Name: Why Statistics Don’t Capture The Full Extent Of The Systemic Bias In Policing
Brand: fivethirtyeight.com
SKU: fivethirtyeight-com-88
Availability: OnlineOnly

critic

Why Statistics Don’t Capture The Full Extent Of The Systemic Bias In Policing

preview

Graphics by, demonstrators have spent the past few weeks protesting against racial disparities in the country’s criminal justice system. There’s of to back them up: Black and Hispanic people are , , and . Once stopped, police are to against, and . And then once in jail, Black defendants are , which in turn . And when convicted, sentencing is against , with Black defendants .The data seems to overwhelmingly point to a criminal justice system riven by racial bias. But, remarkably, it could be even more overwhelming than some studies make it seem. That’s because of a statistical quirk called “collider bias,” a kind of selection bias that means that the crime data that shows racial bias is, itself, biased by racist practices. If you thought crime data showed clear evidence of racism before, understanding how collider bias affects these analyses might make it even clearer.To understand how all this works, we’re going to get mathy. In particular, we’re going to be talking a lot about denominators, which, if you’ll remember your fourth grade math lessons, are the numbers on the bottom of a fraction. For our purposes, though, you can think of them as the universe of people who are being studied.Police engage with only a small subset of the population they see, so if we look at statistics about those interactions, the stats we get are informed by that smaller sample. And if there’s bias in who the police choose to interact with — if it’s not a random sample — that can change the relationships you see in the data.There are two main ways that researchers approach this problem — by using a “population denominator” and an “encounter denominator.”The former, for example, compares the fraction of Black people in the general population who are arrested or harmed to that same statistic for white people. That’s how you get studies that show , compared to 39 out of 100,000 white men and boys — a risk that is 2.5 times higher. Seems straightforward enough. But because Black and white people encounter the police at different rates to begin with, using population denominators ; being more likely to encounter police means that you’re , too.For this reason, many researchers choose to look at the set of people who have encountered the police. This is the “encounter denominator.” The setup is simple: You look at all the people who had recorded encounters with police — data which is — and calculate the proportion that involved the use of force. But this approach has a different issue, as a recent paper pointed out — , then looking at discrepancies in the resulting interactions won’t give you the full picture. This is because of something called “collider bias.”“The vast majority — 99.9 percent of the data — we never get to see,” said Dean Knox, a professor of politics at Princeton and one of the authors of the study. “We just don’t see all the times when police officers are encountering civilians on the street. And that’s a huge problem, because among the data that you do get to see — the stops, and the arrests, and the use of force that officers record — those are already contaminated, because officers have discretion in who they choose to engage.”The fact that researchers’ data comes from a biased sample — who the police choose to stop — rather than the full sample of possible stops might skew the conclusions we draw from it. “If we’re using administrative data, we always need to be aware of what world those data capture, and what they don’t capture,” said Allison P. Harris, a political science professor at Yale who studies racial disparities in the criminal justice system. “With policing, we just can never know what there’s no record of.”One example of an encounter denominator approach is a 2019 study by Roland Fryer, an economist at Harvard. He found that . At first blush, that would seem like evidence that the police are not racially biased — every demographic is being treated equally, after all.But we know that police officers are more likely to stop Black and Hispanic people than white ones — and that more of those stops are unfounded. Researchers measure this with something called the “hit rate,” or the rate at which contraband is actually found on the people who were stopped. A lower hit rate implies bias because it means that the decision to search someone was made with less evidence. White people stopped in New York City, for example, who were stopped. White drivers stopped by the police were than nationally.As political scientists , among others, have pointed out, that complicates Fryer’s findings. All of a sudden, what at first appeared to be equal treatment actually suggests unequal treatment. Because of the initial discrimination in who gets stopped, the sample of stopped people isn’t the same across races. The different hit rate indicates that stopped white people are actually more likely to have contraband, on average, than stopped Black people. In other words, in a world without discrimination in who was stopped — if the Black and white people who were stopped were equally likely to be engaged in criminal activity — you’d see an even bigger disparity in outcomes.It’s unintuitive, I know, but let’s break it down visually.“The data we have pick up halfway through the encounter, after officer bias very likely has already exerted an effect,” said Mummolo, a professor at Princeton. And without knowing the racial composition of the people who are sighted by police but aren’t stopped, it’s impossible to fully correct for the bias. The paper provides a suggestion for how to estimate bias in one part of the chain — use of force in encounters — but it doesn’t capture any of the potential disparities that occur beforehand that make the encounter likelier.So if racial bias in police stops makes it hard to estimate racial disparities in how people are treated while stopped, you might be wondering: well, doesn’t this affect all of the other components of the chain, too?“We need to think about this as this complex system,” said Knox. There are biases layered on biases — social decisions like how society allocates resources in education versus policing, and individual decisions like who police choose to stop or arrest and who is ultimately charged or convicted. “It just compounds all the way down the line,” Knox said.The thing about systemic racism is it’s just that: systemic.Ultimately, the fact that these biased events build on each other should be a reminder that we’re not capturing the entire process surrounding the stats we’re looking at. Essentially, all the discrimination we can actually measure at each specific stage is an underestimate of how much is actually happening.Laura Bronner is a senior applied scientist at ETH Zürich and FiveThirtyEight’s former quantitative editor.Filed underYou are now subscribed!Email AddressSign me up

Graphics by, demonstrators have spent the past few weeks protesting against racial disparities in the country’s criminal justice system. There’s of to back them up: Black and Hispanic people are , , and . Once stopped, police are to against, and . And then once in jail, Black defendants are , which in turn . And when convicted, sentencing is against , with Black defendants .The data seems to overwhelmingly point to a criminal justice system riven by racial bias. But, remarkably, it could be even more overwhelming than some studies make it seem. That’s because of a statistical quirk called...

Why Statistics Don’t Capture The Full Extent Of The Systemic Bias In Policing

No article scores yet.

CRITIC REVIEWS

PUBLIC REVIEWS