Lawyer-turned-data-scientist David Colarusso analyzed 2.2 million sentencing records from Virginia to determine the relationship between race, income and treatment in the criminal justice system.
Colarusso uses his analysis as a primer on data analysis, explaining regression and data normalization in simple language accessible to people without a statistics background.
But then he gets to the conclusion, which shows that race swamps class as a factor in sentencing, saying that "for a black man in Virginia to get the same treatment as his Caucasian peer, he must earn more than half a million dollars a year."
What's more, "the same holds for American Indians, Asian or Pacific Islanders, and Hispanics."
Update: Peer review works! Some of Colarusso's readers reviewed his work and found an error that put the number closer to $90,000 than $500,000. He writes, "Additionally, the offset for Asian defendants turned out to be a little less than half that of Black, Native, and Hispanic defendants."
I did all of my analysis with freely available tools, and there’s nothing stopping you from picking up where I left off. In fact, I hope that a few of you will look at this GitHub repo and do exactly that. However, it’s important to note that you need a solid foundation in statistics to avoid making unwarranted claims due to lack of experience.
And beware the danger zone! As Drew Conway (creator of the Venn Diagram above) points out, “It is from [that] part of the diagram that the phrase ‘lies, damned lies, and statistics’ emanates.”
That being said, there is nothing magic here. You can also discover hidden truths. My advice? Be suspicious of answers that reinforce your existing assumptions. Do your work in the open. When confidentiality allows, share both your findings and your data. Have someone check your math. Listen to feedback, and always be ready to change your mind
Uncovering Big Bias with Big Data
The Justice Department today announced indictments for 80 individuals on charges they ran a massive business email and money laundering scam that operated in part out of Southern California. DoJ’s 145-page indictment was unsealed Thursday, and charges 80 named individuals with conspiracy to commit mail and bank fraud, plus aggravated identity theft and money laundering. […]
Last Mother's Day, grandmother Rhonda Mengert was subjected to a pat-down search at Tulsa airport, wherein a TSA agent felt a panty-liner in her underwear; she was then forced to strip down and show her panty-liner to a female TSA agent. Naturally, she filed suit against the TSA.
Few states have voting machines that are simultaneously more obviously defective and more ardently defended by the state government than Georgia, where 16-year-old touchscreen systems are prone to reporting ballots cast by 243% of the eligible voters and where gross irregularities in election administration sends voters to the wrong polling places or sends co-habitating husbands […]
There’s no shortage of stories about the benefits of cannabidiol, that benign (and non-psychoactive) cousin of THC. Some have been using it for years to deal with pain, stress, and sleeplessness. And the more people use it, the more discussion there is about how to use it. While there’s no shortage of quality edibles on […]
Are we done with capsule coffee makers yet? Sure, they’re easy. But they are not so easy on the environment, and it’s debatable whether they actually make a better cup. Luckily, there’s never been a better time to switch back to the good old reliable drip method – especially when drip coffeemakers have quietly been […]
If there’s one thing that stayed consistent through the last decade or so of tech industry turmoil, it’s the love affair between techies and Linux. There’s just a ton you can do with the OS, and its open-source format means you can customize your rig from the ground up. Apparently not content with that level […]