Last May, Dave at Euri.ca took at crack at expanding Gabriel Rossman's excellent post on spurious correlation in data. It's an important read for anyone wondering whether the core hypothesis of the Big Data movement is that every sufficiently large pile of horseshit must have a pony in it somewhere. As O'Reilly's Nat Torkington says, "Anyone who thinks it’s possible to draw truthful conclusions from data analysis without really learning statistics needs to read this."
* If good looks and smarts are distributed normally, and
If good looks and smarts have nothing to do with each other, and
If movie producers want both smarts and looks
Then, by observing employed actors we’ll assume that looks and smarts have a negative correlation
Even though we constructed this experiment with no correlation
Here’s a graph of 250 randomly generated points (with no correlation). With the red circles representing “actors who are smart and good looking enough to get a job (looks+smarts>2), and lighter blue x’s representing “people who wanted to be actors”
Clearly if we only look at actors with jobs, we’ll see a clearly negative correlation between smarts and good looks. In fact, some brilliant actors are less attractive than an average person, and some gorgeous actors are dumber than an average person. Even more interesting though, is that if we try to rule out bias by looking at aspiring but unsuccessful actors as well, we’ll find that they exhibit a similar correlation...
You’re probably polluting your statistics more than you think
(via O'Reilly Radar)
Evil Mad Scientist Labs have released their latest set of nerdy Valentines ready for you to print, glue on cardstock, and use to win your true love’s heart.
Brian, a graduate student of Applied Mathematics at Columbia University, has a Tumblr called Fouriest Series where he posts his math and physics visualizations. His explanations are clearly written. He also provides the Mathematica code he used to create his animations. From his post about chaos and double pendulums: Summarized by mathematician Edward Lorenz, “Chaos […]
Writing in Slate, Cathy “Weapons of Math Destruction” O’Neill, a skeptical data-scientist, describes the ways that Big Data intersects with ethical considerations.
Light used to just be one of two things: on or off. Simple as that. Either a flood of yellow or total darkness. Then the dimmer switch happened and you could adjust the brightness to meet your seductive needs and suddenly everyone looked a little better in the gentler light. And now your luminary universe […]
Projects will always need management. And now with the tech gold rush it feels like there are more projects than ever with fewer managers than there’s demand for. But it takes too much time and money to go back to school full time so luckily the Project Management Professional certification training course is now 96% […]
If you’ve been blessed enough to avoid them yourself, you’ve definitely heard the horror stories. Late night, crushing out a ton of work, writing, coding, anything, then boom – your computer crashes. The battery blows, you spill water or coffee all over the place, or it just shuts down with no explanation, and you’re screwed. […]