Sloppy statistics: Do 50% of Americans really think married women should be legally obligated to change their names?
Jill Filipovic wrote an opinion column for The Guardian yesterday, arguing against the practice of women taking their husbands' names when they get married. It ended up linked on Jezebel and found its way to my Facebook feed where one particular statistic caught my eye. Filipovic claimed that 50% of Americans think a women should be legally required to take her husband's name.
First, some quick clarification of my biases here. Although I write under a hyphenate, I never have legally changed my name. I've never had a desire to do so. In my private life, I'm just Maggie Koerth and always will be. That said, I personally take issue with the implication at the center of Filipovic's article — that women shouldn't change their names and that to do so makes you a bad feminist. For me, this is one of those personal decisions where I'm like, whatever. Make your own choice. Just because I don't get it doesn't mean you're wrong.
But just like I take objection to being all judgey about personal choices, I also take objection to legally mandating personal choices, and I was kind of blown away by the idea that 50% of my fellow Americans think my last name should be illegal.
So I looked into that statistic. And then I got really annoyed.
Read the rest
The good folks on the most-excellent BBC Radio/Open University statistical literacy programme More or Less decided to answer a year-old Reddit argument about how many Lego bricks can be vertically stacked before the bottom one collapses.
They got the OU's Dr Ian Johnston to stress-test a 2X2 Lego in a hydraulic testing machine, increasing the pressure to some 4,000 Newtons, at which point the brick basically melted. Based on this, they calculated the maximum weight a 2X2 brick could bear, and thus the maximum height of a Lego tower:
The average maximum force the bricks can stand is 4,240N. That's equivalent to a mass of 432kg (950lbs). If you divide that by the mass of a single brick, which is 1.152g, then you get the grand total of bricks a single piece of Lego could support: 375,000.
So, 375,000 bricks towering 3.5km (2.17 miles) high is what it would take to break a Lego brick.
"That's taller than the highest mountain in Spain. It's significantly higher than Mount Olympus [tallest mountain in Greece], and it's the typical height at which people ski in the Alps," Ian Johnston says.
"So if the Greek gods wanted to build a new temple on Mount Olympus, and Mount Olympus wasn't available, they could just - but no more - do it with Lego bricks. As long as they don't jump up and down too much."
Sarah Kliff at the Washington Post digs into new research out today from The American Journal of Clinical Nutrition. She writes about correlation and causality, and how to read statistics more intelligently.
“I was constantly amazed at how often claims about associations of specific foods with cancer were made, so I wanted to examine systematically the phenomenon,” e-mails study author John Ioannidis ”I suspected that much of this literature must be wrong. What we see is that almost everything is claimed to be associated with cancer, and a large portion of these claims seem to be wrong indeed.”
Among the ingredients in question for their purported relation to cancer risk: veal, salt, pepper spice, ﬂour, egg, bread, pork, butter, tomato, lemon, duck, onion, celery, carrot, parsley, mace, sherry, olive, mushroom, tripe, milk, cheese, coffee, bacon, sugar, lobster, potato, beef, lamb, mustard, nuts, wine, peas, corn, cinnamon, cayenne, orange, tea, rum, and raisin.
Now: combine all of them into one recipe and do the study again, I say.
Nate Silver's been in the news a lot these last few days: looking at some stories, you'd think he'd won the election, not Mr. Obama. A statistician, his rigorous polling analysis riled, then humiliated political pundits, whose imaginary political horse-race was rejected by Silver's cold, hard numbers.
And what numbers they were. His "prediction"--though really just the most likely probability among many scenarios offered by his model--nailed the electoral college total on the night. Read the rest
Read the rest
I'm a nervous flyer. But I'm a lot better at it then I used to be. That's because, a few years ago, I learned that it's actually pretty common to survive a plane crash. Like most people, I'd assumed that the safety in flying came from how seldom accidents happened. Once you were in a crash situation, though, I figured you were probably screwed. But that's not the case.
Looking at all the commercial airline accidents between 1983 and 2000, the National Transportation Safety Board found that 95.7% of the people involved survived. Even when they narrowed down to look at only the worst accidents, the overall survival rate was 76.6%. Yes, some plane crashes kill everyone on board. But those aren't the norm. So you're even safer than you think. Not only are crashes incredibly rare, you're more likely to survive a crash than not. In fact, out of 568 accidents during those 17 years, only 71 resulted in any fatalities at all.
I was talking about this fact with a pilot friend over the weekend, and he mentioned one crash in particular that is an excellent example of the statistics in action. On July 19, 1989, United Airlines Flight 232 lost all its hydraulic controls and landed in Sioux City, Iowa, going more than 100 mph faster than it should have been. You can see the plane breaking apart and bursting into flames in the video above. Turns out, that's what a 62% survival rate looks like. (All the pilots you can hear talking in the video survived, too.)
In 2007, Popular Mechanics examined 36 years of NTSB reports and found that the majority of surviving passengers were sitting in the back of the plane. But that seems to depend a lot on the specifics of the crash and may not be a reliable predictor of future results.
The election is next week. And, with that in mind, Salon's Paul Campos has posted a helpful reminder explaining what the statistics at the fivethirtyeight blog actually mean (and what they don't).
In particular, you have to remember that, while Nate Silver gives President Obama a 77.4 percent chance of winning the presidential election, that's not the same thing as saying that Obama is going to win.
Suppose a weather forecasting model predicts that the chance of rain in Chicago tomorrow is 75 percent. How do we determine if the model produces accurate assessments of probabilities? After all, the weather in Chicago tomorrow, just like next week’s presidential election, is a “one-off event,” and after the event the probability that it rained will be either 100 percent or 0 percent. (Indeed, all events that feature any degree of uncertainty are one-off events – or to put it another way, if an event has no unique characteristics it also features no uncertainties).
The answer is, the model’s accuracy can be assessed retrospectively over a statistically significant range of cases, by noting how accurate its probabilistic estimates are. If, for example, this particular weather forecasting model predicted a 75 percent chance of rain on 100 separate days over the previous decade, and it rained on 75 of those days, then we can estimate the model’s accuracy in this regard as 100 percent. This does not mean the model was “wrong” on those days when it didn’t rain, any more than it will mean Silver’s model is “wrong” if Romney were to win next week.
What Silver is predicting, in effect, is that as of today an election between a candidate with Obama’s level of support in the polls and one with Mitt Romney’s level of support in those polls would result in a victory for the former candidate in slightly more than three out of every four such elections.
Matthew Lasar's long Ars Technica feature, "Have we lost 41 percent of our musicians? Depends on how you (the RIAA) count" does an excellent job of digging into RIAA CEO Cary Sherman's claim that the number of working musicians in the USA has declined by 41 percent. After checking the RIAA's math, Lasar finds a gigantic discrepancy between the figures they cite and the conclusions they reach. But then Lasar delves further into the underlying sources, as well as government and industry stats, and finds that basically, the number of musicians working in America may have slightly declined, but is also projected to rise.
It is worth ending this cautionary tale with a review of the BLS's own occupational handbook projection for musician/singer employment in the near future. Note that the handbook cites a much higher employment figure for both trades in 2010 than mentioned in the above tables: about 176,200 musicians and singers. That's because it comes from the Bureau's National Employment Matrix, I was told, which adds additional data sources.
Employment for musicians and singers is expected to grow by ten percent over the decade—"about as fast as the average for all occupations," the government notes:
The number of people attending musical performances, such as orchestra, opera, and rock concerts, is expected to increase from 2010 to 2020. As a result, more musicians and singers will be needed to play at these performances.
There will be additional demand for musicians to serve as session musicians and backup artists for recordings and to go on tour. Singers will be needed to sing backup and to make recordings for commercials, films, and television.
The methodology is straightforward. You take your subject and slide them into an fMRI machine, a humongous sleek, white ring, like a donut designed by Apple. Then you show the subject images of people engaging in social activities — shopping, talking, eating dinner. You flash 48 different photos in front of your subject's eyes, and ask them to figure out what emotions the people in the photos were probably feeling. All in all, it's a pretty basic neuroscience/psychology experiment. With one catch. The "subject" is a mature Atlantic salmon.
And it is dead.
Read the rest
Genius science writer Ed Yong used to work for a cancer charity, so he's seen how the cancer research sausages get made. In a new post at Not Exactly Rocket Science, Ed takes you on a brief tour of the factory, explaining why even good data doesn't necessarily mean what you think it means.
The post is based around a new study that says 16.1% of all cancers worldwide are caused by infections. This statistic is talking about stuff like HPV—viruses and other infections that can prompt mutations in the cells they infect. Sometimes, those mutations propagate and become a tumor.
That statistic tells us that infections play a role in more cancers than most laypeople probably think, Ed says. It gives us an idea of the scale of the problem. But you have to be careful not to read too much into that 16.1%.
The latest paper tells us that 16.1% of cancers are attributable to infections. In 2006, a similar analysis concluded that 17.8% of cancers are attributable to infections. And in 1997, yet another study put the figure at 15.6%. If you didn’t know how the numbers were derived, you might think: Aha! A trend! The number of infection-related cancers was on the rise but then it went down again.
That’s wrong. All these studies relied on slightly different methods and different sets of data. The fact that the numbers vary tells us nothing about whether the problem of infection-related cancers has got ‘better’ or ‘worse’. (In this case, the estimates are actually pretty close, which is reassuring. I have seen ones that vary more wildly. Try looking for the number of cancers caused by alcohol or poor diets, if you want some examples).
And that's only one of the complications involved in understanding cancer statistics. You really should read Ed's entire post. After you do, a lot of apparent inconsistencies in cancer data will make a lot more sense to you. For instance: What about the cancers caused by radiation exposure?
Read the rest
Bruce Schneier comments on an NYT report on cybercrime that shows that there's just not much money to be had in being a ripoff artist. Dinei Florêncio and Cormac Herley wrote:
A cybercrime where profits are slim and competition is ruthless also offers simple explanations of facts that are otherwise puzzling. Credentials and stolen credit-card numbers are offered for sale at pennies on the dollar for the simple reason that they are hard to monetize. Cybercrime billionaires are hard to locate because there aren’t any. Few people know anyone who has lost substantial money because victims are far rarer than the exaggerated estimates would imply.
The authors frame cybercrime as a "tragedy of the commons," where the overfishing (overphishing) by crooks has reduced everyone's margins to nothing, making it hard graft indeed. Meanwhile, cybercrime estimates are subject to the same lobbynomics used to calculate losses from music downloading and profits from drug seizures:
Suppose we asked 5,000 people to report their cybercrime losses, which we will then extrapolate over a population of 200 million. Every dollar claimed gets multiplied by 40,000. A single individual who falsely claims $25,000 in losses adds a spurious $1 billion to the estimate. And since no one can claim negative losses, the error can't be canceled.
Why the DHS's pre-crime biometric profiling is doomed to fail, and will doom passengers with its failures
In The Atlantic, Alexander Furnas debunks the DHS's proposal for a "precrime" screening system that will attempt to predict which passengers are likely to commit crimes, and single those people out for additional screening. FAST (Future Attribute Screening Technology) "will remotely monitor physiological and behavioral cues, like elevated heart rate, eye movement, body temperature, facial patterns, and body language, and analyze these cues algorithmically for statistical aberrance in an attempt to identify people with nefarious intentions." They'll build the biometric "bad intentions" profile by asking experimental subjects to carry out bad deeds and monitoring their vital signs. It's a mess, scientifically, and it will falsely accuse millions of innocent people of planning terrorist attacks.
First, predictive software of this kind is undermined by a simple statistical problem known as the false-positive paradox. Any system designed to spot terrorists before they commit an act of terrorism is, necessarily, looking for a needle in a haystack. As the adage would suggest, it turns out that this is an incredibly difficult thing to do. Here is why: let's assume for a moment that 1 in 1,000,000 people is a terrorist about to commit a crime. Terrorists are actually probably much much more rare, or we would have a whole lot more acts of terrorism, given the daily throughput of the global transportation system. Now lets imagine the FAST algorithm correctly classifies 99.99 percent of observations -- an incredibly high rate of accuracy for any big data-based predictive model. Even with this unbelievable level of accuracy, the system would still falsely accuse 99 people of being terrorists for every one terrorist it finds. Given that none of these people would have actually committed a terrorist act yet distinguishing the innocent false positives from the guilty might be a non-trivial, and invasive task.
Of course FAST has nowhere near a 99.99 percent accuracy rate. I imagine much of the work being done here is classified, but a writeup in Nature reported that the first round of field tests had a 70 percent accuracy rate. From the available material it is difficult to determine exactly what this number means. There are a couple of ways to interpret this, since both the write-up and the DHS documentation (all pdfs) are unclear. This might mean that the current iteration of FAST correctly classifies 70 percent of people it observes -- which would produce false positives at an abysmal rate, given the rarity of terrorists in the population. The other way of interpreting this reported result is that FAST will call a terrorist a terrorist 70 percent of the time. This second option tells us nothing about the rate of false positives, but it would likely be quite high. In either case, it is likely that the false-positive paradox would be in full force for FAST, ensuring that any real terrorists identified are lost in a sea of falsely accused innocents.
Here's a clip of a Danish TV show discussing ACTA, which Denmark has fiercely advocated in favor of. It starts with the head of a rightsholder society and the Danish trade minister quoting dodgy statistics about the extent and cost of piracy, and then demonstrates that these statistics are patently false, and finally, brings out those responsible for quoting them and gets them to admit their errors. Priceless.
You can see both the Danish Trade Minister and the head of a Danish music rights organization (and famous Danish musician) Ivan Pedersen appear on a TV show below (with English subtitles). On the show, a well-informed presenter focuses on how both of these ACTA defenders claimed that 95% of music downloaded in Denmark was unauthorized, and carefully shows how that's simply false -- and then gets both of the ACTA defenders to admit that the numbers were wrong.
The NYT's Andrew Ross Sorkin quotes Barry Ritholtz's digging into how Facebook's IPO documents define "active" users and finds that many of them may never visit the site. Facebook counts you as "active" if your only involvement with the service is setting it up to republish your Twitter feed, or if you click "Like" buttons but never log in to the actual service. This should matter to investors, since Facebook earns no advertising revenue from those users, though it may earn some other income by reselling the private details of their browsing habits as gleaned from its tracking cookies.
In other words, every time you press the “Like” button on NFL.com, for example, you’re an “active user” of Facebook. Perhaps you share a Twitter message on your Facebook account? That would make you an active Facebook user, too. Have you ever shared music on Spotify with a friend? You’re an active Facebook user. If you’ve logged into Huffington Post using your Facebook account and left a comment on the site — and your comment was automatically shared on Facebook — you, too, are an “active user” even though you’ve never actually spent any time on facebook.com.
“Think of what this means in terms of monetizing their ‘daily users,’ ” Barry Ritholtz, the chief executive and director for equity research for Fusion IQ, wrote on his blog. “If they click a ‘like’ button but do not go to Facebook that day, they cannot be marketed to, they do not see any advertising, they cannot be sold any goods or services. All they did was take advantage of FB’s extensive infrastructure to tell their FB friends (who may or may not see what they did) that they liked something online. Period.”