Oft-cited stats about child abduction puts kidnappers behind every bush. But the numbers are old and frequently mangled, distorting our understanding of genuine risks to children.Read the rest
It is no surprise that critics and viewers alike agree that The Godfather is the "best film" among the ~2600 films considered on Rotten Tomatoes, with a 100% score among professional reviewers and a 98% score from the audience. It is perhaps somewhat more surprising to learn which films divide those two groups; thanks to Benjamin Moore, we can contemplate that...
“Overrated” and “underrated” are slippery terms to try to quantify. An interesting way of looking at this, I thought, would be to compare the reviews of film critics with those of Joe Public, reasoning that a film which is roundly-lauded by the Hollywood press but proved disappointing for the real audience would be “overrated” and vice versa.
To get some data for this I turned to the most prominent review aggregator: Rotten Tomatoes...
On the whole it should be noted that critics and audience agree most of the time, as shown by the Pearson correlation coefficient between the two scores (0.71 across >1200 films). [But] using our earlier definition it’s easy to build a table of those films where the audience ending up really liking a film that was panned by critics:
Here we’re looking at those films which the critics loved, but paying audiences were then less enthused:
The latest installment in Randall Munroe's XKCD "What If?" series is called Paint the Earth and it is amazing. One of Munroe's readers wanted to know "Has humanity produced enough paint to cover the entire land area of the Earth?" and Munroe uses this as a springboard for explaining Fermi estimation, a powerful, counter-intuitive tool that has applications in many fields.
Read the rest
Read the rest
XKCD's "Frequency" - using blinking GIFs to visualize the relative frequency of the momentous and trivial
In Frequency, the latest XKCD cartoon, Randall Munroe has assembled a grid of animated GIFs representing various events in the universe, each keyed to blink in the frequency in which they occur in reality. As with the best of Munroe's work, it's a mix of the trenchant and the silly, and the juxtapositions are smart and provocative. There's real genius in putting "50,000 plastic bottles are produced" and "50,000 plastic bottles are recycled" next to each other, the former blinking much more often than the latter -- but the best part is "A Sagittarius named Amelia drinks a soda," just above them, mixing up the alarming and the humorous.
The other juxtapositions are just as delicious -- one birth/one death; China builds a car/Japan builds a car/Germany builds a car/US builds a car/someone else builds a car; someone buys "To Kill a Mockingbird"/someone's cat kills a mockingbird -- and so on. This being XKCD, you can be sure that Munroe has an absurdly well-thought-through process for establishing and documenting his numbers, too.
If you're the type of person who really needs some good visuals to make a concept stick in your head, this series of YouTube videos made by the British Psychological Society Media Centre will help you remember the meanings behind statistical concepts like "correlation", "frequency distributions", and "sampling error". There are four videos in the series so far, and they do a great job of painting pictures around abstract ideas. Bonus: Soothing music.
Between 1980 and 2000, a complicated war raged in Peru, pitting the country’s government against at least two political guerilla organizations, and forcing average people to band together into armed self-defense committees.Read the rest
Business Insider's Walter Hickey did the math on Monopoly, calculating the most frequently landed-up squares (taking into account dice probability, Go To Jail events, and Community Chest/Chance cards) and conceived of a supposedly optimal strategy for buying and building upon property. I still hate Monopoly, but I suspect that this would make it less boring (for a while, at least).
As a woman, you do become less fertile as you get older, eventually culminating in menopause and the end of your potential babymaking years. But what does "less fertile" mean, and at what age, and how quickly does the drop-off in fertility happen?
According to this really fascinating piece by Jean Twenge at The Atlantic, some of the commonly cited scare stats — that one in three women ages 35 to 39 will not be pregnant after a year of trying, say — are based on extremely old data collected from historical birth records that don't necessarily reflect what's happening with real women who are alive right now. That statistic mentioned above, for instance, comes from French records (likely those collected by local church baptismal registries) for the years 1670 to 1830.
That matters because fertility is affected by things like quality of nutrition, infection rates, and even childhood illnesses — all of which have changed drastically for the average Western woman since the 19th century.
Look at more modern records, and the outlook for post-30 babymaking is completely different.
Read the rest
Sloppy statistics: Do 50% of Americans really think married women should be legally obligated to change their names?
Jill Filipovic wrote an opinion column for The Guardian yesterday, arguing against the practice of women taking their husbands' names when they get married. It ended up linked on Jezebel and found its way to my Facebook feed where one particular statistic caught my eye. Filipovic claimed that 50% of Americans think a women should be legally required to take her husband's name.
First, some quick clarification of my biases here. Although I write under a hyphenate, I never have legally changed my name. I've never had a desire to do so. In my private life, I'm just Maggie Koerth and always will be. That said, I personally take issue with the implication at the center of Filipovic's article — that women shouldn't change their names and that to do so makes you a bad feminist. For me, this is one of those personal decisions where I'm like, whatever. Make your own choice. Just because I don't get it doesn't mean you're wrong.
But just like I take objection to being all judgey about personal choices, I also take objection to legally mandating personal choices, and I was kind of blown away by the idea that 50% of my fellow Americans think my last name should be illegal.
So I looked into that statistic. And then I got really annoyed.
Read the rest
The good folks on the most-excellent BBC Radio/Open University statistical literacy programme More or Less decided to answer a year-old Reddit argument about how many Lego bricks can be vertically stacked before the bottom one collapses.
They got the OU's Dr Ian Johnston to stress-test a 2X2 Lego in a hydraulic testing machine, increasing the pressure to some 4,000 Newtons, at which point the brick basically melted. Based on this, they calculated the maximum weight a 2X2 brick could bear, and thus the maximum height of a Lego tower:
The average maximum force the bricks can stand is 4,240N. That's equivalent to a mass of 432kg (950lbs). If you divide that by the mass of a single brick, which is 1.152g, then you get the grand total of bricks a single piece of Lego could support: 375,000.
So, 375,000 bricks towering 3.5km (2.17 miles) high is what it would take to break a Lego brick.
"That's taller than the highest mountain in Spain. It's significantly higher than Mount Olympus [tallest mountain in Greece], and it's the typical height at which people ski in the Alps," Ian Johnston says.
"So if the Greek gods wanted to build a new temple on Mount Olympus, and Mount Olympus wasn't available, they could just - but no more - do it with Lego bricks. As long as they don't jump up and down too much."
Sarah Kliff at the Washington Post digs into new research out today from The American Journal of Clinical Nutrition. She writes about correlation and causality, and how to read statistics more intelligently.
“I was constantly amazed at how often claims about associations of specific foods with cancer were made, so I wanted to examine systematically the phenomenon,” e-mails study author John Ioannidis ”I suspected that much of this literature must be wrong. What we see is that almost everything is claimed to be associated with cancer, and a large portion of these claims seem to be wrong indeed.”
Among the ingredients in question for their purported relation to cancer risk: veal, salt, pepper spice, ﬂour, egg, bread, pork, butter, tomato, lemon, duck, onion, celery, carrot, parsley, mace, sherry, olive, mushroom, tripe, milk, cheese, coffee, bacon, sugar, lobster, potato, beef, lamb, mustard, nuts, wine, peas, corn, cinnamon, cayenne, orange, tea, rum, and raisin.
Now: combine all of them into one recipe and do the study again, I say.
Nate Silver's been in the news a lot these last few days: looking at some stories, you'd think he'd won the election, not Mr. Obama. A statistician, his rigorous polling analysis riled, then humiliated political pundits, whose imaginary political horse-race was rejected by Silver's cold, hard numbers.
And what numbers they were. His "prediction"--though really just the most likely probability among many scenarios offered by his model--nailed the electoral college total on the night. Read the rest
Read the rest
I'm a nervous flyer. But I'm a lot better at it then I used to be. That's because, a few years ago, I learned that it's actually pretty common to survive a plane crash. Like most people, I'd assumed that the safety in flying came from how seldom accidents happened. Once you were in a crash situation, though, I figured you were probably screwed. But that's not the case.
Looking at all the commercial airline accidents between 1983 and 2000, the National Transportation Safety Board found that 95.7% of the people involved survived. Even when they narrowed down to look at only the worst accidents, the overall survival rate was 76.6%. Yes, some plane crashes kill everyone on board. But those aren't the norm. So you're even safer than you think. Not only are crashes incredibly rare, you're more likely to survive a crash than not. In fact, out of 568 accidents during those 17 years, only 71 resulted in any fatalities at all.
I was talking about this fact with a pilot friend over the weekend, and he mentioned one crash in particular that is an excellent example of the statistics in action. On July 19, 1989, United Airlines Flight 232 lost all its hydraulic controls and landed in Sioux City, Iowa, going more than 100 mph faster than it should have been. You can see the plane breaking apart and bursting into flames in the video above. Turns out, that's what a 62% survival rate looks like. (All the pilots you can hear talking in the video survived, too.)
In 2007, Popular Mechanics examined 36 years of NTSB reports and found that the majority of surviving passengers were sitting in the back of the plane. But that seems to depend a lot on the specifics of the crash and may not be a reliable predictor of future results.
The election is next week. And, with that in mind, Salon's Paul Campos has posted a helpful reminder explaining what the statistics at the fivethirtyeight blog actually mean (and what they don't).
In particular, you have to remember that, while Nate Silver gives President Obama a 77.4 percent chance of winning the presidential election, that's not the same thing as saying that Obama is going to win.
Suppose a weather forecasting model predicts that the chance of rain in Chicago tomorrow is 75 percent. How do we determine if the model produces accurate assessments of probabilities? After all, the weather in Chicago tomorrow, just like next week’s presidential election, is a “one-off event,” and after the event the probability that it rained will be either 100 percent or 0 percent. (Indeed, all events that feature any degree of uncertainty are one-off events – or to put it another way, if an event has no unique characteristics it also features no uncertainties).
The answer is, the model’s accuracy can be assessed retrospectively over a statistically significant range of cases, by noting how accurate its probabilistic estimates are. If, for example, this particular weather forecasting model predicted a 75 percent chance of rain on 100 separate days over the previous decade, and it rained on 75 of those days, then we can estimate the model’s accuracy in this regard as 100 percent. This does not mean the model was “wrong” on those days when it didn’t rain, any more than it will mean Silver’s model is “wrong” if Romney were to win next week.
What Silver is predicting, in effect, is that as of today an election between a candidate with Obama’s level of support in the polls and one with Mitt Romney’s level of support in those polls would result in a victory for the former candidate in slightly more than three out of every four such elections.
Matthew Lasar's long Ars Technica feature, "Have we lost 41 percent of our musicians? Depends on how you (the RIAA) count" does an excellent job of digging into RIAA CEO Cary Sherman's claim that the number of working musicians in the USA has declined by 41 percent. After checking the RIAA's math, Lasar finds a gigantic discrepancy between the figures they cite and the conclusions they reach. But then Lasar delves further into the underlying sources, as well as government and industry stats, and finds that basically, the number of musicians working in America may have slightly declined, but is also projected to rise.
It is worth ending this cautionary tale with a review of the BLS's own occupational handbook projection for musician/singer employment in the near future. Note that the handbook cites a much higher employment figure for both trades in 2010 than mentioned in the above tables: about 176,200 musicians and singers. That's because it comes from the Bureau's National Employment Matrix, I was told, which adds additional data sources.
Employment for musicians and singers is expected to grow by ten percent over the decade—"about as fast as the average for all occupations," the government notes:
The number of people attending musical performances, such as orchestra, opera, and rock concerts, is expected to increase from 2010 to 2020. As a result, more musicians and singers will be needed to play at these performances.
There will be additional demand for musicians to serve as session musicians and backup artists for recordings and to go on tour. Singers will be needed to sing backup and to make recordings for commercials, films, and television.
The methodology is straightforward. You take your subject and slide them into an fMRI machine, a humongous sleek, white ring, like a donut designed by Apple. Then you show the subject images of people engaging in social activities — shopping, talking, eating dinner. You flash 48 different photos in front of your subject's eyes, and ask them to figure out what emotions the people in the photos were probably feeling. All in all, it's a pretty basic neuroscience/psychology experiment. With one catch. The "subject" is a mature Atlantic salmon.
And it is dead.
Read the rest
Genius science writer Ed Yong used to work for a cancer charity, so he's seen how the cancer research sausages get made. In a new post at Not Exactly Rocket Science, Ed takes you on a brief tour of the factory, explaining why even good data doesn't necessarily mean what you think it means.
The post is based around a new study that says 16.1% of all cancers worldwide are caused by infections. This statistic is talking about stuff like HPV—viruses and other infections that can prompt mutations in the cells they infect. Sometimes, those mutations propagate and become a tumor.
That statistic tells us that infections play a role in more cancers than most laypeople probably think, Ed says. It gives us an idea of the scale of the problem. But you have to be careful not to read too much into that 16.1%.
The latest paper tells us that 16.1% of cancers are attributable to infections. In 2006, a similar analysis concluded that 17.8% of cancers are attributable to infections. And in 1997, yet another study put the figure at 15.6%. If you didn’t know how the numbers were derived, you might think: Aha! A trend! The number of infection-related cancers was on the rise but then it went down again.
That’s wrong. All these studies relied on slightly different methods and different sets of data. The fact that the numbers vary tells us nothing about whether the problem of infection-related cancers has got ‘better’ or ‘worse’. (In this case, the estimates are actually pretty close, which is reassuring. I have seen ones that vary more wildly. Try looking for the number of cancers caused by alcohol or poor diets, if you want some examples).
And that's only one of the complications involved in understanding cancer statistics. You really should read Ed's entire post. After you do, a lot of apparent inconsistencies in cancer data will make a lot more sense to you. For instance: What about the cancers caused by radiation exposure?
Read the rest
Bruce Schneier comments on an NYT report on cybercrime that shows that there's just not much money to be had in being a ripoff artist. Dinei Florêncio and Cormac Herley wrote:
A cybercrime where profits are slim and competition is ruthless also offers simple explanations of facts that are otherwise puzzling. Credentials and stolen credit-card numbers are offered for sale at pennies on the dollar for the simple reason that they are hard to monetize. Cybercrime billionaires are hard to locate because there aren’t any. Few people know anyone who has lost substantial money because victims are far rarer than the exaggerated estimates would imply.
The authors frame cybercrime as a "tragedy of the commons," where the overfishing (overphishing) by crooks has reduced everyone's margins to nothing, making it hard graft indeed. Meanwhile, cybercrime estimates are subject to the same lobbynomics used to calculate losses from music downloading and profits from drug seizures:
Suppose we asked 5,000 people to report their cybercrime losses, which we will then extrapolate over a population of 200 million. Every dollar claimed gets multiplied by 40,000. A single individual who falsely claims $25,000 in losses adds a spurious $1 billion to the estimate. And since no one can claim negative losses, the error can't be canceled.
Why the DHS's pre-crime biometric profiling is doomed to fail, and will doom passengers with its failures
In The Atlantic, Alexander Furnas debunks the DHS's proposal for a "precrime" screening system that will attempt to predict which passengers are likely to commit crimes, and single those people out for additional screening. FAST (Future Attribute Screening Technology) "will remotely monitor physiological and behavioral cues, like elevated heart rate, eye movement, body temperature, facial patterns, and body language, and analyze these cues algorithmically for statistical aberrance in an attempt to identify people with nefarious intentions." They'll build the biometric "bad intentions" profile by asking experimental subjects to carry out bad deeds and monitoring their vital signs. It's a mess, scientifically, and it will falsely accuse millions of innocent people of planning terrorist attacks.
First, predictive software of this kind is undermined by a simple statistical problem known as the false-positive paradox. Any system designed to spot terrorists before they commit an act of terrorism is, necessarily, looking for a needle in a haystack. As the adage would suggest, it turns out that this is an incredibly difficult thing to do. Here is why: let's assume for a moment that 1 in 1,000,000 people is a terrorist about to commit a crime. Terrorists are actually probably much much more rare, or we would have a whole lot more acts of terrorism, given the daily throughput of the global transportation system. Now lets imagine the FAST algorithm correctly classifies 99.99 percent of observations -- an incredibly high rate of accuracy for any big data-based predictive model. Even with this unbelievable level of accuracy, the system would still falsely accuse 99 people of being terrorists for every one terrorist it finds. Given that none of these people would have actually committed a terrorist act yet distinguishing the innocent false positives from the guilty might be a non-trivial, and invasive task.
Of course FAST has nowhere near a 99.99 percent accuracy rate. I imagine much of the work being done here is classified, but a writeup in Nature reported that the first round of field tests had a 70 percent accuracy rate. From the available material it is difficult to determine exactly what this number means. There are a couple of ways to interpret this, since both the write-up and the DHS documentation (all pdfs) are unclear. This might mean that the current iteration of FAST correctly classifies 70 percent of people it observes -- which would produce false positives at an abysmal rate, given the rarity of terrorists in the population. The other way of interpreting this reported result is that FAST will call a terrorist a terrorist 70 percent of the time. This second option tells us nothing about the rate of false positives, but it would likely be quite high. In either case, it is likely that the false-positive paradox would be in full force for FAST, ensuring that any real terrorists identified are lost in a sea of falsely accused innocents.
Here's a clip of a Danish TV show discussing ACTA, which Denmark has fiercely advocated in favor of. It starts with the head of a rightsholder society and the Danish trade minister quoting dodgy statistics about the extent and cost of piracy, and then demonstrates that these statistics are patently false, and finally, brings out those responsible for quoting them and gets them to admit their errors. Priceless.
You can see both the Danish Trade Minister and the head of a Danish music rights organization (and famous Danish musician) Ivan Pedersen appear on a TV show below (with English subtitles). On the show, a well-informed presenter focuses on how both of these ACTA defenders claimed that 95% of music downloaded in Denmark was unauthorized, and carefully shows how that's simply false -- and then gets both of the ACTA defenders to admit that the numbers were wrong.
The NYT's Andrew Ross Sorkin quotes Barry Ritholtz's digging into how Facebook's IPO documents define "active" users and finds that many of them may never visit the site. Facebook counts you as "active" if your only involvement with the service is setting it up to republish your Twitter feed, or if you click "Like" buttons but never log in to the actual service. This should matter to investors, since Facebook earns no advertising revenue from those users, though it may earn some other income by reselling the private details of their browsing habits as gleaned from its tracking cookies.
In other words, every time you press the “Like” button on NFL.com, for example, you’re an “active user” of Facebook. Perhaps you share a Twitter message on your Facebook account? That would make you an active Facebook user, too. Have you ever shared music on Spotify with a friend? You’re an active Facebook user. If you’ve logged into Huffington Post using your Facebook account and left a comment on the site — and your comment was automatically shared on Facebook — you, too, are an “active user” even though you’ve never actually spent any time on facebook.com.
“Think of what this means in terms of monetizing their ‘daily users,’ ” Barry Ritholtz, the chief executive and director for equity research for Fusion IQ, wrote on his blog. “If they click a ‘like’ button but do not go to Facebook that day, they cannot be marketed to, they do not see any advertising, they cannot be sold any goods or services. All they did was take advantage of FB’s extensive infrastructure to tell their FB friends (who may or may not see what they did) that they liked something online. Period.”
The Centers for Disease Control and Prevention, and the World Health Organization, say that H5N1 bird flu kills some 60% of the human beings it manages to infect. Basically, it hasn't infected many people—because it can't be spread from person to person—but most of the people it does infect die.
But this might not be the full story.
After I posted a summary of the current controversies surrounding H5N1 research, I got an interesting email from Vincent Racaniello, a professor of microbiology at Columbia University Medical Center. Racaniello points out that the 60% death rate statistics are based on people who show up at hospitals with serious symptoms of infection. So far, there've only been about 600 cases. And, yes, about 60% of them have died.
However, they don't necessarily represent everybody who has contracted H5N1.
A death rate is only as good as statistics on the rate of infection. If you've got an inaccurate count of the number of people infected, your death rate is going to be wrong. And there's some evidence that might be the case with H5N1.
In a recent study of rural Thai villagers, sera from 800 individuals were collected and analyzed for antibodies against several avian influenza viruses, including H5N1, by hemagglutination-inhibition and neutralization assays. The results indicate that 73 participants (9.1%) had antibody titers against one of two different H5N1 strains. The authors conclude that ‘people in rural central Thailand may have experienced subclinical avian influenza virus infections’. A subclinical infection is one without apparent signs of illness.
If 9% of the rural Asian population has been subclinically infected with avian H5N1 influenza virus strains, it would dramatically change our view of the pathogenicity of the virus. Extensive serological studies must be done to determine the extent of human infection with avian H5N1 influenza viruses. Until we know how many individuals are infected with avian influenza H5N1, we must refrain from making dire conclusions about the pathogenicity of the virus.
At Bloomberg Business Week, Vali Chandrasekaran makes me incredibly happy by creating a series of six infographics demonstrating the ridiculous connections you can make when you start confusing correlation and causation. Did a conspiracy of baby Avas cause the U.S. housing market to implode? Was Michele Bachmann's candidacy doomed by the end of Staten Island Cakes? Are scientists raising the global average temperature in order to increase their own research funding? Find out here!