What Nate Silver is actually telling you about the election

The election is next week. And, with that in mind, Salon's Paul Campos has posted a helpful reminder explaining what the statistics at the fivethirtyeight blog actually mean (and what they don't).

In particular, you have to remember that, while Nate Silver gives President Obama a 77.4 percent chance of winning the presidential election, that's not the same thing as saying that Obama is going to win.

Suppose a weather forecasting model predicts that the chance of rain in Chicago tomorrow is 75 percent. How do we determine if the model produces accurate assessments of probabilities? After all, the weather in Chicago tomorrow, just like next week’s presidential election, is a “one-off event,” and after the event the probability that it rained will be either 100 percent or 0 percent. (Indeed, all events that feature any degree of uncertainty are one-off events – or to put it another way, if an event has no unique characteristics it also features no uncertainties).

The answer is, the model’s accuracy can be assessed retrospectively over a statistically significant range of cases, by noting how accurate its probabilistic estimates are. If, for example, this particular weather forecasting model predicted a 75 percent chance of rain on 100 separate days over the previous decade, and it rained on 75 of those days, then we can estimate the model’s accuracy in this regard as 100 percent. This does not mean the model was “wrong” on those days when it didn’t rain, any more than it will mean Silver’s model is “wrong” if Romney were to win next week.

What Silver is predicting, in effect, is that as of today an election between a candidate with Obama’s level of support in the polls and one with Mitt Romney’s level of support in those polls would result in a victory for the former candidate in slightly more than three out of every four such elections.

Read the full story at Salon.com


  1. In which “77 %” is explained using small words:

    would result in a victory for the former candidate in slightly more than three out of every four such elections.

    When do you learn about the concept of “percent”?  Fourth grade?  But this is just wasted bytes.  Anyone who is going to read this article knows all these things.  The pundits going after Silver aren’t ignorant about percentages because they’re stupid; they may not even be ignorant about percentages.  They don’t care.  Figuring out what is correct doesn’t get them think tank chairs and invitations to connected-person cocktail parties.  These are not confused people.  These are not misinformed people.  These are bad people.

    1. These pundits, some of them, are actually as ignorant as they seem. Which is not to say they’re stupid — they’re trying to do a thing which is inherently difficult for human brains, which is to cross back and forth between expressions of dynamic ranges and discrete, fixed elements without losing the meaning of the numbers along the way.

      If they knew how hard was, they’d probably be more cautious in their opinions, but these people don’t get into their positions through humility or caution. Pundits are formed when a constant stream of cogent certainties flows out from good bone structure, and to hope for statistical nuance in such formations is 99% futile.

  2. If Romney wins next week, wingers will of course claim Silver was wrong. If Obama wins, they’ll claim the vote counting was wrong (and therefor Silver was wrong, and probably in on the plot with his socialist Muslim ACORN allies).

    While a certain amount of the storm surge of vitriol directed at Silver is the result of — or at least aided by — poor math skills, the actual numbers and interpretation thereof are mostly irrelevant. Silver thinks it’s likely that Obama will win, so he’s an enemy.

  3. I understand the weather analogy, but it seems a bit weak because of the assumption that we have lots of identical predictions (i.e. tomorrow’s chance of rain is 75%).  How would you go about assessing the model’s accuracy if your data points had a variety of different values for the probability of rain tomorrow?

    (Also, I have to get it off my chest that, after reading the fivethirtyeight blog post, the election seems like a really stupid and arbitrary game.)

  4. I wonder if Romney will win since lazy people will stay home after reading Obama “already won” by nearly 80 percent of the vote?


    1. Really? Give readers of BoingBoing more credit. Does your impression of the political climate in the USA lead you to believe this is even remotely possible? That people will read an article on Salon.com and then abandon their plan to vote?  Sheesh!

      1. Give readers of BoingBoing more credit.

        I wasn’t referring to Boingers. Talking ’bout the general public who deserves little or no credit. Especially when you consider they expected all kinds of “hope & change” without themselves voting out enough republicans to thwart filibusters.

        Does your impression of the political climate in the USA lead you to believe this is even remotely possible?

        Um, same country that voted in George Bush… twice. Yes, I think it’s VERY possible in the United States of Amnesia.

        That people will read an article on Salon.com and then abandon their plan to vote? Sheesh!

        No, not most Salon readers, but many people are reading that statistic elsewhere and they sure will. I think you underestimate just how lazy and pacified many Americans are. Once again, I refer you to two (2) Bush terms.

        1. Okay, fair point. I can’t imagine not voting, and that was my initial reaction when I contemplated that. But it’s true: too many people won’t bother.

          1. I could totally see right wing radio airing these statistics to mobilize their base while also attempting to pacify their enemy. I just hope they’re not smart enough…

        1. He may not have lived up to all of the hype, but I think that electing Barack Obama was a smart move. McCain would have been another Reagan/Bush clone, but a “safe” choice. Electing Obama added some clarity to how we are divided and where each side stands. Romney is such a Swiss-flip-flop candidate that the Republicans are confused: do we like him, or do we vote to defeat Obama, etc? Democrats are succeeding in repairing the damage wrought by 8 years of Bush and KNOW they have a superior candidate and a better platform, regardless of the outcome. Republicans can’t say that.

  5. I personally think Nate Silver is being extremely conservative in his estimates.

    If you look at the overall trend, Obama has been leading by a thin margin every time Nate Silver has checked in the last 6 months. And at this point 538 is updating it more than once a day — and Obama is still ahead, every time. Of course, things can change suddenly in a different direction (for example, there might be a giant hurricane that affects voting…)

    Not, mind you, in every single poll! But in the aggregate analysis of those polls (which is not probabilistic, it’s just math) Obama has held a continuous lead since the start. Claming that will change in a week starts to seem ridiculous. The only conclusion I can draw is that Nate’s model is hedging its bets pretty severely.

    1. Yup. However…. I trust Nate’s stats creds over yours.

      Nate’s plugging all this data into his models and running the models thousands of times a day to get these statistics. In all those simulations, 23% of the time Romney still wins, even with all these things going for Obama.

      Don’t forget that there are huge numbers of unknowns (voter turnout etc.) that can’t be easily grasped by those of use just looking at polls, but that his model is attempting to account for.

  6. Of course, while Nate Silver is much preferable to the ‘intuitive’ hand-wringing drama that passes for political coverage here in America, he isn’t even close to the most scientifically / mathematically sound poll analyst out there.  That honor probably goes to Dr. Sam Wang of the Princeton Election Consortium:

    http://election.princeton.edu/ (the website seems to having issues today, probably from increased traffic)

    Wang not only refuses to weight his model based on gut-feeling (unlike Silver, who does), he publishes all of his work on the PIC website, usually in MatLab-friendly code, so that anyone can replicate his calculations.  Silver still relies on a ‘secret sauce’ that he refuses to divulge.

    Dr. Wang’s current prediction?  99% chance of an Obama victory, or 95%, if you want to accept the more skeptical (non-Bayesian) model.  Romney is stuffed.

    1.  While I like to believe polls are correct in this respect, I would like to don my tin foil hat for a moment and remind folks that there are numerous experts who see all sorts of ways a close election like this could be rigged in favor of Romney. Let’s not forget that his son Trig is employed by a company that has sold and services electronic voting machines in 6 states, including Ohio, and that Diebold, the leading manufacturer of electronic voting machines, is a major Republican contributor.

      That said, I’m not entirely thrilled about an Obama reelection but I’ll take it over Mitt Romney any day.

      1. This seems a combination of concern-trolling and standard Neo-Con bipartite consensus reality.  But reality (and math, and science, etc) are not formed around consensus.  There are no credible sources that are calling this for Romney, and concerns about mass voter fraud seem paranoid at best, despite vague allusions to ‘numerous experts’.  The race is, mathematically, not close, despite the vested interests of quite a lot of people in implying otherwise.

        1. Aren’t the machines riddled with bugs – deliberately or otherwise? Why the hell does Trig have to get into THIS line of business? And isn’t voter-suppression being openly pursued by Republicans all over the place? Isn’t disenfranchising the vulnerable a Great American Tradition?

          First slavery, then Jim Crow, then the War on Drugs and attendant felony records: face it, a very large and powerful section of your society has never and will never accept the black vote. And if they can deny votes to  other 47 percenters, why wouldn’t they?  After all, the worse that can happen to the fixers is that a handful of political opponents will scream about it and be shushed into line by their own “sensible” allies.

    2. I am not good at math, so I have to rely on my fallible intuition, but any model that predicts a 99% chance victory doesn’t seem realistic to  me.

    3. Wang not only refuses to weight his model based on gut-feeling (unlike Silver, who does)

      Citation about Nate using gut feeling?

      As far as Nate has explained in his posts, the “house-effects” weighting are entirely data-driven: if Rasmussen is consistently leaning 1% Republican, then the model automatically corrects for that, without Nate having to step in.

      Likewise, I think that the erosion of a poll’s weight over time is similarly data-driven. Looking at past data, Nate’s algorithm’s can calculate the relative coupling between a poll’s data eight weeks before an election and the actual results. If the historical data suggest a weak coupling, then the poll can calculate how much to erode the weight by over time.

  7. This is why all elections should be settled with a 7-election series, to ensure that the best candidate really wins.  Who is up for (at least 3) more election seasons after this one…

  8. Well, the Salon article explains the probabilities in terms of frequency out of many similar events, but that’s not the same as one event. It’s a different branch of statistics.

  9. I know a thing or two about statistics and I certainly don’t think a ‘75% chance that Obama is going to win’ mean Obama is going to win; in fact I hate those odds. I mean, if somebody handed me a four-chambered revolver and told me there was one bullet in it, no way would I put it to my head and pull the trigger.

  10. The “chance of rain” analogy is a bit misleading. Unless I am sorely misinformed, if you are in Chicago and there is a 75% chance of rain tomorrow, that doesn’t mean that there is a 75% chance that Chicago will be rained upon, it means that if you stand still all day outdoors someplace in Chicago you have a 75% chance of getting wet. There is almost a 100% chance that rain will happen somewhere in Chicago; lower percentages indicate expected rain coverage of the area rather than an absolute chance of rain anywhere.

  11. I think that people also look at 77% and see 77% being a very large proportion of 100% and so see the election as being in the bag. It’s easy to forget that you should really think of it as starting from 50%, as that’s the point where it’s a coin flip. Below 50% means that Romney’s ahead and 55% would mean that it’s still basically a coin flip, just one where one side of the coin is a tiny bit more weighted than the other.

    Seen this way, 77% is really not that much bigger than 50%…

Comments are closed.