Image 01 Image 03

If Nate Silver cannot be wrong, how can he be right?

If Nate Silver cannot be wrong, how can he be right?

I have no implicit pro- or anti-Nate Silver bias. 

He was a journolist and in 2008 got some secret help with his predictions from the Obama campaign, but since joining The NY Times I think he has raised his game.

I find the whole focus on Silver and his presidential election “model” to be particularly annoying, in part because there seems to be a lack of transparency.  Is the methodology and how it is used to come up with specific percentages fully disclosed?

Whatever, Silver-mania (both pro and con) seems to have invaded way too many bodies. 

I was in an SUV last Wednesday night in Austin with a group of people at a conference heading to dinner.  A woman in the car got a beep on her phone, and she exclaimed words to the effect: “I just got an alert, Nate Silver has Obama’s chances of winning at 72.3 percent.”  (I may be off slightly on the percentage, but that’s pretty close.)

At that moment, if I did not know it before, I understood that this Nate Silver thing had gone way too far.

I’m not a numbers guy, never claimed to be.  My primary strength when it comes to numbers has been to reduce complicated damages calculations to an explanation so simple that even someone as statistically illiterate as me can understand it.  If I can understand the numbers, anyone can.

But there’s something that doesn’t require you be a numbers guy or gal which bothers me about Silver-mania.

Silver cannot be wrong because his model, whatever it is, merely puts a likelihood of success on the election.  Silver’s model could predict a 75% chance of an Obama win, but if Romney wins the model was not wrong because his model allowed for a 25% chance of a Romney win.

If Silver cannot be wrong, how can he be right?  Heads he wins, tails you lose.  Calling all philosophy majors!

I’ll leave you with this.  My first experience with Silver-mania was in his pre-NY Times days, when he was covering the Coakley-Brown race.  According to Wikipedia, Silver’s 538 model was pure genius:

FiveThirtyEight writers Schaller, Gelman, and Silver also gave extensive coverage to the January 19, 2010 Massachusetts special election to the U.S. Senate. The “538 model” once again aggregated the disparate polls to correctly predict that the Republican Scott Brown would win.[40]

The source in the footnote was Silver’s January 18, 2010 blog entry, 538 Model Posits Brown as 3:1 Favorite posted at 5:26 p.m. the day before the election.

Well, yeah, duh, about 14 hours before the polls opened Silver joined the rest of the world in predicting a Brown win, but he was consistently predicting doom for Brown and was the last person to jump on board.

Here’s the pertinent chronology from my post in February 2010, Beware Statisticians Bearing Advice:

Nate Silver, January 5, 2010, Wicked Awesome Thoughts on Massachusetts Special Election:

Rasmussen is supposed to have a poll out tomorrow (Tuesday) on the Massachusetts Senate Special Election, which will take place on the 19th. There’s been some speculation, mostly from Republican blogs but also from some Democratic analysts, that the Republican candidate, Scott Brown, might have a chance, which would potentially wreck the Democrats’ chances to pass health care reform.

I’ll be curious to see what Rasmussen and the other pollsters (PPP? Suffolk?) have to say, and bears remembering that special elections are highly unpredictable affairs. But I’d be somewhat surprised if the election turns out to be especially competitive.

Nate Silver, January 14, 2010, OK, It’s a Toss-Up:

Earlier today I tweeted about how there wasn’t enough evidence to describe the Massachusetts special election as a “toss-up”, as some other forecasters have done, based on the information available to us at that time.

Well, now there’s some new evidence. And it isn’t good for Martha Coakley.In particular, the evidence is a Suffolk University poll that shows the Republican, Scott Brown, ahead by 4 points, 50-46.

Nate Silver, January 17, 2010, 538 Still Rates Massachusetts as Toss-up:

Please don’t be too enthralled with/scared by the specific numbers below. They’re based a number of assumptions which may not be valid. However, I agree with the characterization that the model comes to, which is that the Massachusetts special election should continue to be regarded as a toss-up.

Nate Silver, January 19, 2010, Defying Odds, Republican Brown Becomes Next Senator from Massachusetts:

In an outcome that would have been unthinkable just weeks ago, Republican Scott Brown has become the Senator-elect from Massachusetts.

Silver was as wrong as anyone could be in the Brown-Coakley race, yet came out a hero because his model predicted a 75% chance Brown would win late in the day before the election.

I neither a Nate Silver lover nor hater be. 

But when someone has a model which cannot be proven right or wrong, I’m not buying whatever it is he’s selling.

DONATE

Donations tax deductible
to the full extent allowed by law.

Tags: