Jay Cost doesn’t see malevolent intent, just a history of oversampling Democrats consistently across a variety of polling organizations:
The clear answer is: they skew Democratic. In fact, every recent registered voter poll with party spreads I could identify had a Democratic advantage that exceeded the quarter-century trend. And the average of all these polls together is 35 percent Democratic to 29.5 percent Republican, or D +5.5
Importantly, these polls show Obama with an average lead over Mitt Romney of 3.5 points. But if they have a 2.5-point Democratic oversample, then what we are really talking about is perhaps a 1-point Obama lead.
Incidentally, this puts these polls much more in line with the Rasmussen poll, which has consistently found a toss-up race. Right now, Rasmussen – a poll of likely voters – sees an R+1.4 advantage in party identification. That is entirely defensible, in my opinion, given the weakness in the economy….
Here’s my bottom line. It is very difficult to model the turnout for a presidential election this far away from November. There are a lot of tough choices that pollsters must make, and it is not fair to single any pollster out for the decisions it ultimately goes with. Nevertheless, we can and should still be smart consumers of political polling. We need to keep the historical spread between the two sides in mind, and be cautious of polls that show a relatively wide Democratic advantage over the GOP. They are probably underestimating the GOP’s electoral strength.
I’ve never seen anyone analyze the “hang up” effect, namely, that Republicans (particularly conservatives) hang up on pollsters more than Democrats (particularly liberals). If anyone has a link to a study of this phenomenon, even if debunking it, please post in the comments.
In the meantime, don’t get overly pessimistic about the polls. We have enough problems with Operation Demoralize without getting apoplectic.
Update: Reader Charles forwards this analysis by Nate Silver:
Over the past six presidential election cycles, likely voter polls have been 0.7 points to 2.5 points more favorable to the Republican candidate than registered voter polls….
For the time being, our model is giving Mr. Romney credit for a 1.6-point shift when it sees a registered voter poll and has him just a tiny bit behind Mr. Obama despite this. Needless to say, if the gap comes in toward the higher end of the historical range — or at something higher still, like a three-point difference in his favor — Mr. Romney’s chances of winning the election will improve.
DONATE
Donations tax deductible
to the full extent allowed by law.
Comments
According to Craig Shirley in Rendezvous With Destiny about the Reagan v. Carter battle, it wasn’t until the last few days before the election that the bottom began to drop for Carter. Let’s hope the same thing happens again!
I follow the poll analysts closely- there is no reason to be down about the polls. Rasmussen’s 1.4 R advantage is cautious at this point. From what I am reading we can expect R+3 turnout comparable to 2004 election. In 2010 we had an R+6 advantage which was a tsunami. If we could replicate that Obama would lose in a landslide. Jay Cost is right though it is very difficult for pollsters to gauge that at this point.
I am not sure why conservatives would hang up on pollsters but I will assume they have reasons. I believe it in our interests to answer them since the polls tend to set the tone for horse race coverage. It matters if we let them know we would crawl through broken glass to vote in November
I’ve seen many observers point to an R landslide.
1) Wisconsin, a Repub elected in Brooklyn, the ’10 midterm
all show anti-Dem strength
2) Dems appear to be flailing and wild. Panic mode.
3) Polls showing a close race oversample Dems
Apparently in many elections the polls show Dems leading or a close race till just before the actual election. At that point the pollsters have to show accuracy to ensure their client’s confidence.
Good pts, Jim. Also remember that Obama ceded 40% of the vote to virtual unknowns in state primaries. When Lyndon Johnson fared so poorly in 1968, many took it as an indicator that he was finished. And Johnson was up against the very well known Gene McCarthy.
Nonetheless, I’d breathe a little more freely if the polling showed an actual Romney lead.
Not only should they factor in hang-ups, but what about all the folks, myself included, who no longer have land lines. It seems most likely these are younger people- younger, transient professionals who use their cell phones exclusively. Pollsters don’t have access to these people, unless, like Zogby, they did on-line polls, which I guilty of ignoring because they are so long and tedious.
My husband and I are in our late 40’s and all of our children are old enough for their own cell phone, so last fall, we ditched the land line on which I used to get pollster calls. How many more are there like me?
If this election ends up being a +3 Republican election, we will win, according to the Cook Partisan Voting Index, not only Florida (29), Virginia(13), and Ohio (18) swingstates, but take in also Iowa(6), New Hampshire(4), Colorado(9), North Carolina(15).
We also would win: Nevada(6), New Mexico(5), Wisconsin and Minnesota (with their 20 electoral votes)(!), and we would win PENNSYLVANIA (20) while coming in a close 2nd in New Jersey and Oregon(!!!)
Here is a map with beautiful red plastered all over it with a R +3 victory…
http://www.270towin.com/2012_election_predictions.php?mapid=pbH
Conservatives hang up because they have seen too many pollsters manipulate polls to politically disadvantage their cause. Why should they cooperate in that?
THe only poll that counts is the one that happens in November. If anyone thinks this Chicago Pol is not going to do any conceivable thing to stay elected you are naive. The dirty games haven’t even begun.
I am glad to see you writing about Operation Demoralize! This scheme needs more press!
It has been proved repeatedly that Rasmussen polls are among the MOST ACCURATE of all. Unlike the lunatic-left fake polls conduced by Washington Post, NYT, abc news, cbs, and the other lamestreamers, Rasmussen does an honest, real-life sampling – not a skewed mixed designed to favor the d-cRAT socialists by INCREASING the percentage of blacks and other lunatic-left leaning groups (as PPP, GfK/AP and the Washington Post, in particular, do) or INCREASING the sampling in blue-states and cities (as the NYT and A(lways)B(iased-socialist)C(lowns), in particular do), or inflating the percentage of d-cRAT socialists in the sample (as ALL of the leftist pollsters/propagandists/LIARS do).
Believe Rasmussen – ignore the others.
Two words…
Preference cascade.
Keep your heads up. I predict a sea-change election.
Agree with Mary Sue up there^^. Mainly I look at RCP averages because they seem to use a fairly good group of pollsters. I stay away from Politico’s choices as they sure strike me as cherry picked.
And no I dont draw too much from the polls beyond the obvious and consistant things like Romney’s strength relative to 2008 and McCain. RCP has just added that feature so comparisons to 2008 are easy via single click.
And as for Carter V Reagan…most polls had it too close to call right up to the election. Not that we should bank on that but its sorta a optomistic thought.
RCP shows an average lead by Obama at just under 2% today and whats actually remarkeable is how close the various polls are with one another. Go figure…the recent Fox poll shows the largest lead for Obama by +4. Without that..the average would be just a bit over +1 for The Pharoah.
I orgot to mention why polls dont bother me at the moment.
Obama can go on and on about telling his story and other nonsense. Two facts remain that he cannot outrun or outsmart.
His economic legacy can be summarized as:
1. Presided over the longest period of unemployment over 8% in recent history
2. General Moters
http://money.cnn.com/quote/quote.html?symb=GM&iid=HP_Last5
I can tell you that the analysts I follow hold Fox’s pollster in fairly low esteem. I agree with you completely about keeping in mind how bad polls can skew the averages too. If anything I am pretty optimistic about where Romney is in the polls right now. I guess the registered voter polls are masking a 2 -3 point Romney lead. We probably will not see Romney taking a lead in polls until the conventions or whenever the pollsters switch to likely voter polls so keep that in mind.
The most important thing right now is that Romney hasn’t been knocked out despite Obama’s best efforts. Obama has an uphill climb and destroying Romney early is really one of the few cards he had to play.
1. It looks like this post was cut off in mid-word.
2. I’ve never seen anyone analyze the “hang up” effect, namely, that Republicans (particularly conservatives) hang up on pollsters more than Democrats (particularly liberals). If anyone has a link to a study of this phenomenon, even if debunking it, please post in the comments.
I only know my own case. A few months ago I got a robo-polling call, supposedly from Rasmussen, about the economy. As the questions progressed, they became so personal that I hung up.
After that incident I no longer respond to polls: I don’t want my opinions, together with my name, put into databases that are sold to all comers. Moreover, many “polls” are just come-ons for fundraising pitches.
3. Judging by the moonbat site I monitor, the Left has not skipped a beat. They switched seamlessly from blaming the Tea Party for the CO atrocity to blaming the Right for rebutting the Tea Party smear.
When you hate, you really limit what you can even consider. Hence, I am not suprized by what you read from the moonbattery.
BTW, appreciate you taking my six the other day. I am a nice guy. Not an easy guy…but nice.
It was a small thing I did, but you’re quite welcome.
Damned polls and statistics are dodgy…
Bless WAJ for trying to wade through the muck and the mire to bring some clarity… although that inherently revolves around some analysis and postulation.
Just remember Obama only won in 2008 by 8 million votes, and there are now 25 million out of work due to his voracious ineptitude.
We will prevail despite efforts to demoralize…
For a righteous man falls seven times, and rises again.
But the wicked are brought down by calamity.
I’d be curious to hear more about the hang-up effect as well. I can tell you I hang up on every robo-call or pollster call that comes to my phone.
If I don’t get an answer to my hello right when I say it, then I know I’ve been auto-dialed and I’m being switched to the next available operator. That’s my hang-up cue.
Now, I don’t math well, but if Obama is up 3.5%, but there is a 2.5% oversampling of Dems, his lead isn’t going to be “perhaps . . . one point.” Operating on the assumption that that oversample came mainly at the cost of Republican voters, you’d have to subtract the 2.5% oversample, THEN factor in that some, maybe even most, of the corrected oversample is probably going to go for Romney. Thus, Romney’s probably slightly ahead.
The reason there aren’t any studies is that the “respondent rate” is considered proprietary information by the polling companies: they don’t want their competition to know what theirs is.
My understanding based on conversations with someone in the academic end of it is that when they first started measuring in the ’80s, the respondent rate was over 75%; 25 years later, according to my source, it is down in the 25% range. This makes it more difficult to get a truly random sample.
The idea that Republicans and/or conservatives were more likely than Democrats and liberals to decline being polled comes from anecdotal experience when the Republican turnout often exceeds predictions. But there is no scientific basis for the claim that I am aware of.
The scientific evidence is apparent, liberals believe if they vote and possibly donate $3 they’ll get a seat at the table.
Conservatives want to be left alone, will work hard and eventually buy their own damned table.
I usually hang up on pollsters. Seriously, I get 2-3 calls a week, and I never let ’em get past the “poll” or “survey” word. Why? I’ve been kept on these stupid product opinion polls for 15-20 minutes too many times. However, of late, I stay on the call because it’s the political season. In the past month, I’ve been called by Dewhurt’s people twice, which is evident by their questions.
As I see it, a big problem with polls is that too many of them have these statements that you must rate from ‘strongly disagree’ all the way to ‘strongly agree’. Too often, the statements are rambling, indirect and obtuse, resulting in something you disagree/agree with, in part. Further, one tends get exhausted after 15 to 20 of these questions, and I think it often reflects that in the polls.
If anyone has a link to a study of this [hang up] phenomenon, even if debunking it, please post in the comments.
I’m a little baffled as to how one would conduct this study. Do you call them back and ask why they hung up?
This occurs off the top of my head…
Correlate the “hang-ups” against voter party registration rolls.
How hard is that?
Obama’s Enemies List
July 19, 2012, 7:20 p.m.
First an Obama campaign website called out Romney donor Frank Vandersloot. Next the IRS moved to audit him—and so did the Labor Department.
Clearly a case of under-sampling those Democrats.
Yes & the one way to finally break Americans teetering faith in the Feds is to publish the political donations of the IRS staff. It is out there but has not been picked by by the wider media.
If citizens connect that to the individual & collective efforts of the IRS & if they see these as punitive then all hell will break out.
It is a simple discretionary matter to prioritize investigations after having checked the targets party registration , the electoral district results & make an educated guess.
The ERA used the Crucify 5 tactics in the SE states before being picked up by a slip up.
I am sure most IRA staff are ethical but if some have to suffer for the sins of others -well too bad.as the Left would call.
haha IRS not IRA.
The IRS & The IRA is another topic entirely .
The Minneapolis Star Tribune’s Minnesota Poll is one very biased poll, one that is typically well reported by local media types, who typically ignore its bias. I’ve heard some local folks mention that they heavily discount the Minnesota Poll’s results yet not sure that’s the best way to deal with that bias.
Being aware of the under-polling that goes on might be the only way to prepare for this election. And, especially not to become disheartened by poll results between now and November 6th.
I am not very smart, but I see a “hole” in the poling. For example, poling shows that 18-29 year olds favor Obama. Similarly, 60% of Hispanics favor Obama. Single females favor Obama. Blacks overwhelmingly favor Obama. My question is how many of the 18-29 year old single Hispanic women are included in each individual category? Let’s give her a name, Maria. Is Maria counted 3 times, once as a single female, once as an Hispanic and once as an 18-29 year old? Am I missing something?
Or… since she’s a Democrat, she gets to vote three times.
A plus 1 for Obama after all they’ve spent is disastrous for them. This is very very near a tipping point.
I don’t answer polls anymore, in fact though I have a landline I don’t answer it. If someone wants to get me on the phone leave a message and I will call back, otherwise call me on my cell phone, the number I give out sparingly. If by chance a pollster gets through to me I hang-up like I do on the telemarketers. I am in a swing state and we get inundated with political calls. Also, I am becoming more and more wary of who is asking questions. It’s called self-protection.
The only way to poll is to go to a mall & offer $50 for shoppers to fill out a form in private.
You can choose your sample pretty easily. People are remarkably honest if you pay hem.
Of course that is if you really want to know.
I can’t remember the last time I went to the mall. I don’t believe it was 2012. The principle is the same, but the sample might be skewed. Pay your phone poll responders. We’ll stick around for a couple of spare bucks. If Nielsen or Gallup want my time, they can pay me.
So what about the years before 2009? We know he turned over more than two decades of returns to the McCain campaign during the veepstakes vetting process. What was in them? “Mitt’s taxes were complex, but clean. He overpaid his taxes.
You’ve undoubtably seen Dick Morris’s analytics that the undecideds breaking for the challenger NOT the encumbent. So if you see 45 Obama, 45 Romney and 10% undecided, at election time, the 10% NEVER go to the incumbent. His math on this goes back 30-40 years.
As per hang ups- annectdotally, I used to hang up. Now I lie and give liberal answers to skew the poll in favor of the libs.
I can only speak from personal experience about the “hang up effect”. We have two little ones, so there is no time to talk to the pollsters.
Once I got a call from pollster regarding a state issue, don’t remember which. I was walking to a nearby grocery, and I decided to talk to him. I remember talking to him as I was walking through the isles. After that — never.
I don’t know if the word got out that I answer such calls or what, but I started getting lots and lots of them. I blocked the numbers. A married woman, I’m in GOP demo.
Look at it this way. The media has a vested interest in making Conservatives, and conservative candidates look like they are losing. They have an institutional bias to the political left. Over the past 10-12 years, this has become increasingly evident with the rise of the interwebs, and a countre voice, as well as FoxNews (which at least pretends to give both sides of any given argument/issue).
Because of this, Legacy Media, must show that conservative views/policies belong to a small minority of the voting/resident poplulation. Additionally, this reinforces their bias that those precepts they hold dear, are in fact the same held by a majority of the nation.
This isn’t so, at least with likely voters. Furthermore, outside of major metropolitan areas, voters are much more likely to at least “lean” conservative…because it’s not in their best interest to pay higher taxes, and support layabouts and donothings…with the exception of Metro-Balt/DC/NorthernVA…who are mostly employed by the bloated Fed bureaucracy.
I suspect that as things stand right now, Obama will lose, perhaps by as much as 7% points. My neighbors in the heavily Jewish enclave of Pikesville, MD (just outside of balt) are…highly disenchanted with Obama, and the direction the leadership of the Democratic party has taken the country. When you toss in the massive tax increase that Martin O’Malley foisted on us, they are beginning to squirm. Hiring is stagnant and has been since Oct. 2008, with very few businesses hiring new employees because of the weight of new State/Fed regulations (mostly from ObamaCare, but certainly new reg’s from MD).
I don’t think Obama will win this Nov, and with his comments last weekend that the Gov’t is responsible for all of “your success”, “you didn’t do it, someone else did.” That there is now a strong chance that the GOP will increase their majority in the House, and possible take enough seats in the Senate, to gain control of that as well.
There are two very important developments in polling that are not mentioned publicly by pollsters.
1) The non-response rate,
and
2) The rate at which people lie to the polls.
The mass turn over to cell phones and unlisted land line phone numbers is playing hob with whether political polls are in fact statistically representative of the actual voting public.
The polling companies just don’t know and the only alternative is much more expensive face to face polling surveys. Most media political polling companies cannot do them let alone do them well.
The second issue of lying is even more touchy with pollsters because telemarketers — in addition to making people less likely to answer without call screening — also makes people actively hostile and much more likely to lie than answer truthfully.
The extent of the second problem was explained to me in 2008 by a friend who moonlighted with political polling firms from his academic social researcher day job.
He explained to me what it would take to do accurate political polling election model, if you assumed a “hostile field protocol,” after a political blogger I read spotted the following in a TIPPs poll.
The TIPPs poll asked the following question:
“Do you display the American Flag”
…with the result being: “Obama 38/ McCain 52/ undecided 9.
and the political blogger stated something to the effect:
“People who wave a flag are undecided? Does anyone really believe this?”
For which my day-job academic said the following:
Your blogger seems to have hit on the idea of hostile field polling questions without being a pollster.
Yeah, that would be the start of it; if I were getting paid to do it and I thought of the flag question, I’d use it.
Typically for a sample size big enough to cover 67 stratifications (the current [2008] minimum number you need to adequately cover the American public — you’d need more to do it state by state, maybe as much as 3x as many) you need around 1200 respondents.
To do a good ‘hostiles protocol’ you need about 15 stand in questions for the one that you think they’re lying about (if you think they’re lying about more than one, you can overlap some questions, but obviously that produces a spurious correlation in the results. Or at least that’s obvious to me and the people who know what they’re doing; I’ve seen a lot of researchers screw that one up).
The 15 stand in questions all need to correlate with each other at about 60% or better.
And now you see why no network even tried it this time (2008).
To get those 15 questions you probably would need to test around 100 good ideas, in about 25 nationwide surveys (since they won’t all start out in the same survey and you have to see how they interact, so you’d test say 7 surveys in your first round … maybe 3/4 of questions would look promising in the second round, so you’d test 5 …. then 3 … then start doing mix-and-matches that would easily use up another 10 surveys).
You’d have to pay for 25 polls to make your polls be accurate again.
Forget all ideology; Fox wasn’t going to pay for anything like that any more than MSNBC.
By 2012, though, they may all have to.
If anyone has heard of any whisper of any of the major political polling firms that has done 24 national polls, of 1,200 people each, just testing out “how is the public lying to pollsters this election” questions?” Please mention it!
At a guess it would cost $100,000 to $250,000 to set up the initial questions, with $50,000 of analysis per poll adjustment, and $ 500,000 per poll using best face to face polling techniques to get that answer.
And the answer would only get you the _Right Polling Question Methodology_.
You would still have to do the polls using that methodology.
And to be sure, you would have to take the same poll via face to face, and via phone only, to know the difference between the two polling methods so you could sell them via cost/quality/speed to your political consumers.
The trick is to sell the “right polling question methodology” answers to each of the real polling companies. Divide the cost of getting the adjustments among a number of polling companies and suddenly the cost doesn’t look so bad.
From what I understand, polling companies do not sell proprietary methods to one another in an election cycle.
People who invent the new methodology lose their best people to other companies, that hire them away from the guys with the new methodology, after an election cycle.
That is why the best pollsters are strict partisans for party candidates who don’t sell their stuff to media outlets.
Another point my polling semi-pro made was that the reason Bill Clinton was so good, and one of the reasons Hillary got blind sided by Obama, is that Bill Clinton kept the same polling team from two years before he was president through his entire presidency.
IOW, “Methodology” is not just the right questions. It is also having an experienced team of poll takers and analysts who understand what the data is telling them in time to be useful to the candidate. For that there is no substitute for an experienced polling team.
Hillary’s New York Senate campaign was deeded that 10-year old polling team by Bill.
She got rid of that team for money burn rate and internal campaign power reasons before she started her Presidential run. (This is the heart of the enmity between Morris and Hillary.)
That was why Hillary Clinton was always a day late and a dollar short with Obama. Her replacement polling team was no where near as good as the one Bill gave her, even if it was “cheaper.”
The old saying that “There nothing more expensive than a second best military” also applies to presidential campaign polling firms.
You’ve undoubtedly seen Dick Morris’s analytics that the undecideds breaking for the challenger NOT the incumbent.
According to the academic I mentioned above, that is a date dependent statement.
Overall, undecided tend to break against the incumbent. This is why the 50% approval number for incumbents is so important for them.
However, _When_ they break in a Presidential race is hugely important, almost as important as by how much.
Undecideds that break before early September overwhelmingly break for the challenger.
Those who break after early September break for the incumbent.
In so many words, challenger party leaning partisans and aligned independents commit early to their guy.
Incumbent leaning loose partisans and aligned independents break for their guy in October.
True independents at the last few days before the election are generally incumbent leaning voters who cannot resolve their distaste between the incumbent and the challenger.
Good years for the incumbent see them hold their nose and vote.
Bad years for the incumbentsee them stay home in large numbers or vote for the incumbent, usually the former.
This is why you see the Democratic tradition of the “October surprise” against incumbent Republican Presidents and why Clinton in in 1996 and Obama now spend huge wads of cash early in swing states.
Obama’s “failure” in his recent heavy campaign spending in swing states against Romney should be judged on that basis.
Obama’s team is doing damned well to stay afloat with their partisans in this economy.
The political poll is readily manipulated by the poller and the polled. Responding to one simply encourages fraud.
Political polling is a social disease.
I am a likely Romney voter. My phone system is Ooma (VoI) which allows you to block an infinite number of calls from any phone numbers, which I do frequently.
The newer Panasonic cordless phones also allow to to call block up to thirty numbers.
Needless to say my preferences will never appear in polling.
New technology is allowing people to effortlessly pre-screen phone calls.
Over the past six presidential election cycles, likely voter polls have been 0.7 points to 2.5 points more favorable to the Republican candidate than registered voter polls….
*dryly* Did they check places with clean voter roles vs those with vote-by-mail that don’t clean their registries?
Washington is HORRIBLE– My folks got a ballot for me several years after I moved.
[…] PROF. JACOBSON: About Those Polls. […]