The Trail: A Daily Diary of Campaign 2008

Archives

The Pollster

What if the Polls Were Right?

By Jon Cohen
Democratic Pollster Peter Hart has a contrarian view on the latest polling kerfuffle.

Hart, one-half of the polling team behind the NBC-Wall Street Journal poll, this afternoon posted an insightful online comment to a Washington Post article about the failure of New Hampshire polls to predict Sen. Hillary Rodham Clinton's victory there.

Maybe they weren't wrong after all.

Instead, Hart contends, we've just witnessed a repeat of 1948, the year of the polling industry's most famous failure. That year, public polls stopped collecting data on the presidential election in early- to mid-October, thinking New York Gov. Thomas E. Dewey had the race sewn-up. After Election Day, the Detroit Free Press ran the box-score "Truman 304, Pollsters-0."

In the vastly accelerated political world of 2008, perhaps polling only two, or three or three-and-a-half days of what was a five-day campaign after Iowa was the functional equivalent of stopping polling weeks before the election in 1948.

Unfortunately, data to fully test this hypothesis aren't yet public (AAPOR, the American Association of Public Opinion Research, and others have called on pre-election pollsters to release their data for analysis).

The network exit poll's "time of decision" question is too crude a gauge to offer a clean read on late-deciders. Polls are good at measuring opinions; they are less accurate on assessments of past behavior. And of the question's five options for answers, three were within the week, potentially leading to an overstatement of the percentage of those making up their minds just in advance of elections. (When did you finally decide to read this blog -- just now, in the last 10 seconds, sometime in the last minute, this morning or before that?)

One intriguing tidbit in the available data -- testable with other surveys -- is that comparing exit poll numbers to the last poll from CNN-WMUR-Univ. of New Hampshire shows a much bigger movement to Clinton among women than among men. In the pre-election poll released Sunday, 34 percent of women supported Clinton, while the exit poll showed 46 percent of women voting for the New York senator. The change among men was plus four percentage points.

Is this partial evidence for the "tear" effect so many have latched onto? A return to the "gender gap" polls showed before Iowa and the media storm that followed? As was evident last night, much more data analysis -- and more elections -- await.

Hart's full comment is available below, as is an analysis from GOP pollster Neil Newhouse, who served as the other half of the NBC-Journal polling brain-trust when Bill McInturff worked on the McCain campaign last year. Newhouse foresees a great deal of pollster soul searching in the days ahead. (Yes, we have souls.)

From Peter Hart,"It is deja vu all over again."

The problem with the polling performance in New Hampshire is exactly the same as it was some 60 years ago with the "shocking win" election of Harry S. Truman. The pollsters concluded their polling before the voters made up their mind for the final time. Because this election was but a five day campaign, stopping polling after day 3 (Sunday) was equivalent to the 1948 presidential polling when they concluded in early- to mid-October. In a five day campaign, it was a mistake to think the final decision would be made "so early." It is after all, New Hampshire. The motto of the state should be: "where big mo comes to die." Add Senator Obama's name to a long list to other "sure NH winners" -- George H. W. Bush in 1980, Mondale in 1984, and George W. Bush in 2000.

From Neil Newhouse, "We just didn't see it coming!"

With Hillary Clinton's victory last night, any shred of reputation that pollsters have for being accurate barometers of public opinion goes out the window.

The problem I have with most explanations for why the polling was so far off is that the very same pollsters that blew the call on the Democratic Presidential Primary last night nailed the GOP primary results. So, how could they get one so wrong and the other so right? (Sounds like the beginning of a bad country and western song.)

Wouldn't a late shift toward Hillary have impacted the GOP primary, too?

(And, no, there is no need to call for a federal investigation into Clinton vote-rigging; someone would have come forward by now!)

Actually, we've seen a similar pattern of polling not matching actual vote in a couple of instances involving well-known African American candidates where, in a competitive election, pre-election polling shows African American candidates receiving significantly more white votes than the candidate actually gets on election day. (Though, nothing quite like last night's turn-around.)

The most immediate problem with that theory is that the Iowa polling and caucus results sure didn't seem to reflect that behavior. (There had to be an exception.)

Regardless of the specific reason why the polls were wrong last night, one thing's for sure -- this is an election campaign for the books, providing an abrupt back-to-earth lesson for all of us who study public opinion for a living.

Posted at 7:55 PM ET on Jan 9, 2008  | Category:  The Pollster
Share This: Technorati talk bubble Technorati | Tag in Del.icio.us | Digg This
Previous: Bill Richardson to Exit Race | Next: Obama Draws Throng in Jersey City


Add 44 to Your Site
Be the first to know when there's a new installment of The Trail. This widget is easy to add to your Web site, and it will update every time there's a new entry on The Trail.
Get This Widget >>


Comments

Please email us to report offensive comments.



there is the professional world of warcraft power leveling here. welcome.

Posted by: jimelyyes | May 7, 2008 5:46 AM

there is the professional world of warcraft power leveling here. welcome.

Posted by: jimelyyes | May 1, 2008 10:46 PM

Posted by: kevinlarmee | January 15, 2008 6:05 PM

This observer does not really understand the need for constant polling and the publishing of constant results. I believe that the way people think and vote is affected in a negative way by such polling. There is a bandwagon syndrome that comes about that wouldn't be there to the present degree without constant polling. Peoples votes are often affected by what they see on television, even on election day itself. Let people make up their minds on their votes through thoughtful study rather than soundbite poll results. Our Country and our Democracy will be better served by its media and by its electorate.

Posted by: HaroldFCrockettJr | January 15, 2008 11:50 AM

Actually, there were three fundamental errors in the polls, and one of them may be unfixable.

The first fundamental error was the wrong choice of model. Pollsters used the standard model of a primary. They chose the model that does not consider interrelations between the Democratic and Republican Primaries. In most primaries it is enough to see what each party is doing. In this election there will be too much inter party migration, and those voters, who show up as undecided, are harder to account for.

The second error was in fact the real Dewey error. Pollsters distributed undecideds as if they would split like the decideds split. when there is a significant body of undecided voters, you CANNOT assume that they will vote like the rest of the population.

The third error is probably unfixable for a while.

Unless your sample properly resembles your population, your results will be unreliable. This is the Landon error. Pollsters trying to call the Roosevelt Landen election relied on telephones for polling. Since at the time, telephone ownership was skewed heavily to the more wealthy, and there for more Republican, portion of the population, The telephone pollsters saw a strong Landon landslide. FDR thought about as much of pollsters in 1936 as Truman did in 1948. More and more people are using strictly cell phones for their telephone communications, and are therefor not in the sample used by virtually polling organizations as their base. These voters are NOT REPRESENTATIVE of the population as a whole. Leaving them out of your sample makes your sample nonrepresentative, and if those people were highly likely to vote in a particular way, say for either Obama or McCain, then problems one and two, added to problem three, makes your poll highly unreliable.

Maybe for the next ten to fifteen years we can pay a bit less attention to a deeply flawed, and currently unrepairable, system.

Posted by: ceflynline | January 13, 2008 3:03 PM

Two points:
[1] It seems to me more likely that any machine-rigging was part of a larger-scale campaign than one candidate would manage. That is, I suspect it would be more likely performed by a small group of "Republicans" than by Hilary Clinton herself. The goal is to end up with the Democratic candidate they think they can beat. All these machines are still produced & owned by Republicans... I think of the group behind this as "Karl-Rovians," actually.

[2] This is very important: It is not enough simply to "watch" or "guard" the electronic voting machines. Based on first-hand experience I can't discuss here, I have plenty of evidence that it is possible to remotely affect the operation of one PC from another nearby PC/gizmo, even when they are not connected by a wired network, and the affected machine has no known bluetooth capability. I do not know how this is accomplished, but I have witnessed it. The second machine seems to need to be within 10-15 feet of the affected machine, but it can operate through walls, ceilings, etc. An electronic voting machine is merely a sort of PC, so I've been afraid for some months now that this would happen.

Posted by: techwatcher | January 11, 2008 12:59 PM

Well, what ABOUT a recount? Did anyone notice an unusual amount of waddling Duck People at the polling places? Or was Carville up there scaring voters away by showing his Beelzebubba-ish face and intimidating those of French Canadian descent by cursing at them in Cajun? Who WAS that furtive-looking Oriental guy rumored to be handing out spiked fortune cookies to voters in Manchester?

Posted by: sawargos | January 10, 2008 5:29 PM

Yo, Obamites! That wasn't racism; it was the Tear Bag (kayoed Muskie in NH in 72, but, however genuine, worked like a charm for Ankleless Annie among distaff Demos et al.)! That, er, tactic (?) can't be re-done, though it sure as hell can be re-run; so hail & beware The Senatrix's reincarnation as ... the "Tear BAG!" Oprah time again.

Posted by: sawargos | January 10, 2008 5:20 PM

If Obama does not ask for a recount by tomorrow, it's too late, right? This will give us some foresight into whether he will fight or just roll over like Al Gore did in 2000.

Posted by: JakeD | January 10, 2008 4:25 PM

I have looked at the numerous allegations of voter fraud in New Hampshire and I too believe that Ms. Clinton stole the New Hampshire election. There should be a recount.

Posted by: orinleevp62 | January 10, 2008 2:29 PM

I get difficulties to understand it. All Polls were wrong!!! Not even one was a bit closer to the results. Why did Iowa get it right. And if they were wrong how will we be convinced to trust other Polls in future? Always 1 plus 1 is 2 not 3!!!

Posted by: fridamulindayahoocouk | January 10, 2008 1:28 PM

How many absentee ballots were cast? Could it be that the outcome of the election was decided before the polls were open? And maybe that the polsters were right?
billybob

Posted by: bilmor | January 10, 2008 12:05 PM

Reason for a recount in NH?

here's a report of a provocative pattern with regard to the votes cast in the Democratic primary in New Hampshire. As the article details

http://digg.com/2008_us_elections/Reason_for_a_recount_in_NH


.

Posted by: usadblake | January 10, 2008 11:38 AM

Someone said, "there's just no point in propagating conspiracy theories when you could be out there persuading people to vote". I disagree.

The polls were right in Iowa. The polls are wrong, only when en-masse diverts from the truth.

True, weather was sunny and 60, an opportunity for aarp to get out while the younger crowd plays. College was not in session because hence the early primary date move.

Too close to be just a little bit wrong in the polls. that may be what someone is betting on. An investigation needs to commence in the event of a dark and sinister element at work.

This happens to be politics where all is at stake.

Posted by: jim | January 10, 2008 11:28 AM

I've posted examples from Obama's blog of his supporters already crying "RACISM!" too.

Posted by: JakeD | January 10, 2008 11:02 AM

biffle ,stop whinning . over 90% of NH uses optical scanners.

blake, crying racism? So did Clinton loose in Iowa because of Sexism.
Grow UP!

Posted by: newagent99 | January 10, 2008 9:51 AM

Race Factor, the "Bradley Effect?

In your opinion did the race factor throw off New Hampshire polls?

http://www.youpolls.com/details.asp?pid=1492

.

Posted by: usadblake | January 10, 2008 9:30 AM

Volcanojon says "The Clinton machine has been setting up New Hampshire for at least eight years. They could have rigged it."

Yes, or maybe the fact that they've been "setting up" NH for 8 years meant that they had a superb get-out-the-vote and ground operation.

Don't get me wrong, I'm an Obama supporter; there's just no point in propagating these conspiracy theories when you could be out there persuading people to vote Obama.

Posted by: s361883 | January 10, 2008 5:21 AM

The Clinton machine has been setting up New Hampshire for at least eight years. They could have rigged it. Who can or will investigate?

Posted by: volcanojon | January 10, 2008 5:07 AM

EVERYONE, to one extent or another, can change their minds. There's a difference between being currently perplexed, and being open to new facts that might later appear - but that's not how the question was asked, was it? Two very different scenarios were combined in one yes/no question.

I still think the change was that women didn't want to see the first serious woman candidate humiliated and embarrassed by losing in the double-digits, as the polls were predicting. The 'last minute' difference was not who they supported so much as their determination to vote even if it meant waiting in line for awhile. The numbers went up. I'd guess that the answer to the question "are you likely, very likely, somewhat likely or not at all likely to vote?" is what really changed. The "somewhat likelies" turned out on the day.

Meanwhile, the polls encouraged the Obama supporters to be complacent and not make that extra effort to get even more people to the polls.

"The only poll that counts is the one in the election booth."

Posted by: TomJx | January 10, 2008 3:08 AM

The pollsters and those in the media who mis-used the horse race polls to coronate Obama and/or bury Clinton overlooked A CRITICAL FACTOR in the New Hampshire data: 35 to 40% were "undecided or could change their mind."
If that many people tell the pollster they're undecided or could change their mind, the pollsters should have reported the results as being too close to call.
As a consequence, I'm seeing a lot of crow pie eating today and justifiably so. The pollsters and media blew it.

Posted by: iopsc | January 9, 2008 10:41 PM

As unsettling as it might be, consider the following possibility:

Millions of Americans over 40 have admired the rhetoric of Martin Luther King and what he accomplished - from a comfortable distance. They do not consider themselves racist, but recognize roles and differences.

Inside the privacy of the polling booth, maybe Obama's message of change is both reminiscent and scary to some.

Just a possibility.

Posted by: gicgltd | January 9, 2008 10:40 PM

If Hillary wins the nomination, and then the election, I hope she'll remember the time the polls were wrong in NH, and -- as president -- make a decision every once in a while that she thinks is right, polls be damned. She might still learn how to lead now that she has reasons she can love to distrust polls.

Posted by: elatia | January 9, 2008 10:20 PM

Voters vote a pollsters don't vote.

I believe the NH vote is an indictment of the news medias concious and unconcious attempts to create the news and manipulate the public.

Posted by: hhkeller | January 9, 2008 9:50 PM

taking an average of the last polling out of NH showed that Obama at 38.3% which is relatively close to what he got with 36%. The polling just failed to get the Clinton surge at the end which averaged 30% vs. the actual 39%. She was polling mid to high 30s in late December so it could be those voters returned to her very late.

Posted by: labrat94720 | January 9, 2008 9:48 PM

..thank god Hill pulled it off. No need to annoint a rookie before we even know what he stands for. LET THE RACE BEGIN!

Go Hill !

Posted by: aboyzboi | January 9, 2008 9:26 PM

There is some warranted suspicion growing that the vote was indeed rigged. Obama's results at paper ballot locations reflect the polling results, whereas his results at electronic locations do not.

http://www.legitgov.org/nh_machine_vs_paper.html

The only way to know is for a candidate to call for a recount before this Friday.

Posted by: biffle | January 9, 2008 9:16 PM

What if the polls were right AND Hillary Clinton stole the primary?

Posted by: JakeD | January 9, 2008 8:44 PM

The comments to this entry are closed.

 
 

© 2009 The Washington Post Company