What if the Polls Were Right?
By Jon Cohen
Democratic Pollster Peter Hart has a contrarian view on the latest polling kerfuffle.
Hart, one-half of the polling team behind the NBC-Wall Street Journal poll, this afternoon posted an insightful online comment to a Washington Post article about the failure of New Hampshire polls to predict Sen. Hillary Rodham Clinton's victory there.
Maybe they weren't wrong after all.
Instead, Hart contends, we've just witnessed a repeat of 1948, the year of the polling industry's most famous failure. That year, public polls stopped collecting data on the presidential election in early- to mid-October, thinking New York Gov. Thomas E. Dewey had the race sewn-up. After Election Day, the Detroit Free Press ran the box-score "Truman 304, Pollsters-0."
In the vastly accelerated political world of 2008, perhaps polling only two, or three or three-and-a-half days of what was a five-day campaign after Iowa was the functional equivalent of stopping polling weeks before the election in 1948.
Unfortunately, data to fully test this hypothesis aren't yet public (AAPOR, the American Association of Public Opinion Research, and others have called on pre-election pollsters to release their data for analysis).
The network exit poll's "time of decision" question is too crude a gauge to offer a clean read on late-deciders. Polls are good at measuring opinions; they are less accurate on assessments of past behavior. And of the question's five options for answers, three were within the week, potentially leading to an overstatement of the percentage of those making up their minds just in advance of elections. (When did you finally decide to read this blog -- just now, in the last 10 seconds, sometime in the last minute, this morning or before that?)
One intriguing tidbit in the available data -- testable with other surveys -- is that comparing exit poll numbers to the last poll from CNN-WMUR-Univ. of New Hampshire shows a much bigger movement to Clinton among women than among men. In the pre-election poll released Sunday, 34 percent of women supported Clinton, while the exit poll showed 46 percent of women voting for the New York senator. The change among men was plus four percentage points.
Is this partial evidence for the "tear" effect so many have latched onto? A return to the "gender gap" polls showed before Iowa and the media storm that followed? As was evident last night, much more data analysis -- and more elections -- await.
Hart's full comment is available below, as is an analysis from GOP pollster Neil Newhouse, who served as the other half of the NBC-Journal polling brain-trust when Bill McInturff worked on the McCain campaign last year. Newhouse foresees a great deal of pollster soul searching in the days ahead. (Yes, we have souls.)
From Peter Hart,"It is deja vu all over again."
The problem with the polling performance in New Hampshire is exactly the same as it was some 60 years ago with the "shocking win" election of Harry S. Truman. The pollsters concluded their polling before the voters made up their mind for the final time. Because this election was but a five day campaign, stopping polling after day 3 (Sunday) was equivalent to the 1948 presidential polling when they concluded in early- to mid-October. In a five day campaign, it was a mistake to think the final decision would be made "so early." It is after all, New Hampshire. The motto of the state should be: "where big mo comes to die." Add Senator Obama's name to a long list to other "sure NH winners" -- George H. W. Bush in 1980, Mondale in 1984, and George W. Bush in 2000.
From Neil Newhouse, "We just didn't see it coming!"
With Hillary Clinton's victory last night, any shred of reputation that pollsters have for being accurate barometers of public opinion goes out the window.
The problem I have with most explanations for why the polling was so far off is that the very same pollsters that blew the call on the Democratic Presidential Primary last night nailed the GOP primary results. So, how could they get one so wrong and the other so right? (Sounds like the beginning of a bad country and western song.)
Wouldn't a late shift toward Hillary have impacted the GOP primary, too?
(And, no, there is no need to call for a federal investigation into Clinton vote-rigging; someone would have come forward by now!)
Actually, we've seen a similar pattern of polling not matching actual vote in a couple of instances involving well-known African American candidates where, in a competitive election, pre-election polling shows African American candidates receiving significantly more white votes than the candidate actually gets on election day. (Though, nothing quite like last night's turn-around.)
The most immediate problem with that theory is that the Iowa polling and caucus results sure didn't seem to reflect that behavior. (There had to be an exception.)
Regardless of the specific reason why the polls were wrong last night, one thing's for sure -- this is an election campaign for the books, providing an abrupt back-to-earth lesson for all of us who study public opinion for a living.
Please email us to report offensive comments.
Posted by: jimelyyes | May 7, 2008 5:46 AM
Posted by: jimelyyes | May 1, 2008 10:46 PM
Posted by: kevinlarmee | January 15, 2008 6:05 PM
Posted by: HaroldFCrockettJr | January 15, 2008 11:50 AM
Posted by: ceflynline | January 13, 2008 3:03 PM
Posted by: techwatcher | January 11, 2008 12:59 PM
Posted by: sawargos | January 10, 2008 5:29 PM
Posted by: sawargos | January 10, 2008 5:20 PM
Posted by: JakeD | January 10, 2008 4:25 PM
Posted by: orinleevp62 | January 10, 2008 2:29 PM
Posted by: fridamulindayahoocouk | January 10, 2008 1:28 PM
Posted by: bilmor | January 10, 2008 12:05 PM
Posted by: usadblake | January 10, 2008 11:38 AM
Posted by: jim | January 10, 2008 11:28 AM
Posted by: JakeD | January 10, 2008 11:02 AM
Posted by: newagent99 | January 10, 2008 9:51 AM
Posted by: usadblake | January 10, 2008 9:30 AM
Posted by: s361883 | January 10, 2008 5:21 AM
Posted by: volcanojon | January 10, 2008 5:07 AM
Posted by: TomJx | January 10, 2008 3:08 AM
Posted by: iopsc | January 9, 2008 10:41 PM
Posted by: gicgltd | January 9, 2008 10:40 PM
Posted by: elatia | January 9, 2008 10:20 PM
Posted by: hhkeller | January 9, 2008 9:50 PM
Posted by: labrat94720 | January 9, 2008 9:48 PM
Posted by: aboyzboi | January 9, 2008 9:26 PM
Posted by: biffle | January 9, 2008 9:16 PM
Posted by: JakeD | January 9, 2008 8:44 PM
The comments to this entry are closed.