Network News

X My Profile
View More Activity
About Chris Cillizza  |  On Twitter: The Fix and The Hyper Fix  |  On Facebook  |  On YouTube  |  RSS Feeds RSS Feed

Parsing the Polls on the Ohio Senate Race

Two polls came out this week in the hotly contested Ohio Senate race. The first, which was conducted by Mason-Dixon Polling & Research, showed Sen. Mike DeWine (R) with a 47 percent to 36 percent lead over Rep. Sherrod Brown (D). The second, conducted by Brown campaign pollster Diane Feldman, showed the Democrat with a 45 percent to 44 percent edge.

The surveys were in the field at nearly the same time (April 24-26 for Mason-Dixon, April 24-27 for Feldman) and had similar sample sizes (625 likely voters for Mason-Dixon, 800 likely voters for Feldman).

So, what gives?

This week's Parsing the Polls is dedicated to trying answer that question.

Let's start with an apples-to-apples comparison of how these surveys were conducted and a bit of background on the two survey firms.

First, the background. Mason-Dixon has been one of the nation's leading independent polling organizations for several decades. According to Mason-Dixon's Web site, it has conducted survey research for more than 250 news organizations and has regularly polled in every state since 1983.

Larry Harris of Mason-Dixon said the firm has worked in Ohio for decades and pointed to its polling in the 2004 presidential race in the state as evidence of its accuracy. In its final poll conducted in this central presidential battleground, Mason-Dixon showed President George W. Bush with a 48 percent to 46 percent edge over John Kerry; after the votes were counted, Bush had achieved a 51 percent to 49 percent victory. (A number of other independent pollsters had Kerry leading Bush by several points in Ohio heading into Election Day.)

Feldman, for her part, is handling the polling not just for Brown's Senate race but also for Rep. Ted Strickland's (D) gubernatorial campaign. When television personality Jerry Springer was weighing a challenge to Sen. George Voinovich (R) in 2003, he hired Feldman (along with pollster Paul Maslin) to test the viability of his candidacy. Feldman was also part of the Democratic National Committee's "Ohio Election Task Force," a group convened following the 2004 election to examine potential voting irregularities in the Buckeye State.

In short, both firms have sterling credentials in the state.

Now, onto the methodology the two firms used to arrive at their results. (Stay with me because this goes deep into the weeds of survey research.)

Feldman used the voter file to compile her her polling sample; Mason-Dixon used random digit dialing.

Each method has its pros and cons. Using the voter file ensures that the people interviewed have actually voted in past elections and gives a pollster a good chance of developing a tight screen of who will vote in traditionally low turnout races. But not every person in the voter file has a listed phone number, nor is every person in the voter file guaranteed to vote again -- some may have died or moved from the state since the file was last updated.

The major advantage for random digit dialing is that it allows a pollster to reach a wider swath of people, including those with unlisted phone numbers. But random digit dialing makes it harder to determine who actually answers the phone in a household -- i.e. is the person of voting age, a registered voter or are they honest about their voting history.

Another major difference between the two surveys is how the firms ensure that their sample is reflective of the geography and past voting patterns in the state. Feldman uses a technique called "cluster sampling" to ensure proper racial and ethnic diversity. In cluster sampling, voters who live in a specific area are broken into groups of 30 or so. The person making the calls must reach one (and only one) person in each cluster; the goal is to contact hard-to-reach voters (especially the poor) to get the most representative sample possible.

Mason-Dixon relies on its past work in the state to determine how to determine the racial and geographic mix of its sample. As Harris puts it, "If I don't have a certain minority population from Cuyahoga County [Cleveland], I am going to be so wrong it's embarrassing. We know what that number should be."

What do these differences in methodology mean? They point to the fact that political polling is both an art and a science. The science end is easily understandable -- a certain amount of people are polled producing results and a margin of error in which those results fall. But determining who should be surveyed and how many of those people make the cut is the trick.

In 2004 -- when Bush and Kerry waged an all-out campaign for Ohio -- 5.7 million people turned out to vote. Two years earlier, in the first midterm election of Bush's presidency, just 3.2 million Ohioans voted. Most observers agree that turnout in 2006 will be higher than 2002 but lower than 2004. But how much higher? Or how much lower? The art of polling is figuring out the answer to that question.

The results on Election Day will let us grade which pollsters had it right. Until then, we wait and speculate. Speaking of which, speculate away about the Ohio race and the two polls we are comparing in the comments section below.

One other note: Some political pros are dismissive of surveys conducted by partisan pollsters. After all, wouldn't the pollster charged with electing Brown to the Senate have a vested interest in releasing numbers that show him ahead? Yes and no.

Partisan pollsters are working to elect candidates, but a major part of that equation is producing accurate numbers to help guide campaigns to victory. It doesn't serve their interests to cook up numbers that are entirely misleading to their clients and could hamper their ability to attract top-tier races in future cycles.

We'll explore this subject more deeply in a future post.

By Chris Cillizza  |  May 3, 2006; 10:50 AM ET
Categories:  Parsing the Polls , Senate  
Save & Share:  Send E-mail   Facebook   Twitter   Digg   Yahoo Buzz   Del.icio.us   StumbleUpon   Technorati   Google Buzz   Previous: Ohio Primary Results: Good News For Dems?
Next: Chatting With The Fix: Damage Control Time for Sen. Allen?

Comments

Uh...Contrarian Genius,

the polls were taken during the week, not the weekend.

Posted by: Correct Genius | May 5, 2006 10:51 AM | Report abuse

As a professional political operative, I'm disappointed that The Fix overlooks the obvious flaw; any poll conducted over a weekend of "likely" voters is flawed, by a mean of about 7-9 percent (trends higher in the south). Basically, a "likely" voter is an adult with above average mobility, a commitment to, for example, taking an hour off work to vote; in the most plainspoken terms, likely voters are extraordinarily unlikely to be home over the weekend to answer a pollster's questions, regardless of methodology of selection, questioning or dialing.

Posted by: The Contrarian Genius | May 4, 2006 5:04 PM | Report abuse

This further proves my feeling that Rahm Emmanuel, Chuck Schumer and the rest of the pipe-hitting Big D Mafia are spineless Toads for threatening Hackett with political cement shoes.

Let's not forget that prior to the Hackett Shakedown, the UAW endorsed him over Brown.. a huge slap for the Union-friendly Brown. Somebody didn't like it, threw a hissy fit, and now the Dems can look forward to six more years of DeWine.

Folks, this is Ohio: DeWine will flog Gay Marraige and a host of other red meat issues and will edge out Brown in the end.

You can't buy, produce, triangulate, poll, package or paint on what people like Hackett have. Brown's going to find this out the hard way.

Posted by: Vicenzo | May 4, 2006 10:53 AM | Report abuse

An interesting exercise is to review election results and polls done close to an election.

Mason Dixon/Plain Dealer
April 21-24
Blackwell 50%
Petro 29%
margin 21%- almost double actual results

Flannery 11%
Strickland 64%
margin 53%- close

Final Primary Results
Blackwell 56%
Petro 44%
margin 12%

Flannery 20%
Strickland 80%
margin 60%

And the winner in pre-election predictions-
Columbus Dispatch
April 30
Blackwell 56%
Petro 44%
margin 12%- Dead on

Flannery 14%
Strickland 86%
margin 62%- off by 2 pts

Posted by: RMill | May 3, 2006 3:36 PM | Report abuse

As important or more so than the snapshot poll numbers are trends.

Survey USA
US Senate Approval Ratings
Date-------- Feb Mar Apr
DeWine (R)* 43% 46% 48%

DeWines' approval numbers have been on a slight but steady upswing since Feb.

Rasmussen
Date-------- Feb 16 Mar 28 Apr 19
DeWine(R)* 46% 45% 43%
Brown (D) 37% 42% 41%

This trend is not reflected in the polling numbers, which show a slight but steady decline in Rasmussen polls.

Other polls

Zogby/WSJ
March

DeWine (R)* 37%
Brown (D) 45.9%

Mason Dixon/Plain Dealer
April 21-24
DeWine (R)* 47%
Brown (D) 36%

These show almost opposite results one month apart.

Final results from Tuesday's primary showed both candidates easily defeating their unknown opposition, with Brown getting 75% and DeWine 71% respectively.

Zogby polls are done by web, which some have criticized the methods. Methodology can play a large role in results. Columbus Dispatch does mail-in surveys. And then the traditional phone polls, which miss many people who have gone wireless.

Also make sure to check error rates and who and how many people are sampled.

Zogby +/- 3.5% error; ?
Rasmussen +/- 4.5%; 500 likely voters
Mason Dixon +/- 4%; 625 frequent voters

How are likely/frequent voters identified? Is sample size sufficient representation of total population.

What all this information tells me is that the race is likely to be very close, within 5 points and that while general feelings towards the incumbant Senator are below 50%, they are on a decided upswing but that has not translated broadly to specific match up questions.

It also tells me that this race will not be decided by likely voters, but those on the fringe that are drawn out for other issues and candidates, who will vote in this race.

Will there be a high level of conservative evangelicals who will come out for the Republican gubernatorial candidate and who not be disappointed in the incumbants track record enough to vote for him?

Will a backlash of anti-scandal, anti-GOP sentiment cause independents to swing to the Democratic candidate, especially with the moderate Democratic Governor candidate?

Both candidates will be well funded, with the incumbant having an edge but probabaly not enough to make much difference.

Posted by: RMill | May 3, 2006 3:28 PM | Report abuse

I agree with Will in Seattle

That digit dialing does not include cell phones. And this tends to be younger voter who lately have been titling towards democrats. So I think some of the polls are skewed towards republicans. If the republicans are scared now from what they see in the polls wait till election night in 6 months.

"oh and digit dialing still doesn't pick up cell phones. people with no land lines are furious right now, "

Posted by: Polling Fraud | May 3, 2006 3:04 PM | Report abuse

I think CJ sums it up pretty well. Anytime I see one poll that is way off the others in the field I sort of discount it. I think they do tell us though that this race is gonna be close. Brown is going to paint Dewine as a Bush lackey and hope for W's poll number nose dive to take him down too. Not to bad a strategy if things keep up like they are now.

Posted by: Andy R | May 3, 2006 2:49 PM | Report abuse

Chris,

Great post.

Thanks explaining why two polling firms come up with almost contradictory results. But, the problem is when the public sees these type of polls contradict each other they start thinking all the polls are invalid. (Maybe that is what the political stategist is trying to do?) The reputation of polling has still not recovered from the exit poll errors that occured in the 2000 Presidential Bush-Gore race.

Posted by: Polling Fraud | May 3, 2006 2:42 PM | Report abuse

The most recent Rasmussen poll (4/19) has it 43-41 for DeWine with a 4.5 MOE. It also has 11% undecided. Their 3 month average shows it 45-40. Most of the polling I have seen has had this race consistently in the MOE, so I find M-D's poll hard to believe. I think the race will be close and decided at the end. Moderates may decide to split their vote and vote Strickland and DeWine. Strickland is definitely the moderate in the governors race.

Posted by: CJ | May 3, 2006 1:51 PM | Report abuse

oh and digit dialing still doesn't pick up cell phones. people with no land lines are furious right now, so don't be surprised - look for polls with a door-to-door precinct level comparison check if you want the real picture.

Posted by: Will in Seattle | May 3, 2006 1:46 PM | Report abuse

Well, Chris, you blew it on the main blog today when you pretended the GOP has a chance to retain either the Senate or the House.

Go with your gut, man. There's a tidal wave coming, and the survivors will be few and far between.

Also, watch MT and WA for some major slaughter.

Posted by: Will in Seattle | May 3, 2006 1:45 PM | Report abuse

Nice clear explanation. Problems with telephone polling date right back to the Literary Digest poll of 1936, when Alf Landon carried the phone-owners but not the voters.

But with so many houses using more than one line, how does random calling avoid duplication? And with cell phones so common, how do they avoid calling up a lot of sullen teens and/or harassed freeway drivers?

Posted by: Kakuzan | May 3, 2006 1:19 PM | Report abuse

The polling shows a tight race getting tighter. Diane Feldman was Paul Wellstone's only pollster. I worked with her on the first two campaigns ('90 & '96), and her early numbers provided us with a roadmap for turning out key base voters and persuading the undecideds. The M-D numbers in both races consistently slanted Republican and predicted a significant Wellstone loss in the 1990 race. In both races Feldman's numbers were right and M-D was way off. The methodology only goes so far, it's how campaigns turn out base and target persuable voters. Feldman's Ohio clients would be wise to closely follow her numbers. One of the last things Wellstone said to me in 2002 was that he was not worried about the close polls, because Diane's numbers were always better and were proven on two election days.

Posted by: Scott | May 3, 2006 1:09 PM | Report abuse

I think the underhanded way the Democrats in Ohio kicked Paul Hackett out of the chance to run for the Senate is a key reason people gets disgusted with politics.

At a time the Democrats are trying to use Iraq and Afghanistan vets to clobber Republicans, here is a strong leader who might have defeated either Jeanne Schmitt in a Congress rematch or won against DeWine tossed out completely. Now I wonder if Hackett has a good taste of dirty politics with the Democrats?

Posted by: Sally | May 3, 2006 12:36 PM | Report abuse


As a political scientist that has done some polling in my time my initial reaction is that the Feldman approach should "capture" a higher proportion of knowledgable voters and hence have a lower percentage of "undecideds" etc. With random digit dialing the "undecideds" increase and name recognition for a sitting senator gives DeWine his apparent advantage(note both polls give DeWine about the same share). My guess is that the "independent" or no party share of the Mason Dixon poll is higher and these are more undecided. It would be interesting to compare the percentages of support for the two candidates by party identifiers for both polls. I suspect that they would agree.

Posted by: John Bing | May 3, 2006 11:30 AM | Report abuse

As a friend and supporter of Sherrod Brown, I found this article very helpful.

I heard about the M-D poll some time ago; the Feldman numbers were news.

Hazarding a guess, I'd be inclined to accept the M-D numbers.

Having just nominated an extremely conservative candidate for Governor, I would guess Sherrod's numbers go up among those who paint DeWine with the same brush.

But those independents who pick and choose among the candidates may give DeWine credit for a degree of moderation and independence that insulates him from the corruption and incompetence threatening to sweep other statewide Republicans from office.

Sherrod's going to have to work hard to persuade them otherwise, and hope that Ted Strickland, our gubernatorial candidate, keeps his opponent on the defensive as radically out of touch with mainstream Ohioans.

Posted by: Kurt Landefeld | May 3, 2006 11:27 AM | Report abuse

Chris,

great post - I use similar type polls in my government class to demonstrate the pros and cons of polling - I will use your post as a great explanation of how polls work and why somtimes we should accept them with a grain of salt, especially this early in the election cycle

Thanks for helping to educate my students.

Bobby Wightman-Cervantes
www.balancingtheissues.com

Posted by: Bobby Wightman-Cervantes | May 3, 2006 11:20 AM | Report abuse

Copy of Email that I sent Rasmussen asking about their polling and margin of error. I noticed it when their numbers have been from other polling fims everyday for the last 2 months. After I asked about it, they changed their website so that you can no longer see a running daily poll for every day that month. You can now only see todays poll not yesterdays. It is as if they changed the website to hide their statistical errors.

To Rasmussen Reports Representative:

President Bush Job Approval
Poll Date Approve Disapprove Spread
RCP Average 04/21 - 04/30 35.2% 59.3% -24.1%
USA Today/Gallup 04/28 - 04/30 34% 63% -29%
CBS News 04/28 - 04/30 33% 58% -25%
Rasmussen 04/28 - 04/30 40% 59% -19%
Cook/RT Strategies 04/27 - 04/30 36% 59% -23%
NBC/WSJ 04/21 - 04/24 36% 57% -21%
CNN 04/21 - 04/23 32% 60% -28%
FOX News 04/18 - 04/19 33% 57% -24%
Pew Research 04/07 - 04/16 35% 55% -20%
Gallup 04/10 - 04/13 36% 59% -23%
LA Times/Bloomberg 04/08 - 04/11 39% 57% -18%
CNN/USA Today/Gallup 04/07 - 04/09 37% 60% -23%


Why are your numbers for "Bush Job Approval" so far off from other leading polling firms ? You are +10 points to Fox News ?

How can your numbers be +6 to +10 off ? Rassussen is nowhere close to every leading polling firm in the entire nation. How can your polling be that far off ? What is your margin of error ?

Please respond with an answer.

Your polling seems unreliable.

Posted by: Polling Fraud | May 3, 2006 11:05 AM | Report abuse

The comments to this entry are closed.

 
 
RSS Feed
Subscribe to The Post

© 2010 The Washington Post Company