Network News

X My Profile
View More Activity

A note on polling


(Via Bruce Bartlett.)

By Ezra Klein  |  January 25, 2010; 10:00 AM ET
Save & Share:  Send E-mail   Facebook   Twitter   Digg   Yahoo Buzz   StumbleUpon   Technorati   Google Buzz   Previous: If the Federal Reserve chairman falls in the forest and everyone is around to hear...
Next: How presidents polarize


2, 3, and 4 are pretty valid - but suggestion 1 shows the cartoon author to be lacking in understanding.

A 52-48 poll with a +/-5 MoE is not "a toss-up", just as a 52-47-1 poll with a +/-5 MoE is not "a lock" because it meets the MoE.

MoE is the spread that pollster decides (based in large part, but not exclusively, on sample size) is necessary to reproduce the "winner" of the poll some large percentage of the time if it were to be repeated. Typically, but not always, that's 95% of the time. The 95% number is arbitrary.

So the 52-47-1 race is deemed 95% likely to go to "Ms. 52".. while the 52-48 race doesn't meet that threshold - but isn't a 50/50 (toss up) proposition for Ms 52 either! The poll still says she is more likely to win than her opponent - i.e. more than 50% likely, but less than 95%. And having an advantage is horse race news.

Why is 95% confidence inherently newsworthy, but 94%, 90% or even 75% are not as long as the confidence level is noted in some way? Noting the MoE is one non specific way of doing so.

Here's another way of putting it: do you want to repeatedly bet at even money on "mr 48" in the 52-48 race? Of course not - he loses most of the time. Not 95% of the time, but most of it. And that's interesting horse race insight. You can certainly say "mr 48" is more likely than not to be trailing.

Now horse race coverage at the expense of substantive coverage sucks, but that's a different story.

Posted by: patrickinmaine | January 25, 2010 10:22 AM | Report abuse

#1 is wrong more generally. If the expectation is that one candidate is supposed to win handily, then the fact that the two are within the margin of error is indeed newsworthy. Drawing this cartoon right after the Mass. special election just shows that understanding statistics and understanding politics are two different things.

Posted by: tomtildrum | January 25, 2010 11:23 AM | Report abuse

I think the point of #1 is a change of less than the margin is not a story. And a lead of less than the margin is not a story in terms of "X is leading Y." That it's closer or wider than expected is a story, unless it's closer or wider by less than the margin.

Posted by: OpieCurious | January 25, 2010 11:38 AM | Report abuse

Yeah guys, he's not saying about a poll which shows candidates within the margin of error. His point is that if you have two polls on the same issue and their results are in the margin of error, it's not a news story (ie., "Candidate O" polls at 51% approval in May and 52% approval in June when the polls have a margin of error of +/- 2%)

Posted by: MosBen | January 25, 2010 12:28 PM | Report abuse

I think it is fair to say that item #1 in the cartoon is flawed. Margin of error is typically a 95% confidence interval. If you are willing to take more than a 1 in 20 chance of being wrong, then the margin of error gets smaller. If the margin of error is 5% and candidate A has a 4% lead, then there can still be a high probability that Candidate A is going to win. It's just not 95%. So if a poll switches from Candidate A is leading by 4% to losing by 4%, then a significant shift has occurred. Just because the probability of A winning was less than 95% before and it is has not sunk below 5% now does not mean that nothing of interest has happened.

Posted by: adonsig | January 25, 2010 2:31 PM | Report abuse

#2 is so, so right that I have a hard time expressing how right it is.

Posted by: jeffwacker | January 25, 2010 5:49 PM | Report abuse

The comments to this entry are closed.

RSS Feed
Subscribe to The Post

© 2010 The Washington Post Company