Examining the Gallup generic ballot question
One week ago, Gallup released a weekly tracking poll that showed Republicans with a 10 point edge on the so-called generic ballot question -- "If the election for Congress were being held today, which parties' candidate would you vote for in your congressional district?" -- a finding that set off a rash of media coverage that the Democrats' hold on their House majority was rapidly slipping away.
(We dedicated the lead item of our "Morning Fix" column to the Gallup generic findings -- and what they meant -- while Post pollster Jon Cohen offered his take in a story for the paper.)
On Tuesday, Gallup released its latest numbers, which showed Democrats and Republicans tied at 46 percent on the generic ballot -- a stunning reversal that had Democratic operatives crowing that the media had jumped the gun on writing their party's political obituary.
"With eight weeks until Election Day, Republicans and Beltway pundits may want to hold off on calling the race for the House before voters cast their ballots," wrote Democratic Congressional Campaign Committee deputy executive director Jennifer Crider in an e-mail to reporters touting the latest Gallup findings.
What explains the radical shift? Gallup polling director Frank Newport offered this explanation:
"Last week marked the return of President Barack Obama from his 10-day vacation, and included his national address to announce the official end of combat operations in Iraq. The president's three-day job approval rating rose to 47% for Aug. 29-31 -- a level it had reached only once since mid-July. Last week also brought media commentary in the aftermath of conservative talk-show host Glenn Beck's massive rally in Washington, D.C. It is not clear if these or other factors affected Americans' voting preferences as measured by the generic ballot."
The Fix spoke to a number of pollsters -- both Democratic and Republican -- seeking their take on the fluctuation in Gallup numbers. Their responses boiled down to a few general thoughts, which are listed from simplest to most complex below.
1. The last Gallup weekly track was an outlier: Every poll has a margin for error -- and with it the chance that the numbers represent an anomaly. Taking last week's track out of the mix, Gallup had shown single-digit Republican edges on the generic ballot for several weeks and, in fact, the tracking survey the week before the 10-point Republican generic lead showed it 46 percent Republican, 43 percent Democrat. Pollsters on both sides insisted that the generic ballot simply doesn't fluctuate that quickly barring some major national event or scads of television advertising -- neither of which happened in the week between +3 for Republicans and +10. For his part, Newport cites the numbers leading up to the +10 finding as the main impetus for his conclusion that the GOP was poised to make "significant gains" in the House this fall; "It was this consistency in the Republican lead that led to our emphasis on its potential implications in my write-up last week, not just one week's polling," wrote Newport in an email to the Fix this morning.
2. Registered voters versus likely voters: The Gallup weekly track is based on registered voters and the organization doesn't release the tighter screen of only those likely to vote. Compare Gallup's tie among registered voters on the generic ballot to the registered voters findings of the latest Washington Post/ABC News and NBC/Wall Street Journal polls and there is considerable consistency. The Post/ABC survey showed Republicans with a 47 percent to 45 percent generic ballot lead among registered voters while NBC/WSJ showed the two parties tied at 43 percent. Among likely voters, however, Republicans had a 13-point edge in the Post/ABC poll and a nine-point margin in the NBC/WSJ poll -- suggesting that Republicans enjoy a considerable enthusiasm edge in the fall. Even the latest Gallup numbers affirm that enthusiasm gap; 50 percent of self identified Republicans say they are very enthusiastic about voting this fall while just 25 percent of self described Democrats say the same. While the difference between registered voters and likely voters doesn't explain the variance in Gallup data -- both weekly tracks were of registered voters -- it does suggest that the latest Gallup generic numbers may not be far off.
3. Weighting party identification: Gallup does not weight their results for party identification. What this means -- in as close to layman's terms as possible -- is that Gallup uses random digit dialing, which is exactly what it sounds like, to build their national sample. Then Gallup asks respondents whether they consider themselves Republicans, Democrats or Independents. Some other pollsters -- particularly those who do political work for partisan clients -- typically take that raw party ID data and "weight" it, meaning that they adjust it to reflect what they believe to be the actual partisan breakdown in the country. Gallup -- as well as the Post/ABC and NBC/Wall Street Journal -- chooses not to weight based on party identification; "We expect changes in party ID when there are broad shifts in sentiment in favor of Democrats or Republicans since shifts in political sentiment can affect [party identification] as well as affecting the ballot," wrote Newport in an email to the Fix. At issue is that when conducting a weekly tracking poll without weighting for party ID, there is the real possibility that the sample will either skew far more Republican (or Democratic) than the reams of available data suggests it should.
Regardless of the reason for the Gallup numbers, the back and forth serves as a reminder -- to the Fix and other political watchers -- that no one poll contains absolute truth.