JONATHAN CHAIT MAY 25, 2010
-
Read Later
READ LATERAvailable only to subscribers. SUBSCRIBE TODAY
-
Listen
ARTICLE AUDIO
- Font Size
Regarding my previous item, a couple friends have suggested that Rasmussen deserves the benefit of the doubt because it accurately predicted the 2008 election. Well, yes and no. Yes in the sense that a record of success carries real weight. But no in the sense that Rasumussen was not producing strong outlier results in 2008. Here, again, is Nate Silver:
The bottom line is this: the sample included in Rasmussen's polling is increasingly out of balance with that observed by almost all other pollsters. This appears to create a substantial house effect, irrespective of whether Rasmussen subsequently applies a likely voter screen.
It also appears to be a relatively new facet of their polling. If one looks at the partisan identification among all adults in polls conducted in September-November 2008, Rasmussen gave the Democrats at 6.5-point edge, versus an average of 8.7 points for the other pollsters; their house effect was marginal if there was one at all.
In other words, the question at hand is the odd tilt in Rasmussen that has appeared since the start of 2009. This has yet to be tested against electoral reality. There's historical precedent to trust Rasmussen's results, but very little to trust the current incarnation of Rasmussen's results. The problem is compounded by Rasmussen's odd habit of not conducting polls of races close to election time:
Yesterday the nation had several hot races, including the House special election in PA-12, primaries in both parties in Kentucky and Arkansas, and the Democratic Senate primary in Pennsylvania.
And somehow, Rasmussen was nowhere to be found. Yet this past week, Rasmussen found time to poll Colorado, California, and those burning Idaho senate and governor races. He even polled the general election in Arkansas, ignoring the imminent primaries -- the better to show Arkansas Republican primary voters who their strongest candidate was.
You see, the thing about Rasmussen is that he cares only about setting the narrative that Democrats are doomed. And it's hard to build those narratives if you screw up polling actual elections.
I excerpt the above from Kos because of its data points, not its argument. To me, the thesis that Rasmussen is deliberately trying to create pro-GOP narratives, and narrowing its profile of polls that can be tested against reality, is one I'm open to but consider far from settled. The point is that Rasmussen's findings have changed dramatically since 2009 and they haven't been tested. For instance, they stopped polling the iconic Brown-Coakley race a week before the election.
10 comments
I think that Rasmussen confecting pro-GOP narratives is inferentially sound, Jonathan, but this matter does bear further investigation.
- liberal reformer
May 25, 2010 at 5:04pm
I am a poll-tard. I wonder if someone might be kind enough to enlighten. Perhaps expand upon the nearly universal (but for Rasmussen) impartiality presumably in the exclusive service of objectivity. Now we all know for certain that politicians of all stripes will tell you exactly what various polling data means and just what that means to you. Sometimes even political writers will enlist all of that valuable info to better provide a dispositional context for the unwashed. (like me) So tells me.... are there better polls than others. Or is it a matter of it all depends upon which way the wind blows for the blower? If I recall properly Tarot and tea leaves would have been more profitable divination tool in the 1994 turnover than all but one outfit. Who was it sniffed that one out? It must have something to do with the questions and how they are asked.
- jacko
May 26, 2010 at 8:06am
This means Rasmussen has become the FOX News of polling.
- drousma
May 26, 2010 at 8:21am
On Rasmussen's Obama job approval ratings, one weird thing is that the approval numbers are roughly comparable to Gallup and other polliing firms - pretty steadily in the high 40's for some weeks if not months. (Low 40's in Rasmussen's latest release, but that could be an outlier.) But Rasmussen's UNfavorable percentages are MUCH higher than the competition: in the mid 50's these days, compared to Gallup's and others' mid-40's. Not sure if that's an artefact of Rasmussen's pro-GOP weighting or what. I wish Nate would comment on that specific element of Rasmussen's Obama job approval numbers.
- rriley
May 26, 2010 at 8:56am
Chait just doesn’t get it. How stupid do you have to be to qualify for liberal blogger? Isn't there a floor? Rasmussen is polling Likely Voters, almost all the other outfits are polling adults or Registered Voters. It doesn't stretch the imagination to understand that voters who disapprove of Obama -- productive private sector types who will be paying for his stupidity -- are more likely to vote than liberals. This is probably why Rasmussen does very well in predicting elections -- likely voters are better predictors than registered voters or adults.
- mr_rationale
May 26, 2010 at 9:18am
Okay, mr_rationale, how does your Likely Voter theory to explain my point above? Rasmussen's "likely voters" generate roughly comparable job APPROVAL ratings for Obama as the other polls, but Rasmussen finds much more Obama UNfavorable sentiment than the competition. Why do Rasmussen's alleged hard-headed realists approve what Obama is doing the same as Gallup etc. respondents, but disapprove him 10 percentage points more? I say it shows a problem in Rasmussen's methodology.
- rriley
May 26, 2010 at 9:51am
- There is a difference between an organization or a business quoting or bragging about favorable data from a survey and using faulty numbers to guide them. An advertiser who has reason to suspect television or radio ratings and a politician who relies of overly optimistic polls has more to lose than fans or pundits who are deceived. I really don't care if FOX follows the advice of Dick Morris, Rove or picks a poll that coincides with their mission. I mean, Sarah Palin is also their consultant and I hope the opponents of Democrats nationwide retain her services. Neither party can afford to ignore unflattering data that is true. Unlike the business of entertaining a leader must provide results, not just applause.
- michaelg
May 26, 2010 at 10:58am
Jonathan Chait is a lot brighter than you are, mr_rationale, in case you haven't noticed (I know that you haven't). When a pollster's polls are consistently outliers, something is wrong. You figure it out (if you can). There are either methodological flaws or ideological biases at work, or maybe even both, in some cases. And Rasmussen seems to be cherry-picking certain congressional races. If John Zogby's polling didn't pass the smell test the way Rasmussen's polling doesn't (and Zogby is pretty bad, often enough), would you give him a pass? To ask this question is to answer it.
- liberal reformer
May 26, 2010 at 11:52am
What I find interesting is that whenever anyone criticizes Rasmussen, what you hear back is 'he was among the most accurate in the past x Presidential elections and better than the libruls at Gallup, yada yada'. I suppose that makes some sense- Presidential elections unlike the rest are sufficiently visible and high stakes for everyone to hang their hat on. The thing is, unlike in prior campaigns, I was paying pretty close attention to the polling during Obama McCain, and could swear there was some funny business going on with Rasmussen. Much like he has been since, Rasmussen was a total outlier polling that race right up until the very last week. In the last week, I think he had Obama close in the neighborhood of 5 points from very slight edge to near about the final margin he ended up with. Does anyone out there really think this last week surge was an accurate depiction of what was going on with the electorate? I can't remember anything in the news cycle or otherwise that would explain it. Moreover, none of the other pollsters picked up anything of the sort. Herein we have a similar accusation of, effectively, conspiracy to defraud as the one insinuated at Kos and cited by The Fabulous Mr. Chait. In this case though, clearly the race was too visible to simply stop polling it, not to mention important to the credibility of Rasmussen polls more broadly, which is a pollster's ultimate currency. Meanwhile, the phantom last minute surge and consistent outliers throughout the campaign leading up to it are lost down the memory hole, as the now universally accepted meme is 'Rasmussen correctly called the Presidential race'. Perhaps that might be worth digging into.
- I Majorajam
May 26, 2010 at 12:50pm
Ramussen just using a different sample (Likely Voters). Of course their results differ. Nothing sinister -- different results from a different sample. Almost all polling operation DO NOT survey likely voters. Of the following list of polling firms surveying O approval only Rasmussen uses LV (Likely Voters). The remainder use A (Adults) or RV (Registered Voters) as their sample Gallup 5/24 - 5/26 1547 A 46 47 -1 Rasmussen Reports 5/24 - 5/26 1500 LV 45 53 -8 CBS News 5/20 - 5/24 1054 A 47 43 +4 Quinnipiac 5/19 - 5/24 1914 RV 48 43 +5 CNN/Opinion Research 5/21 - 5/23 1023 A 51 46 +5 NBC News 5/20 - 5/23 700 A 48 45 +3 FOX News 5/18 - 5/19 900 RV 45 46 -1 Democracy Corps (D) 5/15 - 5/18 1000 RV 47 47 Tie Associated Press/GfK 5/7 - 5/11 1002 A 49 50 -1 NBC News/Wall St. Jrnl 5/6 - 5/10 1000 A 50 44 +6 Ipsos/McClatchy 5/6 - 5/9 1016 A 52 43 +9 Pew Research 5/6 - 5/9 994 A 47 42 +5 11 out of 12 not using likely voters
- mr_rationale
May 27, 2010 at 3:21pm