Network News

X My Profile
View More Activity

The Shoddy Statistics of Super Freakonomics

super_freakonomics.JPGSuper Freakonomics is getting a lot of flak for its flip contrarianism on climate change, most of which seems based on incorrectly believing solar panels are black (they're blue, and this has surprisingly large energy implications) and misquoting important climate scientists.

But before people begin believing that the problem with Super Freakonomics is that it annoys environmentalists, let's be clear: The problem with Super Freakonomics is it prefers an interesting story to an accurate one. This is evident from the very first story on the very first page of the book.

Under the heading "putting the freak in economics," the book lays out its premise: Decisions that appear easy are actually hard. Take, for example, a night of drinking at a friend's house. At the end of the night, you decide against driving home. This decision, the book says, seems "really, really easy." As you might have guessed, we're about to learn that it's not so easy. At least if you mangle your statistics.

The next few pages purport to prove that drunk walking is eight times more dangerous than drunk driving. Here's how they do it: Surveys show that one out of every 140 miles driven is driven drunk. "There are some 237 million Americans sixteen and older; all told, that's 43 billion miles walked each year by people of driving age. If we assume that 1 out of every 140 of those miles are walked drunk -- the same proportion of miles that are driven drunk -- then 307 million miles are walked drunk each year."

"If we assume."

But why should we assume that? As the initial example demonstrates, a lot of people walk drunk when they would otherwise drive drunk. That substitution alone suggests that a higher proportion of walking miles are drunk miles. Other people walk, or take transit, when they know they'll be drinking later. That's why they're walking and not driving. That skews the numbers and makes it impossible to simply "assume" parity.

Then there's the implicit assumption that the two types of drunk miles are the same. But there are a number of reasons to question this presumption. For one, the miles walked drunk are probably disproportionately urban, while the miles driven drunk are probably disproportionately rural and suburban. But driving an urban mile drunk is probably a lot more dangerous than driving a rural mile drunk, just as walking an urban mile drunk is probably much more dangerous than walking a rural mile drunk.

For another, the levels of drunkenness probably differ. A lot of those miles driven drunk are probably miles driven by someone who's had a couple of glasses of wine and still feels competent to drive, even though their blood alcohol level is a shade above 0.8. That is to say, a lot of those miles are driven by someone who's not very drunk. A lot of those miles walked drunk are probably walked by someone who is so wrecked that his friends wouldn't let him drive, or he never would imagine trying. It is, of course, safer to be less drunk than more drunk when you're around vehicles, whether you're walking near them or driving by them.

You can go on and on in this vein. It's terrifically shoddy statistical work. You'd get dinged for this in a college class. But it's in a book written by a celebrated economist and a leading journalist. Moreover, the topic isn't whether people prefer chocolate or vanilla, but whether people should drive drunk. It is shoddy statistical work, in other words, that allows people to conclude that respected authorities believe it is safer for them to drive home drunk than walk home drunk. It's shoddy statistical work that could literally kill somebody. That makes it more than bad statistics. It makes it irresponsible.

But hey, it makes for a fun and unexpected opener.

By Ezra Klein  |  October 16, 2009; 2:25 PM ET
Categories:  Books  
Save & Share:  Send E-mail   Facebook   Twitter   Digg   Yahoo Buzz   Del.icio.us   StumbleUpon   Technorati   Google Buzz   Previous: Joe Lieberman (Probably) Won't Support a Filibuster Against Health-Care Reform
Next: How to Improve the Finance Bill

Comments

how about somebody quantitative and non-shoddy? http://www.washingtonpost.com/wp-dyn/content/article/2009/10/15/AR2009101503894_pf.html (Richard Whitcomb obituary)

Posted by: bdballard | October 16, 2009 2:42 PM | Report abuse

I haven't read it, but one example followed by "You can go on and on in this vein" leaves me wanting a critique that's a bit more quantitative, especially if the argument of general statistical shoddiness is intended add substance to the "black-vs.-blue" issue.

Posted by: westling | October 16, 2009 3:02 PM | Report abuse

I'm shocked. A transport economist who does shoddy numbers on high speed rail because taking the anti position seems like a sexier angle does shoddy numbers on drunk driving because it makes for a sexier angle. I'm SHOCKED that there is gambling going on in my casino.

Posted by: BruceMcF | October 16, 2009 3:13 PM | Report abuse

Gah. Economists can't even explain *the economy*, and here they are playing with everything else?

Please, stick to your day job before trying to do everyone else's.

Posted by: Sophomore | October 16, 2009 3:18 PM | Report abuse

Well, I once was walking home completely drunk and fell into a ditch. Surely that has to count.

Posted by: leoklein | October 16, 2009 3:20 PM | Report abuse

Isn't it a false choice no matter how you slice it? Even if walking drunk _were_ somehow more dangerous than driving drunk, there are other choices out there, like, say, calling a cab.

I'll tell you what is dangerous, though: hang-gliding drunk. If you have to choose between driving drunk and hang-gliding drunk, please, for your own safety, drive.

Posted by: horsecow | October 16, 2009 3:26 PM | Report abuse

Driving home drunk in your H2 might be safer...for you. I don't recall the last time I heard someobody was killed by a drunk walker. And that's what the laws are for anyway...nobody particularly cares what a drunk does to themselves...it's what they do to other people that gets us riled up enough to pass laws, etc.

Posted by: luko | October 16, 2009 3:33 PM | Report abuse

Thanks Ezra, and well said.

Posted by: CorkExaminer | October 16, 2009 3:44 PM | Report abuse

Your move, Slate.

Posted by: Jenn2 | October 16, 2009 3:59 PM | Report abuse

"Driving home drunk in your H2 might be safer...for you. I don't recall the last time I heard someobody was killed by a drunk walker."

Bingo -- the missing element here is the question, dangerous to whom? While it's certainly possible that a drunk pedestrian could cause harm to someone else -- stumbling into the street, causing a driver to swerve suddenly and hit a bystander, a tree, or another car -- I'd bet that's a lot less of a problem than drunk drivers killing other drivers (and their own passengers, though you can argue culpability on the passengers' part as well, or at least find them less sympathetic as victims).

Posted by: Janine1 | October 16, 2009 4:01 PM | Report abuse

As good scientists they should run an experiment. They should both go out and get sloshed. One then walks home; the other drives. I bet they fight over who walks.

Posted by: bronxilla | October 16, 2009 4:21 PM | Report abuse

Awesome post.

Posted by: Drew_Miller_Hates_IDs_That_Dont_Allow_Spaces | October 16, 2009 6:50 PM | Report abuse

Ezra:
The black vs. blue issue is completely bogus. In either case solar power is a huge win because it provides power that doesn't come with CO2 emission. Since CO2, once in the atmosphere, contributes to global warming for about a century, the aggregate heat not trapped difference is colossal. Joe Romm's blog, climateprogress.org, has numerical details.

Which makes the book even more wretched than you thought. Is Levitt's agenda simply to shock, or is he actively trying to shred his credibility forever?

Posted by: student16 | October 16, 2009 8:12 PM | Report abuse

The freakonomics folks have always been kind of creepy. Their goal has always been the cool counter-intuitive conclusion, no matter how they get there. Most of what they've done is kind of frivolous, dressed up as "serious economic theory". This gives them credibility, which makes them dangerous when they attempt to address actual issues.

Posted by: stevedwight | October 16, 2009 10:46 PM | Report abuse

In the first Freakonomics, I believe they argued that driving was as safe as flying by
1) comparing them per hour of travel, rather than per mile, and
2) looking only at fatalities.

Not surprisingly, these two factors made cars look safer than measuring per mile and including injuries would have.

Now that they want to compare (drunk) driving to (drunk) walking, they decide that per mile is a better measure than per hour. I suspect they assume whatever makes driving seem better.

Posted by: Erik6 | October 17, 2009 12:18 AM | Report abuse

I believe you are quite wrong about the relative safety of urban and rural driving. I believe that, in cities (not beltways not suburbs cities) people are generally driving to slowly to kill themselves or other drivers. My rock solid evidence for this claim is this google search http://tinyurl.com/yhcbzdx which gives a dead link with an abstract including

"... SAFEST STATE FOR DRIVING Rhode Island MOST DANGEROUS Wyoming ..."

And if you can't trust a dead link at google, what can you trust ?

Posted by: rjw88 | October 17, 2009 5:49 AM | Report abuse

Ceteris paribus, economists like to assume a lot of things.

Posted by: steveh46 | October 17, 2009 11:34 AM | Report abuse

"That substitution alone suggests that a higher proportion of walking miles are drunk miles."

People walk long distances for exercise. My wife walked 3.5 miles last night. This probably counterbalances your "drunk walking buddies" assumption, making LD's initial assumption a little more realistic.

Posted by: BeatKing11 | October 17, 2009 1:26 PM | Report abuse

Perhaps Klein is correct that the statistics in SuperFreakonomics is poor. His example here, however, fails to establish that.

Why? Because he didn't actually give the entire details of what the book is claiming. Further, if it is a bad assumption that the same proportion of miles driven drunk are walked drunk, then why does Klein follow that accusation with a serious of even more tenuous assumptions:
- "miles walked drunk are probably disproportionately urban"
- "miles driven drunk are probably disproportionately rural and suburban"
- driving/walking an urban mile drunk is probably a lot more dangerous than driving/walking a rural mile drunk
- "a lot of those miles are driven by someone who's not very drunk"
- "a lot of those miles walked drunk are probably walked by someone who is so wrecked that his friends wouldn't let him drive, or he never would imagine trying"

Probably... probably... probably? That's your response to one possibly invalid assumption?

In fact, the idea that walking drunks are more likely to be "wrecked" to be extremely questionable, especially in urban settings. Is it so inconceivable that people walk/take transit not because they are or plan on getting trashed but because it's easier than trying to find parking in a dense metro area? For every staggering drunk walking down the street, there are many, many moderately drunk people walking (consider how many buddies are holding up their staggering drunk friends... that alone balances against the numbers). I make no claim as to which group are more drunk, because I (like Klein, apparently) do not have the statistics to back such claims. But I find Klein's own assumptions to be even more questionable than those he reports from Levitt and Dubner.

Posted by: agaudio | October 19, 2009 11:40 AM | Report abuse

The comments to this entry are closed.

 
 
RSS Feed
Subscribe to The Post

© 2010 The Washington Post Company