Network News

X My Profile
View More Activity
Posted at 1:20 PM ET, 02/22/2011

Your doctor doesn't know what works

By Ezra Klein

Let's say your doctor does everything right. She follows the best medical guidelines. She only prescribes treatments that have been vetted by the Infectious Diseases Society of America and are widely believed to help. She never tries to make a quick buck, or give a patient false hope, by ordering a test or procedure that's not likely to help. In that world, how many of the treatments that she's prescribing will have high-quality clinical studies supporting their use?

One in seven. Yikes. Michael Mandel looks on the bright side:

The conventional wisdom was that breakthroughs in understanding the human genome would provide better treatments within the existing structure of medicine – like putting better windows or a new floor or more comfortable furniture into an existing house. The implicit belief is that we could build on existing medical knowledge, add in the new knowledge of the genome, and quickly get to new products.

But what if the main lesson of the past ten years is that the house itself has rotten foundations and needs to be rebuilt completely? What if the biosciences sector can’t afford to take anything for granted from existing medicine because too much of it is not evidence-based?

This is both bad news and good news. Starting from scratch and rebuilding foundations is obviously a daunting and expensive task, and helps explain why the biosciences sector has struggled to produce breakthrough treatments. On the other hand, when you are rebuilding foundations, a lot of progress can be made without anything visible from the outside.

By Ezra Klein  | February 22, 2011; 1:20 PM ET
Save & Share:  Send E-mail   Facebook   Twitter   Digg   Yahoo Buzz   StumbleUpon   Technorati   Google Buzz   Previous: Why do unions care about Medicaid?
Next: Lunch Break


I wonder if the situation described here should really be so alarming. Even if a treatment recommendation is not supported directly by a clinical study, there may exist other kinds of evidence for it. The underlying biological mechanisms behind a condition and its treatment may be well understood, for instance. Beyond that, surely some weight should be given to the intuitions and assumptions of a group of experts.

A higer-level comment: I feel like over the past maybe decade, an idea has taken hold in many areas of life and fields of study: most of what we think we know, we don't. Part of me thinks this is exactly right; human beings have a great capacity for self delusion. But part of me wonders if we have become too quick to doubt anything not supported by a perfectly designed and executed study.

Posted by: erniep | February 22, 2011 2:18 PM | Report abuse

Doctors are human, they make mistakes just like you and me. The moment we all understand that we'll all be better off.

Posted by: visionbrkr | February 22, 2011 2:25 PM | Report abuse

What I like is hearing story after story of patients who come in to my wife's office saying that they need [fill in the blank] test(s), because they read about it on the internet.

Of course said high-end test(s) very expensive, and are usually prescribed later in the diagnostic process if needed...not as Step 1.

Just wait until these impatient and demanding Americans who expect to get an appointment the same day they call, and expect doctors to order every test imaginable upon demand, get a load of Obamacare and government regulatory actions which will lead to wait times to which we are not accustomed, and will restrict or even eliminate the more effective/expensive tests as a means of achieving cost control.

Posted by: dbw1 | February 22, 2011 2:46 PM | Report abuse

This shouldn't be surprising, and it has nothing to do with doctors "being human" or "making mistakes".

Clinical medicine, as practiced on a day to day basis, isn't a science. It has scientific underpinnings, for sure, but the vast field of medical research, divided into hyper-specialized domains, both experimental and clinical, doesn't feed directly or smoothly into working practice.

That's why you have metastudies and an implicit hierarchy of journals, but even then, you're dealing with a flow of studies and data that would be hard to process on a full-time basis, not to mention commercial and governmental sources chipping in with their own takes.

So the pursuit of best practices is always about tacking the low-hanging fruit, such as generational predispositions towards certain prescribing practices or treatment recommendations that were commonplace in their med-school or residency years, or specialists' tendencies to evaluate patients through the narrow focus of their own specialization.

Or, to put it another way, you treat the medical profession like a patient, not a machine.

Posted by: pseudonymousinnc | February 22, 2011 3:04 PM | Report abuse

Medicine is mostly case studies, which are essentially built up out of anecdote. The field of medicine and available treatments is vast, and it seems unlikely that you could do "high-quality" studies for all interventions. Big controlled trials are very expensive and difficult, and for treatments that are already considered effective you have difficult ethical issues in coming up with a control--so it has to be done observationally if that's even possible. More data helps, but you cannot wave a magic wand and have a pile of irrefutable evidence appear.

Posted by: me2i81 | February 22, 2011 3:06 PM | Report abuse

dbw1: your argument is incoherent.

Do you think that those "impatient and demanding" people should be given exactly what they ask for? You imply otherwise, and you can't have it both ways.

Posted by: pseudonymousinnc | February 22, 2011 3:12 PM | Report abuse


When I started this article, I thought you were going to make a point about the availability of comparative studies in medical treatments. And perhaps how the mandated use of computers in diagnosis could be used to help the doctor review all possible causes for a certain set of symptoms.

You know, what the Right Wing calls "Taking Away a Doctor's Choice", or "Putting the Government Between You and Your Doctor".

After all, you wrote a lot about ineffective treatment being a significant cause of medical costs, and how the VA and Mayo Clinic used these techniques to help drive costs down.

But you did not do any of these things.

Did I miss something?

Posted by: grat_is | February 22, 2011 5:36 PM | Report abuse

Hmm this fact suggests that funding and research into which medical procedure or medicine works the best would not only result in better health outcomes for patients but in cheaper health care. We shouldn't definitely fund more research into this; oh wait never-mind republican are against can't do it now that they control the house.

Posted by: blueman3 | February 22, 2011 7:30 PM | Report abuse

Ezra, this is misleading.

The article says:

They found only one in seven treatment recommendations from the Infectious Diseases Society of America (IDSA) -- a society representing healthcare providers and researchers across the country -- were based on high-quality data from clinical trials.

By contrast, more than half the recommendations relied solely on expert opinion or anecdotal evidence.

End Quote

So, first, that one in seven could be for a very common one-seventh of conditions that account for most of the doctors visits. The other six-sevenths could be for much rarer conditions.

Second the other six in seven could still have strong evidence even if they don't have what's called, "high-quality data from clinical trials". This may mean very expensive double-blind trials over many years.

Quoting the article: "These data reinforce that absolute certainty in science or medicine is an illusion," an editorial in the journal notes. "Rather, evaluating evidence is about assessing probability."

So, often you can really know a lot about the odds from the evidence you have, even if there haven't been long term double blind trials yet. Anecdotal evidence can be powerful -- if it's interpreted intelligently, as can epidemiological evidence. Keep in mind, if a buddy eats a berry in the woods and immediately keels over, that's just one anecdote -- but when combined with other knowledge about human similarity, biology, etc., it's still very strong evidence that there's a high probability that you shouldn't eat those berries.

Posted by: RichardHSerlin | February 22, 2011 9:11 PM | Report abuse

I was going to say almost exactly what RichardHSerlin said, but not as well.

In addition to his many good points, remember that the default expectation for a doctor is to prescribe a treatment, any treatment.

So if the condition is mild and may just as likely go away on its own, the physician is expected to do *something* anyway: "Well, it's probably nothing serious, but it wouldn't hurt to do X" -- where X is a treatment recommendation not based on high-quality data from clinical trials, and the physician and patient both actually know that and are OK with it.

"Well, it wouldn't hurt to take an aspirin," or "it wouldn't hurt to get more rest."

If these recommendations are included in the study, they will seriously skew it.

Posted by: dpurp | February 22, 2011 9:54 PM | Report abuse

This is EXACTLY why I haven't been enrolled in health insurance - what they cover doesn't work! I hate to sound quackish, but I have found better results from herbal remedies which have been proven over hundreds of years.

Posted by: lroberts1 | February 22, 2011 11:06 PM | Report abuse

Post a Comment

We encourage users to analyze, comment on and even challenge's articles, blogs, reviews and multimedia features.

User reviews and comments that include profanity or personal attacks or other inappropriate comments or material will be removed from the site. Additionally, entries that are unsigned or contain "signatures" by someone other than the actual author will be removed. Finally, we will take steps to block users who violate any of our posting standards, terms of use or privacy policies or any other policies governing this site. Please review the full rules governing commentaries and discussions.

characters remaining

RSS Feed
Subscribe to The Post

© 2011 The Washington Post Company