Thought-provoking essay on cause and correlation in modern science

Science is the best method we have for understanding the world. That doesn't mean that everything scientists ever think they've figured out is correct. And it doesn't mean that we're doing science in the best way possible right now.

For a great illustration of this, I recommend reading Jonah Lehrer's new piece in WIRED, about the problems we run into as we learn more about individual parts of complex systems and then assume that we understand the big picture of how those parts work together. A lot of scientific research, particularly in medicine, operates off assumptions like this and it can lead to big mistakes. Case in point: Back pain. In this excerpt, Lehrer explains how MRI technology that allowed doctors to get a better look at the spines of people with back pain led them to make inaccurate conclusions about what was causing the back pain.

The lower back is an exquisitely complicated area of the body, full of small bones, ligaments, spinal discs, and minor muscles. Then there’s the spinal cord itself, a thick cable of nerves that can be easily disturbed. There are so many moving parts in the back that doctors had difficulty figuring out what, exactly, was causing a person’s pain. As a result, patients were typically sent home with a prescription for bed rest.

This treatment plan, though simple, was still extremely effective. Even when nothing was done to the lower back, about 90 percent of people with back pain got better within six weeks. The body healed itself, the inflammation subsided, the nerve relaxed.

Over the next few decades, this hands-off approach to back pain remained the standard medical treatment. That all changed, however, with the introduction of magnetic resonance imaging in the late 1970s. These diagnostic machines use powerful magnets to generate stunningly detailed images of the body’s interior. Within a few years, the MRI machine became a crucial diagnostic tool.

The view afforded by MRI led to a new causal story: Back pain was the result of abnormalities in the spinal discs, those supple buffers between the vertebrae. The MRIs certainly supplied bleak evidence: Back pain was strongly correlated with seriously degenerated discs, which were in turn thought to cause inflammation of the local nerves. Consequently, doctors began administering epidurals to quiet the pain, and if it persisted they would surgically remove the damaged disc tissue.

But the vivid images were misleading. It turns out that disc abnormalities are typically not the cause of chronic back pain. The presence of such abnormalities is just as likely to be correlated with the absence of back problems, as a 1994 study published in The New England Journal of Medicine showed. The researchers imaged the spinal regions of 98 people with no back pain. The results were shocking: Two-thirds of normal patients exhibited “serious problems” like bulging or protruding tissue. In 38 percent of these patients, the MRI revealed multiple damaged discs. Nevertheless, none of these people were in pain. The study concluded that, in most cases, “the discovery of a bulge or protrusion on an MRI scan in a patient with low back pain may frequently be coincidental.”

This is a complicated problem without a clear solution right now. But we definitely need to have discussions like this so that we can work toward making science and medicine better.

Via Espen in Submitterator


  1. Already submitterated just for the record. But you guys’ volume has been great lately, helping keep me occupied.

  2. Wait… so we’re not allowed to be wrong and learn from our mistakes? I mean, I get it, we don’t know as much as we think we know. But that’s no excuse to throw up our hands. The example given for torcetrapib is important, because perhaps without going through the trials and ultimately discovering its failure, we wouldn’t have been able to add that knowledge to our understanding of the whole interrelated cholesterol mess.

    So to my mind, anyway, this is an indication of the success of science, not its failure.

    1. I’m not sure what article you think you read.

      Do you think having one’s back permanently maimed, because some scientist decided to label normal anatomical structures as “abnormal,” doesn’t qualify as “failure?”

      1. I said it wasn’t a failure of science. We learn from our failures in trials. No doubt, it’s a tragedy, but if we didn’t do anything unless we were absolutely guaranteed that nothing would go wrong, we wouldn’t do anything at all, nor would we learn anything.

      2. Doctors had patients with back pain, and new technology let them see there were bulges and “odd” things happening with the associated spinal structures. There was no reason to run an (expensive) MRI on folks with no back pain to see whether those odd structures showed up in them. Correlation became causation – but incorrectly. Then one day they decided to check it out, and lo and behold, those abnormal structures – while perhaps part of some other issue, or perhaps not – had nothing to do with the back pain. The view on what causes back pain and how to treat it changed as a result.

        The only failure here was accepting correlation for causation, which is the whole point of the article – that we cannot be quick to do that, however “obvious” and tempting it might be.

        1. See the thing about the “there was no reason to run MRIs on healthy people” argument means that this was a failure of medical care, not science, because the scientific method most definitely prescribes MRIs on healthy people in that situation.

  3. I’ve known several people who suffered back pain, and had surgery to remedy it. In none of the cases did the pain improve; in all of them their debilitation increased after surgery, some of them receiving multiple “experimental” procedures in an attempt to relieve the damage caused by previous operations.

  4. I think I will still continue to conclude that the herniated lumbar discs that are impinging directly against my sciatic nerve are much more likely to be the cause of my low back pain than depression (I’m not depressed) or smoking (I don’t smoke).

    1. What this article is saying, is that doctors were too quick to look at an instrument scan, and assume that what they were looking at was the cause of the pain, rather than a symptom, then use radical surgery to “fix” the “problem,” often causing more, permanent, damage.

      An alternative explanation is that the lumbar discs may not be the problem in and of themselves; it may be that your muscles are pulling on your back in a way that deforms the discs, and pinches your nerves. The solution in that case wouldn’t be to remove the disc, it would be to retrain your muscles to stop doing that, through postural adjustments and exercise.

      1. “The solution in that case wouldn’t be to remove the disc, it would be to retrain your muscles to stop doing that, through postural adjustments and exercise.”

        The problem is that this results in a physical therapist being paid, instead of a doctor and a pharmaceutical company. I’m sure you can see why this prescription is disfavored (or, more precisely, why the medical industry is largely set up to at least not encourage and perhaps actively discourage doctors from learning this lesson).

  5. i’m really glad to read this. a month ago i had such bad “old man lower back” that every time i got up from the computer, i had to clutch myself for a few seconds.  now, though, it’s totally back to normal. benign neglect was, it turned out, the best treatment.  had i taken homeopathic fairy dust, i’d be singing its praises instead.

      1. I chose the lower back pain example because I have that issue myself. Luckily, I also had an amazing, evidence-based doctor who explained that researchers had done all these studies comparing the effectiveness of various treatments … drugs, surgery, chiropractic, etc. But they had to keep stopping these studies because the control of moderate exercise seemed to work better than any of those things. 

        So now I exercise regularly. When I keep it up at at least 3 times a week, my back doesn’t hurt. 

          1.  Oh I’m sure, that’s a big part of back pain in general.  I was just sharing my personal experience.

  6. A doctor misusing new equipment leading to a poor diagnosis is a far cry from what a scientist does. A scientist would know that any good hypothesis has to be taken in the context of the systems it interacts with or is affected by. Otherwise, it ain’t a valid hypothesis.

    1. This isn’t about individual doctors misusing new equipment (though certainly that is a problem), it’s about students going to medical school, and learning these causal doctrines as accepted truth, that they then play out on their patients.

      1. And that’s why any decent doctor keeps up-to-date on the field – medicine is simply one of the fastest-moving fields there is. My dad illustrated this vividly over Christmas by pointing out that of all the drugs in the (thick, heavy) PDR when he was in medical school, only a handful are still used; similar things can be said of a lot of treatments. While the basics of anatomy and patient care are very much the same, most of his drug and treatment facts are obsolete, so doctors have to keep up-to-date. At the same time, they have to be careful not to be swept away in the latest treatment or diagnosis until it’s been properly tested and established. A tricky balancing act, to be sure.

        1. Few doctors can do that, not with the workloads they’re expected to carry. I’ve discussed new research and treatments with doctors who had never heard of such things. The system itself is designed to discourage it.

  7. It’s a fairly common mistake: when you run a test only when there’s a problem apparent, you find a correlation where none really exists. The classic example is asking “Does drinking water make you more likely to be a criminal?”. The answer is obviously “No.”. But if you take a sample of jail inmates, find out how much water they drank and crunch the numbers, you’ll find there’s a very high correlation between any consumption of water and criminal behavior. That isn’t because drinking water causes criminal behavior, it’s because *everybody* drinks water including criminals.

    If you want to ask “Does X cause Y?”, you can’t select your sample based on Y and look at the occurrence of X. You have to select samples of X and not-X, and then look at whether the occurrence of Y differs significantly between them.

    1. What do you do when you have samples of [A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, Z1, A22, HM-23], and [not-A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V, not-W, X, Y, Z, Z1, A22, HM-23]?

      1. Option one: Look for simpler subsets of the data/simpler versions of the problem, if they exist, and then test to see if the conclusions carry over. This is easier but will give more wrong results.

        Option two: collect a whole heck of a lot of data and accept that your analysis has to be 29-dimensional. This is much more expensive.

    2. Naw, if everybody drinks water, criminals and non-criminals alike, then there is not a “very high correlation between any consumption of water and criminal behavior.”  The correlation in this example is zero.

  8. Please note that medicine and science are not the same thing. Many medical procedures are based on case studies or trial and error. With a real scientific approach the control (analysing a group of healthy individuals with the same technique) would have been done.

    1. I am neither a scientist nor a doctor but I find it astonishing that the results of the MRI scans of patients suffering back pain were not compared against a control group for well over a decade.

  9. I think the WIRED passage does not use the word “correlated” correctly.

    This sentence says that people with degenerated discs were MORE likely to have back pain, compared to people without degenerated discs: “Back pain was strongly correlated with seriously degenerated discs, which were in turn thought to cause inflammation of the local nerves.”

    By contrast, this sentence says that people with degenerated discs were LESS likely to have back problems, compared to people without degenerated discs: “The presence of such abnormalities is just as likely to be correlated with the absence of back problems….”

    The author probably meant the first sentence to say “people with back pain commonly have degenerated discs” and the second sentence to say “people without back pain also commonly have degenerated discs.”

  10. Why did it take 20 years of using an MRI machine to diagnose back pain before a set of control images was created? Did they use one of those life-size plastic skeletons for a control?

    How could you use an imaging technology for 20 years without looking at a few symptomless people?

    I’m pretty sure that if you asked a bunch 12th graders how to set up an experiment, at least a few of them would be able to tell you that it involves a control group and that large sample sizes are better than small ones.

    It seems like this is a failure of basic scientific competence rather than a reason to question the limits of the scientific method.

  11. But you might have a situation where prisoners only had water to drink and people outside drank soda, beer, wine, tea, etc. Then you would have a positive correlation of “People who drink more water than other drinks are likely to be criminals.”

  12. Sounds like doctors who decided that back abnormalities caused back pain did not do a controlled test of this hypothesis. That means they were not doing science.

  13. This is actually a pretty bad example of the root problem, namely the confusion of cause-effect with correlation.  In the back pain example, there’s not even a correlation to begin with!

    A better examples is from Helmut Sies writing in Lancet in the 1970s… birth rates in Germany have declined over the past 5 decades, and the population of wild storks also declined.  Correlation, yes.  Cause and effect, probably not!

  14. The causal shortcuts spoken of in the article are exactly why I’m skeptical of GMOs sold as solutions to poverty and starvation – “feeding the poor”. Apart from the standard capitalist fallacy that the solution should be created afresh rather than change from within (through more equitable food/money/power/energy distribution) we build a solution to sell. But the only evidence we have is that it grows better without this or that. 

    Absolutely *no* long term impact studies on what happens when your GMO crop cross pollinates with my organic crop accidentally and we all end up eating Monsanto’s crop. But we have no idea that tweaking this or that within the crop wont affect our nutritional uptake, or any number of other potential dietary problems…

    Anyway, my 2c

Comments are closed.