Overadjustment bias: when adjusting makes a study worse
Overadjustment bias explained: why adjusting for the wrong variable can distort a finding, and how the birth weight paradox shows it happening.
By Manouchehr Hessabi, MD, MPH
Overadjustment bias happens when an analysis controls for a variable that lies on the causal path between an exposure and an outcome, which removes part of the very effect the study set out to measure. Controlling for a common effect of two variables, called a collider, is a related error that can create an association that does not exist. Both errors run against a common intuition, that a study which "adjusted for" more factors is automatically more trustworthy.
This is an explainer about study methods. It is educational and not a substitute for personal medical advice.
What is overadjustment bias?
Three terms first. An exposure is whatever might influence health, such as a medication, a behavior, or a contaminant. An outcome is the health state being studied. To adjust for, or control for, a variable means to remove its influence statistically, by putting it in a regression model, by analyzing people separately within its levels (stratification), or by studying only people with one value of it (restriction).
In a 2009 paper in Epidemiology, Schisterman, Cole, and Platt observed that the word overadjustment had been used inconsistently, and proposed a precise definition. They define overadjustment bias as control for an intermediate variable, or a descending proxy for one, on a causal path from exposure to outcome. An intermediate variable, also called a mediator, is a step in the chain through which the exposure produces its effect. A "descending proxy" is a measured variable that is itself caused by such an unmeasured step.
The same paper separates this from a milder problem. Unnecessary adjustment is control for a variable that does not change the bias of the estimate but may change its precision, meaning the width of its confidence interval.
Why isn't adjusting for more variables always better?
Adjustment is the standard remedy for confounding. A confounder is a common cause of the exposure and the outcome: a variable with causal arrows pointing out toward both. Leaving a confounder uncontrolled can make an exposure look harmful or protective when it is neither, so controlling for it is correct.
Mediators and colliders have different structures. A mediator sits between exposure and outcome, with an arrow coming in from the exposure and an arrow going out toward the outcome. A collider is a common effect, a variable with arrows pointing in from two directions. Schisterman and colleagues show that overadjustment bias rests on a different causal structure from confounding or selection bias. The practical point is that the decision to adjust depends on where a variable sits in the causal structure, not on how strongly it is associated with the outcome.
The authors also draw a contrast that matters for anyone tempted to solve the problem with more data. Overadjustment bias is not a finite-sample bias, so it does not shrink as a study grows. The inefficiency from unnecessary adjustment, by contrast, is a function of sample size. A very large study can carry overadjustment bias at full strength while looking extremely precise.
What does adjusting for a mediator do?
Suppose an exposure affects an outcome partly through a mediator. The total effect of the exposure includes every pathway, including the one through the mediator. Schisterman and colleagues explain that if an analyst controls for the mediator, the total causal effect of the exposure on the outcome cannot be consistently estimated, and the observed association will typically be biased toward the null, that is, toward no effect. If the mediator is the only path, the adjusted association will typically be null in expectation even when the exposure has a real effect.
Their worked example is the relation between maternal smoking and neonatal mortality, using US birth certificate records. Birth weight is a strong predictor of infant death and is often adjusted for in such studies. Following earlier work by Basso and colleagues, the authors consider a causal diagram in which the link between birth weight and mortality is due to an unmeasured condition, such as a malformation or an infection, that affects both. Under that diagram, adjusting for birth weight when estimating the effect of smoking represents overadjustment.
Controlling for a mediator does not always reflect a mistake in arithmetic. It can reflect a change of question. An analysis that holds the mediator fixed is asking something closer to how much of the effect travels by other routes, which is a different question from the total effect. Trouble starts when a paper answers one question and reports it as the other.
What is collider bias?
Collider bias is easiest to see in a hypothetical example published in the International Journal of Epidemiology by Cole and colleagues in 2010. One hundred people attend a meeting. Ten arrive already infected with influenza. Each person eats either a chicken sandwich or an egg-salad sandwich, 50 of each, chosen at random, and the egg-salad sandwiches are tainted. In this imagined world, a high fever has only two causes: influenza or a tainted sandwich.
Because sandwiches were handed out at random, influenza and sandwich type are unrelated. About five influenza cases are expected in each sandwich group. Now look only at the people who develop a fever, which means conditioning on a common effect. Among feverish people who ate chicken, every one must have influenza, a risk of 5 out of 5. Among feverish people who ate egg salad, the influenza risk is 5 out of 50. The authors note that this association is introduced by conditioning on a common effect, namely fever, between two variables that were independent to begin with.
Cole and colleagues call this selection bias, since the same distortion arises when a study enrolls people on the basis of a common effect, such as being a hospitalized case. Under Schisterman's definition, overadjustment bias refers specifically to intermediate variables, so collider bias is best treated as a close relative with its own name. A single variable can, however, be both a mediator and a collider, which is exactly what happens in the next example.
What is the birth weight paradox?
Birth weight is a strong predictor of infant death. In 1991 US data, infants born to smokers had higher risks of low birth weight and of infant mortality than infants born to nonsmokers. Yet among low-birth-weight infants, mortality was lower for infants born to smokers, with a relative rate of 0.79. A relative rate below 1 means a lower rate in the exposed group.
One proposed explanation was that maternal smoking is somehow beneficial for small babies. In a 2006 paper in the American Journal of Epidemiology, Hernández-Díaz, Schisterman, and Hernán used causal diagrams to show that the inverse association can appear even with no beneficial effect of smoking at all. Birth weight is affected by the exposure and also shares common causes with the outcome. Among low-birth-weight infants, those born to smokers may owe their small size largely to smoking, while other low-birth-weight infants may owe it to causes associated with high mortality, such as birth defects. Comparing the two groups within the low-birth-weight stratum therefore compares infants who became small for very different reasons.
The authors' conclusion is stated in proportion. Under realistic causal diagrams, adjustment for birth weight is unwarranted when the goal is to estimate the overall effect of prenatal exposures on infant mortality. Even for estimating direct effects, it may be invalid when an unmeasured common cause of low birth weight and mortality exists. Justifying such an adjustment, they write, requires specifying the causal question and the assumptions about the underlying biological mechanisms.
None of this suggests that smoking during pregnancy is safe. The paradox is a lesson about how an analysis can manufacture a protective-looking association, which is the opposite of evidence for one.
How can a reader spot possible overadjustment in a paper?
No checklist settles the question, but a few questions help a reader judge whether a list of adjusted variables deserves trust:
- When was each adjusted variable measured? A variable measured before the exposure cannot be caused by it. One measured after the exposure began might be.
- Could the exposure cause the variable? If so, it may be a mediator, and adjusting for it may remove part of the effect of interest.
- Could both the exposure and the outcome, or their causes, affect the variable? If so, it may be a collider.
- Did the authors state their causal assumptions? A causal diagram, or a plain description of which variables were treated as confounders and why, lets readers check the reasoning.
- Do the authors report results with and without a questionable variable? A large shift is a signal to read the justification carefully.
These questions do not convict a study. They separate adjustments made for a stated reason from adjustments made because a variable was available.
Frequently asked questions
Is overadjustment the same as confounding? No. Confounding comes from a common cause of exposure and outcome left uncontrolled. Overadjustment bias comes from controlling for a variable on the causal path, and Schisterman and colleagues show the two rest on different causal structures.
Does a bigger sample fix overadjustment bias? No. The same paper shows it is not a finite-sample bias. A larger study estimates the wrong quantity more precisely.
Is collider bias a kind of selection bias? Cole and colleagues describe it that way, because enrolling or analyzing people on the basis of a common effect produces the same distortion as conditioning on it statistically.
Is it always wrong to adjust for a mediator? Not always. It changes the question from the total effect to something closer to a direct effect, and the birth weight paper shows that even that narrower question requires stated assumptions about unmeasured common causes.
Closing note
Adjustment is a tool for answering a causal question, and it works only when the variables chosen fit that question's structure. Readers who want to see how these design choices play out in applied work may find the peer-reviewed publications listed on this site a useful place to continue.