The imprinting effect of covid-19 vaccines: an expected selection bias in observational studies

Specification of the target trial

Consider a target trial with the following eligibility criteria for participation: aged 18 years or older; received the second dose of an mRNA vaccine at least 90 days previously but not yet received the third dose; no previous laboratory confirmed SARS-CoV-2 infection; and not part of a population with special vaccination requirements (eg, nursing home residents, healthcare workers). The intervention would have three components (fig 1). Firstly, eligible people would be randomly assigned to either immediate receipt of a booster (third dose) of an mRNA covid-19 vaccine, or no further vaccine doses. Secondly, all participants would have to remain uninfected for, say, six months after randomisation. Thirdly, all participants would be intentionally infected with omicron at six months. The outcome of interest would be a laboratory confirmed omicron reinfection at least three months after the first omicron infection. (A variation would be to assign a random time of infection within, for example, six months after randomisation.)

Fig 1

Design of a hypothetical target trial to evaluate the direct effect of a covid-19 vaccine booster dose, not mediated through first infection, on risk of reinfection

• Download figure
• Open in new tab
• Download powerpoint

Individuals would be followed from randomisation. However, because the outcome cannot occur (by definition) until nine months after randomisation, the cumulative incidence curves for both groups would stay at zero for the first nine months. Therefore, if no reinfection can truly occur during the first three months after a first infection and assuming no deaths or losses to follow-up occur, individuals can be equivalently followed from nine months after randomisation until the earliest of omicron reinfection or administrative end of follow-up (end of the study).

This target trial is not feasible because in the real world we cannot force people to become infected at a time of our choosing. But that is beside the point, because the aim of this thought experiment is to specify the causal question as unambiguously as possible, rather than to design an actual trial. If this target trial could be performed, we could use its data to quantify the (controlled) direct effect of a covid-19 booster on a second omicron infection that is not mediated through the first omicron infection by simply comparing the risk of reinfection between individuals assigned to booster and individuals assigned to no booster. If this direct effect exists, then immune imprinting has occurred.

Emulation of the target trial is not possible

When a target trial cannot be carried out, observational data are often used from human populations to emulate it.89 In fact, the previous observational study4 implicitly tried to emulate the target trial described in the previous section by adjusting for factors that may confound the effect of the booster on reinfection and by restricting the analysis to individuals with a first infection after the booster. Let us suppose that adjustment for confounding was successful and therefore the observational study appropriately accounted for the lack of randomised assignment of the booster. Even in that setting, the selection of individuals who had a first omicron infection is expected to introduce bias10 because in the real world, an infection is expected to occur more often among people with higher susceptibility.11 Therefore, if the booster prevents infections, it is essentially guaranteed that people who received the booster and subsequently had a first infection are, on average, more susceptible to reinfection than people who did not receive the booster and subsequently had a first infection. In the absence of data on individual susceptibility, an observational study cannot unbiasedly estimate the direct effect of a booster because the restriction on first infection introduces selection bias.

To see this graphically, we use causal diagrams that are referred to as directed acyclic graphs.1213 The first graph in figure 2 represents the (randomised) target trial specified previously, and the second graph represents the observational analysis restricted to people with a first infection. Both graphs include the variables booster (yes or no), confirmed infection in period 1 (yes or no), confirmed infection in period 2 (yes or no), and an unmeasured “susceptibility” variable that represents individual characteristics that increase the risk of infection (eg, subclinical immunosuppression, occupational and behavioural factors) or of receiving a diagnosis of infection (eg, testing behaviour, access to the health system). Period 1 ends at the earliest of six months or a first infection, and period 2 starts right after the end of period 1.

Fig 2

Simplified causal diagrams representing a hypothetical target trial in which participants are randomly assigned to booster (yes or no) and then intentionally infected with omicron, and an observational study. The dotted arrow represents the direct effect of the booster, not mediated through first infection, on risk of reinfection. The boxed text represents the conditioning of the analysis on individuals with value “yes”

• Download figure
• Open in new tab
• Download powerpoint

In the target trial, there is no arrow from booster to infection in period 1, which, by design, occurs in all individuals, regardless of whether they were assigned to booster or no booster (fig 2). In addition, there is no arrow from susceptibility to first infection because, under the intervention of the target trial, infection is guaranteed regardless of individual susceptibility. Therefore, the unconditional association between booster and infection in period 2 is an unbiased estimator of the direct effect of booster on reinfection (the dotted arrow) because everybody had a first infection. The graph also represents a variation of the target trial in which individuals are randomly assigned to infection or no infection in period 1.

In the observational study, there are arrows into infection in period 1 from both booster, which prevents infections,1415 and susceptibility, which increases the risk of infection (fig 2). Also, when the observational study is restricted to people with infection in period 1, a box is placed around infection in period 1 to represent the conditioning of the analysis on individuals with value “yes” on that variable. In graph theory, infection in period 1 is regarded as a collider because it is a common effect of booster and susceptibility.16 Therefore, restricting to those with infection in period 1 equal to “yes” is a form of collider stratification, which is expected to induce a non-causal association between booster and susceptibility, and, because susceptibility is associated with second infection, between booster and second infection.101316 That is, the association between booster and reinfection among those with a first infection combines the direct effect of booster on reinfection (the dotted arrow), if any, and the selection bias induced by conditioning on first infection.

If the booster had no direct effect (ie, if the dotted arrow did not exist) then the risk of infection in period 2 would be expected to be greater for those who received the booster than for those who did not receive the booster. This higher risk in the booster group compared with the no booster group is entirely the result of selection bias and thus has no causal interpretation as a harmful effect of the booster on reinfection. In fact, all that this increased risk indicates is that those who become infected despite receiving a booster are people who are more susceptible to reinfection.

Replication of the selection bias in simulated data

We designed a simplified simulation to quantify the magnitude of the selection bias under the causal directed acyclic graph for the observational study in figure 2 (without an arrow from first to second infection, for simplicity). We simulated a dataset of 10 million people with a normally distributed susceptibility variable, of whom 65% were randomly assigned to booster. We assumed that the booster decreased the probability of infection in period 1 by 50% and had no effect on the probability of infection in period 2—that is, the booster had no direct effect on reinfection. We considered separate scenarios in which the risk of infection is increased between two and eight times per each standard deviation in susceptibility (see supplementary file for details and computer code).

For a realistic risk of infection of 10% in period 1, the odds ratio of infection in period 2 for booster compared with no booster ranged between 1.04 and 1.37, depending on the assumed distribution of susceptibility to infection in the population. As expected, restricting the observational analysis to individuals with a first omicron infection results in a higher risk of reinfection in the booster group even if the booster had zero effect on reinfection. It is all selection bias.

Replication of the selection bias in real world data

We conducted the observational analysis using linked individual level data from three Spanish population registries (vaccination registry (REGVACU), laboratory results registry (SERLAB), and national health system registry).15

We identified individuals eligible for the target trial starting on 1 January 2022, when omicron accounted for >90% of the circulating variants in Spain. We then assigned those who received a booster to the booster group, and for each of them we randomly chose a matched control who had not received a booster up to that week. The matching factors included sex, age (±5 years), province, time since primary vaccination (±14 days), and type of primary vaccine (BNT162b2 (Pfizer-BioNTech) or mRNA-1273 (Moderna)).

In an attempt to emulate a target trial in which all participants are infected with omicron within 10 months of booster assignment, we restricted the analysis to individuals with a laboratory confirmed SARS-CoV-2 infection in the next 10 months (and further matched eligible individuals on week of infection)—that is, we only considered for the analysis individuals with an omicron infection, and further matched each individual in the booster group with the control individual on week of infection. As mentioned, restricting to individuals with infection induces uncontrollable selection bias because of differential selection between groups that depends on the largely unknown individual susceptibility.

We then followed individuals from day 90 after infection until the earliest of a confirmed SARS-CoV-2 reinfection, death, discontinuation of registration in the national health service database, or administrative censoring at end of the study (31 October 2022). To estimate the per protocol effect, we censored at receipt of any additional vaccine dose. The cumulative incidence (risk) in each group was estimated using the Kaplan Meier method17 and compared between groups through risk ratios. Bootstrapping with 500 samples was used to compute centile based 95% confidence intervals.

Of 12 749 506 initially eligible individuals, 1 704 904 experienced a first infection in the study period. Of these, 425 741 (25.0%) had received a booster dose before the infection. We could exactly match 249 226 (58.5%) individuals with a booster to the same number of controls, with a median age of 44 years. A total of 201 266 (80.8%) matched pairs remained under follow-up 90 days after the infection and were included in the analysis. During a maximum follow-up of 211 days (mean 133 days) 1794 reinfections occurred, with a six month risk over the full period of 0.59% in the booster group and 0.54% in the control group. The risk ratio of reinfection in the booster group compared with the no booster group was 1.08 (95% confidence interval 0.97 to 1.20) at six months of follow-up (nine months post-infection), but varied between 1.03 (0.93 to 1.17) in days 0-90 of follow-up and 1.20 (0.98 to 1.45) in days 91-180. As expected, the booster was associated with a higher risk of reinfection.