RR:C19 Evidence Scale rating by reviewer:
Potentially informative. The main claims made are not strongly justified by the methods and data, but may yield some insight. The results and conclusions of the study may resemble those from the hypothetical ideal study, but there is substantial room for doubt. Decision-makers should consider this evidence only with a thorough understanding of its weaknesses, alongside other evidence and theory. Decision-makers should not consider this actionable, unless the weaknesses are clearly understood and there is other theory and evidence to further support it.
***************************************
Review:
The article is a retrospective cohort study to evaluate the effectiveness of Pfizer bivalent booster against hospitalization and death in individuals aged >64 years in Israel. The study was performed from September to December 2022, including 662,701 individuals that received primary series with BNT162b2, including 85,314 (14%) that received the booster at any time during the study period. A multivariable Cox proportional-hazards regression model was used to estimate vaccine effectiveness (VE). The authors concluded that the bivalent booster increased protection against severe outcomes, with large confidence interval for VE against death.
There are significant points regarding the methods of this study design which can potentially reduce the quality of evidence. First, Israel started offering a booster dose (third BNT162b2 dose) in September of 2021 and, sometime later, a fourth dose. In September of 2022, it started a campaign with bivalent BNT162b2, which represents its potential fifth dose. Therefore, a detailed flowchart is necessary informing the number of doses received by each group and previous infection status. Individuals aged 65 years or older with only two doses may have a higher risk of severe outcome compared to those who received their last booster. Second, the study lacks a clear classification of hospitalization due to COVID-19, an important issue since it differs among studies of vaccine effectiveness.
The description of statistical analysis lacks clarity regarding the choice of confounders. It also misses essential information about the interval between the primary series and the bivalent booster. Increased protection with the higher interval between doses has been demonstrated and could have influenced the final results[1]. The table 2,3 and 4 suffer from the table 2 fallacy [2], which can disinform an inattentive reader.
Even considering the relatively small number of events, and the short follow-up period (more than half of individuals have less than 35 days of follow-up), the study provided potential evidence of additional protection of bivalent vaccine against hospitalization [HR 95%CI= 0.19 (0.08-0.42)] and death 0.14 (0.02-1.03). However, it lacks comparison against individuals with a monovalent booster, precluding any conclusion regarding the superiority of a bivalent booster.
References:
1. Tenforde MW, Weber ZA, Natarajan K, Klein NP, Kharbanda AB, Stenehjem E, Embi PJ, Reese SE, Naleway AL, Grannis SJ, DeSilva MB. Early estimates of bivalent mRNA vaccine effectiveness in preventing COVID-19–associated emergency department or urgent care encounters and hospitalizations among immunocompetent adults—VISION Network, nine states, September–November 2022. Morbidity and Mortality Weekly Report. 2022 Dec 30;71(5152):1616-24.
2. Westreich D, Greenland S. The table 2 fallacy: presenting and interpreting confounder and modifier coefficients. American journal of epidemiology. 2013 Feb 15;177(4):292-8.
Creative Commons Attribution 4.0 International License (CC-BY 4.0)
