RR:C19 Evidence Scale rating by reviewer:
Potentially informative. The main claims made are not strongly justified by the methods and data, but may yield some insight. The results and conclusions of the study may resemble those from the hypothetical ideal study, but there is substantial room for doubt. Decision-makers should consider this evidence only with a thorough understanding of its weaknesses, alongside other evidence and theory. Decision-makers should not consider this actionable, unless the weaknesses are clearly understood and there is other theory and evidence to further support it.
***************************************
Review:
In this observational study by Pawloski et al., the authors examine whether a documented history of non-COVID-19 vaccines is associated with a protective effect against SARS-CoV-2 infection. In a population of patients seeking care at a single health system with at least one documented SARS-CoV-2 test results (negative or positive), the authors examined the association between vaccination history as recorded in the electronic health record and the risk of having a positive SARS-CoV-2 test result. The authors examined the protective effect of 18 commonly used vaccines and accounted for potential differences between those with and without vaccination history using propensity score matching. The authors also conducted sensitivity analyses to determine the potential impact of unmeasured confounding and a “healthy user” effect on the observed results. Among patients with an available test result, the authors report a protective effect against a positive SARS-CoV-2 test among patients with a documented history of 7 non-COVID-19 vaccines (Polio, Haemophilus influenza type B, High Dose/Adjuvanted Flu, Hepatitis A/Hepatitis B, Measles/Mumps/Rubella, Varicella and PCV13). This has implications for identifying potential preventative interventions against SARS-CoV-2 in the period before an approved SARS-CoV-2 vaccine is widely available.
The authors outlined a robust statistical analysis plan that addresses important confounding-related issues, yet two methodological limitations still have the potential to influence the observed results, including: (1) the potential for selection bias due to the reliance on a population of patients with prior care at a single healthcare system that also sought COVID-19 testing within the system, (2) the reliance on immunization history as recorded in the electronic health record to measure the receipt of a vaccine in the prior 5 years.
As it relates to selection bias, it is difficult to understand the true impact of vaccination status on the risk of SARS-CoV-2 in a study population that excludes patients without a test result. If we assume most patients without a test result would be SARS-CoV-2 negative if tested, then any difference in vaccination history among these patients could potentially bias the observed results. If patients that never sought a test are also less likely to have an up-to-date vaccination history (due to being younger or having fewer comorbidities), vaccinated and unvaccinated SARS-CoV-2 negative patients will not be excluded equally from the study, thus leading to a potentially biased relative risk estimate. One potential sensitivity analysis to account for this possibility could be to compare the risk of a positive SARS-CoV-2 test result among a propensity-matched cohort of vaccinated and unvaccinated patients with established care in the healthcare system, with the assumption that those without a test result are SARS-CoV-2 negative.
Additionally, the reliance on immunization records in the electronic health record, although recognized as a limitation by the study authors, is a point of concern. The electronic health record may be a less than optimal source for 5-year vaccination history with vaccines that are not recommended on an annual basis. Without the performance characteristics of the algorithm used to define vaccination status for each vaccine (i.e., sensitivity, specificity, positive predictive value, negative predictive value), it is difficult to determine the potential impact of measurement bias on the observed results.
In summary, this is an interesting, well-described and robust study that is potentially informative for understanding the potential benefit of existing non-COVID-19 vaccines against SARS-CoV-2 infection. However, the results of the study should be considered in context of the known limitations of the study population and the measurement bias inherent in the assessment of vaccination status from the electronic health record.
Creative Commons Attribution 4.0 International License (CC-BY 4.0)
