Review 1: "SARS-CoV-2 Correlates of Protection from Infection Against Variants of Concern"

RR:C19 Evidence Scale rating by reviewer:

• Potentially informative. The main claims made are not strongly justified by the methods and data, but may yield some insight. The results and conclusions of the study may resemble those from the hypothetical ideal study, but there is substantial room for doubt. Decision-makers should consider this evidence only with a thorough understanding of its weaknesses, alongside other evidence and theory. Decision-makers should not consider this actionable, unless the weaknesses are clearly understood and there is other theory and evidence to further support it.

***************************************

Review: In this manuscript, the authors use data from the PHIRST-C cohorts, studied in part to evaluate the effects of viral interactions (among SARS-CoV-2, influenza, RSV, etc.) on health outcomes. In the reported study, the authors evaluate evidence for infection-induced neutralizing antibody (nAb) titers acting as immune correlates of protection (CoPs), that is, antibody markers carrying part of the protective effect of immunity induced by prior infection against subsequent re-infection. As noted by the authors, CoPs are often of interest in vaccine efficacy clinical trials, where any given immune marker may capture a portion of the putative protective effect of vaccination; in particular, immune correlates analyses to evaluate CoPs were the subject of much study in the wake of the COVID-19 pandemic, yet CoPs are not usually studied in the context of natural immunity acquired by previous infection. While the data source exploited in this analytic study (the PHIRST-C cohorts) seems quite rich, and much useful information is provided on how neutralizing antibody measurements were extracted for their evaluation as candidate CoPs, the analytic study falls far short of achieving some of its claimed goals, especially in the area of causal mediation analysis. While the authors claim in the manuscript body that standard/modern techniques from causal mediation analysis are used to inform their evaluation of direct and indirect effects through particular nAb measurements, the Methods section makes clear that this was not the case. In the relevant section, the authors refer to a classical paper of Halloran and Struchiner that outlines direct and indirect effects under (partial) interference between units (note that this is not the same setting as mediation analysis, though both subfields use overlapping terminology) and define "causal parameters" that somehow seem to be completely devoid of any notion of potential outcomes (counterfactual random variables). What's more, in the Methods section, no mention is made of identification conditions necessary to evaluate the causal parameters of interest from the observed data -- one of the most basic steps of conducting a causal inference analysis; in the main text, the authors note two identification assumptions (about exposure-outcome confounding and mediator-outcome confounding) but neither return to these nor clarify their role in their causal inference procedure. From a statistical perspective, many of the assumptions made are simplistic, with parametric models used to define key quantities, without any mention of whether such modeling choices may be justified based on the available data and the study conducted. Overall, the authors provide a glimpse at evidence that would be interesting, were it not for some of the fundamental causal inference and statistical issues that come up.