RR:C19 Evidence Scale rating by reviewer:
***************************************
Review:
This study found that both symptom reactions to mRNA vaccines and objective biomarker measures are associated with a higher neutralizing antibody response to vaccination, which is a known correlate of protection for COVID-19. These findings suggest that public health messaging could seek to reframe systemic symptoms after vaccination as potential markers of benefit from the vaccine.
The overall conclusion of this study, “In the context of low ongoing vaccine uptake, our convergent symptom and biometric findings suggest that public health messaging could seek to reframe systemic symptoms after vaccination as desirable,” is reasonably well supported by the study and appropriate.
Study design elements supporting interpretability of results include (1) restricting to SARS-CoV-2 negative individuals and two-dose recipients of the Moderna or Pfizer mRNA vaccine, (2) measuring nAB at 6 months as well as at one month post-vaccination; and (3) the study of both objective biomarkers (measured by biometrics) and symptoms as predictors of nAB.
The statistical analyses provide credible results, and the results are effectively presented in figures and tables. There are some limitations in the statistical methodology employed and hence on the robustness and reliability of conclusions.
One limitation is reliance on linear mixed models with their parametric assumptions, which are likely misspecified and hence causing biased estimates of association parameters. For example, for the result on the relationship of dose 2 symptoms with nAB, the manuscript did not report on whether the use of a log-linear association assumption was a good fit to the data. Nonparametric analysis would be possible for this data set and would provide more robust results. Nonparametric analysis would be possible not only for inferences (e.g., for conditional means of nAB), but also for the quantification of individual-level prediction accuracy.
Relatedly, because symptoms and biometric input variables were not randomized, it is not clear whether these input variables have a causal effect on nAB, versus an alternative explanation of unaccounted for unmeasured confounding (beyond vaccine type, age, sex, BMI that were adjusted for). Of course, it is hardly possible to randomize these input variables; the point here is that the lack of a sensitivity analysis to quantify robustness of conclusions to potential unmeasured confounding limits understanding about the confidence level in the statistical inferences.
The data analysis studies all predictors one at a time, thus not allowing for potential characterization of joint associations of predictors with nAB. In particular, does the article provide a rationale for how come the symptoms data and the biometrics data are not studied together as predictors of nAB? Relatedly, is there any overlap in the reported symptom categories, for example, does ‘feeling unwell’ include other types of symptoms like chills, fatigue, fever, etc.? Showing the intercorrelations of symptoms provides additional information.
Will the difference in the number of participants in the symptoms analysis and in the biometrics analysis (363 vs 167) cause any bias (give the use of p-values to flag inferred differences)?
A scope limitation of the conclusions of this study are that it is unknown how much of the effect of symptoms on nAB (or biometrics readouts on nAB) would translate to an effect on SARS-CoV-2 infection/COVID-19 outcomes.
The significance of the study findings are limited by the lack of interpretation in terms of the literature on how the estimated nAB associations are expected to correspond to vaccine efficacy or relative vaccine efficacy. For example, if certain symptoms correlate with 1.6-fold higher nAB geometric mean compared to an absence of those symptoms, what is the translation of the 1.6-fold difference to the expected difference in vaccine efficacy (or relative vaccine efficacy)?