Review 1: "Effectiveness of COVID-19 Vaccines Against Variants of Concern in Ontario, Canada"

RR:C19 Evidence Scale rating by reviewer:

• Strong. The main study claims are very well-justified by the data and analytic methods used. There is little room for doubt that the study produced has very similar results and conclusions as compared with the hypothetical ideal study. The study’s main claims should be considered conclusive and actionable without reservation.

***************************************

Review:

The manuscript deals with a very timely and relevant topic, i.e., effectiveness of the Health Canada-approved vaccines against the various variants recently circulating in Ontario and most of Canada. This manuscript is particularly timely given the recent concerns over vaccine effectiveness, especially with single doses, against Delta. The authors present a logical classification of VOCs based on sequence data available and have made conservative assumptions in cases where sequence data was not available. The sample size is substantial, with 421,073 individuals identified with symptoms and almost 70,000 of these testing positive for VOC or non-VOC SARS-CoV-2. The authors then analyzed partial (1 dose) and full (2 doses) vaccine effectiveness based on disease severity for Alpha, Beta/Gamma, and Delta. The authors report that mRNA-1273 was more effective than BNT162b2 and ChAdOx1 at preventing symptomatic infection with Alpha, and both mRNA-1273 and BNT162b2 showed very high effectiveness after 2 doses.

Protection against symptomatic infection with Beta/Gamma was lower for all three vaccines after 1 dose, but rose to 84% for BNT162b2 after 2 doses. An absence of test-positive cases in individuals who received 2 doses of mRNA-1273 or ChAdOx1 prevented similar analysis for these vaccines. Regarding Delta, the percentages ranged from 56% for BNT162b2 to 72% for mRNA-1273, which is higher than I would have expected in the context of single doses. When hospitalization or death was the outcome analyzed, effectiveness was generally higher in all situations, as expected.

Overall, these findings appear to be rigorously analyzed and the results are, for the most part, consistent with data emerging from similar studies in other parts of the world. It is interesting to note that the single dose data against Delta was higher than the original 33% that is often quoted. The authors have proposed Click here to access/download;Review;Review Russell.docx some potential reasons to explain the heterogeneity in vaccine estimates. The test-negative design appropriately controls for healthcare-seeking behaviour bias. Given the known differences in sequences of the VOCs, I would have preferred a positive test for Delta rather than a proxy measure of 501Y/484K negativity, but given the additional considerations such as date and geographical location, Delta inference should be intact. In general the authors have provided a balanced description of potential limitations of the study.

In summary, I believe this to be a valid study with claims that are well-supported by the data, and the availability of vaccine effectiveness data with specific reference to the various VOCs circulating in Canada makes this a timely and publicly-relevant study. I agree that the data indeed support the notion of delaying second doses in order to maximize the number of individuals with partial protection, but I caution against such a claim because with Delta thought to be the least impacted by single doses, and this study detecting Delta by proxy, I would not make overly strong claims on delaying second doses in the presence of Delta.