RR:C19 Evidence Scale rating by reviewer:
Reliable. The main study claims are generally justified by its methods and data. The results and conclusions are likely to be similar to the hypothetical ideal study. There are some minor caveats or limitations, but they would/do not change the major claims of the study. The study provides sufficient strength of evidence on its own that its main claims should be considered actionable, with some room for future revision.
This is an important study showing a concerning lack of efficacy of the ChAdOx1 nCoV-19 (AZD1222) COVID-19 vaccine, better known as the AstraZeneca or Oxford COVID-19 vaccine. Madhi and colleagues report on the Phase III trial of the vaccine in South Africa, where a very high prevalence of a more virulent mutation of the original SARS-COV-2 virus has been causing global concern. It accounted for 93.9% of the cases of COVID-19 in this trial. This vaccine had been previously reported to have around 70% efficacy in preventing symptomatic COVID-19, based mostly on the UK portion of their global trial .
In the present report, there was no statistically significant difference in the number of cases of COVID-19 due to the B.1.351 variant between those given the vaccine and those given placebo. This is an important and novel finding that has global implications, as it implies the AZD1222 vaccine will not be suitable for use in areas where this mutation is prevalent. This currently chiefly involves South Africa but the same mutation has been found in other countries. Of special concern is that the variant rapidly became the predominant strain in South Africa, implying it has some sort of evolutionary advantage over other strains and may do the same elsewhere. Containment of B.1.351 variants will therefore become a special priority for health care systems. Two other vaccine manufacturers (Novavax and Janssen/Johnson & Johnson) have reported a higher degree of effectiveness against the South African strain, around 50-60%, though these results have not been published in a peer-reviewed journal at the time of writing. In light of these findings, it is likely that AstraZeneca will quickly want to produce an updated version of their vaccine to provide better protection against new variants that have emerged in the UK, South Africa, and Brazil—precisely the countries where it has sited their Phase III trial. Nevertheless, this latest report adds to recent bad publicity around its safety, though no concerns around safety have ever been found in any of their randomized double-blind placebo-controlled trials.
The study has important limitations that should be emphasized. Firstly, there were only 42 cases of COVID-19, so it is underpowered. The 95% confidence interval for efficacy, therefore, straddles both zero efficacy and the 50% threshold that most regulators would accept as good enough to gain a license. Secondly, the median age of participants was only 31 years—not much over half of that reported in some of the other vaccine trials. This matters because younger people have generally higher rates of vaccine efficacy, while older people are more likely to have serious diseases . The under-recruitment of older participants is inexcusable in a trial of a condition where most of those with serious adverse outcomes are elderly. Thirdly, there were insufficient cases of severe COVID-19 to make any comment on efficacy. This matters because severely restrictive pandemic public health measures have been taken due to the high incidence of severe COVID-19. We need vaccines to reduce deaths and hospitalizations and reduce pressure on healthcare systems. Arguably this can only be demonstrated via a reduction in severe cases of COVID-19.
 Voysey M, Clemens SAC, Madhi SA, et al. Safety and efficacy of the ChAdOx1 nCoV-19 vaccine (AZD1222) against SARS-CoV-2: an interim analysis of four randomised controlled trials in Brazil, South Africa, and the UK. Lancet 2020;397:99-111.
 Soiza RL, Scicluna C, Thomson EC. Efficacy and safety of COVID-19 vaccines in older people. Age Ageing 2021;50:279-283.