RR:C19 Evidence Scale rating by reviewer:
Potentially informative. The main claims made are not strongly justified by the methods and data, but may yield some insight. The results and conclusions of the study may resemble those from the hypothetical ideal study, but there is substantial room for doubt. Decision-makers should consider this evidence only with a thorough understanding of its weaknesses, alongside other evidence and theory. Decision-makers should not consider this actionable, unless the weaknesses are clearly understood and there is other theory and evidence to further support it.
Rochman and colleagues perform an in silico mutagenesis experiment to evaluate the single point mutant landscapes for SARS-CoV-2 RBD from three different variants (“WT” – Wuhan Hu-1, Delta, Gamma) against binding to human ACE2 and a neutralizing antibody (nAb) CV30. Epistasis, as determined by non-linear mutational effects on binding affinity (either to ACE2 and/or to nAb), was judged to be “limited” but with more substantial destabilization of the nAb-RBD binding interface for the Gamma strain. The authors conclude that the “modest ensemble of mutations relative to the WT shown to reduce vaccine efficacy might constitute the majority of all possible escape mutations.” In the current form of this manuscript, these strong claims are not supported but may yield some insight by the data and methods used. Decision-makers should consider the claims in this study not actionable (except to prompt further research) unless the weaknesses are clearly understood and there are other theories and evidence to further support them.
There are three major concerns with the analysis presented that render the claims unsupported and thus not actionable. However, note that these concerns can be addressed in a substantially revised manuscript.
First, the computational method undergirding all analysis and subsequent claims is not appropriately benchmarked to experimental results. The authors compare a small part of the escape mutants obtained computationally on the wild type with the ones obtained by the Bloom group validating the method (Greaney et al., 2020; Starr et al., 2020a, 2020b, 2021). However, there are no complete datasets presented showing a correlation between these previously experimentally determined ACE2-RBD affinities and nAb-RBD affinities and the computational results. For example, one would expect to see correlation coefficients presented, or other methods of assessing reliability with the computational method. This is especially important as the authors use a highly unusual Rosetta method for determining relative binding affinities (DDG). While many aspects of their use of Rosetta correspond to best practices, the authors incorporate a mutation without repacking and minimizing the surrounding positions to the introduced mutation. Any novel in silico method should be appropriately benchmarked with experimental datasets to ensure the validity of further analysis. For example, the authors set a threshold value of 13 for the receptor cost “to remove from consideration mutations that likely produce steric or charge clashes in the structure” but neither in the main text nor in the SI is it explained why and how they get this value. An additional matter of concern for the validity of the in silico approach is that their analysis yields 241 escape mutants for the wild type, which is much higher than the number of escape mutants obtained experimentally for diverse neutralizing antibodies targeting the same epitope (~30) (Greaney et al., 2020; Francino-Urdaniz et al., 2021).
Second, the authors use CV30 as the only nAb in their analysis, as they claim “the antibodies most critical for assessing the risk of vaccine escape appear to reside within the RBD(9) and are well represented by CV30.” It is unclear by what metric this claim is assessed for all competitive inhibitors to RBD, as different competitive inhibitors have very different escape mutant profiles (Greaney et al., 2020; Francino-Urdaniz et al., 2021). Additionally, nAbs that competitively inhibit ACE2 that derive from different germlines neutralize with different attack angles (Barnes et al., 2020). Therefore, concluding that the identified escape mutants could escape vaccine neutralization as a whole is not supported by the evidence from the current form of this manuscript. It would also help with strengthening the developed computational method to prove that different antibodies can be tested. This should be mentioned in the limitations and the introduction to properly set the scope of this research.
Third, in the “limitations” section the authors do a commendable job of describing several limitations of their analysis, particularly in ascribing fitness, and thus epistasis, of viral strains to changes in ACE2-RBD binding affinity (which they measure in silico). However, these limitations are not appropriately described in the abstract and significance section, which is inappropriate given that the major claims of the current form of the manuscript are impacted by these limitations. Any revision should appropriately note such important limitations in the abstract and significance section.
Barnes, C. O. et al. (2020) ‘Structures of Human Antibodies Bound to SARS-CoV-2 Spike Reveal Common Epitopes and Recurrent Features of Antibodies’, Cell, 182(4), pp. 828-842.e16. doi: 10.1016/j.cell.2020.06.025.
Francino-Urdaniz, I. M. et al. (2021) ‘One-shot identification of SARS-CoV-2 S RBD escape mutants using yeast screening’, Cell Reports. Elsevier Inc., p. 109627. doi: 10.1016/j.celrep.2021.109627.
Greaney, A. J. et al. (2020) ‘Complete Mapping of Mutations to the SARS-CoV-2 Spike Receptor-Binding Domain that Escape Antibody Recognition’, Cell Host and Microbe, 29(1), pp. 44-57.e9. doi: 10.1016/j.chom.2020.11.007.
Starr, T. N. et al. (2020a) ‘Deep Mutational Scanning of SARS-CoV-2 Receptor Binding Domain Reveals Constraints on Folding and ACE2 Binding’, Cell. Elsevier, 182(5), pp. 1295-1310.e20. doi: 10.1016/j.cell.2020.08.012.
Starr, T. N. et al. (2020b) ‘Resource Deep Mutational Scanning of SARS-CoV-2 Receptor Binding Domain Reveals Constraints on Folding and ACE2 Binding ll Deep Mutational Scanning of SARS-CoV-2 Receptor Binding Domain Reveals Constraints on Folding and ACE2 Binding’, Cell. Elsevier, 182(5), pp. 1295-1310.e20. doi: 10.1016/j.cell.2020.08.012.
Starr, T. N. et al. (2021) ‘Complete map of SARS-CoV-2 RBD mutations that escape the monoclonal antibody LY-CoV555 and its cocktail with LY-CoV016’, Cell Reports Medicine, 2(4), p. 100255. doi: 10.1016/j.xcrm.2021.100255.