RR:C19 Evidence Scale rating by reviewer:
Reliable. The main study claims are generally justified by its methods and data. The results and conclusions are likely to be similar to the hypothetical ideal study. There are some minor caveats or limitations, but they would/do not change the major claims of the study. The study provides sufficient strength of evidence on its own that its main claims should be considered actionable, with some room for future revision.
***************************************
Review:
This work tries to answer an important question about the specificity of SARS-CoV-2 serological assays in patients with chronic inflammatory conditions. The research is well-grounded in and appears motivated by the existing literature on the development and validation of serological assays for measuring anti-drug antibodies in such patients. The authors demonstrate that pre-pandemic serum samples from such patients can give false-positive results when using commercially available LFAs and ELISA kits under at least some circumstances. Future work may try to build on this research in larger samples across a larger range of chronic inflammatory conditions or try to understand which molecular mechanisms are most likely to lead to results such as those observed here, and these results can be used to justify the importance of separately validating SARS-CoV-2 serological assays in these cohorts.
Understandably, only small sample sizes are available here, but it is enough to demonstrate that this is a problem rather than having a large enough sample to estimate its precise magnitude. For me, the main weakness is that there are no negative controls from pre-pandemic sera from individuals without chronic inflammatory diseases that have been blindly tested at the same time in the same way. Thus, it cannot be completely ruled out that lab- or operator-specific factors, or perhaps even optimistic reports of some assay’s specificity by the manufacturers, do not explain at least some of the results here.
I completely agree with the authors that it is important for patients with these conditions to have access to well-calibrated diagnostic tests. Based solely on the data presented here I was not however convinced by the argument that these particular issues may be relevant for antibody screening in the general population – although 6% of the global population (around 420 million) may suffer from such diseases, only a small fraction of these are represented in this study based on global estimates of prevalence indicated for MS, RA and SLE in the introduction (about 10 million in total?) and we do not (yet) have evidence that testing in patients with other chronic inflammatory diseases would be similarly affected. It may also be worth noting that the results for IgG seem less affected than the results for IgM—this suggests that most such false-positives that arise erroneously imply recent infection rather than infection some time ago, which in turn suggests that—in the absence of well-validated serological assays—testing patients again sometime later (by when IgM levels should have dropped) could help to distinguish between true and false positives.
I would also add that although the sample sizes here were necessarily small, the raw data may actually be enough to allow you to do some more sophisticated statistical analyses of the specificity of these assays in these patient groups—the authors have generated a dataset where they have tested the same samples from the same patients with a panel of assays whose specificity and sensitivity has been reported by the manufacturers. Wonderfully, the authors have provided the raw data in the supplementary material and this is, therefore, a possibility for anyone who cares to try.