Skip to main content
SearchLoginLogin or Signup

Review 2: "Validation of a Saliva-Based Test for the Molecular Diagnosis of SARS-CoV-2 Infection"

This study finds that saliva samples can be used for population screening at a high rate of sensitivity and specificity, supporting other studies already showing that saliva tests can be comparable to NP swabs for COVID-19 testing. While reliable, there are some study flaws.

Published onOct 21, 2021
Review 2: "Validation of a Saliva-Based Test for the Molecular Diagnosis of SARS-CoV-2 Infection"
1 of 2
key-enterThis Pub is a Review of
VALIDATION OF A SALIVA-BASED TEST FOR THE MOLECULAR DIAGNOSIS OF SARS-CoV-2 INFECTION
Description

ABSTRACTBackgroundSince the beginning of the pandemic, clinicians and researchers have been searching for alternative tests to improve screening and diagnosis of SARS-CoV-2 infection (Y. Yang et al., medRxiv 2020; W. Wang et al., 2020.3786; A Senok et al., Infect Drug Resist 2020). Currently, the gold standard for virus identification is the nasopharyngeal (NP) swab (N. Sethuraman et al., JAMA 2020; A.J. Jamal et al Clinical Infect Disease 2021). Saliva samples, however, offer clear practical and logistical advantages (K.K.W To et al, Clinical Microb and Infect; A.L. Wylle et al. N Engl J Med 2020; N. Matic et al, Eur J Clin 2021) but due to lack of collection, transport, and storage solutions, high-throughput saliva-based laboratory tests are difficult to scale up as a screening or diagnostic tool (D. Esser et al., Biomark Insights 2008; E. Kaufman et al., Crit Rev Oral Biol Med2002). With this study, we aimed to validate an intra-laboratory molecular detection method for SARS-CoV-2 on saliva samples collected in a new storage saline solution, comparing the results to NP swabs to determine the difference in sensitivity between the two tests.MethodsIn this study, 156 patients (cases) and 1005 asymptomatic subjects (controls) were enrolled and tested simultaneously for the detection of the SARS-CoV-2 viral genome by RT-PCR on both NP swab and saliva samples. Saliva samples were collected in a preservative and inhibiting saline solution (Biofarma Srl). Internal method validation was performed to standardize the entire workflow for saliva samples.ResultsThe identification of SARS-CoV-2 conducted on saliva samples showed a clinical sensitivity of 95.1% and specificity of 97.8% compared to NP swabs. The positive predictive value (PPV) was 81% while the negative predictive value (NPV) was 99.5%. Test concordance was 97.6% (Cohen’s Kappa=0.86; 95% CI 0.81-0.91). The LoD of the test was 5 viral copies for both samples.ConclusionsRT-PCR assays conducted on a stored saliva sample achieved similar performance to those on NP swabs and this may provide a very effective tool for population screening and diagnosis. Collection of saliva in a stabilizing solution makes the test more convenient and widely available; furthermore, the denaturing properties of the solution reduce the infective risks belonging to sample manipulation.

RR:C19 Evidence Scale rating by reviewer:

  • Potentially informative. The main claims made are not strongly justified by the methods and data, but may yield some insight. The results and conclusions of the study may resemble those from the hypothetical ideal study, but there is substantial room for doubt. Decision-makers should consider this evidence only with a thorough understanding of its weaknesses, alongside other evidence and theory. Decision-makers should not consider this actionable, unless the weaknesses are clearly understood and there is other theory and evidence to further support it.

***************************************

Review: The manuscript by Bulfoni et al. is a validation study of an RT-PCR based diagnostics test for SARS-CoV-2 using saliva as specimen mixed into a proprietary saline solution. Their protocol involved extracting total RNA from preserved saliva using Qiagen kit and an automated extractor, followed by a 1-step RT-PCR using Roche LightCycler master mix and primers that target the envelope (E) gene of SARS-CoV-2, run on a Roche LightCycler 480 instrument. Paired saliva and NP swab samples were collected, processed, and analyzed from 1161 test subjects to establish sensitivity, specificity, and percent agreement of their protocol to the conventional NP swab approach. Their main finding was that saliva could be used as biological matrix for identification of SARS-CoV-2 infection. As the world continues to struggle to mitigate the spread of COVID-19, and access to vaccines is highly heterogenous, effective testing for COVID-19 is a topic of intense interest throughout the scientific community and general public worldwide. For an assay to be effective, it has to have the following features: convenient and non-invasive sample collection method that can be done by non-healthcare workers with minimal PPE or through self-collection, high sensitivity and specificity, hours (not days) turnaround time, low cost, lack of supply chain bottlenecks (most notably, no need for RNA isolation), and massive scalability. Several studies have already shown that saliva is comparable to NP swab as specimen for SARS-CoV-2 testing. In the US alone, the FDA has already given emergency use approval (EUA) to various RT-PCR based tests that use saliva as biological matrix (https://www.fda.gov/medicaldevices/coronavirus-disease-2019-covid-19-emergency-use-authorizations-medical-devices/in-vitrodiagnostics-euas-molecular-diagnostic-tests-sars-cov-2). All these tests have undergone thorough validation studies for their claims, and parameters such as collection, transport of sample, storage solutions, and processing have been clearly defined and backed with data in their respective instructions for use (IFUs) prior to FDA issuance of an EUA. Thus it is not fair to say in the manuscript that there are “no systematic studies are available yet, in particular concerning saliva collection and analysis validation based on both patients and asymptomatic or healthy groups.” As the authors have alluded to in their intro and discussion, not all saliva tests are the same, i.e. the sensitivity of RT-PCR using saliva samples varies depending on the collection technique (swabbing the tongue and/or cheek is different than collecting 1 mL of saliva drooled into a tube) as well as important differences in processing methods between different saliva tests. Hence, the statements addressing the scalability of their assay for mass testing should be clarified or addressed. Other major concerns: Table 3 showed that a positive concordance between saliva and NP swab was observed in 89 patients who showed symptoms at median 11 days prior to sample collection/testing. The total number of observations only add up to 152 out of the 156 patients admitted for COVID symptoms. Furthermore, according to their results in Table 3, out of the 152 samples tested, 109 tested positive in saliva, 94 tested positive in NP swabs, and 89 tested positive for both saliva and NP swab. However, their description in the paragraph after Table 3 says “Data regarding Ct values were available for 112/156 molecular tests based on saliva and for 71/156 NP swabs”, which does not match the numbers on Table 3 and makes it confusing for the readers. How would a sample be called positive if there were no Ct values measured? How come there were 94 positives in NP swabs but only 71 have available Ct values? The authors have to go back and double check their results. In addition, Figure 1 caption statement has to be modified to reflect the true number of paired Ct values that are compared in the plot. Same goes for Figure 2 and Table 4 description. Are there 89 or 71 samples that tested positive for both saliva and NP swab? They described their assay validation in the methods section, but no data could be found in the results section to support their claims on their assay LoD. While they did show that saliva collected in saline solution was superior to fresh saliva in terms of stability and outcome of assay by comparing 25 matched samples (saliva mixed with saline solution yielded 7 more positives compared to fresh saliva, Figure 3), it was not clear how many out of the 25 patient samples were true positives and true negatives. In addition, they described measuring the temporal stability of saliva and at different storage temperature (18-25 and 4C), and reported that “Our data indicates that viral RNA remains protected even at RT and at 4C up to 48h if collected in saline medium”, but the data backing up this claim could be found nowhere in the manuscript. Similarly, the entire section on “Validation of analytical procedure” has no data shown to back up their claims.

Minor concerns:

1) Their LoD has to be defined more clearly, instead of 5 copies per reaction, either convert this to 1 copy/uL (i.e. 5 copies per 5uL in each reaction) or 1000 copies/mL.

2) In their multiplex PCR assay, they described primers targeting the SARS-CoV-2 E gene, but did not mention any other targets for an internal control. Did they use primers for RnaseP, for example?

3) Because their RT-PCR based test only detects for the wild type E gene of SARS-CoV-2, the authors have to demonstrate that the primer recognition sites are not impacted by the presence of SARS-CoV-2 genetic variants in a patient sample. Adding another SARS-CoV-2 target to the multiplex PCR assay will help reduce the impact of a continuously mutating virus to this labdeveloped COVID-19 diagnostics test.

4) The final sentence in the discussion part lacks coherence and were not supported by data that they presented: “The test may also look for viral proteins to screen large numbers of asymptomatic people and in a future perspective it can be used also to characterize SARS-CoV-2 variants, which are considered more transmissible for their compartmentalization in saliva, their spread and high replication rate.


Comments
0
comment
No comments here
Why not start the discussion?