RR:C19 Evidence Scale rating by reviewer:
***************************************
Review:
BACKGROUND
The COVID-19 pandemic placed a tremendous strain on global healthcare systems as many hospitals at pandemic epicenters reached inpatient and intensive care unit (ICU) capacity despite considerable public health measures to “flatten the curve”. Multiple facets of medical care – from outpatient primary and urgent care to inpatient emergency and critical care – were faced with the task of triaging patients towards home quarantine, hospital admission, ICU escalation, or comfort-focused care. Especially in the early months of the pandemic, these decisions were being made with scarce data regarding the potential risks and benefits of these decisions and either limited or skewed data on the morbidity and mortality associated with SARS-CoV-2. In addition to guidance from the World Health Organization and national public health and medical agencies (such as the Centers for Disease Control and Prevention and National Institutes of Health in the United States of America and the National Health Service in the United Kingdom), healthcare providers likely relied on either their clinical intuition or risk stratification tools that were previously validated for pneumonia (CURB-651), sepsis (qSOFA2), or acute illness more generally (NEWS23,4). However, the utility of these tools for guiding decision-making for COVID-19 is unknown, and the present study by Bradley et al.5 aimed to assess the performance of these clinical scores and their component variables for predicting early death (all-cause mortality within 72 hours of admission) and both all-cause mortality and ICU admission within 30 days of admission.
MAIN FINDINGS
Among a prospective sample of 830 patients admitted to 7 hospitals in northwest England, death occurred in 7.6% of patients within 72 hours of admission and 36.1% within 30 days of admission, and ICU admission occurred in 15.0% of patients within 72 hours of admission and 17.1% within 30 days of admission. The performance of CURB-65, NEWS2, and qSOFA scores for prediction of 72-hour and 30-day mortality and 30-day ICU admission was measured using the area under the curve (AUC) of receiver operator characteristics (ROC) curves, sensitivity and specificity (Table 1). Overall, the performance of these pre-COVID-19 clinical risk scores at their traditional thresholds was only modest (AUC 0.56-0.78), being the least accurate in predicting ICU admission. The authors also developed new prognostic scores (CUCA-SF, CUCAF-SR, CAS-FC) from variables independently associated with outcomes in a backwards selection multivariable model (age, Rockwood clinical frailty score, respiratory rate, pulse oximetry, supplemental oxygen requirement, blood urea nitrogen, radiographic pulmonary consolidation, and C-reactive protein) that performed comparatively better (AUC 0.77-0.83).
Table 1. Summary of diagnostic performance of prognostic scores from Bradley et al. (2020)
| AUC | Sensitivity | Specificity |
CURB-65 (high risk: ≥2) |
|
|
|
72-hour mortality 30-day mortality ICU admission | 0.75 0.74 0.63 | 0.85 0.80 0.37 | 0.47 0.59 0.41 |
NEWS2 (high risk: ≥5) |
|
|
|
72-hour mortality 30-day mortality ICU admission | 0.78 0.67 0.65 | 0.92 0.83 0.82 | 0.31 0.37 0.32 |
qSOFA (high risk: ≥2) |
|
|
|
72-hour mortality 30-day mortality ICU admission | 0.66 0.62 0.56 | 0.42 0.27 0.11 | 0.84 0.87 0.80 |
CUCA-SF (high risk: ≥2) |
|
|
|
72-hour mortality | 0.82 | 0.98 | 0.13 |
CUCAF-SR (high risk: ≥2) |
|
|
|
30-day mortality | 0.77 | 0.98 | 0.14 |
CAS-FC (high risk: ≥2) |
|
|
|
ICU admission | 0.83 | 0.59 | 0.88 |
Abbreviations: AUC, area under the curve.
STUDY REVIEW
Strength of Evidence: Strong
Claims are very well-supported by the data and methods used. Decision-makers should consider the claims in this study actionable without reservation based on the methods and data.
The major claim made by the authors or the presented data is that extant risk stratification scores (CURB-65, NEWS2, qSOFA) offer insufficient predictive value for early clinical deterioration and mortality, and the authors suggest that novel prognostic scores should be created to more accurately guide clinical decision-making in patients with COVID-19. Recent studies have attempted to do just that and have demonstrated reasonable predictive value using measurements that would be readily accessible in clinical practice (Table 2), and the authors also developed predictive scores from clinical factors that were significantly associated with outcomes of interest in COVID-19 patients, which did perform slightly better than predictive tools not specific to COVID-19.
Table 2. Diagnostic performance of diagnostic scores from recently published reports
| AUC | Sensitivity | Specificity |
CALL score (high risk: ≥10)6 | 0.91 | 0.45 | 0.97 |
NOCOS calculator (high risk: ≥51.6%)7 | 0.82 | 0.89 | 0.54 |
qCSI score (high risk: ≥4)8 | 0.81 | 0.79 | 0.79 |
4C mortality score (high risk: ≥9)9 | 0.78 | 0.93 | 0.41 |
CALL score for predicting clinical progression included: comorbidities, age, lymphocyte count, and lactate dehydrogenase. NOCOS calculator for predicting 7-day survival included: serum blood urea nitrogen, age, absolute neutrophil count, red cell distribution width, oxygen saturation, and serum sodium. qCSI score for respiratory failure within 24 hours included: respiratory rate, minimum recorded pulse oximetry, and nasal cannula flow rate requirement. 4C mortality score for predicting in-hospital mortality included: age, sex, number of comorbidities, respiratory rate, pulse oximetry on room air, Glasgow coma scale, serum urea, and C-reactive protein. Abbreviations: AUC, area under the curve.
The cumulative evidence from these studies, and others reported in recent systematic reviews of clinical prediction models in COVID-19, suggest that age, quantifications of respiratory compromise (respiratory rate, pulse oximetry nadir, supplemental oxygen requirement, radiographic pulmonary consolidation), measurements of inflammation (C-reactive protein, lactate dehydrogenase, interleukin), and evidence of multiorgan involvement (creatinine and blood urea nitrogen, transaminases and bilirubin) are among the strongest predictors of mortality and clinical deterioration.10,11 What stands out from these studies is the absence of clinical biomarkers more specific to COVID-19 pathophysiology or more reflective of complications increasingly recognized with COVID-19, such as coagulopathy and neurological involvement (stroke, encephalopathy). Further research should be done, and is underway, to more accurately guide healthcare provider decisions regarding diagnosis, need for hospitalization, likelihood of clinical deterioration, and mortality in COVID-19 patients.
REFERENCES
1. Lim WS, van der Eerden MM, Laing R, et al. Defining community acquired pneumonia severity on presentation to hospital: an international derivation and validation study. Thorax. 2003;58(5):377-382.
2. Seymour CW, Liu VX, Iwashyna TJ, et al. Assessment of Clinical Criteria for Sepsis: For the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016;315(8):762-774.
3. Royal College of Physicians. National Early Warning Score (NEWS) 2: Standardising the assessment of acute-illness severity in the NHS. Updated report of a working party. London: 2017.
4. Pimentel MAF, Redfern OC, Gerry S, et al. A comparison of the ability of the National Early Warning Score and the National Early Warning Score 2 to identify patients at risk of in-hospital mortality: A multi-centre database study. Resuscitation. 2019;134:147-156.
5. Bradley P, Frost F, Tharmaratnam K, Collaborators N-C, Wootton DG. The utility of established prognostic scores in COVID-19 hospital admissions: a multi-centre prospective evaluation of CURB-65, NEWS2, and qSOFA. medRxiv. 2020.
6. Ji D, Zhang D, Xu J, et al. Prediction for Progression Risk in Patients with COVID-19 Pneumonia: The CALL Score. Clin Infect Dis. 2020.
7. Levy TJ, Richardson S, Coppa K, et al. Development and Validation of a Survival Calculator for Hospitalized Patients with COVID-19. medRxiv. 2020.
8. Haimovich A, Ravindra NG, Stoytchev S, et al. Development and validation of the quick COVID-19 severity index (qCSI): a prognostic tool for early clinical decompensation. Ann Emerg Med. 2020.
9. Knight SR, Ho A, Pius R, et al. Risk stratification of patients admitted to hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: development and validation of the 4C Mortality Score. medRxiv. 2020.
10. Gupta RK, Marks M, Samuels THA, et al. Systematic evaluation and external validation of 22 prognostic models among hospitalised adults with COVID-19: An observational cohort study. medRxiv. 2020.
11. Wynants L, Van Calster B, Collins GS, et al. Prediction models for diagnosis and prognosis of covid-19 infection: systematic review and critical appraisal. BMJ. 2020;369:m1328.
Creative Commons Attribution 4.0 International License (CC-BY 4.0)
