Skip to main content
SearchLoginLogin or Signup

Review 1: "Hydroxychloroquine in the treatment of outpatients with mildly symptomatic COVID-19: A multi-center observational study"

This observational study is potentially informative. It suggests an association between outpatient hydroxychloroquine use decreased hospitalization rate. However, both RR:C19 reviewers expressed concern about the study design, variable definitions, and generalizability.

Published onOct 22, 2020
Review 1: "Hydroxychloroquine in the treatment of outpatients with mildly symptomatic COVID-19: A multi-center observational study"
1 of 2
key-enterThis Pub is a Review of
Hydroxychloroquine in the treatment of outpatients with mildly symptomatic COVID-19: A multi-center observational study

Background: Hydroxychloroquine has not been associated with improved survival among hospitalized COVID-19 patients in the majority of observational studies and similarly was not identified as an effective prophylaxis following exposure in a prospective randomized trial. We aimed to explore the role of hydroxychloroquine therapy in mildly symptomatic patients diagnosed in the outpatient setting. Methods: We examined the association between outpatient hydroxychloroquine exposure and the subsequent progression of disease among mildly symptomatic non-hospitalized patients with documented SARS-CoV-2 infection. The primary outcome assessed was requirement of hospitalization. Data was obtained from a retrospective review of electronic health records within a New Jersey USA multi-hospital network. We compared outcomes in patients who received hydroxychloroquine with those who did not applying a multivariable logistic model with propensity matching. Results: Among 1274 outpatients with documented SARS-CoV-2 infection 7.6% were prescribed hydroxychloroquine. In a 1067 patient propensity matched cohort, 21.6% with outpatient exposure to hydroxychloroquine were hospitalized, and 31.4% without exposure were hospitalized. In the primary multivariable logistic regression analysis with propensity matching there was an association between exposure to hydroxychloroquine and a decreased rate of hospitalization from COVID-19 (OR 0.53; 95% CI, 0.29, 0.95). Sensitivity analyses revealed similar associations. QTc prolongation events occurred in 2% of patients prescribed hydroxychloroquine with no reported arrhythmia events among those with data available. Conclusions: In this retrospective observational study of SARS-CoV-2 infected non-hospitalized patients hydroxychloroquine exposure was associated with a decreased rate of subsequent hospitalization. Additional exploration of hydroxychloroquine in this mildly symptomatic outpatient population is warranted.

RR:C19 Evidence Scale rating by reviewer:

  • Potentially informative. The main claims made are not strongly justified by the methods and data, but may yield some insight. The results and conclusions of the study may resemble those from the hypothetical ideal study, but there is substantial room for doubt. Decision-makers should consider this evidence only with a thorough understanding of its weaknesses, alongside other evidence and theory. Decision-makers should not consider this actionable, unless the weaknesses are clearly understood and there is other theory and evidence to further support it.



This preprint presents an observational study on the association between outpatient hydroxychloroquine exposure and hospitalization in patients with SARS-CoV-2 infection. Data from 1274 patients are analyzed using logistic regression analysis and Kaplan-Meier analysis, and the authors find a decreased rate of hospitalization from COVID-19 among the hydroxychloroquine exposed. I have a few comments on the preprint and on this finding.

Observational studies have several limitations, not least regarding the possibilities to achieve unbiased treatment effect estimates. In contrast to randomized trials, bias cannot be prevented in the study design by randomizing patients to treatment and control and blinding the physicians to given treatment. Instead, the unavoidable bias has to be adjusted for in the statistical analysis. This may seem as a reasonable alternative, but successfully identifying all factors that induce bias usually requires information that isn't available. As a consequence, the findings of observational studies are more uncertain than what is reflected by p-values and confidence intervals.

The authors’ analysis approach is to compensate for the bias caused by the non-random treatment allocation using propensity score matching. This compensation seems to be adequately performed technically, and its weaknesses are clearly discussed. However, two further sources of bias may need to be considered when interpreting the presented findings.

First, the primary outcome measure is need for hospitalization. It is not clearly described how the need for hospitalization was defined or has been assessed, but the decision to hospitalize a patient seems more subjective than well-defined outcomes such as death, cancer diagnosis or myocardial infarction. In a well-designed randomized trial (with masking of treatment allocation), the decision to hospitalize patients would ideally have been independent of the given treatment. The design of this observational study does not prevent the physicians' expectations regarding the effect of hydroxychloroquine treatment to influence their decisions to hospitalize patients. The authors have also not made any attempt to adjust for this possible bias. It is unclear if it can be evaluated.

Second, the authors measure the estimated effects of hydroxychloroquine treatment on the risk of hospitalization using odds ratios. The odds ratio is a technically correct measure, and it is often useful for communicating the clinical significance of a finding. However, one interpretational problem with the odds ratio is that it exaggerates the relative risk, and the exaggeration may, depending on baseline risk, be substantial. In this study, 30.6% of the cohort required hospital care, and this implies that the presented finding (OR 0.53, 95% CI, 0.29, 0.95) is not directly interpretable as the relative risk of hospitalization after hydroxychloroquine treatment. It could have been a better approach to use a statistical method that directly estimates the relative risk (see e.g. Zhang J, Yu KF. What’s the Relative Risk? A Method of Correcting the Odds Ratio in Cohort Studies of Common Outcomes. JAMA 1998;280:1690-1691).

No comments here
Why not start the discussion?