RR:C19 Evidence Scale rating by reviewer:
Potentially informative. The main claims made are not strongly justified by the methods and data, but may yield some insight. The results and conclusions of the study may resemble those from the hypothetical ideal study, but there is substantial room for doubt. Decision-makers should consider this evidence only with a thorough understanding of its weaknesses, alongside other evidence and theory. Decision-makers should not consider this actionable, unless the weaknesses are clearly understood and there is other theory and evidence to further support it.
The major claim of the manuscript is that ondansetron use was found to be associated with lower COVID-19 mortality, especially in mechanically ventilated patients, in the analysis of the AdventHealth RECOVER-19 electronic health records (EHR) data. Such association was indeed observed in:
1) the discovered network of one of the 19 pre-defined patient subpopulations (specifically, the "inpatients age less than 60" group, which includes 1,328 people out of the total 16,277 PCR tested positive)
2) the follow-up multivariate regression analysis (focusing on the 3,082 inpatients and 25 selected predictors).
The manuscript is one of the two recent reports (the other is Bayat et al. 2021 on Open Forum Infectious Disease, Volume 8, Issue 7, ofab336) on the observed lower mortality with ondansetron use. While this provides valuable insight into a potential option of COVID-19 medication, causal implication and generalizability of the results cannot be strongly justified by the current findings.
The following are the major limitations of the current study. The first part of the analysis is Bayesian Network (BN) structure discovery for the whole cohort (16,277 PCRT tested positive patients) or the eighteen pre-defined subpopulations, using a proprietary software (bAIcis®) that implements a two-phase score-based BN method using Bayesian information criterion (BIC) scores. To interpret the findings as causal networks, the implicit assumptions include:
1) any unspecified distributional assumptions posited by the proprietary algorithm, for example, if continuous variables are assumed to be Gaussian;
2) that there exist no unobserved confounders;
3) that any variable in the network will be conditionally independent with all other variables that are not direct causes given all its direct causes (the so-called causal Markov condition)
4) that there must exist a DAG that the true distribution is faithful to. However, these assumptions are either untested or untestable in the study, and for instance the interaction between 1)-2) might lead to additional bias.
In addition, only one of the nineteen tested cohorts reports ondansetron as a significant parent of mortality, which leads to a question on reliability of the network findings. Other unselected candidate networks with high BIC-based scores or results from a state-of-the-art constraint-based BN method might be provided to further justify the current result. Lastly, the manuscript then focuses on Ondansetron in downstream analysis, which is solely based on the fact that it is among the only three significant predictors with negative association with mortality; other potential interventions might have been neglected such as preventing some of the manipulatable predictors with positive mortality association. The second part is a confirmative multivariate regression analysis. This part focuses on the 3,082 hospitalized patients. However, the analysis uses only 25 variables and their interactions or quadratic terms, which were selected based on ad hoc graph trimming and association testing of the discovered BN network. This further worsens the concern regarding unmeasured confounding for the probability of receiving ondansetron in the cohort, which is crucial regarding whether the current result resembles that of a hypothetical ideal random trial study. Moreover, only two out of the five imputed datasets selected ondansetron, and only one out of the five selected ondansetron: onVentilator. This is inconsistent with the high percentages of being selected for these two factors in the next round of multiple imputation, which again raises question in reliability of the result. For the task of estimating the ondansetron effect on mortality and its effect modification by ventilator usage, some other locally efficient estimation strategies might also be considered to further improve the chance of resembling the results of a hypothetical random trial while incorporating network estimation results, such as one-step estimator (Bickel et al., 1993), targeted maximum likelihood estimation (van der Laan and Rubin, 2006; Petersen et al., 2007), and double machine learning (Chernozhukov et al., 2018).
In summary, the manuscript highlights an important finding regarding the negative association of ondansetron use and COVID-19 mortality, which calls for further investigation in confirming the possible causal relationship. Meanwhile, in a recent meta-analysis by Tariq et al. (2020; on Mayo Clin Proc, 2020 Aug, 95(8):1632–1648), the overall mortality was estimated as 2.1% (95% CI [0.2%, 4.7%]) by pooling 42 COVID-19 studies, whereas the subgroup mortality in patients with gastrointestinal (GI) symptoms including vomiting was estimated as 0.4% (95% CI [0% to 1.1%]). It remains unclear whether it is the ondansetron use directly causing the lower mortality, or if there exists other undiscovered mechanisms for the lower mortality among patients with GI symptoms or ondansetron use. Despite the promising results, the potential confounding issue is yet to be alleviated. It also calls for further investigation on potential antivirus pathways of 5-HT3 receptor antagonists, as in vivo evidence has not been fully established.