Review 1: "Prospective Study of Machine Learning for Identification of High-risk COVID-19 Patients"

Jason Hsu

doi:doi:10.1162/2e3983f5.485ea89e

Prospective study of machine learning for identification of high-risk COVID-19 patients

by Mateo Frausto-Avila, Roberto de J. León-Montiel, Mario A. Quiroz-Juárez, and Alfred B. U’Ren

Show Description
Published on Feb 23, 2024
www.medrxiv.org

Description

The Coronavirus Disease 2019 (COVID-19) pandemic constituted a public health crisis with a devastating effect in terms of its death toll and effects on the world economy. Notably, machine learning methods have played a pivotal role in devising novel technological solutions designed to tackle challenges brought forth by this pandemic. In particular, tools for the rapid identification of high-risk COVID-19 patients have been developed to aid in the effective allocation of hospital resources and for containing the spread of the virus. A comprehensive validation of such intelligent technological approaches is needed to ascertain their clinical utility; importantly, it may help develop future strategies for efficient patient classification to be used in future viral outbreaks. Here we present a prospective study to evaluate the performance of state-of-the-art machine-learning models proposed in PloS one 16, e0257234 (2021), which we developed for the identification of high-risk COVID-19 patients across four identified clinical stages. The model relies on artificial neural networks trained with historical patient data from Mexico. To assess their predictive capabilities across the six, registered, epidemiological waves of COVID-19 infection in Mexico, we measure the accuracy within each wave without retraining the neural networks. We then compare their performance against neural networks trained with cumulative historical data up to the end of each wave. Our findings indicate that models trained using early historical data exhibit strong predictive capabilities, which allows us to accurately identify high-risk patients in subsequent epidemiological waves—under clearly varying vaccination, prevalent viral strain, and medical treatment conditions. These results show that artificial intelligence-based methods for patient classification can be robust throughout an extended period characterized by constantly evolving conditions, and represent a potentially powerful tool for tackling future pandemic events, particularly for clinical outcome prediction of individual patients.

RR:C19 Evidence Scale rating by reviewer:

Potentially informative. The main claims made are not strongly justified by the methods and data, but may yield some insight. The results and conclusions of the study may resemble those from the hypothetical ideal study, but there is substantial room for doubt. Decision-makers should consider this evidence only with a thorough understanding of its weaknesses, alongside other evidence and theory. Decision-makers should not consider this actionable, unless the weaknesses are clearly understood and there is other theory and evidence to further support it.

***************************************

Review: This research assesses advanced machine learning models for identifying COVID-19 high-risk patients across various stages, utilizing artificial neural networks trained on historical data from Mexico. The findings reveal significant predictive accuracy over several COVID-19 waves, highlighting AI's potential as a crucial tool in patient classification, resource allocation, and improving decision-making for future health crises.

Comments are as follows:

More explicit descriptions of the machine learning algorithms and data preprocessing steps could enhance reproducibility.
Authors can enrich their discussion by referencing and engaging with previous literature on AI models for predicting severe outcomes or mortality in COVID-19 cases, to contextualize their findings within the broader research landscape.
Clarify how these AI tools can be integrated into current clinical protocols and the potential impact on patient management.
It would be beneficial for the authors to discuss how their models might perform in different healthcare systems or populations.
A more thorough discussion of the study's limitations, especially regarding the dataset's geographic and demographic scope, could provide a clearer context for the findings.

To summarize, insights from this study holds great potential for enhancing the distribution of healthcare resources and improving patient management during health crises.

Review 1: "Prospective Study of Machine Learning for Identification of High-risk COVID-19 Patients"

Connections