Skip to main content
SearchLoginLogin or Signup

Review 2: "Prospective Study of Machine Learning for Identification of High-risk COVID-19 Patients"

Reviewers praised the models' effectiveness in identifying less severe cases but suggested improvements in language consistency, figure descriptions, and experimental clarity. They also recommended more precise explanations of technical details.

Published onJul 26, 2024
Review 2: "Prospective Study of Machine Learning for Identification of High-risk COVID-19 Patients"
1 of 2
key-enterThis Pub is a Review of
Prospective study of machine learning for identification of high-risk COVID-19 patients
Prospective study of machine learning for identification of high-risk COVID-19 patients
Description

The Coronavirus Disease 2019 (COVID-19) pandemic constituted a public health crisis with a devastating effect in terms of its death toll and effects on the world economy. Notably, machine learning methods have played a pivotal role in devising novel technological solutions designed to tackle challenges brought forth by this pandemic. In particular, tools for the rapid identification of high-risk COVID-19 patients have been developed to aid in the effective allocation of hospital resources and for containing the spread of the virus. A comprehensive validation of such intelligent technological approaches is needed to ascertain their clinical utility; importantly, it may help develop future strategies for efficient patient classification to be used in future viral outbreaks. Here we present a prospective study to evaluate the performance of state-of-the-art machine-learning models proposed in PloS one 16, e0257234 (2021), which we developed for the identification of high-risk COVID-19 patients across four identified clinical stages. The model relies on artificial neural networks trained with historical patient data from Mexico. To assess their predictive capabilities across the six, registered, epidemiological waves of COVID-19 infection in Mexico, we measure the accuracy within each wave without retraining the neural networks. We then compare their performance against neural networks trained with cumulative historical data up to the end of each wave. Our findings indicate that models trained using early historical data exhibit strong predictive capabilities, which allows us to accurately identify high-risk patients in subsequent epidemiological waves—under clearly varying vaccination, prevalent viral strain, and medical treatment conditions. These results show that artificial intelligence-based methods for patient classification can be robust throughout an extended period characterized by constantly evolving conditions, and represent a potentially powerful tool for tackling future pandemic events, particularly for clinical outcome prediction of individual patients.

RR:C19 Evidence Scale rating by reviewer:

  • Potentially informative. The main claims made are not strongly justified by the methods and data, but may yield some insight. The results and conclusions of the study may resemble those from the hypothetical ideal study, but there is substantial room for doubt. Decision-makers should consider this evidence only with a thorough understanding of its weaknesses, alongside other evidence and theory. Decision-makers should not consider this actionable, unless the weaknesses are clearly understood and there is other theory and evidence to further support it.

***************************************

Review: The paper discusses a study aimed at evaluating the performance of state-of-the-art machine learning models for identifying high-risk COVID-19 patients. 

These models were initially proposed in a previous study (PloS One, 2021) and have been trained on historical patient data from Mexico. 
The goal of the study is to validate these models to ensure their clinical utility,  which is crucial for efficient patient classification and resource allocation during the COVID-19 pandemic and future viral outbreaks.

The tests are interesting, and seems showing that, excluding stage 3/4 patients, the models trained on the first two stages seem robust with respect to time-shift.
However, I have some concerns:

  • Authors should report the experimental settings they used to train their models: did they use the same balanced setting they propose in [17] or they use a different setting? 

  • To provide reliable estimates showing the reliability of the model in the clinical practice and its translational results, it would be better to assess the models on a stratified setting where the positive/negative ratio visible in the input data is maintained; this is because we assume that the input data represents the real latent distribution behind the data.

  • Are the models trained/tested by using multiple holdouts, repeated cross-validation, or just one shot of training/test? 

  • How are the hyperparameter chosen (number of layers, activation functions, and so on)?

  • For each training phase, authors should provide a table with the exact number of patients used for training/test/validation 
    (I know the total amount of patients is in Figure 2 but still I would prefer to have a clear idea of the training/validation/test splits).

  • Besides auc, it would be nice to also have aucpr

  • Plot 4 should maintain the same y axis scale between figures to allow comparison between models trained on consecutive scales

  • Besides the time shift authors test with models trained on the whole data. Plus, for each time slot considered, they perform 4 experiments (considering four stages). Essentially, they perform several experiments. 

  • To improve clarity, conciseness, it would be nice to have a table, or picture, or schema to summarize them at the beginning of the 
    methods section.

Generally speaking, visualizations should be improved:

  • The study could benefit from a comparison with other existing models and approaches for identifying high-risk COVID-19 patients.

  • Although the paper highlights the need for further research, it lacks a concrete plan or suggestions for future studies to address the identified limitations.

Connections
1 of 3
Comments
3
AIShred GEP ECOTECH:

https://maps.google.com/url?q=https://mn.aishred.com

https://maps.google.ad/url?q=https://mn.aishred.com

https://maps.google.ae/url?q=https://mn.aishred.com

https://maps.google.com.af/url?q=https://mn.aishred.com

https://maps.google.com.ag/url?q=https://mn.aishred.com

https://maps.google.al/url?q=https://mn.aishred.com

https://maps.google.am/url?q=https://mn.aishred.com

https://maps.google.co.ao/url?q=https://mn.aishred.com

https://maps.google.com.ar/url?q=https://mn.aishred.com

https://maps.google.as/url?q=https://mn.aishred.com

https://maps.google.at/url?q=https://mn.aishred.com

https://maps.google.com.au/url?q=https://mn.aishred.com

https://maps.google.az/url?q=https://mn.aishred.com

https://maps.google.ba/url?q=https://mn.aishred.com

https://maps.google.com.bd/url?q=https://mn.aishred.com

https://maps.google.be/url?q=https://mn.aishred.com

https://maps.google.bf/url?q=https://mn.aishred.com

https://maps.google.bg/url?q=https://mn.aishred.com

https://maps.google.com.bh/url?q=https://mn.aishred.com

https://maps.google.bi/url?q=https://mn.aishred.com

https://maps.google.bj/url?q=https://mn.aishred.com

https://maps.google.com.bn/url?q=https://mn.aishred.com

https://maps.google.com.bo/url?q=https://mn.aishred.com

https://maps.google.com.br/url?q=https://mn.aishred.com

https://maps.google.bs/url?q=https://mn.aishred.com

https://maps.google.bt/url?q=https://mn.aishred.com

https://maps.google.co.bw/url?q=https://mn.aishred.com

https://maps.google.by/url?q=https://mn.aishred.com

https://maps.google.com.bz/url?q=https://mn.aishred.com

https://maps.google.ca/url?q=https://mn.aishred.com

https://maps.google.cd/url?q=https://mn.aishred.com

https://maps.google.cf/url?q=https://mn.aishred.com

https://maps.google.cg/url?q=https://mn.aishred.com

https://maps.google.ch/url?q=https://mn.aishred.com

https://maps.google.ci/url?q=https://mn.aishred.com

https://maps.google.co.ck/url?q=https://mn.aishred.com

https://maps.google.cl/url?q=https://mn.aishred.com

https://maps.google.cm/url?q=https://mn.aishred.com

https://maps.google.cn/url?q=https://mn.aishred.com

https://maps.google.com.co/url?q=https://mn.aishred.com

https://maps.google.co.cr/url?q=https://mn.aishred.com

https://maps.google.com.cu/url?q=https://mn.aishred.com

https://maps.google.cv/url?q=https://mn.aishred.com

https://maps.google.com.cy/url?q=https://mn.aishred.com

https://maps.google.cz/url?q=https://mn.aishred.com

https://maps.google.de/url?q=https://mn.aishred.com

https://maps.google.dj/url?q=https://mn.aishred.com

https://maps.google.dk/url?q=https://mn.aishred.com

https://maps.google.dm/url?q=https://mn.aishred.com

https://maps.google.com.do/url?q=https://mn.aishred.com

https://maps.google.dz/url?q=https://mn.aishred.com

https://maps.google.com.ec/url?q=https://mn.aishred.com

https://maps.google.ee/url?q=https://mn.aishred.com

https://maps.google.com.eg/url?q=https://mn.aishred.com

https://maps.google.es/url?q=https://mn.aishred.com

https://maps.google.com.et/url?q=https://mn.aishred.com

https://maps.google.fi/url?q=https://mn.aishred.com

https://maps.google.com.fj/url?q=https://mn.aishred.com

https://maps.google.fm/url?q=https://mn.aishred.com

https://maps.google.fr/url?q=https://mn.aishred.com

https://maps.google.ga/url?q=https://mn.aishred.com

https://maps.google.ge/url?q=https://mn.aishred.com

https://maps.google.gg/url?q=https://mn.aishred.com

https://maps.google.com.gh/url?q=https://mn.aishred.com

https://maps.google.com.gi/url?q=https://mn.aishred.com

https://maps.google.gl/url?q=https://mn.aishred.com

https://maps.google.gm/url?q=https://mn.aishred.com

https://maps.google.gr/url?q=https://mn.aishred.com

https://maps.google.com.gt/url?q=https://mn.aishred.com

https://maps.google.gy/url?q=https://mn.aishred.com

https://maps.google.com.hk/url?q=https://mn.aishred.com

https://maps.google.hn/url?q=https://mn.aishred.com

https://maps.google.hr/url?q=https://mn.aishred.com

https://maps.google.ht/url?q=https://mn.aishred.com

https://maps.google.hu/url?q=https://mn.aishred.com

https://maps.google.co.id/url?q=https://mn.aishred.com

https://maps.google.ie/url?q=https://mn.aishred.com

https://maps.google.co.il/url?q=https://mn.aishred.com

https://maps.google.im/url?q=https://mn.aishred.com

https://maps.google.co.in/url?q=https://mn.aishred.com

https://maps.google.iq/url?q=https://mn.aishred.com

https://maps.google.is/url?q=https://mn.aishred.com

https://maps.google.it/url?q=https://mn.aishred.com

https://maps.google.je/url?q=https://mn.aishred.com

https://maps.google.com.jm/url?q=https://mn.aishred.com

https://maps.google.jo/url?q=https://mn.aishred.com

https://maps.google.co.jp/url?q=https://mn.aishred.com

https://maps.google.co.ke/url?q=https://mn.aishred.com

https://maps.google.com.kh/url?q=https://mn.aishred.com

https://maps.google.ki/url?q=https://mn.aishred.com

https://maps.google.kg/url?q=https://mn.aishred.com

https://maps.google.co.kr/url?q=https://mn.aishred.com

https://maps.google.com.kw/url?q=https://mn.aishred.com

https://maps.google.kz/url?q=https://mn.aishred.com

https://maps.google.la/url?q=https://mn.aishred.com

https://maps.google.com.lb/url?q=https://mn.aishred.com

https://maps.google.li/url?q=https://mn.aishred.com

https://maps.google.lk/url?q=https://mn.aishred.com

https://maps.google.co.ls/url?q=https://mn.aishred.com

https://maps.google.lt/url?q=https://mn.aishred.com

https://maps.google.lu/url?q=https://mn.aishred.com

https://maps.google.lv/url?q=https://mn.aishred.com

https://maps.google.com.ly/url?q=https://mn.aishred.com

https://maps.google.co.ma/url?q=https://mn.aishred.com

https://maps.google.md/url?q=https://mn.aishred.com

https://maps.google.me/url?q=https://mn.aishred.com

https://maps.google.mg/url?q=https://mn.aishred.com

https://maps.google.mk/url?q=https://mn.aishred.com

https://maps.google.ml/url?q=https://mn.aishred.com

https://maps.google.com.mm/url?q=https://mn.aishred.com

https://maps.google.mn/url?q=https://mn.aishred.com

https://maps.google.com.mt/url?q=https://mn.aishred.com

https://maps.google.mu/url?q=https://mn.aishred.com

https://maps.google.mv/url?q=https://mn.aishred.com

https://maps.google.mw/url?q=https://mn.aishred.com

https://maps.google.com.mx/url?q=https://mn.aishred.com

https://maps.google.com.my/url?q=https://mn.aishred.com

https://maps.google.co.mz/url?q=https://mn.aishred.com

https://maps.google.com.na/url?q=https://mn.aishred.com

https://maps.google.com.ng/url?q=https://mn.aishred.com

https://maps.google.com.ni/url?q=https://mn.aishred.com

https://maps.google.ne/url?q=https://mn.aishred.com

https://maps.google.nl/url?q=https://mn.aishred.com

https://maps.google.no/url?q=https://mn.aishred.com

https://maps.google.com.np/url?q=https://mn.aishred.com

https://maps.google.nr/url?q=https://mn.aishred.com

https://maps.google.nu/url?q=https://mn.aishred.com

https://maps.google.co.nz/url?q=https://mn.aishred.com

https://maps.google.com.om/url?q=https://mn.aishred.com

https://maps.google.com.pa/url?q=https://mn.aishred.com

https://maps.google.com.pe/url?q=https://mn.aishred.com

https://maps.google.com.pg/url?q=https://mn.aishred.com

https://maps.google.com.ph/url?q=https://mn.aishred.com

https://maps.google.com.pk/url?q=https://mn.aishred.com

https://maps.google.pl/url?q=https://mn.aishred.com

https://maps.google.pn/url?q=https://mn.aishred.com

https://maps.google.com.pr/url?q=https://mn.aishred.com

https://maps.google.ps/url?q=https://mn.aishred.com

https://maps.google.pt/url?q=https://mn.aishred.com

https://maps.google.com.py/url?q=https://mn.aishred.com

https://maps.google.com.qa/url?q=https://mn.aishred.com

https://maps.google.ro/url?q=https://mn.aishred.com

https://maps.google.ru/url?q=https://mn.aishred.com

https://maps.google.rw/url?q=https://mn.aishred.com

https://maps.google.com.sa/url?q=https://mn.aishred.com

https://maps.google.com.sb/url?q=https://mn.aishred.com

https://maps.google.sc/url?q=https://mn.aishred.com

https://maps.google.se/url?q=https://mn.aishred.com

https://maps.google.com.sg/url?q=https://mn.aishred.com

https://maps.google.sh/url?q=https://mn.aishred.com

https://maps.google.si/url?q=https://mn.aishred.com

https://maps.google.sk/url?q=https://mn.aishred.com

https://maps.google.com.sl/url?q=https://mn.aishred.com

https://maps.google.sn/url?q=https://mn.aishred.com

https://maps.google.so/url?q=https://mn.aishred.com

https://maps.google.sm/url?q=https://mn.aishred.com

https://maps.google.sr/url?q=https://mn.aishred.com

https://maps.google.st/url?q=https://mn.aishred.com

https://maps.google.com.sv/url?q=https://mn.aishred.com

https://maps.google.td/url?q=https://mn.aishred.com

https://maps.google.tg/url?q=https://mn.aishred.com

https://maps.google.co.th/url?q=https://mn.aishred.com

https://maps.google.com.tj/url?q=https://mn.aishred.com

https://maps.google.tl/url?q=https://mn.aishred.com

https://maps.google.tm/url?q=https://mn.aishred.com

https://maps.google.tn/url?q=https://mn.aishred.com

https://maps.google.to/url?q=https://mn.aishred.com

https://maps.google.com.tr/url?q=https://mn.aishred.com

https://maps.google.tt/url?q=https://mn.aishred.com

https://maps.google.com.tw/url?q=https://mn.aishred.com

https://maps.google.co.tz/url?q=https://mn.aishred.com

https://maps.google.com.ua/url?q=https://mn.aishred.com

https://maps.google.co.ug/url?q=https://mn.aishred.com

https://maps.google.co.uk/url?q=https://mn.aishred.com

https://maps.google.com.uy/url?q=https://mn.aishred.com

https://maps.google.co.uz/url?q=https://mn.aishred.com

https://maps.google.com.vc/url?q=https://mn.aishred.com

https://maps.google.co.ve/url?q=https://mn.aishred.com

https://maps.google.co.vi/url?q=https://mn.aishred.com

https://maps.google.com.vn/url?q=https://mn.aishred.com

https://maps.google.vu/url?q=https://mn.aishred.com

https://maps.google.ws/url?q=https://mn.aishred.com

https://maps.google.rs/url?q=https://mn.aishred.com

https://maps.google.co.za/url?q=https://mn.aishred.com

https://maps.google.co.zm/url?q=https://mn.aishred.com

https://maps.google.co.zw/url?q=https://mn.aishred.com

https://maps.google.cat/url?q=https://mn.aishred.com

?
Alan Dewey:

The "Prospective Study of Machine Learning for Identification of High-risk COVID-19 Patients" uses machine learning to predict severe outcomes by analyzing clinical and demographic data. Early identification of high-risk patients can improve interventions and resource allocation. The study also supports the development of sss application online to streamline assessments for healthcare providers.