Review 2: "A 5-transcript Signature for Discriminating Viral and Bacterial Etiology in Pediatric Pneumonia"

RR\ID Evidence Scale rating by reviewer:

• Potentially informative. The main claims made are not strongly justified by the methods and data, but may yield some insight. The results and conclusions of the study may resemble those from the hypothetical ideal study, but there is substantial room for doubt. Decision-makers should consider this evidence only with a thorough understanding of its weaknesses, alongside other evidence and theory. Decision-makers should not consider this actionable, unless the weaknesses are clearly understood and there is other theory and evidence to further support it.

***************************************

Review: Pediatric pneumonia is often misdiagnosed due to complex clinical presentation and overlap between bacterial and viral etiologies, leading to incorrect or delayed treatments and excessive costs. A minimal 5-transcriptomic signature is able to accurately distinguish between bacterial and viral origin, as well as between bacterial pneumonia and controls, with high accuracy on an independent validation cohort. This signature is accurate across specific pathogens and disease severity, and may be an effective early diagnostic tool to improve pneumonia management.

This study affirms a compelling direction for transcriptomic analysis of pneumonia etiology. Some of the main claims are generally supported, although there are important caveats to the assessment of the 5-transcript signature’s performance and its comparison to the 2-transcript signature in Herberg et. al. Other claims are not strongly supported by the results and would benefit from clarification or revision.

1. Introduction:

   • The background and motivations for the study are thorough, but do not clearly indicate whether this study is focused on CAP, HAP, or both.

   • The three aims provided at the end of the section may be relevant, but only the third aim is described in the abstract and the article title.

2. Results:

   • Greater clarity on the selection of highly variable genes for PCA would better match the other clearly described criteria in this section. It is unclear how the selection of 100 most variable genes conveys the overall variation in expression for all 5,486 DEGs.

   • It is unclear why different fold change cutoffs are applied at different stages, especially when subsets defined by different cutoffs are compared (as with Wallihan et. al.)–this may be potentially misleading.

   • The process of discovering the n=36 candidate genes would benefit from greater clarity and streamlining–it is unclear in the text how or why the differential expression analysis moves from a comparison of pneumonia vs. healthy controls, to DB vs. DV, to the final subset. Are these analyses independent or successive subsets of one another?

   • It may benefit clarity to indicate how definitive vs. probable cases are defined, especially to support the claim of lower clinical confidence in PB + PV.

   • The derivation of the training, initial validation, and independent validation sets is unclear. The initial validation may be circular with the training set if derived from the same definitive cases, particularly if the probable cases alone perform poorly. The relationship between training and initial validation should be made clear to avoid misleading assessments of performance.

   • The co-expression analysis between pneumonia etiologies is compelling, but is not clearly related to the main goal of developing the 5-transcript signature.

3. Discussion:

   • The investigation of specific targets in relation to the co-expression analysis, including the significance of TLR signalling, exceeds the discussion of the signature and is not clearly relevant to the signature.

   • It may be misleading to compare the training AUC of 0.95 to the validation AUC of 0.88 for the 2-transcript signature. The reported AUC of 0.95 should be clearly indicated as achieved in training.

   • The claim that the 5-transcript signature demonstrates superior discriminant power compared with the 2-transcript model in Herberg et. al. should be carefully articulated. Herberg et. al. report higher training and validation AUC in an independent cohort (0.955 and 0.974 per the given citation). It is not clear that the 5-transcript signature generalizes more successfully.

   • The claim that the 5-transcript signature is effective independently of the causal pathogen responsible is not supported by the previous paragraph reporting poor performance on Mycoplasma pneumoniae and would benefit from revision