Review 2: "Short-term effect of the COVID-19 pandemic on suicidal ideation: A prospective cohort study"

RR:C19 Evidence Scale rating by reviewer:

• Potentially informative. The main claims made are not strongly justified by the methods and data, but may yield some insight. The results and conclusions of the study may resemble those from the hypothetical ideal study, but there is substantial room for doubt. Decision-makers should consider this evidence only with a thorough understanding of its weaknesses, alongside other evidence and theory. Decision-makers should not consider this actionable, unless the weaknesses are clearly understood and there is other theory and evidence to further support it.

***************************************

Review:

The pre-print paper “Short-term effect of the COVID-19 pandemic on suicidal ideation: A prospective cohort study” presented a cohort study of 6,683 individuals investigating the potential impact of COVID-19 pandemic on suicidal ideation in Japan. The research participants were members of an internet survey panel recruited by a Japanese survey company. The data for suicidal ideation and other covariates were collected through an online self-administrated questionnaire in two waves, with the first wave conducted on 24 January 2020 (T1; pre-pandemic) and the second in 27-30 April 2020 (T2; during the epidemic). At T1, a total of 125,011 individuals aged 20 years or above were sent the online questionnaire; the sample was selected according to the demographics (sex, age, and geographic area) of the Japanese population in the 2016 census. Of them 9,982 (8%) completed the questionnaire at T1, and 6,683 of the 9,982 (67%) completed the same questionnaire at T2. Suicidal ideation was evaluated using a 6-item scale developed by the first author previously. Changes in suicidal ideation scores between T1 and T2 were assessed using paired t-test. Factors associated with the suicidal ideation score at T2 were identified using multivariable linear regression. The authors concluded that suicidal ideation level was lower during the pandemic than the pre-pandemic time point (mean (?) score = 1.59 vs 1.71; t=5.87; p<0.001), albeit with a very small effect size (r=0.07). The following factors were found to be associated with suicidal ideation score at T2: being younger, unstable employment, no children, low-income, and receiving psychiatric care.

I view the main conclusion about slightly lower level of suicidal ideation during the COVID-19 pandemic as potentially informative. This claim could not be seen as reliable at this stage because a substantial proportion (33%) of the T1 sample was lost to follow-up at T2, and those who did not complete the T2 questionnaire could be more likely to have suicidal ideation than those who did. This was recognized by the authors as the main limitation of the study. Although they reported “no discernible difference in demographic characteristics between the T1 and T2 respondents,” they could have further examined any difference in other variables associated with suicidal ideation at T2 between those who did and did not participate the T2 survey. This would allow some assessment of the impact of attrition (drop-out) on the results.

Another concern with the study is the statistical approaches used. The suicidal ideation score is likely to be skewed and this makes the use of t-test and linear regression potentially problematic. The statistical evidence for very small difference in the suicidal ideation score between T1 and T2 appeared to be questionable due to the use of inappropriate statistical tests. How the effect size estimate (r=0.07) was derived also requires more detailed explanations.

Last, the survey study received no approval from any ethics review bodies. The authors gave the reason of no such review body at the first author’s institution. I wonder if the authors could have sought approval from the second author’s institute or other reviewing body in the region.