Review 1: "Evaluating the Effect of Olfactory Training on Improving the Sense of Smell in Patients with COVID-19 with Olfactory Disorders: A Randomized Clinical Trial Study"

RR:C19 Evidence Scale rating by reviewer:

• Potentially informative. The main claims made are not strongly justified by the methods and data, but may yield some insight. The results and conclusions of the study may resemble those from the hypothetical ideal study, but there is substantial room for doubt. Decision-makers should consider this evidence only with a thorough understanding of its weaknesses, alongside other evidence and theory. Decision-makers should not consider this actionable, unless the weaknesses are clearly understood and there is other theory and evidence to further support it.

***************************************

Review:

This study evaluating an olfactory training program on improving smell in COVID-19 patients is potentially informative but requires overhaul. 

Chemosensory dysfunction is one of the unique symptoms of COVID-19. Starting from 2020, there have been numerous papers published, reporting the symptoms, the process of recovery, and the treatments. Current paper is another attempt to report the effects of olfactory training on recovery from COVID-19-induced olfactory dysfunction. The study was conducted in Iran and olfactory training was conducted for 6 weeks, and the participants  self-evaluated their senses before and after the olfactory training.

To publish research papers on COVID-19-induced olfactory dysfunction conducted in various countries may provide insights into the genetic and environmental differences of COVID-19 symptoms and it can be meaningful. However, unfortunately, I encountered multiple problems that made me think this paper is far from ready to be published.

First of all, let’s go through the reviewers’ guideline: “We ask you to grade that assessment using the RR\ID Strength of Evidence Scale and then to comment to defend your assessment with a written review”. I evaluate this paper to be a level of “Potentially informative” in the scale. The definition of “Potentially informative” level is: “The main claims made are not strongly justified by the methods and data, but may yield some insight”, which is what the paper is. As I wrote above, although there are papers on the effects of olfactory training, I consider that reports on its effects on COVID-19-induced olfactory dysfunction and reports from different countries would be informative and we are in need of them. And yet, the main claims are not justified by the way they conducted the study and the way data are presented.

There are several reasons that I consider their paper is at this level in the “Strength of evidence scale table”. First of all, I noticed the large difference in the average age of the participants in the intervention group (which is the olfactory training group) from that of the control group. The average ages of each group were 45.12 and 76.13, respectively. Olfactory sense declines in elderlies. If the average age is different this much, and as the average age of the control group was higher, there is a possibility that the lack of improvement in the control group was due to their higher average age. In other words, it is difficult to distinguish the effects of olfactory training and the influences of age because of this large difference in the average of their age in the two groups. Secondly, I am amazed by the large P values. Probability values cannot become larger than 1, and I was astonished to see the P values in Table 1 and Table 2, like for example, P= 884.0. Thirdly, I can see that the scores, which indicate the level of olfactory sense/dysfunction, of intervention group and control group are about 24 and 23 at the pre-training stage in Figure 2 and in the text of the paper it says 24.32 and 22.85, respectively, which matches Figure 2. What I don’t understand is the mean scores of both groups in Table 2, which are 32.24 and 85.22, respectively, at the before training stage, and 60.19 and 52.22 at the post-intervention stage. This does not match the text and Figure 2. I wonder how to understand these discrepancies in the numbers. Fourth question, which I actually noticed first, is the name of a chemical compound “citronol”. I thought it could be just a mistyping of citronellal, which is a major chemical constituent of the smell of lemon, and I have also thought if there is a possibility that citronellal is also called citronol, as some chemical compounds are called in multiple ways. I checked PubChem and I confirmed there is no citronol in PubChem. In a Google search, citronol retrieved results indicating that it is a name of a company that manufactures a super hand cleaner called citronol. I don’t know how to interpret this. It could be a simple typing mistake of citronellal, or it could mean some other chemical compound, and I am not sure if they used super hand cleaner.

As you can see, although I consider the goal is interesting, there are multiple fundamental problems in the paper. Thus, I selected “Potentially informative”, and the conclusion of the study is not substantiated by the methods and results presented.