RR:C19 Evidence Scale rating by reviewer:
Reliable. The main study claims are generally justified by its methods and data. The results and conclusions are likely to be similar to the hypothetical ideal study. There are some minor caveats or limitations, but they would/do not change the major claims of the study. The study provides sufficient strength of evidence on its own that its main claims should be considered actionable, with some room for future revision.
“Flatten the curve” has been a widely-used appeal in public health communication since early on in the COVID-19 pandemic. In the paper “Misunderstanding ‘Flattening the Curve’”, Lalwani and colleagues argue that understanding this appeal “requires an understanding of which curve needs to be flattened.”
The authors convincingly show that most of their American participants did not understand the relationship between graphs of new daily cases and cumulative cases. At the same time, participants were highly confident in the accuracy of their (mis)conceptions. These findings replicate prior research showing that people have difficulty understanding the link between graphs of change and accumulation over time and extend it to the COVID-19 pandemic. The authors show that these difficulties indeed apply also to two graphs widely used to depict the development of the COVID-19 pandemic.
The authors further find that participants’ accuracy and their confidence in their judgments predicted support for stay-at-home policy measures aimed at mitigating the COVID-19 crisis. This suggests that understanding the link between daily and cumulative cases has implications for peoples’ acceptance of social distancing policies. The authors argue that a brief training on the link between daily and cumulative cases led to a modest improvement in participants’ understanding of daily and cumulative trends and to a short-term increase in support for stay-at-home policies.
We have two main reservations with this paper, which, for us, however, do not question that this is a well-conducted and valuable piece of work. The first reservation regards the comparison of the training group and the control group. One issue is that the training encompassed a number of elements: Participants were provided with a tutorial, were asked questions with incentives for correct answers, received feedback, and were explicitly told the difference between daily cases and cumulative cases graphs. To be sure, the choice of a training with several elements is understandable in work that aims to make a timely contribution to an ongoing pandemic. However, it makes it difficult to judge which elements of the training are relevant. More crucial in our view, however, is that participants who failed the comprehension check in the training group were excluded from analysis. Specifically, a sub-sample of people with a low probability to learn the message was removed from the training group but no such exclusion was done in the control group. We assume that the participants were excluded both from the pre- and the post measure. Therefore, this exclusion probably does not affects pre-post comparisons very much. It is possible, however, that the exclusion biases the comparison between training and control group and might lead to an overestimation of the training effect. For this reason, it would be a valuable addition to the manuscript if the results without the exclusion were also reported.
Our second reservation is that the authors conclude that the results imply that people do not understand the concept of “flattening the curve”. Yet, strictly speaking, this assumption is not tested directly and this conclusion is therefore somewhat premature. In addition, one needs to take into account that – as the authors correctly point out – the appeal to “flatten the curve” can and has been combined with both curves of daily new cases and cumulative cases. The authors apply the term of a “flattening” trend narrowly to curves with a zero slope and correctly deduce that flattening daily new cases in this way is not sufficient to prevent an overload of the health system. Public health messaging, however, has been using the term also to refer to an increasing and decreasing curve of daily new cases with a low (flat) maximum. We therefore consider the authors’ conclusion that Americans do not understand the concept of “flattening the curve” an assumption that still needs to be tested.
Additionally, one may mark that the sample was not representative for the American public which might limit the generalisability of the current findings. Non-representative samples are used in many psychological studies and also in this study we consider their use non-problematic for most conclusions. In the current case, however, they can become problematic if one expects that the – sometimes very surprising – numerical results (for example that 48 % percent of participants assign all graphs incorrectly) can be generalised to Americans.
Apart from these points, the study is well-designed and well-powered. The authors make a reliable case for the implications of miscomprehension of graphs on the acceptance of stay-at-home policies.