Review 1: "Four Stylized Facts about COVID-19"

RR:C19 Evidence Scale rating by reviewer:

• Reliable. The main study claims are generally justified by its methods and data. The results and conclusions are likely to be similar to the hypothetical ideal study. There are some minor caveats or limitations, but they would/do not change the major claims of the study. The study provides sufficient strength of evidence on its own that its main claims should be considered actionable, with some room for future revision.

***************************************

Review:

The article examines death data across 24 countries and 25 US states. The authors find that (1) death growth rates fell within about 30 days. (2) Following this period, on average, death growth rates hovered around zero in the countries they study. (3) Variation across countries decreased in the first ten days and remains relatively low onwards. (4) Use several SIR-type compartmental models to estimate that the effective reproduction number has remained around one after the first month of the epidemic. The authors conclude that these four stylized facts challenge the conclusion that government-mandated non-pharmaceutical interventions have been critical to reducing the spread of COVID-19 and associated deaths.

I think the article is interesting and provides a note of caution to overstating the importance of government-mandated non-pharmaceutical interventions. Most evaluations of non-pharmaceutical interventions are based on mathematical models –similar to the ones used by the authors–, and few studies have examined the effects of non-pharmaceutical using a causal framework. Therefore, it may be problematic to attribute reductions in transmission to a specific intervention (see, for example, Haushofer & Metcalf, Science 368, 1063-1065, 2020). This is an important message.

The authors conclude that their stylized facts challenge the conclusion that non-pharmaceutical policies and social distancing have been essential to reduce the spread of COVID-19. I think this is an overstatement of their findings and would encourage adding some nuance to this conclusion.

The authors note that most of the observed decline in the effective reproduction number and its variability across countries is probably explained by a decline in transmission (as opposed to herd immunity). They raise three hypotheses, which are well-accepted and increasingly documented in the epidemiological literature. The first hypothesis is that human behavior is essential to reduce transmission and that humans as spontaneously to avoid contagion. It is well-known that human behavior has a significant role in epidemics, including fear dynamics, mobility, economics (e.g., Vespignani, Funk, Fenichel, Chowell, Perrings, Epstein, etc.). Data on mobility in the current pandemic has shown that people have modified their behavior even before large-scale non-pharmaceutical interventions, such as stay-at-home policies, have been put in place by governments. I agree that this aspect is not considered nearly enough in epidemiological models. However, all the countries included in the authors’ analysis have imposed fairly strict mandatory non-pharmaceutical interventions, recommended social distancing, and the population in those countries has been exposed to intense situational awareness campaigns (e.g., through the government, mass media, academia, social networks, etc.). Would people have acted differently had governments taken no public health actions? They probably would, although there is no counterfactual to be sure. Further, the authors lump together a vast set of countries and note that on average, death growth rates decline quickly, and across country variation also declines. However, they consider no heterogeneities in the analysis to account, for example, for government-mandated policies that may affect social distancing or perceived risk differently.

The second hypothesis is that humans do not interact randomly (“homogeneous mixing” is a ubiquitous assumption in epidemiological compartmental models) but in networks. This surely affects the conclusions of studies that use compartmental models, but not necessarily the conclusions from statistical or phenomenological models. Several statistical models have also shown substantial effects of non-pharmaceutical policies. I agree with the authors that some may overstate the impact of a government-mandated non-pharmaceutical intervention, depending on study design. However, I am not convinced this possibility challenges the conclusion that government-mandated interventions have been essential.

The third hypothesis is unobserved factors, which I agree are always important in science and are always a potential source of bias.

Two additional comments: I think the inclusion criteria for countries and states is not clear. The authors state they included “U.S. states and other countries that have experienced more than 1000 cumulative deaths due to COVID-19 by July 22”, but many countries with >1000 deaths by July 22 are left out of the sample for no apparent reason (e.g., Bolivia, Ecuador, Colombia, Indonesia, Iraq, Guatemala, Pakistan, Romania, South Africa etc.). The results would probably be similar, but I think this needs clarification.

For their estimate of the effective reproduction numbers, the authors assume that mortality data can be directly compared to case incidence, which may be problematic. For example, mortality can be affected by population age structure (e.g., Japan compared to Peru) or by who is getting infected, improvements in clinical practice from experience with the disease or available treatments, or changes in testing among hospitalized patients when the epidemic is already established. I think this assumption needs further examination.