Review 2: "Four Stylized Facts about COVID-19"

RR:C19 Evidence Scale rating by reviewer:

• Potentially informative. The main claims made are not strongly justified by the methods and data, but may yield some insight. The results and conclusions of the study may resemble those from the hypothetical ideal study, but there is substantial room for doubt. Decision-makers should consider this evidence only with a thorough understanding of its weaknesses, alongside other evidence and theory. Decision-makers should not consider this actionable unless the weaknesses are clearly understood and there is other theory and evidence to further support it.

***************************************

Review:

“Four Stylized Facts About COVID-19” analyzes the cross-country dynamics of the evolution of the COVID-19 epidemic, focusing in particular on the growth rate of daily COVID-19 deaths. It finds that, in all countries and regions considered, such growth rates (i) fell to zero within the 30 days following the record of the 25th cumulative death, and (ii) hovered around zero or below thereafter. The authors then use these findings to conclude that there exists a yet-to-be understood factor that naturally pushes the reproduction rate of COVID-19 downwards. They also argue that empirical studies analyzing the effects of government-mandated non-pharmaceutical interventions (NPIs) may overstate their effectiveness if they fail to account for this factor.

Although the authors employ sophisticated methods, I believe that the authors’ findings are not sufficient to substantially justify the claims they make regarding the reliability of empirical studies analyzing the effectiveness of NPIs. I organize my review around two main points.

1. Country sample and authors findings: the authors claim that they select all countries and U.S. states that had recorded at least 1,000 COVID-19 deaths by late July 2020 to carry out their study. The criteria used to select the countries covered seems arbitrary as it excludes small countries. Choosing a per capita deaths threshold would have been more appropriate. But this is not my main concern. Following their criteria, the authors select 23 countries and 25 U.S. states. I have followed COVID-19 news myself and reading that only 24 countries had recorded more than 1,000 deaths by late July 2020 felt odd. To verify this, I sourced daily data on COVID-19 deaths in all countries of the world from the dataset of the European Centre for Diseases Prevention and Control. The countries that had experienced at least 1,000 deaths by the end of July are almost twice as many (42) as the 24 selected by the authors.

I then calculated the growth rate of daily COVID-19 deaths since the 25th cumulative death for the countries that should be in the sample but are not included and found that for many it did not fall to 0 within the first 30 days from the 25th cumulative death nor it did hover at or below 0 after that. Figure 1 below depicts the growth rate of COVID-19 deaths in important countries such as Bolivia, Colombia, Indonesia, and South Africa, which the authors do not include in their study. For none of these countries, the growth rate of daily deaths stabilized at or below zero at any point in the period considered. I believe that the main findings of the authors would not be the same had they picked a more representative sample.

2. Authors conclusions from their findings: the authors conclude that, since all the countries that they considered experienced a decline in the growth rate of COVID-19 deaths, there must exist an intrinsic factor that naturally pushes downwards the reproduction rate of COVID-19 and that not accounting for this factor risks overstating the effectiveness of government-mandated NPIs in reducing the spread of COVID-19.

In my view, the fact that many (not all) countries experienced a decline in the COVID-19 reproduction rate is not enough to claim that there must exist a yet-to-be-understood factor that naturally pushes the rate of COVID-19 infections down. Most governments around the world (even in Sweden, which the authors often cite as an example of laissez-faire) have adopted some measures to reduce the number of COVID-19 infections. It is precisely these measures that may well have put downward pressures on the reproduction rate of COVID-19. Ignoring the intensity of government interventions to reduce infections and claiming that the downward pressure in the reproduction rate is the result of some other unobserved factor seems counterintuitive to me.

The authors also observe that the relaxation of NPIs in many countries over the summer of 2020 did not lead to another immediate surge of COVID-19 infections and deaths. The authors wrote their paper in August 2020. I believe that, had they waited a bit longer, they would not have made the same claim. Indeed, many countries that relaxed NPIs in the summer months experienced a surge in infections in the fall.

Since there is considerable variation in the intensity of government-mandated NPIs, a good way to analyze whether NPIs have any effect on COVID-19 reproduction rate is to check whether countries/regions in which the intensity of NPIs is higher have experienced faster declines. I myself have conducted one such analysis exploiting exogenous variation across local labor markets in the tightness of the government-mandated national lockdown in Italy and came to a very different conclusion from that of the authors (see Ciminelli and Garcia- Mandicó, 2020).

REFERENCES

Ciminelli, Gabriele and Garcia-Mandico, Silvia, Business Shutdowns and COVID-19 Mortality (2020). Available at SSRN: http://dx.doi.org/10.2139/ssrn.3683324