Review 1: "The infection fatality rate of COVID-19 inferred from seroprevalence data"

Timothy Hallett

doi:doi:10.1162/2e3983f5.b78d1b40

The infection fatality rate of COVID-19 inferred from seroprevalence data

by John Ioannidis

Show Description
dx.doi.org

Description

Objective To estimate the infection fatality rate of coronavirus disease 2019 (COVID-19) from data of seroprevalence studies. Methods Population studies with sample size of at least 500 and published as peer-reviewed papers or preprints as of July 11, 2020 were retrieved from PubMed, preprint servers, and communications with experts. Studies on blood donors were included, but studies on healthcare workers were excluded. The studies were assessed for design features and seroprevalence estimates. Infection fatality rate was estimated from each study dividing the number of COVID-19 deaths at a relevant time point by the number of estimated people infected in each relevant region. Correction was also attempted accounting for the types of antibodies assessed. Secondarily, results from national studies were also examined from preliminary press releases and reports whenever a country had no other data presented in full papers of preprints. Results 36 studies (43 estimates) were identified with usable data to enter into calculations and another 7 preliminary national estimates were also considered for a total of 50 estimates. Seroprevalence estimates ranged from 0.222% to 47%. Infection fatality rates ranged from 0.00% to 1.63% and corrected values ranged from 0.00% to 1.31%. Across 32 different locations, the median infection fatality rate was 0.27% (corrected 0.24%). Most studies were done in pandemic epicenters with high death tolls. Median corrected IFR was 0.10% in locations with COVID-19 population mortality rate less than the global average (<73 deaths per million as of July 12, 2020), 0.27% in locations with 73-500 COVID-19 deaths per million, and 0.90% in locations exceeding 500 COVID-19 deaths per million. Among people <70 years old, infection fatality rates ranged from 0.00% to 0.57% with median of 0.05% across the different locations (corrected median of 0.04%). Conclusions The infection fatality rate of COVID-19 can vary substantially across different locations and this may reflect differences in population age structure and case-mix of infected and deceased patients as well as multiple other factors. Estimates of infection fatality rates inferred from seroprevalence studies tend to be much lower than original speculations made in the early days of the pandemic.

RR:C19 Evidence Scale rating by reviewer:

Potentially informative. The main claims made are not strongly justified by the methods and data, but may yield some insight. The results and conclusions of the study may resemble those from the hypothetical ideal study, but there is substantial room for doubt. Decision-makers should consider this evidence only with a thorough understanding of its weaknesses, alongside other evidence and theory. Decision-makers should not consider this actionable, unless the weaknesses are clearly understood and there is other theory and evidence to further support it.

***************************************

Review:

The concept of estimating the Infection Fatality Ratio (IFR) by dividing reported deaths by an estimated of those ever infected through a seroprevalence estimate is, in principle, a good one. The IFR is an important concept and it is very difficult to estimate. The paper does a good service in collating available data and offering the computations. Many of the limitations are noted, and I agree with these. However, the limitation are so important that I do not believe that substantial conclusions can (yet) be drawn from this analysis. I do not believe that the evidence presented here supports such a strong statement as “… the fact that its IFR is typically more lower than originally feared”. I do believe that these data are useful and add to the emerging picture on IFR and that a continued collation of data will contribute to more firm conclusions being drawn.

The main limitations that I note are as follows:

1. Is it recognised that the IFR varies _very_ substantially by age. This (i) makes an average IFR not very meaningful per se, (ii) means that differences in the age-composition of populations and the pattern of spread of the epidemic substantially confound the analysis (to such an extent that it is not overcome by the basic <70 vs >70 years old split presented). This also means that summaries (e.g. the the median across a range of countries included in the analysis) potentially over-simplifies things.

2. There is little accounting for the variable quality of the data that go into the estimates. That is, some studies will be of very high quality and give a high likelihood of a good estimate of IFR, but others will not (due to factors like non-representative samples in the serosurvey or variable methods for counting deaths that can be attributed to COVID-19). It would be useful to show how indicators of quality (decided a priori) relate the resulting IFR estimate. Related to this point, it is interesting that estimates of IFR are higher in settings with higher death counts overall. This could be a real effect (less care available due to overburdened hospitals, or more infections among older people) but it could also be the result of a bias (e.g. countries with have more complete death reporting have commissioned more representative serosurveys). Without further information it is difficult to tease these apart.

3. The inclusion of data from blood donors does - as the author notes - potentially introduce the health-volunteer effect by which a lower prevalence of infection is measured in the sample, thereby increasing the apparent IFR.

4. The method for adjusting for the performance characteristics of the tests used is not well explained or justified. It seems to me that substantial uncertainty is introduced by this and it ought to be communicated in the analysis. Similarly, the correspondence of the cumulative deaths reported and the period of the survey is not always perfect or perfectly known but uncertainty within this may be important and this should also be reflected in the estimates of IFR.

5. The completeness of the reporting is not fully assured. I note that the reports used are drawn from a search of “PubMed (LitCOVID), medRxiv, bioRxiv, and Research Square”. Could this miss “grey literature”, official government reports etc (which may be among the most reliable reports)?

Review 1: "The infection fatality rate of COVID-19 inferred from seroprevalence data"

Connections