RR:C19 Evidence Scale rating by reviewer:
Potentially informative. The main claims made are not strongly justified by the methods and data, but may yield some insight. The results and conclusions of the study may resemble those from the hypothetical ideal study, but there is substantial room for doubt. Decision-makers should consider this evidence only with a thorough understanding of its weaknesses, alongside other evidence and theory. Decision-makers should not consider this actionable, unless the weaknesses are clearly understood and there is other theory and evidence to further support it.
***************************************
Review:
The researchers tested for seropositivity (covid antibodies) in adjacent zip code (ZIPC) areas of Chicago that had officially very different case rates; the hypothesis was that relative seropositivity should reflect official case rates. In fact, the authors found that antibody rates were similar in the paired areas (even though case %s were rather different). This suggests that other factors other than underlying frequency of infection explained why more cases were detected in some areas than others.
The article undertakes several possible analyses that might help explain strong variations in case rates but not seropositivity. Not all of these alternative efforts are well documented (such as cluster analysis). They do, however, describe logistic regression to possibly explain variations, present simple box plots, and describe at length how the recruitment strategy tried to solicit representative samples of socio-economic groups in the adjacent ZIPC areas. There is good discussion of the validity of antibody assay methods. Several explanations for why seropositivity did not reflect case rates are discussed (an admittedly endless debate point).
The article’s conclusions would be better justified and could potentially move to a ‘Reliable’ rating if these aspects were improved:
Some more description (maybe just a 1-2 sentences each) of attempted cluster analysis methods & “weighting data to census characteristics” to show that these ways of handling the data did not explain observed seropositivity differences. Negative results are still results. Logistic regression is used which is valid … but reliant on just 5 observations. This limitation (such a very small dataset) could be better acknowledged.
The reliability of the findings depends on the quality of the antibody test data, and showing that the data had minimal bias (between group) in how they were collected. Would be ideal to document that there was not important chronological variation when responses came in or in case/infection rate growth in most areas during the monitoring period (this is partly described for ZIPC 60655). Show that the unrepresentative sampling was not clearly biased between areas; that comparable %’s of possible persons in each sub-group compared to base population was achieved in the pairs. This could be done well with a well-designed infographic. I suggest focus on race as the key factor (potential variation in recruitment %’s) that readers will be interested in. Authors acknowledged that African Americans were under-represented among respondents.
Are there differences from the census in the distribution of key (essential) workers in the paired areas? This would relate to suggested explanation 2 in the Discussion that persons in the lower case rate areas had lower doses and hence milder (more likely asymptomatic) infections. I wondered as I read this if socially marginalised groups had poor access to tests, and in fact, case rates in more racially/culturally diverse rates might tend to be under-counted (compared to more affluent ZIPC areas).
There are a number of small typos & Table 2 would benefit from repeat headers, & use of fewer raw numbers (%’s will suffice). % of persons age 65+ could be added to Table 2 to help see if higher case rate areas had more older adults in them (a possible explanation the authors discuss)
Creative Commons Attribution 4.0 International License (CC-BY 4.0)
