RR:C19 Evidence Scale rating by reviewer:

• Reliable. The main study claims are generally justified by its methods and data. The results and conclusions are likely to be similar to the hypothetical ideal study. There are some minor caveats or limitations, but they would/do not change the major claims of the study. The study provides sufficient strength of evidence on its own that its main claims should be considered actionable, with some room for future revision.

***************************************

Review:

This preprint describes a study aiming “to contribute to current understanding about potential hesitancy towards a COVID-19 vaccine by evaluating demographic differences in US adults’ intentions to receive a potential coronavirus vaccine.” One good justification for exploring demographic differences is that “hesitancy could risk exacerbating existing disparities in COVID-19 cases and mortality—especially among African Americans, Latinos, and other racial-ethnic groups as well as lower SES groups” (p.4) – if it is differentially distributed. Likewise, as the author notes, other demographic variables such as age, cohabitation/conjugal status, employment status, and where one lives could make a difference, as may one’s political identity. Overall, in the population surveyed, only half intended to get vaccinated; 20% said no; 30% were not certain. If the demographic variables identified were predictive or more or of less ‘vaccine hesitance’, then targeted, tailored outreach would be justified. The investigator acknowledged, insightfully, that the ‘unsure’ group has too often been overlooked and merits the most scrutiny.

Beyond asking how demographics linked up to whether a person thought they would accept a COVID-19 vaccination, the author explored ‘motivations’ to do so. These included, for instance, ‘to protect my family’. As such, the options were too coarse-grained to be of much use in really understanding hesitancy. In addition, motivations for avoiding vaccination were not queried. This particular oversight was an artifact of the nature of the investigation, which drew on data already being collected for the Associated Press (AP)-NORC Center for Public Affairs Research poll - data which were likely not collected under the same mindset as would be data collected for an IRB-approved research study. This was not noted in the limitations section, nor were the poll’s methods elsewhere defended. There is no reason to assume on the face of it that the data were ‘bad.’ But given the options included in the present project, we can deem the ‘motivations’ options lacking in the kind of nuance that would give them utility.

The key findings of the paper were regarding demographic differences in intent to receive a SARS-C0V-2 vaccine. The RR:C19 project solicits reviews that ask “whether the overall conclusions and main claims should be taken seriously or not.” In this case, unless a statistician invited to review the paper identifies an error, they can – although linking demographic differences in yes/no/unsure answers to the motivations surveyed was somewhat of a leap. Unsure persons were not asked the reason-specific items. Recoding those items as if they were, and then answered ‘unsure’ seemed an unjustified stretch. The uncertainty could have had nothing to do with one of those five factors.

Notably, the kind of knowledge about target populations necessary for fruitful outreach is not the focus of this research, which asks only if targeted outreach may be needed. Nonetheless, the author did, in the end, ask how such outreach might be focused and some of the ideas, although speculative, could be useful, including the implicit suggestion at the end that early adoption may hinge on factors that can be leveraged, and the observation that context matters.

This preprint describes a study aiming “to contribute to current understanding about potential hesitancy towards a COVID-19 vaccine by evaluating demographic differences in US adults’ intentions to receive a potential coronavirus vaccine.” One good justification for exploring demographic differences is that “hesitancy could risk exacerbating existing disparities in COVID-19 cases and mortality—especially among African Americans, Latinos, and other racial-ethnic groups as well as lower SES groups” (p.4) – if it is differentially distributed. Likewise, as the author notes, other demographic variables such as age, cohabitation/conjugal status, employment status, and where one lives could make a difference, as may one’s political identity. Overall, in the population surveyed, only half intended to get vaccinated; 20% said no; 30% were not certain. If the demographic variables identified were predictive or more or of less ‘vaccine hesitance’, then targeted, tailored outreach would be justified. The investigator acknowledged, insightfully, that the ‘unsure’ group has too often been overlooked and merits the most scrutiny.

Beyond asking how demographics linked up to whether a person thought they would accept a COVID-19 vaccination, the author explored ‘motivations’ to do so. These included, for instance, ‘to protect my family’. As such, the options were too coarse grained to be of much use in really understanding hesitancy. In addition, motivations for avoiding vaccination were not queried. This particular oversight was an artifact of the nature of the investigation, which drew on data already being collected for the Associated Press (AP)-NORC Center for Public Affairs Research poll - data which were likely not collected under the same mindset as would be data collected for an IRB-approved research study. This was not noted in the limitations section, nor were the poll’s methods elsewhere defended. There is no reason to assume on the face of it that the data were ‘bad.’ But given the options included in the present project, we can deem the ‘motivations’ options lacking in the kind of nuance that would give them utility.

The key findings of the paper were regarding demographic differences in intent to receive a SARS-C0V-2 vaccine. The RR:C19 project solicits reviews that ask “whether the overall conclusions and main claims should be taken seriously or not.” In this case, unless a statistician invited to review the paper identifies an error, they can – although linking demographic differences in yes/no/unsure answers to the motivations surveyed was somewhat of a leap. Unsure persons were not asked the reason-specific items. Recoding those items as if they were, and then answered ‘unsure’ seemed an unjustified stretch. The uncertainty could have had nothing to do with one of those five factors.

Notably, the kind of knowledge about target populations necessary for fruitful outreach is not the focus of this research, which asks only if targeted outreach may be needed. Nonetheless, the author did, in the end, ask how such outreach might be focused and some of the ideas, although speculative, could be useful, including the implicit suggestion at the end that early adoption may hinge on factors that can be leveraged, and the observation that context matters.

Regardless, in terms of the question as to whether the demographic factors identified made a difference, the evidence to support the conclusion (yes) was, as per RR:C19 standards, “Reliable: The ‘main study claims are generally justified by its methods and data” and the findings are “actionable with limitations.” Again, barring objections from a statistician (which I am not), the paper could be published with revisions.
While one might argue ‘of course demographic factors make a difference; anyone who keeps up with the nightly news knows this: why investigate?’ it always is necessary to explore the evidence for our assumptions. If common sense were enough, we might all agree that the world is flat.

Regardless, in terms of the question as to whether the demographic factors identified made a difference, the evidence to support the conclusion (yes) was, as per RR:C19 standards, “Reliable: The ‘main study claims are generally justified by its methods and data” and the findings are “actionable with limitations.” Again, barring objections from a statistician (which I am not), the paper could be published with revisions.

While one might argue ‘of course demographic factors make a difference; anyone who keeps up with the nightly news knows this: why investigate?’ it always is necessary to explore the evidence for our assumptions. If common sense were enough, we might all agree that the world is flat.