Review 1: "Effect of Informational Videos on Vaccine Perceptions Among Unvaccinated Individuals: A Randomized Control Trial"

RR:C19 Evidence Scale rating by reviewer:

Not informative. The flaws in the data and methods in this study are sufficiently serious that they do not substantially justify the claims made. It is not possible to say whether the results and conclusions would match that of the hypothetical ideal study. The study should not be considered as evidence by decision-makers.

***************************************

Review:

The authors state that they have developed and tested evidence-based videos that address the concerns of vaccine-hesitant individuals. They also state that the participants expressed significantly more positive attitudes towards COVID-19 vaccines, and this demonstrates the interventions public outreach potential. The authors state that their study shows high voluntary engagement with the intervention which provides justification for broader impact.

This preprint explores the impact of informational videos on vaccine intentions, vaccine-efficacy beliefs and concerns about side effects in a survey population in the US who were not vaccinated for the COVID-19 virus.  Vaccination was perceived by a segment of the population as harmful during the COVID-19 pandemic. Public health officials worldwide fear that this aversion of vaccines will carry over to other life-saving vaccinations, causing an uptick in previously controlled disease, eg measles, as well as seasonal flu etc. Thus, interventions that could be delivered remotely to recalcitrant individuals could have great public health significance, especially if the intervention was accepted by these individuals.

This study tested short informational videos, as well as if the videos were optional vs compulsory, among unvaccinated participants in an on-line randomized controlled trial with outcomes as stated above. Secondary outcomes included information-seeking behavior and trust in health care professionals.  The intervention groups were rather interesting.  The control group watched an unrelated placebo video; the first intervention group had the option of watching up to 4 informational videos.  The second intervention group was made to watch a vaccine-technology video while the other 3 videos were optional.  It would have been helpful to the reader to know how watching the video was made compulsory, e.g. was there no other option but to go to the video next screen in the flow?

The methods are a little confusing and are not fully described.  For example, the statement that the maximum number of unvaccinated US residents that could be recruited at the time of the study was estimated to be 7,000 is confusing.  In Figure 1, however, the consort diagram shows a different flow for recruitment of participants than what I stated in the text.  A total of 35,623 people were screened for eligibility and 11,228 were eligible and invited (31.5%).  Of those 72% (8,069) agreed and were randomized.  In addition, the paper cites those in each group who completed treatment as those who were randomized in each group – according to the consort diagram that was not the numbers that were allocated to each group by randomization. No description of the attrition is given and the video required group had twice the number who were out of the sample as the placebo group (16% vs 8%).  Allocation to the groups was on a 2:1:2 fashion – no explanation of that allocation is given.  Moreover, the authors state that the study was double-blinded, as participation was anonymous and “participants were unaware of ... parallel versions of the survey…”.  This description does not qualify this study as double-blinded.

According to the procedures described, participants were consented online, demographics were collected and then randomized into one of the three groups.  Videos were watched and then participants completed surveys collecting information on the primary and secondary outcomes. Analysis was then done comparing responses to the survey questions across groups – no adjustment for baseline intentions, beliefs, concerns, information-seeking behaviors, or trust in healthcare was done, as there were no baseline assessments conducted pre-intervention viewing.  There is no way that the authors could be sure that randomization erased all differences in these variables at baseline.  Thus, their conclusions that the interventions improved vaccine intentions and resulted in more positive attitudes is not substantiated by their methodology.

Their conclusions are not justified because of this fatal flaw.  The study did show, however, that people who had not been vaccinated for COVID-19 were amenable to viewing informational videos about the vaccine and that on-line delivery of these short informational videos are feasible.  A future study should be designed to assess pre-post differences in outcomes, then these interventions would be useful to public health agencies to address vaccine hesitancy.