Review 3: "Promoting Public Health with Blunt Instruments: Evidence from Vaccine Mandates"

RR:C19 Evidence Scale rating by reviewer:

• Strong. The main study claims are very well-justified by the data and analytic methods used. There is little room for doubt that the study produced has very similar results and conclusions as compared with the hypothetical ideal study. The study’s main claims should be considered conclusive and actionable without reservation.

***************************************

Review: Abouk et al. (2024) examine the impact of COVID-19 vaccination mandates for healthcare workers in the U.S., given pre-existing workforce shortages in the industry. They used data drawn from two primary sources – (i) the Current Population Survey (CPS) and (ii) the Centers for Disease Control and Prevention (CDC) and employed difference-in-differences (DiD) methods to analyze the effect of these mandates on employment. Their findings indicate that vaccine mandates may have exacerbated healthcare workforce shortages, with a notable 6% decline in the probability of working in healthcare and higher departure rates among healthcare-specific occupations. The study findings are well-founded using the data and the methodology employed.

The effectiveness of vaccine mandates in general and COVID-19 vaccine mandates in particular are still debatable. Proponents of the healthcare industry mandate contended that it was essential to reduce disease transmission within healthcare facilities, where patients are particularly susceptible to infections like COVID-19 and thus need safeguarding. Conversely, critics expressed worries about potential adverse effects on the economy, individual rights, and vaccine safety, among other issues (Canning et al., 2022). Abouk et al. (2024) provide a thorough quantitative analysis designed to address the question of how healthcare workers respond to the COVID-19 vaccination mandates in their employment decisions. The authors rightly pointed out that the literature on the response of healthcare workers to vaccine mandates is ex-ante ambiguous because some of the workers might perceive there would be a safe workplace and join the healthcare sector. In contrast, others might be vaccine-hesitant and prefer to stay out of the industry which will further propagate the prevailing shortages of healthcare workers.

Abouk et al. (2024) well exploited the DiD model starting from the conventional TWFE analysis to a newly expanding strand of DiD Literature such as event study analysis by de Chaisemartin and d’Haultfoeuille (2020). Using this model, they attempted to accommodate the non-absorbing and staggered nature of the mandate and address the issues related to parallel trends assumption in the DiD method.

A notable caveat of the paper that could have been addressed is the absence of an investigation into healthcare workers' employment responses to increases in COVID-19 cases, deaths, and hospitalization rates. Such circumstances might deter individuals from joining the healthcare sector due to increased responsibilities and long working hours. The paper would have been strengthened by investigating the existence of systematic differences between mandated and non-mandated states. Such analysis would be essential to disentangle the effects of mandates from these other influencing factors.

Abouk et al. (2024) contribute significantly to the quantitative methodology for causal identification of the impact of vaccine mandates through secondary data analysis. Their research serves to inform governmental policies and programs, highlighting the pragmatic application of their findings in shaping public health strategies.

Minor comments:

• While the paper explores the impact of COVID-19 vaccine mandates on healthcare workers, it does not address the implications of mandate removals. This omission leaves a gap in understanding the full picture of policy effects. We suggest the authors include an analysis or discussion on the impact of removing these mandates. Specifically, how might lifting mandates affect healthcare workforce shortages and public health outcomes?

• The paper indicates that COVID-19 vaccine mandates reduce the probability of working in the healthcare industry by 6%, with a higher impact on healthcare-specific occupations (7%) compared to non-healthcare occupations (5%). The authors could provide insights into the reasons behind the higher attrition rates among healthcare-specific occupations. Are these workers more vaccine-hesitant, or are there other underlying factors?

• The authors suggest that while COVID-19 vaccine mandates increased vaccination rates, they also exacerbated workforce shortages in healthcare, leading to negative implications for public health and patient care. This is a crucial point. The paper would benefit from a discussion of alternative strategies to increase vaccination uptake without impacting workforce shortages.

The most important takeaway is that state COVID-19 vaccine mandates for healthcare workers appear to have exacerbated workforce shortages in the healthcare industry, leading to a 6% decline in healthcare employment. This suggests that while these mandates aimed to increase safety and reduce disease transmission, they also had unintended consequence of worsening existing workforce shortages, particularly among healthcare-specific occupations, leading to significant employment declines and potential impacts on healthcare quality.