Review 2: "Promoting Public Health with Blunt Instruments: Evidence from Vaccine Mandates"

RR:C19 Evidence Scale rating by reviewer:

• Reliable. The main study claims are generally justified by its methods and data. The results and conclusions are likely to be similar to the hypothetical ideal study. There are some minor caveats or limitations, but they would/do not change the major claims of the study. The study provides sufficient strength of evidence on its own that its main claims should be considered actionable, with some room for future revision.

***************************************

Review: This paper uses a difference-in-difference model to evaluate the effects of state COVID-19 vaccination mandates among healthcare industry workers on their choices to work in the healthcare industry. Using monthly CPS data, the authors found that the probability of working in the healthcare industry decreased by 0.72 percentage points (or 6% in relative to the pre-treatment mean) following the vaccine mandate. The authors addressed an important question of how a specific health policy could have (unexpected) spillover effects on other outcomes. The authors have done a great job in executing their DD models and conducted very detailed checks to demonstrate the robustness of their results. Overall, I believe the authors’ conclusions are very well supported by the data and method used. The authors have done a great job in executing their DD model and conducted very detailed checks to demonstrate the robustness of their results. Overall, I believe the authors’ conclusions are very well supported by the data and method used.

However, one of my major concerns is whether there were other contemporaneous events happening at the same time as the vaccine mandate that could confound the DD estimates. I think there are a couple of things worth considering. First, one of the key reasons for many healthcare workers leaving the healthcare industry during the pandemic period was burnout. The time of the state vaccine mandate overlapped with a new round of COVID-19 cases/hospitalization surge during the second half of 2021. As more cases and hospitalizations might increase healthcare workers’ workload, they might be more likely to leave during this period. Many states that adopted vaccine mandates were also populated states, which might indicate much higher increases in COVID-19 cases and hospitalizations, requiring much more work. It is likely people left the healthcare industry in mandate states because of the burnout from the growing trajectory of COVID-19 cases. Second, there were also more widespread health worker strikes or protests in 2021, which might also trigger people to leave the healthcare industry. If these strikes were more likely to happen in the mandate states, the DD estimates could be confounded. Third, another way to think about this might be related to the changes in other job opportunities over time. Would it be possible that mandate states had growing opportunities for other jobs with better payment, work-life balance, and benefits than healthcare providers during the study period? I think having some discussion around these points could be very helpful.