Review 3: "College campuses and COVID-19 mitigation: clinical and economic value"

RR:C19 Evidence Scale rating by reviewer:

• Reliable. The main study claims are generally justified by its methods and data. The results and conclusions are likely to be similar to the hypothetical ideal study. There are some minor caveats or limitations, but they would/do not change the major claims of the study. The study provides sufficient strength of evidence on its own that its main claims should be considered actionable, with some room for future revision.

***************************************

Review:

Losina et al present the results and discussion of a simulation modeling study investigating the effectiveness, cost, and cost-effectiveness of 24 different strategies for preventing COVID-19 transmission on college campuses. Campuses have emerged as important hot spots for COVID-19 transmission. The impact on students and faculty is clear. Further, many jurisdictions and school districts are concerned that campus hot spots could spill into the larger community if infected students are engaging with people and services off-campus. This paper employs simulation modeling to experiment with various control strategies and to estimate both effectiveness and health economic outcomes.

The analytic approach is to employ dynamic microsimulation. The authors employed the model to simulate 24 combinations of distancing, masking, isolation, and testing strategies to control COVID-19. The outcomes of the model include cases of COVID-19 among students and faculty, costs, and incremental cost-effectiveness ratios ($/quality-adjusted life-year gained).

For those who have become accustomed to reading modeling studies of COVID-19, it is important to note that this is not the traditional epidemiologic modeling approach that many investigators have employed to project COVID-19 trends. The approach to modeling transmission in this paper involves estimating a rate of transmission per-person per-contact hour for various groups of people (students, faculty, community), and then modeling how interventions change contact hours. Ultimately, the paper does not attempt to project college incidence over the long- or medium-term. Instead, the goal is to provide actionable data for immediate decision-making with a one semester time horizon (the urgent decision of the hour). Among simulation modelers, the best approach to modeling transmission is hotly debated. Any such debate, however, should be saved for an analysis over the longer term. The assumptions about transmission in this analysis likely have no effect on the policy conclusions of the work.

The results tables of the paper provide estimates for cases prevented by each strategy, as well as costs and cost-effectiveness. Several high-level important findings should inform ongoing discussion about college campuses, and perhaps even other settings:

First, masking and social distancing emerge as the most effective policies that should be the structural pillars of infection control. While there is a great deal of discussion about testing on campuses and how often we should be testing to ensure safety, it is clear from this paper that testing should be seen as an additional, supplementary intervention to perhaps layer on top of an effective masking and distancing policy. The implications are important – frequent testing does NOT free campus residents from masking and distancing requirements. It is not possible to test to safety and there is no effective policy on college campuses that avoids the need for masks and distance.

Second, the paper provides a very nice counterfactual estimate – how many cases of COVID-19 might there have been had a college decided to not re-open and to instead remain entirely remote. Importantly, that number is NOT zero. In fact, the counterfactual scenario of remote learning often has MORE cases of COVID-19 among students than re-opening school with a strong masking and distancing policy. Too often, there is cognitive bias in discussion about COVID-19, in which a given scenario is implicitly compared to a presumption of total safety should we remain at home. It is helpful to have this model demonstrate that remote is not always best or safest and to provide some estimate of how much community risk remains when schools are closed.

Third, there is no plan that prevents all COVID-19 on campus. Any school that is open now will eventually see COVID-19 among students and staff. That eventuality does not mean that colleges should remain closed (see above). It is essential that every institution have a clear plan for how they will react when cases appear campus and to not panic when that eventuality comes to pass.

Finally, the analysis makes clear that if we hope to implement more routine and frequent COVID-19 testing in the U.S. we need new technology. The incremental cost-effectiveness ratio of implementing routine testing even once every two weeks was over $800,000/QALY gained. In comparison, most things that we do in U.S. healthcare have incremental cost-effectiveness ratios in the $150,000 - $200,000/QALY range. Only when the cost of testing was reduced to $2 per test did routine longitudinal testing become cost-effective. Rather than use our resources to expand testing, we should instead invest those dollars to improving more effective prevention interventions – like masks and distancing.

In summary, this paper is an outstanding application of simulation modeling to make relevant policy conclusions and to provide an evidence base for decision making in near real-time. The methods are somewhat different than other simulation models in the literature, but they are appropriate for the decisions being made. For students of decision science, this work should become a role model for how to use simulation modeling to inform difficult decisions in uncertain times.