Review 1: "Evaluation of Four Interventions using Behavioural Economics Insights to Increase Demand for Voluntary Medical Male Circumcision in South Africa through the MoyaApp: A Quasi-Experimental Study"

RR:C19 Evidence Scale rating by reviewer:

• Potentially informative. The main claims made are not strongly justified by the methods and data, but may yield some insight. The results and conclusions of the study may resemble those from the hypothetical ideal study, but there is substantial room for doubt. Decision-makers should consider this evidence only with a thorough understanding of its weaknesses, alongside other evidence and theory. Decision-makers should not consider this actionable, unless the weaknesses are clearly understood and there is other theory and evidence to further support it.

***************************************

Review: This study tests the impact of different wording on forms on increasing demand for voluntary medical male circumcision among users of a phone application in South Africa. The authors find that a “foot-in the-door” approach that encourages users to take the first step in this process by booking an appointment for information increases submission of a contact form. However, those users were less likely to make a booking compared to users who submitted one of the other types of forms.

Given the health benefits of VMMC it is important to identify low-cost ways to increase demand for the procedures. While this study finds some interesting results in terms of how different wording might influence people’s interest in engaging with the intervention, I was confused by some aspects of the study and analysis which make it difficult to interpret the results.

1. First, it wasn’t clear what the purpose of the MoyaApp was. Who uses this app and why? This affects who is in the sample and how to generalize the results beyond this study.

2. Since the different forms were rotated over time, it seems likely that people could have viewed different versions of the form. How was this accounted for? For example, even if they submitted the “Reserved for You” form, they may have seen some of the other forms before that. It’s possible that the cumulative effect of viewing different forms is different from the effect of viewing one form only.

3. The authors note that more than a third of those who submitted a form were not contacted. It wasn’t clear to me what happened here, or what it has to do with submitting multiple forms. While this does not affect the primary outcome, given that it is such a large proportion, I think it needs to be clarified to understand the secondary outcomes.

4. The text of the forms are quite different in terms of what they are suggesting. Some suggest that submitting the form is equivalent to signing up to get a VMMC, while others suggest submitting a form are more about getting information. I’m not sure if these are apples-to-apples comparisons.

5. Lastly, I think it would be helpful to have an outcome that assesses bookings as a proportion of those who viewed the form so that the denominator is consistent. Otherwise, it is hard to assess the overall effect of the interventions: the foot-in-the-door increases form submissions, but has reduced bookings among those people: what is the overall effect?

The most important takeaway for the lay public is: circumcision can help reduce risk of HIV infection. The wording of flyers to encourage people to undergo voluntary medical care treatment could influence people’s interest in the procedure. However more research is needed to understand how this interest can be converted into increased circumcision rates.