Review 1: "Virus Testing Optimisation Using Hadamard Pooling"

RR\ID Evidence Scale rating by reviewer:

• Potentially informative. The main claims made are not strongly justified by the methods and data, but may yield some insight. The results and conclusions of the study may resemble those from the hypothetical ideal study, but there is substantial room for doubt. Decision-makers should consider this evidence only with a thorough understanding of its weaknesses, alongside other evidence and theory. Decision-makers should not consider this actionable, unless the weaknesses are clearly understood and there is other theory and evidence to further support it.

***************************************

Review: The authors introduce a novel approach for setting up pooled sample testing using Hadamard's S matrices. They describe and provide helpful examples about the procedure for selecting rows for the reduced matrix, but they do not provide the code that they used without request. They do provide precalculated reduced matrix options for 7x7 to 27x27 Hadamard S matrices. When the number of positive samples is 1, their approach can efficiently detect the positive sample by matching the analysis vector with one of the unique columns of the reduced matrix. When more than 1 sample is positive there is a higher chance of an ambiguous outcome where the result does not correspond to any single column and individual testing is required for samples that could have resulted in the observed outcome vector. One challenge that the authors did not address is how robust the approach is to the failure of an assay, particularly as the size of the matrix grows and a correct outcome relies on a larger number of assays to work correctly. When using other approaches (e.g. hypercube pooling, multidimensional pooling), a single failed test will still generally point to a subset of samples that includes the positive one(s), and individual testing of this smaller subset will reveal the positive samples. But with this approach, a failed test could potentially result in a false positive identification or point to a set of samples that may not contain the positive sample. It would be important to estimate the error rate due to failed tests for each of the proposed matrices. The authors should discuss this and offer optimal methods to recover from such an outcome. The authors compare the number of tests per person for different numbers of samples and their approach is very efficient compared to some other methods. However, it would be helpful if they expanded on additional benefits/limitations of their method compared to the others presented. There are other considerations rather than minimizing the number of tests that can be compared. For example, the authors claim that their approach improves computational efficiency, but they did not provide any comparison or description of other methods. There is also not a comparison of how many rounds of testing is required to get to the final result for each method (slightly more tests may be tolerable if the answer can be achieved in one step).

Some Comments:

• Figure 1 caption says samples 1,2,3, and 5 go into Pool A, but shouldn’t it be samples 3, 5, 6, and 7?

• Change to: “In the case of pool D (highlighted), samples 2, 3, 4, and 7 would be grouped.”

• Second paragraph of Results and Discussion, second sentence refers to Figure 2, but I think it should be Figure 3. Also, it seems that 1+2, and 1+4 would result in the vector [0111], but 1+5 and 1+6 would result in [1011].