Skip to main content
SearchLoginLogin or Signup

Review 1: "Sample Size Calculations for Variant Surveillance in the Presence of Biological and Systematic Biases"

Published onApr 25, 2023
Review 1: "Sample Size Calculations for Variant Surveillance in the Presence of Biological and Systematic Biases"
1 of 2
key-enterThis Pub is a Review of
Sample Size Calculations for Variant Surveillance in the Presence of Biological and Systematic Biases
Sample Size Calculations for Variant Surveillance in the Presence of Biological and Systematic Biases
Description

SUMMARY As demonstrated during the SARS-CoV-2 pandemic, detecting and tracking the emergence and spread of pathogen variants is an important component of monitoring infectious disease outbreaks. Pathogen genome sequencing has emerged as the primary tool for variant characterization, so it is important to consider the number of sequences needed when designing surveillance programs or studies, both to ensure accurate conclusions and to optimize use of limited resources. However, current approaches to calculating sample size for variant monitoring often do not account for the biological and logistical processes that can bias which infections are detected and which samples are ultimately selected for sequencing. In this manuscript, we introduce a framework that models the full process— including potential sources of bias—from infection detection to variant characterization, and we demonstrate how to use this framework to calculate appropriate sample sizes for sequencing-based surveillance studies. We consider both cross-sectional and continuous sampling, and we have implemented our method in a publicly available tool that allows users to estimate necessary sample sizes given a specific aim (e.g., variant detection or measuring variant prevalence) and sampling method. Our framework is designed to be easy to use, while also flexible enough to be adapted to other pathogens and surveillance scenarios.

RR:C19 Evidence Scale rating by reviewer:

  • Potentially informative. The main claims made are not strongly justified by the methods and data, but may yield some insight. The results and conclusions of the study may resemble those from the hypothetical ideal study, but there is substantial room for doubt. Decision-makers should consider this evidence only with a thorough understanding of its weaknesses, alongside other evidence and theory. Decision-makers should not consider this actionable, unless the weaknesses are clearly understood and there is other theory and evidence to further support it.

***************************************

Review:

As suggested by the title, the objective of this paper is to describe a method of calculation of sample size to (i) detect a variant, and (ii) estimate its prevalence, when the sample is biased for various reasons such as heterogeneity of disease severity. The method is extended to the setup where genomic surveillance is an ongoing process. The paper addresses an important question, but the proposed method has many limitations as acknowledged by the authors. These assumptions are so strong that the practical application of the method could be drastically restricted. There are certain other concerns also that make the results of the study doubtful.

Among several assumptions that their method makes, the assumption of homogeneous and representative sampling is understandable. But the model used in this paper for calculation of sample size requires several parameters such as the proportion of the variant in the population, the sensitivity of the test for that variant, and the probability that the detected infection meets the quality threshold for genomic study. These parameters are difficult to estimate in most practical setups. When all these are known or can be estimated, the exercise simply reduces to their appropriate multiplication, called the “coefficient of detection” in the study. The paper states that only the ratio of variant coefficients is necessary for sample size calculations and not the raw values, but the rationale of the concerned equation in the paper is not fully explained. A full and more clear explanation would have helped the reader to understand and use their method. The purpose of this equation is to calculate actual prevalence from the observed prevalence but the example they give calculates observed prevalence from the actual prevalence. It is not clear why the observed prevalence is needed when the actual prevalence is known.

In addition to the concerns mentioned above, some other aspects are not fully explained. The paper gives a method to calculate sample size to detect ‘at least one’ case of the variant of concern (VoC) whereas the sample size should be for detecting the first case (and not at least one case) because the first case is enough to detect a variant. Secondly, the sample size formula for estimating the prevalence is a Gaussian approximation to binomial, whereas, for an extremely low prevalence of a variant in this setup, Poisson may be a better approximation. Although Poisson too approximates Gaussian for sufficiently large n but without this intermediary step the formula is less credible because of the approximations involved and the requirement of an extremely large sample.

Overall, the authors have raised an important issue regarding the sample size needed to detect a variant and to estimate its prevalence but the formulas they advocate need more clarity to be of practical utility.

Comments
1
Vape Factory:

https://linevoom.line.me/post/1166176315561563552

https://coub.com/ecigator

https://sway.office.com/M5DCGkdFqRLzFQmb?ref=Link

https://www.homify.in/ideabooks/9128064/what-is-a-vape

https://unsplash.com/@vapecig

https://bookmarkswing.com/story13707746/vape-wholesale-ecig-manufacturer-factory

https://social.msdn.microsoft.com/Profile/Vape%20Factory

https://community.dynamics.com/members/vape-factory

https://community.powerbi.com/t5/user/viewprofilepage/user-id/475172

https://social.technet.microsoft.com/profile/vape%20factory/

https://band.us/band/88711848/post/1

https://band.us/band/88711848

https://www.gog.com/u/vapecig

https://gab.com/vapecig

https://gab.com/vapecig/posts/108900503280340362

https://vapecignew.blogspot.com/

https://vapecignew.blogspot.com/2022/08/people-who-are-addicted-to-electronic.html

https://vapecignew.blogspot.com/2022/11/what-is-electronic-cigarette.html

https://feedly.com/i/entry/fMe7/G3F2H0GZKZqc985OThfnRZ20w3IYeiFP7W28UA=_181aede9e10:102487c:37ab6f60

https://list.ly/i/8074989

https://www.pearltrees.com/matthewma

https://padlet.com/ecigatornew/vapecigarette

https://padlet.com/ecigatornew/vapecigarette/wish/2401482710

https://www.kdpcommunity.com/s/profile/0058V00000DRnLz?language=en_US

https://about.me/vapeecig/

https://www.instructables.com/member/ecigatornew/

https://www.deviantart.com/ecigator

https://www.deviantart.com/ecigator/art/History-Of-Electronic-Cigarette-428511854

https://www.minds.com/vapecig/

https://www.minds.com/groups/profile/1444201952551374863/feed

https://medium.com/@ecigatornew

https://vapecignew.livejournal.com/

https://vapecignew.livejournal.com/profile

https://vapecignew.livejournal.com/621.html

https://vapecignew.livejournal.com/503.html

https://vapecignew.livejournal.com/775.html

https://waoop.com/read-blog/14471

https://waoop.com/vaporcig

https://www.vsv7.com/read-blog/38579

https://www.vsv7.com/vaporecig

https://www.vsv7.com/post/92465_https-ecigator-com.html

https://www.myshareshow.com/read-blog/33658

https://www.myshareshow.com/vaporecig

https://www.myshareshow.com/post/79494_https-ecigator-com.html

http://networks786.ovh/read-blog/45206

http://networks786.ovh/vaporecig

http://networks786.ovh/post/92097_https-ecigator-com.html

https://somee.social/ecigator

https://somee.social/posts/680744

https://www.mycutiepies.com/vaporecig

https://www.mycutiepies.com/post/40084_https-ecigator-com.html

https://www.themetwallynetwork.com/vaporecig

https://www.themetwallynetwork.com/post/17532_https-ecigator-com.html

https://wanzani.com/vaporecig

https://wanzani.com/post/191015_https-ecigator-com.html

https://www.menagerie.media/read-blog/11384

https://www.menagerie.media/vaporecig

https://huduma.social/vapecig

https://huduma.social/posts/197902

https://gezelligkletsen.nl/read-blog/7879

https://gezelligkletsen.nl/vaporecig

https://gezelligkletsen.nl/post/16187_https-ecigator-com.html

https://theavtar.com/read-blog/8457

https://theavtar.com/vaporecig

https://theavtar.com/post/18490_https-ecigator-com.html

https://prosface.com/blogs/40695/People-who-are-addicted-to-electronic-cigarettes

https://prosface.com/pages/vapecignew

https://prosface.com/posts/40696

http://anonimsocial.r91601v6.beget.tech/blogs/1384/Can-minors-buy-e-cigarettes

http://anonimsocial.r91601v6.beget.tech/pages/vaporecig

http://anonimsocial.r91601v6.beget.tech/posts/1385

https://priti-nag.tribe.so/user/vape_factory

https://priti-nag.tribe.so/post/are-electronic-cigarettes-harmless-630e21d305f1aac90e1ebbb9

https://community.telecomdrive.com/user/vape_factory

https://community.telecomdrive.com/question/vape-electronic-cigarette-disposable-vape-pen-ecig-manufacturer-and-vape-fa--6310dc3446430e158a0a347c

https://everyonespace.tribe.so/user/vape_factory

https://everyonespace.tribe.so/post/how-to-choose-an-electronic-cigarette-6310ab15513b775575ef9b3e

https://www.vevioz.com/vaporecig

https://www.vevioz.com/post/275412_https-ecigator-com.html

https://hiqy.in/vaporecig

https://hiqy.in/post/44731

https://workplace.vidcloud.io/social/read-blog/92892

https://workplace.vidcloud.io/social/read-blog/92899

https://talkotive.com/read-blog/77487

https://talkotive.com/read-blog/77489

https://linkspreed.com/read-blog/138146

https://linkspreed.com/read-blog/138151

https://www.pittsburghtribune.org/read-blog/8929

https://www.pittsburghtribune.org/read-blog/8931

https://respeak.net/read-blog/49134

https://respeak.net/read-blog/49135

https://social.x-vendor.com/read-blog/65424

https://social.x-vendor.com/read-blog/65428

https://us.eurl.live/health/what-are-the-top-5-disposable-vapes

https://amazonsale.io/read-blog/5479

https://www.timessquarereporter.com/others/is-it-cheaper-to-buy-disposable-vapes-online

https://reviewsconsumerreports.net/business/who-will-ship-vape-products

https://www.pdf24x7.com/pdf/is-vape-worse-than-smoking

https://edu.pittsburghtribune.org/ebooks/what-s-the-best-brand-of-vape

https://www.nasseej.net/blogs/1415/How-to-Choose-an-Electronic-Cigarette

https://www.nasseej.net/posts/1416

https://www.hypebunch.com/vapecig

https://www.hypebunch.com/post/54248_https-ecigator-com-vape-ecig-health-buy-best.html

https://corosocial.com/vapecig

https://corosocial.com/read-blog/164827

https://corosocial.com/read-blog/164828

https://corosocial.com/read-blog/164829

https://www.lawcodev.com/vapecig

https://www.lawcodev.com/post/250494_ecigator-is-one-of-the-well-known-vape-brands-spun-off-from-giftsoar-technology.html

https://www.lawcodev.com/vaporecig

https://www.lawcodev.com/read-blog/236579

https://www.lawcodev.com/read-blog/236580

https://www.lawcodev.com/read-blog/236581

https://www.palpodia.com/vapecig

https://www.palpodia.com/post/62079_ecigator-is-one-of-the-well-known-vape-brands-spun-off-from-giftsoar-technology.html

https://www.palpodia.com/read-blog/18519

https://www.palpodia.com/read-blog/18520

https://www.palpodia.com/read-blog/18521

https://friend007.com/read-blog/36015

https://friend007.com/vapecig

https://friend007.com/post/193096_https-ecigator-com.html

https://www.bhimchat.com/vaporecig

https://www.bhimchat.com/vapecig

https://www.bhimchat.com/post/76334_ecigator-is-one-of-the-well-known-vape-brands-spun-off-from-giftsoar-technology.html

https://www.bhimchat.com/post/76336_ecigator-is-one-of-the-well-known-vape-brands-spun-off-from-giftsoar-technology.html

https://pharmatalk.org/read-blog/28528

https://pharmatalk.org/vapecig

https://pharmatalk.org/post/78393_https-ecigator-com.html

https://www.camfect.com/read-blog/15048

https://www.camfect.com/post/751902_https-ecigator-com.html

https://www.camfect.com/vapecig

https://community.tccwpg.com/vapecig

https://community.tccwpg.com/vaporecig

https://community.tccwpg.com/read-blog/147468

https://www.xaphyr.com/vapecig

https://www.xaphyr.com/pages/ecignew

https://www.xaphyr.com/blogs/180526/Types-of-Vape

https://www.xaphyr.com/blogs/228151/What-is-the-most-popular-vape-product

https://www.sholltna.com/read-blog/698

https://www.sholltna.com/post/1421_https-ecigator-com.html

https://www.sholltna.com/vapecig

https://www.emazoo.com/blogs/329733/How-to-Choose-an-Electronic-Cigarette

https://www.emazoo.com/vapecig

https://www.emazoo.com/posts/329734

https://www.addonface.com/vapecig

https://www.addonface.com/post/35148_https-ecigator-com.html

https://addandclick.com/read-blog/91440

https://addandclick.com/vapecig

https://addandclick.com/post/247273_https-ecigator-com.html

https://community.wongcw.com/vapecig

https://community.wongcw.com/posts/257048

https://community.wongcw.com/pages/ecignew

https://community.wongcw.com/blogs/257050/What-are-the-four-precautions-when-using-a-vape

https://community.wongcw.com/blogs/257051/How-to-use-a-vape-to-enjoy-it-more

https://community.wongcw.com/blogs/257053/Here-we-will-introduce-the-basic-usage-of-each-type

https://bumppy.com/tm/vapecig

https://bumppy.com/tm/vaporecig

https://www.rolonet.com/posts/177913

https://www.rolonet.com/vapecig

https://www.rolonet.com/pages/vaporcig

https://www.rolonet.com/blogs/177915/Vape-situation-in-Indonesia

https://www.rolonet.com/blogs/179932/Basic-Structure-of-Vape

https://www.as7abe.com/wall/blogs/post/12959

https://www.as7abe.com/wall/blogs/post/12960

https://www.as7abe.com/wall/blogs/post/12961

https://www.exoltech.ps/blogs/post/22823

https://www.exoltech.ps/blogs/post/22824

https://www.globhy.com/read-blog/7111

https://www.globhy.com/vapecig

https://www.globhy.com/post/11660_https-ecigator-com.html

https://community.sprintally.com/vapecig

https://community.sprintally.com/post/69761_ecigator-is-one-of-the-well-known-vape-brands-spun-off-from-giftsoar-technology.html

https://community.sprintally.com/read-blog/16050

https://community.sprintally.com/vaporecig

https://community.sprintally.com/post/69763_ecigator-is-one-of-the-well-known-vape-brands-spun-off-from-giftsoar-technology.html

http://socialnetwork.linkz.us/vapefactory

http://socialnetwork.linkz.us/posts/40163

http://socialnetwork.linkz.us/pages/vaporecig

http://socialnetwork.linkz.us/blogs/40165/What-are-the-four-precautions-when-using-a-vape

http://socialnetwork.linkz.us/blogs/40166/I-want-to-enjoy-vape-in-a-box

http://socialnetwork.linkz.us/blogs/40167/What-is-a-VAPE

https://wiwonder.com/vapecig

https://wiwonder.com/post/194507_https-ecigator-com.html

https://blacksnetwork.net/vapecig

https://blacksnetwork.net/post/64097_ecigator-is-one-of-the-well-known-vape-brands-spun-off-from-giftsoar-technology.html

https://blacksnetwork.net/vaporecig

https://www.buzzbii.com/vapecig

https://www.buzzbii.com/post/290480_https-ecigator-com.html

https://lifesspace.com/vapecig

https://lifesspace.com/post/144449_https-ecigator-com.html

https://www.unitymix.com/vapecig

https://www.unitymix.com/vaporecig

https://www.unitymix.com/post/71615_ecigator-is-one-of-the-well-known-vape-brands-spun-off-from-giftsoar-technology.html

https://hunch.lighthouseapp.com/users/1278184

https://fixholdandflip.lighthouseapp.com/projects/159773-dyebold/tickets/10-differences-between-e-cigarettes-and-real-cigarettes

https://fixholdandflip.lighthouseapp.com/projects/159773-dyebold/tickets/11-vape-factory

https://vape.lighthouseapp.com/projects/160007/quit-smoking-with-electronic-cigarettes

https://uconnect.ae/vapecig

https://uconnect.ae/post/32328_ecigator-is-one-of-the-well-known-vape-brands-spun-off-from-giftsoar-technology.html

https://uconnect.ae/vaporecig

https://degentevakana.com/users/view/41544

https://degentevakana.com/blogs/view/136969

https://degentevakana.com/blogs/view/136981

https://buddybio.com/vapecig

https://buddybio.com/vaporecig

https://buddybio.com/post/100459_ecigator-is-one-of-the-well-known-vape-brands-spun-off-from-giftsoar-technology.html

https://buddybio.com/read-blog/37380

https://social.kubo.chat/vapecig

https://social.kubo.chat/post/16392_ecigator-is-one-of-the-well-known-vape-brands-spun-off-from-giftsoar-technology.html

https://social.kubo.chat/vaporecig

https://social.kubo.chat/post/16393_ecigator-is-one-of-the-well-known-vape-brands-spun-off-from-giftsoar-technology.html

https://social.kubo.chat/read-blog/11286

https://social.kubo.chat/read-blog/12158

https://actsfile.com/read-blog/24019

https://actsfile.com/vapecig

https://actsfile.com/post/38704_https-ecigator-com.html

https://teachin.id/angelauy

https://teachin.id/posts/132759

https://teachin.id/blogs/132760/Let-s-get-the-basics-of-vape

http://www.barberlife.com/vapecig

http://www.barberlife.com/post/137727_ecigator-is-one-of-the-well-known-vape-brands-spun-off-from-giftsoar-technology.html

http://www.barberlife.com/vaporecig

http://www.barberlife.com/read-blog/96666

http://www.barberlife.com/read-blog/96667

http://www.barberlife.com/read-blog/96668

https://shapshare.com/read-blog/178392

https://shapshare.com/vapecig

https://shapshare.com/post/297463_https-ecigator-com.html

https://goodshape1.com/read-blog/39424

https://goodshape1.com/vapecig

https://goodshape1.com/post/92637_ecigator-is-one-of-the-well-known-vape-brands-spun-off-from-giftsoar-technology.html

https://goodshape1.com/vaporecig

https://facethai.net/vapecig

https://facethai.net/post/128834_https-ecigator-com.html

https://expressafrica.et/vapecig

https://expressafrica.et/post/121663_https-ecigator-com.html

https://demo.wowonder.com/vapecig

https://demo.wowonder.com/post/412797_ecigator-is-one-of-the-well-known-vape-brands-spun-off-from-giftsoar-technology.html

https://demo.wowonder.com/vaporecig

https://wo.barataa.com/vapecig

https://wo.barataa.com/read-blog/108702

https://wo.barataa.com/read-blog/108708

https://wo.barataa.com/read-blog/108714

https://www.cyberpinoy.net/vapecig

https://www.cyberpinoy.net/post/6090_ecigator-is-one-of-the-well-known-vape-brands-spun-off-from-giftsoar-technology.html

https://www.cyberpinoy.net/read-blog/5093

https://www.cyberpinoy.net/read-blog/5094

https://www.cyberpinoy.net/read-blog/5095

https://www.speakfreelee.com/vapecig

https://www.speakfreelee.com/post/36922_ecigator-is-one-of-the-well-known-vape-brands-spun-off-from-giftsoar-technology.html

https://www.merexpression.com/vapecig

https://www.merexpression.com/post/144782_ecigator-is-one-of-the-well-known-vape-brands-spun-off-from-giftsoar-technology.html

https://www.merexpression.com/read-blog/76898

https://www.merexpression.com/read-blog/76899

https://www.merexpression.com/read-blog/76900

https://socialmodem.com/1675956120339436_593

https://4african.com/vapecig

https://4african.com/read-blog/1073

https://4african.com/read-blog/1074

https://4african.com/read-blog/1075

https://www.chumsay.com/1676012290389810_5101

https://www.chumsay.com/post/17075_ecigator-is-one-of-the-well-known-vape-brands-spun-off-from-giftsoar-technology.html

https://advpr.net/vapecig

https://advpr.net/post/51203_ecigator-is-one-of-the-well-known-vape-brands-spun-off-from-giftsoar-technology.html

https://advpr.net/read-blog/49628

https://advpr.net/read-blog/49631

https://advpr.net/read-blog/49632

https://assignmento.org/vapecig

https://assignmento.org/post/7348_ecigator-is-one-of-the-well-known-vape-brands-spun-off-from-giftsoar-technology.html

https://assignmento.org/read-blog/1986

https://assignmento.org/read-blog/1987

https://assignmento.org/read-blog/1988

https://blue.et/1675959939817498_5954

https://blue.et/post/16729_ecigator-is-one-of-the-well-known-vape-brands-spun-off-from-giftsoar-technology.html

https://blue.et/read-blog/997

https://blue.et/read-blog/998

https://blue.et/read-blog/999

https://www.nhlfriends.com/vapecig

https://www.nhlfriends.com/post/21734_ecigator-is-one-of-the-well-known-vape-brands-spun-off-from-giftsoar-technology.html

https://www.nhlfriends.com/read-blog/6780

https://www.nhlfriends.com/read-blog/6781

https://www.nhlfriends.com/read-blog/6782

https://www.jointcorners.com/vapecig

https://www.jointcorners.com/post/14208_ecigator-is-one-of-the-well-known-vape-brands-spun-off-from-giftsoar-technology.html

https://www.jointcorners.com/read-blog/9767

https://www.jointcorners.com/read-blog/9768

https://www.jointcorners.com/read-blog/9769

https://kryza.network/vapecig

https://kryza.network/post/15411_ecigator-is-one-of-the-well-known-vape-brands-spun-off-from-giftsoar-technology.html

https://socialcreditu.com/vapecig

https://socialcreditu.com/post/172085_ecigator-is-one-of-the-well-known-vape-brands-spun-off-from-giftsoar-technology.html

https://www.truthsocialviet.com/vapecig

https://www.truthsocialviet.com/post/40731_ecigator-is-one-of-the-well-known-vape-brands-spun-off-from-giftsoar-technology.html

https://www.truthsocialviet.com/read-blog/28197

https://www.truthsocialviet.com/read-blog/28201

https://www.truthsocialviet.com/read-blog/28202

https://mysportsgo.com/profile/vapecig

https://mysportsgo.com/profile/vapecig/action_id/443287

https://mysportsgo.com/Blog/1326/vape

https://mysportsgo.com/Blog/1327/why-should-i-choose-a-disposable-vape-pen

https://mysportsgo.com/Blog/1328/do-disposable-vape-get-you-high

https://www.vkay.net/vapecig

https://www.vkay.net/post/2563_ecigator-is-one-of-the-well-known-vape-brands-spun-off-from-giftsoar-technology.html

https://www.vkay.net/read-blog/2299

https://www.vkay.net/read-blog/2300

https://www.vkay.net/read-blog/2301

https://www.noifias.it/vapecig

https://friendstrs.com/vapecig

https://friendstrs.com/post/8069_ecigator-is-one-of-the-well-known-vape-brands-spun-off-from-giftsoar-technology.html

https://www.voyage-to.me/vapecig

https://www.voyage-to.me/post/54154_ecigator-is-one-of-the-well-known-vape-brands-spun-off-from-giftsoar-technology.html

https://uchat.umaxx.tv/vapecig

https://uchat.umaxx.tv/post/64428_ecigator-is-one-of-the-well-known-vape-brands-spun-off-from-giftsoar-technology.html

https://uchat.umaxx.tv/read-blog/51945

https://uchat.umaxx.tv/read-blog/51946

https://uchat.umaxx.tv/read-blog/51947

https://cursedmetal.com/vapecig

https://cursedmetal.com/posts/72573

https://cursedmetal.com/pages/vapecig

https://cursedmetal.com/blogs/72574/How-to-Use-a-Vape

https://cursedmetal.com/blogs/77701/Quit-smoking-with-electronic-cigarettes

https://globuzzer.mn.co/members/12637002

https://globuzzer.mn.co/posts/26607803

https://stemfemmes.mn.co/members/15010520

https://stemfemmes.mn.co/posts/about-me-32307820

https://stemfemmes.mn.co/posts/is-vaping-harder-to-quit-than-smoking

https://topbazz.com/vapecig

https://topbazz.com/post/7037_ecigator-is-one-of-the-well-known-vape-brands-spun-off-from-giftsoar-technology.html

https://topbazz.com/read-blog/547

https://topbazz.com/read-blog/549

https://topbazz.com/read-blog/550

https://community.fyers.in/user/vape_factory

https://www.ekademia.pl/@ecigatornew/post/is-2-nicotine-a-lot-in-a-vape

https://www.ekademia.pl/@ecigatornew/blog

https://www.ekademia.pl/@ecigatornew/post/why-amazon-does-not-sell-vapes

https://www.ekademia.pl/@ecigatornew/post/how-much-should-a-beginner-vape

https://kbb-hub.mn.co/members/13924142

https://kbb-hub.mn.co/posts/about-me-29644806

https://kbb-hub.mn.co/posts/29644865

https://friendtalk.mn.co/members/13813267

https://friendtalk.mn.co/posts/what-is-a-vape

https://www.fuzia.com/article_detail/776046/10-surprising-benefits-of-using-disposable-vapes

https://uberant.com/article/1901623-the-best-disposable-vape-brands-to-try-in-2023/