243 - Data Integrity Assessment of Clinical Trials in Neonatology for Individual Participant Data Meta-Analyses: A Case Study of Cord Management in Preterm Infants

Friday, April 28, 2023

5:15 PM – 7:15 PM ET

Poster Number: 243
Publication Number: 243.127

Kylie E. Hunter, NHMRC Clinical Trials Centre, University of Sydney, Camperdown, New South Wales, Australia; Nipun Shrestha, The University of Sydney, Sydney, New South Wales, Australia; Angie Barba, NHMRC Clinical Trials Centre, The University of Sydney, Camperdown, New South Wales, Australia; Mason Aberoumand, University of Sydney, Sydney, New South Wales, Australia; Sol Libesman, University of Sydney, Camperdown, Sydney, New South Wales, Australia; Jonathan G. Williams, NHMRC Clinical Trials Centre, University of Sydney, Camperdown, New South Wales, Australia; Jannik Aagerup, University of Sydney, Sydney, New South Wales, Australia; James X. Sotiropoulos, University of Sydney, Sydney, New South Wales, Australia; Ben W. Mol, Monash University, PRAHAN, Victoria, Australia; Rui Wang, Monash University, Melbourne, Victoria, Australia; Wentao Li, Monash University, Melbourne, Victoria, Australia; Lisa Askie, World Health Organization, Geneva, Geneve, Switzerland; Anna Lene Seidler, University of Sydney, Camperdown, New South Wales, Australia

Presenting Author(s)

Kylie E. Hunter, MPH BA (Hons) (she/her/hers)

Senior Evidence Analyst
NHMRC Clinical Trials Centre, University of Sydney
Camperdown, New South Wales, Australia

Background:

Concerns about the trustworthiness of published clinical trials are increasing, resulting in retractions or expressions of concern across various disciplines, including neonatology. Thus, it has been argued that we can no longer trust evidence based on published aggregate data alone, and checking individual participant data (IPD) is optimal to derive the most trustworthy systematic reviews and clinical guidelines. However, to date there is limited guidance on how to conduct integrity checks for datasets of completed studies.

Objective:

We aimed to develop a user-friendly IPD integrity tool, and test this in a large IPD meta-analysis evaluating umbilical cord management in preterm infants.

Design/Methods:

We conducted a literature review to collate and group existing data integrity items into key themes. Items were refined by discussion among an expert advisory group and included in a standardized integrity tool based on consensus. Where possible, checks were automated using R Markdown. We tested this tool in 61 neonatal trials with IPD and 43 trials with only published data available. Each trial was assessed by two independent reviewers. The collective pattern of potential integrity issues for each trial were discussed among our team and/or with trialists (as needed) to determine whether a trial should be included in our meta-analysis. Upon completion, we held an evaluation workshop to refine and improve our tool.

Results:

We developed an integrity tool comprising study level and IPD-specific items (Fig 1). Not all items are weighted equally. In our case study, 55 (90%) of the 61 trials contributing IPD had at least one potential integrity issue. Most were minor inconsistencies or genuine errors that were resolved via consultation. Three IPD trials were excluded from our evidence synthesis due to integrity concerns, including major discrepancies between IPD and published data and lack of association between variables known to be highly correlated, i.e. gestational age and birthweight (Fig 2). Nine trials with only published data were excluded due to multiple concerns regarding registration, randomization, ethics, non-communication and plausibility.

Conclusion(s):

We developed a tool to help researchers check, and then make decisions about trials that appear to have integrity issues. Our case study revealed a number of problematic trials in neonatology, highlighting the need for such comprehensive checks. Our tool can ensure that only highly trustworthy studies are included in meta-analyses, and subsequently inform clinical guidelines in pediatrics and neonatology.