Introduction: Plain language summaries (PLSs) make complex healthcare evidence accessible to patients and the public. Large language models (LLMs) may assist in generating accurate, readable PLSs. This study explored using the LLM Claude 2 to create PLSs of evidence reviews from the Agency for Healthcare Research and Quality (AHRQ) Effective Health Care Program. Methods: We selected 10 evidence reviews published from 2021-2023, representing a range of methods and topics. We iteratively developed a prompt to guide Claude 2 in creating PLSs which included specifications for plain language, reading level, length, organizational structure, active voice, and inclusive language. PLSs were assessed for adherence to prompt specifications, comprehensiveness, accuracy, readability, and cultural sensitivity. Results: All PLSs met the word count. We judged one PLS as fully comprehensive; 7 mostly comprehensive. We judged 2 PLSs as fully capturing the PICO elements; 5 with minor PICO errors. We judged 3 PLSs as accurately reporting the results; 4 with minor result errors. We judged 3 PLSs as having major result errors for incorrectly reporting total participants. Five PLSs met the target 6th-8th grade reading level. Passive voice use averaged 16%. All PLSs used inclusive language. Conclusions: LLMs show promise for assisting in PLS creation but likely require human input to ensure accuracy, comprehensiveness, and the appropriate nuances of interpretation. Iterative prompt refinement may improve results and address the needs of specific reviews and audiences. As text-only summaries, the AI-generated PLSs could not meet all consumer communication criteria, such as textual design and visual representations. Further testing should explore how to best leverage LLM support in drafting PLS text for complex evidence reviews.

Background: There are few randomised controlled trials (RCTs) in paediatric surgery, and the risk of bias of these studies is unknown. There is also little known about the methodological or reporting quality of systematic reviews and meta-analyses in paediatric surgery. We aimed to determine the risk of bias and reporting quality of recent RCTs, systematic reviews and meta-analyses in paediatric surgery, and the associations with study characteristics. Methods: We searched MEDLINE, Embase, Cochrane Library, JBI EBP Database, Centre for Reviews and Dissemination, Web of Science and high-impact journals for all RCTs, systematic reviews and meta-analyses in paediatric surgery published in 2021. We assessed the risk of bias and reporting quality of RCTs using the RoB 2 and CONSORT tools, respectively. We assessed the same parameters for systematic reviews and meta-analyses using the ROBIS and PRISMA tools. Findings: We identified 82 RCTs and 289 systematic reviews or meta-analyses in paediatric surgery published in 2021. More than half (n=46, 56%) of RCTs and almost all (n=278, 96%) systematic reviews and meta-analyses were at high risk of bias. Only one (1%) RCT and 4 (1%) systematic reviews and meta-analyses were adequately reported. We found that lower risk of bias and higher reporting quality in RCTs, systematic reviews, and meta-analyses were associated with the presence of a published protocol. For systematic reviews and meta-analyses, a higher PRISMA score was associated with being a Cochrane review. Surprisingly, we found that more than half of systematic reviews and meta-analyses (n=162, 56.1%) had no risk of bias assessments. Conclusions: Recently published RCTs, systematic reviews, and meta-analyses in paediatric surgery are at high risk of bias and have poor reporting quality. We suggest strategies for how trialists, systematic reviewers and other stakeholders across the research lifecycle can design, conduct and report higher quality research in paediatric surgery.

Introduction Methods guidance and appraisal tools for systematic reviews require specification of the question and eligibility criteria for the review (‘PICO for the review’). Less emphasis has been given to specifying the question and criteria for each synthesis (‘PICO for each synthesis’), yet decisions about which studies to include in each synthesis can critically influence the utility and findings of a review. This paper describes the rationale and methods for developing the InSynQ (Intervention Synthesis Questions) tool for planning and reporting synthesis questions in reviews of interventions. The aim is to provide transparency about the basis of the tool and contribute to evidence on methods for developing guidance for research conduct and reporting. Methods Informed by EQUATOR Network methods, we (1) established a project group; (2) examined reporting of the ‘PICO for each synthesis’ in published reviews; (3) reviewed existing reporting guidance and the Cochrane Handbook for Systematic Reviews of Interventions; (4) drafted items with elaboration, explanation and examples; (5) conducted consultation meetings, an evaluation survey and pilot test; (6) incorporated feedback. Results Existing reporting guidelines do not distinguish the review and synthesis PICO, with limited coverage of the elements needed to specify the PICO for each synthesis. Using the PRISMA 2020 format, the draft tool contained 10 items with essential and additional reporting elements, explanation and examples. Revisions arising from feedback (>30 people), included adding an eleventh item on consumer and stakeholder involvement, a figure explaining PICO for each synthesis, and integrating examples into elements/explanations. All respondents to the evaluation survey (12 people) said the tool would help them plan or appraise synthesis questions. InSynQ is available at https://InSynQ.info. Conclusion Transparent reporting of the development process contributes to the evidence-base for methods to develop guidance. It may improve uptake of InSynQ, in turn enhancing the clarity of syntheses.