Interpretation
Women’s and newborn health research often involves the engagement of women, partners, and families in challenging circumstances. Care is required to ensure that core outcomes have relevance to the lived experience of patients and families involved, alongside the key scientific questions posed. As clinical trials do not always capture and / or publish these patient important outcomes, the use of systematic reviews alone cannot be solely relied upon to generate potential outcomes for the Delphi process.(100) Qualitative methods are therefore recommended to capture these outcomes and enhance research quality and the prospect of implementation success.(7, 14, 101-103) In our review only four of the identified studies used qualitative methods alongside systematic reviews of trial outcomes to explore patient and family perspectives when generating outcomes to consider. Despite this, the majority of studies included patients and representatives in the Delphi process. Only initiating involvement at this point, however, risks missing the relevance to the lived experience of participants, suggesting that researchers are uncertain how to optimise engagement so they can constructively contribute to the COS.(104) Funding limitations may also influence researchers use of interviews in COS development, as can be time and resource intensive.(103) Whilst the use of qualitative methods is strongly advocated, there is currently also minimal evidence to support any impact upon final core outcome selection. Research is required to explore this impact and provide clarity around the use of patient and public involvement in core outcome set development and encourage researchers to engage with this process.
Less than half of the studies identified in our review clarified stakeholder involvement in the core outcome development process.(14) We also observed that all consensus processes were undertaken in English, despite representation from international participants in the Delphi process. The involvement of global stakeholders, including patient and public as research partners, has potential implications for increased global participation and subsequent uptake in non-English speaking countries. The feasibility and of global acceptability of COS needs further exploration, however, to develop guidance for researchers which inform stakeholder selection and expectations regarding geographical representation.
Wide variation was also found in the number of outcomes selected for the Delphi process (15 - 263) the number of participants involved (24 - 412). and the resulting number of outcomes in the final core outcome set (6 - 48). The range of outcomes/statements entered into the consensus process may affect attrition; long and complex rounds may deter participation however reducing statements to minimise attrition may introduce bias to the study.(105, 106) Too large or small panel size may also result in smaller response rates or few participants in the final rounds; previous reviews of Delphi studies have shown that increasing panel size does not improve results .(106-109) Arguably a more effective approach is to consider balanced stakeholder representation to ensure all views are captured and considered.(110)
The majority of protocols did not address attrition bias or panel size. Published COS development studies shared little information regarding which stakeholder groups had the highest attrition levels or the effect of attrition bias.(111). Published standards for protocol development and COS reporting suggest attrition bias is discussed as a potential limitation where relevant, however clearer examples of the impact of attrition bias upon the degree of consensus and the reasons for participant withdrawal would be valuable.(13, 15)
The number of outcomes contributing to the final core outcome set will vary, which may impact on implementation. A balance has to be struck between larger outcome sets and the use of broad domains to summate areas, particularly when definitions may vary between populations or settings. There is also a risk that broad domains may be unhelpful in meta-analysis. The uptake of COS may provide insight into the impact of the wide variation in outcome set size on both clinical and research implementation. Where COS include a large number of outcomes or when outcome domains are included, further refinement is required to ensure they can be implemented within future research.
Whilst this review has not specifically addressed implementation or on-going clinical relevance, they are important considerations when examining COS success. Discussion on implementation was limited and no study discussed the assessment of on-going clinical relevance of the COS. These aspects could be explored through engagement with colleagues and routine re-examination of and citation in published protocols, randomised trials, systematic reviews and prospective registry records. Objectively demonstrating the uptake of COS in this way can quantify their contribution to improving the value of future research, and researchers should commit to supporting this implementation as part of the core outcome development process.