Interpretation
Women’s and newborn health research often involves the engagement of
women, partners, and families in challenging circumstances. Care is
required to ensure that core outcomes have relevance to the lived
experience of patients and families involved, alongside the key
scientific questions posed. As clinical trials do not always capture and
/ or publish these patient important outcomes, the use of systematic
reviews alone cannot be solely relied upon to generate potential
outcomes for the Delphi process.(100) Qualitative methods are therefore
recommended to capture these outcomes and enhance research quality and
the prospect of implementation success.(7, 14, 101-103) In our review
only four of the identified studies used qualitative methods alongside
systematic reviews of trial outcomes to explore patient and family
perspectives when generating outcomes to consider. Despite this, the
majority of studies included patients and representatives in the Delphi
process. Only initiating involvement at this point, however, risks
missing the relevance to the lived experience of participants,
suggesting that researchers are uncertain how to optimise engagement so
they can constructively contribute to the COS.(104) Funding limitations
may also influence researchers use of interviews in COS development, as
can be time and resource intensive.(103) Whilst the use of qualitative
methods is strongly advocated, there is currently also minimal evidence
to support any impact upon final core outcome selection. Research is
required to explore this impact and provide clarity around the use of
patient and public involvement in core outcome set development and
encourage researchers to engage with this process.
Less than half of the studies identified in our review clarified
stakeholder involvement in the core outcome development process.(14) We
also observed that all consensus processes were undertaken in English,
despite representation from international participants in the Delphi
process. The involvement of global stakeholders, including patient and
public as research partners, has potential implications for increased
global participation and subsequent uptake in non-English speaking
countries. The feasibility and of global acceptability of COS needs
further exploration, however, to develop guidance for researchers which
inform stakeholder selection and expectations regarding geographical
representation.
Wide variation was also found in the number of outcomes selected for the
Delphi process (15 - 263) the number of participants involved (24 -
412). and the resulting number of outcomes in the final core outcome set
(6 - 48). The range of outcomes/statements entered into the consensus
process may affect attrition; long and complex rounds may deter
participation however reducing statements to minimise attrition may
introduce bias to the study.(105, 106) Too large or small panel size may
also result in smaller response rates or few participants in the final
rounds; previous reviews of Delphi studies have shown that increasing
panel size does not improve results .(106-109) Arguably a more effective
approach is to consider balanced stakeholder representation to ensure
all views are captured and considered.(110)
The majority of protocols did not address attrition bias or panel size.
Published COS development studies shared little information regarding
which stakeholder groups had the highest attrition levels or the effect
of attrition bias.(111). Published standards for protocol development
and COS reporting suggest attrition bias is discussed as a potential
limitation where relevant, however clearer examples of the impact of
attrition bias upon the degree of consensus and the reasons for
participant withdrawal would be valuable.(13, 15)
The number of outcomes contributing to the final core outcome set will
vary, which may impact on implementation. A balance has to be struck
between larger outcome sets and the use of broad domains to summate
areas, particularly when definitions may vary between populations or
settings. There is also a risk that broad domains may be unhelpful in
meta-analysis. The uptake of COS may provide insight into the impact of
the wide variation in outcome set size on both clinical and research
implementation. Where COS include a large number of outcomes or when
outcome domains are included, further refinement is required to ensure
they can be implemented within future research.
Whilst this review has not specifically addressed implementation or
on-going clinical relevance, they are important considerations when
examining COS success. Discussion on implementation was limited and no
study discussed the assessment of on-going clinical relevance of the
COS. These aspects could be explored through engagement with colleagues
and routine re-examination of and citation in published protocols,
randomised trials, systematic reviews and prospective registry records.
Objectively demonstrating the uptake of COS in this way can quantify
their contribution to improving the value of future research, and
researchers should commit to supporting this implementation as part of
the core outcome development process.