The future: potential ways to deal with outliers
Quality assurance is a constant process incumbent on individual
clinicians and institutions to maintain practice within acceptable
standards. Therefore when negative outliers appear it is logical that
“bringing them back” would involve similar active processes (14). As
part of a new paradigm, we propose the following:
- Novel outcomes The quality performance equilibrium is in
flux. As newer technology for clinical diagnosis and treatment is
constantly emerging, so too should our methods of quality assurance.
As such, the definitions outliers must be adaptable and relevant.
Outlier detection should not rely on mortality alone, which is a poor
discriminator in quality of care. Further endpoints are required that
are automatically collated or patient reported, which may include
quality of life measures, patient satisfaction, imaging outcomes and
biomarkers. As an example, analysing post-surgical patient quality of
life and satisfaction in a well-run institution may highlight its
positive performance, compared to focusing on an unexpected
in-hospital mortality it recently suffered. As well as novel outcomes,
robust statistical methods that accurately and appropriately classify
outliers are required, using both supervised classification strategies
and proximity-detection methods. Differentiating between binary and
continuous indicators, logistic regression and propensity adjusted
scoring and their influences on data hierarchy should be
sought(15,16). Statistical methods need to go through cycles of
adaptation as outcomes changes and patient populations evolve.
- Positive outliers The emergence of a positive outlier must
not only prompt reward, but attempt to shift the equilibrium of
clinical performance in the wider field. Perhaps this excellent
performance arose from a novel clinical method or management process.
With the GMC evolving into a weapon used by healthcare leaders to
force outstanding competency and expose negative outliers, there is
little being done to reward the excellent. Such mechanisms are
psychologically straining for surgeons who are already in a high-risk
field and stand to gain very little if they demonstrate good outcomes.
Leaders of the NHS on a national, and even a local and departmental
level, can evoke more permanent change by encouraging and nurturing
those who excel as positive outliers. Individuals and institutions
that can go on to encourage others to follow suit(17).
- Negative outliers Focusing on negative outcomes alone in
identifying and then improving underperformers is not the optimum
method(6). In fact, newer studies are suggesting that outlier
detection using outcomes analysis may well be avoided, and the
subsequent knee-jerk reward-punishment strategies imposed to bring
back outliers, can be harmful, especially when involving blame(7,18).
In a recent national survey(13), 58% of UK cardiothoracic surgeons
opposed reporting of surgeon-specific mortality. Studies highlight an
increase in risk-averse and loss-averse behaviour from surgeons who
fear taking on high-risk cases, especially when there is no reward for
taking on such cases yet a high punishment when death does occur. The
result of such measures has seen more risk-averse behaviour by
surgeons wanting to avoid operating on high-risk patients (the
patients who arguably need the most attention) to circumvent poorer
outcomes (13). A fairer and less punitive attitude with negative
outliers is therefore essential: this is especially true if a large
body of social science purports that outlier occurrence is indeed
random and unpredictable..
- Leadership The need for strong leadership and management is
paramount(19,20). Rescuing an outlier requires the unit to challenge
the status quo and make significant changes in a timely manner to
return to the accepted mean. This would rely on an executive with good
management capabilities, effective communication and a departmental
culture conducive to adaptation and change(14). The Bristol Enquiry
(21) found that whilst poorer outcomes were being detected and
potential causes identified, there was not a strong enough leadership
to take command of making significant improvements. Furthermore, the
implementation of effective leadership and robust business models will
ensure that data driven decision making in healthcare systems is
upheld in a structured and innovative fashion.