The future: potential ways to deal with outliers
Quality assurance is a constant process incumbent on individual clinicians and institutions to maintain practice within acceptable standards. Therefore when negative outliers appear it is logical that “bringing them back” would involve similar active processes (14). As part of a new paradigm, we propose the following:
  1. Novel outcomes The quality performance equilibrium is in flux. As newer technology for clinical diagnosis and treatment is constantly emerging, so too should our methods of quality assurance. As such, the definitions outliers must be adaptable and relevant. Outlier detection should not rely on mortality alone, which is a poor discriminator in quality of care. Further endpoints are required that are automatically collated or patient reported, which may include quality of life measures, patient satisfaction, imaging outcomes and biomarkers. As an example, analysing post-surgical patient quality of life and satisfaction in a well-run institution may highlight its positive performance, compared to focusing on an unexpected in-hospital mortality it recently suffered. As well as novel outcomes, robust statistical methods that accurately and appropriately classify outliers are required, using both supervised classification strategies and proximity-detection methods. Differentiating between binary and continuous indicators, logistic regression and propensity adjusted scoring and their influences on data hierarchy should be sought(15,16). Statistical methods need to go through cycles of adaptation as outcomes changes and patient populations evolve.
  2. Positive outliers The emergence of a positive outlier must not only prompt reward, but attempt to shift the equilibrium of clinical performance in the wider field. Perhaps this excellent performance arose from a novel clinical method or management process. With the GMC evolving into a weapon used by healthcare leaders to force outstanding competency and expose negative outliers, there is little being done to reward the excellent. Such mechanisms are psychologically straining for surgeons who are already in a high-risk field and stand to gain very little if they demonstrate good outcomes. Leaders of the NHS on a national, and even a local and departmental level, can evoke more permanent change by encouraging and nurturing those who excel as positive outliers. Individuals and institutions that can go on to encourage others to follow suit(17).
  3. Negative outliers Focusing on negative outcomes alone in identifying and then improving underperformers is not the optimum method(6). In fact, newer studies are suggesting that outlier detection using outcomes analysis may well be avoided, and the subsequent knee-jerk reward-punishment strategies imposed to bring back outliers, can be harmful, especially when involving blame(7,18). In a recent national survey(13), 58% of UK cardiothoracic surgeons opposed reporting of surgeon-specific mortality. Studies highlight an increase in risk-averse and loss-averse behaviour from surgeons who fear taking on high-risk cases, especially when there is no reward for taking on such cases yet a high punishment when death does occur. The result of such measures has seen more risk-averse behaviour by surgeons wanting to avoid operating on high-risk patients (the patients who arguably need the most attention) to circumvent poorer outcomes (13). A fairer and less punitive attitude with negative outliers is therefore essential: this is especially true if a large body of social science purports that outlier occurrence is indeed random and unpredictable..
  4. Leadership The need for strong leadership and management is paramount(19,20). Rescuing an outlier requires the unit to challenge the status quo and make significant changes in a timely manner to return to the accepted mean. This would rely on an executive with good management capabilities, effective communication and a departmental culture conducive to adaptation and change(14). The Bristol Enquiry (21) found that whilst poorer outcomes were being detected and potential causes identified, there was not a strong enough leadership to take command of making significant improvements. Furthermore, the implementation of effective leadership and robust business models will ensure that data driven decision making in healthcare systems is upheld in a structured and innovative fashion.