Variables
There were a number of similarities between variables in the AIME and
previously described models. For example; antiarrhythmic agents were
significant in the model by McElnay [44] and antipsychotics in
models by Trivalle and Nguyen et al. [12, 46]. Previous
hospitalisation also featured in the model by Nguyen [46], and
previous drug allergy was a variable in the GerontoNet ADR risk score,
and the MOAT model [37, 41]. All models included the ‘number of
medications’ as a predictor which highlights the importance of
deprescribing.
A clinically informative model development strategy must assess a
comprehensive range of variables and this was a strength of our study.
To the best of our knowledge, no other study has included laboratory
tests assessing coagulation indices such as INR and aPTT, yet they are
commonly used by clinicians to identify high-risk patients for urgent
assessment and diagnosis. By leveraging the digital capabilities of our
hospital’s EMR we obtained a comprehensive dataset. This resulted in a
final model which included serum sodium and INR, as predictors. Given
that these variables have been identified as clinically important for
patient prioritisation [24] and are modifiable, we anticipate that
their inclusion will enhance the application of the AIME model and its’
translation into practice. Whilst variables such thromboembolism or
supratherapeutic INR may occur later during the course of
hospitalisation, the advantages of “real-time” digital surveillance
means that as the patient’s variables fluctuate so does the probability
of medication harm. This is consistent with the dynamic nature of
patient risk and a predictive model such as the AIME can assist with
ongoing clinical prioritisation.
Limitations
Despite our best attempt to develop a model with an EPV of 10 or greater
we were unable to achieve this given the larger-than expected number of
variables included in the multivariable modelling phase of our study.
However, simulation studies have shown that lower EPVs can produce
stable models, and that model instability occurs with EPVs below 4
[47].
For clinical applicability three variables (INR, serum sodium and number
of medications) were categorised. To minimise the risk of bias we
examined variables and selected thresholds based on optimal cut-off in
ROC analysis (for number of medications), and clinical relevance as
guided by pharmacist survey and focus groups.
Whilst we evaluated a comprehensive set of variables there were others,
such as frailty which has been correlated with polypharmacy and
non-adherence, that may have warranted inclusion[48]. However, at
the time this study was conducted we were unable to quantify these
variables with precision. Plans for external evaluation of the AIME
model include testing these variables, as we anticipate they could
improve model performance.
The retrospective nature of our study means that it is possible that
medication-harm events were undetected. We chose this method as a
practical means of evaluating harm in a reasonably sized patient cohort.
Studies have used ICD-10 coding as an efficient approach to evaluate the
impact of serious harm events in the hospital setting[49]. In
addition to using ICD-10 codes, we comprehensively examined the medical
records of flagged patients to detect any additional medication events
that may not have been documented or coded. The hospital’s incident
database was also reviewed to identify medication incidents that
resulted in actual patient harm.
The AIME includes two variables (antipsychotic use and history of
medication allergy) that marginally exceed the traditional 5% level of
significance. This stems from the principal that prognosis (as opposed
to causality assessment), is based on risk estimation, and thus it is
statistically acceptable (and recommended) to include predictors with
p-values greater than the 5% threshold. This is especially true if the
predictor is known to be correlated with the outcome from prior research
and its inclusion has a large effect size and enhances model performance
[27]. Other predictive models such as the BADRI ADR score and the
MOAT model have followed a similar approach [11, 37].
Future direction
The delivery of a comprehensive medication management services, for
every inpatient, is not feasible in busy Australian public hospitals
which serve a growing and aging population. Therefore, we must
prioritise those at greatest risk of medication harm for timely
interventions. To date, a limited number of risk prediction models have
been developed for identifying hospital inpatients at high-risk of
medication harm and none have undergone impact evaluation. Our study
shows that the AIME model has an acceptable degree of predictive
accuracy and potential clinical utility.
The availability of complete EMRs means that predictive models could be
embedded into digital systems. This would enable patient risk to be
iteratively estimated and monitored in real time using approaches such
as surveillance dashboards, available to all clinicians at any time. The
increasing availability of hospital digital data and machine learning
methods of modelling provide an exciting opportunity to gain deeper
insights and improve the quality of patient care [50]. The role of
e-health in improving medication safety is predicted to be fundamental.
Clinicians must embrace
opportunities to utilise a hospital’s digital capabilities to mitigate
avoidable harm, improve patient outcomes and optimise health system
efficiencies, and the AIME model offers this opportunity.
Acknowledgements
The authors would like to express their sincere thanks and appreciation
to Professor Bill Venables for his expert statistical guidance, and to
Mr Karl Winckel and Dr Christopher Morris who assisted with review and
rating of medication events.
References
(1) Budnitz, D.S., Lovegrove, M.C., Shehab, N. & Richards, C.L.
Emergency hospitalizations for adverse drug events in older Americans.N. Engl. J. Med. 365 , 2002-12 (2011).
(2) Leendertse, A.J., Van Den Bemt, P.M.L.A., Bart Poolman, J., Stoker,
L.J., Egberts, A.C.G. & Postma, M.J. Preventable Hospital Admissions
Related to Medication (HARM): Cost analysis of the HARM study.Value. Health. 14 , 34-40 (2011).
(3) Classen, D.C., Pestotnik, S.L., Evans, R.S., Lloyd, J.F. & Burke,
J.P. Adverse drug events in hospitalized patients: Excess length of
stay, extra costs, and attributable mortality. JAMA.277 , 301-6 (1997).
(4) Dhingra, N. WHO Global Patient Safety Challange - Medication
safety .
<http://www.who.int/patientsafety/campaigns/en/>
(2016). Accessed April 2017 2017.
(5) Roughead, E.E., Semple, S.J. & Rosenfeld, E. The extent of
medication errors and adverse drug reactions throughout the patient
journey in acute care in Australia. Int. J. Evid. Based. Health.14 , 113-22 (2016).
(6) Paradissis, C., Coombes, I.D., Donovan, P., Doran, E., Mckean, M. &
Barras, M.A. The type and incidence of adverse drug events in ageing
medical inpatients and their effect on length of hospital stay.JPPR. 47 , 347-54 (2017).
(7) Davies, E.C., Green, C.F., Taylor, S., Williamson, P.R., Mottram,
D.R. & Pirmohamed, M. Adverse drug reactions in hospital in-patients: a
prospective analysis of 3695 patient-episodes. PLoS. ONE.4 , e4439 (2009).
(8) O’Sullivan, D. et al. The impact of a structured pharmacist
intervention on the appropriateness of prescribing in older hospitalized
patients. Drugs. Aging. 31 , 471-81 (2014).
(9) Australian Commission on Safety and Qaulity in Healthcare (ACSQHC).Hospital-acquired complications .
<https://www.safetyandquality.gov.au/our-work/indicators/hospital-acquired-complications/>
(2019).
(10) Shekelle, P., Wachter, R.M., Provonost, P., Schoelles, K.,
Mcdonald, K. & et al. (2013). Making Health Care Safer 2: An
Updated Critical Analysis of Patient Safety Practices. Comparative
Effectiveness Review No. 211 (Agency for Healthcare Research and
Quality, Rockville, MD, 2013).
(11) Tangiisuran, B. et al. Development and validation of a risk
model for predicting adverse drug reactions in older people during
hospital stay: Brighton Adverse Drug Reactions Risk (BADRI) model.PLoS. One. 9 , e111254 (2014).
(12) Trivalle, C., Burlaud, A. & Ducimetière, P. Risk factors for
adverse drug events in hospitalized elderly patients: A geriatric score.Eur. Geriatr. Med. 2 , 284-9 (2011).
(13) Moons, K.G.M. et al. Risk prediction models: I. Development,
internal validation, and assessing the incremental value of a new
(bio)marker. Heart. 98 , 691e8 (2012).
(14) Anderson, K.M., Odell, P.M., Wilson, P.W. & Kannel, W.B.
Cardiovascular disease risk profiles. Am. Heart. J. 121(1
Pt 2) , 293-8 (1991).
(15) Lane, D.A. & Lip, G. Use of the CHA2DS2-VASc and HAS-BLED Scores
to Aid Decision Making for Thromboprophylaxis in Nonvalvular Atrial
Fibrillation. Circulation. 126 , 860–5 (2012).
(16) Pisters, R., Lane, D.A., Nieuwlaat, R., de Vos, C.B., Crijns, H. &
Lip, G. A novel user-friendly score (HAS-BLED) to assess 1-year risk of
major bleeding in patients with atrial fibrillation. The Euro Heart
Survey. Chest 138 , 1093-100 (2010).
(17) Falconer, N., Barras, M.A. & Cottrell, W.N. Systematic review of
predictive risk models for adverse drug events in hospitalised patients.Bri. J. Clin. Pharmacol. 84 , 846-64 (2018).
(18) Bates, D.W., Boyle, D.L., Vander Vliet, M.B., Schneider, J. &
Leape, L. Relationship between Medication Errors and Adverse Drug EventsJ. Gen. Intern. Med. 10 , 199-205 (1995b).
(19) Edwards, I.R. & Aronson, J.K. Adverse drug reactions: definitions,
diagnosis and management. Lancet. 356 , 1255-9 (2000).
(20) Aronson, J.K. & ferner, R.E. Classification of Terminology in Drug
Safety. Drug. Saf. 28 , 851-70 (2005).
(21) Hallas, J. et al. Drug related hospital admissions: the role
of definitions and intensity of data collection, and the possibility of
prevention. J. Intern. Med. 228 , 83-90 (1990).
(22) Morimoto, T., Gandhi, T.K., Segar, A.C., Hsieh, T.C. & Bates, D.W.
Adverse Drug Events and Medication Errors: detection and classification
methods. Qual. Saf. health. Care. 13 , 306-14 (2004).
(23) Schumock, G.T. & Thornton, J.P. Focusing on the preventability of
adverse drug reactions. Hosp. Pharm. 27 , 538 (1992).
(24) Falconer, N., Barras, M. & Cottrell, N. How hospital pharmacists
prioritise patients at high risk for medication harm. Res. Soc.
Admin. Pharm. 15 , 1266-73 (2018).
(25) Australian Medicines Handbook 2017 (online) (Australian
Medicines Handbook Pty Ltd: Adelaide, 2017).
(26) Levey, A.S. et al. A new equation to estimate glomerular
filtration rate. Ann Intern Med 150 , 604-12 (2009).
(27) Steyerberg, E.W. Clinical Prediction Models : a practical
approach to development validation and updating. (2009).
(28) Moon, K.G.M. et al. Critical Appraisal and Data Extraction
for Systematic Reviews of Prediction Modelling Studies: The CHARMS
Checklist. PLoS. Med. 11 , e1001744 (2014).
(29) Heinz, G., Wallisch, C. & Dunkler, D. Variable selection – A
review and recommendations for the practicing statistician.Biometrical 431-49 (2017).
(30) R Core Team. R: A language and environment for statistical
computing. (R Foundation for Statistical Computing, Vienna, Austria,
2018).
(31) R: A language and environment for statistical computing. . (R Core
Team (2013), R Foundation for Statistical Computing, Vienna, Austria.,
2013).
(32) Collins, G.S., Reitsma, J.B., Altman, D.G. & Moons, K.G.M.
Transparent reporting of a multivariable prediction model for individual
prognosis or diagnosis (TRIPOD): the TRIPOD Statement. Eur. J.
Clin. Invest. 45 204–14 (2015).
(33) Parekh, N. et al. Medication-related harm in older adults
following hospital discharge: development and validation of a prediction
tool. BMJ. Qual. Saf. , bmjqs-2019-009587 (2019).
(34) Vickers, A.J. & Elkin, E.B. Decision Curve Analysis: A Novel
Method for Evaluating Prediction Models. Med. Decis. Making.26 , 565-74 (2006).
(35) Scott, I.A., Hilmer, S.N. & Reeve, E. Reducing Inappropriate
Polypharmacy: The Process of Deprescribing. JAMA. Intern. Med.175 , 827-34 (2015).
(36) Onder, G. et al. Development and validation of a score to
assess risk of adverse drug reactions among in-hospital patients 65
years or older: The GerontoNet ADR risk score. Arch. Int. Med.170 , 1142-8 (2010).
(37) Geeson, C., Wei, L. & Dean Franklin, B. Development and
performance evaluation of the Medicines Optimisation Assessment Tool
(MOAT): a prognostic model to target hospital pharmacists’ input to
prevent medication-related problems,. BMJ. Qual. Saf. , 1-12
(2019).
(38) Sakuma, M., Bates, D.W. & Morimoto, T. Clinical prediction rule to
identify high-risk inpatients for adverse drug events: the JADE Study.Pharmacoepidemiol. Drug. Saf. 21 , 1221-6 (2012).
(39) O’Mahony, D., O’Connor, M.N., Eustace, J., Byrne, S., Petrovic, M.
& Gallagher, P. The adverse drug reaction risk in older persons
(ADRROP) prediction scale: derivation and prospective validation of an
ADR risk assessment tool in older multi-morbid patients. Eur. Ger.
Med. 9 , 191-9 (2018).
(40) Bates, D.W. et al. Incidence of Adverse Drug Events and
Potential Adverse Drug Events: Implications for Prevention. JAMA274 , 29-34 (1995).
(41) Onder, G. et al. Development and validation of a score to
assess risk of adverse drug reactions among in-hospital patients 65
years or older: the GerontoNet ADR risk score. Arch. Intern. Med.170 , 1142-8 (2010).
(42) Schneider, J.K., Mion, L.C. & Frengley, J.D. Adverse drug
reactions in an elderly outpatient population. Am. J. Hosp.
Pharm. 49 , 90-6 (1992).
(43) Parameswaran Nair, N., Chalmers, L., Peterson, G.M., Bereznicki,
B.J., Castelino, R.L. & Bereznicki, L.R. Hospitalization in older
patients due to adverse drug reactions – the need for a prediction
tool. Clin. Interv. Aging. 11 , 497-505 (2016).
(44) McElnay, J.C., McCallion, C.R., Al-Deagi, F. & Scott, M.G.
Development of a Risk Model For Adverse Drug Events in the Elderly.Clin. Drug. Investig. 13 , 47-55 (1997).
(45) Kaiser, E.A., Lotze, U. & Schäfer, H.H. Increasing complexity:
which drug class to choose for treatment of hypertension in the elderly?Clin. Interv. Aging. 9 , 459-75 (2014).
(46) Nguyen, T. et al. Improving medication safety: Developement
and impact of a multivariate model-based strategy to target high-risk
patients. PLOS. One. 12 , 1-13 (2017).
(47) Vittinghoff, E. & McCulloch, C.E. Relaxing the rule of ten events
per variable in logistic and Cox regression. Am. J. Epidemiol.165 , 710-8 (2007).
(48) Scott, I.A., Gray, L.C., Martin, J.H. & Mitchell, C.A. Minimizing
Inappropriate Medications in Older Populations: A 10-step Conceptual
Framework. Am. J. Med. 125 , 529-37.e4 (2012).
(49) Walter, S.R., Day, R.O., Gallego, B. & Westbrook, J.I. The impact
of serious adverse drug reactions: a population-based study of a decade
of hospital admissions in New South Wales, Australia. Br. J. Clin.
Pharmacol. 83 , 416-26 (2017).
(50) Hutchinson, L. et al. Models and Machines: How Deep Learning
Will Take Clinical Pharmacology to the Next Level. Clin Pharmaco.
Ther. 8 , 131-4 (2019).