Discussion
Assessment of adverse events that occur during therapy to determine whether they represent an ADR versus the underlying disease of interest is problematic. The development of tools to provide a more objective approach rather than relying solely on clinical intuition would permit a more evidence-based approach to care, notably for patients for whom the nihilistic approach of simply avoiding the drug may place them at risk or compromise their care. The Naranjo Scale was developing at the University of Toronto in the late 1970’s to help address this issue [14]. This scale was developed and validated in a cohort of patients undergoing psychotherapy and the various domains on the scale reflect this and the practice of psychopharmacology in the 1970s. As the practice of medicine has evolved this has made practical implementation of the scale more problematic; as an example, while the use of a placebo (item 6 on the Naranjo Scale) was considered at the time the scale was developed this would be unlikely to be a realistic option currently [Fig 1]. This and other elements of the scale have made made the Naranjo Scale less helpful in a clinical context, and consequently groups have developed alternate approaches. One such approach, developed by investigators at the University of Liverpool and Alder Hey Children’s Hospital, was the Liverpool Causality Assessment, created based on data from in-patients at Alder Hey [19].
This study was conducted to examine the value of screening suspected ADR cases prior to any investigations using history and clinical presentation alone in a ambulatory care setting and to compare the Naranjo Scale to the Liverpool Causality Assessment. The two published assessment tools: the older Naranjo Scale and newly developed tool by the Liverpool group were correlated with well studied diagnostic tests such as the allergy skin test, LTA, iPTA, RAST and oral challenge [22-25].
Both assessment tools have a high sensitivity in predicting ADR cases but poor specificity. Liverpool causality assessment tool (97.2% ± 2.4%) proved to be the more sensitive tool compared to its predecessor, Naranjo scale (81.2% ± 5.69%). This is likely related to design; the Naranjo Scale is a summative score of all the questions to determine the likelihood of ADR while the Liverpool instrument has a flowchart format. This significantly impacted the score distribution as all cases lost two points on the Naranjo Scale due to following two reasons. First, a score of 0 was given for the question addressing whether the reaction reappeared when a placebo was given; as noted above placebo challenge was not administered at the LHSC Clinical Pharmacology clinic. Secondly, our study attempted to retrospectively score charts assuming no diagnostic tests were performed for all cases, therefore a score of 0 was given for the last question addressing if there were any objective evidence supporting ADR. This essentially made it impossible for any cases to score “Definite” on Naranjo Scale as the maximum score any case can receive was 8/10, which is equivalent to “Probable”. The placebo question was rejected by the Liverpool group when their tool was developed as it was not a common practice currently [19]. Moreover, the Liverpool tool’s flowchart format made it possible for the cases to achieve “Definite” without having an objective evidence to support ADR.
The low specificities of the two tools were expected as the authors of both Naranjo and Liverpool assessment tool designed the questions to evaluate the sequence of events and reaction response to drug initiation and/or cessation. The tools were not designed to examine ADR causality directly or mechanistically but to screen suspected ADR cases in the clinic setting in a timely manner prior to ordering more expensive, time consuming diagnostic tests.
Predictive values for both Naranjo and Liverpool tools were similar in comparison but both tools are more accurate with its negative results than predicting true positives. Both tools had negative predictive value of approximately 60% and positive predictive value of just over 30%. This can be related back to the above explanation for low specificities observed. The results from either tool are not diagnostically reliable and should only be used to screen patients in conjunction to clinical picture to aid in decision making to pursue further investigations.
This is the first study to compare the newer Liverpool Causality Assessment Tool with its predecessor, Naranjo Scale, independently from the developers of the two tools and in an ambulatory setting. Our study population was also independently selected from the cases previously identified in the papers describing the two tools[14, 19]. The tools were applied in more than 500 suspected ADR cases and the large sample size allowed this study to determine the sensitivities and specificities with narrow 95% confidence intervals.
As previously described by Gallagher et al. , the Liverpool assessment tool demonstrated a moderate inter-rater reliability with a global kappa score of 0.48[19]. However in our study, only one rater scored all charts and the potential for inter-rater difference was not a concern. By having only one rater, all charts were scored with consistent interpretation and application of the data.
It should be noted that only a handful number of cases that scored at the polar ends on both Liverpool and Naranjo scales, either “Unlikely/Doubtful” or “Definite”. This may have led to skewing of the negative predictive values as only a limited number of samples were available for statistical analysis. A future study identifying more suspected ADR cases that score at the extreme ends would be a useful addition to our findings, although in the case of the Naranjo Scale given the items used to scoring findings of Definite are quite uncommon.
The Liverpool Causality Assessment tool has superior sensitivity compared to the Naranjo Scale. The Liverpool tool has an observed sensitivity of 97.2% ± 2.4% which suggests that it can be a useful screening tool in the clinic setting. Nonetheless its specificity and predictive values are low and should not be used as the sole method of measuring the likelihood of ADR in suspected cases. This reserves clinicians to consider the overall clinical impression and use the tool as a supplementation when making ADR diagnosis or decisions to pursue further investigations. Other diagnostic tests should be performed for understanding the mechanism of ADR or to prove direct causality between a drug and symptoms. Liverpool Causality Assessment Tool is useful for screening ADR prior to pursuing diagnostic investigations, such as during initial patient encounter.
The utility of ADR screening tools such as the Liverpool causality assessment tool or Naranjo scale has not been verified prospectively in an ambulatory or primary care setting. As well, the value of these tools perceived by clinicians using them in clinical settings should be evaluated by collecting data after applying the tools when suspecting ADRs in patients suggesting that more research is needed to refine causality instruments for the evaluation of potential ADRs.