Trigger Tool-Based Automated Adverse Event Detection in Electronic Health Records: Systematic Review

BACKGROUND: Adverse events in health care entail substantial burdens to health
care systems, institutions, and patients. Retrospective trigger tools are often
manually applied to detect AEs, although automated approaches using electronic
health records may offer real-time adverse event detection, allowing timely
corrective interventions.
OBJECTIVE: The aim of this systematic review was to describe current study
methods and challenges regarding the use of automatic trigger tool-based adverse
event detection methods in electronic health records. In addition, we aimed to
appraise the applied studies' designs and to synthesize estimates of adverse
event prevalence and diagnostic test accuracy of automatic detection methods
using manual trigger tool as a reference standard.
METHODS: PubMed, EMBASE, CINAHL, and the Cochrane Library were queried. We
included observational studies, applying trigger tools in acute care settings,
and excluded studies using nonhospital and outpatient settings. Eligible articles
were divided into diagnostic test accuracy studies and prevalence studies. We
derived the study prevalence and estimates for the positive predictive value. We
assessed bias risks and applicability concerns using Quality Assessment tool for
Diagnostic Accuracy Studies-2 (QUADAS-2) for diagnostic test accuracy studies and
an in-house developed tool for prevalence studies.
RESULTS: A total of 11 studies met all criteria: 2 concerned diagnostic test
accuracy and 9 prevalence. We judged several studies to be at high bias risks for
their automated detection method, definition of outcomes, and type of statistical
analyses. Across all the 11 studies, adverse event prevalence ranged from 0% to
17.9%, with a median of 0.8%. The positive predictive value of all triggers to
detect adverse events ranged from 0% to 100% across studies, with a median of
40%. Some triggers had wide ranging positive predictive value values: (1) in 6
studies, hypoglycemia had a positive predictive value ranging from 15.8% to 60%;
(2) in 5 studies, naloxone had a positive predictive value ranging from 20% to
91%; (3) in 4 studies, flumazenil had a positive predictive value ranging from
38.9% to 83.3%; and (4) in 4 studies, protamine had a positive predictive value
ranging from 0% to 60%. We were unable to determine the adverse event prevalence,
positive predictive value, preventability, and severity in 40.4%, 10.5%, 71.1%,
and 68.4% of the studies, respectively. These studies did not report the overall
number of records analyzed, triggers, or adverse events; or the studies did not
conduct the analysis.
CONCLUSIONS: We observed broad interstudy variation in reported adverse event
prevalence and positive predictive value. The lack of sufficiently described
methods led to difficulties regarding interpretation. To improve quality, we see
the need for a set of recommendations to endorse optimal use of research designs
and adequate reporting of future adverse event detection studies.