\documentclass[10pt]{article}

\usepackage{fullpage}
\usepackage{setspace}
\usepackage{parskip}
\usepackage{titlesec}
\usepackage[section]{placeins}
\usepackage{xcolor}
\usepackage{breakcites}
\usepackage{lineno}
\usepackage{hyphenat}





\PassOptionsToPackage{hyphens}{url}
\usepackage[colorlinks = true,
            linkcolor = blue,
            urlcolor  = blue,
            citecolor = blue,
            anchorcolor = blue]{hyperref}
\usepackage{etoolbox}
\makeatletter
\patchcmd\@combinedblfloats{\box\@outputbox}{\unvbox\@outputbox}{}{%
  \errmessage{\noexpand\@combinedblfloats could not be patched}%
}%
\makeatother


\usepackage{natbib}




\renewenvironment{abstract}
  {{\bfseries\noindent{\abstractname}\par\nobreak}\footnotesize}
  {\bigskip}

\titlespacing{\section}{0pt}{*3}{*1}
\titlespacing{\subsection}{0pt}{*2}{*0.5}
\titlespacing{\subsubsection}{0pt}{*1.5}{0pt}


\usepackage{authblk}


\usepackage{graphicx}
\usepackage[space]{grffile}
\usepackage{latexsym}
\usepackage{textcomp}
\usepackage{longtable}
\usepackage{tabulary}
\usepackage{booktabs,array,multirow}
\usepackage{amsfonts,amsmath,amssymb}
\providecommand\citet{\cite}
\providecommand\citep{\cite}
\providecommand\citealt{\cite}
% You can conditionalize code for latexml or normal latex using this.
\newif\iflatexml\latexmlfalse
\providecommand{\tightlist}{\setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}}%

\AtBeginDocument{\DeclareGraphicsExtensions{.pdf,.PDF,.eps,.EPS,.png,.PNG,.tif,.TIF,.jpg,.JPG,.jpeg,.JPEG}}

\usepackage[utf8]{inputenc}
\usepackage[english]{babel}



\usepackage{float}








\begin{document}

\title{Evaluating the Impact of EMR Implementation in the Australian Health
System: A Scoping Literature Review}



\author[1]{Michelle Middleton}%
\affil[1]{Affiliation not available}%


\vspace{-1em}



  
  \date{May 26, 2024}


\begingroup
\let\center\flushleft
\let\endcenter\endflushleft
\maketitle
\endgroup





\selectlanguage{english}
\begin{abstract}
Background

An EMR is an electronic medical record, also known as an electronic
health record (EHR), and is a computerized database containing patient
information (demographics, admission data, clinical notes, test results,
medications, etc.) and accounting information (McGraw-Hill, 2002). EMRs
often include additional features to streamline care, such as decision
aids for clinicians, and theoretically improve efficiency and reduce
human error in healthcare (Australian Digital Health Agency, 2021;
Duckett, 2018b).

The Australian public health system has only started to implement EMRs
and related technologies to advance digital health over the past decade
or so, amongst a complex and fragmented funding system shared between
the Commonwealth and the States and Territories (Duckett, 2018b; Jedwab
et al., 2019).

To ensure compliance with the 2020-25 National Health Reform Agreement
(Australian Government Department of Health, 2022), so that value for
money can be achieved in an area where competition for resources and
funding is fierce, this scoping literature review will identify the
available evidence, what opportunities there are for improvement, and
where Australia has excelled.

\subsection*{Evidence check question}\label{evidence-check-question}

This review was guided by the question: `Has the success of electronic
medical record (EMR) implementation been evaluated both quantitatively
and qualitatively in the Australian health care system?'

\subsection*{Objective}\label{objective}

To review available evidence evaluating EMR implementation across all
health settings in Australia, to identify current knowledge gaps, and
provide recommendations for future research on evaluation strategies.

\subsection*{Summary of methods}\label{summary-of-methods}

A scoping literature review of primary, peer-reviewed, academic
literature was performed by a single researcher to identify and map the
available evidence over the past decade (2012-2022) in Australia only.
The PRISMA-ScR methodology was followed, with a critical appraisal of
individual evidence sources assessed against the MMAT integrity
checklist. A thematic analysis of findings was then performed on the
shortlisted articles. Because there was no primary data collection
involved, ethical approval was not required (Griffith University, 2022).

\subsection*{Key findings~}\label{key-findings}

25 papers met the inclusion criteria. Approximately half of the studies
(n=12) were qualitative, nine were quantitative in nature, and four were
mixed-methods studies. There was an even split of studies in the
Victorian and New South Wales (NSW) settings (32\% each) and just 8\% in
Queensland. The remainder did not define the location.

14 studies targeted nurses, three targeted midwives, allied health
professionals, and aged care workers, seven targeted pharmacy staff, and
eight targeted doctors/medical staff. Only seven were pre-post studies,
most of which (n=5) were quantitative \& based in NSW.

11 of the 25 studies disclosed which EMR system was being assessed, the
majority of which were from Cerner (Cerner Millennium).

High-level findings were grouped into the following topics: patient
safety, efficiency, medication management, usability, documentation,
workforce satisfaction, and patient outcomes (articles could be assigned
more than one topic).

Further analysis allowed the above topics to be grouped into the
following high-level themes for discussion:

\begin{itemize}
\tightlist
\item
  Quality \& safety
\item
  Service delivery
\item
  Workforce-related
\end{itemize}

\subsection*{Conclusion}\label{conclusion}

There is no consistent framework to evaluate EMR implementation in
Australia, making return on investment, and impacts on healthcare
delivery difficult to define and compare. Overall, Australia has
answered the call to arms to increase its evidence base. However, given
the complexity of designing and executing research in this field, the
quality and quantity of available evidence may not be sufficient to
drive policy reform or recommendations for future evaluation strategies,
particularly since most of the evidence is qualitative. The evidence
captured in this scoping review generally supports EMR implementation,
demonstrating benefits such as improved efficiency, safety, and patient
outcome; though is constrained by research based on various EMR systems
in different settings and among different user groups. The use of
validated, standardized evaluation tools such as WOMBAT, STAMP, and
NuHISS is advocated to ensure consistency and reliability in future
evaluations.%
\end{abstract}%



\sloppy


\section*{\texorpdfstring{{Executive
summary}}{Executive summary}}

{\label{581789}}

\subsection*{\texorpdfstring{{Background}}{Background}}

{\label{background}}

An EMR is an electronic medical record, also known as an electronic
health record (EHR), and is a computerized database containing patient
information (demographics, admission data, clinical notes, test results,
medications, etc.) and accounting information (McGraw-Hill, 2002). EMRs
often include additional features to streamline care, such as decision
aids for clinicians, and theoretically improve efficiency and reduce
human error in healthcare (Australian Digital Health Agency, 2021;
Duckett, 2018b).

The Australian public health system has only started to implement EMRs
and related technologies to advance digital health over the past decade
or so, amongst a complex and fragmented funding system shared between
the Commonwealth and the States and Territories (Duckett, 2018b; Jedwab
et al., 2019).

To ensure compliance with the 2020-25 National Health Reform Agreement
(Australian Government Department of Health, 2022), so that value for
money can be achieved in an area where competition for resources and
funding is fierce, this scoping literature review will identify the
available evidence, what opportunities there are for improvement, and
where Australia has excelled.

\subsection*{\texorpdfstring{{Evidence check
question}}{Evidence check question}}

{\label{evidence-check-question}}

This review was guided by the question: `Has the success of electronic
medical record (EMR) implementation been evaluated both quantitatively
and qualitatively in the Australian health care system?'

\subsection*{\texorpdfstring{{Objective}}{Objective}}

{\label{objective}}

To review available evidence evaluating EMR implementation across all
health settings in Australia, to identify current knowledge gaps, and
provide recommendations for future research on evaluation strategies.

\subsection*{\texorpdfstring{{Summary of
methods}}{Summary of methods}}

{\label{summary-of-methods}}

A scoping literature review of primary, peer-reviewed, academic
literature was performed by a single researcher to identify and map the
available evidence over the past decade (2012-2022) in Australia only.
The PRISMA-ScR methodology was followed, with critical appraisal of
individual evidence sources assessed against the MMAT integrity
checklist. A thematic analysis of findings was then performed on the
shortlisted articles. Because there was no primary data collection
involved, ethical approval was not required (Griffith University, 2022).

\subsection*{\texorpdfstring{{Key
findings}}{Key findings}}

{\label{key-findings-results}}

25 papers met the inclusion criteria. Approximately half of the studies
(n=12) were qualitative, nine were quantitative in nature, and four were
mixed-methods studies. There was an even split of studies in the
Victorian and New South Wales (NSW) settings (32\% each) and just 8\% in
Queensland. The remainder did not define the location.

14 studies targeted nurses, three targeted midwives, allied health
professionals, and aged care workers, seven targeted pharmacy staff, and
eight targeted doctors/medical staff. Only seven were pre-post studies,
most of which (n=5) were quantitative \& based in NSW.

11 of the 25 studies disclosed which EMR system was being assessed, the
majority of which were from Cerner (Cerner Millennium).

High-level findings were grouped into the following topics shown
in~\textbf{Figure 1} below.

\par\null\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/Figure-1/Figure-1}
\caption{{Evidence bases of shortlisted articles by topic (articles could be
assigned more than one topic)
{\label{853347}}%
}}
\end{center}
\end{figure}

\par\null

Further analysis allowed the above topics to be grouped into the
following high-level themes for discussion:

\begin{itemize}
\tightlist
\item
  Quality \& safety
\item
  Service delivery
\item
  Workforce-related
\end{itemize}

\subsection*{\texorpdfstring{{Conclusion}}{Conclusion}}

{\label{conclusion}}

There is no consistent framework to evaluate EMR implementation in
Australia, making return on investment, and impacts on healthcare
delivery difficult to define. Overall, Australia has answered the call
to arms to increase its evidence base. However, given the complexity of
designing and executing research in this field, the quality and quantity
of available evidence may not be sufficient to drive policy reform or
recommendations for future evaluation strategies, particularly since
most of the evidence is qualitative. The evidence captured in this
scoping review generally supports EMR implementation, demonstrating
benefits such as improved efficiency, safety, and patient outcome;
though is constrained by research based on various EMR systems in
different settings and among different user groups. The use of
validated, standardized evaluation tools such as WOMBAT, STAMP, and
NuHISS is advocated to ensure consistency and reliability in future
evaluations.

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

\par\null

\section*{Main Body}

{\label{579369}}\par\null

\section*{Background}

{\label{852508}}

An EMR is an electronic medical record, also known as an electronic
health record (EHR), and is a computerized database containing patient
information (demographics, admission data, clinical notes, test results,
medications, etc.) and accounting information (McGraw-Hill, 2002). EMRs
often include additional features to streamline care, such as decision
aids for clinicians, to theoretically improve efficiency and reduce
human error in healthcare (Australian Digital Health Agency, 2021;
Duckett, 2018b).

The Australian public health system has only started to implement EMRs
and related technologies to advance digital health over the past decade
or so, amongst a complex and fragmented funding system shared between
the Commonwealth and the States and Territories (Duckett, 2018b; Jedwab
et al., 2019).

The World Health Organisation in their National eHealth Strategy Toolkit
(World Health Organization \& International Telecommunication Union,
2012) emphatically states the importance and benefits of results-based
management (a framework used by the United Nations). This is crucial in
the health sector to optimize scarce resources, improve accountability,
and enhance the sustainability and effectiveness of operations or
program activities. Monitoring and evaluation of eHealth strategies
(including EMR implementation) enable quantification of outcomes being
delivered and can help communicate this to stakeholders (for example,
for investment purposes), and allow corrective action to address
outcomes not achieved.

In 2013, The Victorian Auditor-General's Report on Clinical ICT Systems
in the Victorian Public Health Sector identified that the department and
health services are unable to report on benefits realization or outcomes
from clinical ICT systems (EMRs). Therefore, the Department of Health
cannot have effective financial oversight or be able to assess value for
money between varying EMR / clinical ICT systems (Victorian
Auditor-General's Office (VAGO), 2013).

Although this is the landscape in Victoria, it should be noted that the
digital health strategies of other states all emphasize the importance
of ensuring value for money is obtained from technologies, processes,
and systems through the use of a benefits realization framework (ACT
Government (Health), 2019; Government of Western Australia Department of
Health, 2019; Northern Territory Government, 2020; NSW Government, 2021;
Queensland Government, 2015; Rockliff, 2021).

Australia needs to bolster its own national evidence base surrounding
the outcomes of EMR implementation because Australia faces unique
geographical, funding, and political challenges not faced by other
origins of literature such as the US and Europe (Jedwab et al., 2019).

Almost ten years on from when Australia started its digital health
journey of EMR rollouts (Duckett, 2018b; Jedwab et al., 2019), how much
practice has been translated into research, accessible for future
learning and optimization of decision-making?

To comply with the 2020-25 National Health Reform Agreement (Australian
Government Department of Health, 2022), so value for money can be
achieved in an area where competition for resources and funding is
fierce, this scoping literature review will identify the current
evidence base, where there are opportunities for improvement, and where
Australia has excelled.

\section*{Definitions and key terms}

{\label{671926}}

This report will reference the technology in question as EMR(s), which
includes research using the interchangeably used terms in \textbf{Table
1 below} .

\subsubsection*{Table 1 Definitions of interchangeable EMR key terms used
in search
strategy}

{\label{851436}}\selectlanguage{english}
\begin{longtable}[]{@{}ll@{}}
\toprule
Term & Definition \& Rationale for Use in Search\tabularnewline
\midrule
\endhead
Electronic Health Record (EHR) & Defined as ``a longitudinal electronic
record of patient health information produced by encounters in one or
more care settings,''(Nelson \& Staggers, 2016) compared with an EMR's
definition being a ``\ldots{}resource used in a single healthcare
setting to capture patient data. Term often used interchangeably with
electronic health record.''(Nelson \& Staggers, 2016)\tabularnewline
Electronic Patient Record (EPR) & ``An~electronic record~of
health-related information on an individual that conforms to nationally
recognized interoperability standards and that can be drawn from
multiple sources while being managed, shared, and controlled by the
individual.'' (IGI Global, 2022)\tabularnewline
Digital Health (Record) (DHR) & ``{[}A{]}n umbrella term referring to a
range of technologies that can be used to treat patients and collect and
share a person's health information, including mobile health and
applications, electronic health records, telehealth and telemedicine,
wearable devices, robotics and artificial intelligence.'' (Australian
Institute of Health and Welfare, 2020) Term included as it encapsulates
EHRs.\tabularnewline
Electronic Medication Management System(s) (EMMS) & ``\ldots{}a broad
term covering all computer systems involved. It is a closed loop system
that encompasses prescribing, administration, pharmacy review, smart
infusion pumps, automated dispensing cabinets, barcode medication
administration and anything that has electronic medicines datasets or
encompasses medication management processes. There are a number of
electronic medication systems available. These vary from software for
individual practitioners to stand-alone systems for specialties (e.g.
oncology, intensive care), and hospital or district-wide systems with or
without an integrated, fully electronic medical record.'' (Pearce \&
Whyte, 2018) This definition highlights EMMS can be part of EMRs, hence
were included in the search. The Australian Commission on Safety \&
Quality in Healthcare also use this terminology when communicating with
the healthcare sector (Australian Commission on Safety and Quality in
Health Care, 2017).\tabularnewline
\bottomrule
\end{longtable}

\section*{Methods}

{\label{652485}}

A scoping literature review is a systematic method to map the available
evidence of a topic and can be used to identify and analyse knowledge
gaps. Outcomes of a scoping review can be used to direct further
research, or even support recommendations, depending on the topic (Munn
et al., 2018; Peters et al., 2021; PRISMA, 2021; Rethlefsen et al.,
2021). The research question is to determine the evidence base
surrounding EMR implementation in Australia, so a scoping literature
review would be the most appropriate methodology.

The review included five key phases:

\begin{enumerate}
\tightlist
\item
  Identifying the research question
\item
  Identifying and selecting relevant studies
\item
  Evaluation of data
\item
  Thematic analysis
\item
  Summarising and reporting results
\end{enumerate}

Prior to data collection, an exploratory search was performed to
ascertain scope and eligibility criteria suitable for the time and
researcher constraints.

\section*{Research question \&
objectives}

{\label{120199}}

This review was guided by the question: `Has the success of electronic
medical record (EMR) implementation been evaluated both quantitatively
and qualitatively in the Australian health care system?' broken down
into the objectives in \textbf{Table 2} .

\subsubsection*{Table 2: Research question breakdown and
objective}

{\label{208004}}\selectlanguage{english}
\begin{longtable}[]{@{}ll@{}}
\toprule
\begin{minipage}[b]{0.47\columnwidth}\raggedright\strut
Question breakdown\strut
\end{minipage} & \begin{minipage}[b]{0.47\columnwidth}\raggedright\strut
Objective\strut
\end{minipage}\tabularnewline
\midrule
\endhead
\begin{minipage}[t]{0.48\columnwidth}\raggedright\strut
Has the success of EMR implementation been evaluated in Australia?\strut
\end{minipage} & \begin{minipage}[t]{0.48\columnwidth}\raggedright\strut
Perform a scoping literature review to find evidence of evaluation of
EMRs in the Australian context.\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.48\columnwidth}\raggedright\strut
If so, has this been quantitatively or qualitatively?\strut
\end{minipage} & \begin{minipage}[t]{0.48\columnwidth}\raggedright\strut
Perform a comparative analysis of the literature review to determine
what measures of evaluation/success have been published and whether they
are qualitative or quantitative.\strut
\end{minipage}\tabularnewline
\bottomrule
\end{longtable}

The search strategy is outlined below. The studies were categorised
based on similarities in their main objectives and findings to enable
thematic analysis.

\section*{Scope}

{\label{237512}}

The scope of this review was confined to:

\begin{itemize}
\tightlist
\item
  Primary academic literature reporting on Australian outcomes of EMR
  systems.
\item
  Literature published over the last decade only due to the recency of
  technology (Jedwab et al., 2019).
\item
  Outcomes reported in all healthcare settings: hospitals, aged care,
  allied health, and primary and community health.
\item
  EMR technology only (not supplemental technologies linked to EMRs such
  as secure messaging).
\end{itemize}

\section*{Data sources and search
strategy}

{\label{676577}}

The initial search was undertaken in April 2022 on studies published
between 2012 and 2022, in the following databases accessed via Griffith
Library and Google Scholar search engines: Medline, Web of Science and
Scopus. \textbf{Appendix 1} provides a list of search query combination
terms used to search the databases. The key term EMR is also used
interchangeably with the terms in \textbf{Table 3 below} (Zurynski et
al., 2021), which were also used to capture the full range of research.
Refer to \textbf{Table 1} for definitions of these terms.

\subsubsection*{Table 3: Search terms used in scoping literature
review}

{\label{521702}}\selectlanguage{english}
\begin{longtable}[]{@{}ll@{}}
\toprule
Key term & Search terms / Boolean Operators\tabularnewline
\midrule
\endhead
EMR & EMR OR ``electronic medical record*'' OR EHR OR ``electronic
health record'' OR ``digital health record'' or EPR OR ``electronic
patient record'' OR EMMS or ``electronic medication management
system''\tabularnewline
Evaluation & Evaluat* OR success OR adopt* OR outcome* OR ``outcome
measure*'' OR ``performance measure'' OR ``impact assess*'' OR measure
AND success OR monitor* OR ``success rate''* OR perception OR ``key
performance indicator*'' OR KPI* OR impact* OR measure* OR result* OR
positive OR reduc* OR error* OR ``error rate''\tabularnewline
Australia & Australia*\tabularnewline
\bottomrule
\end{longtable}

\section*{Ethics}

{\label{639443}}

Patients or the public were not involved in the design or reporting of
this review, and there was no funding source or conflicts of interest.
According to Griffith University policy, no ethical approval was
required (Griffith University, 2022). This study was not formally
registered.

\section*{Evidence grading}

{\label{523649}}

Since the primary objective of this study was to identify the type of
Australian literature available, of which quality is one component, the
Mixed Methods Appraisal Tool (MMAT) was deemed the most efficient and
suitable integrity checklist to evaluate the evidence due to its
simplicity (Hong, Quan Nha, Fabregues, Sergi, et al., 2018; Hong, Quan
Nha, Pluye, Pierre, et al., 2018).

\section*{Eligibility criteria}

{\label{174446}}

Eligibility criteria are summarised in \textbf{Table 4} below.

\section*{}

{\label{271339}}

\subsubsection*{Table 4: Eligibility
criteria}

{\label{730228}}\selectlanguage{english}
\begin{longtable}[]{@{}llll@{}}
\toprule
Inclusion Criteria & Justification & Exclusion Criteria &
Justification\tabularnewline
\midrule
\endhead
Primary academic research. & Most relevant research type to answer
research question. & Study protocols, case series, technical
descriptions, conference abstracts and grey literature. & Not primary
research and unable to be graded using the MMAT criteria.\tabularnewline
Qualitative research, quantitative research \& mixed methods research. &
Critical to answering research question. & Non-English language. & Not
relevant to the research question. Also, to refine article selection
based on constraints of researcher capacity.\tabularnewline
Peer reviewed research only. & Quality assurance. & Systematic reviews,
scoping reviews and meta-analyses. & Not suitable for MMAT criteria and
based on preliminary literature review, all included research that was
conducted outside of Australia which is outside of the scope of the
research question.\tabularnewline
All healthcare settings (primary \& community care, aged care, allied
health and hospital / tertiary). & EMRs or their equivalent are
applicable in all these settings and within the scope of the research
question. & Duplicate research. & Eliminates double counting /
overestimation of amount of research in the field, critical to
addressing the research question.\tabularnewline
Research conducted in Australia only between 2012 and 2022. & Australia
has a unique health system which may impact outcomes of EMR
implementation. Given the background research highlighting most research
is from the U.S, this criterion provides a point of differentiation for
this research, addresses the research question, and provides a narrow
enough scope for the researcher's constraints. & Research not directly
related to outcomes of EMR implementation (e.g. for telehealth or mobile
applications). & Does not address the research question.\tabularnewline
\bottomrule
\end{longtable}

\subsection*{Approach}

{\label{approach}}

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses
extension for Scoping Reviews (PRISMA-S) was used to guide article
selection and data extraction, represented in \textbf{Figure 2~}(PRISMA,
2021).

\par\null\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/Figure-2/Figure-2}
\caption{{PRISMA-ScR approach used in this scoping revie3
{\label{231476}}%
}}
\end{center}
\end{figure}

\subsubsection*{Figure 2:~ PRISMA-ScR approach used in this scoping
review}

{\label{405069}}

\section*{Data extraction}

{\label{528926}}

Articles that met the eligibility criteria were extracted using EndNote,
where duplicates were removed. Studies were then organized by topic in
Microsoft Excel (supplemental data files 1-3 available upon request ).

\section*{Thematic analysis}

{\label{492618}}

Pivot tables were used to analyse results and perform a thematic
analysis. Topics that were initially identified in the data extraction
phase were then grouped into themes, with some reorganisation of topics
(for example, there were three articles in a miscellaneous column, which
were reassessed and regrouped accordingly).

\section*{Results and Thematic
analysis}

{\label{741878}}

25 papers met the eligibility criteria and were broadly grouped into the
topics shown in \textbf{Figure 3 below} .

\par\null\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/Figure-11/Figure-11}
\caption{{Evidence base of shortlisted articles by topic (articles could be
assigned more than one topic)
{\label{947668}}%
}}
\end{center}
\end{figure}

\par\null

Almost 50\% of studies were qualitative in nature (see \textbf{Figure
4~}below).

\par\null\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/Figure-4/Figure-4}
\caption{{Scoping review results: data categorisation
{\label{278459}}%
}}
\end{center}
\end{figure}

\par\null

Seven of the 25 shortlisted articles were pre-post studies. These were
most likely to be quantitative in nature, as demonstrated
in~\textbf{Figures 5 \& 6} below.~

\par\null\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/Figure-5/Figure-5}
\caption{{Scoping review results: data breakdown of pre-post studies
{\label{231028}}%
}}
\end{center}
\end{figure}\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/Figure-6/Figure-6}
\caption{{Scoping review results: pre-post study data types
{\label{695278}}%
}}
\end{center}
\end{figure}

12 of the 25 articles disclosed the EMR system analyzed in the studies.
The most common system reported on in the literature, based on the
parameters of this scoping review was Cerner Millennium (a quarter of
studies who disclosed their EMR). Cerner as an overall brand was
represented in 50\% of the shortlisted studies.

Research was primarily conducted in Victoria and NSW, as demonstrated by
\textbf{Figures 7} \textbf{\& 8} below.

\par\null\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/Figure-7/Figure-7}
\caption{{Scoping review results: research by state
{\label{433036}}%
}}
\end{center}
\end{figure}\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/Figure-8/Figure-8}
\caption{{Scoping review results: research by state over time
{\label{211838}}%
}}
\end{center}
\end{figure}

Since 2017, there has been an increase in published research evaluating
EMR implementation, primarily in Queensland and Victoria. The sentiment
of findings relative to state is demonstrated in \textbf{Figure 9} . The
articles were classified according to the criteria outlined
in~\textbf{Table 5} .

\subsubsection*{Table 5: Sentiment
criteria}

{\label{737200}}\selectlanguage{english}
\begin{longtable}[]{@{}ll@{}}
\toprule
Sentiment & Definition\tabularnewline
\midrule
\endhead
Positive & If findings were described with the inclusion of words such
as `improvement,' `satisfaction,' or implied promotion of the EMR system
in relation to the article topic(s).\tabularnewline
Negative & If findings were described with the inclusion of words such
as `deterioration,' `dissatisfaction,' or implied the EMR system
resulted in worsening of outcomes being reported.\tabularnewline
Mixed / neutral & If findings did not fall into the positive or negative
sentiments as above. For example, reporting on a metric but not implying
whether this is an improvement or worsening of current practice -- this
was seen in some studies reporting on change in medication review
practices.\tabularnewline
\bottomrule
\end{longtable}\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/Figure-9/Figure-9}
\caption{{Scoping review findings: sentiment of findings
{\label{467691}}%
}}
\end{center}
\end{figure}

Most articles included in this scoping review were of high to very-high
quality, based on the MMAT grading system (see \textbf{Figure 10} ), and
this has increased over recent years (demonstrated in \textbf{Figure 11}
), as has the total amount of published research (\textbf{Figure 12} ).
An article published in 2022 (South et al., 2022) was excluded in the
formation of \textbf{Figure 12} because this scoping review was
conducted in the first quarter of 2022, meaning there would be fewer
articles published this year to date, which would nullify the trend.

\par\null\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/Figure-10/Figure-10}
\caption{{Articles grouped by MMAT grading
{\label{110081}}%
}}
\end{center}
\end{figure}\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/Figure-111/Figure-111}
\caption{{Articles grouped by MMAT grading over time
{\label{264248}}%
}}
\end{center}
\end{figure}\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/Figure-12/Figure-12}
\caption{{Quantity of Australian research published over time
{\label{598933}}%
}}
\end{center}
\end{figure}

\subsubsection*{Health workforce groups}

{\label{health-workforce-groups}}

Health workforce groups were generally represented evenly, with the
exception of allied health staff and midwives being underrepresented in
the literature. This impacts usability and worker satisfaction, as this
often depends on the features being used in the EMR (Lloyd et al.,
2021).

\subsubsection*{Context}

{\label{context}}

Most studies (n=18) were conducted in tertiary hospitals. Two studies
included primary care, four were based in aged care, and one was in a
government-funded regional drug and alcohol service.

\subsubsection*{Approaches to evaluation: existing
standards}

{\label{approaches-to-evauation-existing-standards}}

10 of the 25 studies did not use a standardized tool, industry key
performance indicators (KPIs) or acknowledged audit criteria to evaluate
their area of interest.

One study stated they used `documentation standards' but did not further
define what they were (Wang et al., 2012). Existing auditing guidelines
included:

\begin{itemize}
\tightlist
\item
  Therapeutic drug monitoring guidelines (Firman et al., 2021)
\item
  The `Five Moments of Antimicrobial Prescribing' (Devchand et al.,
  2019)
\item
  National Inpatient Medication Chart (NIMC) audit criteria (McLain et
  al., 2017)
\item
  Aged Care Accreditation Standards (Jiang et al., 2016)
\item
  Current national screening tools for substance abuse, falls, and
  pressure injuries (Curtis et al., 2021)
\end{itemize}

Some studies also referenced the use of work sampling (E. Munyisia et
al., 2014; E. N. Munyisia et al., 2012).

\subsubsection*{Approaches to evaluation: time and motion studies and
before-and-after
studies}

{\label{approaches-to-evaluation-time-and-motion-studies-and-before-and-after-studies}}

Time and motion studies benefitted from the use of industry-validated
tools, such as the Work Observation Method By Activity Training (WOMBAT)
(Bingham et al., 2021; Westbrook et al., 2019) which was most popular,
and Suggested Time and Motion Procedures (STAMP) structured observation
tool (Walker et al., 2020).

An interrupted time series (ITS) analysis was used in in the most recent
study by South et al. (2022): a before and after study of 355,709
hospital discharges, over an eight-year period, at a paediatric teaching
hospital.

In 2013 standard Department of Health Emergency Department (ED) KPIs
were used to determine the impact of an EMR on ED service delivery
(Mohan et al., 2013).

\subsubsection*{Approaches to evaluation: user
experience}

{\label{approaches-to-evaluation-user-experience}}

For user experience, Lloyd et al. (2021) used the validated National
Usability-Focused HIS Scale (NuHISS) tool successfully, and promoted its
future use. Dabliz et al., (2021) used the Unified Theory of Acceptance
and Use of Technology (UTUAT) model.

\subsubsection*{Links between evidence
themes}

{\label{links-between-evidence-themes}}

As referenced in \textbf{Figure 3} , articles often reported on more
than one topic. For example, 10 of the 25 shortlisted studies reported
on workforce satisfaction, and the same proportion reported on
usability. Other topics reported together were typically medication
management \& efficiency, and medication safety with medication
management. These will be elaborated on in the discussions section.

\par\null\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/Figure-13/Figure-13}
\caption{{Common topic combinations: usability \& workforce satisfaction
{\label{878685}}%
}}
\end{center}
\end{figure}

\textbf{Figure 13} above shows there has been an increase in usability
and workforce satisfaction studies over time. Despite this, there have
been many recommendations from authors that more research is needed in
this area (Baysari et al., 2019; Dabliz et al., 2021; Lloyd et al.,
2021; E. N. Munyisia et al., 2012). Only four of the 10 studies
reporting on usability stated the system used, three of which were
Cerner. 50\% of studies reporting on workforce satisfaction were related
to Cerner.

\subsection*{Literature themes}

{\label{literature-themes}}

Topics were grouped into the themes in \textbf{Figure 14} below, based
on criteria in \textbf{Table 6} .

\subsubsection*{Table 6: Thematic analysis
criteria}

{\label{801514}}\selectlanguage{english}
\begin{longtable}[]{@{}lll@{}}
\toprule
Theme & Topic & Article attributes\tabularnewline
\midrule
\endhead
Quality \& safety & Patient outcomes & Morbidity \& mortality rates,
treatment times.\tabularnewline
& Patient satisfaction & If patients felt EMRs impacted on their
care.\tabularnewline
& Medication safety & Error rates and types regarding EMR-based
medication practices.\tabularnewline
& Regulatory requirements & EMRs and their role in compliance with
accreditation standards.\tabularnewline
Service delivery & Documentation & Quality, completion rates,
comprehensiveness.\tabularnewline
& Efficiency & Changes in length of time required to complete tasks,
including documentation.\tabularnewline
& Medication management & Changes in workflow of medication practices,
including medication review, and antimicrobial
stewardship.\tabularnewline
Workforce factors & Workforce satisfaction & Acceptance and opinion of
EMRs on workforce morale and workflow.\tabularnewline
& Usability & How the design of EMRs influences the above topics,
including documentation, efficiency, medication management and patient
safety.\tabularnewline
\bottomrule
\end{longtable}\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/Figure-14/Figure-14}
\caption{{Thematic analysis grouping
{\label{340988}}%
}}
\end{center}
\end{figure}

A summary of the shortlisted articles is in \textbf{Table 7} .

\subsubsection*{Table 7: Shortlisted articles and characteristics
summary}

{\label{362829}}\selectlanguage{english}
\begin{longtable}[]{@{}llllllllllll@{}}
\toprule
\begin{minipage}[b]{0.06\columnwidth}\raggedright\strut
Paper citation\strut
\end{minipage} & \begin{minipage}[b]{0.06\columnwidth}\raggedright\strut
State\strut
\end{minipage} & \begin{minipage}[b]{0.06\columnwidth}\raggedright\strut
MMAT Score\strut
\end{minipage} & \begin{minipage}[b]{0.06\columnwidth}\raggedright\strut
Method\strut
\end{minipage} & \begin{minipage}[b]{0.06\columnwidth}\raggedright\strut
Context\strut
\end{minipage} & \begin{minipage}[b]{0.06\columnwidth}\raggedright\strut
Discipline\strut
\end{minipage} & \begin{minipage}[b]{0.06\columnwidth}\raggedright\strut
Brand\strut
\end{minipage} & \begin{minipage}[b]{0.06\columnwidth}\raggedright\strut
Topics\strut
\end{minipage} & \begin{minipage}[b]{0.06\columnwidth}\raggedright\strut
Theme\strut
\end{minipage} & \begin{minipage}[b]{0.06\columnwidth}\raggedright\strut
Evaluation measures\strut
\end{minipage} & \begin{minipage}[b]{0.06\columnwidth}\raggedright\strut
Evaluation time\strut
\end{minipage} & \begin{minipage}[b]{0.06\columnwidth}\raggedright\strut
Key findings\strut
\end{minipage}\tabularnewline
\midrule
\endhead
\begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
South et al., 2022\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
VIC\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
4\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Quantitative\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Large paediatric teaching hospital (tertiary \& quaternary
services)\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Multiple / all\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Epic\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Patient outcomes\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Quality \& safety\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
355,709 hospital discharges ITS analysis\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
2 years pre and post EMR implementation\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Sustained decrease (22\%) in in-hospital mortality rate following EMR
implementation, resulting in one less death per fortnight hospital.
Supports investment in EMR systems.\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Wynter et al., 2021\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
VIC\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
5\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Qualitative\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Tertiary metropolitan public hospital\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Nurses Midwives\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
-\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Workforce satisfaction Usability\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Workforce factors\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Focus groups (retrospective)\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
12 months post-EMR implementation\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Nurses and midwives had mixed experiences of EMR. Significant training
\& ongoing support required 12 months following implementation. Negative
impacts associated with workflow and patient care.\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Witkowski et al., 2021\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
VIC\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
5\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Qualitative\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Tertiary metropolitan public hospital, orthopaedic outpatient
clinic\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Medical professionals (intern, Resident, Non-accredited registrar,
accredited registrar, consultant)\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
-\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Documentation\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Service delivery\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Retrospective medical record review\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Not defined\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Overall improvement in orthopaedic documentation in EMR vs paper record
but deficiencies in documentation remain. Deficiencies not solved solely
with technology. Discharge summaries completed 100\% EMR vs 82.5\%
paper.\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Lloyd et al., 2021\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
All\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
5\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Qualitative\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Multiple: Hospitals Primary care\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Nurses Medical professionals / doctors\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
-\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Workforce satisfaction Usability\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Workforce factors\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Retrospective, observational online cross-sectional survey. Use of
validated NuHISS tool.\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Not defined\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Nurses and medical professionals have different experiences with EMR
usability, dependent on the sector they work in and usability feature
measured. Technical quality features were more positively experienced by
doctors in the primary care sector than nurses as well as ease of
obtaining patient information and prevention of errors. In hospitals,
nurses experiences with EMRs were more positive regarding support for
routine task completion, learnability, ease of obtaining patient
information and entry of patient data.\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Firman et al., 2021\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
QLD\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
5\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Qualitative\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Tertiary hospital\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Multiple\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
-\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Documentation Medication management Medication safety\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Service delivery Quality \& Safety\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Records assessed for appropriateness of timing of collection, compliance
to recommended Therapeutic Drug Monitoring (TDM) guidelines, and
pharmacist documentation.\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
2-year retrospective audit. 2016 (paper) vs 2018 (ieMR).\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
No significant differences between ieMR and paper but there was
increased sufficient data to calculate BMI (86\%) vs 14\% paper
based.\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Dabliz et al., 2021\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
NSW\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
5\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Qualitative\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Tertiary 750-bed teaching hospital, specialised 12-bed outpatient
oncology unit\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Nurses Medical professionals / doctors Pharmacists\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Cerner\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Workforce satisfaction Usability\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Workforce factors\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Semi-structured interviews assessing usability, using validated unified
theory of acceptance and use of technology (UTUAT)\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
6 months post EMR implementation\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Nurses demonstrated overall satisfaction with EMR. Doctors and
pharmacists co-perceived usability problems related to restricted
autonomy and system complexity. UTAUT framework is useful to evaluate
usability of EMR. EMR design can reduce mental effort to search for
patient information.\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.06\columnwidth}\raggedright\strut
Curtis et al., 2021\strut
\end{minipage} & \begin{minipage}[t]{0.06\columnwidth}\raggedright\strut
NSW\strut
\end{minipage} & \begin{minipage}[t]{0.06\columnwidth}\raggedright\strut
4\strut
\end{minipage} & \begin{minipage}[t]{0.06\columnwidth}\raggedright\strut
Quantitative\strut
\end{minipage} & \begin{minipage}[t]{0.06\columnwidth}\raggedright\strut
4 NSW Hospitals\strut
\end{minipage} & \begin{minipage}[t]{0.06\columnwidth}\raggedright\strut
Nurses\strut
\end{minipage} & \begin{minipage}[t]{0.06\columnwidth}\raggedright\strut
Cerner\strut
\end{minipage} & \begin{minipage}[t]{0.06\columnwidth}\raggedright\strut
Documentation\strut
\end{minipage} & \begin{minipage}[t]{0.06\columnwidth}\raggedright\strut
Service delivery\strut
\end{minipage} & \begin{minipage}[t]{0.06\columnwidth}\raggedright\strut
Observational pre-post: completion of industry screening tools for
substance abuse, falls, and pressure injuries\strut
\end{minipage} & \begin{minipage}[t]{0.06\columnwidth}\raggedright\strut
1 year pre-and post-EMR implementation\strut
\end{minipage} & \begin{minipage}[t]{0.06\columnwidth}\raggedright\strut
Increased efficiency noted post-EMR implementation (nursing). Substance
use screening \& Waterlow screening increased after EMR but OMS (falls)
sightly decreased. Proportion with all 3 screens completed increased
post-EMR. Overall completion rate low (documentation completion cannot
be solved with technology alone).\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Bingham et al., 2021\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
VIC\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
5\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Quantitative\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Quaternary hospital -- medical / surgical wards\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Nurses\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
-\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Medication management Efficiency\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Service delivery\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Longitudinal quantitative direct observational pre and post time and
motion study - using validated Work Observation Method by Activity
Timing (WOMBAT)\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Immediately prior to, and 6 months post- EMR implementation\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Non-significant increase in proportion of time spent on direct care with
EMR vs paper. Nurses spent more tasks at bedside vs nursing station.A
significant proportion on time spent in transit and indirect work was
reduced post EMR. Less interruptions with EMR.\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Walker et al., 2020\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
QLD\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
5\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Quantitative\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Tertiary hospital\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Nurses\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Cerner\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Documentation Medication management Efficiency\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Service delivery\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Structured, continuous observation time and motion (STAMP)
observation\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Pre-post (2.5 years): 18 months post-EMR implementation\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
The move from paper-based records to integrated electronic health record
did not significantly change the amount of nursing time at the bedside,
or for the preparation and administration of ordered medications.
Significant increase in median time for documentation activities 18
months following rollout\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Schwarz et al., 2020\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
QLD\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
4\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Qualitative\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Hospital\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Allied health professionals\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Cerner\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Workforce satisfaction Usability Efficiency\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Workforce factors Service delivery\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Cross-sectional electronic survey\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Varied: largest site survey disseminated 7 months
post-implementation\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Minimal effect on workplace anxiety and stress, quality of patient care,
speed or efficiency. Inability to practice EMR hinders
implementation.\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.06\columnwidth}\raggedright\strut
Qian et al., 2020\strut
\end{minipage} & \begin{minipage}[t]{0.06\columnwidth}\raggedright\strut
QLD\strut
\end{minipage} & \begin{minipage}[t]{0.06\columnwidth}\raggedright\strut
4\strut
\end{minipage} & \begin{minipage}[t]{0.06\columnwidth}\raggedright\strut
Quantitative\strut
\end{minipage} & \begin{minipage}[t]{0.06\columnwidth}\raggedright\strut
Government funded regional drug and alcohol service\strut
\end{minipage} & \begin{minipage}[t]{0.06\columnwidth}\raggedright\strut
-\strut
\end{minipage} & \begin{minipage}[t]{0.06\columnwidth}\raggedright\strut
-\strut
\end{minipage} & \begin{minipage}[t]{0.06\columnwidth}\raggedright\strut
Documentation\strut
\end{minipage} & \begin{minipage}[t]{0.06\columnwidth}\raggedright\strut
Service delivery\strut
\end{minipage} & \begin{minipage}[t]{0.06\columnwidth}\raggedright\strut
Longitudinal retrospective descriptive data analysis of patient
records\strut
\end{minipage} & \begin{minipage}[t]{0.06\columnwidth}\raggedright\strut
40-month longitudinal post-EMR implementation\strut
\end{minipage} & \begin{minipage}[t]{0.06\columnwidth}\raggedright\strut
Error rates change over the course of the implementation phases but
eventually stabilise. Among all types of errors, service option error
accounted for more than half. Errors can help to tailor training.\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Eden et al., 2020\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
QLD\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
5\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Qualitative\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Large public tertiary university hospital (one of Australia's first
digital tertiary-care university hospitals)\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
All, including administrative staff, executive staff, \& patients\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
-\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Workforce satisfaction Usability Medication management\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Workforce factors Service delivery\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Single-site case study: semi-structured interviews + focus groups,
observations and documentation\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Immediately \& 6 months post-EMR implementation\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
New positive perception of EMR: improvements in accountability for care,
individual career development, time management, positive impacts re
secondary use of data, but mixed findings re availability of real-time
data. Many participants attributed success to the hospital's effective
preparation of the workforce, such as establishing a change-ready
culture.\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Westbrook et al., 2019\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Not Stated\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
3\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Quantitative\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Australian \& English teaching hospitals (wards of)\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Pharmacists\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
-\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Medication management Efficiency\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Service delivery\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Parallel, cross-country, direct observational time and motion studies,
using validated WOMBAT tool.\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
6-month + post-EMR implementation\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Increased time spent in medication review post-implementation. Workgroup
impacted pharmacists the same in Australia and UK -- supports increased
applicability of international studies to Australia.\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Devchand et al., 2019\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
VIC\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
4\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Quantitative\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Tertiary care referral hospital, ICU\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Infectious disease clinician, ICU consultants, Infectious disease
fellow, Pharmacists\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Cerner\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Medication safety Medication management\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Quality \& Safety Service delivery\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Cross-sectional prospective review: The ''5 moments of Antimicrobial
Prescribing'' to antimicrobial stewardship (AMS) compliance.\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
12 months pre- and post-implementation\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Integration of EMR with an ICT-AMS program allowed implementation of a
new AMS service, which was associated with a high clinician compliance
and improved antibiotic appropriateness. The ''5 moments of
Antimicrobial Prescribing'' metric provides a framework for measuring
AMS recommendation compliance.\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Baysari et al., 2019\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
NSW\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
4\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Qualitative\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Paediatric hospital\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Pharmacists\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Cerner\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Workforce satisfaction Medication management\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Workforce factors Service delivery\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Semi structured interviews\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
4 months pre- and 1-year post-EMR implementation\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
EMR implementation increased pharmacists' workload with additional
tasks: they were required to review more information and absorb
additional information, which increased stress.\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Van de Vreede, Clifford \& McGrath, 2018\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
VIC\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
4\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Qualitative\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
8 hospitals\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
All, including midwives\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Multiple\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Workforce satisfaction Usability Medication management Patient
safety\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Workforce factors\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Survey\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Various -- during \& post-EMR implementation\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
56\% agree EMRs reduced potential for medication errors and 58\%
consider EMRs have introduced new types of errors. Most common issues
raised re non-intuitive processes, time-consuming and overreliance on
technology. 48\% have observed, investigated, or reviewed near
misses/errors in EMRs. Errors identified (195 of 664) were incorrect
patient selection and incorrect dose scheduling, resulting in dose
duplication.\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Poon et al., 2018\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
VIC\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
1\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Mixed methods\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Hospital, outpatient dermatology clinic\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Medical staff\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Custom\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Workforce satisfaction Usability Efficiency Patient safety\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Workforce Factors Service delivery Quality \& Safety\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Qualitative data obtained by survey, combined with administrative data
(number of patients seen per session).\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
6-12 months post-EMR implementation\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Dermatology outpatient clinic numbers overall fell by 12\% compared with
the year before implementation - most attributed to reduction in number
of review patients.\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
McLain et al., 2017\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Not stated\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
3\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Qualitative\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
2 tertiary teaching hospitals\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
-\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
CSC MedChart (Turner)\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Documentation Medication management Medication safety\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Service delivery Quality \& Safety\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Retrospective descriptive study assessing paper and EMRs against
National Inpatient Medication Chart (NIMC) criteria\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Post-EMR implementation, time period not defined.\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Results appeared superior (re compliance with NIMC audit criteria) for
EMR charts than paper NIMCs for all parameters, except documentation of
patient weight and indication for medication. EMRs demonstrated a higher
rate of pharmacist review of medication orders than NIMC. EMRs increase
clarity of prescription, reducing errors associated with these, such as
error prone abbreviations (EPAs). EMRs almost entirely eliminate omitted
doses due to system design.\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.06\columnwidth}\raggedright\strut
Jiang et al., 2016\strut
\end{minipage} & \begin{minipage}[t]{0.06\columnwidth}\raggedright\strut
Not stated\strut
\end{minipage} & \begin{minipage}[t]{0.06\columnwidth}\raggedright\strut
5\strut
\end{minipage} & \begin{minipage}[t]{0.06\columnwidth}\raggedright\strut
Qualitative\strut
\end{minipage} & \begin{minipage}[t]{0.06\columnwidth}\raggedright\strut
Aged care\strut
\end{minipage} & \begin{minipage}[t]{0.06\columnwidth}\raggedright\strut
Managerial \& administration staff\strut
\end{minipage} & \begin{minipage}[t]{0.06\columnwidth}\raggedright\strut
-\strut
\end{minipage} & \begin{minipage}[t]{0.06\columnwidth}\raggedright\strut
Regulatory requirements\strut
\end{minipage} & \begin{minipage}[t]{0.06\columnwidth}\raggedright\strut
Quality \& Safety\strut
\end{minipage} & \begin{minipage}[t]{0.06\columnwidth}\raggedright\strut
Retrospective qualitative content data analysis using Aged Care
Accreditation Standards\strut
\end{minipage} & \begin{minipage}[t]{0.06\columnwidth}\raggedright\strut
Not defined\strut
\end{minipage} & \begin{minipage}[t]{0.06\columnwidth}\raggedright\strut
The proportion of residential aged care homes using EMRs that met all
accreditation standards was significantly higher than those using paper
records. Minimal positive benefit having EMR linking to
accreditation.\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Qian, Yu \& Hailey, 2015\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
NSW\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
4\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Mixed methods\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Aged care (high level of care)\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Medications staff: Registered nurses Enrolled Nurses Personal carers
with qualifications in medication management\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
-\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Usability Medication management Medication safety Efficiency\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Workforce factors Quality \& Safety Service delivery\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Time-motion observation case-study\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
18 months+ post-EMR implementation\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
EMRs improved compliance with documentation requirements, freedom from
signing twice, reducing types of medication documentation. No difference
in time found on various activities in a medication round using EMR vs
paper.\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Munyisia, Yu \& Hailey, 2014\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
NSW\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
4\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Quantitative\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Aged care (high level of care)\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Registered nurses, Enrolled nurses, Personal carers\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
-\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Efficiency\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Service delivery\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Observational work sampling (validated): prospective longitudinal cohort
case study\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Pre-post 25 month longitudinal study\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Successful introduction of an EMR system in a nursing home may not
interfere with nursing staff time on direct care duties.\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Mohan, Bishop \& Mallows, 2013\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
NSW\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
4\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Quantitative\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Tertiary teaching hospital\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Emergency department (ED) staff\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Cerner\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Patient outcomes Efficiency\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Quality \& Safety Service delivery\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Retrospective quantitative observational cross-sectional analytic study
(or case control as control was prior to EMR) using government KPIs for
public EDs\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
3 months post-EMR implementation\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Decreased ED KPIs 6 months post EMR implementation: statistically
significant increased wait time, total treatment time, increased in did
not wait rate, increased proportion of ambulance offload times longer
than 30 mins.\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Fairley et al., 2013\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
VIC\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
4\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Mixed methods\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Primary care (sexual health)\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Medical professionals / doctors Nurses\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
-\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Patient outcomes Workforce satisfaction Documentation Usability
Medication management Efficiency\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Quality \& Safety Service Delivery Workforce factors\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Pre-post observational study\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Not defined\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
EMR improved efficiency (5\% more consultations per hour) \& was popular
with staff and patients. No difference in quality of paper vs EMR
records.\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Wang, Yu \& Hailey 2013\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Not stated\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
5\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Qualitative\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Aged care\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Registered Nurses Enrolled Nurses\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
-\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Documentation\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Service delivery\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Retrospective audit to assess completeness, comprehensiveness \&
compliance with record standards\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Not defined\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Overall completeness and comprehensiveness rates of admission forms were
poor, but higher in electronic records vs paper. An increase in
comprehensiveness rate was found in the admission forms in the EMRs vs
paper (40\% vs 28.6\% p\textless{}0.01).\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Munyisia, Yu \& Hailey, 2012\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
NSW\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
5\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Mixed methods\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Aged care\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Aged care worker Primary care giver\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
-\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Workforce satisfaction Documentation Usability Efficiency\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Workforce factors Service delivery\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Longitudinal cohort case study using work-sampling technique.\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
6-12 months post-EMR implementation\strut
\end{minipage} & \begin{minipage}[t]{0.08\columnwidth}\raggedright\strut
Introduction of electronic documentation might not lead to increased
efficiency in documentation for caregivers. Primary caregivers found it
was easier to search for electronic records rather than manually
searching and retrieving paper ones. Longer documentation times were
associated with poor user interfaces. Mixed paper/electronic charting
may hinder realization of documentation efficiency.\strut
\end{minipage}\tabularnewline
\bottomrule
\end{longtable}

\section*{Discussion}

{\label{278652}}

The above themes are discussed in further detail below as part of the
thematic analysis. Some articles were re-grouped based on the
overarching themes.

\subsection*{Service delivery}

{\label{service-delivery}}

\subsubsection*{Documentation}

{\label{documentation}}

The major advantage of computerized documentation is legibility, which
was highlighted in multiple studies (Qian et al., 2015, 2020; Wang et
al., 2012).

Data entry in EMRs primarily differs from entering data in paper records
in that the former allows field selection options such as drop-down
menus. This introduces a new type of error that can harm the integrity
of documentation. Qian and colleagues (2020) discovered among all types
of errors, the service option error accounted for more than half.

Compared with their paper counterparts, an increased rate of completion
of documentation, such as discharge summaries, management plans, and
screening proformas in EMRs was demonstrated in studies by Curtis and
Witkowski (Curtis et al., 2021; Witkowski et al., 2021). Completion of
key demographic and patient data (patient weight, for example) impacts
subsequent workflows, including medication delivery, which was seen to
be a benefit in Firman's study (Firman et al., 2021).

Whereas overall completion tended to fare better with EMR vs paper
systems (Wang et al., 2012), caveats remain that other factors influence
completion rates. These include demands on the health workforce,
training, and motivation (Curtis et al., 2021, 2021; Wang et al., 2012).

Not all studies reported positive findings in this area. Fairley et al.
found no difference in quality between paper and electronic records
(Fairley et al., 2013).

Most research in this area highlighted that documentation practices and
standards can vary over time, which hinders the ability to draw
conclusions about improvements longitudinally. Most of the studies use a
nationally recognized paper form template to audit electronic records,
which may not be tailored for electronic format (McLain et al., 2017).
Studies reporting on documentation were often very domain-specific: for
example, research in the residential aged care setting conducted by Wang
et al., (2013) reported on factors that would not be applicable to a
surgical ward in a hospital, such as `membership'. A resident's
cognitive capacity may also contribute to incomplete data entry (E. N.
Munyisia et al., 2012; Wang et al., 2012).

Optimization and continuous education regarding documentation
requirements are key to improvements in this area, and it was also
recommended that further research be conducted to solidify the link
between documentation and outcomes of care (Curtis et al., 2021; E. N.
Munyisia et al., 2012; Wang et al., 2012).

\subsubsection*{Efficiency}

{\label{efficiency}}

Most studies reported increased efficiency after EMR implementation
(Bingham et al., 2021; Curtis et al., 2021; Fairley et al., 2013; E.
Munyisia et al., 2014; Witkowski et al., 2021). Several studies provided
statistically significant evidence of more patients being treated in the
same amount of time post-implementation compared to pre-implementation:
Witkowski et al., (2021) demonstrated a 19.5\% increase in patient
reviews; Fairley (2013) demonstrated 5\% more consultations per hour,
and Curtis (2021) showed nursing staff were caring for more patients of
a similar acuity in the same amount of time.

Negative impacts on efficiency were related to increased time taken for
medication reviews by pharmacists (Westbrook et al., 2019) and the use
of mixed paper and electronic documentation systems (E. N. Munyisia et
al., 2012; Walker et al., 2020).

Overall, recommendations centralized around guidance for standardization
and proformas, combined with a need for education and sustained
continuous improvement practices (Curtis et al., 2021; E. N. Munyisia et
al., 2012; Schwarz et al., 2020; Witkowski et al., 2021).

User-experience design improvements were recommended to enhance
documentation features for efficiency gains by Bingham et al., (2021),
Munyisia et al., (2014), Qian et al., (2015, 2020), and Walker et al.,
(2020), as poorly designed user interfaces can result in longer
documentation times (E. Munyisia et al., 2014).

Limitations within this topic were mostly due to the observational
nature of the studies, which often limits sample size and
generalizability, even when standardized techniques such as STAMP and
WOMBAT are used (Bingham et al., 2021; E. Munyisia et al., 2014; Qian et
al., 2015; Walker et al., 2020; Westbrook et al., 2019). The presence of
an observer in time and motion studies could have led to the Hawthorne
effect, though this was noted in the limitations sections of these
papers, and steps taken to minimize the effects (Mohan et al., 2013; E.
Munyisia et al., 2014; Walker et al., 2020).

\subsubsection*{Medication management}

{\label{medication-management}}

The majority of papers reporting the impact of EMRs on medication
management offered mixed findings and were often reported impartially
(Baysari et al., 2019; Bingham et al., 2021; Firman et al., 2021; McLain
et al., 2017). Several studies demonstrated a higher rate of pharmacist
review of medication orders in EMR systems than in paper systems
(Baysari et al., 2019; Firman et al., 2021; McLain et al., 2017;
Westbrook et al., 2019). However, few reported on whether this was a
positive change or a negative one. Baysari et al., (2019) reported this
had negative impacts on pharmacy staff wellbeing.

Medication management seems to be uniquely impacted by the transition to
an EMR, in that most studies report cannibalization of some tasks which
results in either no change in completion to perform tasks, or in
several instances, an increase in time required to perform medication
management activities (Baysari et al., 2019; Bingham et al., 2021;
Westbrook et al., 2019). This led to the suggestion that regular reviews
of workflow planning post-EMR rollout are crucial for a safer and more
streamlined transition from paper to digital systems (Baysari et al.,
2019).

EMRs can, however, support additional initiatives to improve medication
management: one study demonstrated improved antimicrobial stewardship
compliance using a modified add-on to an existing EMR (Devchand et al.,
2019).

\subsection*{Quality and safety}

{\label{quality-and-safety}}

\subsubsection*{Patient outcomes.}

{\label{patient-outcomes.}}

Few shortlisted studies reported on patient outcomes and findings were
mixed. The most recent study in this review reported a clinically
significant, sustained 22\% decrease in in-hospital mortality post-EMR
implementation and supports ongoing investment in these systems (South
et al., 2022).

However, an older study reported a statistically significant
deterioration in all ED KPIs (including ambulance offload times
\textgreater{}30 mins and total treatment time) (Mohan et al., 2013).

Mixed impacts on patient care were reported by Wynter and colleagues
(Wynter et al., 2021).

\subsubsection*{Patient satisfaction}

{\label{patient-satisfaction}}

Studies considering the patient satisfaction of care related to EMR
implementation were rare. One such study was conducted at a large urban
primary care sexual health Centre in 2013 by Fairley et al. and found
that patient satisfaction with their care was unchanged following EMR
implementation. Given the increasing emphasis on the consumer viewpoint
in healthcare transformation (Australian Commission on Safety and
Quality in Health Care, 2022), one would expect to see this perspective
being captured in future EMR research.

\subsubsection*{Medication safety}

{\label{medication-safety}}

This scoping review identified EMRs almost eliminate certain types of
medication errors such as error-prone abbreviations (EPAs), omitted
doses, and errors related to clarity of prescriptions (McLain et al.,
2017; Qian et al., 2015; Van de Vreede et al., 2018). However, they
introduce other errors, such as incorrect patient selection and
incorrect dose scheduling resulting in dose duplication (Van de Vreede
et al., 2018).

Several authors state EMR design changes could help mitigate some of
these new errors by modifying drop-down lists, for example. The same
authors argued that electronic systems help identify errors easier than
paper-based systems, which can drive quality and safety improvement
projects (Qian et al., 2015; Van de Vreede et al., 2018).

McLain et al., (2017) highlighted national medication audit criteria
need to be adapted to electronic systems, as the current criteria were
designed for paper-based systems and fall short in areas assessing EMRs.
Since the publication of this research the Australian Commission on
Safety and Quality in Health Care (ACSQHC) has revised its audit
criteria, but are still not suitable for auditing EMRs (Australian
Commission on Safety and Quality in Health Care, 2018). The ACSQHC has,
however, published guidance on the display of on-screen medicines
information (Australian Commission on Safety and Quality in Health Care,
2017), and has also produced a comprehensive guide to the safe
implementation of EMRs (Australian Commission on Safety and Quality in
Health Care, 2019).

Reliability was compromised in some of these studies when mixed paper
and electronic medication systems were in use (Dabliz et al., 2021; Qian
et al., 2015).

\subsubsection*{Regulatory requirements}

{\label{regulatory-requirements}}

One study reported on EMRs as contributing to compliance with
Residential Aged Care Accreditation standards (Jiang et al., 2016).
However, the link between accreditation and patient safety and quality
of care has recently been contested (Duckett, 2018a).

\subsection*{Workforce factors}

{\label{workforce-factors}}

\subsubsection*{Workforce satisfaction}

{\label{workforce-satisfaction}}

Different user groups reported different levels of satisfaction with
EMRs (Baysari et al., 2019; Dabliz et al., 2021; Lloyd, 2021; Schwarz et
al., 2020; Wynter et al., 2021).

Nurses generally had positive acceptance for EMRs (Dabliz et al., 2021;
Fairley et al., 2013; Lloyd, 2021; E. N. Munyisia et al., 2012; Van de
Vreede et al., 2018), whereas pharmacists and medical staff were more
likely to report issues with automation (Dabliz et al., 2021), safety
risks (Van de Vreede et al., 2018), and increased workload (Baysari et
al., 2019).

Baysari and colleagues (2019) discovered pharmacists are often the
cohort teaching other healthcare staff how to use the system and
reviewing additional information as part of a changed workload. This can
increase stress and anxiety in the pharmacy workforce (Baysari et al.,
2019).

This led to the recommendation that further research aimed at different
user groups is vital to target education and improve user experience
pathways (Dabliz et al., 2021; Lloyd, 2021).

Sample sizes were a common limitation in this topic (Baysari et al.,
2019; Lloyd, 2021; Schwarz et al., 2020; Wynter et al., 2021), as was a
lack of generalizability due to system brand (Baysari et al., 2019;
Dabliz et al., 2021; Fairley et al., 2013).

\subsubsection*{Usability}

{\label{usability}}

Usability varies between user groups due to their workflows. Nurses and
medical professionals have different experiences with EMR usability,
which also depends on the area of work and which feature is measured.
This often hinders the generalizability of findings (Lloyd et al.,
2021).

The greatest usability issues were related to protocol-mandated care,
whereby if a user wanted to order outside of an order set, for example,
this created difficulty (Dabliz et al., 2021).

Well-designed user interfaces can `\ldots{}reduce the mental energy
required searching for important information and the time taken to
achieve this', (Dabliz et al., 2021) whereas poorly designed interfaces
were associated with increased levels of dissatisfaction and longer
times to perform tasks (E. N. Munyisia et al., 2012).

Both Lloyd et al., and Dabliz et al., (2021) advocate the need for
multidisciplinary usability studies to represent different user groups
and their associated environment.

Lloyd et al., (2021) promoted the use of the NuHISS tool to measure the
usability of EMRs in the Australian context.

When an interface is less than optimal, all research in this area
focused on the need for continuous improvement, utilizing lessons
learned, and support for staff (Dabliz et al., 2021; Lloyd et al., 2021;
E. N. Munyisia et al., 2012).

\subsection*{Limitations}

{\label{limitations}}

The limitations of this study were that only English language papers
were included due to the assumption that Australian research would be
conducted and published in the English language. There is a very small
possibility that researchers have assessed EMR implementation in the
Australian context but have published in another language.

Grey literature, scoping, and systematic reviews were also excluded
based on constraints and compatibility with the quality checklist used,
meaning some industry data could have been missed.

Only a single researcher with a time constraint of 14 weeks was able to
perform this scoping review. Hence personal researcher bias cannot be
excluded from this study.

The MMAT checklist was used as a broad indicator of quality to
contribute to answering the research question. Grading literature is not
within the typical methodology of scoping reviews, so should be
interpreted with caution (PRISMA, 2021; Rethlefsen et al., 2021; Subbe
et al., 2021).

\section*{Conclusion}

{\label{444111}}

This is the first scoping review, to the author's knowledge, to
systematically determine how EMR implementation is evaluated in the
Australian context. This is in response to government reports exposing a
current lack of evaluation frameworks to assess EMRs, and the fact that
EMRs are a relatively new addition to the Australian healthcare system
compared to other nations, primarily occurring over the past decade
(Duckett, 2018a; Jedwab et al., 2019). Previous reviews have often
either focused on a particular topic (Subbe et al., 2021) or workforce
group (Jedwab et al., 2019), and refer to international data, which is
often stated as a limitation and/or knowledge gap in these studies.

This scoping review rigorously analyzed the literature and out of the 25
articles found, the themes that were most evident were in quality and
safety, and service delivery, though in recent years there has been an
increase in studies reporting on workforce factors (satisfaction and
usability). Workforce factors have been identified as important by
authors such as Lloyd et al (2021)and Dabliz et al (2021), since
different workforce groups are likely to report different outcomes.

Studies overall were mostly qualitative in nature, with only 16\% being
mixed methods, and just over a third being quantitative. Only seven of
the 25 shortlisted articles were pre-post studies, reflecting the
difficulty in designing and implementing such studies.

To date, most health workforce groups have been evenly represented,
though there is limited research on how EMRs affect midwives and allied
health professionals.

The system in use was not consistently referenced in the literature. If
it was, the brand was most likely to be Cerner (Millennium). The
differing brands of EMRs were cited as common limitations in most
studies, restricting generalisability. Generalisability was also often
restricted due to the specialty and/or setting under scrutiny.

Healthcare is a complex system, with multiple disciplines and
workstreams. EMRs traverse all these systems, yet there is no consistent
framework to determine if EMRs present value for money, or indeed
improve patient care. An evaluative framework that incorporates one or
more validated tools such as the WOMBAT or STAMP for time and motion
studies, and NuHISS for user experience could be a recommendation.

This review solidifies the following benefits of EMRs:

\begin{itemize}
\tightlist
\item
  They provide a huge advantage regarding legibility and ease of access
  to patient records. This can reduce errors associated with paper
  records, including poor readability and abbreviation-prone errors.
\item
  EMRs have been shown to generally improve the efficiency of multiple
  workflows, except for pharmacists.
\item
  Most worker dissatisfaction with EMRs was related to change management
  and the EMR interface, though this varied across disciplines.
\end{itemize}

Research gaps include lack of patient viewpoint, non-medication-related
patient safety outcomes (e.g. mortality rate, improvements in clinical
outcomes), and how usability and EMR design impacts patient outcomes.

This review demonstrates the need to address the above research gaps,
and to ultimately design uniform and validated outcome measures and
frameworks to drive consistency across EMR evaluations. This will ensure
benefits are tracked, realized, and maintained. Overall, the articles in
this scoping review provide evidence to support the continued rollout of
EMR systems across Australia, and have even drawn parallels with
international findings (Lloyd, 2021; Westbrook et al., 2019). This
indicates Australian policymakers could rely on international evidence,
as well as that conducted in Australia. Whether the current selection of
evidence is sufficient to guide policy or digital strategy in Australian
healthcare remains to be seen.

\subsection*{Future research}

{\label{future-research}}

Future research could be to use the same thematic analysis applied to
global literature (for example, the U.S \& Canada), and compare with
findings of this study to determine if outcomes and themes are the same.
This would permit the scientific community to apply with more certainty
non-Australian research to Australian healthcare settings, and would
also allow researchers to determine the proportion of Australian
evidence in relation to the worldwide evidence base.

\subsection*{Reference List}

{\label{reference-list}}

1. ACT Government (Health). (2019). \emph{Digital Health Strategy
2019-2029,} (pp. 1--19). ACT Government.
\url{https://health.act.gov.au/sites/default/files/2019-05/Digital\%20Health\%20Strategy\%202019-2029.pdf}

2. Australian Commission on Safety and Quality in Health Care, (ACSQHC).
(2017). \emph{National guidelines for on-screen display of medicines
information} (pp. 1--78) {[}Guide, user guide or guidelines{]}.
Australian Commission on Safety and Quality in Health Care.
\url{https://www.safetyandquality.gov.au/publications-and-resources/resource-library/national-guidelines-screen-display-medicines-information}

3. Australian Commission on Safety and Quality in Health Care, (ACSQHC).
(2018). \emph{National Standard Medication Chart (NSMC) Audit Guide} (p.
5) {[}Guide, user guide or guidelines{]}. Australian Commission on
Safety and Quality in Health Care.
\url{https://www.safetyandquality.gov.au/sites/default/files/2019-05/nsmc-audit-guide-for-hospitals-v1.0-august-2018\_1.pdf}

4. Australian Commission on Safety and Quality in Health Care, (ACSQHC).
(2019). \emph{Electronic Medication Management Systems A guide to safe
implementation} (Guide, Guidelines Version 3.1; pp. 1--168). Australian
Commission on Safety and Quality in Health Care.
\url{https://www.safetyandquality.gov.au/sites/default/files/2019-08/sq19-030\_emms\_functional\_requirements\_acc\_01n\_5\_aug\_2019.pdf}

5. Australian Commission on Safety and Quality in Health Care, (ACSQHC).
(2022). Partnering with Consumers Standard. \emph{Standards} .
\url{https://www.safetyandquality.gov.au/standards/nsqhs-standards/partnering-consumers-standard}

6. Australian Digital Health Agency. (2021). \emph{Australia's National
Digital Health Strategy} (pp. 1--63).
\url{https://www.digitalhealth.gov.au/sites/default/files/2020-11/Australia\%27s\%20National\%20Digital\%20Health\%20Strategy\%20-\%20Safe\%2C\%20seamless\%20and\%20secure.pdf}

7. Australian Government Department of Health. (2022). \emph{2020-25
National Health Reform Agreement (NHRA)} {[}Internet{]}. Commonwealth of
Australia.
\url{https://www.health.gov.au/initiatives-and-programs/2020-25-national-health-reform-agreement-nhra}

8. Australian Institute of Health and Welfare, (AIHW). (2020). What is
digital health? {[}Government Website{]}. \emph{Digital Health} .
\url{https://www.aihw.gov.au/reports/australias-health/digital-health}

9. Baysari, M. T., Hardie, R., Barclay, P., \& Westbrook, J. I. (2019).
Effects of an electronic medication management system on pharmacists'
work in a paediatric hospital. \emph{Journal of Pharmacy Practice and
Research} , \emph{49} (4), 317--323.
\url{https://doi.org/10.1002/jppr.1507}

10. Bingham, G., Tong, E., Poole, S., Ross, P., \& Dooley, M. (2021). A
longitudinal time and motion study quantifying how implementation of an
electronic medical record influences hospital nurses' care
delivery.\emph{International Journal of Medical Informatics (Shannon,
Ireland)} ,\emph{153} , 104537--104537.
\url{https://doi.org/10.1016/j.ijmedinf.2021.104537}

11. Curtis, K., Qian, S., Yu, P., White, J., Ruperto, K., Balzer, S., \&
Munroe, B. (2021). Does electronic medical record redesign increase
screening of risk for pressure injury, falls and substance use in the
Emergency Department? An implementation evaluation. \emph{Australasian
Emergency Care} , \emph{24} (1), 20--27.
\url{https://doi.org/10.1016/j.auec.2020.04.002}

12. Dabliz, R., Poon, S. K., Ritchie, A., Burke, R., \& Penm, J. (2021).
Usability evaluation of an integrated electronic medication management
system implemented in an oncology setting using the unified theory of
the acceptance and use of technology. \emph{BMC Medical Informatics and
Decision Making} , \emph{21} (1), 4--4.
\url{https://doi.org/10.1186/s12911-020-01348-y}

13. Devchand, M., Stewardson, A. J., Urbancic, K. F., Khumra, S.,
Mahony, A. A., Walker, S., Garrett, K., Lindsay Grayson, M., \&
Trubiano, J. A. (2019). Outcomes of an electronic medical record
(EMR)-driven intensive care unit (ICU)-antimicrobial stewardship (AMS)
ward round: Assessing the Five Moments of Antimicrobial Prescribing.
\emph{Infection Control and Hospital Epidemiology} , \emph{40} (10),
1170--1175. \url{https://doi.org/10.1017/ice.2019.218}

14. Duckett, S. (2018a). To keep patients safe in hospitals, the
accreditation system needs an overhaul. \emph{Health and Aged Care
{[}Grattan Institute{]}} .
\url{https://grattan.edu.au/news/to-keep-patients-safe-in-hospitals-the-accreditation-system-needs-an-overhaul/}

15. Duckett, S. (2018b). \emph{What can we learn from Australia's My
Health Record experience?} {[}Internet{]}. Grattan Institute.
\url{https://grattan.edu.au/news/what-can-we-learn-from-australias-my-health-record-experience/}

16. Fairley, C. K., Vodstrcil, L. A., Huffam, S., Cummings, R., Chen, M.
Y., Sze, J. K., Fehler, G., Bradshaw, C. S., Schmidt, T., Berzins, K.,
\& Hocking, J. S. (2013). Evaluation of Electronic Medical Record (EMR)
at Large Urban Primary Care Sexual Health Centre. \emph{PloS One}
,\emph{8} (4), e60636--e60636.
\url{https://doi.org/10.1371/journal.pone.0060636}

17. Firman, P., Whitfield, K., Tan, K., Clavarino, A., \& Hay, K.
(2021). The impact of an electronic hospital system on therapeutic drug
monitoring. \emph{Journal of Clinical Pharmacy and Therapeutics}
,\emph{46} (6), 1613--1621. \url{https://doi.org/10.1111/jcpt.13497}

18. Government of Western Australia Department of Health. (2019).
\emph{WA Health Digital Strategy 2020-2030} . Government of Western
Australia Department of Health.
\url{https://ww2.health.wa.gov.au/~/media/Files/Corporate/Reports-and-publications/Digital-strategy/Digital-Strategy-2020-2030.pdf}

19. Griffith University. (2022). \emph{Getting started on your health
project {[}Powerpoint slides{]}} (7706MED Health Project, p. 16).
Griffith University,.

20. Hong, Quan Nha, Fabregues, Sergi, Bartlett, Gillian, Boardman,
Felicity, Cargo, Margaret, Dagenais, Pierre, Gagnon, Marie-Pierre,
Griffiths, Frances, Nicolau, Belinda, O'Cathain, Alicia, Rousseau,
Marie-Claude, \& Vedel, Isabelle. (2018). The Mixed Methods Appraisal
Tool (MMAT) version 2018 for information professionals and researchers.
\emph{Education for Information} , \emph{34} , 1--7.
\url{https://doi.org/10.3233/EFI-180221}

21. Hong, Quan Nha, Pluye, Pierre, Fabregues, Sergi, Bartlett, Gillian,
Boardman, Felicity, Cargo, Margaret, Dagenais, Pierre, Gagnon,
Marie-Pierre, Griffiths, Frances, Nicolau, Belinda, O'Cathain, Alicia,
Rousseau, Marie-Claude, \& Vedel, Isabelle. (2018). \emph{Mixed Methods
Appraisal Tool (MMAT)v, version 2018} . Industry Canada.
\url{http://mixedmethodsappraisaltoolpublic.pbworks.com/w/file/fetch/127916259/MMAT\_2018\_criteria-manual\_2018-08-01\_ENG.pdf}

22. IGI Global. (2022). What is Electronic patient record (EPR).
\emph{What Is Electronic Patient Record (EPR)} .
\url{https://www.igi-global.com/dictionary/ict-healthcare-management-developments-applications/9490}

23. Jedwab, R. M., Chalmers, C., Dobroff, N., \& Redley, B. (2019).
Measuring nursing benefits of an electronic medical record system: A
scoping review. \emph{Collegian (Royal College of Nursing, Australia)}
,\emph{26} (5), 562--582.
\url{https://doi.org/10.1016/j.colegn.2019.01.003}

24. Jiang, T., Yu, P., Hailey, D., Ma, J., \& Yang, J. (2016). The
Impact of Electronic Health Records on Risk Management of Information
Systems in Australian Residential Aged Care Homes. \emph{Journal of
Medical Systems} , \emph{40} (9), 1--7.
\url{https://doi.org/10.1007/s10916-016-0553-y}

25. Lloyd, S. (2021). A National Survey of EMR Usability: Comparisons
between medical and nursing professions in the hospital and primary care
sectors in Australia and Finland. \emph{International Journal of Medical
Informatics} . \url{https://doi.org/10.1016/j.ijmedinf.2021.104535}

26. Lloyd, S., Long, K., Oshni Alvandi, A., Di Donato, J., Probst, Y.,
Roach, J., \& Bain, C. (2021). A National Survey of EMR Usability:
Comparisons between medical and nursing professions in the hospital and
primary care sectors in Australia and Finland. \emph{International
Journal of Medical Informatics (Shannon, Ireland)} , \emph{154} ,
104535--104535. \url{https://doi.org/10.1016/j.ijmedinf.2021.104535}

27. McGraw-Hill. (2002). EMR. In \emph{McGraw-Hill Concise Dictionary of
Modern Medicine} . McGraw-Hill.
\url{https://medical-dictionary.thefreedictionary.com/EMR}

28. McLain, S. E., Pogue, T. L., Richardson, K. L., Westcoast, P.,
Gilbert, A. V., Heywood, M. J., Welch, S. A., Keily, J. V., \& Patel, B.
K. (2017). Applying the National Inpatient Medication Chart audit to
electronic medication management systems: What does it tell
us?\emph{Journal of Pharmacy Practice and Research} , \emph{47} (2),
96--102. \url{https://doi.org/10.1002/jppr.1209}

29. Mohan, M. K., Bishop, R. O., \& Mallows, J. L. (2013). Effect of an
electronic medical record information system on emergency department
performance. \emph{Medical Journal of Australia} , \emph{198} (4),
201--204. \url{https://doi.org/10.5694/mja12.10499}

30. Munn, Z., Peters, M. D. J., Stern, C., Tufanaru, C., McArthur, A.,
\& Aromataris, E. (2018). Systematic review or scoping review? Guidance
for authors when choosing between a systematic or scoping review
approach.\emph{BMC Medical Research Methodology} , \emph{18} (1), 143.
\url{https://doi.org/10.1186/s12874-018-0611-x}

31. Munyisia, E. N., Yu, P., \& Hailey, D. (2012). The impact of an
electronic nursing documentation system on efficiency of documentation
by caregivers in a residential aged care facility. \emph{Journal of
Clinical Nursing} , \emph{21} (19pt20), 2940--2948.
\url{https://doi.org/10.1111/j.1365-2702.2012.04157.x}

32. Munyisia, E., Yu, P., \& Hailey, D. (2014). The effect of an
electronic health record system on nursing staff time in a nursing home:
A longitudinal cohort study. \emph{Australasian Medical Journal}
,\emph{7} (7), 285--293. \url{https://doi.org/10.4066/AMJ.2014.2072}

33. Nelson, R., \& Staggers, N. (2016). \emph{Health Informatics An
Interprofessional Approach} (2nd ed.). Elsevier.

34. Northern Territory Government. (2020). \emph{Strengthening our
Health System Strategy (2020-2025)} (pp. 1--19). Health Network Northern
Territory.
\url{https://www.ntphn.org.au/wp-content/uploads/2021/08/Health-System-Strategy-Updated-18-11-2020-1.pdf}

35. NSW Government. (2021). \emph{Digital NSW Designing our Digital
Future}(pp. 1--14). NSW Government.
\url{https://www.digital.nsw.gov.au/sites/default/files/DigitalStrategy.pdf?msclkid=db80b285b79511ec98af6520f8efb384}

36. Pearce, R., \& Whyte, I. (2018). Electronic medication management:
Is it a silver bullet?~\emph{Aust Prescr} ,~\emph{41} , 32--33.
\url{https://doi.org/10.18773/austprescr.2018.012}

37. Peters, M. D. J., Marnie, C., Colquhoun, H., Garritty, C. M.,
Hempel, S., Horsley, T., Langlois, E. V., Lillie, E., O'Brien, K. K.,
Tuncalp, O., Wilson, M. G., Zarin, W., \& Tricco, A. C. (2021). Scoping
reviews: Reinforcing and advancing the methodology and application.
\emph{BMC} ,\emph{10} (263).
\url{https://doi.org/10.1186/s13643-021-01821-3}

38. PRISMA. (2021). \emph{PRISMA for Scoping Reviews} {[}Internet{]}.
PRISMA. \url{http://www.prisma-statement.org/Extensions/ScopingReviews}

39. Qian, S., Munyisia, E., Reid, D., Hailey, D., Pados, J., \& Yu, P.
(2020). Trend in data errors after the implementation of an electronic
medical record system: A longitudinal study in an Australian regional
Drug and Alcohol Service. \emph{International Journal of Medical
Informatics (Shannon, Ireland)} , \emph{144} , 104292--104292.
\url{https://doi.org/10.1016/j.ijmedinf.2020.104292}

40. Qian, S., Yu, P., \& Hailey, D. M. (2015). The impact of electronic
medication administration records in a residential aged care
home.\emph{International Journal of Medical Informatics (Shannon,
Ireland)} ,\emph{84} (11), 966--973.
\url{https://doi.org/10.1016/j.ijmedinf.2015.08.002}

41. Queensland Government. (2015). \emph{21st Century Healthcare eHealth
Investment Strategy} (p. 10). Queensland Health.
\url{https://www.health.qld.gov.au/\_\_data/assets/pdf\_file/0031/442939/ehealthinvestmentstrategy.pdf}

42. Rethlefsen, M. L., Kirtley, S., Waffenschmidt, S., Ayala, A. P.,
Moher, D., Page, M. J., Koffel, J. B., \& PRISMA-S, G. (2021). PRISMA-S:
an extension to the PRISMA Statement for Reporting Literature Searches
in Systematic Reviews. \emph{BMC} , \emph{10} (39).
\url{https://doi.org/10.1186/s13643-020-01542-z}

43. Rockliff, J. (2021). \emph{Digital strategy to help better deliver
health services} . Tasmanian Government Department of Health,.
\url{https://www.health.tas.gov.au/news/news/digital-strategy-help-better-deliver-health-services}

44. Schwarz, M., Coccetti, A., Draheim, M., \& Gordon, G. (2020).
Perceptions of allied health staff of the implementation of an
integrated electronic medical record across regional and metropolitan
settings. \emph{Australian Health Review} , \emph{44} (6), 965--972.
\url{https://doi.org/10.1071/AH19024}

45. South, M., Cheng, D., Andrew, L., Egan, N., \& Carlin, J. (2022).
Decreased in-hospital mortality rate following implementation of a
comprehensive electronic medical record system. \emph{Journal of
Paediatrics and Child Health} , \emph{58} (2), 332--336.
\url{https://doi.org/10.1111/jpc.15718}

46. Subbe, C. P., Tellier, G., \& Barach, P. (2021). Impact of
electronic health records on predefined safety outcomes in patients
admitted to hospital: A scoping review. \emph{BMJ Open} , \emph{11} (1),
e047446--e047446. \url{https://doi.org/10.1136/bmjopen-2020-047446}

47. Van de Vreede, M., Clifford, J., \& McGrath, A. (2018). Staff
experience and perceptions of the safety and risks of electronic
medication management systems in Victorian public hospitals.
\emph{Journal of Pharmacy Practice and Research} , \emph{48} (1),
18--25. \url{https://doi.org/10.1002/jppr.1327}

48. Victorian Auditor-General's Office (VAGO). (2013). \emph{Clinical
ICT systems in the Victorian Public Health Sector {[}Powerpoint
presentation{]}} (pp. 1--20). VAGO.
\url{https://www.audit.vic.gov.au/sites/default/files/20131030-Clinical-ICT-Systems-presentation.pdf\#:~:text=Clinical\%20ICT\%20systems\%20in\%20the\%20Victorian\%20public\%20health,ICT\%20systems.\%20Findings\%20\%E2\%80\%93\%20Planning\%20and\%20implementation\%209}

49. Walker, R. M., Burmeister, E., Jeffrey, C., Birgan, S., Garrahy, E.,
Andrews, J., Hada, A., \& Aitken, L. M. (2020). The impact of an
integrated electronic health record on nurse time at the bedside: A
pre-post continuous time and motion study. \emph{Collegian (Royal
College of Nursing, Australia)} , \emph{27} (1), 63--74.
\url{https://doi.org/10.1016/j.colegn.2019.06.006}

50. Wang, N., Yu, P., \& Hailey, D. (2012). Description and comparison
of quality of electronic versus paper-based resident admission forms in
Australian aged care facilities. \emph{International Journal of Medical
Informatics (Shannon, Ireland)} , \emph{82} (5), 313--324.
\url{https://doi.org/10.1016/j.ijmedinf.2012.11.011}

51. Westbrook, J. I., Li, L., Shah, S., Lehnbom, E. C., Prgomet, M.,
Schofield, B., Cresswell, K., Slee, A., Coleman, J. J., McCloughan, L.,
\& Sheikh, A. (2019). A cross-country time and motion study to measure
the impact of electronic medication management systems on the work of
hospital pharmacists in Australia and England. \emph{International
Journal of Medical Informatics (Shannon, Ireland)} , \emph{129} ,
253--259. \url{https://doi.org/10.1016/j.ijmedinf.2019.06.011}

52. Witkowski, C., Kimmel, L., Edwards, E., \& Cosic, F. (2021).
Comparison of the quality of documentation between electronic and paper
medical records in orthopaedic trauma patients. \emph{Australian Health
Review} . \url{https://doi.org/10.1071/AH2112}

53. World Health Organization \& International Telecommunication Union.
(2012). \emph{National eHealth strategy toolkit.} (ISBN: 9789241548465;
pp. 1--223). WHO. \url{https://apps.who.int/iris/handle/10665/75211}

54. Wynter, K., Holton, S., Nguyen, L., Sinnott, H., Wickramasinghe, N.,
Crowe, S., \& Rasmussen, B. (2021). Nurses' and midwives' experiences of
the first phase of the implementation of an electronic medical records
system. \emph{Australian Health Review} .
\url{https://doi.org/10.1071/AH21118}

55. Zurynski, Y., Ellis, L. A., Tong, H. L., Laranjo, L., Clay-Williams,
R., Testa, L., Meulenbroeks, I., Turton, C., \& Sara, G. (2021).
Implementation of electronic medical records in mental health settings:
Scoping review. \emph{JMIR Mental Health} , \emph{8} (9),
e30564--e30564. \url{https://doi.org/10.2196/30564}

\section*{Appendices}

{\label{626133}}

\subsection*{Appendix 1: List of search term
combinations}

{\label{appendix-1-list-of-search-term-combinations}}

Between each keyword the ``AND'' Boolean operator was used. For example,
the first line search term entered into the search engine would be EMR
AND Evaluat* AND Australia*. Similarly, the fifth line would read EMR
AND ``outcome measure'' AND Australia*.\selectlanguage{english}
\begin{longtable}[]{@{}l@{}}
\toprule
EMR Evaluat* Australia*\tabularnewline
\midrule
\endhead
EMR success Australia*\tabularnewline
EMR adopt* Australia*\tabularnewline
EMR outcome* Australia*\tabularnewline
EMR outcome measure* Australia*\tabularnewline
EMR performance measure Australia*\tabularnewline
EMR impact assess* Australia*\tabularnewline
EMR monitor Australia*\tabularnewline
EMR success rate Australia*\tabularnewline
EMR perception Australia*\tabularnewline
EMR key performance indicator* Australia*\tabularnewline
EMR KPI* Australia*\tabularnewline
EMR impact* Australia*\tabularnewline
EMR measure* Australia*\tabularnewline
EMR result* Australia*\tabularnewline
EMR positive Australia*\tabularnewline
EMR reduc* Australia*\tabularnewline
EMR error* Australia*\tabularnewline
EMR error rate Australia*\tabularnewline
electronic medical record* Evaluat* Australia*\tabularnewline
electronic medical record* success Australia*\tabularnewline
electronic medical record* adopt* Australia*\tabularnewline
electronic medical record* outcome* Australia*\tabularnewline
electronic medical record* outcome measure* Australia*\tabularnewline
electronic medical record* performance measure Australia*\tabularnewline
electronic medical record* impact assess* Australia*\tabularnewline
electronic medical record* success Australia*\tabularnewline
electronic medical record* monitor Australia*\tabularnewline
electronic medical record* success rate Australia*\tabularnewline
electronic medical record* perception Australia*\tabularnewline
electronic medical record* key performance indicator*
Australia*\tabularnewline
electronic medical record* KPI* Australia*\tabularnewline
electronic medical record* impact* Australia*\tabularnewline
electronic medical record* measure* Australia*\tabularnewline
electronic medical record*benefit* Australia*\tabularnewline
electronic medical record* positive Australia*\tabularnewline
electronic medical record* reduc* Australia*\tabularnewline
electronic medical record* error* Australia* implement*\tabularnewline
electronic medical record* efficiency* Australia*
implement*\tabularnewline
electronic medical record* error rate Australia*\tabularnewline
EHR Evaluat* Australia*\tabularnewline
EHR success Australia*\tabularnewline
EHR adopt* Australia*\tabularnewline
EHR outcome* Australia*\tabularnewline
EHR outcome measure* Australia*\tabularnewline
EHR performance measure Australia*\tabularnewline
EHR impact assess* Australia*\tabularnewline
EHR success Australia*\tabularnewline
EHR monitor Australia*\tabularnewline
EHR success rate Australia*\tabularnewline
EHR perception Australia*\tabularnewline
EHR key performance indicator* Australia*\tabularnewline
EHR KPI* Australia*\tabularnewline
EHR impact* Australia*\tabularnewline
EHR measure* Australia*\tabularnewline
EHR result* Australia*\tabularnewline
EHR positive Australia*\tabularnewline
EHR reduc* Australia*\tabularnewline
EHR error* Australia*\tabularnewline
EHR error rate Australia*\tabularnewline
electronic health record Evaluat* Australia*\tabularnewline
electronic health record success Australia*\tabularnewline
electronic health record adopt* Australia*\tabularnewline
electronic health record outcome* Australia*\tabularnewline
electronic health record outcome measure* Australia*\tabularnewline
electronic health record performance measure Australia*\tabularnewline
electronic health record impact assess* Australia*\tabularnewline
electronic health record success Australia*\tabularnewline
electronic health record monitor Australia*\tabularnewline
electronic health record success rate Australia*\tabularnewline
electronic health record perception Australia*\tabularnewline
electronic health record key performance indicator*
Australia*\tabularnewline
electronic health record KPI* Australia*\tabularnewline
electronic health record impact* Australia*\tabularnewline
electronic health record measure* Australia*\tabularnewline
electronic health record result* Australia*\tabularnewline
electronic health record positive Australia*\tabularnewline
electronic health record reduc* Australia*\tabularnewline
electronic health record error* Australia*\tabularnewline
electronic health record error rate Australia*\tabularnewline
digital health record Evaluat* Australia*\tabularnewline
digital health record success Australia*\tabularnewline
digital health record adopt* Australia*\tabularnewline
digital health record outcome* Australia*\tabularnewline
digital health record outcome measure* Australia*\tabularnewline
digital health record performance measure Australia*\tabularnewline
digital health record impact assess* Australia*\tabularnewline
digital health record success Australia*\tabularnewline
digital health record monitor Australia*\tabularnewline
digital health record success rate Australia*\tabularnewline
digital health record perception Australia*\tabularnewline
digital health record key performance indicator*
Australia*\tabularnewline
digital health record KPI* Australia*\tabularnewline
digital health record impact* Australia*\tabularnewline
digital health record measure* Australia*\tabularnewline
digital health record result* Australia*\tabularnewline
digital health record positive Australia*\tabularnewline
digital health record reduc* Australia*\tabularnewline
digital health record error* Australia*\tabularnewline
digital health record error rate Australia*\tabularnewline
EPR Evaluat* Australia*\tabularnewline
EPR success Australia*\tabularnewline
EPR adopt* Australia*\tabularnewline
EPR outcome* Australia*\tabularnewline
EPR outcome measure* Australia*\tabularnewline
EPR performance measure Australia*\tabularnewline
EPR impact assess* Australia*\tabularnewline
EPR success Australia*\tabularnewline
EPR monitor Australia*\tabularnewline
EPR success rate Australia*\tabularnewline
EPR perception Australia*\tabularnewline
EPR key performance indicator* Australia*\tabularnewline
EPR KPI* Australia*\tabularnewline
EPR impact* Australia*\tabularnewline
EPR measure* Australia*\tabularnewline
EPR result* Australia*\tabularnewline
EPR positive Australia*\tabularnewline
EPR reduc* Australia*\tabularnewline
EPR error* Australia*\tabularnewline
EPR error rate Australia*\tabularnewline
electronic patient record Evaluat* Australia*\tabularnewline
electronic patient record success Australia*\tabularnewline
electronic patient record adopt* Australia*\tabularnewline
electronic patient record outcome* Australia*\tabularnewline
electronic patient record outcome measure* Australia*\tabularnewline
electronic patient record performance measure Australia*\tabularnewline
electronic patient record impact assess* Australia*\tabularnewline
electronic patient record success Australia*\tabularnewline
electronic patient record monitor Australia*\tabularnewline
electronic patient record success rate Australia*\tabularnewline
electronic patient record perception Australia*\tabularnewline
electronic patient record key performance indicator*
Australia*\tabularnewline
electronic patient record KPI* Australia*\tabularnewline
electronic patient record impact* Australia*\tabularnewline
electronic patient record measure* Australia*\tabularnewline
electronic patient record result* Australia*\tabularnewline
electronic patient record positive Australia*\tabularnewline
electronic patient record reduc* Australia*\tabularnewline
electronic patient record error* Australia*\tabularnewline
electronic patient record error rate Australia*\tabularnewline
electronic medication management system* Australia*\tabularnewline
\bottomrule
\end{longtable}

\subsection*{Appendix 2: Completed PRISMA-ScR Checklist
(overleaf)}

{\label{appendix-2-completed-prisma-scr-checklist-overleaf}}\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/image2/image2}

\end{center}
\end{figure}\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/image3/image3}

\end{center}
\end{figure}

\selectlanguage{english}
\FloatBarrier
\end{document}

