\documentclass[10pt]{article}

\usepackage{fullpage}
\usepackage{setspace}
\usepackage{parskip}
\usepackage{titlesec}
\usepackage[section]{placeins}
\usepackage{xcolor}
\usepackage{breakcites}
\usepackage{lineno}
\usepackage{hyphenat}





\PassOptionsToPackage{hyphens}{url}
\usepackage[colorlinks = true,
            linkcolor = blue,
            urlcolor  = blue,
            citecolor = blue,
            anchorcolor = blue]{hyperref}
\usepackage{etoolbox}
\makeatletter
\patchcmd\@combinedblfloats{\box\@outputbox}{\unvbox\@outputbox}{}{%
  \errmessage{\noexpand\@combinedblfloats could not be patched}%
}%
\makeatother


\usepackage{natbib}




\renewenvironment{abstract}
  {{\bfseries\noindent{\abstractname}\par\nobreak}\footnotesize}
  {\bigskip}

\titlespacing{\section}{0pt}{*3}{*1}
\titlespacing{\subsection}{0pt}{*2}{*0.5}
\titlespacing{\subsubsection}{0pt}{*1.5}{0pt}


\usepackage{authblk}


\usepackage{graphicx}
\usepackage[space]{grffile}
\usepackage{latexsym}
\usepackage{textcomp}
\usepackage{longtable}
\usepackage{tabulary}
\usepackage{booktabs,array,multirow}
\usepackage{amsfonts,amsmath,amssymb}
\providecommand\citet{\cite}
\providecommand\citep{\cite}
\providecommand\citealt{\cite}
% You can conditionalize code for latexml or normal latex using this.
\newif\iflatexml\latexmlfalse
\providecommand{\tightlist}{\setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}}%

\AtBeginDocument{\DeclareGraphicsExtensions{.pdf,.PDF,.eps,.EPS,.png,.PNG,.tif,.TIF,.jpg,.JPG,.jpeg,.JPEG}}

\usepackage[utf8]{inputenc}
\usepackage[ngerman,english]{babel}



\usepackage{float}








\begin{document}

\title{From clinical guidelines to clinical care -- employee involvement makes
a positive difference to implementation.}



\author[1]{Liv Kleve}%
\author[1]{Line Skarstein}%
\author[1]{Irene Elgen}%
\affil[1]{Haukeland University Hospital}%


\vspace{-1em}



  
  \date{June 13, 2024}


\begingroup
\let\center\flushleft
\let\endcenter\endflushleft
\maketitle
\endgroup





\selectlanguage{english}
\begin{abstract}
Objectives Implementation of new knowledge into routine care is a
complex endeavour. Involving employees in the change process, good
planning and communication as well as a commitment to training has been
highlighted as important factors for successful implementation.
Acknowledging change as a process may also be helpful. The aim of this
paper was to describe the initial phase of the implementation process in
changing to evidence-based practisepractices within a child and
adolescent mental health service. Method Prior to the five-year project,
an external service evaluation was carried out. The employees expressed
a need for a clear direction from management to guide their clinical
practice. A vision and strategy for the service was developed. Employees
participated in the process of developing clinical standards during the
first phase of implementation. Results Fixsen's four stage model and the
PSDA circle were used to guide the implementation process. The employees
developed a template for a clinical standard based on national and
international clinical guidelines. During the period, 17 clinical
standards were established and 10 new evidence-based methods were
implemented. All service leads (13) and a group of senior clinicians
(32) were invited to participate in an evaluation five years after the
initial service evaluation. There was overall agreement that the mental
health service was developing positively providing high quality services
for children and adolescents. In addition, both groups agreed that the
introduction of clinical standards was important in ensuring quality
care. Conclusion Involving employees in the implementation process
seemed to be an important factor in changing a mental health service.%
\end{abstract}%



\sloppy


\textbf{Method}

Prior to the five-year project, an external service evaluation was
carried out. The employees expressed a need for a clear direction from
management to guide their clinical practice. A vision and strategy for
the service was developed. Employees participated in the process of
developing \emph{clinical standards} during the first phase of
implementation.

\textbf{Results} Fixsen's four stage model and the PSDA circle were used
to guide the implementation process. The employees developed a template
for a\emph{clinical standard} based on national and international
clinical guidelines. During the period, 17 clinical standards were
established and 10 new evidence-based methods were implemented.

All service leads (13) and a group of senior clinicians (32) were
invited to participate in an evaluation five years after the initial
service evaluation. There was overall agreement that the mental health
service was developing positively providing high quality services for
children and adolescents. In addition, both groups agreed that the
introduction of clinical standards was important in ensuring quality
care.

\textbf{Conclusion}

Involving employees in the implementation process seemed to be an
important factor in changing a mental health service.

\textbf{Running tittel}

Implementation of clinical guidelines

\textbf{Keywords} Clinical guidelines, healthcare, evidence-based
medicine, clinical safety

\textbf{Introduction}

In line with an expansion in research and proven methods during recent
decades, expectations of delivering evidence-based practice are
increasing. In many ways this has challenged existing culture within
mental health services where clinical autonomy has long been cherished
and a disregard for clinical guidelines reported \textsuperscript{1}.
The difficulty in integrating research findings into clinical care has
been described as the ``research practice gap'' \textsuperscript{2,3}.

Barriers to changing practice have been reported to include a number of
factors such as administrative constraints (eg. lack of time), negative
staff attitudes, anxiety about changing practice, information overload
as well as a lack of culture and leadership \textsuperscript{1,4}. There
continues to be uncertainty about clinically meaningful and sustainable
effects of clinical guidelines on patient outcomes and how best to
implement such guidelines for maximal benefit \textsuperscript{5,6}.
While several promising implementation strategies are
described\textsuperscript{7}, the evidence of their effectiveness
remains less evaluated \textsuperscript{8-12}\textbf{.} The general
advice is therefore to use low cost approaches when planning
organisational or clinical changes \textsuperscript{4,7}.

The use of evidence-based practice in mental health services is
associated with proficient organizational
cultures\textsuperscript{13-16}. Clinicians who work in proficient
cultures report that they are expected to be effective and there is
evidence that mental health organizations with proficient cultures
provide higher quality service and better outcomes
\textsuperscript{14,17-19}.

Alongside organisational cultures the concept of employee-driven
innovation (EDI) is receiving increasing attention
\textsuperscript{20,21}. Proponents of EDI suggest that in the course of
their working days employees capture significant information and insight
into the running of an organisation and its potential for improvement.
Furthermore, they claim that EDI provides higher self-esteem to
employees through empowerment. Alignment between employee-driven ideas
and management priorities is important to ensure that ideas are
implemented, to improve morale and showing employees that they are being
listened to. In support of this claim, it appears that communication
that makes use of active participation and involvement, using for
instance audits and feedback systems, have greater effect than passive
dissemination \textsuperscript{10,11}. Training and performance
management also have positive impact on patient and staff outcome
\textsuperscript{6,22}.

Fixsen et al. has identified that good implementation includes at least
four stages; exploration, installation, first implementation and full
implementation \textsuperscript{8}. Although the initial exploration
stage is considered particularly important, this is often neglected by
managers \textsuperscript{10}.

With respect to the more practical ways to facilitate the change
process, Deming's PDSA circle (Plan-Do-Study-Act) is frequently used as
a practical feedback system tool during all stages of
implementation\textsuperscript{23}.

In summary, factors that facilitate the implementating of EBP include
developing a culture that is positive to change as well as alignment
between managers and staff in priorities for the service. Further,
change requires good exploration, planning, communication as well as a
commitment to training and involvement of employees. There continues to
be a call for research to improve methods for
implementation\textsuperscript{4,7,13}.

The present paper describes and evaluates the first stage of a five-year
service quality improvement project. The overall aim of the project was
to optimize clinical care through introducing a clear direction for the
service, disseminating a range of evidence-based methods and identifying
effective organisational models for low incidence client groups.

\textbf{Methods}

\emph{Background}

The process of change took place in a child and adolescent mental health
service (CAMHS) consisting of seven outpatient teams, two specialized
teams and three hospital wards (Haukeland university hospital). This
change process was part of a five-year project following a critical
service evaluation.

Prior to the five-year project, an external service evaluation had been
carried out by the hospital's human resource (HR) department in 2012.
The aim of this evaluation was to examine to what extent the
department's organizational structure and clinical practice were
experienced as beneficial. The evaluation consisted of interviews with
40 individuals; service leads, key senior clinicians, trade union and
safety representatives, service users and staff from collaborating
units.

The results of the service evaluation highlighted satisfaction with
levels of staff resources and expertise. But a perceived exclusive focus
on operational and administrative issues, a failure to address the wide
variety of approaches practiced within the service, and the absence of a
clear strategy specifically to guide clinical practice were sources of
significan disatisfaction.

\emph{CAMHS vison and strategy}

Based on the recommendations from the evaluation carried out by HR in
2012, a working group consisting of managers, clinicians, union as well
as user representatives, developed a vision and service strategy. The
conclusion was a vision; \emph{``to develop excellence in service
delivery, equality in access across the service, a patient centred
approach and evidence based practice''} and a service strategy which
identified a range of clinical and organisational areas in need of
development. The role of the key champion for implementing the vision
and strategy was passed onto a member of staff who had previous
experience in the area of implementation.

\emph{Implementing the first stage of the CAMHS vision and strategy.}

In addition to develop a plan for implementating new methods,
consideration was given to the process of change. It was decided to
involve employees in the development of the service and to use Fixsen's
four stages of implementation to structure the
process\textsuperscript{8}. The first stage, the exploration stage,
involves creating a team, assessing needs, exploring evidence and
usability of interventions. The process described below belongs to the
exploration stage.

A steering group consisting of the director of service and two
consultant psychiatrists, coordinated activities during the exploration
stage (two members were part of the management team and all had
background as researchers). The details of the involvement of the
broader staff group is described below.

\emph{Clinical standards}

The goal was to develop a tool that provided staff with quick access to
relevant research literature and recommendations for evidence based
practice. These tools would serve as \emph{clinical standards} and
contribute to identify the requested direction of the service. Success
metrics would involve clinicians perceiving the standards as useful and
that later service development adhered to recommendations.

In order to provide easy access to relevant research literature, it was
decided to use Norwegian, English and American clinical guidelines as
the main source of information. The term clinical guidelines refers to
evidence based recommendations for health and care with the aim of
optimizing patient care, how to diagnose and treat a range of medical
conditions. These are informed by a systematic reviews of evidence. The
purpose of guidelines is to improve clinical effectiveness by
implementation of evidence based care in daily
practice\textsuperscript{24,25}. Pairs of clinicians were asked to
summarize relevant clinical guidelines. The summary of each guideline
gave rise to discussions highlighting service gaps and needs. Key
information from guidelines was operationalized to increase the
feasability for clinical care. Clinicians created drafts of clinical
standards. Five drafts of clinical standards underwent piloting before
the final clinical standard template was established. The piloting took
place within the service during a 6 months period.

\emph{Evaluation of the service development and process of change.}

In 2017, five years after the initial service evaluation, a review of
achievements and a structured evaluation of employees' perception of the
service developments and process of change was conducted.

\emph{Evaluation of employees' perception of service develompment and
process of change}

The HR department developed a structured questionnaire in collaboration
with the CAMHS management team. The questions were consistently worded
in a positive manner. There is a risk of acquiescence or extreme
response bias when using only positively worded
questions\textsuperscript{26,27}. However it is well-known that
including a mix of both positively and negatively worded questions can
also create new challenges (such as confusion and incorrect
answers)\textsuperscript{28}. The questionnaire was distributed and
collected electronically, asking about areas of implementation and
clinical management.

The seven questions are presented in Table 1. Each question had a score
where 1 was ``totally disagree'', 2 ``disagree'', 3 ``both disagree and
agree'', 4 ``agree'', and 5 ``completely agree''.

All employees or their equivalent replacements (if left the service)
that were interviewed in 2012 were included (service leads and senior
clinicians).

\textbf{Statistical analyses}

Descriptive analyses were used to describe the outcome measures with
mean, standard

deviation (Table 1, seven answers). We compared mean values of the seven
answers for the senior clinicians (N=32) and the service leads (N=13)
using Paired Sampled t-test. Secondly, a linear regression analysis was
carried out using one question ``CAMHS is developing in the right
direction\selectlanguage{ngerman}» as the dependent variable and two other questions as
independent variables. This was done to explore possible factors
facilitating the development of the CAMHS. A \emph{p} -value of
\textless{}0.05 was considered statistically significant. The SPSS
statistical package version 24.0 (IBM SPSS Statistics, IBM Corporation)
was used for all analyses \textsuperscript{29}.

\textbf{Results}

\emph{Clinical standards}

Initially the draft clinical standards provided by clinicians varied
from 10 to 15 pages. As stated previously, the goal was to develop a
tool that would provide staff with quick access to relevant research
literature and recommendations for evidence based practice. To succeed,
it was considered that standards should be possible to read by busy
clinicians, perceived as clinically useful and that subsequent staff
training adhered to recommendations.

A template limited to a two page, operationalized description included
the following elements: When to use, diagnostic criteria, assessment,
and treatment methods. In addition, there were notes recommending case
formulation and regular points for evaluation. The standards included
references to the clinical guidelines that formed the basis of the
standards \textsuperscript{24,25}. Clinicians responsible for summaries
and the steering group negotiated the final content of the standards.

During the five-year period, 34 senior clinicians participated in
developing altogether 17 clinical standards. Examples of the content of
clinical standards included categories of disorders such as eating
disorders, anxiety disorders, mood disorders, ADHD, Autism.

Clinical standards became available for all staff in the hospital
``electronic quality control handbook'' and updated as new guidelines
have become available.

Although it could be argued that training and implementation of new
methods belong to the implementation stages \textsuperscript{8} it is,
for the purpose of highlighting the effects of the clinical standards,
considered relevant to report that between 2013 and 2017 staff received
training in 10 new evidence based methods, all recommended in clinical
guidelines and thus local clinical standards. The process of
implementing these methods as well as the organisation of the service is
not addressed in this paper.

\emph{Evaluation of perception of change and process}

Evaluation of employee's perception of change took place in 2017, five
years following the initial evaluation and was performed asking seven
questions. Results are given in Table 1. For the invited employees 45 of
60 (75\%) completed the questionnaire in 2017. All service leads
participated (13 (29\%)). More than one third (18/45, 40\%) of the
respondents were also interviewed in the 2012 evaluation.

\emph{Table 1 in here}

The results of the evaluation expressed that both service leads and
clinicians tend to agree that the mental health service was developing
positively, providing a high quality service for children and
adolescents (Table 1). Furthermore, both groups agreed that the
introduction of clinical standards was particularly important in
ensuring quality care. The highest score was found for the statement
``the development of clinical standards has contributed to clarify the
clinical direction of the service'' with a score of 4.5 (SD: 0.6). There
were no differences in ratings between service leads and clinicians on
this item.

On the question about how ``they perceived themselves to be involved in
developing clinical standards'' there was a statistically significant
difference between service leads and clinicians with the clinicians
rating themselves as having been slightly less involved than service
leads.

In order to explore employee involvement in the process, a linear
regression analysis was carried out. The score of the question ``CAMHS
is developing in the right direction» was used as the dependent variable
and the two variables; question 1) ``The clinical standards have
contributed to clarify the professional direction of CAMHS'', and
question 2) ``Employees in CAMHS have been positively involved in the
development of clinical standards'', as independent variables. Only
employee experience of involvement was significant with 14\% of the
explained variance of CAMHS development in the right direction (adjusted
R: 0.14; 95\% CI 0.1 to 0.7; \emph{p}\textless{} 0.05).

\textbf{Discussion}

During a five-year period, managers and clinicians were involved in
developing clinical standards as the first stage of implementing a CAMHS
vision and strategy. The content of the standards was based on the
advice given in clinical guidelines and contained operationalised and
clinically relevant information. In addition, recommendations
effectuated a staff-training plan. 17 standards were developed and 10
evidenced based methods were implemented during the time period.

All service leads and a group of senior clinicians participated in a
structured evaluation of the process. There was an overall contentment
with the service development and staff agreed that the introduction of
clinical standards provided a clear direction for the service and
quality of care.

Involving employees in the change process seemed to be one of the key
factors in successfully changing the service. The knowledge that passive
dissemination of information has limited effect on staff behaviour made
us actively use EDI in the implementation process. We suggest that the
participation of service leads and clinicians created ownership,
engagement and a sense of responsibility for developing a shared
culture, variables recognised as important in facilitating
implementation \textsuperscript{20}. The 2017 evaluation outcomes
appears to support this understanding, where staff partialy attributed
the positive service development to the involvement of employees. This
finding is also in line with the notion of EDI \textsuperscript{20}.

An important aspect of the strategy was to establish evidence based
knowledge in the clinical setting. One of the barriers to implementing
clinical guidelines include information
overload\textsuperscript{1}\textbf{.} By creating clinical standards
that presented the content of clinical guidelines succinctly, focusing
only on key aspects of assessment and interventions, research
information was transported into the clinical setting and as such
contributed to reduce the ``research practice gap'' \textsuperscript{2}.
The fact that the recommendations set out in clinical standards were
followed up with training and implementation of recommended methods, has
most likely added to the positive perception of change
\textsuperscript{6}.

Managers often ignore and disregard the stage of exploration as
unnecessary and too time-consuming \textsuperscript{5}. However, in this
project the exploration stage was seen as essential in clarifying the
strategy and to facilitate employees' motivation for developing and
adopting changes. This is in line with previous reported findings that
lack of time is often a barrier to changing practice\textsuperscript{5}.

\emph{Strength and limitations}

The strength of this study was the focus on the challenging area of
implementing evidence based practice in the real world of clinical care.
This is important in order to maximise the quality and effectiveness of
health care services.

There are however limitations to this report. A low cost approach
together with a focus on service development has been employed which has
affected the amount of data, eg. a lack of comparison group, collected
in this study.

Furthermore, the relatively positive response from employees may be an
overestimation of actual satisfaction. Satisfaction levels were however
only one factor in evaluating service improvements which was a
limitation.

\textbf{Conclusion}

In conclusion, the initial step of implementing clinical guidelines has
been achieved through involving employees in developing succinct
clinical standards as a vehicle for the communication of evidence based
practice. We attribute this to leadership, sufficient time spent to plan
and communicate the strategy as well as involving employees in the
process. Following up recommendations with a training plan and
implementation of new methods has also been vital to keep up a momentum
of change.

\textbf{Abbreviations}

AACAP American Academy of Child and Adolescent Psychiatry

CAMHS Child and Adolescent Mental Health Service

EDI Employee-Driven Innovation

NICE National Institute for Clinical Excellence

PDSA Plan, Do, Study, and Act

\textbf{Declarations}

Ethics

The project is a service evaluation project and according to Norwegian
law does not require ethics approval (see link). Consent to participate
in the study was ensured through the participation process which was
voluntary and
anonymous.\emph{https://helseforskning.etikkom.no/reglerogrutiner/soknadsplikt/sokerikkerek?p\_dim=34999\&\_ikbLanguageCode=n}

\emph{Consent for publication}

Not applicable

\emph{Availability of data and material}

The datasets used and/or analysed during the current study are available
from the corresponding author on reasonable request

\emph{Competing interests}

The authors declare that they have no competing interests.

\emph{Funding}

No funding was obtained for this study

\emph{Authors' contributions}

First author, LK, has designed the study in collaboration with all other
authors and written the paper. Second author, LSS, conducted the
evaluation in collaboration with all other authors and participated in
writing the paper. Third author, IE, analyzed the data and participated
in structuring and writing of the paper.

\textbf{References}

1. Forsner, T., Hansson, J., Brommels, M.~Wistedt, A. Å., Forsell,
Y.\emph{Implementing clinical guidelines in psychiatry: a qualitative
study of perceived facilitators and barriers.} BMC Psychiatry,
2010.\textbf{10} : p. 8.

2. Bero, L.A.,\emph{~} Grilli, R., Grimshaw J.M., Harvey, E., Oxman,
A.D., Thomson, M.A. \emph{Closing the gap between research and practice:
an overview of systematic reviews of interventions to promote the
implementation of research findings. The Cochrane Effective Practice and
Organization of Care Review Group.} Bmj, 1998. \textbf{317} (7156): p.
465-8.

3. Flottorp, S., and Aakhus, E. \emph{Implementeringsforskning:
vitenskap for forbedring av praksis.} Norsk Epidemiologi, 2013: p.
187-196.

4. Baker,~R., Camosso\selectlanguage{english}-Stefinovic,~J., Gillies~~C., Shaw,~E.J.,
Cheater~~F., Flottorp,~S. and Robertson,~N.\emph{Tailored interventions
to overcome identified barriers to change: effects on professional
practice and health care outcomes.} Cochrane Database Syst Rev, 2010(3):
p. Cd005470.

5. Bighelli, I., Ostuzzi, G., Girlanda, F., Ciprian, A., Becker, T.,
Koesters, M., Barbui. C. \emph{Implementation of treatment guidelines
for specialist mental health care.} Cochrane Database Syst Rev,
2016.\textbf{12} : p. Cd009780.

6. Patterson,~M., Rick, J.,~Wood, S.,~Carroll, C.,~Balain, S.,
and~Booth, A..\emph{Systematic review of the links between human
resource management practices and performance.} Health Technol Assess,
2010. \textbf{14} (51): p. 1-334, iv.

7. Weinmann, S., Koesters, M., and Becker, T. \emph{Effects of
implementation of psychiatric guidelines on provider performance and
patient outcome: systematic review.} Acta Psychiatr Scand,
2007.\textbf{115} (6): p. 420-33.

8. Fixsen, D. L., Naoom S.F., Blase, K.A., Friedman, R.M., Wallace, F.,
Burns,B., Carter, W., Paulson, R., Schoenwald, S., Barwick, M.,
Chambers, D., Petrila,J., Rivard, J. and Shern, D. \emph{Implementaion
research: A synthesis of literature.} USF University of South Florida,
2005.

9. Gagliardi, A.R. and S. Alhabib, \emph{Trends in guideline
implementation: a scoping systematic review.} Implement Sci,
2015.\textbf{10} : p. 54.

10. Grimshaw, J., Shirran, L., Thomas, R., Mowatt, G., Fraser, C., Bero,
L., Grilli, R., Harvey, E., Oxman A. and O'Brien, M. \emph{Changing
provider behavior: an overview of systematic reviews of
interventions.}Med Care, 2001. \textbf{39} (8 Suppl 2): p. Ii2-45.

11. Grimshaw, J.;~Thomas, R.;~MacLennan, G.;~Fraser, C.;~Ramsay,
C.R.;~Vale, L.;~Whitty, P.;~Eccles, M.P.;~Matowe, L.;~Shirran,
L.;~\emph{Wensing, M.J.P.}~;~\emph{Dijkstra, R.F.}~;~Donaldson, C.
\emph{Effectiveness and efficiency of guideline dissemination and
implementation strategies.} Health Technol Assess, 2004. \textbf{8} (6):
p. iii-iv, 1-72.

12. Powell, B.J., Proctor, E.K., Glisson, C.A. Kohl, P.L.,~ Ragavan,R.,
Brownson, RC., Stoner, B.P., Carpenter, C.R. and Palinkas, L.A. \emph{A
mixed methods multiple case study of implementation as usual in
children's social service organizations: study protocol.} Implement Sci,
2013. \textbf{8} : p. 92.

13. Aarons, G.A., Sommerfeld, D.H., and Walrath-Greene,
C.M.\emph{Evidence-based practice implementation: the impact of public
versus private sector organization type on organizational support,
provider attitudes, and adoption of evidence-based practice.} Implement
Sci, 2009. \textbf{4} : p. 83.

14. Glisson, C., Williams,N.J., Hemmelgarn, A., Proctor, E. and Green,
P. \emph{Increasing clinicians' EBT exploration and preparation behavior
in youth mental health services by changing organizational culture with
ARC.} Behav Res Ther, 2016. \textbf{76} : p. 40-6.

15. Novins, D.K., Green, A. E., Legha, R. K. and Aarons,
G.A.\emph{Dissemination and implementation of evidence-based practices
for child and adolescent mental health: a systematic review.} J Am Acad
Child Adolesc Psychiatry, 2013. \textbf{52} (10): p. 1009-1025.e18.

16. Rohrbach, L.A., Graham J.W., and Hansen, W.B. \emph{Diffusion of a
school-based substance abuse prevention program: predictors of program
implementation.} Prev Med, 1993. \textbf{22} (2): p. 237-60.

1. Forsner T, Hansson J, Brommels M, Wistedt AA, Forsell Y. Implementing
clinical guidelines in psychiatry: a qualitative study of perceived
facilitators and barriers. \emph{BMC psychiatry.} 2010;10:8.

2. Bero LA, Grilli R, Grimshaw JM, Harvey E, Oxman AD, Thomson MA.
Closing the gap between research and practice: an overview of systematic
reviews of interventions to promote the implementation of research
findings. The Cochrane Effective Practice and Organization of Care
Review Group. \emph{BMJ (Clinical research ed).} 1998;317(7156):465-468.

3. Flottorp S, Aakhus E. Implementeringsforskning: vitenskap for
forbedring av praksis. \emph{Norsk Epidemiologi.} 2013:187-196.

4. Baker R, Camosso-Stefinovic J, Gillies C, et al. Tailored
interventions to overcome identified barriers to change: effects on
professional practice and health care outcomes. \emph{The Cochrane
database of systematic reviews.} 2010(3):Cd005470.

5. Bighelli I, Ostuzzi G, Girlanda F, et al. Implementation of treatment
guidelines for specialist mental health care. \emph{The Cochrane
database of systematic reviews.} 2016;12(12):Cd009780.

6. Patterson M, Rick J, Wood S, Carroll C, Balain S, Booth A. Systematic
review of the links between human resource management practices and
performance. \emph{Health technology assessment (Winchester,
England).}2010;14(51):1-334, iv.

7. Weinmann S, Koesters M, Becker T. Effects of implementation of
psychiatric guidelines on provider performance and patient outcome:
systematic review. \emph{Acta psychiatrica
Scandinavica.}2007;115(6):420-433.

8. Fixsen D, Naoom S, Blase K, Friedman R, Wallace F. Implementaion
research: A synthesis of literature. \emph{USF University of South
Florida.} 2005.

9. Gagliardi AR, Alhabib S. Trends in guideline implementation: a
scoping systematic review. \emph{Implementation science :
IS.}2015;10:54.

10. Grimshaw JM, Shirran L, Thomas R, et al. Changing provider behavior:
an overview of systematic reviews of interventions. \emph{Medical
care.}2001;39(8 Suppl 2):Ii2-45.

11. Grimshaw JM, Thomas RE, MacLennan G, et al. Effectiveness and
efficiency of guideline dissemination and implementation
strategies.\emph{Health technology assessment (Winchester,
England).}2004;8(6):iii-iv, 1-72.

12. Powell BJ, Proctor EK, Glisson CA, et al. A mixed methods multiple
case study of implementation as usual in children's social service
organizations: study protocol. \emph{Implementation science :
IS.}2013;8:92.

13. Aarons GA, Sommerfeld DH, Walrath-Greene CM. Evidence-based practice
implementation: the impact of public versus private sector organization
type on organizational support, provider attitudes, and adoption of
evidence-based practice. \emph{Implementation science : IS.} 2009;4:83.

14. Glisson C, Williams NJ, Hemmelgarn A, Proctor E, Green P. Increasing
clinicians' EBT exploration and preparation behavior in youth mental
health services by changing organizational culture with
ARC.\emph{Behaviour research and therapy.} 2016;76:40-46.

15. Novins DK, Green AE, Legha RK, Aarons GA. Dissemination and
implementation of evidence-based practices for child and adolescent
mental health: a systematic review. \emph{Journal of the American
Academy of Child and Adolescent Psychiatry.}2013;52(10):1009-1025.e1018.

16. Rohrbach LA, Graham JW, Hansen WB. Diffusion of a school-based
substance abuse prevention program: predictors of program
implementation. \emph{Preventive medicine.} 1993;22(2):237-260.

17. Glisson C, Williams NJ. Assessing and changing organizational social
contexts for effective mental health services. \emph{Annual review of
public health.} 2015;36:507-523.

18. Olin SS, Williams N, Pollock M, et al. Quality indicators for family
support services and their relationship to organizational social
context. \emph{Administration and policy in mental
health.}2014;41(1):43-54.

19. Williams NJ, Glisson C. Reducing turnover is not enough: The need
for proficient organizational cultures to support positive youth
outcomes in child welfare. \emph{Children and youth services
review.}2013;35(11).

20. Hoyrup S, Moller K. \emph{Emplyee-Driven Innovation. A new
approach.} 1 ed. London: Palgrave Macmillan; 2012.

21. Smith P, Ulhoi JP, Kesting P. Mapping key antecedents of
employee-driven innovations. \emph{International Journal of Human
Resources Development and Management.} 2012;12(3):224-236.

22. Proctor EK, Landsverk J, Baumann AA, et al. The implementation
research institute: training mental health implementation researchers in
the United States. \emph{Implementation science : IS.} 2013;8:105.

23. Donnelly P, Kirk P. Use the PDSA model for effective change
management. \emph{Education for primary care : an official publication
of the Association of Course Organisers, National Association of GP
Tutors, World Organisation of Family Doctors.} 2015;26(4):279-281.

24. American Academy of Child \& Adolescent
Psychiatry.\emph{https://www.aacap.org/AACAP/About\_AACAP/Home.aspx}.

25. When designing usability questionnaires, does it hurt to be
positive? Proceedings of the International Conference on Human Factors
in Computing Systems, CHI; 2011; Vancouver, BC, Canada.

26. Salazar MS. The dilemma of combining positive and negative items in
scales. \emph{Psicothema.} 2015;27(2):192-199.

27. Sauro J, Lewis JR. When designing usability questionnaires, does it
hurt to be positive? Proceedings of the International Conference on
Human Factors in Computing Systems, CHI; May 7-12, 2011; Vancouver, BC,
Canada.

28. van Sonderen E, Sanderman R, Coyne JC. Ineffectiveness of reverse
wording of questionnaire items: let's learn from cows in the
rain.\emph{PloS one.} 2013;8(7):e68967.

29. IBMCorp. \emph{IBM SPSS Statistics for windows.} 24 ed. Armonk, NY:
IBM Corp; 2017.

\textbf{Hosted file}

\verb`Table!_10.08.2020.docx`  available at \url{https://authorea.com/users/350795/articles/475527-from-clinical-guidelines-to-clinical-care-employee-involvement-makes-a-positive-difference-to-implementation}

\selectlanguage{english}
\FloatBarrier
\end{document}

