How the Consolidated Framework for Implementation Research Can Strengthen Findings and Improve Translation of Research Into Practice: A Case Study

Although randomized, controlled trials (RCTs) provide the lowest risk of bias in assessment of treatment outcomes, they have a number of important limitations. They offer only data on predefined and measurable outcomes and do not explain how and under which social and process-oriented circumstances the intervention works, or which contextual factors are essential for the sustainment of intervention effects. Therefore, process evaluations are strongly recommended when comprehensively evaluating complex interventions (Richards & Hallberg, 2015) to understand the mechanisms through which the intervention achieves its outcomes (Grant, Treweek, Dreischulte, Foy, & Guthrie, 2013; Medical Research Council, 2006). Accompanying research is particularly important in multicenter trials in which the intervention may have been implemented in different ways (Grant, Dreischulte, Treweek, & Guthrie, 2012; Grant et al., 2013). Understanding how the components of the intervention and the context vary across sites can assist in interpreting differences and similarities in results.

In this article, the authors outline how a complex nurse-led complementary and alternative medicine (CAM) intervention will be evaluated in the context of an RCT. A multifaceted nursing intervention was developed within the Complementary Nursing in Gynecologic Oncology (CONGO) study, consisting of an evidence-based CAM nursing package to target physical and psychological symptoms of patients with cancer at each new cycle of their chemotherapy regimen, as well as for the time between (Klafke et al., 2014). The components of the intervention are distinct but intertwined elements, complementing and supporting one another. They include (a) resource-oriented counseling; (b) a nursing package, including evidence-based CAM (e.g., propolis for treating mucositis, aloe vera gel application for treating hand-foot syndrome, acupressure band to prevent or treat nausea); and (c) evidence-based information material on CAM. All interventions were performed by trained CONGO nurses. The complex intervention, which comprises a pragmatic trial according to the Medical Research Council (2006) framework, was tested in an RCT in two outpatient cancer departments in Germany (a university hospital and a community hospital) (Klafke et al., 2015). Both sites had previous experience providing selected complementary therapies (i.e., aromatherapy, compress) to patients with cancer (Neuberger et al., 2012). However, none had yet provided potentially interested patients a standardized supportive CAM therapy package. The primary patient-centered outcome of the RCT is health-related quality of life. Secondary endpoints assessed are physical (e.g., fatigue, nausea, pain, mucositis) and psychosocial (e.g., anxiety, depression, self-efficacy, social support, spiritual well-being, patient competence) symptoms (Klafke et al., 2015). The CAM trial tests whether the developed intervention improves patient-centered outcomes. To receive additional information on the implementation process of the complex intervention characteristics, its inherent structures, and procedures, the suggestion is that further data are collected and additional research is conducted in a systematic process evaluation (Grant et al., 2013; Medical Research Council, 2006).

The overall aim of the process evaluation outlined is to provide an additional explanatory basis for comprehensively interpreting the main study outcomes. The following central research questions guided the concept of this analytic procedure and will be addressed throughout the process evaluation:

•  How was the planned intervention performed in routine care?

•  Which factors supported or distracted its implementation?

•  How have key organizational structures been built and sustained?

For identifying the relevant factors affecting the realization of the intervention, as well as the planning of possible future knowledge transfers, the following six objectives have been developed for making the analytic procedures more concrete:

1.  To assess intervention fidelity, or the degree to which the planned intervention was realized

2.  To explore the intervention advantages and disadvantages from different perspectives, in terms of (anticipated or nonanticipated) positive and negative outcomes

3.  To understand mechanisms of change in patients, including dose-response relationships

4.  To identify implementation facilitators and barriers (determinants and mechanisms of change) in the two different healthcare settings

5.  To explore the experiences with the intervention from different perspectives (patients, their significant others, nursing staff and staff colleagues, as well as other clinical decision makers)

6.  To derive stakeholder-informed recommendations on how to deliver the intervention and adapt it for use in other healthcare settings

Methods

Design and Theoretical Concept

Implementation of an intervention in practice is in its essence a social process; therefore, it is important to evaluate the social effectiveness of the intervention and highlight other social and interpersonal factors contributing to the integration of the intervention (Damschroder et al., 2009; Rod, Ingholt, Sorensen, & Tjornhoj-Thomsen, 2014). The Consolidated Framework for Implementation Research (CFIR) provides a theoretical framework for process evaluation of the implementation of complex interventions in healthcare practice. It synthesizes central concepts of earlier models of implementation and evaluation research and provides a comprehensive structure to describe the processes and factors that enable explanation of the effects or variation of the tested intervention. The CFIR framework has been applied successfully in several studies (Breimaier, Heckemann, Halfens, & Lohrmann, 2015; Damschroder & Lowery, 2013; Ilott, Gerrish, Booth, & Field, 2013; Kirk et al., 2016) to gain deeper insight into and understanding of how to analyze and understand change processes in healthcare settings. The CFIR framework specifies five core domains for the analysis of the implementation process (Damschroder et al., 2009; Gaglio & Glasgow, 2012): intervention characteristics, outer setting, inner setting, characteristics of the individuals, and the process of implementation.

The intervention characteristics domain includes information on evidence level, adaptability, trialability, complexity, design, and cost of the intervention, which need to be made transparent for other healthcare settings. The outer setting domain reports on patient needs and resources, external policy, and regulations. The inner setting domain assesses information such as the relevant organizational structure of the healthcare setting, the capacity for implementation change, and the readiness for implementation change. The characteristics of the individuals provide a better understanding of how the involved individuals conduct decisions and how much choice and agency they have in these. The process of implementation domain analyzes the impact of individual and organizational communication of change, involvement of key stakeholders, and feedback processes. According to the intertwined structure of the CFIR, these five domains interact and influence one another during implementation processes (Damschroder & Lowery, 2013; Kirk et al., 2016), and the authors consider them essential to analyze the intervention’s major drivers before the planning and formation of implementation processes.

The CFIR was applied to guide the overall study design of the process evaluation (Damschroder et al., 2009; Kirk et al., 2016); however, to comprehensively unravel the healthcare structure providing the basis for the long-term delivery of complementary supportive therapy, some emphasis also needs to be placed on intervention fidelity, which is not part of the CFIR framework. This concept refers to the concern about whether the intervention was conducted as it was initially planned and documented (Horner, Rew, & Torres, 2006). Therefore, it is critical to include measures and standardized monitoring instruments from the beginning of the study to understand expected and unexpected outcomes of the intervention after the study’s completion (Bosak, Pozehl, & Yates, 2012). In considering the CFIR domains, as well as the concept of intervention fidelity, several discrete interrelated studies were designed and conducted (see Table 1), following a mixed-methods approach to answer the research objectives of the process evaluation.

Data Collection

To provide a better understanding of how the application of the CFIR may be useful, the authors will describe how the framework has been considered in the study design and in the CONGO study’s process evaluation. Data from the following five studies were collected to gain insight into the key implementation concepts specified by the CFIR framework.

Study 1: The aim of the first study is to examine CFIR core concepts of inner and outer settings by assessing human and organizational resources of the two recruiting centers. In line with the design of the process evaluation, objectives 3, 4, and 6 of the process evaluation will be addressed.

A retrospective analysis of organizational data in both sites will be conducted within the two different healthcare settings (the National Center for Tumor Diseases and the Community Hospital Karlsruhe), in which the complex CAM intervention of the CONGO study was implemented in July 2014. Data regarding human resources (e.g., staff fluctuation, qualification and health professions, sick leave), and organizational aspects (e.g., hierarchical structures, meetings and reporting, documentation, patient pathways) will be collected pre- and poststudy for both sites. External influences or relevant changes (e.g., media reports, relevant publications, medical innovation) that may have an impact on the implementation process within the organization will be collected and described.

Study 2: The aim of the second study is to identify the enabling and hindering factors for the implementation of the study. As part of this overarching aim, objectives 2, 4, 5, and 6 of the process evaluation will be addressed.

Repeated focus groups, ranging from 6–8 participants, with the assigned CONGO nurses will be conducted throughout the implementation process, allowing insight into their perspectives and understanding of the intervention practices from the beginning. Each focus group meeting will be moderated by a member of the research team. The moderator will stimulate and guide the group discussions by focusing on different themes that were identified as important to explore since the start of the intervention (see Table 2).

In addition, interviews with stakeholders from the two healthcare settings will be conducted to understand their motivational factors to implement the CAM intervention. The semistructured interview guideline will focus on aspects relevant when implementing change in healthcare settings on the macro-, meso-, and microlevel (Parkin, 2009).

Study 3: The third study aims to understand how patients and their families experience the CAM intervention. Therefore, objectives 2, 3, and 5 of the process evaluation will be addressed.

Patients (range = 20–30 patients) will be purposefully sampled (Patton, 2002) so that interviewees will represent as many varying aspects as possible (e.g., stage/type of cancer, intervention/control group, high/low CAM use). A semistructured interview guideline will be developed. Figure 1 illustrates a sample of core questions that will be addressed and explored by adequate follow-up prompts and probes.

Study 4: The fourth study aimed to evaluate the competence and knowledge gain of the assigned CONGO nursing staff members in the following areas: professional competence, methodologic competence, nursing competence, and social competence. In particular, objectives 2, 4, and 5 were addressed.

A mixed-methods approach (interviews and questionnaires) was applied to analyze in which competence areas the trained CONGO nursing staff gained experience. An interview guideline, based on an established competence framework (Wittneben, 2003), was developed to explore how the nurses perceived their competence gain. Additional semistructured interviews were conducted to examine the perspectives of nursing staff not directly involved in the CONGO study but working in the same outpatient clinic in which the CAM intervention was conducted.

In addition, a questionnaire was developed and piloted in which the CONGO nurses self-assess their (un)certainty on the following competence areas relevant for conducting the CAM nursing intervention throughout the CONGO study: communicating with patients with cancer in general, conducting information and study briefings, counseling patients based on a resource-oriented approach, and conducting the CAM interventions. This questionnaire was based on a questionnaire applied for evaluation of KoMPASS, an intensive communication training for oncologists (Vitinius et al., 2013). The described questionnaire was administered to the assigned CONGO nurses at nine time points throughout the study.

Study 5: As part of the aim of the fifth study, intervention fidelity was investigated, and objectives 1 and 4 of the study evaluation were addressed.

In this study, a mixed-methods approach (documentation analysis and participant observation) was applied to assess intervention fidelity. An electronic case report file (eCRF) was developed to monitor the study. Every patient invited for the study was registered and assigned a study participation number, which is documented in the eCRF. If patients agreed to participate in the study, they were assessed at each cycle of their chemotherapy regimen, and intervention-relevant data were documented accordingly. Patients’ symptoms, which have been specified in their medical health record or mentioned during their counseling interview before each chemotherapy cycle, were documented in the eCRF. In addition, if patients in the intervention group received CAM interventions during their stay or for home self-use, this was also documented in the eCRF. Regular monitoring visits, based on random sampling, were made to verify that the assigned CONGO nurses conducted the interventions according to the study protocol and to solve possible occurring discrepancies between the medical health record and the eCRF. In addition, patients were handed a diary in which they kept a record of symptoms and symptom management at home between chemotherapy cycles. This can provide relevant information to explain the intervention fidelity of the patients.

To analyze the developed structures and processes based on the study’s protocol, six participant observation sessions were conducted at each study site (Girtler, 2001; Lüders, 2003). The analysis was based on the field notes collected during the observations (Emerson, Fretz, & Shaw, 1995).

Data Analysis

Overall data analysis consisted of two steps. As a first step, each study was analyzed individually in line with its specific research objectives.

Quantitative data analysis: The structural data from study 1 were analyzed descriptively. Data from study 4 (competence gain) and 5 (intervention fidelity) were also subject to descriptive quantitative analysis. SPSS®, version 21.0, assisted in analyzing the data sets.

Qualitative data analysis: The data sets from studies 2, 3, and 4 were comprised of qualitative material; discursive data of the focus groups and patient, nurse, and healthcare professional interviews were subject to thematic analysis (Braun et al., 2014) or content analysis (Mayring, 2000). All recorded interviews and focus groups were transcribed verbatim. The qualitative software program ATLAS.ti was used to facilitate data analysis. Initial codes were identified, compared, and then further grouped into categories. These categories were subject to a further process of analysis, resulting in the identification and justification of certain themes relevant for answering the research questions. The observational data and field notes from study 5 were analyzed according to principles of qualitative data analysis (Braun et al., 2014; Emerson et al., 1995; Girtler, 2001).

After individual analysis of all studies, the main research findings of the RCT and the process evaluation will be integrated in a second step. All data will be subject to a framework analysis (Ritchie & Lewis, 2003; Ritchie, Lewis, Nicholls, & Ormston, 2013). This framework method is widely used for structuring and identifying differences and similarities in qualitative data to complement quantitative results, enabling concrete recommendations for implementation strategies to be revealed (Davy, Harfield, McArthur, Munn, & Brown, 2016; Gale, Heath, Cameron, Rashid, & Redwood, 2013). The following three integrative research levels will be considered when framing the data according to the CFIR concepts:

•  How can the intervention results be implemented into healthcare settings?

•  How and under which circumstances did the intervention work best?

•  Which new unexpected concepts did occur and need to be addressed?

According to this established framework method, all data will be coded and then thematically analyzed using the CFIR framework. An advantage of the described framework method lies in its flexibility, as it is not aligned with an epistemologic approach and can be applied deductively and inductively (Gale et al., 2013). The authors aimed to frame the data material based on the CFIR’s five core concepts, allowing room to add additional concepts that may have been identified during the analytic process.

Ethical Considerations

The ethical approval for the CONGO study and its process evaluation has been granted by the ethics committees of the University of Heidelberg and the State Medical Council of Baden-Wurttemberg. The CONGO study has been registered with the German Clinical Trials Register (https://drks-neu.uniklinik-freiburg.de/drks_web) under DRKS00006056 and within the Database Health Services Research (Datenbank Versorgungsforschung Deutschland) under VfD_CONGO_14_003552.

Trial Status

Patient recruitment for the CONGO study started in July 2014 together with the accompanying process evaluation. Data collection for the RCT was finished in December 2016. Data collection of all five studies of the process evaluation was finished in mid-2017. Data analysis of studies 4 and 5 have been completed; the other analyses are ongoing, and final results are expected at the end of 2017. Results of the framework analysis integrating the findings of the RCT and the process evaluation are expected at the beginning of 2018.

Discussion

This article outlines the protocol for a process evaluation applying multiple methods to understand how and under which circumstances the complex CAM nurse-led intervention has been introduced into routine practice, and under which circumstances this intervention affects healthcare deliveries, teamwork, communication, decision-making processes, and subsequent patient outcomes. This mixed-methods process evaluation is a first attempt to document the impact of implementing a supportive cancer care program at two German outpatient cancer clinics.

A strength of the CONGO process evaluation lies in the application of the established CFIR framework (Damschroder et al., 2009; Kirk et al., 2016), which structures the analytical components comprehensively. This framework guided the application of different research techniques, providing a comprehensive approach for analyzing the different objectives. To date, the authors can confirm that their selected methods have been appropriate for the research aims of this process evaluation. By applying this mixed-methods process evaluation, the authors expect to provide answers for the micro-, meso-, and macrosystem processes inherent in outpatient clinics providing chemotherapy treatment with integrated supportive programs (Parkin, 2009).

The authors have already finished the data collection of the RCT and the parallel process evaluation. The primary analysis has been completed, as well as some substudies of the process evaluation, supporting the authors’ confidence that the planned approach is suitable to answer the research aims. The authors acknowledge the high acceptance with the methods involved in studies 2 and 3 (focus groups and semistructured interviews, respectively), as the study participants (CONGO patients, CONGO nurses, experts, and colleagues) were keen and interested to contribute to the process analysis, and they have created a rich pool of qualitative data. In addition, the evaluation questionnaire developed for study 4 can be considered appropriate; it could be comprehensively answered by the participants, so it was deemed applicable. These first positive feedback processes encouraged the authors to continue with the process evaluation as planned.

Limitations

The results of this study will be subject to certain limitations. The authors decided to apply the CFIR framework for the process evaluation of their study, as it provides a comprehensive approach to constructs that have been identified to influence implementation of interventions in healthcare practice. Because the CFIR is comprised of 39 constructs within the 5 core domains, addressing all constructs was not possible. The use of a specific framework also may have led to neglecting domains or constructs included in other frameworks, such as Promoting Action on Research Implementation in Health Services (Rycroft-Malone, 2004) or the Knowledge to Action Framework (Graham et al., 2006), which also may have been relevant for the process evaluation of the current study.

Because the CONGO study has been implemented in only two different healthcare settings, the process evaluation focuses on those setting providers, which may reduce the potential to generalize results. In addition, the findings from the CONGO study and its process evaluation may not be applicable to all healthcare systems outside of Germany/Europe, where health care is organized differently, or where the interest to integrate CAM is not widespread. The CONGO process evaluation was designed by researchers affiliated with the CONGO study, and no completely neutral researcher is involved to guide the data analysis of the framework analysis of the process evaluation. However, to prevent possible social desirability bias, patients, nurses, and stakeholders were interviewed, and initial data analysis was performed by a neutral person who is not directly part of the core research team.

Implications for Nursing

This article has twofold implications for nursing practice and research. On one hand, this article highlights the practical benefits for nurse managers and researchers when translating research findings into practice. After composing and conducting an interventional process with a sound study design, the questions after finishing the study remain: How can the results be translated into everyday practice? Which organizational structure is needed? To answer these questions and more, healthcare professionals can draw on the five core concepts of the CFIR framework and consider its answers in the routine implementation and integration processes. In the case of the CONGO study, which the authors described as an example in this article, it is relevant to know how the planned intervention and study results can be translated into practice so that patients can benefit from complementary supportive care. In considering the structured findings from the CFIR-influenced process evaluation, such a translation process may be easier and more efficient in practice. On the other hand, the article outlines factors that are essential but usually are not considered in oncologic nursing research studies.

Including a process evaluation in interventional study designs from the beginning can help strengthen study protocols. Each individual interventional study requires a process-analytical frame so that the setting and factors promoting or hindering the intervention in everyday routine care are clearly understood. Without this, translating RCT results into clinical practice is difficult. In addition, such translations, which do not address real-life settings, may not be sustainable and, therefore, may be inefficient. The authors strongly recommend conducting a process evaluation in addition to RCTs, and they present the CFIR framework as one suitable comprehensive option, particularly when potential interest exists in integrating the tested intervention in different healthcare settings.

Conclusion

This mixed-methods process evaluation of the CONGO study, grounded in a sound and established theoretical framework, namely the CFIR, will evaluate the factors that need to be considered in knowledge transfers as part of future implementation processes of integrative supportive CAM care programs. Supportive CAM care programs should be nurse-led, because oncology nurses spend the most amount of time with the patients and can build an optimistic, trusting relationship. However, such interventions also should be embedded in an interdisciplinary healthcare environment considering the perspectives and interplay of all involved healthcare professionals aiming to improve disease-oriented, as well as patient-oriented, outcomes.

About the Author(s)

Klafke is a post-doctoral researcher and Mahler is a senior researcher, both in the Department of General Practice and Health Services Research at Heidelberg University Hospital; von Hagens is a senior physician in the Department of Gynaecological Endocrinology and Reproductive Medicine at Heidelberg University Women’s Hospital; Wensing is a professor and deputy head in the Department of General Practice and Health Services Research at Heidelberg University Hospital; Schneeweiss is the head of the Division of Gynaecologic Oncology at the National Center for Tumor Diseases at Heidelberg University Hospital; Müller is the director of the Women’s Clinic at the Community Hospital of Karlsruhe; Szecsenyi is a professor and head of the Department of General Practice and Health Services Research at Heidelberg University Hospital; and Joos is a professor and head of the Institute of General Practice and Interprofessional Care at Tuebingen University Hospital, all in Germany. This study was funded by the Bundesministerium für Bildung und Forschung under 01GY1334 (Förderkennzeichen). Klafke and Mahler shared first authorship and supervised the data collection and data analysis and provided statistical support. All authors contributed to the conceptualization and design and the manuscript preparation. Klafke can be reached at nadja.klafke@med.uni-heidelberg.de, with copy to editor at ONFEditor@ons.org. Submitted July 2016. Accepted for publication February 13, 2017.