This material is protected by U.S. copyright law. Unauthorized reproduction is prohibited.
To purchase reprints or request permission to reproduce, e-mail reprints@ons.org.
Development of a Measure to Delineate
the Clinical Trials Nursing Role
Heidi E. Ehrenberger, PhD, RN, AOCN®, and Linda Lillington, DNSc, RN, CCRC
Key Points . . .
Purpose/Objectives: To identify the significant dimensions of the
clinical trials nursing role and to construct a reliable and valid survey
instrument to reflect these dimensions.
The current social organization of clinical research is under-
Design: Methodologic survey.
going a significant transformation, and a better understanding
Setting/Sample: The judge panel consisted of six national nurse ex-
of the research nurse role is critical.
perts. The focus group sample was comprised of 24 clinical research
nurses from the West, Northeast, and Great Lakes regions of the United
Nurses are responsible for empirically demonstrating the skills
States and five research nurses from Canada. The sample for instrument
and knowledge they bring to the clinical research enterprise.
testing consisted of 40 oncology clinical research nurses from the
The systematic development and psychometric testing of a
survey instrument are essential when planning to delineate the
Methods: Several strategies were used to develop the Clinical Trials
clinical trials nursing role.
Nursing Questionnaire© (CTNQ): literature review, conceptualization of
the subscales, development of items for each subscale, development of
the tool, expert judge panel evaluation, focus group testing, administra-
tion of the tool, and psychometric analysis of the results.
Main Research Variables: Frequency and importance of clinical tri-
registered nurses who have been educated and certified in on-
als nursing activities.
cology nursing" (p. 973). Little empirical data have been sys-
Findings: Content validity was established at 0.95. The alpha reliabil-
tematically collected and analyzed for the purpose of describ-
ity coefficient was 0.92 for the frequency scale and 0.95 for the impor-
ing the role and responsibilities of the research nurse in cancer
tance scale. A two-week test-retest reliability of 0.88 was obtained for
care. Mueller (2001) stated, "It will be up to nurses to empiri-
the frequency scale and 0.92 for the importance scale. The final CTNQ
cally demonstrate that the skills and knowledge they bring to
contained 12 sections with 154 items.
clinical research as nurses are qualitatively and quantitatively
Conclusions: The CTNQ has acceptable content validity, internal con-
sistency, and stability reliability. This instrument is promising for the as-
sessment of the research nurse role, and its use in further research is
Heidi E. Ehrenberger, PhD, RN, AOCN  ®, is a nurse consultant in
Implications for Nursing: A valid and reliable measure can be used
Ann Arbor, MI, an adjunct assistant professor in the College of
to delineate the subspecialty of clinical trials nursing, thus providing a
Nursing at the University of Tennessee in Knoxville, and a former
better understanding of how nursing professionals contribute to the
visiting professor of nursing at Lincoln Memorial University in
cancer research enterprise.
Harrogate, TN. Linda Lillington, DNSc, RN, CCRC, is a nurse man-
ager in the General Clinical Research Center at the Harbor­Univer-
sity of California, Los Angeles, Research and Education Institute in
he Oncology Nursing Society (ONS, 1998) position
Torrance, CA. Project support was provided, in part, by the Oncol-
statement on cancer research and cancer clinical
ogy Nursing Society Special Interest Groups project fund, an ONS
Foundation/Oncology Nursing Certification Corporation Education
trials stated that "coordination of clinical trials (e.g., the
Research Grant, and a General Clinical Research Center Grant M01-
coordination of clinical sites, development of standardized
RR00425 from the National Center for Research Resources. (Submit-
treatment orders, symptom management, patient education
ted September 2003. Accepted for publication November 26, 2003.)
and advocacy, facilitation of informed consent, assistance
with participant accrual and retention) is best accomplished by
Digital Object Identifier: 10.1188/04.ONF.E64-E68
different, and therefore more beneficial, than those provided
for use in the present study to guide item generation and the
by other occupational groups" (p. 189). The current social or-
selection of variables that are relevant to the CTN role and
ganization of clinical trials research is undergoing a signifi-
practice situation. Three components are included in the
cant transformation, and a better understanding of the unique
framework: (a) structure, which encompasses professional
nursing role is critical. The number and complexity of clini-
nursing and organizational structure variables; (b) process,
cal trials being conducted are increasing. Regulatory and eco-
which consists of the CTN role components (e.g., clinician,
nomic pressures are mounting. Clinical trial processes need to
educator, administrator) and the ways that the role is enacted;
be completed the right way (procedurally correct and as de-
and (c) outcomes, which include clinical guideline enactment.
fined by the protocol), at the right time (protocol-specified
The structure component consists of variables that may influ-
time points), by the right specialist or specialty service (skill
ence the processes and outcomes of care. These variables in-
set or equipment) in a manner that is compliant with the Good
clude those related to role perception (e.g., role satisfaction),
Clinical Practice (GCP) guideline (International Conference
role characteristics (e.g., years of experience), organizational
on Harmonisation [ICH], 1996; Joshi & Ehrenberger, 2001).
characteristics (e.g., primary work setting), and demograph-
The purpose of this methodologic study was to develop a re-
ics (e.g., geographic location). The process component is
liable and valid measure to delineate the subspecialty of clini-
composed of the CTN's role components in clinical research.
cal trials nursing.
These activities include those related to protocol assessment,
protocol planning, subject recruitment, the informed consent
process, investigational product, implementation and evalua-
tion, data management, and performance of the professional
A work group of the ONS Clinical Trial Nurses (CTNs)
nursing role. The outcome component is conceptualized as en-
Special Interest Group (SIG) membership was formed in Oc-
actment of GCP standards.
tober 2000 because an instrument to measure the domain of
interest did not exist. Core members of the work group are
listed in Figure 1. The intent of the group was to conduct an
ONS SIG-sponsored survey. Gail Mallory, PhD, RN, CNAA,
Several strategies were used to examine the validity and
the current ONS director of research, and Judy DePalma, PhD,
reliability of the measure (i.e., literature review, expert judge
RN, who was the ONS senior research associate at the time
panel evaluation, focus group testing, and pilot testing).
that this study was conducted, were asked to initially serve on
Changes in the instrument, including item reduction and
the work group in an advisory capacity. The work group set
modification of the response format, were made through con-
goals, developed action items, and collaborated in the devel-
sensus of the work group members during the instrument test-
opment of the survey items (Ehrenberger et al., 2003). Each
ing and feasibility process. Before proceeding with the study,
step of instrument development was approached to enhance
approval was obtained from the Lincoln Memorial University
the accuracy, precision, and sensitivity of the final measure
Institutional Review Board (IRB). The project presented sub-
(Mishel, 1998; Waltz, Strickland, & Lenz, 1991).
jects with no more than minimal risk as defined by applicable
federal regulations, thus satisfying the criteria for exemption
Conceptual Basis
status through the IRB.
Item Generation
The Nursing Role Effectiveness Model, based on the struc-
ture, process, and outcome model of quality care, was used to
Sources used for the item generation included a comprehen-
guide instrument development (Irvine, Sidani, & McGillis-
sive review of the published literature, the Manual for Clini-
Hall, 1998; Sidani & Irvine, 1999). This framework has been
cal Trials Nursing (Klimaszewski et al., 2000), an analysis of
used to identify nurses' roles in health care and relate these
25 diverse CTN-related position descriptions, a review of the
roles to specific patient and system outcomes. It was adapted
clinical research coordinator role and certification from the
Association of Clinical Research Professionals (ACRP), the
clinical research associate role and certification from ACRP,
the clinical research professional role and certification from
· Monica Bacon, RN
the Society of Clinical Research Associates, and the CTN ex-
Kingston, Canada
perience of work group members.
· Kathy Burks, RN, MSN, OCN®
Practice standards, including the Standards of Oncology
Baltimore, MD
Nursing Practice (Brant, 1996) and the Statement on the
· Heidi Ehrenberger, PhD, RN, AOCN®
Knoxville, TN
Scope and Standards of Advanced Practice in Oncology Nurs-
· Linda Lillington, DNSc, RN, CCRC
ing (ONS, 1997), also were examined. In addition, the ICH
Torrance, CA
(1996) GCP guideline, an international ethical and scientific
· Kristi Ward, RN, BSN, OCN®
quality standard for the design, conduct, performance, moni-
Indianapolis, IN
toring, auditing, recording, analysis, and reporting of clinical
· Janet Zimmerman, RN, MS
trials was reviewed closely. Numerous discussions occurred
Princeton, NJ
among work group members as preliminary decisions regard-
Figure 1. Core Work Group Members of the Clinical Trial
ing item content, construction, format, and scaling were made.
Nurses Special Interest Group
Guided by the conceptual framework, items were selected and
assembled into a usable format using Dillman's techniques for
Note. Other individuals who provided assistance were Nancy Ellis, RN, BN, St.
questionnaire construction (Crosby, Ventura, & Feldman,
John's, Canada; Nancy LaSota, RN, MSN, AOCN®, Agoura Hills, CA; and
Kathleen Shedlock, MS, MPA, ANP, CS, AOCN®, Manilus, NY.
Table 1. Final Format of the Clinical Trials Nursing
The initial version of the questionnaire contained 12 sec-
tions with a total of 170 items. Sections 1­8 contained a com-
prehensive list of the various and diverse clinical research
Number of Items
nursing activities and responsibilities. The activities within
these eight sections were to be scaled by the respondent for
Protocol assessment
performance (i.e., frequency and importance). Section 9 in-
Protocol planning
cluded items regarding respondents' perceptions and experi-
Subject recruitment
ences in their role as nurses involved in clinical research. Sec-
Informed consent process
tion 10 included items related to the professional characteristics
Investigational product
of the nursing role (e.g., highest degree completed). Section
Implementation and evaluation
11 included items related to the employing organization (e.g.,
Data management
Professional nursing role performance
primary work setting, opportunity for advancement). Section
Professional nursing role perception
12 contained items to assess the demographic characteristics
Professional nursing role characteristics
of the professional nurse.
Organizational characteristics
Expert Judge Panel
Demographic information
Content validity was assessed using a six-member expert
Note. Questions in Sections 1­8 contain a frequency and an importance scale.
judge panel (see Figure 2). A structured procedure for the
evaluation of the content validity was given to each expert
lective results, reviewed by the work group, and finalized
(Lynn, 1986). Each expert independently rated the relevance
through consensus of the work group members.
of each item to an identified objective using a four-point rat-
ing scale: 1 = not relevant, 2 = somewhat relevant, 3 = quite
relevant, and 4 = very relevant. A content validity index was
calculated for each item and section and for the measure over-
Reliability Estimation
all (Lynn; Waltz et al., 1991). The overall content validity in-
Reliability of the present survey instrument was estimated
dex for the instrument was 0.95, which is the proportion of
using internal consistency (Cronbach's alpha) and test-retest
items rated as content valid (a rating of 3 or 4) by the six ex-
methods. The test-retest method provides an estimate of sta-
perts. The judge panel also identified area(s) that could be
bility over time. A sample of oncology clinical research nurses
omitted from the instrument and suggested areas of item im-
(N = 40) in the Southeast, representative of the relevant re-
provement or modification.
search population, completed the questionnaire on two occa-
Focus Group Testing
sions, two weeks apart. An acceptable test-retest reliability of
0.88 was obtained for the frequency scale and 0.92 for the
Focus groups were conducted to further validate the instru-
importance scale. Reliability coefficients typically range from
ment in three different geographic regions of the United
0.00 ­1.00. The closer the correlation coefficient is to 1.00, the
States: the West, Northeast, and Great Lakes. Several mem-
more reliable (stable) the measure is. Analysis of the instru-
bers of the work group were involved in facilitating or mod-
ment's internal consistency reliability revealed an alpha coef-
erating these focus groups. Participants were recruited from
ficient of 0.92 for the frequency scale and 0.95 for the impor-
the local ONS chapters within the geographic region and rep-
tance scale. Each scale was internally consistent. An alpha
resented individuals who currently were engaged in the sup-
coefficient of 0.70 is considered acceptable for a newly devel-
port of clinical research (N = 24). A script was used to guide
oped instrument (Burns & Grove, 1997).
the discussion of the focus groups. Participants were asked to
review the survey questions for clarity, comprehensiveness,
wording, and length. They also were asked to review the in-
Content validity of the Clinical Trials Nursing Question-
structions for clarity. Each focus group lasted approximately
naire© (CTNQ) was assessed systematically and quantified
60 minutes. In addition, five Canadian research nurses exam-
through a two-stage process described by Lynn (1986). The
ined the measure for language and acceptability. The pro-
developmental stage was accomplished through a thorough
posed instrument then was revised for clarity based on the col-
review of the literature, item generation, and instrument for-
mation. The judgment-quantification stage was accomplished
via the use of the expert judge panel. In addition, face valid-
· Jennifer Aikin, RN, MSN, AOCN®
ity was determined via focus groups in which participants
Jeannette, PA
were asked to review the questionnaire for acceptability of the
· Deborah Berg, RN, BSN
format, clarity and understanding of each item, and ease and
Derry, NH
· Donna Berry, RN, PhD
time of completion.
Seattle, WA
Final Questionnaire
· Marge Good, RN, BSN, MPH, OCN®
Wichita, KS
The final survey questionnaire, a self-administered paper-and-
· Angela Klimaszewski, RN, MSN
pencil tool, consisted of 12 sections for a total of 154 items (see
Canton, NY
Table 1). The first eight sections of the questionnaire (120 items)
· Debbie Wujcik, RN, MSN, AOCN®
assess the complex role components of the professional nurse
Nashville, TN
who uses the nursing process in the support of clinical re-
Figure 2. Members of the Expert Judge Panel
search. Examples of the items are given in Figure 3. Activities
the item is not applicable to the current role, then "not appli-
1. Protocol assessment
cable" is a possible response. Section 10 (11 items) contains
­ Assess a protocol for possible risks and inconveniences to the subject.
questions about the professional characteristics of the nursing
­ Consider the ability to maintain the rights, safety, and well-being of the
role (e.g., highest degree completed, years of experience, cer-
tification). Section 11 (9 items) consists of questions related
2. Protocol planning
to the employing organization (e.g., primary work setting, as-
­ Participate in study initiation meetings.
signed position title, opportunity for advancement). Section
­ Identify clinical staff learning needs, including those based on specific
12, the final section of the survey (4 items), contains questions
3. Subject recruitment
about the demographic characteristics of the professional
­ Communicate general information about the nature and goals of clini-
cal research to potential subjects.
­ Apply culturally sensitive recruitment strategies to increase minority
subject participation.
4. Informed consent process
The CTNQ was developed in an attempt to address frequent
­ Explain the study to the potential subject using the basic elements of
requests from the ONS membership regarding the role of the
informed consent (e.g., purpose, benefits, risks).
clinical research nurse. For a newly developed measure, the
­ Assess the potential subject's understanding of the consent form infor-
CTNQ has demonstrated acceptable initial validity and reli-
ability. Additional testing is recommended in a larger, more
5. Investigational product
­ Educate members of the research team about the use of the investiga-
varied population of nurses working in the research setting.
tional product.
An international survey of ONS members working in clinical
­ Provide patient teaching about the investigational product (e.g., poten-
research in various capacities has been completed recently us-
tial side effects).
ing the CTNQ. This larger study will allow additional testing
6. Implementation and evaluation
of the instrument to further establish the reliability and valid-
­ Perform psychosocial assessment of the subject and family.
ity of the tool. Since its development, wide interest in using
­ Assess and document identified toxicities and adverse events per pro-
the tool has been expressed by other groups of nurses. The
tocol-specific criteria.
General Clinical Research Centers (GCRCs) Nurse Manager
7. Data management
Group has requested permission to use the CTNQ to examine
­ Identify problems in data collection and management.
the role and responsibilities of the nurses working in GCRCs
­ Ensure subject records are protected in accordance with applicable regu-
latory requirement(s).
as a means of establishing general competencies required for
8. Professional nursing role performance
basic and advanced levels of practice in these settings. Gradu-
­ Identify and support the discussion of ethical issues related to clinical
ate nursing students in clinical research masters' programs and
the Canadian Clinical Trials Research Nurses SIG also have
­ Participate in the orientation or training of new research staff.
expressed interest in using the tool, suggesting that the delin-
eation of the unique contributions that nursing makes to the
Figure 3. Examples of Items From Subscales 1­8
clinical research setting is needed desperately.
and responsibilities that nurses may assume in a variety of
clinical research nursing roles are scaled by the respondent for
The CTNQ is considered a promising instrument for assess-
frequency and importance.
ing the research nurse role, and its use in further research is
The respondent indicates the frequency with which he or
appropriate, particularly among oncology clinical research
she has performed the activity during the prior year using a
nurses. Previous efforts by Bowen and Rice (1998) to answer
scale of 0 (never, not part of my role), 1 (once or twice), 2 (oc-
the question, "Who is a clinical research nurse?" may now
casionally, as needed), 3 (repeatedly, at various times), and 4
begin to be addressed systematically.
(extremely frequently). The respondent indicates the impor-
tance of the activity to the safe and effective practice of clini-
The authors gratefully acknowledge Angie Stengel, BA, ONS director of
cal trials nursing using a scale of 0 (not important) to 4 (very
membership and leadership, for her assistance with this endeavor.
Section 9 (10 items) asks for the respondents' perceptions
Author Contact: Requests for the Clinical Trials Nursing Question-
and experiences related to the role of a nurse involved in clini-
naire© and permission to use can be sent to Heidi E. Ehrenberger,
cal research. Respondents answer using a five-point Likert-
PhD, RN, AOCN®, at ehrenber@umich.edu, with copy to editor at
type scale from 1 (strongly agree) to 5 (strongly disagree). If
Bowen, K., & Rice, L. (1998). Who is a clinical research nurse? Establishing
methodology: Dillman's total design method. Nursing Research, 38, 56­58.
guidelines and standards of practice for a growing profession. Research
Ehrenberger, H.E., Lillington, L., Bacon, M., Burks, K., Ward, K., &
Nurse, 4(4), 1­4.
Zimmerman, J. (2003). The development of a virtual work group of the On-
Brant, J.M. (1996). Standards of oncology nursing practice. Pittsburgh, PA:
cology Nursing Society Clinical Trials Nurses Special Interest Group: Keys
Oncology Nursing Society.
to productivity [Abstract 237]. Oncology Nursing Forum, 30(2 Suppl.),
Burns, N., & Grove, S.K. (1997). The practice of nursing research: Conduct,
critique, and utilization (3rd ed.). Philadelphia: W.B. Saunders.
International Conference on Harmonisation of Technical Requirements for
Crosby, F., Ventura, M., & Feldman, J. (1989). Examination of a survey
Registration of Pharmaceuticals for Human Use. (1996). Guidance for in-
dustry: E6 good clinical practice: Consolidated guidance. Retrieved April
Oncology Nursing Society. (1998). Cancer research and cancer clinical trials
10, 2003, from http://www.fda.gov/cder/guidance/index.htm
[Position statement]. Oncology Nursing Forum, 25, 973­974.
Irvine, D., Sidani, S., & McGillis-Hall, L. (1998). Finding value in nursing
Sidani, S., & Irvine, D. (1999). A conceptual framework for evaluating the
care: A framework for quality improvement and clinical evaluation. Nurs-
nurse practitioner role in acute care settings. Journal of Advanced Nurs-
ing Economics, 16, 110­116.
ing, 30, 58­66.
Joshi, T.G., & Ehrenberger, H.E. (2001). Cancer clinical trials in the new
Waltz, C.F., Strickland, O.L., & Lenz, E.R. (1991). Measurement in nursing
millennium: Novel challenges and opportunities for oncology nursing.
research. Philadelphia: F.A. Davis.
Clinical Journal of Oncology Nursing, 5, 147­152.
Klimaszewski, A., Aikin, J., Bacon, M., Distasio, S., Ehrenberger, H., & Ford,
For more information . . .
B. (2000). Manual for clinical trials nursing. Pittsburgh, PA: Oncology
Nursing Society.
Lynn, M.R. (1986). Determination and quantification of content validity.
National Institute of Nursing Research
Nursing Research, 35, 382­385.
Mishel, M.H. (1998). Methodological studies: Instrument development. In P.J.
Oncology Nursing Society Clinical Trial Nurses Special
Brink & M.J. Wood (Eds.), Advanced design in nursing research (pp. 235­
Interest Group
286). Thousand Oaks, CA: Sage.
Mueller, M. (2001). From delegation to specialization: Nurses and clinical trial
co-ordination. Nursing Inquiry, 8, 182­190.
Oncology Nursing Society. (1997). Statement on the scope and standards
Links can be found at www.ons.org.
of advanced practice in oncology nursing (2nd ed.). Pittsburgh, PA: Au-