This material is protected by U.S. copyright law. Unauthorized reproduction is prohibited.
To purchase quantity reprints or request permission to reproduce multiple copies, please e-mail reprints@ons.org.
Computerized Symptom and Quality-of-Life
Assessment for Patients With Cancer
Part I: Development and Pilot Testing
Donna L. Berry, PhD, RN, AOCN, Lisa J. Trigg, MN, ARNP, William B. Lober, MD,
Bryant T. Karras, MD, Mary L. Galligan, RN, BSN, Mary Austin-Seymour, MD,
and Stephanie Martin, MSW, LICSW
pain specialist, social worker, nutritionist) for further evaluation. Such
Purpose/Objectives: To develop and test an innovative computerized
system enhancement could greatly facilitate oncology nurses' coordina-
symptom and quality-of-life (QOL) assessment for patients with cancer
tion role in caring for complex patients with cancer.
who are evaluated for and treated with radiation therapy.
Design: Descriptive, longitudinal prototype development and cross-
sectional clinical data.
Setting: Department of radiation oncology in an urban, academic
medical center.
Key Points . . .
Sample: 101 outpatients who were evaluated for radiation therapy,
able to communicate in English (or through one of many interpreters
available at the University of Washington), and competent to understand
Although cancer symptoms and quality-of-life issues are of
the study information and give informed consent. Six clinicians caring for
the highest priority to oncology nurse clinicians, little time is
the patients in the sample were enrolled.
available in most clinical settings for comprehensive screening
Methods: Iterative prototype development was conducted using a
and assessment of problematic areas.
standing focus group of clinicians. The software was developed based on
Advances in computer and Internet technologies have made
survey markup language and implemented in a wireless, Web-based for-
electronic assessment a feasible and attractive method of gath-
mat. Patient participants completed the computerized assessment prior
ering patient-reported information in busy clinical settings.
to consultation with the radiation physician. Graphical output pages with
Patient-generated symptom and quality-of-life assessment is
flagged areas of symptom distress or troublesome QOL issues were
feasible in an ambulatory setting with patients with various
made available to consulting physicians and nurses.
levels of education and prior computer use.
Main Research Variables: Pain intensity, symptoms, QOL, and de-
Instruments: Computerized versions of a 010 Pain Intensity Numeri-
cal Scale (PINS), Symptom Distress Scale, and Short Form-8.
Findings: Focus group recommendations included clinician priorities
Donna L. Berry, PhD, RN, AOCN  , is an associate professor in
of brevity, flexibility, and simplicity for both input interface and output and
Biobehavioral Nursing and Health Systems in the School of Nursing,
that the assessment output contain color graphic display. Patient partici-
Lisa J. Trigg, MN, ARNP, is a predoctoral student in the School of
pants included 45 women and 56 men with a mean age of 52.7 years (SD
Nursing and a National Library of Medicine postgraduate fellow in the
= 13.8). Fewer than half of the participants (40%) reported using a com-
Division of Biomedical and Health Informatics, William B. Lober, MD,
puter on a regular basis (weekly or daily). Completion time averaged 7.8
is a research assistant professor in the Division of Biomedical and
minutes (SD = 3.7). Moderate to high levels of distress were reported
Health Informatics in the Department of Medical Education, and
more often for fatigue, pain, and emotional issues than for other symp-
Bryant T. Karras, MD, is an assistant professor in the Department of
toms or concerns.
Health Services in the School of Public Health and Community Medi-
Conclusions: Computerized assessment of cancer symptoms and
cine, all at the University of Washington in Seattle; Mary L. Galligan,
QOL is technically possible and feasible in an ambulatory cancer clinic.
RN, BSN, is a staff nurse in radiation oncology at the Seattle Cancer
A wireless, Web-based system facilitates access to results and data en-
Care Alliance in Washington; Mary Austin-Seymour, MD, is a profes-
try and retrieval. The symptom and QOL profiles of these patients new to
sor in radiation oncology at the University of Washington Medical
radiation therapy were comparable to other samples of outpatients with
Center in Seattle; and Stephanie Martin, MSW, LICSW, is an oncology
social worker in radiation oncology at the University of Washington
Implications for Nursing: The ability to capture an easily interpreted
Medical Center. This study was supported by grants from the ONS
illustration of a patient's symptom and QOL experience in less than 10
Foundation (RE-07) and the Research Intramural Funding Program
minutes is a potentially useful adjunct to traditional face-to-face interview-
of the School of Nursing at the University of Washington. (Submitted
ing. Ultimately, electronic patient-generated data could produce auto-
May 2003. Accepted for publication March 25, 2004.)
mated "red flags" directed to the most appropriate clinicians (e.g., nurse,
Digital Object Identifier: 10.1188/04.ONF.E75-E83
ancer symptoms and quality-of-life (QOL) issues are
and the apparent impact of such experiences on the dimensions
of the highest priority to oncology nurse clinicians and
of QOL, the consequences of inadequate symptom manage-
researchers (King et al., 1997; Ropka et al., 2002). The
ment are complex and can be overwhelming to patients and
workdays of oncology nurses are focused on symptom assess-
their caregivers. For example, uncontrolled pain may shorten
ment, treatment, and patient teaching. Oncology nurses work
survival (Page & Ben-Eliyahu, 1998), severe mucositis or radia-
in complex healthcare systems along with other professionals
tion-associated skin changes can put patients at risk for addi-
with whom the shared vision of excellent cancer care becomes
tional complications (Wujcik, 1999), and nausea, vomiting, di-
reality. Nurses are the cancer care professionals who approach
arrhea, and anorexia not only can compromise nutritional status
patients as individuals who experience cancer in all aspects of
but also can affect cognition, mobility, and metabolism
their lives: physiologic, emotional, sensory, and behavioral
(Wickham, 1999). Early screening for psychosocial distress
(Heitkemper, Levy, Jarrett, & Bond, 1995). However, clini-
may enable healthcare providers to identify patients at higher
risk and initiate interventions to prevent development of crisis
cians are faced too often with shrinking resources, removing
opportunities for lengthy interpersonal interactions with pa-
events. Depression may increase disability, morbidity, and
tients. Patients' symptom and QOL experiences, reported in
mortality by compromising adherence with treatment, rehabili-
tation, and risk-factor modification. DiMatteo, Lepper, and
a reliable and efficient way, must take their place as a compo-
Croghan (2000) showed that depression was associated with a
nent of the essential information on which complete clinical
assessments, diagnoses, and treatment plans are based. This
threefold decrease in adherence to medical regimens. Yet recent
article is the first of two that report the results of an innova-
trends in managed care to reduce time spent with each patient
have rendered existing screening measures less feasible (Pirl &
tive approach to collecting patient-generated symptom and
Roth, 1999). Making cancer symptoms and QOL issues visible
QOL information during clinical assessment. Part I reports
findings related to the technical development of a computer-
and discussed in the clinic can promote partnership between
clinicians and patients, validating the patients' experiences and
ized assessment program and the outcomes of symptom dis-
enhancing communication and satisfaction (Velikova, Brown,
tress and QOL measures. Part II describes the acceptability
Smith, & Selby, 2002). Rapid, predictive screening may help to
and usability of the program.
reduce healthcare costs and prolonged medical treatments, as
well as enhance QOL and increase patient satisfaction
Literature Review
(Andersen, 1992). The challenge of efficient, systematic, and
meaningful assessment is very important and timely in current
A patient's symptom experience typically is considered a
clinical cancer settings.
component of QOL in the cancer context (Cella, Chang, Lai,
Because of developments in the science of cancer screening,
& Webster, 2002). The relationship between cancer symp-
detection, and treatment, plus collaborative involvement of
toms and QOL is intuitive for cancer clinicians. Not only are
various professionals and consumers, assessment of QOL and
cancer symptoms indications of physiologic changes associ-
symptoms now is an established, integral component of cancer
ated with disease and treatment toxicity, but they also reflect
clinical trials. Systematic assessment occurs less frequently
linkages to patients' perceived reality, including social, psy-
outside of the research setting (Ganz, 1995; Levine & Ganz,
chological, and cultural factors (Faithfull, 1995). Pain and
2002). Many instruments have been developed for use in re-
fatigue are the two preeminent, significant symptoms that
search and may not be adapted easily to clinical patient care
are addressed in virtually every publication about cancer ex-
settings. The clinical interpretation of the meaning of QOL
periences and supportive care of people with cancer. As re-
scores has not been publicized or taught during clinical training
viewed by Cleeland (2000), the high incidence of cancer-
of professionals. Although major comprehensive cancer centers
related pain and fatigue has been studied and reported often,
have developed and reported routine clinical use of
along with dyspnea, depression, and cognitive deficits. The
multisymptom assessment tools (Cleeland, 2000; Portenoy,
prevalence of elevated psychosocial distress among patients
Thaler, Kornblith, Lepore, Friedlander-Klar, Kiyasu, et al.,
with cancer at diagnosis and recurrence is reported to be
1994), the routine clinical use of QOL questionnaires (which
about 30% (Zabora et al., 1997), with clinical depression
include symptom scales) has been reported on a limited basis,
occurring in about 25% (Pirl & Roth, 1999). Given, Given,
typically not in the United States (Detmar, Muller, Schornagel,
Azzouz, Stommel, and Kozachik (2000) noted that most re-
Wever, & Aaronson, 2002; Levine & Ganz; Taenzer et al.,
search literature about cancer treatment and symptoms as-
2000; Velikova et al., 2002; Wright et al., 2003).
sumes that symptoms mediate between types of treatment
Portable electronic technology has been developed and
and diminishing aspects of function. The investigators com-
tested for patient self-reporting of symptom and QOL data
bined two symptom assessment scales, the Memorial Symp-
using validated survey instruments delivered on notebook
tom Assessment Scale (Portenoy, Thaler, Kornblith, Lepore,
computers (Trigg, Berry, Karras, Austin-Seymour, & Lober,
Friedlander-Klar, Coyle, et al., 1994) and the Symptom Dis-
2003). Not only do these devices eliminate the usual steps of
tress Scale (SDS) (McCorkle, 1987), to measure cancer
abstracting patient interview data or computerizing marked
symptoms over one year in 907 patients newly diagnosed
responses, but the interface also permits customized, confi-
with cancer who were aged 65 and older and reported the
dential, and private assistance in completely answering que-
most common symptoms as fatigue, pain, dry mouth, trouble
ries. Patients with diabetes, ulcerative colitis, mental illness,
sleeping, and weakness. Not only did patients with cancer
and cancer have participated successfully in the evaluation of
report frequent symptom experiences, the severity and asso-
computerized clinical assessment strategies. Although the
ciated distress were rated highly as well, particularly for pain
reports are few, the results are positive, data integrity is en-
and fatigue.
hanced by computerized assessment, patients with cancer are
With the incidence of cancer symptoms and groups of symp-
able to complete assessment (Trigg et al.), and they report a
toms at such a high level across various diagnoses and stages
preference for a computerized version (Mullen, Berry, &
sample of 107 outpatients who were newly evaluated for ra-
Zierler, 2004). The time has come for nurses and their col-
diation therapy, able to communicate in English (or through
leagues in oncology settings to deploy electronic systems to
one of many interpreters available at the medical center), and
streamline and prioritize delivery of care to individual patients
competent to understand the study information and give in-
and to build electronic databases with clinical outcome data.
formed consent were invited to participate. Exclusion criteria
The benefits of moving from oral interview or paper ques-
included those who were being evaluated for total body irra-
tionnaire to electronic assessment have been described and
diation, gamma knife stereotactic radiosurgery, or combined
studied since the 1970s and more recently in oncology settings
modality radiation and neurosurgery.
(Allenby, Matthews, Beresford, & McLachlan, 2002; Carlson,
Speca, Hagen, & Taenzer, 2001; Taenzer et al., 2000;
Velikova et al., 2002; Wilkie et al., 2003; Wright et al., 2003).
The Short Form-8 (SF-8) is a short-form, multipurpose
health survey developed by Ware (2000) based on extensive
Four groups of investigators have evaluated formal ac-
ceptability of computerized screening (Allenby et al., 2002;
previous work with the SF-36 and SF-12 (Quality Metrics,
Carlson et al., 2001; Mullen et al., 2004; Wilkie et al., 2003),
Inc., 2004). The subscales and component summary scales of
the SF-8 have been evaluated in population-based studies. The
a n d the University of Washington (Mullen et al.) and
Carlson et al. in Canada have reported on utility. The re-
survey contains one item for each of eight subscales: physical
search group at the University of Washington reported pre-
function, role physical, bodily pain, general health, mental
liminary feasibility of a Web-based assessment in their cur-
health, role emotional, social function, and vitality. Each item
rent program of research as evidenced by high enrollment
allows a response on a five- or six-point categorical scale. The
and completion rates in a diverse sample of adults with can-
paper versions of the SF scales have been used often in the
cer (Trigg et al., 2003).
cancer research setting. Yet the SF-8 does not cover all issues
The radiation therapy setting for the current study provided
considered relevant and important in oncology patient care.
the type of ambulatory cancer care service that will lead the
Among the content not included in the SF-8 and relevant to
future of multidisciplinary oncology care. Nurses, physicians,
the oncology patient care setting and sample are sexuality,
social workers, and nutritionists are focused on coordinated
sleep adequacy, cognitive functioning, and appetite.
cancer care that demands efficient communication of all types
The SDS is a 13-item, cancer-specific symptom assess-
of clinical information. The purpose of this pilot study was to
ment developed and tested by oncology nurse researchers for
develop and test a computerized symptom and QOL assess-
two decades (McCorkle, Cooley, & Shea, 2000). A literature
ment for patients with cancer who are evaluated for and
review and patient interviews were used to generate items
treated with radiation therapy.
for the SDS. McCorkle and Young (1978) used a known
group method to establish construct validity for the SDS.
Scores range from 1365, with a higher score indicating
higher symptom distress. Although cut scores have not been
established empirically, McCorkle et al. recommended inter-
vention for a score indicating moderate distress (25) or se-
This descriptive, multimethod study established the necessary
vere distress (33). The SDS has been used in more than 50
infrastructure, procedures, and database to pilot test a comput-
clinical cancer studies, including multisite, international tri-
erized QOL and symptom assessment in an outpatient clinical
als (McCorkle & Quint-Benoliel, 1983). A substantial body
cancer setting. The study began in June 2001 and continued
of literature now supports the reliability and validity of the
through September 2002. Approval from the Human Subjects
SDS (McCorkle et al.). Results of studies have underscored
Division at the University of Washington was in place at all
the usefulness of the SDS in examining the relationship be-
times during data collection and analysis.
tween symptom distress and QOL. The clinical utility of the
SDS in improving patient outcomes also has been docu-
mented in a number of studies. In fact, the SDS has been rec-
The study was conducted in the department of radiation
ommended as an instrument to screen patients who may be
oncology at the University of Washington Medical Center in
in need of closer follow-up (Degner & Sloan, 1995; Peruselli
Seattle, where a full complement of certified specialists, train-
et al., 1992).
ees, and support staff actively provide state-of-the-science
Three additional items were included to complement these
care to patients with cancer. Traditionally, all new patients
instruments: an 11-point (010) pain-intensity numerical scale
receiving radiation receive a mailed, paper information and
(McGuire, 1984) (see Figure 1) and two 5-point response
health history form with instructions to complete the form and
items related to fever and chills and the impact of cancer on
turn it in at their clinic visit. The health history component of
sexual interest and function.
the one-time form includes several symptom queries in a re-
view of systems framework. For patients receiving radiation
therapy, opportunity for ongoing verbal pain assessment oc-
curs at least during each weekly doctor visit and at times more
Software development and application: The investigative
frequently through additional nursing assessments. Progress
team used a standing focus group of clinicians to determine
notes are available for written documentation.
program modifications needed to improve the specifications
for display. The rapid application development (Beynon-
Davies & Holmes, 1998) and refinement based on feedback
Full-time clinicians (eight total) in radiation oncology were
were intended to ensure that the clinicians would find the
invited to participate in the prototype development evaluation
graphical output useful. The focus group met in the second
of the computerized assessment output. A convenience
and fourth months of the project. Examples of the output were
a v a i l a b l e freely in the open-
source domain.
The software infrastructure
supports secure queries for identi-
fied and deidentified data, with
independent authorization and
auditing structures for clinical re-
porting and research use. It effec-
tively produces a functionally
deidentified research copy of the
Figure 1. Eleven-Point Pain-Intensity Numerical Scale
database while maintaining the
simpler design and maintenance
of a single database. The software
presented, and participants gave suggestions for improvement.
runs on a secure, centrally located server that can be accessed
Focus group discussions were recorded on audiotape.
by any Internet-enabled computer with the proper authorization
codes. The application is implemented as a Web-based appli-
The software is a Web application built with open-source
technology and uses a forms-based Web protocol to commu-
cation accessed over a wireless network. The clinical site has
nicate between a central server and a laptop browser. (See
been equipped with wireless connectivity for this project only,
with a LinkSysTM (Linksys Broadband and Wireless Network-
inset for explanations of technical terms.) The software was
developed iteratively by a programming staff, using the open-
ing, a division of Cisco Systems, Irvine, CA) WAP11 access
point using the 802.11b protocol and a Lucent Gold/OrinocoTM
source text-editing tool GNU Emacs (GNU Project, 2004) and
the Web development environment Allaire HomesiteTM
wireless card (Lucent Technologies, Bell Labs Innovations,
(Macromedia Inc., San Francisco, CA). The application is
Murray Hills, NJ). To ensure data security, access to the Web
application requires 128-bit SSL encryption (Cam & Trigg,
implemented as a completely server-side application, using a
MySQLTM database management system (MySQL-AB, 2003).
2002). Further network security is achieved by restriction of use
The software was written in the PHP: hypertext preprocessor
of the wireless network to authenticated users, from specific
language and runs on the Linux-Apache-MySQL-Perl-PHP
PCs to specific servers. This network security combined with
(LAMP) platform (Dougherty, 2001) with standard personal
the mandatory end-to-end encryption addresses the inherent in-
computer (PC) server hardware. The LAMP platform is at-
securities of wireless networking while supporting the mobil-
ity required for this project.
tractive for this purpose because it is supported widely and
Explanation of Technical Terms
Open source: A philosophy and practice of software development and dis-
system, Apache is a Web server, MySQL is a database program, and
tribution characterized by the free distribution of software source code
Perl, Python, and PHP are server-side programming languages with
and redistribution rights for modified code or derived works. This ap-
their own parsers or machine-reader applications. They are well main-
proach enables volunteer developers or end users to further develop and
tained by the volunteer open-source community and freely distributed.
customize the software by building on existing code. Contrast this to pro-
They frequently are used together in open-source development and dis-
prietary code, which is not made publicly available to end users or soft-
tribution projects because they are reliable, robust, and economically
ware developers in the community. It has been compared to the scien-
attractive tools (Dougherty, 2001).
tific method, where results and methods are shared freely with the
Authentication, authorization, and auditing structures: These are the ba-
scientific community (Dibona, Ockman, & Stone, 1999; Howe, 2004b).
sis of security between Internet applications and sometimes are referred
Server-side processing: A Web programming technique in which the appli-
to as AAA structures (Piskiel, 2000).
cation software runs on the Web server rather than the Web browser on the
Authentication asks, "Who are you, and how do we know you are who
end user, or "client" computer. This enables greater independence from
you claim to be?"
different browsers, at the cost of some decrease in user interface function
Authorization determines, "What are you permitted to do?"
and responsiveness. It also reduces the burden on the client computer for
Auditing services record, "What did you attempt to do with the software,
processing all of the code necessary to display the Web content. Server-
and was it successful or not?"
side processing contrasts with "client-side" processing, where much of the
Wireless network: A computer network connected through transmission of
code is executed by the browser in the end user computer (Howe, 2004a).
radio signals rather than wired connections. Most commonly, wireless
PHP: hypertext preprocessor language: PHP was developed in 1994 by
networks use a standard called 802.11b, or related "a" or "g" standards,
Rasumus Lerdorf to track visitors to his home page. PHP originally stood
which replaces a wired Ethernet network (Howe, 2004c).
for "personal home page," but as the capabilities of PHP increased, it
128-bit SSL (secure socket layer) encryption: A protocol used to encrypt
came to mean PHP: hypertext preprocessor (Ulman, 2001). PHP is a
messages sent through the Internet. It permits server and client comput-
widely used open-source, server-side scripting language used in Web de-
ers to negotiate an encryption algorithm to be used between them and
velopment. PHP "scripts" execute on the server and can be interspersed
can be used by the computers to "authenticate" each other. It was de-
with ordinary hypertext markup language (HTML) that normally is sent
signed by Netscape Communications Corporation to ensure privacy and
directly to the browser in the client computer. These "server-side" PHP
reliability between communicating applications by preventing eavesdrop-
processes can include the collection of data from a Web page, generation
ping, tampering, or message forgery. It does so by supporting data en-
of dynamic page content, and sending and receiving cookies. PHP can be
cryption and server authentication and ensuring message integrity. 128-
used in Web servers on all major operating systems (PHP Group, 2004).
bit refers to the length of the session key generated: The longer the key,
LAMP platform: An acronym for Linux-Apache-MySQL-Perl-Python-PHP,
the more secure the encryption. SSL is built into all major browsers
which comprises the "open-source platform." Linux is an operating
(Freier, Karlton, & Kocher, 1996; Netscape, 2003).
by the SF-8 health survey scores and the single-item sexual
Clinical implementation: At their initial radiation oncol-
ogy appointments, patients were invited to hear about the
activities and interest score; (b) symptoms were measured by
the summative symptom distress score and the pain intensity
computerized assessment and, if agreeable, were given the
number reported; (c) threshold scores were any score on the
laptop computer and consent screens to view. Consenting pa-
tients were asked to complete the computerized QOL and
SF-8, the SDS, or the sexual activities and interest item at or
above the third response for each item; (d) demographics were
symptom assessment using the touch-screen computer in the
queried onscreen to include age, gender, education, and prior
reception and waiting area, where privacy is most available.
In most cases, assessment was completed prior to visits with
computer experience; (e) data completeness was defined as
the percent of the items answered by the participant; and (f)
radiation oncology physicians. The program is intended to be
time to complete was recorded automatically by the software
user friendly and can be completed independently by patients;
program for each item. All numerical data were exported from
however, an assistant or student was assigned to help each pa-
the MySQL database into SPSS 10.0 (SPSS Inc., Chicago,
tient with any questions about the equipment or use of the
computerized questionnaire. The first screen of the program
IL) for analysis. Descriptive frequencies and measures of cen-
queries patients for permission to use the responses to the
tral tendency were used to describe sample characteristics.
questions for two purposes: a clinical database and a research
Percent of answers beyond the clinical threshold values were
study. Patients were able to agree to one, both, or neither pur-
pose. The assistant printed the graphic results of each com-
Standard scoring for the SF-8 and SDS were conducted.
pleted assessment and placed one copy on the clinic chart for
These scores plus the single items of sexual activities and in-
immediate use by the physician during that day's visit, gave
terest and pain intensity were described in measures of central
one copy to the RN, and placed an archive copy in a notebook
in the radiation treatment area, where patients' records are
The application provides a Web-based graphic user interface
Prototype Development
for the patient and survey assistant to use and produces a paper
report (see Figure 2) for the clinician to review prior to or dur-
Five radiation oncology clinicians met with the three inves-
ing the visit with the patient. The patient-entered data automati-
tigators in July 2001 for the initial focus group. An additional
cally populate a MySQL database on the central server.
clinician and another investigator joined the focus group in
September 2001. Physicians, nurses, and a social worker par-
Variables, Measures, and Analyses
ticipated in the iterative prototype development. The first rec-
Data collected during the clinical implementation of the
ommendations addressed clinician priorities of brevity, flex-
program were organized as follows: (a) QOL was measured
ibility, and simplicity for input interface and output. The
participants suggested that the
assessment output contain color
graphics, which normalized the
threshold categorical item values
at three or higher and displayed
the results in two methods: a ver-
tical rank order graph, so high
d i s t r e s s could be seen at the
"top," and a horizontal bar graph
that showed levels above and
below the threshold values for
each item in order of each query.
A final major suggestion in-
cluded placing truncated lan-
guage of the patient's selected
response in the vertical graph.
Web-based versions of the SF-
8 and SDS were developed with
a forms-based Web protocol ap-
proach, using the LAMP plat-
form as described. The instruc-
tions and wording of the original
scale items were maintained.
A l l responses could be an-
swered with a touch selection on
a computer screen or with a key-
b o a r d touchpad. Participants
could move forward and back-
ward by touching arrows at the
bottom of the screen. Skipping
Figure 2. Graphic Results for Clinicians
past items was allowed, but a
Table 1. Symptom Distress, Quality of Life, and Pain
query at the end of the program asked the respondent whether
Intensity Scores
he or she would like to answer the missed questions. The
laptop computer used is lightweight and has a privacy screen
(no one else can see the screen from any angle except that of
or Item
the individual participant).
The computerized QOL and symptom assessment also in-
13 65
13 45
cluded demographic items on separate screens. Acceptability
Quality of life (SF-8)
questions were added to the assessment to query the
Physical component
00 100
19.75 63.59
participant's opinions about using the electronic assessment
Mental component
00 100
18.13 64.24
(Mullen et al., 2004). Selected responses were stored automati-
Pain intensity
0 10
cally in a MySQL database file. A one-page color graphic out-
put was generated to display the results of each patient's entries.
SDS--Symptom Distress Scale; SF-8--Short Form-8
Clinical Findings
Analysis of enrollment records revealed that a total of 121
the SDS and at 0.89 and 0.86, respectively, for the SF-8 physi-
patients in radiation oncology were approached for participa-
cal component scale and mental component scale in this
tion in the study. Of these, 119 were eligible for the study and
sample. Only those with complete data are included in the
108 agreed to view the consent information onscreen. A de-
SDS scale scores. Those scores with missing items on the SF-
tailed analysis of reasons for nonparticipation has been re-
8 were calculated using the missing data estimation technique
ported elsewhere (Trigg et al., 2003). All but one patient be-
employed by Quality Metrics, Inc. (2004). The responses to
g a n answering the computerized assessment, with six
the single item on sexual activities and interest included 44%
choosing that their data be used only for clinical use. There-
reporting minimal or no effect and 27% reporting moderate to
fore, the sample for the following results included 101 partici-
great effect or impact of their diagnosis or disease. Seventy
pants, 45 women and 56 men with a mean age of 52.7 years
percent of the sample replied to this item. The number of par-
(SD = 13.8). Fewer than half of the participants (40%) re-
ticipants who selected responses at or above the threshold
ported using a computer weekly or daily. Although 9% chose
scores (indicating a level of distress or problem important to
not to report educational levels, 38% reported having an un-
address clinically) is listed for each item in Table 2.
dergraduate or graduate degree, and the majority (53%) re-
ported having received technical training, a high school edu-
cation, or less.
Inclusion of end users in the software design process is a
During deployment of the Web-based assessment tool, the
practice that has gained widespread acceptance in software
research team members met and resolved a variety of techni-
design and engineering and might be considered by many as
cal problems, including server, hardware, and wireless de-
a fundamental requirement of the design process. Involving
ployment issues. These occurred largely during system and in-
end users in various ways is an element of several software
frastructure upgrades to improve wireless network security
design approaches, including but not limited to participatory
and correct the code for "bugs" that crept into the software as
design and rapid application development (Beynon-Davies &
a result of vendor upgrades to components of the LAMP plat-
Holmes, 1998), contextual inquiry (Spinuzzi, 2000), value-
form software. Another technical interruption of the survey
sensitive design (Friedman, Kahn, & Borning, 2003), and
occurred as a result of a misunderstanding of the range of the
customer partnering (Hackos, Elser, & Hammer, 1997). These
secure wireless system, when patients (along with the laptop)
approaches represent a trend away from the "design expert"
were seated in a part of the clinic where the wireless signal
model in acknowledgement of the value of the insight and
could not reach. Finally, the research team members found
preferences of those who ultimately will use the end software
that implementation of such an application required that spe-
product. Involving end users is thought to increase empower-
cial attention be paid to how the application is integrated into
ment of the users (Spinuzzi) and result in software that more
general clinic workflow. For example, a number of patient
closely meets the needs of those who will use the system
assessments were interrupted when the residents, eager to stay
(Hackos et al.). The research team members found the clini-
ahead of their caseloads, called for patients early and, there-
cians eager to participate in focus groups whose purpose was
fore, patients did not have time to complete their assessments
to improve the clinician report design.
prior to seeing clinicians.
The diversity in this sample with regard to age, education,
Because of symptom distress, difficulties with the wireless
and computer use affirms not only the willingness to partici-
connection, or interruptions by clinic staff, 19 participants
pate in such a pilot study, but also the ability of literate people
were unable to fully complete the electronic assessment. Of
with minimal education to use a simple, touch-screen com-
the remaining 82 (81%) participants who finished, 70 com-
puter interface. These findings are consistent with a study
pleted the assessment without major interruption. The 12 par-
conducted in cancer clinics and wards in the United Kingdom
ticipants who were interrupted by clinic staff or the appear-
by Wright et al. (2003); education levels were not predictive
ance of the consulting physician paused the assessment and
of using an automated QOL screening program on touch-
continued answering as soon as they could. For those without
screen computers, and younger age was associated with higher
external interruptions, the total time required to answer the
use levels. Most of the British sample also reported that they
symptom and QOL components of the assessment averaged
were not frequent computer users.
7.8 minutes (SD = 3.7).
The average length of time required to answer the ques-
The SDS, SF-8, and pain intensity scores are summarized
tions, 7.8 minutes, was in a reasonable and practical range.
in Table 1. Internal consistency was calculated at a = 0.83 for
Table 2. Participants Responding at or Above Threshold
ture evaluations must include opportunity to complete the
Score for Each Item
assessment and protection from unwarranted interruptions.
Routine application of screening procedures for symptoms
and QOL information will necessitate permanent change in
n Who
% at or Above
the study site's check-in procedures that are founded on time-
efficient patient flow patterns. This is the same conclusion
Short Form-8
reached by Wright et al. (2003); overall compliance with lon-
General health
gitudinal, automated QOL screening in cancer services rose
Physical activities limited by
from an average of 43% to 72% when the procedure changed
physical health (physical func-
from a prospective cohort study without staff prompting to a
regular clinic procedure facilitated by check-in staff.
Difficulty doing daily work be-
The findings describe a sample of patients with cancer who
cause of physical health (role
were about to embark on major cancer therapy. Knowing and
documenting baseline information about patients' symptom
Bodily pain
experiences are accepted universally as requisites to treatment
Energy (vitality)
planning and patient education (Yarbro, Frogge, & Goodman,
3, 4b
Limited social activities (social
2003). Yet routine clinical screening of cancer symptoms and
3, 4b
Emotional problems (mental
other QOL dimensions has not been adapted widely and has
been evaluated and reported infrequently. One reason for this
Emotional problems limiting daily
lack of clinical application is the unfamiliarity with the mean-
activities (role emotional)
ing of QOL and symptom scores on the part of practicing can-
Symptom Distress Scale
cer specialists and oncology nurses (Soni & Cella, 2002). In-
Nausea frequency
deed, the clinician participants in this study voiced no need for
Nausea severity
output in the clinic that reported numerical scores. The results
of item responses in tabular and graphic form, highlighting
moderate to high levels of distress in color, were requested.
Pain frequency
Pain severity
The clinicians in the standing focus group had agreed on a
threshold score of three out of five on the SDS and corre-
Bowel pattern
sponding levels on the SF-8. At this point for each item, the
clinicians believed that either further assessment or interven-
tion would be appropriate. Because almost half of the sample
reported scores at or above the threshold for SDS items related
to fatigue and outlook (worry and fear about the future) and
about a third of the respondents identified threshold scores for
pain frequency and insomnia, intervention certainly would
1 = most frequently at or above threshold
have been appropriate. Hinds, Schum, and Srivastava (2002)
explored the clinical use of SDS threshold item scores in com-
parison to total scale scores in a sample of 77 adolescents with
Other reports of computerized screening in cancer settings
cancer. Using the same threshold of the third response, they
have used somewhat longer instruments, yet completion times
reported underestimation of symptom distress for most of the
were not distinctly different (e.g., 8.6 minutes reported by
items when only the scale score was considered. The authors
Taenzer et al., 2000).
concluded that valuable clinical information is gained by an
Technical difficulties are known to interrupt the perfor-
initial single-item approach to symptom assessment. In this
mance of any mechanistic or electronic system, particularly
study, the SDS mean score of 25 indicates a moderate level of
during pilot testing. In this study, wireless connectivity issues
symptom distress, comparable to four studies of newly diag-
were addressed easily once discovered. Because few authors
nosed (within 100 days of diagnosis) patients with cancer with
have conducted such evaluations of computerized patient as-
varying stages and diagnoses in which the mean SDS score
sessment and even fewer have reported the technical findings,
ranged from 22.827.5 (McCorkle et al., 2000). An analysis
the authors are challenged to place their findings in the con-
of the relationship between item scores and total scores is
text of others' work in health care. Wright et al. (2003) de-
scribed problems with patients inadvertently powering down
The physical component score (PCS) and the mental com-
the PC. A recent report of the computerized PAINReportIT
ponent score (MCS) from the SF-8 indicated worse and some-
(Wilkie et al., 2003) alluded to technical difficulties, describ-
what more variable QOL in both physical and mental compo-
ing an unknown number of comments by users about touch-
nents than in the general population ( X PCS and MCS = 50,
screen insensitivity and 15 system "crashes" occurring in 213
SD = 10). Specific comparisons of these scores to reported
subject sessions on a PC-based system. The current study's
PCS and MCS scores in the cancer literature are less useful
results indicate less severe, quickly correctable technical is-
because of the nature of this pilot study with mixed diagnoses
sues using a wireless, Web-based system.
and stages of disease. Clearly, the findings describe a sample
Most participants who did not complete the computerized
of patients with cancer who perceive themselves to be com-
assessment were interrupted by clinic staff to conduct routine
promised in physical and emotional role or function.
care (e.g., vital signs) and accompany patients from one area
Fatigue, pain, and emotional issues were placed in priority
of the clinic to another to wait for examining clinicians. Fu-
positions by the threshold rankings of symptom distress and
an easily interpreted illustration of a patient's symptom and
QOL items. Although pain frequency was the third most fre-
quently cited symptom of moderate to high distress, pain in-
QOL experience in less than 10 minutes is a potentially use-
ful adjunct to traditional face-to-face interviewing. Ultimately,
tensity was reported as less distressful. This could indicate a
electronic patient-generated data could be used in event-moni-
variety of situations, from well-managed pain control to pain
intensity levels associated with newer diagnoses. Completely
toring systems (Tao, Wang, Shortliffe, & Lussier, 2002) to
produce automated notifications or "red flags" directed to the
consistent with recent literature (National Institutes of Health,
most appropriate clinicians (e.g., nurse, pain specialist, social
2002), this constellation of problems is faced by the majority
of patients with cancer.
worker, nutritionist) for further evaluation and response. Such
system enhancement could greatly facilitate oncology nurses'
This developmental pilot study is limited by the very nature
coordination role in caring for these complex patients with
of its design. Single-institution findings cannot be generalized
beyond the academic radiation oncology setting. Minimal
cancer. Automated entry to a database and clinician access to
results from any Internet access point are added benefits. Suc-
study resources for data collection precluded chart reviews for
cess of future integration of patient-generated data into
additional demographic or clinical factors; therefore, differen-
tial analyses based on ethnicity or cancer diagnosis cannot be
housewide electronic medical record systems will be founded
on developmental efforts such as this study.
The authors wish to acknowledge Terri Whitney, Mary Monahan, RN, MN,
and the staff of the University of Washington Clinical Informatics Research
Group for their excellent assistance throughout the project.
Driven by a clinical need to enhance patient assessment,
this project and its findings reflect a future direction for oncol-
ogy clinical care integrated with informatics-based systems.
Author Contact: Donna L. Berry, PhD, RN, AOCN, can be reached
Computerized assessment of cancer symptoms and QOL is
at donnalb@u.washington.edu, with copy to editor at rose_mary@
feasible in an ambulatory cancer clinic. The ability to capture
Freier, A.O., Karlton, P., & Kocher, P.C. (1996). The SSL Protocol Version
Allenby, A., Matthews, J., Beresford, J., & McLachlan, S.A. (2002). The
3.0. Netscape Communications Corporation. Retrieved January 15, 2004,
application of computer touch-screen technology in screening for psycho-
from http://wp.netscape.com/eng/ssl3/draft302.txt
social distress in an ambulatory oncology setting. European Journal of
Friedman, B., Kahn, P.H., Jr., & Borning, A. (2003). Value sensitive design:
Cancer Care, 11, 245253.
Theory and methods. Retrieved February 1, 2004, from http://www.ischool
Andersen, B.L. (1992). Psychological interventions for cancer patients to
enhance the quality of life. Journal of Consulting and Clinical Psychology,
Ganz, P.A. (1995). Impact of quality of life outcomes on clinical practice.
60, 552568.
Oncology, 9(11, Suppl.), 6165.
Beynon-Davies, P., & Holmes, S. (1998). Integrating rapid application devel-
Given, C.W., Given, B., Azzouz, F., Stommel, M., & Kozachik, S. (2000).
opment and participatory design. IEEE Proceedings Software, 145, 105112.
Comparison of changes in physical functioning of elderly patients with
Cam, K., & Trigg, L. (2002). Cancer electronic quality of life project: Deliv-
new diagnoses of cancer. Medical Care, 38, 482493.
ering wireless Web-based technology at the point of care. Unpublished
GNU Project. (2004). GNU Emacs. Retrieved July 27, 2004, from http://
Carlson, L.E., Speca, M., Hagen, N., & Taenzer, P. (2001). Computerized
Hackos, J., Elser, A., & Hammer, M. (1997). Customer partnering: A new
quality-of-life screening in a cancer pain clinic. Journal of Palliative Care,
approach to needs analysis. IEEE Transactions on Professional Commu-
17, 4652.
nication, 40, 102110.
Cella, D., Chang, C.H., Lai, J.S., & Webster, K. (2002). Advances in quality
Heitkemper, M., Levy, R.L., Jarrett, M., & Bond, E.F. (1995). Interventions for
of life measurements in oncology patients. Seminars in Oncology, 29(3,
irritable bowel syndrome: A nursing model. Gastroenterology Nursing, 18,
Suppl. 8), 6068.
Cleeland, C.S. (2000). Cancer-related symptoms. Seminars in Radiation
Hinds, P.S., Schum, L., & Srivastava, D.K. (2002). Is clinical relevance some-
Oncology, 10, 175190.
times lost in summative scores? Western Journal of Nursing Research, 24,
Degner, L.F., & Sloan, J.A. (1995). Symptom distress in newly diagnosed
ambulatory cancer patients and as a predictor of survival in lung cancer.
Howe, D. (Ed.) (2004a). Client. Retrieved January 15, 2004, from http://
Journal of Pain and Symptom Management, 10, 423431.
Detmar, S.B., Muller, M.J., Schornagel, J.H., Wever, L.D., & Aaronson, N.K.
Howe, D. (Ed.) (2004b). Open source. Retrieved January 15, 2004, from
(2002). Health-related quality-of-life assessments and patient-physician
communication: A randomized controlled trial. JAMA, 288, 30273034.
Howe, D. (Ed.) (2004c). Wireless. Retrieved January 15, 2004, from http://
Dibona, C., Ockman, S., & Stone, M. (Eds.). (1999). Open sources: Voices
from the open source revolution. Retrieved January 15, 2004, from http:
King, C.R., Haberman, M., Berry, D.L., Bush, N., Butler, L., Dow, K.H., et
al. (1997). Quality of life and the cancer experience: The state-of-the-
DiMatteo, M.R., Lepper, H.S., & Croghan, T.W. (2000). Depression is a risk
knowledge. Oncology Nursing Forum, 24, 2741.
factor for noncompliance with medical treatment: Meta-analysis of the
Levine, M.N., & Ganz, P.A. (2002). Beyond the development of quality-of-
effects of anxiety and depression on patient adherence. Archives of Inter-
life instruments: Where do we go from here? Journal of Clinical Oncol-
nal Medicine, 160, 21012107.
ogy, 20, 22152216.
Dougherty, D. (2001). LAMP: The open source Web platform. Retrieved
McCorkle, R. (1987). The measurement of symptom distress. Seminars in
May 20, 2002, from http://www.onlamp.com/pub/a/onlamp/2001/01/25/
Oncology Nursing, 3, 248256.
McCorkle, R., Cooley, M., & Shea, J. (2000). A user's manual for the symp-
Faithfull, S. (1995). `Just grin and bear it and hope that it will go away':
tom distress scale. Unpublished manuscript.
Coping with urinary symptoms from pelvic radiotherapy. European Jour-
McCorkle, R., & Quint-Benoliel, J. (1983). Symptom distress, current con-
nal of Cancer Care, 4, 158165.
Suppl.), S560S573.
cerns and mood disturbance after diagnosis of life-threatening disease.
Spinuzzi, C. (2000, June). Investigating the technology-work relationship: A
Social Science and Medicine, 17, 431438.
critical comparison of three qualitative field methods. Paper presented at
McCorkle, R., & Young, K. (1978). Development of a symptom distress
the Proceedings of IEEE Professional Communication Society Interna-
scale. Cancer Nursing, 1, 373378.
tional Professional Communication Conference, Cambridge, MA.
McGuire, D.B. (1984). The measurement of clinical pain. Nursing Research,
Taenzer, P., Bultz, B.D., Carlson, L.E., Speca, M., DeGagne, T., Olson, K.,
33, 152156.
et al. (2000). Impact of computerized quality of life screening on physician
Mullen, K.H., Berry, D.L., & Zierler, B.K. (2004). Computerized symptom
behaviour and patient satisfaction in lung cancer outpatients.
and quality-of-life assessment for patients with cancer. Part II: Acceptabil-
Psychooncology, 9, 203213.
ity and usability. Oncology Nursing Forum, 31, 896.
Tao, Y., Wang, D., Shortliffe, E.H., & Lussier, Y.A. (2002). Extended at-
MySQL-AB. (2003). MySQL. Retrieved April 1, 2003, from http://www
tributes of event monitor systems for criteria-based notification modalities.
Proceedings of AMIA Symposium, 762766.
National Institutes of Health. (2002, October 26). State-of-the-science confer-
Trigg, L., Berry, D., Karras, B., Austin-Seymour, M., & Lober, W. (2003).
ence on symptom management in cancer: Pain, depression, and fatigue.
Feasibility of patient entered QOL assessment data in a radiation oncology
Retrieved January 21, 2003, from http://consensus.nih.gov/ta/022/
clinic. In H. Marin, E. Marques, E. Hovenga, & W. Goosen (Eds.), eHealth
for all: Designing nursing agenda for the future [CD-ROM]. Sau Palo,
Netscape. (2003). Secure sockets layer. Retrieved January 15, 2004, from
Brazil: NI 2003.
Ulman, L. (2001). PHP for the World Wide Web. Berkley, CA: Peachpit
Page, G.G., & Ben-Eliyahu, S. (1998). Pain kills: Animal models and neuro-
immunological links. Seattle, WA: IASP Press.
Velikova, G., Brown, J.M., Smith, A.B., & Selby, P.J. (2002). Computer-
Peruselli, C., Camporesi, E., Colombo, A.M., Cucci, M., Sironi, P.G., Bellodi,
based quality of life questionnaires may contribute to doctor-patient inter-
M., et al. (1992). Nursing care planning for terminally ill cancer patients
actions in oncology. British Journal of Cancer, 86, 5159.
receiving home care. Journal of Palliative Care, 8(4), 47.
Ware, J. (2000). SF-8 health survey. Retrieved July 27, 2004, from http://
PHP Group. (2004). What can PHP do? Retrieved January 15, 2004, from
Wickham, R. (1999). Nausea and vomiting. In C.H. Yarbro, M.H. Frogge, &
Pirl, W.F., & Roth, A.J. (1999). Diagnosis and treatment of depression in
M. Goodman (Eds.), Cancer symptom management (pp. 228263).
cancer patients. Oncology, 13, 12931302.
Sudbury, MA: Jones and Bartlett.
Piskiel, H. (2000). Rules-driven authorization. Securities Industry
Wilkie, D.J., Judge, M.K., Berry, D.L., Dell, J., Zong, S., & Gillespie, R.
Middleware Council, Inc. Retrieved January 15, 2004, from http://www
(2003). Usability of a computerized PAINReportIt in the general public
with pain and people with cancer pain. Journal of Pain and Symptom
Portenoy, R.K., Thaler, H.T., Kornblith, A.B., Lepore, J.M., Friedlander-Klar,
Management, 25(3), 213224.
H., Coyle, N., et al. (1994). Symptom prevalence, characteristics and dis-
Wright, E.P., Selby, P.J., Crawford, M., Gillibrand, A., Johnston, C., Perren,
tress in a cancer population. Quality of Life Research, 3, 183189.
T.J., et al. (2003). Feasibility and compliance of automated measurement
Portenoy, R.K., Thaler, H.T., Kornblith, A.B., Lepore, J.M., Friedlander-Klar,
of quality of life in oncology practice. Journal of Clinical Oncology, 21,
H., Kiyasu, E., et al. (1994). The Memorial Symptom Assessment Scale:
An instrument for the evaluation of symptom prevalence, characteristics
Wujcik, D. (1999). Infection. In M.H. Frogge, C.H. Yarbro, & M. Goodman
and distress. European Journal of Cancer, 9, 13261336.
(Eds.), Cancer symptom management (3rd ed., pp. 307327). Sudbury,
Quality Metrics, Inc. (2004). Comparisons among SF surveys. Retrieved Au-
MA: Jones and Bartlett.
gust 10, 2004, from http://www.qualitymetric.com/products/Compare
Yarbro, C.H., Frogge, M.H., & Goodman, M. (Eds.). (2003). Cancer symp-
tom management (3rd ed.). Sudbury, MA: Jones and Bartlett.
Ropka, M.E., Guterbock, T., Krebs, L., Murphy-Ende, K., Stetz, K., Sum-
Zabora, J.R., Blanchard, C.G., Smith, E.D., Roberts, C.S., Glajchen, M.,
mers, B., et al. (2002). Year 2000 Oncology Nursing Society Research
Sharp, J.W., et al. (1997). Prevalence of psychological distress among
Priorities Survey. Oncology Nursing Forum, 29, 481491.
cancer patients across the disease continuum. Journal of Psychosocial
Soni, M.K., & Cella, D. (2002). Quality of life and symptom measures in
Oncology, 15(2), 7387.
oncology: An overview. American Journal of Managed Care, 8(18,