Article

Use of the Distress Thermometer in Cancer Survivors: Convergent Validity and Diagnostic Accuracy in a Spanish Sample

Yolanda Andreu Vaillo

Paula Martínez Lopez

María J. Galdón Garrido

cancer survivors, Distress Thermometer, psychosocial distress, diagnostic accuracy
ONF 2019, 46(4), 442-450. DOI: 10.1188/19.ONF.442-450

Objectives: To explore the performance of the National Comprehensive Cancer Network Distress Thermometer (DT) as a distress screening tool in cancer survivors.

Sample & Setting: 236 Spanish adult-onset cancer survivors who visited the Fundación Instituto Valenciano de Oncología in Valencia, Spain, for follow-up appointments.

Methods & Variables: Survivors completed the DT and the Brief Symptom Inventory 18 (BSI-18), which has established a cutoff score for identifying clinically significant distress.

Results: Receiver operating characteristic curve analysis of the DT scores relative to the BSI-18 cutoff score showed good overall accuracy. For a score of 5 or greater, sensitivity, specificity, positive predictive value, negative predictive value, and clinical utility indexes indicated that the DT appeared to be satisfactory for screening but had restricted use for case finding.

Implications for Nursing: Screening for and responding to distress is considered an important part of nursing practice. The DT is suitable for use as a first-stage, quick-detection instrument in a two-step screening process to rule out noncases among Spanish post-treatment cancer survivors.

Jump to a section

    People who have survived cancer frequently experience long-term and late physical effects, such as pain and fatigue (Bower, 2007; Pachman, Barton, Swetz, & Loprinzi, 2012), as well as psychosocial and practical difficulties, such as fear of recurrence and issues surrounding employment, finances, and health and life insurance (Aaronson et al., 2014; de Boer, Taskila, Ojajärvi, van Dijk, & Verbeek, 2009; Duijts et al., 2014; Hoffman, McCarthy, Recklitis, & Ng, 2009; Stanton, 2012). Because of these challenges, cancer survivors are at increased risk for psychosocial distress, even many years after the completion of therapy, although definitive data regarding the prevalence of significant distress are lacking (the reported prevalence ranges from 5% to 43%) (Jefford et al., 2017; Kaiser, Hartoonian, & Owen, 2010; Mitchell, Ferguson, Gill, Paul, & Symonds, 2013; Ploos van Amstel et al., 2013; Wells et al., 2015). Psychosocial distress in cancer is defined by the National Comprehensive Cancer Network ([NCCN], 2017a) as follows:

    A multi-determined unpleasant emotional experience of a psychological, social, spiritual and/or physical nature that may interfere with the ability to cope effectively with cancer, its physical symptoms and its treatment. Distress extends along a continuum, ranging from common normal feelings of vulnerability, sadness, and fears to problems that can become disabling, such as depression, anxiety, panic, social isolation, and existential and spiritual crisis. (p. MS-4)

    In the early 2000s, distress was recognized as the sixth vital sign in cancer care (Bultz & Carlson, 2005); consequently, guidelines of a number of international professional societies and regulatory organizations (e.g., CanCon in Europe and NCCN in the United States) recommended integration of screening for distress as a regular step in the long-term follow-up care of people who have survived cancer (Albreht, Kiasuwa, & Van den Bulcke, 2017; NCCN, 2017b). Physical and psychological effects present challenges for healthcare systems, which have to guarantee appropriate follow-up care and quality of life, moving from how long people live after a diagnosis to how well people can expect to live from diagnosis onward (Albreht et al., 2017).

    The recommendation for the routine use of screening for distress after active treatment may benefit from the availability of an ultrashort instrument, such as the NCCN Distress Thermometer (DT). The DT, developed by Roth et al. (1998), is a single-item self-report method for identifying psychological distress ranging from 0 (no distress) to 10 (extreme distress). Since its introduction, the DT has been widely recommended for screening distress in people with cancer (NCCN, 2017a; Ma et al., 2014). Its potential advantages versus other distress screening tools are its brevity, ease of administration and scoring, and acceptability to healthcare providers and patients. In addition, many studies carried out in different countries and cultures have endorsed the usefulness and adequate accuracy of the DT to correctly identify clinically significant distress (a highly severe and impairing psychological response that requires professional support), despite some shortcomings—in particular, the lack of specificity of the DT to rule out false-positive cases and mixed findings regarding the optimal cutoff score (Donovan, Grassi, McGinty, & Jacobsen, 2014; Ma et al., 2014; Snowden et al., 2011). However, research on the use of the DT in the post-treatment survivor group is inconclusive. Studies in adult-onset cancer survivors are scarce, and the results are mixed. Most studies compare the performance of the DT with that of other longer instruments, like the Hospital Anxiety and Depression Scale (Boyes, D’Este, Carey, Lecathelinais, & Girgis, 2013; Roerink et al., 2013), the Brief Symptom Inventory 18 (BSI-18) (Merport, Bober, Grose, & Recklitis, 2012), the University of Washington Quality of Life scale (Ghazali et al., 2017), and even the Structured Clinical Interview for the Diagnostic and Statistical Manual of Mental Disorders (4th ed., depressive and anxiety disorder modules) (Recklitis, Blackmon, & Chang, 2016). The findings show that the accuracy of the DT to detect clinically significant distress in adult post-treatment survivors is not always satisfactory. The DT has shown higher values of sensitivity than specificity in some cases (72–93 versus 40–42) (Hong & Tian, 2013; Olesen et al., 2018) but higher values of specificity than sensitivity in others (52–60 versus 86–90) (Craike, Livingston, & Warne, 2011; Merport et al., 2012). In addition, there is a lack of consensus about the DT’s optimal cutoff score for clinically relevant distress; scores based on receiver operating characteristic curve analyses range from 2 to 5 across different studies and even in relation to the assessment’s proposed objective (Boyes et al., 2013). Therefore, more studies are needed to evaluate the performance of the DT (Ma et al., 2014) in post-treatment survival.

    In the Spanish context, the DT has been validated in people with heterogeneous cancer in terms of the phase of care (Gil, Grassi, Travado, Tomamichel, & González, 2005; Martínez, Andreu, Galdón, & Ibáñez, 2015; Martínez, Galdón, Andreu, & Ibáñez, 2013) but not specifically in post-treatment survivors. However, this stage of cancer trajectory validation is important; a review by Donovan et al. (2014) showed that cutoff scores for clinically relevant distress vary according to the characteristics of the patients being screened and the setting. Therefore, the aim of the current study was to examine the accuracy of the DT to detect clinically significant distress as assessed by the BSI-18 (criterion measure) in Spanish post-treatment survivors of adult-onset cancer.

    Methods

    Participants and Procedure

    A convenience sample of 254 participants who visited the Fundación Instituto Valenciano de Oncología in Valencia, Spain, for follow-up medical appointments were approached. The study was approved by the institutional ethics committee. To be eligible, the following was required of participants:

    •  Aged 18 years or older

    •  Diagnosed with cancer and without any symptoms or signs of cancer at the time

    •  Finished surgery, chemotherapy, or radiation therapy treatments at least four weeks before the study

    •  Had knowledge of the Spanish language

    Participants received information about the study, and 236 (93%) gave informed consent and completed the questionnaire package.

    Measures

    Sociodemographic and medical data: A self-report form developed for this study collected age, marital status, education level, and employment status. Cancer-related details (cancer type and time elapsed since the end of primary treatment) were gathered using a chart review.

    Distress Thermometer: The DT is a one-item visual analog scale that measures psychological distress (Roth et al., 1998). Individuals indicate on an 11-point scale (with 0 indicating no distress and 10 indicating extreme distress) how distressed they felt in the previous seven days.

    This study used the Spanish version of the DT, which has shown satisfactory diagnostic accuracy (area under the curve [AUC] = 0.82 or greater, sensitivity = 90% or greater, specificity = 64% or greater, predictive positive value [PPV] = 25% or greater, and negative predictive value [NPV] = 97% or greater for a selected DT cutoff of 5) (Martínez et al., 2013, 2015).

    Brief Symptom Inventory 18: The BSI-18 (Derogatis, 2013) is a self-report symptom checklist comprising 18 items rated on a five-point Likert-type scale. Respondents are asked to rate each item according to how they have been feeling during the past seven days. The scale provides an overall measure of psychological distress (Global Severity Index [GSI]) and three symptom scores (somatization, depression, and anxiety). In accordance with Derogatis (2001), scores are transformed into t scores to identify clinically significant distress using gender-specific normative data (t scores of 63 or greater on the GSI or on at least two subscales are classified as “caseness”). The BSI-18 has been used as a criterion scale for the DT (Bevans et al., 2011; Hoffman, Zevon, D’Arrigo, & Cecchini, 2004; Jacobsen et al., 2005; Merport et al., 2012) and has shown satisfactory psychometric properties in studies of a Spanish population of people with cancer (Galdón et al., 2008; Martínez, Conchado, Andreu, & Galdón, 2019). In those studies, the confirmatory factor analyses supported the good structural validity of the instrument, and Cronbach alpha values ranged from 0.91 to 0.81 for GSI and from 0.84 to 0.62 for subscales. In addition, composite reliability values obtained in the second study ranged from 0.9 to 0.69. Regardless of the index used, the lowest values in reliability were obtained by the somatization subscale. For the current study, the GSI and subscales showed adequate reliability (Cronbach alpha GSI = 0.92, Cronbach alpha somatization = 0.73, Cronbach alpha depression = 0.86, Cronbach alpha anxiety = 0.86).

    Data Analysis

    Statistical analysis was performed using IBM SPSS Statistics, version 22.0. Descriptive statistics were used to summarize sociodemographic, medical, and psychological data. Correlations between the DT and BSI-18 (GSI and subscales) were calculated using nonparametric Spearman’s rho (advised for data that do not meet the assumption of normality) to explore the convergent validity of the DT with the BSI-18. The following indices were used to study the accuracy of the DT and determine its optimal cutoff point for clinically significant distress (Doménech, 2004; Franco & Vivo, 2007):

    •  Global measures (AUC, fraction correct)

    •  Occurrence measures (sensitivity, specificity)

    •  Discrimination measures (PPV, NPV)

    Occurrence and discrimination measures were used to calculate clinical utility indexes (UIs). The positive UI ([UI+] = sensitivity × PPV) provides rule-in accuracy (case finding), and the negative UI ([UI−] = specificity × NPV) shows rule-out accuracy (screening) (Mitchell, 2011). The statistical significance level for analyses was p ≤ 0.05.

    Results

    Sample Characteristics

    Sociodemographic and medical data for the 236 participants are shown in Table 1. The mean age was 58 years (SD = 13.4); 65% were women. More than two-thirds of the participants were living with a partner (69%), most had completed primary education (89%), 43% were retired or on sick leave, and 11% were unemployed.

    The most frequent cancer type was gynecologic cancer (36%), followed by prostate cancer (18%), breast cancer (16%), and melanoma (13%). Almost one in four participants (24%) had completed their primary treatment no more than a year before, 43% had exceeded a year but had not yet reached five years post-treatment, and 33% had reached five or more years post-treatment.

    Brief Symptom Inventory 18 and Distress Thermometer Scores

    The mean GSI score was 10.45 (SD = 11.37, range = 0–55), and 16% of participants (37 of 236) met the criteria for caseness of distress. The mean DT score was 4.21 (SD = 3.1, range = 0–10); in accordance with the proposal of distress ranges of Hoffman et al. (2004), 66 participants (28%) reported high levels of distress (DT score of 7 or greater), 70 (30%) reported moderate levels (score of 4 to 6), and 100 (43%) reported low levels (DT score of 3 or less). Figure 1 shows the distribution of negative and positive cases of distress (as measured by the BSI-18) across the DT score range. Two of the 37 positive cases showed low DT scores, 13 had moderate scores, and 22 presented high levels of distress on the DT; 90 of the 191 negative cases had low DT scores, 61 reported moderate scores, and 40 had high DT scores.

    Accuracy and Convergent Validity of the Distress Thermometer

    The AUC was 0.81 (95% confidence interval [0.74, 0.87]), indicating that the DT showed adequate overall accuracy in detecting distress and non-distress cases relative to the BSI-18. A graphic representation of the tradeoff between sensitivity and 1−specificity is shown in Figure 2. To evaluate the optimal cutoff point, several measures of accuracy were calculated for each DT score (see Table 2).

    Taking into account these indicators, and considering screening purposes, a higher sensitivity is of greater value than a higher specificity (Olesen et al., 2018; Vodermaier & Millman, 2011). A score of 5 was considered the optimal cutoff point. The fraction correct for this score showed that 64% of patients would have been given the correct diagnosis of negative or positive. A DT cutoff of 5 allowed the detection of 33 of 37 cases of distress (sensitivity = 89%) and excluded 117 of 199 non-cases (specificity = 59%). The PPV was 29%, and the NPV was 97%. The clinical UIs showed that the rule-out accuracy of the DT was adequate (UI− = 0.57), but its rule-in accuracy was poor (UI+ = 0.26).

    DT scores were significantly associated with GSI scores (r = 0.67, p < 0.001) and with each of the BSI-18 subscales (somatization [r = 0.42, p < 0.001], depression [r = 0.56, p < 0.001], and anxiety [r = 0.62, p < 0.001]).

    Discussion

    The main objective of this study was to examine the performance of the DT as a screening tool in Spanish post-treatment survivors of adult-onset cancer. The findings were similar to those found in previous studies of Spanish people with cancer (Andreu, Galdón, Martínez, & Martínez, 2015; Martínez et al., 2013, 2015) and indicated the adequate global accuracy of the DT scores relative to the BSI-18 cutoff score gold standard (AUC = 0.81). The DT also showed satisfactory convergent validity; it was significantly associated with each of the BSI-18 subscales (anxiety, depression, and somatization) and with its global score (GSI).

    To select the optimal cutoff point of the DT for clinically significant distress (psychological problems of all types with high levels of severity and impairment that require professional support), the authors considered a combined criterion: an optimal sensitivity (to ensure that possible cases of psychological morbidity were not missed) together with clinical UIs that combine occurrence and discrimination measures. This could potentially lead to overdetection of cases and unnecessary further evaluation. In effect, it would increase the social and personal costs associated with false positives. However, it could be argued that overdetection is preferable to underdetection in the survivorship context because of intermittent contact with healthcare providers and fewer opportunities to identify psychological distress (Boyes et al., 2013).

    Based on these considerations, a DT cutoff score of 5 was selected to detect clinically significant distress, which was the same as the cutoff in previous studies of the Spanish cancer population. This DT cutoff point yielded a true negative in 97 of 100 negative tests (NPV) while being really negative (according to the BSI-18) in only 59% of cases (specificity); in addition, this threshold yielded a true positive in 29 of 100 positive tests (PPV) while being really positive (according to the BSI-18) in 89% of cases (sensitivity).

    These data are consistent with data from studies of people who have survived cancer in which the DT showed higher levels of sensitivity than specificity (Boyes et al., 2013; Ghazali et al., 2017; Hong & Tian, 2013; Olesen et al., 2018). The results suggest that the DT is more adequate for routine screening (UI− = 0.57) (to rule out non-cases with minimal false negatives) than for case finding (UI+ = 0.26) because of its substantial number of false positives.

    In terms of cancer trajectory stages, the performance of the DT at the post-treatment survivorship stage is more limited than that obtained at the active treatment stage (Ma et al., 2014). Therefore, some consensus exists that the DT cannot be used as a single screening tool for clinically significant distress (Craike et al., 2011; Hong & Tian, 2013; Merport et al., 2012; Recklitis et al., 2016). Some appropriate strategies that have been suggested to improve its accuracy for people with cancer (Martínez et al., 2013) could also be used in the stage of survivorship. For example, Craike et al. (2011) compared the overall accuracy of the DT to its combined use with the Impact Thermometer (IT) (Akizuki, Yamawaki, Akechi, Nakano, & Uchitomi, 2005). However, the combination of the DT and the IT did not improve its performance.

    Some advantages of the DT are its ease of application and its effectiveness as a tool to discard cases of clinically significant distress. The DT could be used as part of a two-stage screening process, in which it serves as a first step to identify non-cases. This would require a low DT cutoff point that permits a high sensitivity and the addition of a second screening tool with more items to reduce false positives (Boyes et al., 2013; Chambers, Zajdlewicz, Youlden, Holland, & Dunn, 2014; Recklitis et al., 2016). This is the only use of the DT in the cancer survival phase that is currently supported by the NCCN (2017b).

    Limitations

    This study has several limitations. A relatively small number of cancer survivors were recognized as clinically distressed; however, results were calculated using common standard measures, some of which did not consider the prevalence of the event (Franco & Vivo, 2007). In addition, the data of the diagnostic accuracy of the DT were improved by using other parameters like the clinical UIs. The mixed characteristics of the sample in relation to sociodemographic and medical variables may facilitate the generalization of the results to the population of people who have survived cancer. However, the small size of the subgroups precluded an exploration of the performance of the DT as a distress screening tool taking into account relevant variables such as type of cancer and time elapsed since the end of treatment; this would have provided useful additional information about the performance of the DT. Finally, the use of a structured diagnostic interview, considered the gold standard, would have been desirable to assess psychological distress. However, it should be noted that a meta-analysis showed that the self-reported prevalence of any emotional complication was similar to the prevalence obtained using interviews (Mitchell et al., 2011).

    Implications for Nursing

    Screening for and responding to distress is considered an important part of nursing practice as part of a therapeutic person-centered care approach (McCormack & McCance, 2006). The success of any screening program for distress requires nursing staff to embrace distress as a nurse-sensitive outcome and acquire the confidence to provide effective psychosocial care and evidence-based symptom management (Fitch, Howell, McLeod, & Green, 2012). As indicated by Vodermaier, Linden, and Siu (2009), healthcare providers’ infrequent use of high-quality screening instruments in cancer care settings may partially be because of their time constraints and insufficient knowledge about appropriate screening tools. For this reason, it is essential to provide nursing education about the routine use of screening instruments and the conceptual frameworks that guide the assessment process and score responses (Grassi, Nanni, & Caruso, 2010). Given its brevity and ease of administration, the DT should form part of a programmatic approach to detect distress. It could be used as a first mechanism in triage by ruling out a large subgroup of people who do not require future exploration of their distress levels.

    Conclusion

    The DT could be used to screen for distress in the cancer post-treatment survival phase as a first step to rule out non-cases. However, it is essential to include another screening instrument with a higher level of specificity to reevaluate previously identified cases. This second step in the screening process is necessary to avoid overloading scarce available health resources.

    About the Author(s)

    Yolanda Andreu Vaillo, PhD, is a professor in the Department of Personality, Assessment, and Psychological Treatments at the University of Valencia in Spain, Paula Martínez Lopez, PhD, is an associate professor in the Area of Health Sciences at the Valencian International University in Spain, and María José Galdón Garrido, PhD, is a professor in the Department of Personality, Assessment, and Psychological Treatments at the University of Valencia. This research was funded by a grant (PSI201345905-R) from the Ministry of Economy and Competitiveness, Government of Spain. All authors contributed to the conceptualization and design, provided the analysis, and contributed to the manuscript preparation. Andreu Vaillo and Martínez Lopez completed the data collection and provided statistical support. Andreu Vaillo can be reached at yolanda.andreu@uv.es, with copy to ONFEditor@ons.org. (Submitted May 2018. Accepted December 4, 2018.)

     

    References

    Aaronson, N.K., Mattioli, V., Minton, O., Weis, J., Johansen, C., Dalton, S.O., . . . van de Poll-Franse, L.V. (2014). Beyond treatment—Psychosocial and behavioral issues in cancer survivorship research and practice. European Journal of Cancer Supplements, 12, 54–64. https://doi.org/10.1016/j.ejcsup.2014.03.005

    Akizuki, N., Yamawaki, S., Akechi, T., Nakano, T., & Uchitomi, Y. (2005). Development of an Impact Thermometer for use in combination with the Distress Thermometer as a brief screening tool for adjustment disorders and/or major depression in cancer patients. Journal of Pain and Symptom Management, 29, 91–99. https://doi.org/10.1016/j.jpainsymman.2004.04.016

    Albreht, T., Kiasuwa, R., & Van den Bulcke, M. (2017). European guide on quality improvement in comprehensive cancer control. Retrieved from https://cancercontrol.eu/archived/uploads/images/Guide/pdf/CanCon_Guide…

    Andreu, Y., Galdón, M.J., Martínez, P., & Martínez, O. (2015). Precisión diagnóstica del termómetro de distrés en neoplasias hematológicas. Psicooncología, 12, 237–247. https://doi.org/10.5209/rev_PSIC.2015.v12.n2-3.51006

    Bevans, M., Wehrlen, L., Prachenko, O., Soeken, K., Zabora, J., & Wallen, G.R. (2011). Distress screening in allogeneic hematopoietic stem cell (HSCT) caregivers and patients. Psycho-Oncology, 20, 615–622. https://doi.org/10.1002/pon.1906

    Bower, J.E. (2007). Behavioral symptoms in patients with breast cancer and survivors. Journal of Clinical Oncology, 26, 768–77. https://doi.org/10.1200/JCO.2007.14.3248

    Boyes, A., D’Este, C., Carey, M., Lecathelinais, C., & Girgis, A. (2013). How does the Distress Thermometer compare to the Hospital Anxiety and Depression Scale for detecting possible cases of psychological morbidity among cancer survivors? Supportive Care in Cancer, 21, 119–127. https://doi.org/10.1007/s00520-012-1499-3

    Bultz, B.D., & Carlson, L.E. (2005). Emotional distress: The sixth vital sign in cancer care. Journal of Clinical Oncology, 23, 6440–6441. https://doi.org/10.1200/JCO.2005.02.3259

    Chambers, S.K., Zajdlewicz, L., Youlden, D.R., Holland, J.C., & Dunn, J. (2014). The validity of the distress thermometer in prostate cancer populations. Psycho-Oncology, 23, 195–203. https://doi.org/10.1002/pon.3391

    Craike, M.J., Livingston, P.M., & Warne, C. (2011). Sensitivity and specificity of the Distress Impact Thermometer for the detection of psychological distress among CRC survivors. Journal of Psychosocial Oncology, 29, 231–241. https://doi.org/10.1080/07347332.2011.563347

    de Boer, A.G., Taskila, T., Ojajärvi, A., van Dijk, F.J., & Verbeek, J.H. (2009). Cancer survivors and unemployment: A meta-analysis and meta-regression. JAMA, 301, 753–762. https://doi.org/10.1001/jama.2009.187

    Derogatis, L.R. (2001). Brief Symptom Inventory 18. Retrieved from https://www.pearsonclinical.com/products/100000638/brief-symptom-invent…

    Derogatis, L.R. (2013). BSI-18, Inventario Breve de 18 Síntomas. Retrieved from https://www.pearsonclinical.es/producto/110/bsi-18-inventario-breve-de-…

    Doménech, J.M. (2004). Fundamentos de diseño y estadística. UD 3: Fundamentos de la teoría de la probabilidad. Pruebas diagnósticas. Barcelona: Signo. Retrieved from http://datos.bne.es/edicion/bimo0002405601.html

    Donovan, K.A., Grassi, L., McGinty, H.L., & Jacobsen, P.B. (2014). Validation of the Distress Thermometer worldwide: State of the science. Psycho-Oncology, 23, 241–250. https://doi.org/10.1002/pon.3430

    Duijts, S.F.A., van Egmond, M.P., Spelten, E., van Muijen, P., Anema, J.R., & van der Beek, A.J. (2014). Physical and psychosocial problems in cancer survivors beyond return to work: A systematic review. Psycho-Oncology, 23, 481–492. https://doi.org/10.1002/pon.3467

    Fitch, M.I., Howell, D., McLeod, D., & Green, E. (2012). Screening for distress: Responding is a critical function for oncology nurses. Canadian Oncology Nursing Journal, 22, 12–30. https://doi.org/10.5737/1181912x2211220

    Franco, M., & Vivo, J.M. (2007). Análisis de Curvas ROC. Principios básicos y aplicaciones. Madrid: La Muralla.

    Galdón, M.J., Durá, E., Andreu, Y., Ferrando, M., Murgui, S., Pérez, S., & Ibañez, E. (2008). Psychometric properties of the Brief Symptom Inventory 18 (BSI-18) in a Spanish breast cancer sample. Journal of Psychosomatic Research, 65, 533–539. https://doi.org/10.1016/j.jpsychores.2008.05.009

    Ghazali, N., Roe, B., Lowe, D., Tandon, S., Jones, T., Brown, J., . . . Rogers, S.N. (2017). Screening for distress using the Distress Thermometer and the University of Washington Quality of Life in post-treatment head and neck cancer survivors. European Archives of Oto-Rhino-Laryngoly, 274, 2253–2276. https://doi.org/10.1007/s00405-017-4474-2

    Gil, F., Grassi, L., Travado, L., Tomamichel, M., & González, J.R. (2005). Use of distress and depression thermometers to measure psychosocial morbidity among southern European cancer patients. Supportive Care in Cancer, 13, 600–606. https://doi.org/10.1007/s00520-005-0780-0

    Grassi, L., Nanni, M.G., & Caruso, R. (2010). Emotional distress in cancer: Screening policy, clinical limitations and educational needs. Journal of Medicine and the Person, 8, 51–59. https://doi.org/10.1007/s12682-010-0047-5

    Hoffman, B.M, Zevon, M.A., D’Arrigo, M.C., & Cecchini, T.B. (2004). Screening for distress in cancer patients: The NCCN rapid-screening measure. Psycho-Oncology, 13, 792–799. https://doi.org/10.1002/pon.796

    Hoffman, K.E., McCarthy, E.P., Recklitis, C.J., & Ng, A.K. (2009). Psychological distress in long-term survivors of adult-onset cancer: Results from a national survey. Archives of Internal Medicine, 169, 1274–1281. https://doi.org/10.1001/archinternmed.2009.179

    Hong, J.S., & Tian, J. (2013). Sensitivity and specificity of the Distress Thermometer in screening for distress in long-term nasopharyngeal cancer survivors. Current Oncology, 20, e570–e576. http://doi.org/10.3747/co.20.1617

    Jacobsen, P.B., Donovan, K.A., Trask, P.C., Fleishman, S.B., Zabora, J., Baker, F., & Holland, J.C. (2005). Screening for psychological distress in ambulatory cancer patients: A multicenter evaluation of the Distress Thermometer. Cancer, 103, 1494–1502. https://doi.org/10.1002/cncr.20940

    Jefford, M., Ward, A.C., Lisy, K., Lacey, K., Emery, J.D, Glaser, A.W., . . . Bishop, J. (2017). Patient-reported outcomes in cancer survivors: A population-wide cross-sectional study. Supportive Care in Cancer, 25, 3171–3179. https://doi.org/10.1007/s00520-017-3725-5

    Kaiser, N.C., Hartoonian, N., & Owen, J.E. (2010). Toward a cancer-specific model of psychological distress: Population data from the 2003-2005 National Health Interview Surveys. Journal of Cancer Survivorship, 4, 291–302. https://doi.org/10.1007/s11764-010-0120-3

    Ma, X., Zhang, J., Zhong, W., Shu, C., Wang, F., Wen, J., . . . Liu, L. (2014). The diagnostic role of a short screening tool—The Distress Thermometer: A meta-analysis. Supportive Care in Cancer, 22, 1741–1755. https://doi.org/10.1007/s00520-014-2143-1

    Martínez, P., Andreu, Y., Galdón, M.J., & Ibáñez, E. (2015). Improving the diagnostic accuracy of the Distress Thermometer: A potential role for the Impact Thermometer. Journal of Pain and Symptom Management, 50, 124–129. https://doi.org/10.1016/j.jpainsymman.2015.02.004

    Martínez, P., Conchado, A., Andreu, Y., & Galdón, M.J. (2019). Psychometric properties of the Brief Symptom Inventory-18 in a heterogeneous sample of adult cancer patients. Revista Latinoamericana de PsicologÍa, 51. https://doi.org/10.14349/rlp.2019.v51.n1.1

    Martínez, P., Galdón, M.J., Andreu, Y., & Ibáñez, E. (2013). The Distress Thermometer in Spanish cancer patients: Convergent validity and diagnostic accuracy. Supportive Care in Cancer, 21, 3095–3102. https://doi.org/10.1007/s00520-013-1883-7

    McCormack, B., & McCance, T.V. (2006). Development of a frame-work for person-centered nursing. Journal of Advanced Nursing, 56, 472–479. https://doi.org/10.1111/j.1365-2648.2006.04042.x

    Merport, A., Bober, S.L., Grose, A., & Recklitis, C.J. (2012). Can the Distress Thermometer (DT) identify significant psychological distress in long-term cancer survivors? A comparison with the Brief Symptom Inventory-18 (BSI-18). Supportive Care in Cancer, 20, 195-198. https://doi.org/10.1007/s00520-011-1269-7

    Mitchell, A.J. (2011). Sensitivity x PPV is a recognized test called the clinical utility index (CUI+). European Journal of Epidemiology, 26, 251–252. https://doi.org/10.1007/s10654-011-9561-x

    Mitchell, A.J., Chan, M., Bhatti, H., Halton, M., Grassi, L, Johansen, C., & Meader, N. (2011). Prevalence of depression, anxiety, and adjustment disorder in oncological, haematological, and palliative-care settings: A meta-analysis of 94 interview-based studies. Lancet Oncology, 12, 160–174. https://doi.org/10.1016/S1470-2045(11)70002-X

    Mitchell, A.J., Ferguson, D.W., Gill, J., Paul, J., & Symonds, P. (2013). Depression and anxiety in long-term cancer survivors compared with spouses and healthy controls: A systematic review and meta-analysis. Lancet Oncology, 14, 721–732. https://doi.org/10.1016/S1470-2045(13)70244-4

    National Comprehensive Cancer Network. (2017a). NCCN Clinical Practice Guidelines in Oncology (NCCN Guidelines®): Distress management [v.2.2017]. Retrieved from https://www.nccn.org/professionals/physician_gls/pdf/distress.pdf

    National Comprehensive Cancer Network. (2017b). NCCN Clinical Practice Guidelines in Oncology (NCCN Guidelines®): Survivorship [v.2.2017]. Retrieved from https://www.nccn.org/professionals/physician_gls/pdf/survivorship.pdf

    Olesen, M.L., Hansen, M.K., Hansson, H., Ottesen, B., Andersen, K.K., & Zoffmann, V. (2018). The Distress Thermometer in survivors of gynaecological cancer: Accuracy in screening and association with the need for person-centred support. Supportive Care in Cancer, 26, 1143–1150. https://doi.org/10.1007/s00520-017-3935-x

    Pachman, D.R, Barton, D.L., Swetz, K.M., & Loprinzi, C.L. (2012). Troublesome symptoms in cancer survivors: Fatigue, insomnia, neuropathy, and pain. Journal of Clinical Oncology, 30, 3687–3696. https://doi.org/10.1200/JCO.2012.41.7238

    Ploos van Amstel, F.K., van den Berg, S.W., van Laarhoven, H.W., Gielissen, M.F.M., Prins, J.B., & Ottevanger, P.B. (2013). Distress screening remains important during follow-up after primary breast cancer treatment. Supportive Care in Cancer, 21, 2107–2115. https://doi.org/10.1007/s00520-013-1764-0

    Recklitis, C.J., Blackmon, J.E., & Chang, G. (2016). Screening young adult cancer survivors for distress with the Distress Thermometer: Comparisons with a structured clinical diagnostic interview. Cancer, 122, 296–303. https://doi.org/10.1002/cncr.29736

    Roerink, S.H., de Ridder, M., Prins, J., Huijbers, A., de Wilt, H.J., Marres, H., . . . Netea-Maier, R.T. (2013). High level of distress in long-term survivors of thyroid carcinoma: Results of rapid screening using the Distress Thermometer. Acta Oncologica, 52, 128–137. https://doi.org/10.3109/0284186X.2012.723822

    Roth, A.J., Kornblith, A.B., Batel-Copel, L., Peabody, E., Scher, H.I., & Holland, J.C. (1998). Rapid screening for psychologic distress in men with prostate carcinoma: A pilot study. Cancer, 82, 1904–1908.

    Snowden, A., White, C.A., Christie, Z., Murray, E., McGowan, C., & Scott, R. (2011) The clinical utility of the Distress Thermometer: A review. British Journal of Nursing, 20, 220–227. https://doi.org/10.12968/bjon.2011.20.4.220

    Stanton, A.L. (2012). What happens now? Psychosocial care for cancer survivors after medical treatment completion. Journal of Clinical Oncology, 30, 1215–1220. https://doi.org/10.1200/JCO.2011.39.7406

    Vodermaier, A., Linden, W., & Siu, C. (2009) Screening for emotional distress in cancer patients: A systematic review of assessment instruments. Journal of the National Cancer Institute, 101, 1464–1488. https://doi.org/10.1093/jnci/djp336

    Vodermaier, A., & Millman, R.D. (2011). Accuracy of the Hospital Anxiety and Depression Scale as a screening tool in cancer patients: A systematic review and meta-analysis. Supportive Care in Cancer, 19, 1899–1908. https://doi.org/10.1007/s00520-011-1251-4

    Wells, M., Cunningham, M., Lang, H., Swartzman, S., Philip, J., Taylor, L., & Thomson, J. (2015). Distress, concerns and unmet needs in survivors of head and neck cancer: A cross-sectional survey. European Journal of Cancer Care, 24, 748–60. https://doi.org/10.1111/ecc.12370