Original Article
A 67-item stress resilience item bank showing high content validity was developed in a psychosomatic sample

https://doi.org/10.1016/j.jclinepi.2018.04.004Get rights and content

Abstract

Objectives

To develop the first item bank to measure stress resilience (SR) in clinical populations.

Study Design and Setting

Qualitative item development resulted in an initial pool of 131 items covering a broad theoretical SR concept. These items were tested in n = 521 patients at a psychosomatic outpatient clinic. Exploratory and confirmatory factor analysis, as well as other state-of-the-art item analyses and item response theory were used for item evaluation and calibration of the final item bank.

Results

Of the initial item pool of 131 items, we excluded 64 items (54 factor loading <0.5, four residual correlations >0.3, two nondiscriminative item response curves, and four differential item functioning). The final set of 67 items indicated sufficient model fit in confirmatory factor analysis and item response theory analyses. In addition, a 10-item short form with high measurement precision (SE ≤ 0.32 in a theta range between −1.8 and +1.5) was derived. Both the SR item bank and the SR short form were highly correlated with an existing static legacy tool (Connor-Davidson Resilience Scale).

Conclusion

The final SR item bank and 10-item short form showed good psychometric properties. When further validated, they will be ready to be used within a framework of computer-adaptive tests for a comprehensive assessment of the stress construct.

Introduction

There is ample research suggesting that stress plays a major role in the development and maintenance of several medical conditions [1], [2], [3], [4]. However, some people thrive despite of adverse conditions and stay healthy in the face of stressors [5], [6], [7]. Still, to date, there is only little knowledge about why some people stay healthy in the face of adversity and others turn ill. Over the past years, there has been a shift from deficiency-based research to a more resource-oriented approach [8]. Although this turn gave rise to research on protective characteristics, it is not yet well understood which factors are crucial for the maintenance of physical and mental health. Resilience is a relatively new construct that may fill this gap. It has been shown that more resilient individuals are less likely to develop mental disorders, such as depression or anxiety [9], [10], [11]. Resilience can be seen as a “dynamic process encompassing positive adaptation within the context of significant adversity” [12]. Windle [13] suggested three necessary requirements for resilience: (a) the presence of a significant adversity, (b) the presence of resources to compensate the effects of the adversity, (c) positive adaptation or the avoidance of a negative outcome. Fletcher and Sarkar [14] limited these requirements to two core concepts that form the mutual basis of different definitions of resilience: adversity and positive adaptation.

Initial research into resilience first focused on protective factors, although it later turned to the underlying protective mechanisms and processes that help the individual to overcome adversity [12]. Fletcher and Sarkar [14] showed that resilience has been conceptualized in various different ways, i.e., as a trait, a process, or as an outcome. Hence, there is a lack of consistent conceptualization [15]. Therefore, resilience is difficult to operationalize. Various definitions led to the development of different questionnaires all claiming to measure resilience, and a number of reviews evaluated existing resilience questionnaires. As a result, there are inconsistent recommendations about which instrument should be used [16], [17]. Recently, Windle, Bennett, and Noyes [17] suggested that the quality of these questionnaires may be regarded as only mediocre considering a range of quality criteria and, they also questioned the conceptual and theoretical adequacy of a number of the scales, concluding that there is no “gold standard” to measure resilience.

A major concern in the development of resilience scales is the frequent negligence of the target population in the developmental process, which generally leads to insufficient content validity of according measures [17]. This means that oftentimes, the item generation is purely theory driven, the qualitative process is not covered in much depth, and the target population is not involved in focus groups and pilot tests for item generation but only in final data selection [17]. Moreover, it remains unclear whether or not resilience can be described by one global factor or by several subfactors. In addition, it is questionable whether or not these subfactors consistently exist in different populations. For example, the original five-factor solution for the Connor-Davidson resilience scale (CD-RISC) reported by Connor and Davidson [18] could not be replicated consistently [19].

The development of the different resilience questionnaires that exist today is mostly based on classical test theory [18], [20], [21], [22], [23], [24]. Even if different instruments were claimed to measure the same latent construct, comparison between scores from these instruments would be difficult, as each instrument has its own metric. To overcome these problems of comparability, item response theory (IRT) methods can be used to calibrate any number of items measuring the same latent construct on a common scale [25], [26], [27], resulting in a construct-specific but instrument-independent “item bank” [28], [29]. This approach has several advantages over classical test theory, including improved comparability between different subsets of items due to the established common metric, the possibility to develop computer-adaptive tests (CATs) and tailored short forms, and–when making use of CATs–a substantial reduction of response burden for patients [30], [31]. A further advantage is that IRT-based item banks are dynamic tools, meaning that they can be continuously improved [28]. Therefore, the IRT approach could notably advance the operationalization of the concept of resilience, allowing not only for comparisons between studies but also for iterative refinement of the item bank by adding supplemental items. In the future, items of existing resilience measures such as the CD-RISC [18] and the RS-11 [20], [32] could be linked to the established common stress resilience (SR) metric. The establishment of such a common metric covering items of several instruments has already been done for other constructs such as depression [29], anxiety [33], or physical function [34]. Recently, our group developed a web application allowing for direct comparison of scores from different depression, anxiety, or physical function instruments [35].

To date, two studies used modern test theory methods to develop or evaluate items aimed at assessing resilience. One study used IRT methods (two-parameter model) to develop resilience items to explore resilience in spinal cord injury patients [36]. Another study used a one-parameter model (RSM) to evaluate the psychometric properties at the item level of the CD-RISC [37].

In contrast to Victorson et al. [36], in the present study, it was not intended to assess resilience in a specific patient population but to develop items that work in different populations. As a result, patients with a wide range of health conditions that presented at the psychosomatic department of the medical clinic at the Charité were involved in the item generation process. This study aims to overcome aforementioned problems in the development of resilience questionnaires particularly the lack of comparability and the negligence of target population involvement in the instrument development process. Our objectives were to develop a new SR scale thereby involving a clinical population in the item generation process and making use of modern test theory methods for item bank calibration. In detail, this article is aimed at advancing the understanding of resilience, presenting the findings of the development of a content valid item bank, and providing a SR short form.

Section snippets

Methods

We adopted an iterative process for item bank development following the guidelines of the Food and Drug Administration [38] and well-established standard procedures previously described in the literature [28], [29], [30], [39]. Four iterative steps (see Fig. 1) were performed that are described in detail below: (a) concept specification, (b) item pool generation and refinement, (c) data collection, and (d) psychometric item bank development and development of a short form. Conceptual work

Concept specification

During the iterative focus group sessions with health-care professionals, results from focus group sessions with the target population were integrated. The conceptual framework of SR was formulated. We defined resilience as a broad construct with the aim to capture different aspects of a positive aptitude in stress handling. We wanted to assess SR at a personal level and assumed that SR may change over the life course and can be enhanced by interventions. Based on our literature research and

Discussion

We developed a new content valid SR item bank and a 10-item SR short form based on modern test theory methods and involvement of the target population. The item bank development followed iterative qualitative steps and comprehensive statistical analyses to assure the identification of the most essential items assessing SR. The final item bank including 67 items and the 10-item short form showed good psychometric properties as well as high construct validity. After further validation in

Conclusions

The newly developed SR item bank allows for valid measurement of the underlying SR construct in a psychosomatic sample. To ensure content validity and utility of the newly developed items, study methods included iterative focus group discussions, target population feedback, and field testing. Following EFA, CFA, and IRT-analyses, 67 items were retained for the final item bank and a 10-item short form was generated. Initial analyses found good psychometric properties and confirmed the validity

References (68)

  • G. Liegl et al.

    Using Patient Health Questionnaire-9 item parameters of a common metric resulted in similar depression scores compared to independent item response theory model reestimation

    J Clin Epidemiol

    (2016)
  • M.N. Burns et al.

    Do positive or negative stressful events predict the development of new brain lesions in people with multiple sclerosis?

    Psychol Med

    (2014)
  • R. Glaser et al.

    Stress-induced immune dysfunction: implications for health

    Nat Rev Immunol

    (2005)
  • S.S. Luthar

    Resilience and vulnerability: Adaptation in the context of childhood adversities

    (2003)
  • A.S. Masten

    Ordinary magic: resilience processes in development

    Am Psychol

    (2001)
  • G.A. Bonanno

    Loss, trauma, and human resilience: have we underestimated the human capacity to thrive after extremely aversive events?

    Am Psychol

    (2004)
  • G.E. Richardson

    The metatheory of resilience and resiliency

    J Clin Psychol

    (2002)
  • V. Bitsika et al.

    The buffering effect of resilience upon stress, anxiety and depression in parents of a child with an autism spectrum disorder

    J Develop Phys Disabilities

    (2013)
  • C.F. Sharpley et al.

    Does resilience ‘buffer’against depression in prostate cancer patients? A multi-site replication study

    Eur J Cancer Care

    (2014)
  • B.L. Fredrickson et al.

    What good are positive emotions in crisis? A prospective study of resilience and emotions following the terrorist attacks on the United States on September 11th, 2001

    J Pers Soc Psychol

    (2003)
  • S.S. Luthar et al.

    The construct of resilience: a critical evaluation and guidelines for future work

    Child Dev

    (2000)
  • G. Windle

    What is resilience? A review and concept analysis

    Rev Clin Gerontol

    (2011)
  • D. Fletcher et al.

    Psychological resilience

    Eur Psychol

    (2013)
  • N.R. Ahern et al.

    A review of instruments measuring resilience

    Issues Compr Pediatr Nurs

    (2006)
  • G. Windle et al.

    A methodological review of resilience measurement scales

    Health Qual Life Outcomes

    (2011)
  • K.M. Connor et al.

    Development of a new resilience scale: the Connor-Davidson resilience scale (CD-RISC)

    Depress Anxiety

    (2003)
  • L. Campbell-Sills et al.

    Psychometric analysis and refinement of the Connor–Davidson resilience scale (CD-RISC): validation of a 10-item measure of resilience

    J Traumac Stress

    (2007)
  • G. Wagnild et al.

    Development and psychometric evaluation of the resilience scale

    J Nurs Meas

    (1993)
  • P.T. Bartone

    Test-retest reliability of the dispositional resilience scale-15, a brief hardiness scale

    Psychol Rep

    (2007)
  • B.W. Smith et al.

    The brief resilience scale: assessing the ability to bounce back

    Int J Behav Med

    (2008)
  • J. Block et al.

    IQ and ego-resiliency: conceptual and empirical connections and separateness

    J Pers Soc Psychol

    (1996)
  • O. Friborg et al.

    A new rating scale for adult resilience: what are the central protective resources behind healthy adjustment?

    Int J Methods Psychiatr Res

    (2003)
  • S.E. Embretson et al.

    Item response theory for psychologists

    (2000)
  • S.P. Reise et al.

    Item response theory and clinical measurement

    Annu Rev Clin Psychol

    (2009)
  • Cited by (6)

    Conflict of interest: None.

    Funding: This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

    View full text