Original ArticleA 67-item stress resilience item bank showing high content validity was developed in a psychosomatic sample
Introduction
There is ample research suggesting that stress plays a major role in the development and maintenance of several medical conditions [1], [2], [3], [4]. However, some people thrive despite of adverse conditions and stay healthy in the face of stressors [5], [6], [7]. Still, to date, there is only little knowledge about why some people stay healthy in the face of adversity and others turn ill. Over the past years, there has been a shift from deficiency-based research to a more resource-oriented approach [8]. Although this turn gave rise to research on protective characteristics, it is not yet well understood which factors are crucial for the maintenance of physical and mental health. Resilience is a relatively new construct that may fill this gap. It has been shown that more resilient individuals are less likely to develop mental disorders, such as depression or anxiety [9], [10], [11]. Resilience can be seen as a “dynamic process encompassing positive adaptation within the context of significant adversity” [12]. Windle [13] suggested three necessary requirements for resilience: (a) the presence of a significant adversity, (b) the presence of resources to compensate the effects of the adversity, (c) positive adaptation or the avoidance of a negative outcome. Fletcher and Sarkar [14] limited these requirements to two core concepts that form the mutual basis of different definitions of resilience: adversity and positive adaptation.
Initial research into resilience first focused on protective factors, although it later turned to the underlying protective mechanisms and processes that help the individual to overcome adversity [12]. Fletcher and Sarkar [14] showed that resilience has been conceptualized in various different ways, i.e., as a trait, a process, or as an outcome. Hence, there is a lack of consistent conceptualization [15]. Therefore, resilience is difficult to operationalize. Various definitions led to the development of different questionnaires all claiming to measure resilience, and a number of reviews evaluated existing resilience questionnaires. As a result, there are inconsistent recommendations about which instrument should be used [16], [17]. Recently, Windle, Bennett, and Noyes [17] suggested that the quality of these questionnaires may be regarded as only mediocre considering a range of quality criteria and, they also questioned the conceptual and theoretical adequacy of a number of the scales, concluding that there is no “gold standard” to measure resilience.
A major concern in the development of resilience scales is the frequent negligence of the target population in the developmental process, which generally leads to insufficient content validity of according measures [17]. This means that oftentimes, the item generation is purely theory driven, the qualitative process is not covered in much depth, and the target population is not involved in focus groups and pilot tests for item generation but only in final data selection [17]. Moreover, it remains unclear whether or not resilience can be described by one global factor or by several subfactors. In addition, it is questionable whether or not these subfactors consistently exist in different populations. For example, the original five-factor solution for the Connor-Davidson resilience scale (CD-RISC) reported by Connor and Davidson [18] could not be replicated consistently [19].
The development of the different resilience questionnaires that exist today is mostly based on classical test theory [18], [20], [21], [22], [23], [24]. Even if different instruments were claimed to measure the same latent construct, comparison between scores from these instruments would be difficult, as each instrument has its own metric. To overcome these problems of comparability, item response theory (IRT) methods can be used to calibrate any number of items measuring the same latent construct on a common scale [25], [26], [27], resulting in a construct-specific but instrument-independent “item bank” [28], [29]. This approach has several advantages over classical test theory, including improved comparability between different subsets of items due to the established common metric, the possibility to develop computer-adaptive tests (CATs) and tailored short forms, and–when making use of CATs–a substantial reduction of response burden for patients [30], [31]. A further advantage is that IRT-based item banks are dynamic tools, meaning that they can be continuously improved [28]. Therefore, the IRT approach could notably advance the operationalization of the concept of resilience, allowing not only for comparisons between studies but also for iterative refinement of the item bank by adding supplemental items. In the future, items of existing resilience measures such as the CD-RISC [18] and the RS-11 [20], [32] could be linked to the established common stress resilience (SR) metric. The establishment of such a common metric covering items of several instruments has already been done for other constructs such as depression [29], anxiety [33], or physical function [34]. Recently, our group developed a web application allowing for direct comparison of scores from different depression, anxiety, or physical function instruments [35].
To date, two studies used modern test theory methods to develop or evaluate items aimed at assessing resilience. One study used IRT methods (two-parameter model) to develop resilience items to explore resilience in spinal cord injury patients [36]. Another study used a one-parameter model (RSM) to evaluate the psychometric properties at the item level of the CD-RISC [37].
In contrast to Victorson et al. [36], in the present study, it was not intended to assess resilience in a specific patient population but to develop items that work in different populations. As a result, patients with a wide range of health conditions that presented at the psychosomatic department of the medical clinic at the Charité were involved in the item generation process. This study aims to overcome aforementioned problems in the development of resilience questionnaires particularly the lack of comparability and the negligence of target population involvement in the instrument development process. Our objectives were to develop a new SR scale thereby involving a clinical population in the item generation process and making use of modern test theory methods for item bank calibration. In detail, this article is aimed at advancing the understanding of resilience, presenting the findings of the development of a content valid item bank, and providing a SR short form.
Section snippets
Methods
We adopted an iterative process for item bank development following the guidelines of the Food and Drug Administration [38] and well-established standard procedures previously described in the literature [28], [29], [30], [39]. Four iterative steps (see Fig. 1) were performed that are described in detail below: (a) concept specification, (b) item pool generation and refinement, (c) data collection, and (d) psychometric item bank development and development of a short form. Conceptual work
Concept specification
During the iterative focus group sessions with health-care professionals, results from focus group sessions with the target population were integrated. The conceptual framework of SR was formulated. We defined resilience as a broad construct with the aim to capture different aspects of a positive aptitude in stress handling. We wanted to assess SR at a personal level and assumed that SR may change over the life course and can be enhanced by interventions. Based on our literature research and
Discussion
We developed a new content valid SR item bank and a 10-item SR short form based on modern test theory methods and involvement of the target population. The item bank development followed iterative qualitative steps and comprehensive statistical analyses to assure the identification of the most essential items assessing SR. The final item bank including 67 items and the 10-item short form showed good psychometric properties as well as high construct validity. After further validation in
Conclusions
The newly developed SR item bank allows for valid measurement of the underlying SR construct in a psychosomatic sample. To ensure content validity and utility of the newly developed items, study methods included iterative focus group discussions, target population feedback, and field testing. Following EFA, CFA, and IRT-analyses, 67 items were retained for the final item bank and a 10-item short form was generated. Initial analyses found good psychometric properties and confirmed the validity
References (68)
- et al.
Stress, behavior and the heart
Neurosci Biobehav Rev
(2017) - et al.
Stress, depression and Parkinson's disease
Exp Neurol
(2012) - et al.
Resilience and mental health
Clin Psychol Rev
(2010) - et al.
The patient-reported outcomes measurement information system (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005–2008
J Clin Epidemiol
(2010) - et al.
Evaluation of a preliminary physical function item bank supported the expected advantages of the patient-reported outcomes measurement information system (PROMIS)
J Clin Epidemiol
(2008) - et al.
Standardization of depression measurement: a common metric was developed for 11 self-report depression measures
J Clin Epidemiol
(2014) - et al.
The PROMIS physical function item bank was calibrated to a standardized metric and shown to improve measurement efficiency
J Clin Epidemiol
(2014) - et al.
Establishing a common metric for self-reported anxiety: linking the MASQ, PANAS, and GAD-7 to PROMIS Anxiety
J Anxiety Disord
(2014) - et al.
Subjective well-being measures for children were developed within the PROMIS project: presentation of first results
J Clin Epidemiol
(2014) - et al.
The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes
J Clin Epidemiol
(2010)
Using Patient Health Questionnaire-9 item parameters of a common metric resulted in similar depression scores compared to independent item response theory model reestimation
J Clin Epidemiol
Do positive or negative stressful events predict the development of new brain lesions in people with multiple sclerosis?
Psychol Med
Stress-induced immune dysfunction: implications for health
Nat Rev Immunol
Resilience and vulnerability: Adaptation in the context of childhood adversities
Ordinary magic: resilience processes in development
Am Psychol
Loss, trauma, and human resilience: have we underestimated the human capacity to thrive after extremely aversive events?
Am Psychol
The metatheory of resilience and resiliency
J Clin Psychol
The buffering effect of resilience upon stress, anxiety and depression in parents of a child with an autism spectrum disorder
J Develop Phys Disabilities
Does resilience ‘buffer’against depression in prostate cancer patients? A multi-site replication study
Eur J Cancer Care
What good are positive emotions in crisis? A prospective study of resilience and emotions following the terrorist attacks on the United States on September 11th, 2001
J Pers Soc Psychol
The construct of resilience: a critical evaluation and guidelines for future work
Child Dev
What is resilience? A review and concept analysis
Rev Clin Gerontol
Psychological resilience
Eur Psychol
A review of instruments measuring resilience
Issues Compr Pediatr Nurs
A methodological review of resilience measurement scales
Health Qual Life Outcomes
Development of a new resilience scale: the Connor-Davidson resilience scale (CD-RISC)
Depress Anxiety
Psychometric analysis and refinement of the Connor–Davidson resilience scale (CD-RISC): validation of a 10-item measure of resilience
J Traumac Stress
Development and psychometric evaluation of the resilience scale
J Nurs Meas
Test-retest reliability of the dispositional resilience scale-15, a brief hardiness scale
Psychol Rep
The brief resilience scale: assessing the ability to bounce back
Int J Behav Med
IQ and ego-resiliency: conceptual and empirical connections and separateness
J Pers Soc Psychol
A new rating scale for adult resilience: what are the central protective resources behind healthy adjustment?
Int J Methods Psychiatr Res
Item response theory for psychologists
Item response theory and clinical measurement
Annu Rev Clin Psychol
Cited by (6)
Factor structure and psychometric properties of the Connor–Davidson resilience scale (CD-RISC) in individuals with opioid use disorder
2021, Drug and Alcohol DependenceCitation Excerpt :The data demonstrate adequate reliability and validity of the measure within samples of individuals who were either not currently seeking treatment, recently detoxified from opioids, or receiving opioid agonist maintenance pharmacotherapies. Overall, average resilience scores in our sample across all groups (non-treatment seekers, those currently undergoing agonist maintenance treatment, and recently detoxified individuals), were higher compared to other psychiatric populations (Davidson et al., 2005; Obbarius et al., 2018; Torgalsbøen, 2012). Consistent with prior studies, we found that higher resilience was associated with lower depression symptoms, further demonstrating the ability of the CD-RISC 25 to predict psychiatric stability (Min et al., 2013; Smith, 2009).
Burnout, stress and resilience of an Australian regional hospital during COVID-19: a longitudinal study
2022, BMC Health Services ResearchHow can stress resilience be monitored? A systematic review of measurement in humans
2021, Current PsychologyAssociation Between Physical Fitness and Anxiety in Children: A Moderated Mediation Model of Agility and Resilience
2020, Frontiers in Public Health
Conflict of interest: None.
Funding: This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.