Introduction

The evaluation of patient-relevant treatment benefit from drugs and medicinal products is required for approval or reimbursement in several countries, e.g. Australia, Austria, Canada, Finland, France, Great Britain, The Netherlands, New Zealand, Norway, Sweden, and Switzerland [10]. In Germany, regulatory agencies require treatment benefit to be assessed from the patient’s point of view [8, 9].

In general, patients benefit most from treatments satisfying their individual needs. Treatments that are beneficial but do not meet what is the most important to the particular patient will be second choice. Therefore, matching of treatment outcomes and individual treatment preferences is particularly important in the evaluation of treatment value. In dermatology, patients certainly want to get rid of skin alterations and accompanying symptoms. Moreover, their treatment needs can include experiencing fewer side effects of therapy, being able to show themselves in public, or feeling less depressed due to their skin condition.

The recently developed questionnaire ‘Patient Benefit Index’ (PBI) [4] provides a method to assess patient-relevant treatment benefit with two separate questionnaires: The patient needs questionnaire (PNQ) is given before initiation of treatment and offers a list of 23 potentially relevant treatment needs. Patients rate the individual importance of these needs on a scale ranging from 0 = not at all important to 4 = very important. After or during therapy, they rate the extent to which these 23 needs have been accomplished by treatment in the patient benefit questionnaire (PBQ) using a scale ranging from 0 = did not help at all to 4 = helped a lot. As an alternative, patients may tick ‘does/did not apply to me’ in both questionnaires. A total score, the actual PBI, is calculated by weighting all benefit items by the respective importance ratings and summing up the resulting products. The total score ranges from 0 (no benefit) to 4 (maximum benefit).

The items of the PBI had been derived from an open item survey involving n = 50 dermatological patients asking for treatment needs and benefits relevant to them. An expert panel including dermatologists, statisticians, psychologists, and patients reduced the results to a nonredundant list of 23 items.

To date, additional disease-specific PBI versions have been developed and validated for a range of indications including vitiligo [1], cosmetic indications [3], hand eczema [5], and pruritus [6]. The general version of the PBI has been validated for use in patients with ten different skin diseases [3] and for use in patients with acne vulgaris [2].

To enable a more differentiated evaluation of treatment needs and benefits, the PBI items of two samples were reanalysed to determine the dimensional structure of patient needs and to compute patient benefit in these different areas by the PBI formula described above.

Materials and methods

Data

Data from two studies were reanalysed (1) to determine the dimensions of patient needs and (2) to replicate the dimensions identified.

Identification of need dimensions

Data from a cross-sectional study [4] were used for the identification of need dimensions. The samples included 500 patients with ten different diagnoses (n = 50 patients each): acne vulgaris; atopic eczema; autoimmune diseases of the skin; hand and foot eczema; hair diseases; herpes zoster; hyperhidrosis; psoriasis; ulcus cruris; urticaria. All patients had consulted the department of dermatology at the University Clinics of Hamburg. The patients were told to refer to their current or last therapy when filling in the PBI. In this first PBI study, the PNQ did not offer the option ‘does not apply to me’ and patients had to choose ‘not important at all’ if an item did not apply to them. Since both answers are coded as 0 (see below), the difference is not relevant regarding both determination of subscales and computation of the PBI.

Replication of the identified dimensions

For replication of the results, data from an open, uncontrolled, multi-centre longitudinal study of 925 patients with acne vulgaris treated with topical clindamycin + benzoyl peroxide were used. Data were collected in 2005 and 2006 at 205 dermatological practises in Germany including patients with mild to moderate acne vulgaris. Patients and physicians were given questionnaires at three times: at start of medication (T1), after 4–6 weeks (T2), and after 10–12 weeks (T3). In the analysis reported here, only patient-reported data at T1 were used.

Ethics and informed consent

Since only pseudonomyzed anonymous data on file were used in this analysis, it was not submitted to the local ethics committee. In both the cross-sectional study and the acne therapy study, the patients’ informed consent had been obtained.

Factor analysis

Dimensions were identified by exploratory factor analysis on the 23 need items of the PNQ using principal components analysis with varimax rotation.

Since factor analysis requires complete cases (no missing data), the response ‘does not apply to me’ in the PNQ was coded as 0 instead of treating it as missing. Thereby, ‘does not apply to me’ was equated to ‘not important at all’, which is also coded as 0. This numerical equalisation is unproblematic and corresponds to the empirical relative, because in either instance no treatment need exists.

Benefit items of the PBQ, however, would not be suitable for this purpose because in the PBQ, the answer ‘does not apply to me’ cannot be equated with ‘did not help at all’ by coding both as 0. Instead, it has to be coded as missing value; thus, most patients would have at least one missing value and would have to be excluded from the factor analysis.

Factor analysis was conducted on both data sets independently. Bartlett’s test of sphericity was calculated testing whether the variables included in the factor analysis were significantly correlated. Only factors with an Eigen value of one or above were extracted (‘Kaiser’s criterion’). Each item was assigned to the factor on which it loaded highest. Internal consistency of the items loading high on a factor was determined with Cronbach’s alpha. Factor titles were derived by inspection of the respective items loading on a factor. The identified solutions for both data sets were compared by inspection.

All analyses were carried out with SPSS 15.0 for Windows.

Results

Patients

Identification database: cross-sectional study

All the 500 patients included in the study filled in the PBI completely (no missing data). 61.1% were female. They aged 13–97 years (mean 45.0 ± 19.5).

Replication database: acne therapy study

749 of the 925 patients included in the study filled in the PNQ. Amongst these, 421 (56.6%) were female and 323 male (n = 5 missing). They aged 12–30 years (mean 18.8 ± 5.4). Clinical results of this study will be published elsewhere.

Factor analysis

Identification database: cross-sectional study

Bartlett test was highly significant (p < 0.001). Factor analysis revealed five factors explaining 63.0% of the total variance (Table 1, left side). Each item loaded highest on one of the factors and was allocated according to this factor (Table 2).

Table 1 Results of factor analyses: Eigen values and explained variance
Full size table
Table 2 Factor identification
Full size table

The first four factors could easily be interpreted as ‘reducing social impairments’, ‘reducing psychological impairments’, ‘reducing impairments due to therapy’, and ‘reducing physical impairments’. Factor 5 was interpreted as ‘having confidence in healing’.

Cronbach’s alpha of the corresponding subscales was 0.87, 0.85, 0.81, 0.78, and 0.66, respectively. Thus, only the internal consistency of the subscale of factor 5 was slightly not satisfying [7].

Replication database: acne therapy study

Bartlett test was highly significant (p < 0.001). Factor analysis revealed four factors explaining 61.5% of total variance (Table 1, right side). Again, each item loaded highly on only one factor and was thus allocated to the respective factor (Table 3). The factors were interpreted as ‘reducing social and psychological impairments’, ‘reducing impairments due to therapy’, ‘reducing physical impairments’, and ‘having confidence in healing’.

Table 3 Replication
Full size table

Cronbach’s alpha of the corresponding scales was 0.92, 0.81, 0.82, and 0.72. Thus, reliability of all factors was satisfying [7].

This replication solution differed from the solution identified in the identification database in two aspects:

  1. 1.

    Factor 1 represented items concerning the reduction of social and psychological impairments, while these items formed two distinct factors in the solution found in the identification database.

  2. 2.

    Factor 4 of the replication database (having confidence in healing) contains the same items as factor 5 in the solution found in the identification database, plus the item ‘to be healed of all skin alterations’, which in the cross-sectional study loaded highest on the factor ‘reducing physical impairments’.

Decision on subscales

Since both analyses yielded partly different solutions, a decision needed to be made about which classification to select for further use.

The cross-sectional identification sample included patients with a wide range of skin diseases and, thus, results from this study might be more representative of dermatological patients.

In the replication sample, factor 5 might represent the wish to have a clear, trustworthy therapy, which provides long-lasting reduction of skin blemishes being the main symptom of acne vulgaris. This need might derive from a pragmatic expectancy of complete skin healing with low effort and quick diagnosis. However, needs relating to reduction of further physical impairments (e.g. pain, itching) were allocated to factor 3.

In the replication sample, the first factor (psychological and social impairments) contains 11 items––which is nearly half the total item count––while the remaining factors contain only 4 items each. In the identification database, however, these 11 items are relegated to two factors making the subscales more equal in item count. Since a treatment’s benefit might differ regarding psychological and social impairments, we consider it more appropriate to allocate the 11 items to two distinct subscales, as found in the identification database.

The different allocation of the item ‘to be healed of all skin alterations’ in both studies might be explained by the characteristics of the respective study samples. In acne patients (replication database), this is the only item referring directly to their main physical symptom (skin blemishes), which supposedly all of the patients aim to eliminate. By contrast, the identification database patients with other diagnoses, e.g. atgopic dermatitis and ulcus cruris, have several of their symptoms addressed in other items, referring for example to burning sensations, itching, or pain.

Allocating the item to the physical factor would facilitate subscale interpretation because, first, all items referring to physical symptoms would be included in one factor. Secondly, the items covered by the last factor would match well, resulting in a factor containing three items on ‘having confidence in healing’.

On the other hand, reliability of the identification database’s factor 5, not containing the item in question, was rather low (Cronbach’s alpha = 0.66). Therefore, reliability of subscales according to the five-factor solution from the identification database was determined for the replication database. Resulting Cronbach’s alpha values were consistently satisfying, amounting to 0.87 (subscale 1), 0.88 (subscale 2), 0.80 (subscale 4), 0.76 (subscale 4), and 0.74 (subscale 5).

In order to obtain subscales that are at the same time easily interpretable and generalisable to patients with a range of skin diagnoses, we decided to allocate the item ‘to be healed of all skin alterations’ to the factor ‘reducing physical impairments.’

Thus, the solution with five subscales derived from the identification database (Table 3) was chosen for further use.

Comparison of patient needs in both samples

In Figs. 1, 2, 3, 4 and 5, mean importance of treatment needs represented by each of the five subscales is given for the ten diagnose groups of the identification database and for the acne patients of the replication database. In all groups, confidence in therapy (Fig. 5) was rated the most important amongst the five need dimensions. When interpreting the mean importance values, is has to be taken into account that the answer ‘does not apply to me’ had been coded as 0. Thus, a subscale that includes items referring to impairments which many patients are not affected by will have a lower mean. For example, the physical subscale (Fig. 4) covers impairments that only some of the patients experience such as itching, pain, burning sensations and sleep disorders; this might explain the low values in this subscale in hyperhidrosis and hair disease patients.

Fig. 1
figure 1

Importance of treatment needs in dimension 1 ‘Reducing psychological impairments’; cross-sectional identification sample; n = 50 (mean ± SD; range: 0 = not at all important to 4 = very important; does not apply to me = 0)

Full size image
Fig. 2
figure 2

Importance of treatment needs in dimension 2 ‘Reducing social impairments’; cross-sectional identification sample; n = 50 (mean ± SD; range: 0 = not at all important to 4 = very important; does not apply to me = 0)

Full size image
Fig. 3
figure 3

Importance of treatment needs in dimension 3 ‘Reducing impairments due to therapy’; cross-sectional identification sample; n = 50 (mean ± SD; range: 0 = not at all important to 4 = very important; does not apply to me = 0)

Full size image
Fig. 4
figure 4

Importance of treatment needs in dimension 4 ‘Reducing physical impairments’; cross-sectional identification sample; n = 50 (mean ± SD; range: 0 = not at all important to 4 = very important; does not apply to me = 0)

Full size image
Fig. 5
figure 5

Importance of treatment needs in dimension 5 ‘Building confidence into therapy’; cross-sectional identification sample; n = 50 (mean ± SD; range: 0 = not at all important to 4 = very important; does not apply to me = 0)

Full size image

Discussion

In both studies, largely congruent dimensions were found. The solutions differed only in the number of factors (4 vs. 5) and in the allocation of one item. The solution from the identification database with five well interpretable subscales was chosen for further use.

In further studies, benefit subscales can be computed according to the PBI formula described in the introduction with the need and benefit items belonging to the respective dimensions, providing information on patient benefit in different areas. However, the explained variance of 63.0% indicates that more than one-third of the information gathered with the PBI is not covered by the five dimensions. Therefore, the global score should be computed in addition to the subscales––assuming that the non-explained variance is not error variance only.

The dimensions cover a wide range of patient needs including physical, social, and psychological burdens as well as impairments due to treatment and the desire to have confidence in therapy. Since the PBI items include those needs that have repeatedly been mentioned by patients with various skin diseases in an open survey, they should contain the most important and prevalent treatment goals in dermatology. However, it is possible that single, rather infrequent needs will not be covered by the PBI and its subscales.

Generalisability of the present results is a prerequisite for using PBI subscales in studies on patients with various skin diseases. It is supported by the high concordance of the results which has been found in spite of marked differences between both studies: In the replication database (acne therapy study), young patients were asked to participate by their dermatologists at the beginning of the medication. In contrast, in the cross-sectional study, patients of all ages visiting the university clinics consultation with ten distinct skin diseases were asked to help validate the PBI by taking part in the study. These patients differed regarding the duration of the current therapy. However, the results may not be generalizable to those patients who do not consult dermatologists and who could therefore not be included in any of the two databases.

For computing subscales of one of the indication-specific PBI versions, another factor analysis needs to be conducted because these versions contain both items from the PBI-standard version and disease-specific items.

We conclude that patient benefit of dermatological treatments can be measured using five well distinguishable and interpretable subscales of the PBI.