Multicenter Collaborative Panic Disorder Severity Scale

In DSM-IV, panic disorder (with agoraphobia) is defined by unexpected panic attacks, anticipatory anxiety, and fear and avoidance of situations where escape might be difficult in the event of a panic attack. Although semistructured interviews to establish panic disorder diagnoses, such as the Anxiety Disorders Interview Schedule for DSM-IV (1), and self-report measures of constituent panic disorder features, such as the Anxiety Sensitivity Index (2) and Mobility Inventory for Agoraphobia (3), are available, a simple and standardized instrument to assess overall panic disorder severity does not exist. Therefore, we developed and evaluated a brief, clinician rating scale for this purpose—the Panic Disorder Severity Scale.

METHOD

Scale Development and Initial Evaluation

The Panic Disorder Severity Scale, modeled after the Yale-Brown Obsessive Compulsive Scale (4, 5), contains items that assess the severity of seven dimensions of panic disorder and associated symptoms: 1) frequency of panic attacks; 2) distress during panic attacks; 3) anticipatory anxiety (worry about future panic attacks); 4) agoraphobic fear and avoidance; 5) interoceptive fear and avoidance (i.e., apprehension and avoidance of bodily sensations) (6, 7); 6) impairment of or interference in work functioning; and 7) impairment of or interference in social functioning. The scale is administered by a clinician using a scripted interview (in which the past month is used as the reference period). On the basis of the patient's responses, the clinician rates the severity of each feature on a 0 to 4 scale (0=“none,” and higher ratings reflect greater degrees of symptom severity). A composite score is established by averaging the scores on the seven items.

The scale includes an item assessing fear and avoidance of uncomfortable bodily sensations, now known to play a role in panic disorder morbidity (6, 7) but rarely measured. Sensation avoidance leads patients to curtail activities in which uncomfortable bodily sensations may be provoked. Examples include exercise, exposure to a hot or humid climate, emotionally arousing work-related activities (anxiety- or conflict-producing meetings, presentations), leisure activities associated with intense emotionality (exciting sports events, movies, sexual excitement).

The preliminary reliability of the Panic Disorder Severity Scale was evaluated by using six written vignettes derived from clinical cases of panic disorder of varying severity. Before their participation in the multicenter study, 10 experienced clinicians underwent extensive training in use of this scale and other assessment instruments (see next section). After training, these clinicians provided independent ratings of the six vignettes with the Panic Disorder Severity Scale. Excellent interrater reliability was obtained, as reflected by an intraclass correlation of 0.92 (df=2,8). On the basis of this result, the Panic Disorder Severity Scale was included as an outcome measure in the Multicenter Collaborative Treatment Study of Panic Disorder.

Psychometric Evaluation

The subjects (N=186) in the present study were recruited from patients who were participating in the multicenter study. Each patient received a principal DSM-III-R diagnosis of panic disorder with no or mild agoraphobia, as determined by the Anxiety Disorders Interview Schedule, Revised (8). The patients signed statements giving informed consent for the treatment study, including consent for audiotaped assessment interviews. Patients who met the study criteria underwent 2 weeks of self-monitoring of panic and anxiety symptoms (9). In order to proceed to random treatment assignment, the patients were required to report at least one full panic attack or a limited-symptom attack during the self-monitoring period. Eligible patients were randomly assigned to treatment with imipramine, placebo, cognitive behavioral panic-control treatment, or combination treatment (either panic-control treatment plus imipramine or panic-control treatment plus placebo). Posttreatment assessment occurred after 11 treatment sessions.

Pre- and posttreatment evaluations were conducted by 10 independent evaluators who achieved criterion reliability on the Anxiety Disorders Interview Schedule, Revised, and the Panic Disorder Severity Scale by matching scores with an independent evaluator at the Pittsburgh site, who co-rated audiotaped interviews. In addition, training involved matching the independent evaluator's scores on the Clinical Global Impression scale (CGI), which consisted of 7-point ratings of overall panic disorder severity and global improvement (10). For assessment of interrater reliability, a subset of the study group (N=24) was randomly selected to undergo two independent pretreatment assessments with the Panic Disorder Severity Scale by clinicians at the same site. In addition, the audiotaped pretreatment Panic Disorder Severity Scale assessments for another 24 patients were independently co-rated by an independent evaluator at the Pittsburgh site. For the purpose of evaluating convergent and discriminant validity, the following pretreatment measures were analyzed:

1. Anxiety Disorders Interview Schedule, Revised. In addition to providing DSM-III-R diagnoses, the Anxiety Disorders Interview Schedule provides dimensional measures of panic disorder severity (0–8 scale), panic frequency in the past month, fear of panic in the past month (0–8 scale), fear and avoidance ratings (0–4 scale) of 20 situations (range of scores=0 to 80) and 17 activities producing physical sensations (range of scores=0 to 68).

2. Subjective Symptoms Scale. A modified version of the measure introduced by Hafner and Marks (11) was used. The Subjective Symptoms Scale contains 9-point ratings (0–8) of the extent to which anxiety symptoms interfered with five areas of functioning during the past week (work, home management, private leisure, social leisure, family relationships).

3. Anxiety Sensitivity Index. This measure is a 16-item questionnaire designed to assess fear of the symptoms of anxiety (2).

4. Albany Panic and Phobia Questionnaire. This 27-item measure of situational and interoceptive fear comprises three subscales: agoraphobia, social, and interoceptive (7).

RESULTS

Reliability and Latent Structure

With regard to interrater reliability, the random-model intraclass correlation coefficient for the 24 independently rated audiotaped Panic Disorder Severity Scale assessments was 0.87 (df=2,23, p<0.001); for the 24 pairs of separate assessments, the intraclass correlation coefficient was 0.88 (df=2,23, p<0.001). Across these two subgroups, interrater reliability on individual scale items ranged from 0.74 (item 2: distress during panic attacks) to 0.87 (item 4: agoraphobic fear and avoidance).

For the entire study group (N=186), the internal consistency (Cronbach's alpha) of the pretreatment total score on the Panic Disorder Severity Scale was 0.65. Mean scores and standard deviations for individual items and correlations between each item and the total score are presented in table 1. Although the item-total correlations were modest for some items (e.g., items 1 and 5), results indicated that alpha would not increase with the omission of any individual item. Because fair reliability could reflect a lack of unidimensionality of the severity scale, the latent structure of the seven constituent scale items was evaluated by using a linear structural relations program and a maximum-likelihood solution (LISREL 8.14) (12). First, the hypothesized one-factor solution was fit to the data. The results indicated that all seven items had salient factor loadings; the completely standardized lambda-X coefficients were 0.36, 0.49, 0.44, 0.46, 0.35, 0.54, and 0.59 for items 1 through 7, respectively. Indices of overall fit for the one-factor model were as follows: χ²=39.32, df=14, p<0.001; comparative fit index=0.82, incremental fit index=0.83, root mean square error of approximation=0.099, global fit index=0.94.

Inspection of the standardized residuals and modification indices from the theta-delta matrix indicated an association between items 1 and 2 (panic frequency and distress, respectively) that was not adequately accounted for by the single-factor solution. Thus, a two-factor model was fit to the data; items 1 and 2 formed the first factor, and items 3 through 7 formed the second. Relative to the one-factor model, the two-factor model provided a significantly improved fit (nested χ²=14.88, df=1, p<0.001). Indeed, this model provided a good fit for the data (χ²=24.44, df=13, p<0.05; comparative fit index=0.92, incremental fit index=0.92, root mean square error of approximation=0.069, global fit index=0.96. The factor loadings for the first factor were 0.49 and 0.77 (items 1 and 2, respectively), and the factor loadings for the second factor were 0.43, 0.50, 0.38, 0.54, and 0.62 (items 3 through 7, respectively). The two factors were moderately correlated (r=0.55, N=185, p<0.001). Inspection of the standardized residuals and modification indices from the theta-delta and lambda-X matrix suggested no further refinements in the factor model.

Convergent and Discriminant Validity

Table 2 presents correlations of the item and total scores on the Panic Disorder Severity Scale with scores on the Anxiety Disorders Interview Schedule and other questionnaire measures. As shown in this table, the Panic Disorder Severity Scale total score was correlated significantly with the Anxiety Disorders Interview Schedule clinical severity rating of panic disorder (r=0.55, N=145, p<0.001). As a test of convergent and discriminant validity, it was predicted that each item on the Panic Disorder Severity Scale would be most strongly correlated with an Anxiety Disorders Interview Schedule or questionnaire measure assessing a similar or overlapping domain (no predictions were made for item 2 of the Panic Disorder Severity Scale because no criterion measure was available). As summarized in table 2, statistical tests of the relative magnitude of the correlations (13) supported these predictions overall (e.g., Panic Disorder Severity Scale item 1, panic frequency, was more strongly correlated with the Anxiety Disorders Interview Schedule rating of panic frequency than was any other Panic Disorder Severity Scale item). Not surprisingly, the Panic Disorder Severity Scale was more strongly correlated with the Anxiety Disorders Interview Schedule than with questionnaire measures; this finding is attributable in large part to shared-method variance (i.e., the independent evaluator completing the Panic Disorder Severity Scale had previously administered the Anxiety Disorders Interview Schedule). Nevertheless, it is encouraging that, albeit of smaller magnitude, the pattern of correlations between the Panic Disorder Severity Scale and the questionnaires was consistent with the results involving measures from the Anxiety Disorders Interview Schedule, indicating differential associations with the criterion measures in the predicted directions (e.g., Panic Disorder Severity Scale item 4, agoraphobic fear/avoidance, was more strongly correlated with the agoraphobia rating from the Albany Panic and Phobia Questionnaire than was any other item on the Panic Disorder Severity Scale).

Sensitivity to Change

Of a subgroup of 89 patients completing the posttreatment assessment, 69 (78%) and 20 (22%) were classified as treatment responders and nonresponders, respectively, on the basis of CGI ratings of global improvement by independent evaluators. The sensitivity to change of the Panic Disorder Severity Scale was analyzed by using a time-by-group analysis of variance (time: pre-, posttreatment; group: responder, nonresponder). Both of the main effects and the interaction effect were statistically significant (group: F=11.08, df=1,87, p<0.001; time: F=57.15, df=1,87, p<0.001; interaction: F=41.36, df=1,87, p<0.001). Post hoc analyses (Fisher's protected t tests) of the interaction effect indicated that, although posttreatment total scores on the Panic Disorder Severity Scale (mean=1.43, SD=0.67) did not differ from pretreatment scores (mean=1.51, SD=0.49) for the nonresponders (t=0.90, df=19, p<0.38), treatment responders evidenced a significant decline (t=13.83, df=68, p<0.001) from pretreatment (mean=1.62, SD=0.42) to posttreatment (mean=0.64, SD=0.48). Because the study blind had not been broken at this writing, differences between active treatment and placebo are not reported.

DISCUSSION

The Panic Disorder Severity Scale is a seven-item, interview-based scale for assessing the severity of panic disorder. The results reported here indicate that the scale has good interrater and test-retest reliability when administered by trained raters. In addition, the findings provide initial support for the concurrent and discriminant validity of the scale and indicate that the total score is sensitive to clinically meaningful change. However, the modest internal consistency of the total score is notable (alpha=0.65). There are several possible explanations for this result. For instance, whereas data suggest that the features constituting the DSM-IV diagnostic definition of panic disorder form a coherent syndrome (14), research has also documented a partial independence of these symptom domains (e.g., panic frequency is not strongly associated with agoraphobic avoidance) (15). Indeed, the results of confirmatory factor analysis, indicating improved data fit for a two-factor model (whereby the items for panic frequency and panic distress loaded on a separate factor), could be viewed as support of a multidimensional structure of panic disorder.

On the other hand, our results pertaining to the factor structure and internal consistency of the Panic Disorder Severity Scale should be regarded as somewhat tentative, given the nature of the study group. Specifically, the inclusion criteria for enrollment in the Multicenter Collaborative Treatment Study of Panic Disorder included 1) no more than mild agoraphobic avoidance, 2) willingness to discontinue any preexisting medication regimens, and 3) willingness to be randomly assigned to treatment conditions involving cognitive behavior, antidepressant, and placebo therapy. Accordingly, the study selection procedures likely contributed to range restriction for some panic disorder features (agoraphobic avoidance, in particular), which would thereby affect the psychometric behavior of some of the scale items. Thus, future research should examine the psychometric properties of the Panic Disorder Severity Scale in more diverse groups of patients with panic disorder and should use a broader range of convergent and discriminant validity measures.

A final point relates to the relationship between the Panic Disorder Severity Scale and the Yale-Brown Obsessive Compulsive Scale. As noted earlier, the Panic Disorder Severity Scale was modeled after the Yale-Brown Obsessive Compulsive Scale, which is used widely and is generally accepted as a standard measure of obsessive-compulsive disorder symptoms. While this scale initially showed very high internal consistency (4), more recent reports (16) place the alpha in a more modest range, similar to our findings for the Panic Disorder Severity Scale.

Why use the Panic Disorder Severity Scale when panic disorder severity can be assessed in other ways? Other measures of overall severity include the CGI and composite scales or severity ratings derived from semistructured interviews, such as the Anxiety Disorders Interview Schedule. However, single-item Likert-scale ratings such as the CGI typically possess less than optimal psychometric properties because they do not assess the full range of the salient features of panic disorder and because evaluators may interpret scale points idiosyncratically. Moreover, composite or severity ratings from semistructured interviews can be complex and cumbersome. For example, the Anxiety Disorders Interview Schedule is a lengthy semistructured interview, designed to generate DSM-III-R diagnoses. The panic disorder section of the Anxiety Disorders Interview Schedule requires about twice as much time to complete as the Panic Disorder Severity Scale. However, it should be noted that the Panic Disorder Severity Scale is not a diagnostic instrument. Rather, it provides a simple and efficient way to monitor panic disorder severity in patients in research and clinical settings for whom a diagnosis has been established. In short, the Panic Disorder Severity Scale holds a unique place in the assessment of panic disorder, with or without agoraphobia, as the only scale with which to evaluate the panic disorder severity on the basis of a systematic evaluation of DSM-IV criteria.

TABLE 1

Enlarge table

TABLE 2

Enlarge table