The Brown Assessment of Beliefs Scale: Reliability and Validity

Delusions represent a complex yet fundamental concept in psychiatry. They are a cornerstone of psychosis and the core symptom of the DSM diagnosis of delusional disorder. However, the assessment of delusions has received surprisingly little investigation. At this time there is no widely used, reliable, and valid clinician-administered instrument to assess this important construct.

In 1923, Karl Jaspers (1) defined a delusion as “a judgment made which is held on to with full conviction, not only with a consciousness of validity but with a sense of absolute certainty” (p. 135). DSM-IV similarly defines a delusion as “a false belief based on incorrect inference about external reality that is firmly sustained despite what almost everyone else believes and despite what constitutes incontrovertible and obvious proof or evidence to the contrary” (p. 765). A controversial question is whether delusions are a categorical, dichotomous construct—that is, either present or absent—as conceptualized in DSM, or a dimensional construct, with delusionality characterized by a number of components contributing to a continuum of insight that spans a spectrum from good to poor to absent.

Delusionality has traditionally been considered a categorical symptom—that is, delusions are either present or absent. The presence of delusions is important in the diagnosis of schizophrenia, delusional disorder, and mood disorder with psychotic features. In DSM-III-R, the absence of delusional thinking was required for the diagnosis of obsessive-compulsive disorder (OCD), body dysmorphic disorder, and hypochondriasis. It has been argued, however, that delusionality is a continuous rather than a dichotomous construct. Sacks et al. (2), for example, demonstrated that delusions in schizophrenia form and resolve gradually, rather than in an “all or nothing” fashion. Similarly, Strauss (3) described a range of conviction in the delusions of schizophrenia, noting that certain delusions are held with less than complete conviction.

The question of whether delusionality is a continuous rather than a categorical construct has also been extended to other disorders, such as OCD and body dysmorphic disorder (4–6). In 1986, Insel and Akiskal (7) proposed that insight in OCD spans a continuum ranging from good insight, in which patients clearly recognize the excessiveness and/or senselessness of their concerns, to delusional conviction, in which the obsessions are considered realistic and reasonable. In recognition of this possibility, DSM-IV added a “poor insight” specifier to OCD for patients with poor insight into the irrationality of their obsessions and need-to-perform compulsions. In addition, DSM-IV allows patients with no insight into their OCD or body dysmorphic disorder concerns to receive two diagnoses: delusional disorder plus OCD or body dysmorphic disorder. This double-coding acknowledges the possibility that these disorders may span a spectrum of insight and that delusionality may be a continuous rather than a categorical concept. This compromise also underscores the fact that this issue is controversial and remains unresolved.

Another unresolved issue is what the components of delusionality are. Kendler et al. (8) developed a scale (the Dimensions of Delusional Experience) measuring five components of delusions in 52 subjects with schizophrenia: conviction, pressure, extension, systematization, and bizarreness. Correlations among the components were low, suggesting that delusionality has multiple components. The Dimensions of Delusional Experience scale is reliable but lacks specific probes and is not easily applied to disorders other than schizophrenia that may be characterized by delusional thinking, such as OCD, body dysmorphic disorder, and hypochondriasis. A study of an unvalidated self-report scale, the Characteristics of Delusions Rating Scale (9), found few strong correlations among component characteristics, also supporting the notion that delusions are multidimensional. The reliable and valid Scale to Assess Unawareness of Mental Disorder (10), which includes 17 items that measure insight and two ratings of delusions, also supports the notion that insight, like delusionality, has multiple components and can be rated on a continuum. However, most of the items assessed by this scale are not relevant to disorders with delusional variants, such as OCD and body dysmorphic disorder, that are not characterized by formal thought disorder.

Two measures that have been designed to assess delusionality in OCD, the Fixity of Beliefs Scale (11) and a structured interview that assesses multiple dimensions of beliefs (12), demonstrated a range of insight in OCD but do not have established interrater reliability or validity and are not easily applied to disorders other than OCD.

Given the importance of delusions in psychiatry, a reliable and valid clinician-administered instrument is needed to assess the presence and degree of delusionality in a wide variety of psychiatric disorders. Such an instrument is needed to refine diagnostic classification and to resolve some of the classification controversies that were debated for DSM-IV and will be revisited for DSM-V—for example, whether the delusional and nondelusional variants of disorders constitute the same disorder (13). Such a scale is also needed to define homogeneous groups of subjects for research and to examine the relationship of delusionality to treatment outcome and prognosis.

In this report, we present data on a scale to assess delusional thinking in a variety of psychiatric disorders. We selected three diagnoses (OCD, body dysmorphic disorder, and psychotic mood disorder) for this study. OCD and body dysmorphic disorder were selected because of a lack of validated insight/delusionality scales applicable to these disorders. In addition, as previously noted, important questions regarding delusionality in OCD and body dysmorphic disorder were raised during the DSM-IV process and remain unresolved. Similar questions pertain to psychotic mood disorder—for example, whether the psychotic and nonpsychotic variants of major depression are the same or different disorders (14). In addition, including psychotic mood disorder assesses the scale's application to a more “classic” psychotic disorder.

METHOD

Scale Description

The Brown Assessment of Beliefs Scale is a seven-item clinician-administered semistructured scale designed to assess delusionality of beliefs in a broad range of psychiatric disorders. To administer the scale, the dominant belief (obsession, concern, idea, worry, or delusion) that has preoccupied the patient during the past week is first established. For obsessions, the belief and associated consequences underlying the obsessional thought must be determined. An example of an underlying belief in OCD is that AIDS could be contracted by touching any object outside the home. In body dysmorphic disorder, the belief might consist of the thought that one is among the ugliest people in the world or a greatly exaggerated view of the ugliness of minor skin blemishes. An example of a false belief in mood disorder with psychotic features is that one is half angel and half human.

The Brown Assessment of Beliefs Scale has specific probes and five anchors for each item, with descriptions corresponding to each anchor. An example of a probe is the following, which assesses conviction: “How convinced are you that you'll get sick from eating any food that was touched by anyone other than yourself? Are you certain your belief is accurate?” The score for each item ranges from 0 (nondelusional, or least pathological) to 4 (delusional, or most pathological). Ratings represent an average score for the past week.

Scale Development

The scale's original version contained 15 items culled from a review of the literature on delusionality (15). The items conviction, perception of others' views of beliefs, fixity of ideas, stability of beliefs over time, ego-syntonicity, and attempt to disprove beliefs were derived from the literature on delusions (8, 16) and on insight in OCD (4, 7, 11, 12). Several items similar to those in the Yale-Brown Obsessive Compulsive Scale (17, 18) were also derived from the literature on delusions: time preoccupied, time free from concerns, distress, control, and interference with functioning. Three items were derived primarily from the schizophrenia literature on delusions: need to convince others, bizarreness, and coherence (2, 3, 8). Insight—an awareness of the psychiatric etiology of the belief—has been written about extensively (19) and is included in the Scale to Assess Unawareness of Mental Disorder (10, 20). Three items were added following our pilot study: explanation of differing views, ideas/delusions of reference, and extension, which was derived from the scale of Kendler et al. (8).

The original 15-item version of the Brown Assessment of Beliefs Scale was administered to 46 subjects in a pilot study: 18 with OCD, 18 with body dysmorphic disorder, and 10 with mood disorder with psychotic features (15). Subjects were obtained from the clinical practices of the first two authors (J.L.E. and K.A.P.) and from a hospital inpatient unit. One of the authors (J.L.E., K.A.P., or D.A.B.) conducted an audiotaped interview during which the scale was administered; interrater reliability was determined by subsequent independent ratings of 10 audiotaped interviews by the other two interviewers. Interrater reliability was good to excellent. Intraclass correlations (ICCs) were 0.95 for the total score and ranged from 0.81 to 0.99 for individual items, with the exception of interference with functioning and control, which were subsequently deleted from the scale. Cronbach's alpha coefficient was 0.83, and factor analysis yielded three factors (core features, severity, and psychosis), accounting for 67% of the variance.

Study Procedures

Subjects. The 18-item scale was administered to 50 subjects who had not participated in the pilot study; 20 subjects had OCD, 20 had body dysmorphic disorder, and 10 had mood disorder with psychotic features (eight had bipolar affective disorder, and two had major depression). Thirty subjects (60%) were women, and the mean age of all subjects was 35.2 years (SD=10.7). Subjects were referred from OCD and body dysmorphic disorder outpatient programs at two university-based hospitals, from new referrals to these programs, and from hospital inpatient units. All diagnoses were made according to DSM-III-R criteria, by a psychiatrist (J.L.E., K.A.P., or D.A.B.) with extensive clinical experience and expertise in OCD, body dysmorphic disorder, or psychotic mood disorder, on the basis of a detailed clinical interview and available records. In addition, each patient with OCD was assessed with the Yale-Brown Obsessive Compulsive Scale symptom checklist and a semistructured clinical interview to obtain additional information about OCD (21). Subjects with body dysmorphic disorder were diagnosed by using a reliable semistructured diagnostic instrument for body dysmorphic disorder (22) modeled after the Structured Clinical Interview for DSM-IV (23). Subjects were diagnosed as having psychotic mood disorder only if there was diagnostic agreement among the research psychiatrist, the admitting psychiatrist, and the treating psychiatrist based on chart review and patient interview. Patients were at various stages of treatment; therefore, their symptoms were at different levels of severity when the scale was administered. Written informed consent was obtained from all subjects.

Instrument administration. One of four raters administered the Brown Assessment of Beliefs Scale, which was audiotaped. Three of the raters (J.L.E., K.A.P., and D.A.B.) were developers of the scale, and the fourth rater (K.D.A.) was trained in its use by observing five interviews conducted by one of the scale's developers. All raters were experienced clinicians. The dominant belief (obsession, concern, idea, worry, or delusion) that had preoccupied the subject during the previous week was first established. The rater then asked the scale's specific probes.

Reliability. To assess interrater reliability, the audiotaped interviews were independently rated by the three other interviewers. All scoring was done blind to the other interviewers' ratings. Test-retest reliability was evaluated by administration of the Brown Assessment of Beliefs Scale by the same interviewer to 27 patients (16 patients with body dysmorphic disorder and 11 with OCD) on two different occasions separated by a 6-day interval. ICCs were calculated to determine reliability (24).

Discriminant and convergent validity. To assess discriminant and convergent validity, the interviewers administered the Yale-Brown Obsessive Compulsive Scale or the Body Dysmorphic Disorder–Yale-Brown Obsessive Compulsive Scale (a slightly modified version of the scale that assesses severity of body dysmorphic disorder) (25), the Brief Psychiatric Rating Scale (BPRS) (26), the Dimensions of Delusional Experience scale (8), the Scale to Assess Unawareness of Mental Disorder (10), and the Fixity of Beliefs Scale (11). Two self-report scales were completed: the Characteristics of Delusions Rating Scale (a visual analog scale that measures 11 dimensions of delusionality) (9) and the Beck Depression Inventory (27). We created a total score on the Characteristics of Delusions Rating Scale by summing the scores on each item. All ratings were obtained on the same day.

On the day the Brown Assessment of Beliefs Scale was administered, the treating clinician (who did not administer the scale) independently made a global rating of delusionality using anchors from the experimental insight question of the Yale-Brown Obsessive Compulsive Scale (0=excellent insight, 1=good insight, 2=fair insight, 3=poor insight, or 4=no insight; delusional). For patients with OCD or a psychotic mood disorder, not all of the global raters were experts in delusionality. However, the global rater for patients with body dysmorphic disorder had expertise in delusionality and extensive familiarity with the literature on delusionality.

Sensitivity to change. We conducted a separate study to assess the scale's sensitivity to change in delusionality during treatment. As part of a multicenter study of sertraline in the treatment of OCD, the Brown Assessment of Beliefs Scale was administered to 38 consecutive subjects at the study baseline visit and after 16 weeks of unblinded administration of sertraline (mean dose at 16 weeks=190 mg/day, SD=26.6, range=50–200) (28). Three of these subjects also participated in the reliability and validity study described above. Study coordinators and the principal investigators were trained in the administration of the scale by one of the authors (J.L.E.), who visited each site. The training session involved an orientation in which the design, purpose, and administration of the scale were described. To establish interrater reliability, the study investigators watched and discussed videotaped interviews of the scale's administration to two patients with OCD and subsequently independently rated a third interview.

Scale Revision

After extensive experience with administration of the 18-item version of the Brown Assessment of Beliefs Scale, the scale was reduced to seven items because we thought that the length of the 18-item version would limit its use. In addition, we considered certain items relatively weak on the basis of their psychometric properties and the richness of the information supplied. Time occupied by beliefs, time interval free from beliefs, control, interference with functioning, and distress were deleted because these items were primarily measures of illness severity, which is assessed by other instruments, such as the Yale-Brown Obsessive Compulsive Scale, rather than core features of delusionality. In addition, correlations between these items and the total scale score were not strong (r=0.38, 0.37, 0.46, 0.27, and 0.13, respectively). The items coherence, need to convince others (pressure), and extension were dropped because of poor correlation with the total scale score (r=0.02, 0.31, and 0.09, respectively) and because they did not contribute substantial information about delusionality. Stability of beliefs over time was deleted because, although correlation of this item with the total scale score was high (r=0.68), we thought that this concept would be better assessed by prospective ratings with the scale. Although there is substantial literature on the importance of evaluating bizarreness, or plausibility, in delusional patients (29, 30), this item was not retained because of low correlation with the total score (r=0.02) and weak interrater reliability. Ego-syntonicity, a frequently discussed concept in the OCD literature, was dropped because of relatively weak correlation with the total score (r=0.32) and difficulty operationalizing and communicating this abstract concept to patients.

One item, ideas/delusions of reference, is scored but not included in the scale's total score. This item was retained because it yields important data about delusionality, particularly for certain disorders such as body dysmorphic disorder, olfactory reference syndrome, and other types of delusional disorder. However, because ideas of reference are not relevant to all disorders characterized by delusional thinking, this item's score is not included in the total score.

The final seven-item scale can be administered in only 10 to 15 minutes and includes items that have good psychometric properties and, in our view, are most relevant to the construct of delusionality. The scale's final items are conviction, perception of others' views of beliefs, explanation of differing views, fixity of ideas, attempt to disprove beliefs, insight, and ideas/delusions of reference.

Statistical Analysis

ICCs were used to determine interrater and test-retest reliability (24), and Pearson's correlation was used to determine correlations between each item and the total score minus that item. Cronbach's alpha coefficient was used to evaluate internal consistency. Principal components factor analysis was conducted using orthogonal (varimax) rotation. The number of factors was based on an examination of eigenvalues greater than 1. Pearson correlations were also done to determine convergent and discriminant validity, i.e., the relationship between total score on the scale and scores on other scales.

The sensitivity and specificity of the scale in classifying patients as delusional or nondelusional, and the percentage of subjects correctly classified, were calculated for body dysmorphic disorder by using a variety of total score thresholds ranging from 14 to 20 and the “gold standard” rating from the global Yale-Brown Obsessive Compulsive Scale experimental insight question completed by the treating clinician (score of 4=delusional; score of 0–3=nondelusional). This analysis is presented only for patients with body dysmorphic disorder because the global “gold standard” ratings for body dysmorphic disorder were made by a clinician with extensive familiarity with the literature on delusions (K.A.P.).

To evaluate the sensitivity of the scale to change with treatment, a t test comparing mean baseline score and mean posttreatment score was calculated for the 38 patients with OCD who participated in the sertraline study.

RESULTS

Reliability

ICCs demonstrated good agreement across the four raters for the total score and individual item scores (table 1). As shown in table 2, correlations between each item and the total score minus that item ranged from r=0.38 to r=0.85. Cronbach's alpha coefficient was 0.87, indicating acceptable homogeneity of the scale. The individual item test-retest ICCs ranged from 0.79 to 0.98, with a median of 0.95.

Factor Structure

Factor analysis identified one factor with eigenvalues greater than 1 when conducted with the seven items. All seven items loaded strongly together on the factor identified, with loadings ranging from 0.48 to 0.92. This factor accounted for 56% of the variance.

Discriminant and Convergent Validity

As shown in table 3, the total score on the Brown Assessment of Beliefs Scale was not significantly correlated with total score on the Beck Depression Inventory or the Yale-Brown Obsessive Compulsive Scale. The Brown Assessment of Beliefs Scale correlated only weakly with the BPRS, although this correlation reached statistical significance (table 3). As a further test of discriminant validity, we analyzed correlations between the total score on the Brown Assessment of Beliefs Scale and the 18 individual items on the BPRS. All correlations were nonsignificant except for emotional withdrawal (r=0.37, p=0.008) and unusual thinking (r=0.39, p=0.005), which assesses the presence of delusions.

Convergent validity was demonstrated with the Characteristics of Delusions Rating Scale total score and the two items that assess delusional thinking on the Scale to Assess Unawareness of Mental Disorder (table 3). A high correlation was also found between the conviction item on the Brown Assessment of Beliefs Scale and conviction items on the Dimensions of Delusional Experience, the Fixity of Beliefs Scale, and the Characteristics of Delusions Rating Scale (table 3).

Discriminant and convergent validity were also assessed for each diagnosis separately. Findings for each diagnosis were similar to those already described with one exception: correlation of the total score on the Brown Assessment of Beliefs Scale with the BPRS did not reach statistical significance for the OCD group (r=0.22, p=0.36) and the mood disorder group (r=–0.47, p=0.17).

Diagnostic Threshold

Using the “gold standard” rating from the global Yale-Brown Obsessive Compulsive Scale experimental insight question for subjects with body dysmorphic disorder, we calculated the percentage of subjects correctly classified as delusional or nondelusional by the Brown Assessment of Beliefs Scale. Total score thresholds ranged from 14 to 20. In addition to achieving a cutoff point, patients could be classified as delusional only if the first item, conviction, was scored as 4—i.e., the patient was completely convinced that the belief was accurate.

Sensitivity and specificity calculations using a variety of total score thresholds indicated that the threshold for identifying subjects judged to be delusional is a total score of 18 out of a possible total score of 24. This cutoff point, in combination with a score of 4 on the first item, conviction, correctly classified 90% (N=18) of the 20 subjects assessed (six of the six delusional patients and 12 of the 14 nondelusional patients). It produced no false negatives (100% sensitivity)—that is, every patient considered delusional by the expert rater was categorized as delusional by the scale. The specificity was 86%—i.e., two patients considered nondelusional by the expert rater were categorized as delusional by the scale.

Sensitivity to Change

In the sertraline study, the mean Yale-Brown Obsessive Compulsive Scale score decreased by 49% (mean baseline score=25.0, SD=3.6; mean posttreatment score=12.7, SD=5.5). The mean Brown Assessment of Beliefs Scale score decreased by 51% (mean baseline score=8.5, SD=4.2; mean posttreatment score=4.2, SD=4.4) (t=5.47, df=37, p<0.001, paired t test), indicating that the scale is sensitive to change. The correlation between mean change in total Yale-Brown Obsessive Compulsive Scale score and mean change in total score on the Brown Assessment of Beliefs Scale was 0.43 (p<0.008), indicating that improvement in degree of delusionality as measured by the scale was correlated with but not identical to improvement in severity of OCD symptoms.

DISCUSSION

The Brown Assessment of Beliefs Scale, a clinician-administered scale designed to assess degree of delusionality across a range of psychiatric disorders, has good psychometric properties, including interrater reliability, test-retest reliability, and internal consistency. Discriminant validity findings suggest that the scale is not a measure of aspects of illness severity assessed by the BPRS, the Yale-Brown Obsessive Compulsive Scale, or the Beck Depression Inventory. The high correlation with other measures of delusionality suggests that the Brown Assessment of Beliefs Scale is a valid measure of delusionality. Our findings also indicate that the scale is sensitive to change and therefore can be used to measure change with, and efficacy of, treatment.

Advantages of the scale include its brevity, ease of administration, and provision of a continuous measure of delusionality. The determination of a cutoff point for the presence of delusionality in body dysmorphic disorder suggests that the scale is potentially useful in classifying patients according to the DSM-IV categories of delusional versus nondelusional. However, this preliminary finding in body dysmorphic disorder needs to be replicated in other diagnostic groups. Use of the Brown Assessment of Beliefs Scale may shed light on classification controversies about the relationship between delusional and nondelusional variants of disorders (for example, whether the delusional and nondelusional variants of OCD, body dysmorphic disorder, and hypochondriasis are the same or different disorders).

The scale may also aid investigation of treatment efficacy. For example, are serotonin reuptake inhibitors as effective for delusional body dysmorphic disorder and OCD as they are for the nondelusional variants of these disorders, as preliminary data suggest (27, 31)? Or are the delusional variants of these disorders best treated with a combination of a serotonin reuptake inhibitor and a neuroleptic (32)? Some investigators (33), but not others (12, 34, 35), have found that cognitive behavioral therapy is less effective for delusional OCD than for nondelusional OCD; these studies have been hampered by the lack of a reliable and valid measure of delusionality. Our findings indicate that the scale is sensitive to change in patients with OCD and therefore can be used to measure change with, and efficacy of, treatment in this disorder. This finding needs to be replicated in other diagnostic groups (e.g., body dysmorphic disorder, anorexia nervosa, psychotic depression, and other disorders characterized by delusional thinking or a range of delusionality).

In our experience, the scale can be easily administered to patients with other disorders that may be characterized by delusional thinking, such as olfactory reference syndrome, parasitosis, hypochondriasis, and anorexia nervosa. Although anorexia nervosa has been noted to be characterized by delusionality (36, 37), this aspect of the disorder has been virtually unstudied.

The scale can also be used with patients with schizophrenia who are able to focus on the interview and respond meaningfully to the questions. In our experience, however, patients with schizophrenia who have current formal thought disorder may not be able to respond meaningfully to the questions. Patients with delusions related to dementia may also be difficult to assess if cognitive impairment interferes with their understanding of the questions or their ability to respond to them in a meaningful way.

Another caveat concerning scale administration involves determining the underlying belief. Many patients with OCD have concerns about dreaded future events, fearing that if they do not perform a certain compulsion, a dreaded event will occur. When administering the scale, it is important to determine the belief underlying the obsession and to phrase the belief consistently and specifically—e.g., “How certain are you that you will [not might] get AIDS if you do not wash all of your food before eating it?”

One limitation of the scale is the occasional difficulty in obtaining responses pertaining to the past week from patients with a rapidly fluctuating clinical course, in whom the degree of delusional thinking can change even daily. We observed this phenomenon primarily in patients with mood disorder with psychotic features. In such cases, the scale is more useful clinically if current status (e.g., for the past day) is assessed rather than an average for the past week.

Several further limitations of the study should be noted. One of the two scales used to determine convergent validity (the Scale to Assess Unawareness of Mental Disorder) was not rated blind to the Brown Assessment of Beliefs ratings, which may have increased the resulting correlation. Furthermore, there is no perfect “gold standard” available to determine a cutoff point for the presence or absence of delusionality. Although the global “gold standard” ratings were made independently of ratings on the Brown Assessment of Beliefs Scale, they were based solely on the rater's clinical impression of the patient. Nonetheless, excellent sensitivity and specificity were obtained for patients with body dysmorphic disorder, whose global rater had expertise in delusionality. Further work is needed with larger samples of all three diagnostic groups with the use of expert independent ratings of delusionality to replicate and confirm this finding. In addition, the Brown Assessment of Beliefs Scale should be studied in larger samples, and severity of illness could be controlled for to ascertain that the scale is equally effective in patients with different degrees of illness severity.

In the meantime, the Brown Assessment of Beliefs Scale, a reliable and valid measure of delusionality, is easily administered and potentially useful to researchers and clinicians in assessing degree of delusionality and classifying patients with disorders characterized by delusions. It may also further understanding of the relationship of delusionality to prognosis and treatment response.

TABLE 1

Enlarge table

TABLE 2

Enlarge table

TABLE 3

Enlarge table