Concurrent and Predictive Validity of the Personality Disorder Diagnosis in Adolescent Inpatients

Despite common clinical use of the construct of personality disorder in adolescent patients (1–5), the meaning and validity of personality disorder in this group remains uncertain (6–12). DSM-III-R and DSM-IV both note that personality disorders begin in childhood or adolescence and are often recognizable by adolescence; therefore, they allow for the diagnosis of personality disorders in adolescents if the disturbance is pervasive and persistent and not likely limited to a developmental stage. Despite this tentative acceptance, the nature of personality disorders in child and adolescent patients—and the appropriateness of the construct of personality disorder in this younger population—remains a much-debated topic (3, 6, 8–14).

There have been two large-scale studies of the concurrent validity of the diagnosis of personality disorder in adolescents in community-based samples. The Toronto Adolescent Longitudinal Study (15) used DSM-III-R criteria to assess 72 subjects from a community sample at ages 13, 16, and 18 years. They found that adolescent personality disorders were associated with high levels of distress and impairment. Similarly, Bernstein et al. (16) used a large community sample to show that adolescents with personality disorders were more likely to have more social impairment, school problems, contacts with the police, and problems at work.

In recent years there has been increased research in adolescent personality disorders using clinical samples—a more relevant population for the assessment of personality pathology (17–47). However, most of these studies have examined borderline personality disorder exclusively (17–27) or have assessed personality disorders secondary to investigating other disorders (28–40). Overall, these studies have found adolescent personality disorders to be associated with elevated levels of depression, anxiety, anger, dissociation, cognitive distortions and impairment, poor self-concept, attempted and completed suicide, poor treatment response, and such personality traits as novelty-seeking, aggression, assaultiveness, neuroticism, low self-confidence, and hopelessness. In addition, these studies show that personality disorders occur at similar frequencies in adolescents and adults and that personality disorders aggregate in the families of adolescents with personality disorders. Limitations of these studies (e.g., small samples, lack of reliability, lack of comparison groups, cohort differences), however, restrict the extent to which meaningful conclusions can be drawn concerning the concurrent validity of the construct of personality disorder in adolescents.

Previous research in adolescents using community samples has suggested that, although personality disorders are prevalent, have concurrent validity, and are stable over 1–2-month periods (46), they are relatively unstable over long periods of time (15, 16). For example, Bernstein et al. (16) found that adolescent personality disorders did not persist over a 2-year interval. Similarly, adolescent personality disorders have been found to be unstable over long periods of time in inpatient samples (20, 27, 46). For example, in a study of 70 adolescent inpatients diagnosed by using structured clinical interviews and reassessed 2 years later, our own research group found low diagnostic stability for personality disorders (46). However, our research group (20) and Meijer et al. (23) found that a number of symptoms of borderline personality disorder were stable over 2-year and 3-year periods, respectively. Moreover, although adolescent personality disorders may be unstable, such diagnoses may nevertheless predict poor outcome or persistent psychopathology (4, 13).

In the present study, we extend our work on the validity of diagnosis in adolescent inpatients (20, 46). In our previous study, we examined stability over a 2-year period. In the current study, we investigate concurrent and predictive validity of personality disorders in the same adolescent study group by comparing subjects who met DSM-III-R criteria for one or more personality disorders with those who did not meet the criteria for any personality disorder. At baseline, we compared groups on clinician-rated and self-rated measures of psychiatric distress and dysfunction. At 2-year follow-up, groups were compared on the same measures—as well as on a functional assessment of work, school, social relations, family relations, alcohol and drug use, and psychiatric symptoms and treatment. To our knowledge, there have not been any studies assessing the concurrent validity or any prospective studies of the predictive validity of adolescent personality disorders using large clinical study groups.

METHOD

Subjects

Subjects were drawn from adolescent inpatients hospitalized at the Yale Psychiatric Institute between 1986 and 1990. At baseline evaluation, there were 165 such subjects, representing nearly all of the adolescent inpatient admissions during this time. Subjects ranged in age from 12 to 18 years (mean=15.5, SD=1.4). Seventy-two (44%) were female and 93 (56%) were male. All subjects were single, and most were middle class. Of the 165 subjects, 142 were eligible for the follow-up study. A detailed description of the study group and procedures for the Yale Psychiatric Institute Adolescent Follow-Up Study is provided elsewhere (46).

At the time that subjects were invited to participate in the study, written informed consent was obtained after all study procedures had been explained. If the subject was a minor at the time of recruitment, assent was obtained from the subject and consent was obtained from the subject’s parents.

Measures

The Personality Disorder Examination (48) is a semistructured diagnostic interview that assesses the presence or absence of all 11 DSM-III-R personality disorders. According to the manual (48), traits are considered present in adolescent subjects if the traits are pervasive and have persisted for at least 3 years. Previous research with the Personality Disorder Examination has found it useful in assessing personality pathology in adolescent subjects (20, 31–33, 44, 46).

The Global Assessment of Functioning Scale provides a single global rating of functioning and symptoms. We used a modified version of the Global Assessment of Functioning Scale included in DSM-III-R. In this version, scores range from a low of 1 (e.g., needs constant supervision, serious suicide attempt with clear intent and expectation of death) to a high of 90 (e.g., superior functioning in a wide range of activities, no symptoms).

The SCL-90-R (49) is a 90-item, self-report measure of clinical functioning tapping nine domains of distress and symptoms within the previous 7 days. Subjects rate items on a 5-point scale of distress ranging from 0 (not at all) to 4 (extremely). In addition to the nine symptom dimensions, the SCL-90-R yields three global indexes of distress: the global severity index (the grand total), the positive symptom total (number of symptoms rated 1 or higher), and the positive symptom distress index (grand total/positive symptom total). The global severity index is the best single indicator of current level of distress. We compared groups on raw scores because no standardized, T-score value norms exist for adolescent psychiatric inpatients. Raw scores were computed by summing each item on a factor and dividing by the number of items making up the factor. Thus, each factor score could range from 0 to 4.

The Functional Interview (50) is a 55-question, semistructured, hour-long interview that assesses functioning since discharge. Generally, for our patients the time frame assessed was the follow-up period (about 2 years), although some questions asked about functioning in the past month. Several domains of functioning are assessed by the Functional Interview, including living arrangements since discharge, educational and occupational functioning, social and familial relations, further psychiatric treatment and symptoms, alcohol and drug use, and legal difficulties. These domains were selected a priori by Edell and Dipietro (50), and initial validity for the Functional Interview has been reported elsewhere (51).

Procedures

Baseline assessment

At admission, all subjects received a systematic diagnostic evaluation, which included administration of the Personality Disorder Examination, Global Assessment of Functioning Scale, and SCL-90-R. Assessments were conducted by doctoral and master’s level clinicians who had been trained to a high level of reliability.

Personality disorder diagnoses were established by the best-estimate method. Besides the Personality Disorder Examination interview data, information was obtained from admission notes, medical records, and clinician descriptions. This method was in accordance with the LEAD (longitudinal, expert, all-data) standard advanced by Spitzer (52) and others (53). The results of the SCL-90-R were not used in making diagnoses, and the interviewers were blind to the scores.

We began administering the Personality Disorder Examination when it became available in 1988. Accordingly, it was given to approximately two-thirds of the subjects. Pairs of raters assessed the interrater reliability for 26 subjects; the kappas ranged from 0.65 (for paranoid personality disorder) to 1.00 (for histrionic, avoidant, and passive-aggressive personality disorders) (mean kappa=0.84). These reliabilities are similar to those reported in previous studies using adult subjects (53, 54).

Follow-up assessment

Follow-up assessments included the Personality Disorder Examination, Global Assessment of Functioning Scale, SCL-90-R, and the Functional Interview. These were administered by the clinical research team—none of whom was a baseline interviewer, and all of whom were blind to baseline diagnostic and symptom data. Members of the research team all had at least 1 year of clinical experience (average=3.5 years), and all received 40 hours of intensive training by a doctoral-level psychologist (W.S.E.) over a 4-week period. Weekly training meetings continued throughout the follow-up assessment period. As with the baseline assessments, a series of reliability ratings conducted along with the current Personality Disorder Examination interviews confirmed a high degree of ­interrater reliability of diagnosis. Reliabilities at follow-up were comparable to those at baseline. Pairs of raters also assessed the interrater reliability for 25 subjects for the Functional Interview and Global Assessment of Functioning Scale ratings. The intraclass correlations (ICCs) for the Functional Interview ranged from 0.92 (drug use) to 0.99 (employment level, legal difficulties, psychiatric symptoms, social/romantic relationships, and family relationships) (mean ICC=0.97). For the Global Assessment of Functioning Scale ratings, ICC=0.91 for current ratings and ICC=0.96 for past-year ratings.

RESULTS

Eighty-six (61%) of 142 patients were diagnosed with at least one personality disorder. Borderline personality disorder was the most frequent (N=71), and schizoid and obsessive-compulsive personality disorders were the least frequent (N=2 each). Antisocial personality disorder was not assessed because the vast majority of the subjects did not meet the requirement of being at least 18 years old. Comparisons of the groups with (N=86) and without (N=56) personality disorder showed no differences in demographic variables.

Concurrent Validity

As shown in Table 1, univariate tests at baseline indicate that adolescent inpatients with personality disorders were significantly more impaired than adolescent inpatients with no personality disorder in terms of current and past-year Global Assessment of Functioning Scale scores.

Table 1 also presents a comparison of the two groups’ SCL-90-R scores at baseline. Adolescents with personality disorders scored higher on the somatization, obsessive-compulsive, interpersonal sensitivity, depression, anxiety, hostility, and psychoticism scales—as well as on the global severity index, positive symptom total, and positive symptom distress index scales.

Predictive Validity

We performed a series of repeated measures analyses of variance (ANOVAs) on the Global Assessment of Functioning Scale ratings and the SCL-90-R scores, assessing the effect of the two groups (with versus without personality disorder), the two testing times (baseline versus follow-up), and the group-by-time interaction. In terms of personality disorder diagnosis, there was a main effect for the depression and hostility scores of the SCL-90-R: the adolescents with personality disorders scored significantly higher on both scales. In terms of baseline versus follow-up, there was a significant main effect for the current and past-year Global Assessment of Functioning Scale ratings as well as for the SCL-90-R interpersonal sensitivity, depression, anxiety, phobic anxiety, global severity index, and positive symptom total scales. Trends were also found for the hostility, paranoid ideation, and psychoticism scales of the SCL-90-R between baseline and follow-up. In terms of diagnosis-by-time interaction, a trend was found for both the depression and the positive symptom total scores: the adolescents with personality disorders had a 0.69-point drop on the depression scale and a 17-point drop on the positive symptom total index of the SCL-90-R, but the adolescents with no personality disorders did not have similar changes over time. Thus, over time, the scores of the adolescents with personality disorders became more similar to those of the adolescents with no personality disorders; however, only the changes in the depression and positive symptom total scales approached statistical significance.

A series of one-way (with versus without personality disorders) ANOVAs were performed on the Functional Interview composite scores. The means from these univariate tests are shown in Table 2. The adolescents with personality disorders showed significantly greater dysfunction in the areas of drug use and further psychiatric hospitalizations. There were no differences in functioning, however, with regard to current employment, alcohol abuse, legal difficulties, psychiatric symptoms, or social and family relationships.

DISCUSSION

We examined the concurrent and predictive validity of the personality disorder diagnosis in a large group of hospitalized adolescents by comparing subjects with and without personality disorders on various clinician-rated and self-rated measures of distress and dysfunction. Our study represents a refinement over previous studies in a number of respects. First, the use of a large study group ensured that there was adequate power to detect group effects. Second, we studied both boys and girls, allowing for better generalizability. Third, we used a structured interview with established reliability that applied the DSM-III-R criteria. Fourth, we used validated and reliable measures of impairment and distress (i.e., the Global Assessment of Functioning Scale and the SCL-90-R). Fifth, we included both patient-rated (SCL-90-R) and clinician-rated (Global Assessment of Functioning Scale and Functional Interview) data, assuring method variability. Finally, we examined an inpatient group, for whom the study of personality disorders is particularly relevant to clinical practice.

These advantages notwithstanding, our study group was heterogeneous, with high rates of multiple diagnoses; therefore, it is difficult to associate impairment and distress scores clearly with specific personality disorders. Nevertheless, diagnosing several personality disorders in the same subject is a practice that follows DSM-III-R (and DSM-IV) convention. Moreover, such a study group has ecological validity because it represents a clinical reality. Although we had a large total number of subjects, we did not have sufficient numbers to perform t tests for multiple specific diagnoses. Finally, the fact that our findings are based on an inpatient study group precludes extrapolation to nonhospitalized adolescents with personality disorders.

Concurrent Validity

In general, our findings support the concurrent validity of the diagnosis of personality disorder in adolescent inpatients. At baseline, adolescents with personality disorder diagnoses were more impaired according to their ratings on the Global Assessment of Functioning Scale and on all SCL-90-R scales except for phobic anxiety and paranoid ideation. These findings are consistent with those from studies involving community-based adolescent samples (15, 16). Our findings in adolescent inpatients are also generally consistent with those reported for diverse adult community and outpatient samples, indicating that personality disorders are related to functional impairment (55–58).

Predictive Validity

The predictive validity of the adolescent personality disorder construct was modest. We found that the DSM-III-R diagnosis of a personality disorder during adolescence did not predict greater impairment according to the Global Assessment of Functioning Scale or the SCL-90-R. Adolescent personality disorder diagnosis was related to later drug use and to future psychiatric hospitalization, however.

Personality disorder at baseline was associated with greater drug use at follow-up. The co-occurrence of personality disorders—especially borderline personality—and drug use has been well established in adults (59–61). It has generally been accepted that patients with personality disorders who abuse drugs do so as a function of their personality (62, 63). Personality disorders, especially cluster B personality disorders, are associated with deficits in affect regulation and with greater impulsivity. Drug users may be attempting to “self-medicate” by modulating affect through unpropitious means (64). Thus, drug use is a way of coping with an array of affects, simultaneously striving to regulate both arousal and negative mood states by artificially managing arousal and by creating positive affect. Although past studies have shown that personality disorders predate substance use, results from this study neither confirm nor disconfirm such findings. Conversely, some investigators have argued that primary problems with drug use create many of the symptoms that constitute personality disorders, or that a subgroup of patients with personality disorders may appear to have personality disorders because of behaviors associated with drug use (59). Our findings suggest that this is an important area for future research.

We also found that the diagnosis of a personality disorder predicted increased use of inpatient psychiatric services during a 2-year follow-up period. This finding offers clear support for the construct validity of the personality disorder diagnosis in adolescence and is consistent with the findings of Bernstein et al. (16), who reported that the personality disorder diagnosis is predictive of psychological distress and functional impairment at follow-up.

The general lack of predictive validity in our study, coupled with other studies that have found personality disorders in adolescents to lack longitudinal stability (16), indicates that personality disorders—at least DSM-III-R personality disorders—may reflect a point-in-time disturbance rather than chronic impairment in adolescents. Adolescence is a time of turbulence and flux, and acute stressors can provoke disruption and regression (20). It may be that symptoms and signs during periods of acute crisis do not reflect the enduring mode of functioning for the individual. During periods of acute crisis, patients may describe themselves as having elevated psychopathology relative to asymptomatic periods (65–66). However, there are a number of caveats to that interpretation of our data. It may be that our measures were not sensitive enough to detect subtle differences in functioning at follow-up. For instance, the Global Assessment of Functioning Scale is a relatively gross measure of functioning. Likewise, our follow-up Functional Interview may not have been sensitive to subtle differences in functioning in such ­areas as social relationships, family relationships, or psychiatric symptoms.

However, we found differences between patients with and without personality disorders in both drug use and further psychiatric hospitalizations, which are concrete constructs that are easily defined and measured. Functional assessments were based on self-­report, which are vulnerable to reports of pseudo well-being (67). Additionally, it may have been helpful to use multiple informants in our assessments because some investigators have noted that many patients do not adequately describe their typical personality because they do not possess sufficient insight into their own behavior (68). The literature for adults has also failed to demonstrate impressive stability (58).

Another issue is that ours was not a naturalistic study, in that subjects were drawn from a clinical population and received psychiatric treatment. Insofar as treatment is effective for patients with personality disorders (69), treatment variables may have confounded findings. Thus, some of the lack of predictive validity may be due to the beneficial effects of treatment. Data from clinical populations are necessary to advance the understanding of developmental psychopathology (70, 71); however, the lack of predictive validity in treated study groups cannot—without epidemiologic validation—be generalized to the population as a whole (72).

Another possibility is that alternative conceptualizations may be more useful to understanding adolescent personality disorders (1, 3) and that alternative assessment methods may prove useful as well. For example, dimensional approaches to the assessment of adolescent personality disorders may have more validity than categorical classification (73). Indeed, a number of studies have found better predictive validity in adolescents with disruptive behavior disorders when using dimensional approaches than when using categorical approaches (74).

A final issue that may affect the limited predictive validity of DSM-III-R adolescent personality disorders is DSM-III-R’s lack of attention to the developmental complexities of this stage (75, 76). Although DSM-III-R does note that personality disorders have an onset in adolescence or early adulthood, DSM-III-R personality disorder diagnoses rely on symptoms and signs originally designed for adults. Adolescent personality disorders may be phenomenologically similar to adult personality disorders (1), but it is unlikely that they would be completely isomorphic with adult criteria. Therefore, adolescent personality disorders may require a classification system that incorporates a developmental perspective and considers how personality disorders derive from disruptions in normal developmental processes (77). Examining this broader picture of developmental psychopathology in adolescence allows for the examination of heterotypic continuity and helps determine the construct validity of adolescent personality disorders (78).

CONCLUSIONS

Our findings suggest that, although DSM-III-R-defined personality disorders in adolescent inpatients have good concurrent validity, important questions remain concerning the predictive validity of such diagnoses in this population. Future research should consider broader conceptions of personality functioning and should incorporate a developmental perspective as well as dimensional approaches.

Presented in part at the 146th annual meeting of the American Psychiatric Association, San Francisco, May 22–27, 1993. Received Dec. 26, 1996; revisions received Dec. 17, 1998, and March 23, 1999; accepted April 13, 1999. From the Yale Psychiatric Institute and the Department of Psychiatry, Yale University School of Medicine, New Haven, Conn. Address reprint requests to Mr. Levy, Department of Psychology, New York Presbyterian Hospital-Weill Medical College, Cornell University, Westchester Division, 21 Bloomingdale Rd., White Plains, NY 10605; [email protected] (e-mail). The authors thank David Greenfeld for institutional support as well as Diana Bowling, Pamela Harding, Steven Joy, Helen Sayward, Rachel Yehuda, and Kathy Zampano for help in conducting the baseline assessments and Diane Bacon, Robert Cubeta, Eric Hancock, Terry Harran, Foluke Morris, and Jeffrey Muller for help in conducting follow-up assessments.

TABLE 1

Enlarge table

TABLE 2

Enlarge table