Diagnosing Comorbid Psychiatric Disorders in Substance Abusers: Validity of the Spanish Versions of the Psychiatric Research Interview for Substance and Mental Disorders and the Structured Clinical Interview for DSM-IV

Diagnosing and treating comorbid psychiatric disorders in substance abusers have become increasingly important in the last 10 years. Many clinical and epidemiologic studies have revealed a high degree of co-occurrence of substance abuse and psychiatric disorders, a finding that may have important consequences from a health and social point of view (1). However, the identification of reliable and valid diagnoses of comorbid psychiatric disorders in substance abusers has long been problematic. The core problem is that of accurate diagnosis, which has two main difficulties. First, the acute or chronic effects of substance abuse can mimic symptoms of many other mental disorders, making it difficult to differentiate psychiatric symptoms that represent a primary disorder from those representing the effects of acute or chronic substance use or withdrawal. Second, psychiatric diagnoses are defined by their manifestations rather than by direct biological markers. Furthermore, it has been shown that most of the diagnostic criteria used for diagnosing psychiatric disorders offer little specific guidance for determining other psychiatric diagnoses from the clinical records of patients affected by drug abuse/dependence. Studies using Research Diagnostic Criteria (2), DSM-III criteria, and DSM-III-R criteria, even with structured diagnostic instruments such as the Schedule for Affective Disorders and Schizophrenia—Lifetime Version (3), the National Institute of Mental Health Diagnostic Interview Schedule (4), and the Structured Clinical Interview for DSM-III-R (SCID) (5), have shown poor reliability and validity for mood and anxiety disorders in substance abusers (6–11).

In response to the increasing recognition of the relevance of comorbid psychiatric disorders in drug users, DSM-IV placed more emphasis on comorbidity, replacing the dichotomous terms “organic” versus “nonorganic” to distinguish three categories: “primary,” “substance-induced,” and “expected effects” of the substances. In contrast to previous diagnostic systems, DSM-IV provides guidelines for distinguishing psychiatric symptoms in ongoing substance use. In consequence, it can be hypothesized that the use of DSM-IV criteria, through structured diagnostic interview, would be the best strategy to increase the validity of comorbid psychiatric diagnoses in substance abusers.

At the present time, two interviews for psychiatric diagnosis based on DSM-IV criteria have been developed: the DSM-IV versions of the SCID (12, 13) and of the Psychiatric Research Interview for Substance and Mental Disorders (PRISM) (14). The SCID is a semistructured interview that allows diagnosis of primary or substance-induced disorders but provides no more specific guidelines than are stated in the criteria. Reliability data for the SCID in substance-abusing samples are not yet available. The PRISM is a structured interview developed in response to the lack of a diagnostic interview suitable for comorbidity research. Three important characteristics of the PRISM that are specific to comorbidity include 1) adding specific rating guidelines throughout the interview, including frequency and duration requirements for symptoms, explicit exclusion criteria, and decision rules for frequent sources of uncertainty; 2) positioning of the alcohol and drug sections of the PRISM near the beginning of the interview, before the mental disorder sections, so that the history of alcohol and drug use is available at the time of beginning the assessment of mental disorders; and 3) more structured alcohol and drug histories to provide a context for assessing comorbid psychiatric disorders. The first version of PRISM, based on DSM-III-R criteria, showed good to excellent reliability for many diagnoses, including affective disorders, substance use disorders, eating disorders, some anxiety disorders, and psychotic symptoms (15). To address the changes in DSM-IV, the PRISM has been updated and revised to provide diagnoses of primary and substance-induced disorders and to include the expected effects of intoxication or withdrawal. In addition, the revised version of the PRISM provides a method for operationalizing the term “in excess of” the expected effects of substance in chronic substance abusers (16).

In the absence of suitable standardized measures against which to examine the validity of psychiatric diagnoses (i.e., a “gold standard”), the longitudinal evaluation performed by an expert, using all data available (the longitudinal, expert, all data [LEAD] procedure) was conceived as a standard for validating psychiatric diagnoses (17). The LEAD procedure has been employed as a criterion for the assessment of the procedural validity of diagnostic instruments. Nevertheless, the reliability and validity of the LEAD procedure for diagnosing some psychiatric comorbidity according to DSM-III-R criteria have been questioned in previous studies (18, 19).

We hypothesized that, in substance users, the PRISM based on DSM-IV criteria will allow clinicians to obtain psychiatric diagnoses with more concordance to diagnoses based on the LEAD procedure than the SCID in primary versus substance-induced disorders. The aim of the present study was to assess the validity of DSM-IV diagnoses obtained by two diagnostic instruments, the PRISM and SCID, compared with diagnoses based on the LEAD procedure in patients with substance use disorders.

Method

Subjects

A total of 105 patients were recruited from an inpatient (N=85) or outpatient (N=20) program at the Drug Abuse Treatment Center of a 350-bed acute-care teaching hospital in the city of Barcelona, Spain. The subjects were predominately male (N=72, 69%). The mean age of the subjects was 33.3 years (SD=7.7). Most of the patients (N=81, 77%) were living with their families, and 60 patients (57%) were unemployed. With regard to serologic status, 30 (29%) were infected by the human immunodeficiency virus, 42 (40%) by the hepatitis B virus, and 58 (55%) by the hepatitis C virus.

Because the present study was focused on methodological issues and was not designed to address prevalence rates of comorbid psychiatric disorders, the study was conducted with subjects drawn from a group selected according to the patients’ availability during their first 15 days of treatment. After full explanation of the purposes of the study, written informed consent was obtained from all participants. The study protocol was approved by the Ethical Committee of Clinical Trials of the hospital. Exclusion criteria were severe cognitive impairment or a medical disorder that would interfere with the research assessments. Patients were interviewed after their acute withdrawal symptoms subsided.

Measures and Procedures

All patients were diagnosed according to DSM-IV criteria by using Spanish versions of the PRISM, the SCID for axis I disorders, clinical version (20), and the SCID for axis II personality disorders (SCID-II) (21). All patients were also diagnosed by using the LEAD procedure. Diagnoses were described in two time frames: “current” (criteria were met within the past year) and “past” (criteria were met before the previous 12 months).

The Spanish version of the PRISM was obtained after translation of the original version (provided by the authors of PRISM) by a bilingual research translator (whose native language was Spanish) and back-translated by one experienced bilingual research translator (whose native language was English). Following the back-translation, both the Spanish version and the back-translation were sent to the original authors for revision. The final version was the result of corrections for inconsistencies and problems detected by the research team.

The PRISM includes the following disorders: 1) substance use disorders, including substance abuse and dependence for alcohol, cannabis, hallucinogens, licit and illicit opiates, and stimulants; 2) primary affective disorders, including major depression, manic episode (and bipolar I disorder), psychotic mood disorder, hypomanic episode (and bipolar II disorder), dysthymia, and cyclothymic disorder; 3) primary anxiety disorders, including panic, simple phobia, social phobia, agoraphobia, obsessive-compulsive disorder, generalized anxiety disorder, and posttraumatic stress disorder; 4) primary psychotic disorders, including schizophrenia, schizoaffective disorder, schizophreniform disorder, delusional disorder, and psychotic disorder not otherwise specified; 5) eating disorders, including anorexia, bulimia, and binge-eating disorder; 6) personality disorders, including antisocial and borderline personality disorders; and 7) substance-induced disorders, including major depression, mania, dysthymia, psychosis, panic disorder, and generalized anxiety disorder.

The Spanish version of the PRISM was administered in all cases by two research interviewers with a master’s degree and clinical experience with patients with substance abuse or mental disorders (D.S. and M.A.). The interviewers conducted a series of practice interviews together to enhance interrater reliability before beginning their interviews with patients.

We used the Spanish version of the SCID and SCID-II diagnoses (20, 21). The SCID-II involved a two-stage process in which the patient initially completed a self-report questionnaire followed by a detailed clinician inquiry cued by positive responses. The SCID is the semistructured interview more widely used for psychiatric diagnoses in psychiatric population studies. All of the Spanish versions of the SCID and SCID-II were administered by an independent research interviewer experienced in the use of these instruments for mental disorders (G.P.D.) who was blind in respect to diagnoses obtained by the PRISM instrument.

The LEAD procedure diagnosis was conducted by two clinical experts, both psychiatrists, who were experienced in the diagnosis and treatment of psychoactive substance use and other psychiatric disorders (M.T. and R.M.S.). Each patient’s attending psychiatrist served as the expert for that patient. At the beginning of the study, LEAD experts were informed that at the end of assessment they would give all the DSM-IV diagnosis of each patient in two time frames: current (within the past year) and past (before the previous 12 months). To develop the LEAD diagnoses, the experts used the following material: their own clinical interviews during the admission period (interview at admission day, complete standard clinical interview of the department, and daily clinical contact with the patient), the patient’s complete clinical record (including evaluations conducted by staff nurses, evaluations by medical consultants, laboratory results, and, when available, medical records from previous treatment in both inpatient and outpatient settings), and data provided from a family informant about each patient, when possible. The clinical interview was not structured and provided only general guidelines for assessing all clinical areas (i.e., sociodemographic features, family and personal background, employment, legal status, and toxicological, psychopathological, somatic, and psychiatric examination). At the end of the procedure, the experts had to fill in a questionnaire about the presence or absence of all DSM-IV diagnoses listed (current or past) in each patient. The LEAD experts were blind to diagnoses obtained by the PRISM and SCID.

After a mean of 6 days after admission at the Drug Abuse Treatment Center, patients were assessed with the Spanish version of the PRISM and, a mean of 3.9 days later (SD=4.9), with the Spanish version of SCID and SCID-II. The LEAD procedure was conducted at the end of treatment period, after a mean of 13.8 days of admission (SD=5.8).

Although most psychiatric diagnoses were assessed, analysis was done only when the diagnosis was made by all three systems. These included borderline personality disorder and antisocial personality disorder.

Data Analysis Strategy

For each diagnosis assessed by LEAD as the gold standard, the probability of observing an equal or greater number of patients (or alternatively equal or smaller number of patients) with the diagnosis than obtained by the PRISM or SCID was calculated by the McNemar’s test based on binomial distribution (df=1). The cutoff p value for statistical significance was 0.02 (two-tailed) according to Bonferroni’s method (22). The agreement between diagnoses in each pair of assessments was calculated by using the kappa statistic and the 95% confidence interval (23). Values range from 1.00 (perfect agreement) to –1.00 (total disagreement). Kappa values of 0.75 and higher indicate excellent agreement, values from 0.65 to 0.74 indicate good agreement, values of 0.40 to 0.64 indicate fair agreement, and values of 0.39 or lower indicate poor agreement (24). Kappa values were calculated by using two-by-two comparisons of each diagnosis obtained by the different methods, then pairs of diagnoses obtained with PRISM and LEAD (and SCID and LEAD). Only disorders with a prevalence lower than 10/probability of positive response or 10/(1–probability of positive response) were considered to have a reasonable sample size to calculate kappa values (25). For concordance in group diagnosis (i.e., affective disorders) when one patient had more than one affective diagnosis, only one diagnosis was taken into account to calculate the kappa index among the three instruments. Subjects could be counted in more than one diagnostic category if multiple diagnoses were given.

The nonparametric bootstrapping with 1,000 replicates to obtain 95% confidence intervals (bias-corrected) was used to calculate the significance of the difference between the kappas (PRISM-LEAD and SCID-LEAD) (26). The difference was significant at the 5% level if zero was not included in the 95% confidence interval.

McNemar’s test and the kappa statistics with 95% confidence intervals were analyzed with the SPSS 10.0 statistical software package (Chicago, SPSS, Inc.). Bootstrapping was carried out with the STATA 7 program (College Station, Tex., Stata Corp.).

Results

Diagnoses of Substance Use Disorders

The prevalence of current and past diagnoses of substance abuse and dependence obtained by using the PRISM, SCID, and LEAD instruments was fairly similar. The only statistically significant difference found was related to the diagnosis of past cannabis abuse; the SCID diagnosed fewer patients than the LEAD procedure (Table 1).

Overall, concordances of PRISM, SCID, and LEAD instruments in current and past substance use dependence disorders were good or excellent. However, the PRISM and LEAD showed significantly better concordance in current cannabis and cocaine dependence, as well as past alcohol abuse and dependence, than the SCID and LEAD (Table 2). Because of the small number of subjects, a few abuse diagnoses were studied, but the kappas found between diagnostic tools were poor to fair (Table 2).

Diagnoses of Primary, Non-Substance-Use Psychiatric Disorders

Significant differences in the percentages of current and past diagnoses of affective disorders were observed; the PRISM and SCID diagnosed a larger number of patients than the LEAD procedure (Table 1). The PRISM detected a similar number of cases of primary major depression as the LEAD procedure, and the SCID diagnosed a similar number of cases of substance-induced depression but more cases of primary major depression than the LEAD procedure (Table 1). The concordance for any current affective disorders was fair between PRISM and LEAD and poor between SCID and LEAD; the systems differed in the number of patients diagnosed as having primary major depression (Table 2). Concordances for past affective disorders were fair between both PRISM-LEAD and SCID-LEAD pairs, although concordance was better between PRISM and LEAD than between SCID and LEAD in cases of substance-induced depression.

Despite the fact that there were no differences in the rates of current psychotic diagnoses, more diagnoses of past psychosis were obtained with the SCID than with LEAD. For current and past psychotic disorders, the concordance between the PRISM and LEAD was excellent to good and significantly better than concordance between SCID and LEAD (Table 2).

More diagnoses of current and past anxiety disorders were found with the PRISM or SCID than the LEAD procedure; both instruments tended to diagnose more simple phobia, but there were no significant differences for any specific anxiety disorders. The concordance obtained was excellent to good for any anxiety diagnoses and panic with or without agoraphobia. Although no significant differences among the PRISM, SCID, and LEAD were found in regard to personality disorder diagnoses, the SCID tended to diagnose fewer patients as having antisocial personality disorder than the PRISM or the LEAD procedure. In both personality diagnoses, concordances between PRISM and LEAD were good, whereas kappa values for the concordance for these diagnoses between the SCID and LEAD were fair and poor (Table 2).

Discussion

Using the LEAD procedure as a gold standard, we found that the Spanish version of the PRISM seems to be a better structured interview for diagnosing comorbid psychiatric disorders than the Spanish version of the SCID in a group of drug abusers. The main differences are related to major depression, substance-induced psychosis, and borderline personality disorders.

The finding that fewer diagnoses of current and past affective and anxiety diagnoses were obtained with the LEAD procedure than with the PRISM and SCID instruments is probably related to the structure of the PRISM and SCID as opposed to the less structured clinical assessment. In the case of current major depression, the number of diagnoses obtained with the PRISM and the LEAD procedure were very similar (14% and 11%, respectively) with good concordance (kappa=0.68). Since these figures are better than those obtained for the SCID (24% of major depression and kappa=0.28), it can be suggested that the PRISM is a better instrument than the SCID to diagnose current episodes of major depression in drug abusers. For substance-induced depression diagnoses, some considerations should be commented on. First, the higher number of diagnoses obtained with the PRISM in respect to the LEAD procedure and the SCID may be explained as an overdiagnosing effect of the PRISM. Alternatively, the expert, in the absence of specific rating guidelines, might tend to assess the depressive symptoms as expected effects of chronic substance use and, as a consequence, might underdiagnose the substance-induced depressive disorders. Finally, since similar rates of depressive disorders (substance-induced plus primary) were found with the SCID and PRISM, but the SCID obtained more diagnoses of major depression and fewer diagnoses of substance-induced depression than the PRISM, it can be argued that major depression may be overdiagnosed by the SCID in drug users rather than substance-induced depression underdiagnosed by the SCID. Taking into account the good reliability in mood disorders previously described by the PRISM (15), we suggest that future studies of test-retest reliability with the SCID and LEAD procedure will be useful to clarify this point.

With regard to psychotic diagnosis and the limitation of the number of patients analyzed, our findings suggest that the PRISM is a better instrument than the SCID for diagnosing substance-induced psychotic disorders in substance users and that it should be used for differential diagnosis. The finding that the concordance in anxiety disorders between the PRISM and the LEAD procedure was good is in contrast to previous studies using DSM-III-R criteria obtained by the SCID (11), in which anxiety disorders showed poor validity. Furthermore, in respect to diagnoses of panic disorders (with or without agoraphobia), the concordance between the LEAD procedure and each of the other procedures was good. The difference between rates of current and past anxiety disorders seems to be related to simple phobia diagnoses. Simple phobia disorders are the most frequently overlooked diagnoses by clinicians not using structured interviews when patients come for other psychiatric disorders (27); these diagnoses are better recognized by interviews like the PRISM and SCID because of their structured design.

The PRISM and the LEAD procedure were more similar in diagnosing antisocial and borderline personality disorders than the SCID-II. Differences might be due to differences among the three procedures in methodology in obtaining information. The PRISM, like the LEAD procedure, is administered face-to-face with the patient, but the SCID-II involves a two-stage process in which the patient initially completes a self-report questionnaire followed by a detailed clinician inquiry cued by positive responses; this filtering process might explain the divergences observed.

Finally, in terms of substance use disorder diagnoses, the three diagnostic systems showed good or excellent agreement with DSM-IV for most dependence diagnoses except for cannabis, although the small number of patients with a diagnosis of cannabis dependence limits this result. Low concordance for the DSM-IV diagnosis of cannabis dependence has been previously described with other instruments (the Alcohol Use Disorder and Associated Disabilities Interview Schedule—Alcohol/Drug—Revised, Composite International Diagnostic Interview, and Schedules for Clinical Assessment in Neuropsychiatry) (28). Differences in diagnoses of cannabis dependence may be attributable to the nature of the study group and/or the nature of cannabis problems. Participants in this study were in treatment programs mainly for alcohol, heroin, and cocaine problems and, therefore, were not primarily cannabis users. The validity of the cannabis dependence diagnosis obtained with the three methods might be better tested in individuals who seek treatment for cannabis use.

In respect to abuse disorders, the concordances between the PRISM and LEAD and between the SCID and LEAD showed poor to fair agreement for alcohol, cannabis, and hallucinogen abuse. This low agreement on abuse diagnoses have been described in other studies (29–31) and might indicate that conceptualization and operationalization have not yet reached a satisfactory level even with the new diagnostic classification in DSM-IV. This could be attributable to the fact that, although these categories are still not well-defined conditions, they are not merely transitional states between extended use and dependence (32). Additionally, it may be caused by the subjectivity of the criteria of harm or hazard to daily life, in contrast to the more “biological” criteria of withdrawal, tolerance, and craving.

Some limitations of the study have to be taken into account before generalization of the results. The first limitation is related to the LEAD procedure as a gold standard for validation diagnosis. In previous studies using DSM-III-R criteria (18), it was shown that the LEAD procedure yielded good test-retest reliability between experienced clinicians for most psychoactive substance use disorders but that reliability was lower for major depression and anxiety disorders. Later, evaluating the validity of the LEAD procedure according to DSM-III-R criteria in drug abusers, it was found that, overall, LEAD-derived substance use diagnoses showed good concurrent, discriminant, and predictive validity. The validity of comorbid diagnoses obtained using the LEAD procedure was poor for anxiety disorders, fair for borderline personality disorder, and good for major depression and antisocial personality disorder (19). As far as we are aware, no previous studies diagnosing non-substance-use psychiatric diagnoses using DSM-IV have been carried out.

A second limitation is the comparatively small number of patients given some diagnoses and their recruitment primarily from an inpatient substance abuse treatment program, limiting the extent to which these results can be generalized to other patient groups. Another limitation associated with the use of only one investigator for SCID interviews is the possibility that findings may be partly a function of an interviewer effect. Finally, differences in patients’ reported information in different interviews might be considered a possible source of disagreement among the three diagnostic systems used.

In conclusion, the most relevant finding of the present study is that the PRISM seems to be a better instrument than the SCID for diagnosing primary major depression, substance-induced psychotic disorders, and borderline personality disorders in substance-abusing patients, whereas these interviews are similar in diagnosing anxiety disorders in these patients. Other approaches to the validation of psychiatric diagnoses include genetics and other biological markers, and we should still look for better diagnostic criteria in patients with dual diagnoses. The use of the PRISM might improve the characterization of subjects in future epidemiologic studies and clinical trials for therapeutic approaches.

TABLE 1

Enlarge table

TABLE 2

Enlarge table