Validity of Substance Use Self-Reports in Dually Diagnosed Outpatients

Monitoring substance use in patients dually diagnosed with psychiatric illness and substance use disorder is critical for diagnosis and for evaluation of progress. However, Stone et al. (1) (in an emergency room setting) and Shaner et al. (2) (at a clinic administering depot neuroleptic medication) have reported widespread underreporting of drug use by patients with schizophrenia who had positive urine drug screens. These and other authors (1–3) have thus cautioned against relying heavily on self-reports of substance use in patients with schizophrenia.

We are aware of no research, however, that has specifically focused on the validity of substance use self-reports by nonpsychotic, dually diagnosed patients. We therefore investigated the validity of substance use self-reports among two different populations of dually diagnosed outpatients: one group with a mood disorder (bipolar disorder) and one with an anxiety disorder (posttraumatic stress disorder [PTSD]). Self-reports were compared to supervised urine samples obtained on the same day.

METHOD

Subjects were 55 outpatients participating in two separate studies, each designed to pilot test a new group psychotherapy for patients with coexisting substance use disorder and either 1) PTSD in women (N=16) or 2) bipolar disorder (N=15). Each therapy lasted 12 weeks; the bipolar disorder group met weekly, and the PTSD group met twice a week. The bipolar disorder/substance use disorder study also contained a comparison cohort of 24 subjects who received monthly evaluations but no group therapy. All subjects met DSM-IV criteria for substance dependence and either PTSD or bipolar disorder, as diagnosed with the Structured Clinical Interview for DSM-IV Axis I Disorders, Patient Edition (4). After complete description of the study to the subjects, written informed consent was obtained.

Observed urine samples were collected from therapy patients at every treatment session and at monthly evaluation visits from the bipolar comparison cohort. Samples were analyzed for drugs of abuse (excluding alcohol) at a laboratory that was certified by the National Institute on Drug Abuse and that used an enzyme multiplied immunoassay test screen and gas chromatography/mass spectrometry confirmation.

Self-report data for group psychotherapy patients were collected weekly during the 12-week period through use of a questionnaire that asked how many days specific drugs were used during the preceding week. Self-report data for the bipolar comparison cohort were obtained during subjects' monthly evaluations by asking how many days specific drugs were used during the preceding month. There were no negative consequences attached to reporting substance use, and subjects were repeatedly encouraged to be honest in their self-reports.

Percent agreement between self-report and urine data was calculated; instances of negative self-report in the face of a positive urine screen were then compared between groups.

RESULTS

The mean age of our subjects was 35.2 years (SD=7.9). All subjects in the PTSD group were women, by design, and 53.8% of the subjects in the bipolar disorder group were men. The PTSD group included no patients with combat-related disorders. Most of the patients were white (87.3%), unemployed (63.6%), and single (56.4%). Most patients (N=38, 69.1%) had both drug and alcohol dependence; 10 (18.2%) had only alcohol dependence, and seven (12.7%) were only drug dependent. Among the 45 subjects with drug dependence diagnoses, 43 (95.6%) were dependent on more than one drug; cannabis (N=16, 35.6%) and cocaine (N=12, 26.7%) were the most common primary drugs of abuse.

Self-report and urine data were collected on the same day in 275 instances. Overall, our results indicated highly valid self-report data. In only 13 instances (4.7% of all cases) did a subject deny using a drug that was detected by a urine screen. Five of the 13 instances were accounted for by two subjects who repeatedly denied substance use despite positive urine samples. Self-reports and urine results were consistent in 153 (55.6%) of 275 instances, with 34 instances of positive self-reports and urine tests and 119 matched negative results. Among the 122 inconsistent reports, 109 (89.3%) involved overreporting, whereby the subject reported substance use that was not detected by urine testing. Validity of self-report did not vary by specific drug.

The validity of self-report was even greater among subjects in group treatment. Only three (9.7%) of 31 subjects receiving group therapy—two with PTSD and one with bipolar disorder—ever denied known substance use. In contrast, seven (29.2%) of 24 subjects in the bipolar comparison cohort denied known substance use at some point. A comparison of subjects with bipolar disorder in group treatment and those receiving only monthly evaluations revealed less underreporting of substance use among the group therapy cohort (p=0.096, Fisher's exact test).

DISCUSSION

Our finding of highly valid self-reports in two different patient populations provides encouragement regarding the monitoring of substance use in nonpsychotic, dually diagnosed patients. The obvious advantages of self-reports (e.g., low cost, noninvasiveness, ease of administration) make them desirable for use with this population. Moreover, while supervised urine samples offer an objective means of monitoring substance use, they detect only recent use (5). Indeed, subjects in our study reported many instances of use that were not detected by urine tests administered on the same day.

Babor et al. (6) have pointed out that validity of self-reports can vary considerably, depending on the context in which the data are collected. Several characteristics of our design have been previously reported (5, 6) to improve the accuracy of self-reports: data were collected repeatedly over time; subjects knew the research personnel well; subjects had prior knowledge of urine tests; they faced no negative consequences for reporting substance use; and they were strongly encouraged to be honest in their self-reports. Our comparison of two populations of patients with bipolar disorder—one involved in group treatment and one receiving only monthly evaluations—suggests that participation in active treatment may also improve validity; this finding has encouraging implications for clinical populations.

It might be argued that differences in study protocols could have affected the validity of self-reports. For example, data from the PTSD, bipolar therapy, and bipolar comparison groups were obtained twice weekly, weekly, and monthly, respectively. Moreover, the comparison cohort was asked about drug use during the previous 1-month period, while the group therapy patients were asked about drug use during the prior 1-week period. It is thus possible that differences in self-report validity, e.g., between the two groups of bipolar patients, may reflect differences in the frequency of data collection. This is unlikely, however, because a positive urine screen reflects only recent drug use. Therefore, the comparatively lower validity of self-reports among subjects receiving monthly evaluations (i.e., those not in group therapy) is unlikely to have resulted from poor recall (e.g., forgetting about an episode of drug use that occurred several weeks earlier).

Although previous studies (1, 2) have demonstrated the limitations of self-reports in psychotic populations, particularly in acute care settings, our findings have shown that self-reports can be highly valid in nonpsychotic, dually diagnosed outpatients under the conditions specified in this study. The relative contributions of 1) the diagnostic makeup of our patients and 2) the conditions under which the data were collected (e.g., no adverse consequences of reporting drug use, prior knowledge of same-day urine screens) are unclear. Moreover, ours is a relatively small population, and it is not known to what extent our findings can be generalized to substance-dependent or dually diagnosed populations with other sociodemographic or clinical characteristics (e.g., patients with combat-related PTSD). However, our data support the importance of recognizing the heterogeneity of dually diagnosed patients (e.g., psychotic versus nonpsychotic) (7) with respect to monitoring substance use. Moreover, our findings emphasize the need to appreciate the context in which self-reports are gathered (e.g., whether or not they are linked to a negative consequence) when evaluating their validity.

Received Feb. 11, 1997; revision received June 2, 1997; accepted July 10, 1997. From the Alcohol and Drug Abuse Program, McLean Hospital; and the Department of Psychiatry, Harvard Medical School, Boston. Address reprint requests to Dr. Weiss, McLean Hospital, 115 Mill St., Belmont, MA 02178; [email protected] (e-mail). Supported by grants DA-09400, DA-08631, and DA-00326 from the National Institute on Drug Abuse; grant AA-09881 from the National Institute on Alcohol Abuse and Alcoholism; and a grant from the Dr. Ralph and Marian C. Falk Medical Research Trust.