Sources of Diagnostic Uncertainty for Chronically Psychotic Cocaine Abusers

Successful management of patients with co-occurring severe mental illness and substance abuse or dependence requires accurate diagnosis of both the psychiatric and the substance use disorders. Although the diagnosis of substance use disorders in this population has recently received considerable attention (1), the assessment of mental illness has not (2).

Diagnosis of patients with both substance abuse and psychiatric symptoms may be inaccurate or uncertain because abuse of alcohol and drugs can cause a wide range of psychological symptoms and syndromes (3,4). To make an accurate diagnosis in the presence of substance abuse, one must determine what role abused substances have played in the initiation and maintenance of psychological symptoms. This determination depends on the answers to four questions: What drugs has the patient used? What symptoms can these drugs cause? In what way can these drugs alter the symptoms of pre-existing psychiatric disorders? When did symptoms occur relative to drug use?

Unfortunately, this information is often missing or inaccurate, for two reasons. First, patients often do not accurately report substance use (5,6), and, second, little is known about how drugs may mimic or alter psychiatric disorders. These factors create a "catch-22" in which the research needed to develop this knowledge depends on accurate diagnoses that we cannot make until the results of that research are known.

Diagnostic uncertainty is especially problematic among patients who abuse stimulants and have psychotic syndromes resembling schizophrenia. First, stimulants can cause psychotic symptoms in individuals without a history of psychosis. Stimulant abusers frequently develop psychotic symptoms (7,8) that can be reproduced by experimental administration of stimulants (9,10,11). Even among amphetamine abusers who have never been psychotic, large doses of stimulants can cause acute psychotic symptoms strikingly similar to those of schizophrenia (12,13,14,15). Finally, chronic stimulant abuse or dependence, particularly involving amphetamine, may produce a chronic psychosis indistinguishable from schizophrenia that persists long after the patient has stopped using stimulants (16,17).

Second, stimulants can alter the symptoms of schizophrenia. Experimental doses of stimulants, including caffeine, can intensify schizophrenic psychotic symptoms for several hours despite antipsychotic medication (18,19,20). Although most reports show that stimulants worsen schizophrenia, laboratory studies (21,22,23,24) and the subjective experiences of people with schizophrenia (25) suggest that stimulants may sometimes improve schizophrenic symptoms, including psychosis. Thus the effect of stimulants on schizophrenic symptoms is heterogeneous and varies with the symptoms examined, the state of the illness, and the presence of antipsychotic medications. The effect might also vary with individual vulnerability to psychosis, the type of stimulant, the duration of use, the route of administration, and interactions with other abused substances, such as the interaction of cocaine and alcohol (26,27).

Because stimulants can mimic and alter schizophrenic symptoms, stimulant abuse can lead to diagnostic error—either a false positive or a false negative diagnosis of schizophrenia. Such diagnostic errors could lead to inappropriate treatment. On the one hand, a false positive diagnosis of schizophrenia for a patient who actually has a substance-induced psychosis could lead to inappropriately prolonged use of antipsychotic medication. On the other hand, a false negative diagnosis of schizophrenia could lead to a failure to use antipsychotic medication. Diagnostic uncertainty and inaccuracy can also lead to exclusion from appropriate treatment programs. For example, substance abuse treatment programs often accept patients with transient toxic psychoses but exclude those with schizophrenia or uncertain diagnoses.

Although there are many reasons to suspect that comorbid substance abuse and dependence will make diagnosis difficult (2), it is not clear how frequently such problems arise during careful diagnostic evaluation of psychotic substance abusers. The purpose of this study was to determine the frequency and sources of uncertainty in the diagnostic evaluation of chronically psychotic cocaine abusers.

Methods

Subjects

Subjects were recruited on admission to a large, urban Veterans Affairs medical center. Data were drawn from a study, conducted between 1990 and 1995, that compared the efficacy of two experimental treatment programs for cocaine abusers with schizophrenia. One program used case managers to coordinate separate psychiatric and substance abuse services. In the other program, a new treatment unit provided comprehensive integrated services for both disorders. Patients were eligible for the study if the admitting psychiatrist concluded that the patient required psychiatric hospitalization, had a psychotic disorder that might be schizophrenia, and had abused cocaine within the past six months.

The study was approved by the medical center's institutional review board, and all subjects gave informed consent. All patients were hospitalized initially. Each experimental program then attempted to maintain patients in ambulatory care, but success varied. By the 18-month follow-up assessment, patients had received varying durations and intensities of both inpatient and outpatient care.

Altogether, 221 patients were referred to the study by admitting psychiatrists. Forty-nine did not meet study criteria: 13 were too dangerous, ten never had psychotic symptoms, eight never abused stimulants, six had no phone or mailing address, four could not read the consent form, two were too psychotic to consent, two could not participate because it would conflict with current treatment, two planned to move out of the state, one was participating in a conflicting research project, and one had medical problems that were too severe. Seven patients who met study criteria refused to provide consent. Thus a total of 165 subjects completed the initial assessment.

All subjects were males with a mean±SD age of 40±6.4 years. A total of 129 (78 percent) were African American, 28 (17 percent) were Caucasian, and eight (5 percent) were of other races. The mean±SD level of education was 13±1.8 years. Sixty-eight patients (41 percent) had never married, 18 (11 percent) were currently married, and 79 (48 percent) were separated, widowed, or divorced. Seventy-three (44 percent) were homeless.

On average, they had used cocaine regularly for nine years (SD=6.7) and had used cocaine on 13±11.7 days of the 30 days before study entry. A total of 160 patients (97 percent) met DSM-III-R criteria for current cocaine abuse or dependence, 86 (52 percent) for alcohol, 58 (35 percent) for marijuana, 21 (13 percent) for opiates, 17 (10 percent) for sedatives, 12 (7 percent) for amphetamine, and seven (4 percent) for other substances. The mean number of psychiatric hospitalizations for the groups was ten (SD=11).

Thus these patients were severely mentally ill and chronically cocaine dependent individuals. They were characterized by homelessness, many psychiatric hospitalizations, a long duration of cocaine use, and recent extensive use of cocaine as well as other drugs and alcohol.

Diagnostic assessment

All subjects were assessed using the Structured Clinical Interview for DSM-III-R (SCID-R) for axis I disorders (28), supplemented whenever possible by hospital records and by interviews with family members, friends, and other persons familiar with the patient (collateral interviews). The project was affiliated with the UCLA Clinical Research Center for Schizophrenia and Psychiatric Rehabilitation, which is sponsored by the National Institute of Mental Health. The center maintains reliability of measurement across many projects through standardized training and quality assurance.

Using consensus ratings made by the center's senior diagnosticians as "gold-standard" criteria, five master's-level psychologists were trained to high levels of interrater reliability in the use of the SCID-R (overall kappa of at least .75 for symptoms and 100 percent agreement on diagnosis). Our use of the kappa statistic as a measure of association follows Kraemer's recommendation (29) that chance-corrected indexes of association are, in general, more interpretable and meaningful. When using gold standards, Kraemer identifies a family of weighted kappa statistics and notes that the conventional, so-called Cohen's kappa (30) is simply the statistic that derives when false positive and false negative errors are weighted equally.

One statistical feature of our weighted kappa is atypical, namely, that the separate units of observation are not independent subjects but rather items from a scale administered to a single individual. Because separate items within a scale are not independent, conventional significance testing of these weighted kappas is not appropriate. However, we use them only descriptively, and they are not tested for statistical significance. The nonindependence of observations does nothing to compromise the statistic as a descriptive index of the degree of chance-corrected association.

The diagnostic interviewers' initial results were discussed in a weekly case conference involving two senior diagnosticians (the first and third authors). The sources of diagnostic uncertainty were discussed in an effort to identify key questions that may not have been asked or new sources of information. Based on the conference, the diagnostic interviewer collected new information and presented the case again until no further sources of diagnostic information could be identified. This strategy, together with the reassessment described below, is in conformance with the "LEAD standard" proposed by Spitzer (31). LEAD is an acronym for Longitudinal, Expert and All Data and refers to criterion-based diagnoses made by experts reaching consensus after examining data from a variety of sources and from repeated assessments over considerable time.

Family members or other collaterals were available for interviews in about a quarter of the cases but were rarely able to provide key information, such as whether the patient's psychotic symptoms persisted during abstinence. Medical records were available in nearly three-quarters of cases and occasionally provided important diagnostic information, such as previous incoherence in a patient who reported a nonbizarre delusion as the only psychotic symptom.

The scoring of the SCID-R was modified to allow interviewers to record the source of uncertainty on any item. A list of potential sources of uncertainty was developed. The list included insufficient periods of abstinence, inconsistent responses, poor memory, and disorganized speech, among other sources. If uncertainty on individual items led to uncertainty at decision points in the SCID-R algorithm, interviewers completed questioning along each of the branches from the decision point.

For example, on a number of individual items, the unmodified SCID-R requires that a psychotic symptom be counted toward the diagnosis of schizophrenia only if it is judged to be "not organic." The modified scoring allowed the interviewer to score the symptom in one of three ways: definitely organic, uncertain organicity, or definitely not organic. If organicity was uncertain, the sources of uncertainty were recorded, such as an inconsistent history or insufficient periods of abstinence. Similarly, substance abuse and dependence can raise concern that the mood syndromes, chronicity, and deterioration in functioning may have an organic basis. These criteria were also operationalized to allow for recording uncertainty. (A list of these criteria and their operational characteristics is available from the first author.)

In cases of diagnostic uncertainty, these modifications of SCID-R scoring led to a list of possible diagnoses (differential diagnosis) and a list of sources of uncertainty. The sources of uncertainty were keyed to the criteria for schizophrenia. For example, in DSM-III-R the organic exclusion criterion for schizophrenia is met if "it cannot be established that an organic factor initiated and maintained the disorder" (32). For this study, the statement was operationalized with separate criteria for determining whether psychotomimetic substances initiated or maintained the disorder.

For example, a subject reported that persecutory delusions had first started during a period of amphetamine abuse. This temporal relationship suggested that amphetamine might have initiated the delusions, but it did not provide enough information to conclude that amphetamine was an organic etiologic factor. The close temporal relationship might have been coincidental, or the amphetamine might have precipitated a psychotic disorder that then maintained itself. The interviewer then asked about the persistence of the symptom in the absence of amphetamine use. The interviewer would have concluded that amphetamine was etiologic if both of the following had been true: first, that at least twice, persecutory delusions resolved within six weeks of stopping amphetamine use, and, second, that persecutory delusions had never occurred more than six weeks after stopping amphetamine use.

Conversely, the interviewer would have concluded that persecutory delusions were not due to amphetamine if the subject reported that delusions had occurred more than six weeks after stopping amphetamine use. We required two instances of resolution within six weeks to avoid ascribing etiologic significance to chance coincidence of drug use and psychosis. However, in the example discussed here, the subject had been using amphetamine for several years and had never stopped using the drug for more than three weeks. Although persecutory delusions were less prominent during these brief periods of abstinence, they did not resolve.

Thus the interviewer noted that persecutory delusions were present, that it was uncertain whether they were organic, and that the source of the uncertainty was insufficient periods of abstinence. Rating persecutory delusions as present but their organic basis as uncertain resulted in a differential diagnosis that included, among others, amphetamine delusional disorder and schizophrenia. Similar criteria were developed to specify the relationship between abused substances and three other criteria for schizophrenia±_chronicity, deterioration, and mood syndromes.

This project was designed and conducted before publication of DSM-IV(33). However, early drafts from the DSM-IV work group suggested that symptoms should not be considered drug induced if they persisted for more than six weeks after cessation of drug use. Consequently, we used the same time frame in our criteria. In the final version of DSM-IV this time frame was reduced to four weeks.

Assessment of drug abuse

Lifetime and current drug use were assessed using the SCID-R section on substance abuse and dependence and the Addiction Severity Index (ASI) (34). The ASI is a comprehensive structured interview used widely in substance abuse research, and its validity and reliability have been established among mentally ill substance abusers (35,36). In addition. results of urine toxicology tests obtained at study entry were used as evidence of substance use even if subjects denied using the detected substances.

Diagnostic reassessment

Eighteen months after study entry, each subject for whom the original diagnosis was uncertain was reassessed. An attempt was made to resolve SCID-R items originally coded as uncertain. Diagnostic information was gathered through interviews with subjects and from medical records and collateral interviews whenever possible. Medical records often included many notes by psychiatrists, psychologists, nurses, social workers, and case managers in the two experimental programs. Reported abstinence was verified using urine toxicology tests. These data were reviewed with the two senior diagnosticians as described above until no further sources of diagnostic information could be identified.

Of the 165 patients who consented and completed initial assessments, 27 were not available for the 18-month reassessment: 17 could not be found, six had died, two had been dropped from the study because they had become too dangerous, one was not eligible for VA care, and one elected to drop out.

Data tabulation

Cases were classified according to whether a definitive diagnosis was made or diagnosis was uncertain. The frequency with which each diagnosis appeared in the differential diagnoses of the uncertain cases was tabulated, and sources of diagnostic uncertainty were then tabulated based on this frequency. Only uncertainty at the criterion level was tabulated. The frequency with which each criterion for schizophrenia was met, not met, or uncertain was tabulated.

Results

As Table 1 shows, the diagnosis at study entry remained uncertain for 135 of the 165 patients (82 percent). A definitive diagnosis was reached in the remaining 30 cases. The diagnoses included 21 cases of schizophrenia, six of schizoaffective disorder, and three of psychostimulant psychosis. As shown in Table 2, differential diagnoses for these uncertain cases always included schizophrenia or schizoaffective disorder, frequently included psychostimulant psychotic disorder, and sometimes included delusional disorder, mood disorders, and even malingering. For more than half the uncertain cases, psychosis not otherwise specified remained among the differential diagnoses. Generally, this diagnosis was included because deterioration in functioning occurred coincident with drug abuse and might therefore have been due to drug abuse and not to the psychotic disorder.

Table 3 indicates the sources of diagnostic uncertainty. By far, the most frequent source of uncertainty was insufficient periods of abstinence. Poor memory and inconsistent responses often contributed to uncertainty. Surprisingly, psychotic symptoms rarely interfered directly with diagnosis, although psychotic symptoms may have indirectly interfered by contributing to other sources of uncertainty, such as poor memory.

As Table 4 shows, uncertainty was most likely to affect the criteria for exclusion of an organic etiology, deterioration, and exclusion of a mood disorder. In nearly all cases, characteristic psychosis, functional deterioration, and chronicity were clearly present, but extensive abuse of psychotomimetic drugs, especially cocaine, made it impossible to determine whether these features were due to schizophrenia or to ongoing psychotomimetic substance abuse or dependence. Most cases also included extensive mood syndromes, but it was usually not possible to exclude drugs as a cause of these syndromes, generally because of insufficient abstinence. The result was the frequent inability to distinguish between schizophrenia, schizoaffective disorder, and chronic organic psychoses due to a variety of abused drugs, particularly stimulants.

Table 1 also shows that reassessment at 18 months led to definitive diagnoses in 12 additional cases. They included eight additional cases of schizophrenia, two cases of psychostimulant psychotic disorder complicating the course of delusional disorder, one case of delusional disorder alone, and one case of malingering. The two cases of psychostimulant psychotic disorder complicating the course of delusional disorder were diagnosed as such because during prolonged periods of abstinence these patients continued to have psychotic symptoms that did not meet criteria for the characteristic psychotic features of schizophrenia. However, during episodes of stimulant use, other psychotic symptoms occurred such that the psychosis appeared characteristic of schizophrenia±_that is, it met the criterion for characteristic psychosis.

Discussion and conclusions

Despite structured diagnostic procedures, urine toxicology tests, review of medical records, and interviews with collateral sources, both initially and prospectively, we reached a definitive diagnosis in only 25 percent of cases. Most of the uncertainty arose because most subjects first developed psychotic symptoms during a period of psychotomimetic drug abuse and rarely stopped using drugs long enough to determine if the drugs were responsible for psychotic symptoms, mood syndromes, or deterioration in social and occupational functioning. Thus the differential diagnoses for the uncertain cases included mood disorders, substance-induced psychotic disorders, and other psychotic disorders.

These data suggest that for most patients with chronic psychoses and extensive psychotomimetic drug abuse and dependence, treatment must begin without a definitive diagnosis. Thus treatment algorithms and psychiatric services that require definitive diagnoses will be useful for only a minority of patients in this population.

It is important not to overgeneralize these findings. Our study involved a unique clinical population. Patients entered the study during a period of exacerbated symptoms and social problems requiring hospitalization. Fewer obstacles to a definitive diagnosis may exist in a less severely ill population, such as one drawn from an outpatient clinic. All our patients had psychotic disorders resembling schizophrenia. A definitive diagnosis may be less problematic for patients with mood and anxiety syndromes. Indeed, a previous study found that current substance dependence did not worsen the one-week test-retest reliability of mood and anxiety diagnoses using the Structured Clinical Interview for DSM-III-R (37). Unfortunately, the study included too few subjects with schizophrenia to determine the effect of current substance dependence on the reliability of that diagnosis.

All of our patients were men. Women who met our enrollment criteria might have differed in important ways. For example, if women tend to have more frequent and longer periods of abstinence, this factor would lead to less diagnostic uncertainty. All our patients were veterans of military service. Compared with other chronically psychotic stimulant-abusing persons, they probably had histories of less severe psychiatric disorders and a higher level of social functioning, at least when they entered the military. In other populations, more individuals may experience the early onset of psychotic symptoms before psychotomimetic drug abuse obscures the diagnosis.

Finally, the research setting was a large urban hospital. The results might be different in rural settings where psychotomimetic drugs may be less immediately available. Our study had several other limitations. First, the criteria for the modified SCID-R scoring were biased in favor of diagnostic uncertainty in several ways. Substances were judged to be etiologic based on only two considerations±the temporal relationship between the symptoms and substance use and whether or not the substance in question was known to cause that class of symptoms. Clinicians, on the other hand, may consider how characteristic the symptoms are for substance-induced syndromes.

For example, some subjects in our study had the persisting delusion that their drug use was closely monitored by clandestine government agencies using a variety of electronic devices. Some clinicians would consider this delusion likely to be substance induced because the content is not bizarre and is so closely related to illegal drug use. Substances were judged to be etiologic only if the symptoms always began during periods of substance use and had twice resolved within six weeks of stopping drug use. Less stringent definitions would have produced more diagnostic certainty. Finally, the reliability of diagnoses was inferred from the reliability of training interviewers rather than directly through co-rated interviews of the research subjects.

Despite these limitations, the results do indicate that substance abuse and dependence greatly complicate the diagnosis of psychotic symptoms. Diagnostic uncertainties might be resolved if abstinence could be ensured for many weeks, but ensuring such abstinence is rarely possible. Faced with this dilemma, clinicians may be tempted to pick the most likely diagnosis. However, premature diagnostic conclusions could deprive the patient of appropriate treatment. Clinicians should attempt simultaneous treatment of substance abuse or dependence and primary psychotic disorders even for patients who may later be shown to have had a substance-induced psychotic disorder.

In a substantial proportion of cases, it may not be possible to resolve diagnostic uncertainty even after months of treatment. This lack of resolution can occur if treatment fails to produce abstinence from psychotomimetic substances long enough to eliminate diagnostic uncertainties. Even if psychotic symptoms do resolve during periods of abstinence, it may be unclear whether the resolution was due to abstinence, the effects of antipsychotic medications, or the natural course of the psychotic disorder.

Research on dual diagnosis populations should acknowledge and manage diagnostic uncertainty. One approach is to study only those with definitive diagnoses. Unfortunately, such a strategy would make it difficult to generalize the results to clinical populations characterized by diagnostic uncertainty. Another approach is to classify subjects, as we have, into definitive and uncertain categories pending additional information from other sources or from prospective data on the relationship between abstinence and symptoms. This approach avoids the temptation to reach premature diagnostic conclusions based on a perceived probability that drugs are, or are not, etiologic.

Future research should focus on the effects of stimulants and other abused drugs on symptoms of subjects with and without a primary psychotic disorder. Data from such research could allow refinements in diagnostic criteria for distinguishing substance-induced psychosis from other psychotic disorders.

Acknowledgments

This research was supported by grant R01-MH48081 from the National Institute of Mental Health (NIMH) and grant IIR-90-033 from the Veterans Affairs Health Services Research and Development Service. The authors thank Robert P. Liberman, M.D., director of the UCLA Clinical Research Center for his support. The authors are indebted to Helena C. Kraemer, Ph.D., for invaluable statistical consultation during the development of the program to train interviewers in diagnosis and symptom assessment within the NIMH-funded Clinical Research Center for Schizophrenia and Psychiatric Rehabilitation.

When this work was done, all authors were affiliated with the West Los Angeles Veterans Affairs Medical Center and the department of psychiatry and biobehavioral sciences at the University of California, Los Angeles, School of Medicine. Ms. Roberts is currently affiliated with the department of psychology at the University of Washington in Seattle. Ms. Racenstein is with the department of psychology at the Illinois Institute of Technology in Chicago. Dr. Tucker is with the section on psychiatry and law at Rush-Presbyterian-St. Luke's Hospital in Chicago. Send correspondence to Dr. Shaner at the VA Medical Center-West Los Angeles (00MH), 11301 Wilshire Boulevard, Los Angeles, California 90073 (e-mail, [email protected]). This paper was presented at the annual scientific meeting of the College on Problems of Drug Dependence held June 22-27, 1996, in San Juan, Puerto Rico.

Enlarge table

Enlarge table

Enlarge table

Enlarge table