Reduced Neural Tracking of Prediction Error in Substance-Dependent Individuals

Drug addiction is characterized by poor decision making associated with urges to seek and take drugs despite knowledge of harmful consequences. One model of addiction suggests that pathological decision making reflects a disruption in the generation of physiological signals that indicate discrepancies between expected and received outcomes (1–3). These signals, broadly termed prediction error, are driven by dopamine release in the striatum and are critical for learning because they guide adjustments in short-term decisions to achieve long-term goals (4). Addiction is hypothesized to usurp these normal learning processes (5, 6). However, despite research suggesting that drugs adversely affect predictive learning signals in animals (3), few such studies have been conducted in drug users. One challenge is a limited understanding of processes that link physiology (i.e., learning signals) to behavior (i.e., pathological decision making). Computational model-based functional MRI (fMRI) methods have the potential to bridge this gap (2, 7).

Few model-based fMRI studies of prediction error learning have been conducted in drug users. One study in smokers (8) and another in alcohol-dependent men (9) did not find evidence of altered prediction error learning signals in the striatum. While nicotine and alcohol share mechanisms of drug addiction, most of the research suggesting maladaptive predictive learning involves stimulants. Yet, no study to date has investigated prediction error learning signals in stimulant-dependent individuals using a clinically valid measure of decision making.

The Iowa Gambling Task is a reward-related decision-making task involving risk and uncertainty. It is sensitive to decision-making deficits in patients with ventral medial frontal lesions and in clinical populations that include substance users (10–13). The goal of the task is to earn as much money as possible by selecting from four decks of cards. Two of the decks are associated with large gains and occasional large losses that, over time, yield a net loss; the other two decks are associated with small gains and small losses that, over time, yield a net gain. In order to succeed, subjects must learn to forgo the “risky decks” with immediate large gains in favor of “safe decks” with small gains. Task performance is influenced by sensitivity to reward and risk, temporal foresight, mental flexibility, and emotion. Cognitive models have been useful for separating some of these processes and have shown, for example, that compared with healthy subjects, cocaine users’ performance on the task reflects lower sensitivity to loss and lower response consistency (14).

The neural correlates of decision making on the Iowa Gambling Task have been studied (15–18). Most of these studies model fixed conditions (i.e., decision versus no decision or high- versus low-risk trials) rather than modeling changes that track learning over time. To investigate this critical aspect of performance, activity from initial phases of the task can be compared with later phases (18–20). While this provides a good first approximation of learning, computational fMRI allows for stronger inferences of biological mechanisms, such as how well neural structures track trial-to-trial prediction error during learning (21). Computational model-based fMRI studies in healthy subjects suggest that the medial prefrontal cortex and striatum track prediction error during learning on the Iowa Gambling Task (18, 22, 23).

Our goal in this study was to determine whether the tracking of prediction error is altered in stimulant-dependent individuals. We used a computational model of prediction error and a modified version of the Iowa Gambling Task to test the hypothesis that substance-dependent individuals would have reduced tracking of prediction error in the striatum and orbitofrontal cortex relative to comparison subjects. Such a finding could suggest a mechanism for poor decision making in substance users and inform future research aimed at therapy.

Method

Subjects

Forty-two substance-dependent patients and 36 healthy comparison subjects were recruited. Ten of the substance-dependent patients and six comparison subjects were excluded (three were excluded because of diagnostic exclusions, four because of scanner failure, and nine because of head motion exceeding 3 mm). Data are reported on 32 substance-dependent patients (19 of them male; age range, 22–50 years, with a mean of 36.6 years [SD=9.0]) and 30 comparison subjects (15 of them male; age range, 20–49 years, with a mean of 34.5 years [SD=7.5]), on whom some behavioral data have been published previously (24). The two groups had a similar mean IQ (comparison group: mean=103.3, SD=12.7, range=82–126; patient group: mean=102.2, SD=10.3, range=83–125).

Patients with DSM-IV stimulant dependence were recruited from a residential drug treatment program at the University of Colorado Denver Addiction Research and Treatment Service. Other dependence diagnoses and the duration of dependence and abstinence are listed in Table 1. Average abstinence was 1.9 years.

Community comparison subjects were recruited using advertisements, flyers, and a marketing company. Volunteers were excluded if they met DSM-IV criteria for drug or alcohol dependence.

Exclusion criteria included head trauma with loss of consciousness exceeding 15 minutes, neurological disease, schizophrenia, bipolar disorder, and current major depression. All participants provided written informed consent as approved by the Colorado Multiple Institutional Review Board and completed diagnostic and structured interviews, behavioral tasks, and MRI scanning.

Interviews and Tasks

We used the Composite International Diagnostic Interview–Substance Abuse Module (25), a computerized structured interview, to characterize the DSM-IV dependence diagnoses for amphetamine, cocaine, marijuana, alcohol, tobacco, hallucinogens, opioids, inhalants, sedatives, club drugs, and phencyclidine. Specific modules of the Diagnostic Interview Schedule–Version IV were administered to exclude schizophrenia, bipolar disorder, current major depression (<2 months) and to assess for antisocial personality disorder, which was not exclusionary. IQ was assessed with the Wechsler Abbreviated Scale of Intelligence (WASI). The WASI Two-Subtest Form was used to estimate general intelligence (Psychological Corporation, 1999). We also administered the Barratt Impulsiveness Scale, a 30-item self-report questionnaire that provides a measure of impulsivity (26).

We used a modified version of the Iowa Gambling Task (27). Subjects were shown four decks of cards and instructed to try to earn as much money as possible by choosing to either play or pass on a given deck. “Play” resulted in a single positive or negative monetary value, along with the running total. “Pass” resulted in no change. Decks differed in the magnitude and frequency of wins and losses. Two of the four decks resulted in a net gain; the other two decks resulted in a net loss. To perform well, subjects had to learn to “play” good decks and “pass” bad decks over time. Task details have been described elsewhere (24); a diagram of the task is provided in the data supplement that accompanies the online edition of this article. The decision period was 2 seconds, and the outcome period was 4 seconds. Each deck was presented 50 times (for a total of 200 trials) in pseudorandom order, interspersed with 65 6-second fixation trials. Total scan time was 26.4 minutes, divided into three runs of 8.8 minutes each.

Expectancy Valence Model

Prediction error is the difference between expected and received outcome. For each subject and for each trial, prediction error was computed as the difference between the individual’s internally generated expected outcome (termed “valence”) and actual outcome. The expectancy valence model, adapted from Stout et al. (28), deconstructs decision making on the Iowa Gambling Task into three processes: sensitivity to loss (ω), recency or updating (α), and response consistency (c). Model details are provided in the online data supplement.

MRI Acquisition and Preprocessing

Acquisition.

Images were acquired on a 3.0-T scanner using a quadrature head coil. Functional images were acquired with a gradient-echo T₂* blood-oxygen-level-dependent (BOLD) contrast technique, TR=2,000 ms, TE=30 ms, field-of-view 220 mm², 64×64 matrix, 30 slices, 4 mm thick, no gap, angled parallel to the planum sphenoidale. To reduce inferior frontal susceptibility artifact, we used a volume-selective z-shimmed image acquisition (29). MR-compatible goggles were used for visual stimuli, and responses were recorded with a two-button response device.

Preprocessing.

The first four image volumes from each run were excluded for saturation effects. Data were processed and analyzed using SPM8 (http://www.fil.ion.ucl.ac.uk/spm/software/spm8/). Functional data were realigned to the first volume. Data were excluded for head motion exceeding 3 mm. The realigned images were normalized to Montreal Neurological Institute space. Data were smoothed with a 6-mm full width at half maximum Gaussian kernel. The final smoothness of the data after preprocessing was 8.2×8.4×7.9 mm³ and was used in the Monte Carlo simulations to determine cluster-wise corrected threshold levels.

fMRI Data Analysis

First-level model.

fMRI time-series data were analyzed using the general linear model after filtering low-frequency noise, correcting for temporal autocorrelation, and convolving the stimulus function with a canonical hemodynamic response function. The regressor of interest was the trial-to-trial prediction error. Regressors of no interest were deck, run, and play. The play regressor controls for the mean effect of a play response during the stimulus phase. By including this regressor, the variability in signal captured by the prediction error regressor is more likely to capture the processes associated with prediction error computed at the time of the outcome.

Second-level model.

Contrast maps for the prediction error regressor were brought to the second level and analyzed using whole-brain and region-of-interest analyses.

Whole brain.

Whole-brain group comparison was conducted using a two-sample t test with education as a covariate, as this variable differed between groups. The statistical threshold was set at a voxel level of p<0.005, uncorrected, and p<0.01 whole-brain cluster-corrected, corresponding to an extent threshold of 39 voxels based on 10,000 Monte Carlo simulations using AlphaSim in the AFNI program (http://afni.nimh.nih.gov/afni/).

Regions of interest.

Based on the literature, four regions of interest were defined: the ventral striatum, the dorsal striatum, the medial orbitofrontal cortex, and the lateral orbitofrontal cortex. The medial orbitofrontal cortex consisted of Brodmann’s area (BA) 25, BA 32, and medial BA 11. The lateral orbitofrontal cortex consisted of BA 47, BA 12, and BA 10. The boundaries of the ventral and dorsal striatum were manually drawn (30). Regions of interest were created using the Wake Forest University PickAtlas (see the online data supplement). Time-series data were extracted for each region of interest using the MarsBaR toolbox in SPM8.

Statistical Analysis

Demographic and behavioral data were tested for normality and analyzed with two-tailed t tests or chi-square tests, for continuous and categorical variables, respectively.

Modified Iowa Gambling Task performance and valence model parameters.

Performance was analyzed with a 2×2 (group-by-time) repeated-measures analysis of variance on net score. Net score is the number of plays on good decks minus the number of plays on bad decks. Since there were 50 trials for each deck, net score was calculated for the first half and for the second half, consisting of 25 trials each.

The three parameters (ω, α, c) from the expectancy valence model of the Iowa Gambling Task were analyzed with a bootstrapping procedure because the parameters were highly skewed. Parameters were compared across the group by bootstrapping the sampling distribution of the mean difference between groups (5,000 iterations).

Regions of interest.

For each region, fMRI signal (beta estimates) for prediction error tracking was first analyzed in all subjects using a one-sample t test and then compared across groups using a two-sample t test after adjusting for education.

Correlations among fMRI signal, behavioral measures, and drug characteristics.

fMRI signal, duration of abstinence, duration of drug use, impulsivity score, and decision-making parameters were correlated within group using Pearson’s r and Spearman’s rho for normal and nonnormal data, respectively.

Results

Demographic Characteristics and Behavior

The patient and comparison groups were not significantly different in mean age (36.6 years [SD=9.0] and 34.5 years [SD=7.5], respectively), gender distribution (19 men and 13 women compared with 15 men and 15 women), and mean IQ (102.2 [SD=10.3] compared with 103.3 [SD=12.7]). Comparison subjects had more years of education on average than did patients (14.3 years [SD=1.9] compared with 12.0 years [SD=2.3], p<0.001), so education was entered as a covariate in all group comparisons. Patients were more impulsive on average than comparison subjects (mean Barratt Impulsiveness Scale score, 70.1 [SD=9.5] compared with 59.7 [SD=7.0], p<0.001). Thirty-two patients and 11 comparison subjects met criteria for antisocial personality disorder.

Drug Characteristics

Table 1 lists dependence diagnoses and duration of dependence and abstinence.

Behavioral Performance on the Modified Iowa Gambling Task

As shown in Figure 1, a significant group-by-time interaction was observed on net score, which improved over time in the comparison group but not in the patient group (F=4.09, df=1, 58, p=0.05; in the comparison group, the mean net score was 4.62 [SD=1.3] for the first 25 trials and 6.97 [SD=1.7] for the next 25 trials; in the patient group, these values were 4.18 [SD=1.3] and 4.05 [SD=1.62], respectively).

Expectancy valence model parameters (ω, α, c).

Patients’ decisions were significantly less consistent over time than those of the comparison subjects (c=−0.46 and c=−0.01, respectively; p=0.02). Patients also showed a lower sensitivity to loss, although this difference did not reach statistical significance (ω=0.16 and ω=0.26, respectively; p=0.06). There was no group difference in recency (α) (see Table S4 in the online data supplement).

Imaging

Whole-brain analysis.

Figure 2 shows prediction error-related signal in the ventral striatum and medial orbitofrontal cortex in the comparison group and the patient group, as well as the group difference. Patients had weaker tracking of prediction error in the medial orbitofrontal cortex. Table 2 lists significant clusters of lower prediction error-related activity in patients relative to comparison subjects in the left insula, thalamus, and left superior temporal gyrus. There were no regions of higher prediction error-related fMRI signal in patients relative to comparison subjects.

Region-of-interest analysis.

Table 3 presents the results of one-sample t tests across all subjects and two-sample t tests of prediction error fMRI signal. Across all subjects, there was tracking of prediction error in the ventral and dorsal striatum and the medial orbitofrontal cortex. Relative to comparison subjects, patients did not track prediction error as strongly in the medial orbitofrontal cortex (t=2.3, df=61, p=0.01) and the ventral striatum (t=1.9, df=61, p=0.03). There were no group differences in the dorsal striatum or the lateral orbitofrontal cortex.

Correlations.

In the comparison group, decision-making consistency correlated with fMRI signal in the lateral (r=0.39, p<0.05) and medial orbitofrontal cortex (r=0.43, p<0.05). Recency correlated negatively with fMRI signal in the lateral orbitofrontal cortex (r=−0.37, p<0.05). Impulsivity correlated negatively with response consistency (r=−0.38, p<0.05). In contrast, in the patient group, fMRI signal did not correlate with any behavioral measures.

Discussion

Effect of Substance Dependence on Striatal Prediction Error

Relative to comparison subjects, substance-dependent individuals showed a significantly weaker relationship between prediction error and fMRI signal in the ventral striatum and medial orbitofrontal cortex. This finding is consistent with a large body of evidence of altered striatal function related to drug abuse. While it is firmly established that drugs act on the striatal dopaminergic reward circuitry, the development of addiction also likely involves non-reward processes involving learning and decision making (5). Cue learning, for example, is driven by dopaminergic signals that track prediction error and is thought to be altered in addiction. Animals exposed to cocaine show abnormal striatal firing during odor discrimination learning in which rats learned that one odor predicted sucrose and another predicted quinine (3). Cue-selective activity was recorded in dopaminergic striatal neurons. Control rats showed stronger cue-selective activity in the ventral striatum compared with cocaine-treated rats, which is consistent with our results. We found no group difference in activity in the dorsal striatum, which is somewhat inconsistent with other studies. Cocaine addicts, for example, show greater dorsal striatal dopamine release when viewing drug-related compared with neutral videos, suggesting that cocaine shifts the representation of learning signals from the ventral striatum, which involves reward and motivation, to the dorsal striatum, which involves habit learning (31). Dorsal striatal activity is hypothesized to prime individuals to engage in habitual behaviors, such as procuring drugs (6, 32). Our finding that prediction error tracking was represented in the dorsal striatum for both groups, but with no difference between them, may be related to the fact that our decision-making task is unlikely to induce the strong habit-learning that would be expected for drug cues.

Effect of Substance Dependence on Orbitofrontal Cortex Expectancies

Cue-selective firing during prediction learning is not unique to the striatum but is also found in prefrontal cortex neurons (33, 34). Stalnaker et al. (35) showed that cocaine-treated rats fail to develop cue-selective firing in orbitofrontal cortex neurons using the odor discrimination task described above. Furthermore, when the odor-outcome pairings were reversed, these neurons failed to demonstrate normal reversal of cue-selective firing in cocaine-treated compared with saline-treated rats, suggesting that drugs disrupt learning signals processed in the orbitofrontal cortex. Our finding of weaker tracking of prediction error in the medial orbitofrontal cortex in patients relative to comparison subjects is consistent with these studies. Drugs may disrupt outcome expectancies encoded in the orbitofrontal cortex and prevent the striatum from accurately updating prediction error (34). This is consistent with reduced frontal-striatal connectivity reported in alcohol-dependent patients and suggests a disruption in the transfer of prediction error signals from the striatum to the orbitofrontal cortex (9). Another hypothesis is that dopaminergic striatal activity serves as a gating signal to the prefrontal cortex and that drug exposure dampens the ability of natural rewards to open the gating so that the orbitofrontal cortex can influence decision making (5).

Our finding that group differences in prediction error involved the medial more than the lateral orbitofrontal cortex confirms the critical role of the ventromedial prefrontal cortex in performance on the Iowa Gambling Task (27). It also supports the notion that the medial and lateral orbitofrontal cortex play distinct roles in reward-related decision making. The medial orbitofrontal cortex has been more strongly associated with motivation, uncertainty, valuation, and effort. The lateral orbitofrontal cortex has been associated with processing aversive outcomes (36) or suppressing unrewarded responses (37). The lateral orbitofrontal cortex is thought to exert top-down control, planning how to achieve goals and passing the information to the medial orbitofrontal cortex, which evaluates the cost-benefit balance of those plans and estimates and monitors expected value (38). In our study, fMRI signal in the orbitofrontal cortex correlated positively with response consistency and negatively with recency valence estimates in comparison subjects but not in patients, suggesting that the orbitofrontal cortex plays a role in computing expected valence and may be affected in drug users. These correlates were weak, however, and would need to be replicated.

Model-Based fMRI in Drug Users

Although this was not the first study to investigate neural correlates of prediction error in addiction, it is the first to report differences in individuals with stimulant dependence. Park et al. (9) reported no difference in striatal prediction error in alcohol-dependent compared with healthy men performing a probabilistic learning task, while Chiu et al. (8) found no difference in smokers compared with nonsmokers playing a stock market simulation task. Our study differs from these studies in several ways. First, the substance use problems in our population were more severe. Second, we used a non-commercially available z-shim MR pulse sequence that increases signal in the orbitofrontal region (29). Third, the study tasks differed in cognitive demand. Finally, the use of internally generated (i.e., subjective) prediction error signals could be more sensitive than externally computed (i.e., objective) prediction error signals. For example, Stout et al. (39) showed that a cognitive model of internally generated valence predicted behavior on the Iowa Gambling Task better than a linear model based on externally computed valence, consistent with our finding that the valence model described task behavior better than a conventional reinforcement learning model. We argue that brain activity should more closely track internally generated compared with externally computed expectancy.

Cognitive Processes Underlying Decision Making on the Modified Iowa Gambling Task

Relative to the comparison group, the patient group did not show an improvement in decision making over time, consistent with previous studies (12, 40). Performance on the Iowa Gambling Task is influenced by many factors, including reward sensitivity, ability to forgo short-term in favor of long-term gain, impulsiveness, and cognitive flexibility. The expectancy valence model disentangles some of the processes (14) by parameterizing sensitivity to gain and loss (risk), discounting of prior experiences (recency), and consistency between expectancies and choice (28). Despite the modifications to the original Iowa Gambling Task (i.e., subjects were not free to select but rather a deck was selected for them), our results were consistent with previous model studies in drug users (28, 41). The finding that substance users’ decisions were less consistent and less sensitive to loss than comparison subjects’ may reflect impulsiveness and “myopia for the future.”

Most neuroimaging studies of the Iowa Gambling Task model specific trial conditions (i.e., decision versus no decision or high versus low risk). Brain regions implicated in these models include the medial and lateral prefrontal cortex, the cingulate cortex, the striatum, and the insula (16–19). These models, however, fail to account for learning. To approximate this critical aspect of the task, Li et al. (18) compared brain activity from the first and last blocks of the task and found that orbitofrontal cortex activity increased over time. In contrast, Lawrence et al. (19) reported less activity in the orbitofrontal cortex and insula over time. One possibility for these inconsistencies is that individuals learn at different rates. Model-based fMRI accounts for variations in learning and has been applied to the Iowa Gambling Task (18, 23). Christakou et al. (23) found that the right ventromedial prefrontal cortex was sensitive to individuals’ negative expectancies. Although we did not separate positive from negative expectancies, our finding of ventral medial activity related to global expectancy is consistent with these results (23). Li et al. (18) also found that prediction error correlated with activity in the ventral striatum and the middle frontal gyri. Both of these previous studies, however, used externally derived prediction estimates. Our study extends model-based fMRI studies for the first time using internally derived prediction estimates in substance-dependent individuals.

Strengths of this study include the use of a computational model based on internally generated valence, the relatively large sample size of 32 patients, the use of an MR pulse sequence that increases orbitofrontal cortex signal compared with conventional MRI sequences, and a comparison group matched on IQ. Limitations include an inability to directly compare results to the Iowa Gambling Task, an inability to determine whether drug use preceded or antedated neural changes, the abuse of multiple drug classes in our sample, and the lack of control for medications. We cannot exclude the possibility that antisocial personality disorder could account for some results, as this diagnosis was common in both groups.

In summary, decision-making deficits in substance users compared with healthy subjects were related to inconsistent choices and reduced neural tracking of prediction error signals in the striatum and orbitofrontal cortex, suggesting that frontal-striatal pathways involved in learning may be a target for future treatment.