Prediction Error Representation in Individuals With Generalized Anxiety Disorder During Passive Avoidance

Generalized anxiety disorder is an anxiety disorder associated with pervasive worry (1). Anxiety symptoms are associated with poorer decision making in healthy (e.g., 2–4) and clinical populations (2), particularly individuals with generalized anxiety disorder (5), in whom reinforcement-based decision-making deficits are associated with symptom severity (5, 6). Individuals with generalized anxiety disorder fail to fully show healthy optimistic bias, which has been associated with a failure to appropriately represent the value of positive events in the medial prefrontal cortex (7), a region that is critical for reinforcement-based decision making (8). However, the nature of the pathophysiology disrupting reinforcement-based decision making in generalized anxiety disorder remains largely unknown.

Two computational processes that are critical for reinforcement-based decision making are the representation of 1) expected value (EV)—the reinforcement expectancy associated with a stimulus or action; and 2) prediction error (PE)—the difference between the amount of reward or punishment occurring relative to expectations (9). PE signals are thought to trigger reinforcement learning; the greater the PE, the greater the alteration in the reinforcement associated with the stimulus (9). The ventral striatum, the ventromedial prefrontal cortex, and the posterior cingulate cortex are implicated in both the representation of EV during choice and PE signaling during feedback (10, 11). In addition, the anterior insular cortex, the dorsal anterior cingulate cortex/dorsomedial frontal cortex, and the caudate show increased blood-oxygen-level-dependent (BOLD) response as a function of EV to suboptimal choices (12, 13); these regions likely use EV information to guide the individual away from suboptimal choices (12, 14).

Relatively few studies have examined reinforcement-based decision making in generalized anxiety disorder, and most have examined adolescents with the disorder (15–18). These studies have typically reported reduced responsiveness (other than within the anterior insular cortex). Thus, adolescents with generalized anxiety disorder, relative to healthy subjects, showed a reduced increase in activity within the caudate/putamen in anticipation of, and when receiving, reward after responding to high relative to low reward cues (15). Similarly, anxious adolescents (predominantly individuals with generalized anxiety disorder) showed less ventral putamen activity compared with healthy subjects during risky relative to safe choices for monetary rewards (16). Consistent with this, although within the amygdala, adults with generalized anxiety disorder showed reduced increases in amygdala activity for high uncertainty choices (choosing between two objects, both associated with a 50% reward contingency) relative to low uncertainty choices (choosing between two objects associated with, respectively, a 100% and a 50% reward contingency) compared with subjects without anxiety (17). Moreover, recent work has revealed that anticipatory anxiety (induced by the threat of electric shocks) is associated with reduced EV signaling of reward within the ventromedial prefrontal cortex and ventral striatum in healthy adults (19). However, Guyer et al. (18) reported typical recruitment of striatal regions by adolescents with generalized anxiety disorder during the anticipation of reward on the monetary incentive delay task. Moreover, Benson et al. (15) reported that when participants with anxiety were required to choose between two responses, they showed greater increases in activity within the caudate/putamen in anticipation of (and when receiving) reward after high relative to low reward cues compared with healthy subjects. Finally, Engelmann et al. (19) reported that anticipatory anxiety increased EV signaling of punishment within the anterior insular cortex in healthy adults, while Galván and Peris (16) reported that anxious adolescents showed greater anterior insular cortex activity compared with healthy subjects during risky relative to safe choices when responding to avoiding losing money.

Given that relatively few studies have examined reinforcement-based decision making in (particularly adult) individuals with generalized anxiety disorder, and the absence of patient studies utilizing model-based functional MRI (fMRI), we examined the computational processes involved in reinforcement-based decision making in adults with generalized anxiety disorder. Computational model-based imaging techniques allow the direct evaluation of empirically supported components of learning models, such as EV and PE (20). We made two main predictions: first, that individuals with generalized anxiety disorder would show impairment during reinforcement-based decision making; and second, that relative to healthy comparison subjects, they would show reduced EV/PE signaling within the ventromedial prefrontal cortex and the ventral striatum.

Method

Participants

The study included 78 participants 18–50 years of age; 46 had generalized anxiety disorder and 32 were healthy subjects matched on IQ, gender, and age (Table 1). Participants with generalized anxiety disorder were diagnosed according to DSM-IV criteria, based on the Structured Clinical Interview for DSM-IV Axis I Disorders (21) as well as a confirmatory clinical interview by a board-certified psychiatrist. As in our previous work, comorbidity with social anxiety disorder (N=18) was not exclusionary, although none of the participants had any axis I diagnoses other than generalized anxiety disorder and social anxiety disorder, and all had been medication free for at least 6 months. Drug screens for benzodiazepines, cocaine metabolites, opioids, and cannabinoids (THC) were performed, and positive screens were exclusionary for study participation. For the healthy comparison group, any history of psychiatric illness was exclusionary. All participants were in good physical health, as confirmed by a complete physical examination. The National Institutes of Health Combined Neuroscience Institutional Review Board approved all procedures. None of the participants took part in our earlier behavioral investigation of the passive avoidance task (6).

Passive Avoidance Task

Participants completed a passive avoidance task (Figure 1) that has been employed previously (22). Trials began with a 1500-ms image presentation of one of four objects, and participants either responded to the object or refused to respond. The image presentation was followed by a randomly jittered fixation (0–4000 ms) and by a 1500-ms feedback presentation. In trials in which participants chose to respond to an object, one of four outcomes was presented: win $5, win $1, lose $1, or lose $5. All objects could engender each of these outcomes. However, the feedback was probabilistic, with one object resulting in a gain of $18.57 over 10 trials, one a gain of $7.14, one a loss of $18.57, and one a loss of $7.14 (participants were paid for study participation, but not on the basis of their performance). Choosing not to respond resulted in the presentation of a blank screen. Another randomly jittered fixation (0–4000 ms) presentation followed feedback. Participants completed 168 trials (42 trials/object) over three runs.

Based on behavioral data, a learning curve was modeled establishing EVs and PEs for each object for each participant. PE was calculated by subtracting the EV of the current trial (t) from the feedback (F) received on the current trial (t) (coded 1 [object rewarded] or 0 [object punished]) with the formula

EV for the first trial of each object is set to 0 and updated according to the formula

where the EV of the current trial (t) equals the EV of the previous trial (t−1) plus the PE of the previous trial multiplied by the learning rate (α). The learning rate was set at 0.348; a learning rate was calculated for each participant via a model-fitting simulation (see the Supplemental Methods section of the data supplement that accompanies the online edition of this article), and then the average (0.348) was used for the model-based fMRI analyses (average log likelihood across the sample: mean=93.00, SD=21.71).

MRI Parameters

The MRI parameters were identical to those used previously for this task (22).

Imaging Data Preprocessing

Imaging data were preprocessed and analyzed in AFNI (23). At the individual level, functional images from the first five repetitions, collected prior to equilibrium magnetization, were discarded. Functional images from the three time series were motion corrected and spatially smoothed with a 6-mm full width at half maximum Gaussian filter. The time series were normalized by dividing the signal intensity of a voxel at each point by the mean signal intensity of that voxel for each run and multiplying the result by 100. The resultant regression coefficients represented a percentage of signal change from the mean.

After this, four regressors were generated: objects chosen, objects refused, reward received, punishment received. All regressors were created by convolving the train of stimulus events with a gamma variate hemodynamic response function to account for the slow hemodynamic response. The participants’ anatomical scans were then individually registered to the Talairach and Tournoux atlas (24). The individuals’ functional echo planar imaging data were then registered to their Talairach anatomical scan. Linear regression modeling was performed using the four regressors described above plus regressors to model a first-order baseline drift function. Furthermore, the percent signal change at each voxel and time point was modulated by EV (during the choice phase) and by PE (during the feedback phase). This produced a modulated and an unmodulated β coefficient and an associated t statistic for each voxel and regressor.

fMRI Data Analysis

The group analysis of the BOLD data was then performed on the modulated regression coefficients from individual subject analyses. The modulated coefficients were entered into a 2×2 (group [generalized anxiety disorder, healthy comparison] and choice [approach, refuse]) analysis of variance (ANOVA) in the choice phase and a 2×2 (group and feedback type [reward, punishment]) ANOVA in the feedback phase. The results of the analysis using the unmodulated coefficients can be found in the Supplemental Results section of the online data supplement. The 3dClustSim program in AFNI was used to calculate an extent cluster threshold at an initial p value of 0.005. This led to a 0.05 family-wise-error-corrected extent threshold of 18 voxels (486 mm³).

Results

Behavioral Results

A 2×3×2 (group, task run [first, second, third], error type [commission errors, omission errors]) repeated-measures ANOVA was conducted on the error data to test for group differences in task performance over the three runs of the task (Table 1). Significant main effects of group, task run, and error type were observed (F values, 9.24, 9.12, and 46.60, respectively; p values, <0.05). The healthy comparison group made fewer errors (28.57%) than did the generalized anxiety disorder group (36.59%). Moreover, across groups, the participants made significantly fewer mistakes in runs 2 (32.78%) and 3 (30.88%) relative to run 1 (36.24%) (t values, 2.26 and 3.04, respectively; p values, <0.027), although performance did not differ significantly between runs 2 and 3. Across groups, the participants also made more commission (21.74%) than omission errors (11.56%). Notably, a significant group-by-task run interaction was observed (F=6.37, p=0.002). In run 1, error rates did not differ between the two groups (34.93% and 37.15%). However, by runs 2 and 3, error rates were significantly lower in the healthy comparison group (27.40% and 23.38%, respectively) relative to the generalized anxiety disorder group (36.53% and 36.1%) (t values, 2.85 and 3.59; df=76; p values, <0.006).

To test the validity of the learning model, we examined whether it predicted behavior (as a function of EV). In line with the model, there was a significant relationship between predicted and observed behavior (average correlation: r=0.386, one-sample t test [null: r=0]: t=15.79, p<0.001). Next, we conducted a 2×3 (group and task run) repeated-measures ANOVA on the relationship between EV and task performance data. A significant main effect of group was observed (F=6.11, p=0.016) in which a stronger relationship between EV and behavior was observed in the healthy comparison group relative to generalized anxiety disorder group. A group-by-task run interaction was observed that fell short of significance (F=2.63, p=0.075), in which the two groups did not significantly differ with respect to the correlation between EV and behavior in run 1 (t=1.04, p=0.30), but did differ in runs 2 (t=2.26, p=0.03) and 3 (t=2.68, p<0.01).

fMRI Results

We sought in this study to examine whether individuals with generalized anxiety disorder would show abnormal neural recruitment during decision making, and in particular whether they would show disrupted EV representation in the ventromedial prefrontal cortex, ventral striatum, and anterior insular cortex during choice, and disrupted PE representation in the ventromedial prefrontal cortex and ventral striatum during feedback. This was tested in the cue phase using a 2×2 (group and choice [approach, refuse]) ANOVA conducted on the BOLD data modulated by EV and in the feedback phase using a 2×2 analysis (group and feedback type [reward, punishment]) conducted on the BOLD data modulated by PE.

Choice-Phase Data Modulated by Expectancies of Reinforcement

This contrast revealed no regions showing either a main effect of group or group-by-choice interaction that survived our criterion for statistical significance. However, it should be noted that in a follow-up analysis at a more lenient threshold (p=0.02), a significant main effect of group within the ventromedial prefrontal cortex was observed, as were significant group-by-choice interactions within the ventromedial prefrontal cortex, ventral striatum, and caudate (see the Supplemental Results section of the online data supplement). Weaker representation of EV was seen in all three regions in the generalized anxiety disorder group.

Regions showing a main effect of choice are detailed in the Supplemental Results section of the data supplement.

Feedback Data Modulated by PE

Main effect of group.

Regions showing a significant main effect of group included the left ventromedial prefrontal cortex, the right dorsal anterior cingulate cortex/dorsomedial prefrontal cortex, the left and right anterior insular cortex, the left posterior cingulate cortex, and the right ventral striatum (Table 2; Figure 2). In all these regions, the healthy comparison group showed greater modulation of activation by PE than did the generalized anxiety disorder group.

Group-by-feedback type interaction.

This analysis identified regions of the left and right lentiform nucleus that extended into the putamen and, on the right, the caudate (Table 2 and Figure 3). In both regions, the generalized anxiety disorder group showed reduced modulation by punishment prediction errors relative to the healthy comparison group (t=3.01, p<0.005, and t=4.24, p<0.001, for the left and right lentiform nucleus/putamen, respectively). There were no group differences in modulation by reward prediction errors.

Regions showing a main effect of feedback type are detailed in the Supplemental Results section of the data supplement.

Relationship Between Modulated Activation and Behavior

Within the overall sample, significant inverse correlations were observed between modulated response in the left and right anterior insular cortex, the right dorsal anterior cingulate cortex/dorsomedial prefrontal cortex, the left ventromedial prefrontal cortex, the right ventral striatum, and the left posterior cingulate cortex and omission errors (r values, −0.362 to −0.262, p<0.021). Within the generalized anxiety disorder group, a similar pattern of results was observed. Significant inverse correlations were observed between the right anterior insular cortex and the right dorsal anterior cingulate cortex/dorsomedial prefrontal cortex (r values, −0.296 and −0.299, p<0.046). Inverse correlations that fell short of significance were observed between the left anterior insular cortex, the left ventromedial prefrontal cortex, the right ventral striatum, and the left posterior cingulate cortex (r values, −0.259 to –0.283, p<0.10).

Relationship Between Dysfunctional PE Modulation of BOLD Responses and Symptom Levels in Generalized Anxiety Disorder

We conducted preliminary correlational analyses focusing on the striatal regions showing group-by-feedback interactions for the PE modulated BOLD response data. Extent of atypical response, relative to healthy comparison subjects, was not related to generalized anxiety disorder symptoms on the Penn State Worry Questionnaire or the Generalized Anxiety Disorder–7 scale. However, an association was observed between atypical response to punishing feedback in the left lentiform nucleus/putamen and scores on the Global Assessment of Functioning Scale (GAF) (r=−0.32, p=0.03).

Discussion

Our goal in this study was to determine the extent to which individuals with generalized anxiety disorder show dysfunction in the neural systems mediating specific computational components of decision making. There were three main findings. First, individuals with generalized anxiety disorder showed impaired reinforcement-based decision making. Second, during feedback, they showed a reduced correlation between PE and BOLD responses within the ventromedial prefrontal cortex, the ventral striatum, the dorsal anterior cingulate cortex/dorsomedial prefrontal cortex, the anterior insular cortex and the posterior cingulate cortex relative to healthy comparison subjects. Third, individuals with generalized anxiety disorder showed lower correlations between punishment PEs and BOLD responses within the lentiform nucleus/putamen relative to healthy comparison subjects.

Consistent with previous work (2, 5), individuals with generalized anxiety disorder showed impaired reinforcement-based learning as indexed by significantly more errors relative to healthy comparison subjects during a passive avoidance task. The groups did not differ in performance in run 1. However, while the healthy group showed significant improvement in behavioral performance over time, the generalized anxiety disorder group did not. These results are in line with our earlier work involving a more simplistic, behavioral version of the passive avoidance task, in which individuals with generalized anxiety disorder similarly showed impaired performance and reduced learning as a function of time relative to the healthy group (6).

Also consistent with predictions, individuals with generalized anxiety disorder showed reduced PE representation in the ventromedial prefrontal cortex and the ventral striatum, as well as in other regions implicated in the representation of reinforcement value (i.e., the dorsal anterior cingulate cortex/dorsomedial frontal cortex, the anterior insular cortex, and the posterior cingulate cortex [11]) relative to healthy comparison subjects. In most regions, individuals with generalized anxiety disorder showed reduced modulation of BOLD responses by PEs to both rewards and punishments relative to healthy comparison subjects. However, within the lentiform nucleus/putamen, it was specific to punishing feedback. Notably, previous work has reported that individuals with anxiety show both a lower increase in activity within the caudate/putamen in both anticipation and receipt of reward (15) and less putamen activity relative to healthy subjects when responding to risky relative to safe choices (16). However, previous work has not used a computational modeling approach to the decision-making deficits in generalized anxiety disorder. The present findings extend previous reports of reduced striatal activity in adolescents with the disorder when anticipating and receiving reward (15) and when making risky choices (16). Moreover, the present results are consistent with another model-based fMRI study of induced anticipatory anxiety in healthy participants, which also revealed reduced representation of decision-making computations within the ventromedial prefrontal cortex and the ventral striatum after anxiety induction (19).

It is probable that the reduced PE representation in regions implicated in reinforcement processing in the generalized anxiety disorder group contributed to their impaired task performance in this study and in previous work (2, 5). Consistent with this, the performance of the generalized anxiety disorder group in this study was significantly inversely related to the correlation between PE and activity within the anterior insular cortex and dorsal anterior cingulate cortex/dorsomedial prefrontal cortex during feedback. Moreover, there were indicators in this direction for the anterior insular cortex, ventromedial prefrontal cortex, ventral striatum, and posterior cingulate cortex, although they failed to reach statistical significance. All of these regions are implicated in successful decision making (12–14).

Notably, data from this study did not support our predictions with respect to representation of EV in the choice phase. However, analysis at a more lenient threshold (p=0.02) revealed that individuals with generalized anxiety disorder showed a lower correlation between EV and activity within the ventromedial frontal cortex during choice. Given these findings, consistent with predictions, we suspect that our failure to support our predictions with respect to EV representation in the choice phase is due to type II error. However, the possibility that the weak representation of PEs in the generalized anxiety disorder group is more prominent in driving the decision-making deficits seen in this population will need to be directly tested.

The extent of reduced punishment PE representation within the lentiform nucleus/putamen was related to lower GAF scores, but unrelated to more specific indices of generalized anxiety disorder symptoms. Notably, GAF scores specifically and uniquely correlated with degree of impairment on an early simplistic version of the passive avoidance learning task (6). In other words, individuals with generalized anxiety disorder who show greater problems in reinforcement representation, particularly within regions of the striatum implicated in the passive avoidance task, also show greater impairment in social, occupational, and psychological functioning.

A caveat should be considered: the present results may represent a general failure in attention/motivation in generalized anxiety disorder rather than a specific impairment in PE representation. However, this appears unlikely, for two reasons. First, individuals with the disorder do not show generally reduced attention/motivation in terms of either reaction times or error rates (25–27); and second, the reduced PE representation seen here is not coupled with generally reduced responses in individuals with generalized anxiety disorder. Indeed, the anxiety group showed atypically large unmodulated BOLD responses (see the online data supplement). These data are incompatible with a general attention/motivation dysfunction account, and they strengthen the idea of a relatively specific neurocomputational impairment in PE representation in generalized anxiety disorder.

In summary, this study is the first, to our knowledge, to identify computational impairments in PE signaling during decision making in generalized anxiety disorder. The results suggest that the decision-making deficits observed in patients with the disorder are the result of a failure to appropriately represent PE. A notable feature of generalized anxiety disorder is worry over potential future consequences, such as illness or the loss of a job. Within the language of the present study, in day-to-day living, patients with generalized anxiety disorder engage in increased representation of events associated with negative expected value (these are their worries). The deficits in PE signaling seen in this study may contribute to the maintenance of these worries. A healthy individual might form an expectation of a future adverse consequence (imminent job loss). However, the experience of not losing his or her job should generate a positive PE, changing the expected value of future thoughts about employment. Patients with generalized anxiety disorder with impaired PE signaling should be less sensitive to the inconsistency between their expectations and reality and thus be less likely to change the value of items that they worry about. These findings have two broad implications: First, boosting the representation of PE during feedback in individuals with generalized anxiety disorder may be a useful intervention goal. This might be achieved pharmacologically. Dopamine appears to be critically involved in PE signaling (28), and it is possible that dopaminergic agonists might boost the representation of PE (although it should be noted that we know of no work documenting this). Alternatively, it may be possible to increase the representation of PE through computer training techniques—for example, providing the patient experiences with highly salient PEs (e.g., a punishment for a response choice following repeated rewards) and then progressively increasing task difficulty so that punishment is more common and PEs are less salient. Second, investigating the computational processes underlying decision-making deficits in other anxiety disorders may refine diagnostic categories and differential diagnosis.