Disrupted Expected Value and Prediction Error Signaling in Youths With Disruptive Behavior Disorders During a Passive Avoidance Task

Youths with disruptive behavior disorders, including conduct disorder and oppositional defiant disorder, show increased aggression and antisocial behavior (1). Moreover, prognosis is poor, with many presenting with severe pathology in adulthood (2, 3). A major impairment, perhaps contributing to increased risk for antisocial behavior, is in decision making. Youths with severe behavioral problems and psychopathic traits show impairment on the Iowa gambling (4), reversal learning (5), and passive avoidance (6) tasks. The neural basis of this difficulty is only beginning to be understood (6–9).

Two critical computational components of successful decision making involve 1) the appropriate representation of reinforcement expectancies (representing the expected value associated with a stimulus or action) and 2) prediction error signaling (detecting the difference between the amount of reward or punishment received and the amount expected so that changes to reinforcement expectancies can be learned [10]). The ventromedial prefrontal cortex and striatum show increased activity during choice as a function of the expected value of the choice (11, 12). The anterior insular cortex and, to a lesser extent, the dorsal anterior cingulate cortex have been implicated in the avoidance of suboptimal choices (13, 14) and are sensitive to expected value information (15, 16). In typically developing individuals, the ventromedial prefrontal cortex and striatum show increased activity as a function of positive prediction error level (the degree to which reinforcement was better than expected) and decreased activity as a function of negative prediction error level (the degree to which reinforcement was worse than expected [11]). Prediction error signals are thought to trigger reinforcement learning; greater prediction error is associated with greater alteration in the reinforcement associated with the stimulus (10).

There have been reports of reduced responses to reward within the ventromedial prefrontal cortex (6, 7) and increased responses to unexpected punishment within the ventromedial prefrontal cortex and striatum (9) in individuals with disruptive behavior disorders and in youths with symptoms of conduct disorder and substance abuse/dependence (8). However, this previous literature has important limitations. Particularly, no previous study has examined responses during decision making and during reinforcement in this population using model-based functional MRI (fMRI). Computational model-based imaging allows for the testing of specific hypotheses regarding how, not simply where, a function is conducted (i.e., whether computations relating to expected value or prediction error are intact) (17, 18). Model-based imaging techniques allow empirically supported models of learning, such as expected value processing during the decision-making phase (18) and prediction error processing during the feedback phase (11), to be directly evaluated, yielding deeper insight into the psychopathology of disruptive behavior disorders.

The present study involved a probabilistic version of the passive-avoidance decision-making paradigm used previously in this population (6). Critically, model-based regressors were used: those for cue were weighted by expected value, and those for outcome were weighted by prediction error according to learning theory (10). We predicted that during decision making, youths with disruptive behavior disorders would show 1) reduced modulation of blood-oxygen-level-dependent (BOLD) responses in the ventromedial prefrontal cortex by expected value when choosing to respond and 2) reduced modulation of BOLD responses in the anterior insular cortex by expected value when refusing to respond.

Consistent with previous studies (7, 9), we also predicted that during feedback, youths with disruptive behavior disorders would show 1) reduced modulation of BOLD responses in the ventromedial prefrontal cortex and striatum by prediction error when receiving reward (7, 8) and 2) increased modulation of BOLD responses in the ventromedial prefrontal cortex and striatum by prediction error when receiving punishment (8, 9).

Our final prediction concerned callous-unemotional traits. Youths with disruptive behavior disorders are considered to be a heterogeneous group (19–21). Indeed, it has been proposed that the conduct disorder diagnosis in DSM-5 be modified to include a callous-unemotional specifier. Callous-unemotional traits (reduced guilt and empathy) are one of three components of psychopathic traits (the others being impulsivity and narcissism). It was previously suggested that callous-unemotional traits relate to dysfunction in the representation of reinforcement value mediated by the ventromedial prefrontal cortex (21). We thus predicted that the level of callous-unemotional traits in the disruptive behavior disorders group would be inversely associated with modulation of the ventromedial prefrontal cortex activity as a function of expected value.

Method

Participants

Twenty youths with disruptive behavior disorders and 18 healthy comparison youths participated in this study (Table 1). Youths were recruited from the community through advertising, fliers, and referrals from mental health practitioners. Informed assent and consent were obtained from participating children and their parents. This study was approved by the National Institute of Mental Health Institutional Review Board.

All youths and parents completed the Schedule for Affective Disorders and Schizophrenia for School-Age Children–Present and Lifetime Version (K-SADS–PL) (22), conducted by a doctoral-level clinician as part of a comprehensive psychiatric and psychological assessment. Parents also completed the Antisocial Process Screening Device (23), a measure of psychopathic traits (see the data supplement that accompanies the online edition of this article). Youths meeting K-SADS–PL criteria for conduct disorder or oppositional defiant disorder were included in the disruptive behavior disorders group, while healthy comparison subjects did not meet criteria for any K-SADS–PL diagnosis (for more details on the participant characteristics, see the online data supplement).

IQ was assessed with the Wechsler Abbreviated Scale of Intelligence (two-subtest form). The groups did not differ significantly in age (Table 1). However, the healthy comparison group had a significantly higher IQ (t=4.668, df=36, p<0.001), and the disruptive behavior disorders group had a significantly greater proportion of minority youths (χ²=8.155, p=0.004) and male participants (χ²=3.993, p<0.05).

Passive Avoidance Task

Participants were presented with a passive avoidance task, in which four images were presented (Figure 1). Trials began with a 1,500-ms image presentation, in which participants either responded to the object shown or refused to respond. The image presentation was followed by a randomly jittered fixation (0 ms–3,000 ms) and by a 1,500-ms feedback presentation. After responding to an object, participants received one of four outcomes: win $5.00, win $1.00, lose $1.00, or lose $5.00. All objects could engender each of these rewards. However, the feedback is probabilistic such that one object should result in a gain of $18.57 over 10 trials, one a gain of $7.14, one a loss of $18.57, and one a loss of $7.14. Feedback was followed by another randomly jittered fixation (0 ms–3,000 ms). Participants completed 112 trials (28 trials per object) over two runs. Choosing not to respond resulted in no reward or punishment.

Using participants’ behavioral data, a learning curve was modeled establishing expected values and prediction errors for each object. The expected value for the first trial of each object was 0. Prediction error was then calculated based on the feedback, which was coded 1 (object rewarded) or 0 (object punished) with the following formula:

In this formula, the prediction error for the current trial (PE_[t]) equaled the feedback value for the current trial (F_[t]) minus the expected value for the current trial (EV_[t]). Expected value was calculated with the following formula:In this formula, the expected value of the current trial equaled the expected value of the previous trial plus the prediction error of the previous trial multiplied by the learning rate (α), which was set at 0.1 following previous modeling and empirical work (e.g., 24, 25).

MRI Parameters

Participants were scanned using a 1.5-T GE Signa scanner (General Electric, Fairfield, Conn.). A total of 118 functional images per run were taken using gradient echo planar imaging (EPI) sequence (repetition time=2,300 ms; echo time=30 ms; 64×64 matrix; flip angle=90°; field of view=24 cm). Whole-brain coverage was obtained with 31 axial slices (thickness, 4 mm; in-plane resolution, 3.75 mm×3.75 mm). We obtained a high-resolution anatomical scan in register with the EPI data set covering the whole brain (three-dimensional spoiled gradient recalled acquisition in a steady state; repetition time=9 ms; echo time=2.872 ms; field of view=24 cm; flip angle=20°; 124 axial slices; thickness, 1.5 mm; 256×256 matrix).

Imaging Data Preprocessing

Imaging data were preprocessed and analyzed using Analysis of Functional Neuroimages (AFNI) (26). At the individual level, functional images from the first six repetitions, collected prior to equilibrium magnetization, were discarded. Functional images from the two time series were motion corrected and spatially smoothed with a 6-mm full-width half-maximum Gaussian filter. The time series were normalized by dividing the signal intensity of a voxel at each point by the mean signal intensity of that voxel for each run and multiplying the result by 100. Resultant regression coefficients represented a percentage of signal change from the mean.

Next, four regressors were generated: objects chosen, objects refused, reward received, and punishment received. All regressors were created by convolving the train of stimulus events with a gamma variate hemodynamic response function to account for the slow hemodynamic response. The participants’ anatomical scans were then individually registered to the Talairach-Tournoux atlas (27); studies have shown that normalization of brain volumes from ages 7–8 onward does not introduce major age-related distortions in localization or time course of the BOLD signal in event-related fMRI (28). The participants’ functional EPI data were then registered to their Talairach anatomical scans. Linear regression modeling was performed using the four regressors described above plus regressors to model a first-order baseline drift function. Furthermore, the expected value estimates and prediction error values were used to modulate the percent signal change at each voxel and time point. This produced a modulated and unmodulated β coefficient and associated t statistic for each voxel and regressor.

fMRI Data Analysis

The group analysis of the BOLD data was then performed on regression coefficients from individual subject analyses using a series of independent samples t tests. For the decision-phase analysis, activations modulated by the expected value to chosen objects in the disruptive behavior disorders group were compared with those in the healthy comparison group, as were activations to refused objects. Similarly, in the feedback-phase analysis, activations modulated by prediction error in both groups were contrasted for both rewarded and punished trials.

Given our a priori hypotheses, regions of interest, taken from the AFNI software anatomical maps, were obtained for Brodmann’s area 10 (the ventromedial prefrontal cortex) and the insula and caudate. A small-volume-corrected region-of-interest analysis using the ClustSim function in AFNI was used on these regions (initial threshold: p<0.005 with minimum cluster sizes identified for each region of interest at a corrected p value <0.05). For completion, a whole-brain analysis was also conducted (see the online data supplement). Because of the significant group differences in IQ scores and in the proportion of gender and minority youths, activity within the functional regions of interest identified by these four t tests was further analyzed in a series of one-way analyses of covariance (ANCOVAs), with IQ score, gender, and minority status as covariates. For all the regions reported, the introduction of the covariates did not remove the significant differences between groups. In addition, these analyses were repeated after the removal of seven youths in the disruptive behavior disorders group and five in the healthy comparison group so that the groups were matched on IQ.

Results

Behavioral Data Analysis

To test the validity of the model, we examined whether it predicted behavior (as a function of expected value). Consistent with the model, there was a significant relationship between predicted and observed behavior (p<0.001). Additionally, a greater relationship between expected value and behavior in healthy comparison subjects than in youths with disruptive behavior disorders fell short of statistical significance (t=1.911, p=0.06). Moreover, a significantly greater proportion of healthy comparison subjects relative to youths with disruptive behavior disorders showed a significant association between expected value and behavior (χ²=6.65, p=0.01).

fMRI Data Analysis

The goal of our study was to assess whether youths with disruptive behavior disorders showed aberrant modulation of BOLD signal by expected value and/or prediction error (for whole-brain analyses, see the online data supplement). This was tested with a series of t tests contrasting youths with disruptive behavior disorders with healthy comparison youths: first for activation modulated by expected value to chosen and refused objects during the decision phase and second for activation modulated by prediction error to reward and punishment during the feedback phase (Table 2).

Choice-Phase Data Modulated by Expectancies of Reinforcement

Within the ventromedial prefrontal cortex region of interest, the disruptive behavior disorders group, relative to the healthy comparison group, showed significantly reduced modulation of activity as a function of expected value when choosing an object. Within both the anterior insular cortex and caudate regions of interest, youths with disruptive behavior disorders, relative to healthy comparison youths, showed significantly reduced modulation of activity as a function of expected value when refusing to respond to stimuli (Figure 2).

Feedback Data Modulated by Prediction Error

Within the caudate region of interest, the disruptive behavior disorders group, relative to the healthy comparison group, showed significantly reduced modulation of activity as a function of prediction error when receiving reward but significantly increased modulation of activity as a function of prediction error when receiving punishment (Figure 3).

Relationship Between Modulated Activation and the Correlation Between Expected Value and Behavioral Response

Correlation analyses were conducted to examine the relationship between the level of modulated activity in the regions of interest and the degree to which expected value predicted behavior. This revealed that there was a significant relationship for the insula (the greater the modulation of insula activity by expected value, the greater the relationship between expected value and behavior; r=0.323, p<0.05) but not for other regions. However, there were no significant group differences in the strength of this relationship.

Potential Confounds

To account for possible effects of medication and comorbid attention deficit hyperactivity disorder (ADHD), the preceding analysis was repeated first without the two youths in the disruptive behavior disorders group who were taking medication and second without the four youths from this group who met criteria for ADHD. In both cases, the preceding effects of interest were replicated with proximal activations in the same brain regions for all contrasts (see Tables S2 and S3 in the online data supplement).

To account for group differences in IQ, beyond post hoc covariance of the effects of IQ on modulated activation data, the analysis was repeated in an IQ-, age-, and ethnicity-matched sample of 13 youths with disruptive behavior disorders and 13 healthy comparison youths (see Table S4 in the online data supplement). Again, the preceding effects of interest were replicated with proximal activation in the same brain regions for all contrasts. However, the ventromedial prefrontal cortex result was seen only at a higher threshold (p=0.008).

Symptom Severity and Neural Response

Given the group differences in sensitivity to expected value and prediction error information within the ventromedial prefrontal cortex, anterior insular cortex, and caudate, we examined the association between modulated activity in these regions and the severity of callous-unemotional traits in the disruptive behavior disorders group. Contrary to our hypotheses, no significant relationships were found. Moreover, no significant associations were found for these regions in the disruptive behavior disorders group on either the impulsive or antisocial subscale of the Antisocial Process Screening Device.

Discussion

To our knowledge, this is the first study to identify the computational impairments, and their neural correlates, underpinning the decision-making deficits seen in youths with disruptive behavior disorders. Youths with disruptive behavior disorders showed significantly reduced sensitivity to expected value information within the ventromedial prefrontal cortex when choosing objects and within the anterior insular cortex when refusing objects. Moreover, within the caudate, these youths showed reduced modulation by prediction error of responses to reward but increased modulation by prediction error of responses to punishment. In short, our study identified, for the first time, a constellation of impairments involving the computations necessary for successful decision making in youths with disruptive behavior disorders.

Previous studies have suggested that youths with disruptive behavior disorders have reduced reward-related signaling (either during decision making or feedback [6, 7]). However, our study is the first to demonstrate that this reflects reduced modulation by expected value during choice. Poorer representation of expected value will lead to poorer decision making—the individual will be less likely to choose optimally because he or she will be less able to represent how rewarding the available choices are (21). Indeed, behaviorally, our disruptive behavior disorders group showed less of a relationship between choice behavior and expected value signaling.

When not responding to stimuli, healthy comparison youths showed anterior insular cortex activity that increased as a function of expected value. This indicates that activity within this region is modulated according to the level of inappropriateness of the action (the higher the expected value associated with the response, the more the participant should want to choose, not refuse, the object). In other words, this region was sensitive to the conflict between the avoidance response initiated and the level of nonoptimality of this response (as a function of expected value). Consistent with the finding of reduced expected value sensitivity within the ventromedial prefrontal cortex when choosing objects, the disruptive behavior disorders group showed reduced sensitivity to expected value within the anterior insular cortex when refusing objects. Interestingly, given the known responsiveness of the dorsomedial frontal cortex during conflict monitoring (29), one might have hypothesized that there would be group differences, modulated by expected value, in the dorsomedial frontal cortex when refusing objects. While this region was not one of our regions of interest, it should be noted that a large region of the dorsomedial frontal cortex revealed group differences when participants refused objects as a function of expected value (see Table S1 in the online data supplement).

In a previous study, we speculated that prediction error signaling was significantly disturbed in youths with disruptive behavior disorders (6, 9). However, in the absence of studies using model-based fMRI and distinguishing BOLD responses during decision making compared with feedback, this speculation had been impossible to formally test. Our results in the present study support the suggestion that impairment in caudate prediction error signaling, a critical function for successful decision making, is a component of the computational pathophysiology of disruptive behavior disorders. Positive prediction errors (when the rewards are greater than expected) are associated with increased activity (11) and are thought to result in increases to the expected value only associated with reward (10). In our study, healthy comparison youths showed significantly greater positive prediction error signaling than youths with disruptive behavior disorders. Poorer prediction error signaling for rewards in youths with disruptive behavior disorders should result in reduced learning of the expected value associated with stimuli, leading to poorer decision making.

Negative prediction errors (when the punishments are greater than expected) are typically associated with decreases in caudate responsiveness (11). This was seen in our healthy comparison group, but the disruptive behavior disorders group showed the opposite pattern (i.e., significant increases in caudate activity as a function of negative prediction errors). This echoes previous reports of increased caudate/ventromedial prefrontal cortex activity in response to unexpected punishment (8, 9). However, the present data formally demonstrate that this unexpectedly increased activity relates to a lack of appropriate modulation by negative prediction error in youths with disruptive behavior disorders.

The reason for increased, as opposed to decreased, activity in the caudate as a function of prediction error for punishment is unclear. Prediction error signaling is thought to depend on the dopamine system (11, 30, 31). Disrupted prediction error signaling in youths with disruptive behavior disorders would suggest dysfunction in the dopamine system in this disorder. Importantly, our data suggest something more complicated than a simple deficit model. Prediction error-modulated signaling in response to punishment in youths with disruptive behavior disorders was not reduced but fundamentally different from that seen in healthy youths. The disruptive behavior disorders group showed significant positive prediction error-related signaling in response to unexpected punishment, not the significant negative prediction error-related signaling seen here in healthy youths and previously in healthy adults (11). Currently, it is unknown what might cause such a fundamental reorganization of prediction-error punishment signaling.

It should be noted that two predictions were not confirmed by our study. First, we did not find group differences in the modulation of the ventromedial prefrontal cortex by prediction error for either rewards or punishments. Looking at healthy youths, significant modulation of the ventromedial prefrontal cortex activity was seen only for positive prediction errors at a high threshold (p=0.04). Thus, failure to observe group differences in prediction error signaling within the ventromedial prefrontal cortex may reflect specific paradigm parameters limiting activation in the ventromedial prefrontal cortex during feedback in this study.

Second, activity within the regions showing dysfunctional modulation by expected value during decision making and by prediction error during feedback was not moderated by the level of callous-unemotional traits (or impulsivity/narcissism) in the disruptive behavior disorders group. This contrasts with findings that levels of callous-unemotional traits modulate neural responses in youths with disruptive behavior disorders, for example, amygdala responsiveness to fearful expressions (32–34). This may reflect type II error, perhaps relating to insufficient variability of callous-unemotional traits/impulsivity/narcissism in our sample. Alternatively, computational impairments associated with decision making may be seen in disruptive behavior disorders independently of callous-unemotional traits. Additional research is needed to distinguish between these two possibilities.

Three caveats should be considered with respect to these data. First, we did not include an ADHD comparison group. This was because previous work has indicated that youths with ADHD do not present with the pathophysiology found in youths with disruptive behavior disorders (7, 9, 33). Moreover, and mitigating this limitation, it is important to note that our subsequent group analysis excluding disruptive behavior disorders youths with comorbid ADHD revealed extremely similar results, with proximal activations for all contrasts. Second, the medications for two youths in our disruptive behavior disorders group could not be withheld at the time of scanning. However, again mitigating this limitation, the results of our subsequent analysis of variance excluding these participants again identified proximal regions for all contrasts. Third, the disruptive behaviors disorders and comparison groups significantly differed in terms of their IQs. Mitigating this limitation, the results of the initial analysis survived post hoc correction in the ANCOVA analysis of the modulated data from each functional region of interest. In addition, subsequent analyses with an IQ-matched sample again identified proximal regions showing the same significant effects.

In summary, youths with disruptive behavior disorders showed significant disruption in the appropriate modulation of ventromedial prefrontal cortex and insula activity by expected value and caudate activity by prediction error (and, consistent with this, significantly less association between expected value and their behavior than healthy comparison subjects). We believe that these impairments contribute to an individual’s increased risk for goal-directed antisocial behavior. Appropriate expected value signaling is vital in making good behavioral choices. Appropriate prediction error signaling is vital to an individual’s ability to generate appropriate expected values. Disruption in expected value signaling (whether directly or as a result of disrupted prediction error signaling) will lead to poor behavioral choices and, in specific contexts, antisocial behavior (the individual will be less likely to avoid an action associated with potential negative consequences).

These findings have important implications for theory and treatment. Impairments in youths with disruptive behavior disorders have generally been conceptualized in terms of deficits (e.g., 21, 35). However, our data suggest that simple deficit models may be insufficient with regard to the decision-making impairment. Instead, it is possible that the dopaminergic system is profoundly altered in its functional organization. If robust, these findings suggest that interventions need to be designed with this disorganization in mind. For example, treatments simply augmenting responsiveness of target systems are not supported; reward prediction error signaling might be improved, but the impairment in punishment prediction error might be exaggerated, with potentially detrimental developmental effects.