Abnormal Modulation of Amygdala Activity in Schizophrenia in Response to Direct- and Averted-Gaze Threat-Related Facial Expressions

Impaired emotion recognition is a core domain of social cognitive dysfunction in schizophrenia and is strongly linked to functional outcome (1). One possible mechanism for this impairment is abnormal activation of a neural network centered on the amygdala (2–4). Studies examining amygdala functioning in schizophrenia largely report hypoactivation, and a recent meta-analysis supports findings of overall reduced amygdala activation in response to facial emotion processing 5). More nuanced investigations, however, have begun to expose complex patterns of functioning such that in some conditions (e.g., incorrectly labeling fear) schizophrenia patients show hyperactivation relative to healthy comparison subjects, and in others (e.g., correctly labeling anger) they show hypoactivation (6). Such findings confirm that amygdala responses in schizophrenia can be modulated by threat-related expressions, and it appears that these modulated relationships show the greatest associations with disease variables such as flat affect (6) and level of social functioning (7). These links to functional outcome provide a compelling argument for the identification of additional factors that could potentially modulate amygdala functioning in schizophrenia.

The gaze direction of threat-related facial expressions has been found to modulate both emotion recognition accuracy and amygdala response in healthy individuals. Behaviorally, and perhaps because of differing levels of self-relevance (8), emotion recognition accuracy is greater for direct-gaze anger and averted-gaze fear expressions than for averted-gaze anger and direct-gaze fear expressions (9–11; see reference 12 for an exception). Angry faces with a direct gaze are also rated as more intense than angry faces with an averted gaze, and the opposite is true for fear expressions (8, 13). Finally, detection of gaze direction is faster and more accurate when direct gaze is paired with angry expression and averted gaze is paired with fearful expression (14).

With regard to amygdala activation, the earliest report (15) noted that direct-gaze fear and averted-gaze anger expressions produced greater amygdala responses than did their opposite-gaze counterparts. These initial results were interpreted to suggest that the amygdala is particularly sensitive to the ambiguous levels of threat presented by an angry face that is looking away and a fearful face that is making direct eye contact. While subsequent work has raised questions about the true direction of this effect, differential amygdala responses depending on emotion and gaze direction continue to be reported, thus highlighting an important interaction between gaze direction and emotion perception (16–18).

To our knowledge, only one study has examined neural correlates of gaze detection in schizophrenia, and no studies have examined the combined effects of threat-related emotion and gaze direction. Kohler and colleagues (19) manipulated gaze direction of faces expressing neutral emotional states and asked healthy volunteers and patients with schizophrenia to determine whether the face was looking at them or looking away. Despite similar behavioral performances, patterns of amygdala activation varied between groups. Whereas healthy individuals did not show an effect of gaze direction in the amygdala, patients with schizophrenia showed greater left amygdala activation to direct gaze relative to averted gaze.

We sought to further these investigations by examining whether gaze direction of threat-related faces would evoke differential amygdala response in individuals with schizophrenia. Healthy volunteers and schizophrenia patients completed an emotion recognition task of fear and anger expressions with direct and averted gazes while undergoing functional neuroimaging. We hypothesized, first, that across the task as a whole, patients would show reduced amygdala activation relative to healthy comparison subjects. Second, based on the results of Kohler et al. (19), we tentatively hypothesized that patients would show greater amygdala activation to both anger and fear in direct-gaze expressions and therefore show less amygdala modulation as compared to healthy volunteers. Finally, while amygdala activation was the primary focus of this investigation, we also conducted exploratory analyses to identify other brain regions showing sensitivity to gaze manipulations of threat-related faces.

Method

Participants

The original sample included 40 healthy comparison subjects and 46 patients diagnosed with schizophrenia or schizoaffective disorder. Twelve individuals (three comparison subjects and nine patients) were excluded from further analysis because of poor image quality and excessive motion artifact (motion >4 mm). Two other patients were excluded for inadequate coverage of the imaging area and poor task performance (nonresponses >70%). The final sample included 37 healthy comparison subjects and 35 schizophrenia patients (31 with schizophrenia and four with schizoaffective disorder). The sample characteristics are summarized in Table 1. The groups did not differ significantly in gender, handedness, ethnicity, age, or maternal or paternal education level. As expected, the groups differed in education level (t=3.05, df=69, p=0.003), with comparison subjects achieving higher levels than patients. All participants were volunteers at the Schizophrenia Research Center of the University of Pennsylvania Medical Center, and all provided written informed consent after receiving a full description of the study procedures. The University of Pennsylvania ethics review board approved the study.

Diagnoses were confirmed with the Diagnostic Interview for Genetic Studies (20) and self-reported demographic and medical history information. To be eligible for the study, patients had to have a diagnosis of schizophrenia or schizoaffective disorder, depressed type, and no other current axis I or II diagnoses. Comparison subjects also completed all assessment procedures to ensure that they did not currently meet criteria for any axis I or II disorders, never met criteria for a psychotic disorder, and did not have any first-degree family members with a psychotic illness. For both groups, exclusion criteria were current substance use, abuse, or dependence (except nicotine); history of head injury; and any medical conditions known to affect brain function (e.g., uncontrolled hypertension, diabetes mellitus, history of seizures).

Assessments of symptom severity in the patient group using the Scale for the Assessment of Negative Symptoms (SANS; 21) and the Scale for the Assessment of Positive Symptoms (SAPS; 22) identified relatively mild symptom levels. Global ratings on these instruments averaged 1.42 (SD=0.72, range=0–3.0) and 1.04 (SD=1.04, range=0–3.5), respectively. Level of social and occupational functioning was assessed with the Strauss-Carpenter Outcome Scale (23). As expected, patients showed significantly lower levels of functioning than did comparison subjects (t=7.36, df=61, p<0.001). Six patients were receiving stable dosages of first-generation antipsychotics (chlorpromazine equivalents, mean=300.0 mg/day [SD=181.55]); 26 patients were on stable dosages of second-generation antipsychotics (chlorpromazine equivalents, mean=405.0 mg/day [SD=318.77]) (24). Information on medication was unavailable for three patients.

Imaging Stimuli and Task

During scanning, participants were presented with 72 faces individually and asked to identify the emotion expressed by each face. Faces displayed anger, fear, or no emotion, with 24 in each expression category; of these 24 faces, 12 were presented with a direct gaze so that they appeared to be looking directly at the participant, and 12 were presented with a gaze averted by 8 degrees (six to the right and six to the left) so that they appeared to be looking away from the participant.

The task used an event-related design, with each face displayed for 5.5 seconds. Faces were followed by a variable interstimulus interval of 0.5–18.5 seconds in which participants fixed on a complex crosshair comprising a scrambled face image with a crosshair fixation point centered in the display. Total task duration was 12.55 minutes. (Additional information about the task is presented in the data supplement that accompanies the online edition of this article.)

Imaging Procedures

Earplugs were used to muffle scanner noise, and head fixation was aided by foam-rubber restraints mounted on the head coil. Stimuli were rear-projected to the center of the visual field using a PowerLite 7300 video projector (Epson America, Long Beach, Calif.) and viewed through a mirror mounted on the head coil. Stimulus presentation was synchronized with image acquisition using the Presentation software program (Neurobehavioral Systems, Albany, Calif.).

Image Acquisition

Blood-oxygen-level-dependent (BOLD) functional MRI (fMRI) was acquired with a Siemens Trio 3-T (Erlangen, Germany) system with an eight-channel head coil and the following parameters: repetition time=3,000 msec, echo time=32 msec, field of view=240 mm, matrix=128×128, slice thickness=2 mm, gap=0 mm, 30 slices, voxel size=1.875×1.875×2 mm. To reduce partial volume effects in orbitofrontal regions, echo planar images were acquired obliquely (axial/coronal). The slices provided coverage of the temporal lobe and inferior frontal lobes, with good coverage of the ventral regions through the orbitofrontal lobes, midbrain, and fusiform gyrus. This slab acquisition allowed for excellent resolution through our a priori region of interest in the amygdala. A 5-minute magnetization-prepared, rapid acquisition gradient-echo T₁-weighted image (repetition time=1,630 msec, echo time=3.87 msec, field of view=180×240 mm, matrix 192×256×160, voxel size 0.94×0.94×1 mm) was collected for spatial normalization and overlays of functional data.

Statistical Analysis

In order to remain consistent with previous work and to focus our analyses on the threat-related emotions of anger and fear, behavioral and neural responses to neutral faces were included as a covariate of no interest in their respective analyses. Previous work suggests that neutral faces may contain unintended, and variable, emotional significance, and indeed, several studies have demonstrated that neutral faces are commonly perceived as containing some emotional content (25–27).

Behavioral Performance

Response accuracy was assessed with a repeated-measures analysis of variance (ANOVA). Stimulus emotion (anger or fear) and gaze (direct or averted) were entered as within-subject factors, and group (comparison group or patient group) was entered as the between-subjects factor. Analysis of response time is provided in the online data supplement.

Image Analysis

Preprocessing and analyses of fMRI data were performed with FEAT (fMRI Expert Analysis Tool), version 5.98 (from FMRIB's Software Library [FSL], www.fmrib.ox.ac.uk/fsl). Images were slice-time corrected, motion-corrected to the median image using a trilinear interpolation with six degrees of freedom (28), high-pass filtered (100 seconds), spatially smoothed (4-mm full width at half maximum, isotropic), and grand-mean scaled using mean-based intensity normalization. FMRIB's Brain Extraction Tool was used to remove non-brain areas (29). The median functional and anatomical volumes were coregistered and then transformed into the standard anatomical space (T₁ Montreal Neurological Institute template, voxel dimensions of 2×2×2 mm) using trilinear interpolation.

Subject-level time-series statistical analysis was carried out using FMRIB's Improved General Linear Model with local autocorrelation correction (30). The four condition events (direct-gaze anger, averted-gaze anger, direct-gaze fear, and averted-gaze fear) were modeled in the general linear model after convolution with a canonical hemodynamic response function and its temporal derivative. Six rigid body movement parameters were also modeled as nuisance covariates along with two other regressors (direct-gaze neutral and averted-gaze neutral). The resulting contrast images revealing task-dependent activation relative to fixation baseline were then used in group-level analyses.

To examine activation in our a priori region of interest, the right and left amygdala were first anatomically defined using the WFU (Wake Forest University) PickAtlas utility (31). Mean percent signal change in both regions of interest was then estimated using the model described above, and these values were entered into a repeated-measures ANOVA with hemisphere (right versus left), emotion (anger versus fear), and gaze (direct versus averted) as within-subject factors and group (comparison subjects versus patients) as the between-subject factor. The threshold for statistical significance was set at p<0.05 (two-tailed), and significant interactions were probed with follow-up t tests.

To address our exploratory aim of examining additional brain regions showing sensitivity to the interaction of emotion and gaze across groups, a mixed-effects analysis using FMRIB's local analysis of mixed effects was performed to conduct a 2×2×2 (group by emotion by gaze) voxel-wise ANOVA on subject-level contrasts across the entire slab acquisition. Cluster-level correction of p<0.01 was used for all main effects and two-way interactions. The main effect of group produced robust effects; thus, for clarity, results for this contrast are presented at a more conservative threshold of p<0.05 (family-wise error corrected, cluster >50). For the three-way interaction of emotion, gaze, and group, a less stringent cluster-level correction of p<0.05 was applied. For these analyses, findings pertaining to the main effect of group or interactions with group are presented below. All other within-subject findings are presented in the online data supplement.

Results

Behavioral Performance

For response accuracy, both the main effect of group (F=7.96, df=1, 68, p=0.006) and the emotion-by-gaze interaction (F=5.44, df=1, 68, p=0.023) were significant. As expected, on the task as a whole, accuracy was greater in the comparison group (mean=82.6%, SD=11.85) than in the schizophrenia group (mean=73.8%, SD=13.25). There was also an interaction between emotion and gaze, such that direct-gaze anger (mean=82.3%, SD=16.08) and averted-gaze fear (mean=81.3%, SD=17.99) were better recognized than averted-gaze anger (mean=72.7%, SD=20.08) and direct-gaze fear (mean=76.7%, SD=14.73). This interaction did not differ by group (Figure 1).

Analyses of BOLD Signal Change in Amygdala

The repeated-measures ANOVA on mean percent signal change revealed a statistically significant main effect for hemisphere (F=8.41, df=1, 70, p=0.005), indicating greater left amygdala responsivity (mean=0.856, SD=0.331) as compared to the right amygdala (mean=0.775, SD=0.283). A significant main effect was also evident for emotion (F=9.57, SD=1, 70, p=0.003), such that fear expressions (mean=0.859, SD=0.318) evoked greater amygdala responses than did anger expressions (mean=0.773, SD=0.297). Greater amygdala activity was also seen in response to averted-gaze relative to direct-gaze expressions, but this main effect fell short of statistical significance (F=3.11, df=1, 70, p=0.082). Notably, the main effect of group across both regions of interest combined was not significant (F=2.381, df=1, 70, p=0.13). More importantly, however, the three-way interaction between emotion, gaze, and group was significant (F=5.19, df=1, 70, p=0.026 (see Figure 1). Follow-up t tests revealed that this interaction was largely driven by significantly reduced amygdala responses to direct-gaze anger expressions in the patient group. Indeed, patients showed significantly lower signal change for direct-gaze anger expressions than for all other conditions (p<0.01 for all comparisons), and within conditions, they significantly differed from comparison subjects only for direct-gaze anger expressions (t=2.81, df=1, 70, p=0.006), although it should be noted that the between-group comparison for averted-gaze fear expressions approached significance (t=1.72, df=1, 70, p=0.090). Somewhat surprisingly, despite a pattern of differences in comparison subjects that was partially consistent with previous reports, no direct comparisons of amygdala response across conditions reached statistical significance. No other interactions were significant.

Voxel-wise Comparisons Across the Acquisition Slab

Voxel-wise analyses of neural activation revealed a main effect of group, indicating that comparison subjects showed greater activation than patients in several regions typically associated with facial emotion processing, including the fusiform gyrus bilaterally, the right thalamus, the right inferior frontal gyrus, and the cerebellum. In contrast, patients showed greater activation than comparison subjects in the right middle temporal gyrus (Table 2; see also Figure S2 in the online data supplement).

Likewise, multiple regions showed a significant interaction between gaze and group such that comparison subjects showed greater activation to averted-gaze relative to direct-gaze expressions and patients showed an opposite pattern (a contrast of −1, 1, −1, 1 for comparison subjects and 1, −1, 1, −1 for patients for direct-gaze anger, averted-gaze anger, direct-gaze fear, and averted-gaze fear, respectively). These regions included the medial prefrontal cortex, including Brodmann's area 10; the left precentral gyrus; the left lingual gyrus; the left putamen; and the left inferior frontal gyrus (Figure 2). Finally, two clusters demonstrated a significant three-way interaction between emotion, gaze, and group (Figure 2). These clusters included the thalamus and caudate bilaterally and revealed opposite patterns of neural responses in comparison subjects relative to patients, such that comparison subjects showed greater activation to averted-gaze anger and direct-gaze fear relative to direct-gaze anger and averted-gaze fear, and patients showed greater activation to direct-gaze anger and averted-gaze fear relative to averted-gaze anger and direct-gaze fear. Coordinates are provided in Table 2.

Post Hoc Analyses

Given that amygdala responses in patients differed from those in comparison subjects only in response to direct-gaze anger expressions, we conducted post hoc correlational analyses to examine associations between amygdala activation to direct-gaze anger, symptom ratings, and level of functioning in the patient group. We averaged mean percent signal change estimates in response to direct-gaze anger expressions across hemisphere and then calculated bivariate correlations between these measures and the sum of the symptom subscale scores on the SANS and the SAPS and total score on the Strauss-Carpenter Outcome Scale. These relationships were generally weak (r values <0.25) except for a robust negative correlation between amygdala response and anhedonia (r=−0.564, p<0.001) and a robust positive correlation between amygdala response and level of functioning (r=0.587, p<0.001) (Figure 3). These correlations were unchanged when the potential effects of medication were controlled for.

Discussion

We investigated amygdala modulation during emotion recognition in patients with schizophrenia by manipulating gaze direction in threat-related expressions. Across the task as a whole, reductions in neural activation in patients were seen throughout a facial emotion processing network that includes the limbic and thalamic regions, the frontal regions, and the cerebellum. Results demonstrated a three-way interaction between emotion, gaze, and group in amygdala that was driven by significantly decreased activation in patients only to direct-gaze anger expressions. Contrary to previous reports demonstrating generalized amygdala hypoactivation during emotion processing (5), patients and healthy comparison subjects showed comparable levels of amygdala activation to all other stimulus categories. Given that directed anger expressions likely present the clearest indication of self-relevant threat, our findings highlight a specific deficit in facial emotion processing in patients with schizophrenia that may also be related to accurately assessing threat and determining self-relevance (8, 10, 17, 32). These findings are also supportive of previous work demonstrating abnormal amygdala modulation in schizophrenia and are consistent with our previous finding that paranoid patients fail to show greater amygdala activation in response to increasing levels of threat (7). In our previous work, degree of amygdala modulation showed a relationship with social functioning, and in the present study, the degree of amygdala activation to direct-gaze anger expressions was significantly correlated with level of functioning. This finding identifies a robust link between amygdala activation and measurements of social and occupational functioning that may indicate a potential neural basis for impaired functioning in schizophrenia.

In addition to the amygdala, several other regions previously implicated in emotion processing and social cognition showed significant interactions between stimulus characteristics and group. Most notably, both the medial prefrontal cortex and the inferior frontal gyrus demonstrated greater activation to averted-gaze relative to direct-gaze expressions in healthy comparison subjects and an opposite pattern in schizophrenia patients. The medial prefrontal cortex has been consistently linked to theory of mind (the ability to infer the thoughts or intentions of others) (33, 34), and this pattern in healthy subjects may reflect a greater effort to interpret the intentions of an individual who is looking away. Patients, however, show impairments in theory of mind (35, 36), which may be related to the abnormal pattern of activation seen here. Likewise, greater activation in the inferior frontal gyrus for healthy individuals in response to averted-gaze expressions may indicate the greater cognitive demand of processing more ambiguous stimuli. Finally, activations in the caudate and thalamus bilaterally showed a three-way interaction between emotion, gaze, and group. Both the caudate and the thalamus are implicated in processing fear-related stimuli (37, 38), and the caudate has been linked to emotional arousal (39). While tentative, these results may reflect abnormal early arousal responses to potential threat in individuals with schizophrenia and therefore require replication.

Reported neural interactions between stimulus characteristics and group were evident despite the lack of any group interactions in recognition accuracy. Both patients and healthy comparison subjects followed a pattern of behavioral response that is consistent with previous work (9–11). This finding may suggest some preserved ability in schizophrenia to integrate gaze cues and emotional signals. A recent study examining this effect in patients with unilateral amygdala damage reported that only right lobectomy patients showed an abnormal behavioral pattern, suggesting that the effect may be dependent on the right amygdala (10). While our patient group generally showed reduced activity relative to our comparison group, they did show right amygdala activity in response to all stimuli, which may have contributed to an intact normative pattern in emotion recognition accuracy, albeit with reduced accuracy overall.

This study has several limitations. First, while the pattern of neural activation to averted- and direct-gaze fear expressions in our healthy comparison subjects was consistent with previous findings (16, 17), we did not find a significant interaction of gaze and emotion in the amygdala. As N'Diaye and colleagues (17) have pointed out, this may be due to our use of high-intensity emotional expressions. N'Diaye et al. used both mild and high-intensity expressions and found a significant interaction between emotion and gaze in amygdala only for mild expressions. They concluded that the interactive effect of gaze on emotion recognition may be maximized only in conditions where emotional signals are relatively weak and can be enhanced with congruent shifts in gaze (i.e., averted gaze in a fearful expression resulting in a greater portion of exposed eye whites). Alternatively, in the present study we assessed explicit emotion recognition. It is possible that amygdala modulation differs during implicit processing and that modulation to fear is greater when emotion is not directly assessed (15, 16). Second, manipulation of our stimuli to create shifted gaze can be most accurately characterized as horizontal head tilt with concomitant averted gaze rather than a directed expression with averted gaze. While the majority of previous work on this topic has used the latter, work by Hess and colleagues (11) suggests that there may not be a meaningful distinction between the two presentations. Finally, the correlation between amygdala activation and functioning does not imply causality. Additional work will be needed to determine whether other factors (e.g., premorbid functioning, activation of additional brain regions) may account for this finding.

Notwithstanding these limitations, this study provides novel support for abnormal amygdala modulation in patients with schizophrenia that goes beyond generalized reduced activation. More importantly, amygdala activation showed a strong and positive correlation with social and occupational functioning, reinforcing the need for continued research investigating modulators of amygdala responses in schizophrenia.