An fMRI Study of Facial Emotion Processing in Patients With Schizophrenia

Impaired emotional functioning, considered fundamental in schizophrenia (1), manifests in flat, blunted, inappropriate affect and in depression (2). Studies have reported emotion processing deficits in identification, discrimination, and recognition of facial expressions (3, 4). This literature has also examined relations between cognition and emotion, with some evidence that emotion processing deficits are not specific but may represent generalized cognitive impairment (5, 6). Notwithstanding the issue of specificity, studies have shown that emotion processing deficits uniquely relate to symptoms (7, 8) and neurobiological measures (8).

Animal and human investigations of cerebral networks for emotional behavior have implicated the limbic system, primarily the amygdala, hypothalamus, mesocorticolimbic dopaminergic systems, and cortical regions, including the orbitofrontal, dorsolateral prefrontal, temporal, and parts of the parietal cortex (9, 10). Functional neuroimaging has probed the neural circuitry that modulates emotion processing in healthy people during emotion discrimination (11) and mood induction (12) and when presented with evocative stimuli (13).

Facial expressions, salient in emotional behavior, were used in conjunction with functional magnetic resonance imaging (fMRI) to investigate neural circuitry involved in emotion processing. Several paradigms were applied, including mood induction (14), facial matching and emotion identification (15), and comparison of facial and vocal expressions of fear and disgust (16). An increase in activity in the amygdala has been consistently observed across paradigms, particularly when the emotional valence (i.e., whether the emotion is positive [happy] or negative [sad, disgusted]) of the faces was salient. Cognitive or evaluative processing of facial expressions seem to attenuate amygdala response (15, 16) and possibly shift activation leftward (17).

The study of emotion processing with functional imaging in patients with schizophrenia is limited. In a mood induction fMRI study, patients did not show normal increases in amygdala activity during sad affect, despite similar self ratings (18). In another fMRI study, patients showed less activation and were less accurate at identifying emotions (19). Finally, in a task requiring attribution of mental states, patients had more errors and less fMRI signal in the left inferior frontal gyrus (20).

We evaluated whether activation of the amygdala is related to the relevance of the emotional valence of stimuli in healthy people. Blood-oxygen-level-dependent (BOLD) responses were measured while viewing facial displays of emotions (21). The task alternated between a simple emotional valence discrimination and age discrimination. Limbic response was greater during the emotional valence than the age discrimination condition, indicating amygdala response to the relevance of the emotional display. The response was pronounced in the amygdala but was also present in the hippocampus and other limbic regions.

Our goal was to examine the effects of facial emotion processing on activation of limbic and cortical regions in schizophrenia. Impaired emotion processing may relate to diminished amygdala response or to upstream failure to activate cortical components of the neural network. Because patients with schizophrenia perform more poorly than healthy people on complex emotion recognition tasks, administering complex tasks in fMRI would confound interpretation of activation differences. However, if the task is rudimentary, it will show differences in the activated circuitry that are precursors of impaired performance that cannot be attributed to performance effects. We therefore examined simple valence judgment, in which patients perform at par with healthy subjects. We tested the hypothesis that limbic activation, related to emotional relevance of facial stimuli, is diminished in patients with schizophrenia.

Method

Participants

Fourteen patients with schizophrenia and 14 healthy participants were consecutive volunteers for brain-behavior studies at the Schizophrenia Research Center of the University of Pennsylvania. There were 10 men and four women in each group; all subjects were right-handed. Participants underwent a medical examination, laboratory tests, and psychiatric assessment (22). For patients, assessment included the patient version of the Structured Clinical Interview for DSM-IV (SCID) (23) and scales for measuring symptoms and functioning, administered by investigators trained to a high criterion reliability (intraclass correlation [ICC]=0.90). Criteria for entry into the study were a diagnosis of schizophrenia, no concomitant axis I or axis II disorder, and no medical or neurological event or disorder that might affect brain function. Patients were studied while clinically stable (13 were outpatients). Negative symptoms, as per global ratings on the Scale for the Assessment of Negative Symptoms (24) were mild (mean=1.6, SD=1.3, range=0–4.8) as were positive symptoms (per the Scale for the Assessment of Positive Symptoms [25]: mean=0.5, SD=0.5, range=0–1.8) and symptoms of depression (per the Hamilton Depression Rating Scale [26]: mean=6.0, SD=4.7, range=0–12). All patients, except for one neuroleptic-naive participant, had been treated with antipsychotic medications. Ten received olanzapine (mean dose=14.0 mg/day, SD=9.3, range=5–30), and one patient each received risperidone (3 mg/day), clozapine (150 mg/day), and haloperidol (4 mg/day).

Healthy participants, recruited through an advertisement in newspapers, underwent the same screening procedures and were assessed with the SCID nonpatient version (27). They had no history of illness affecting brain function or a history of major psychiatric illness in first-degree relatives. Patients and comparison subjects were matched sociodemographically in terms of age (mean=28.8 years [SD=8.9, range=18–47] and 27.4 years [SD=7.3, range=18–44], respectively) and parental education (mean=13.8 years [SD=2.9, range=8–17] and 14.1 years [SD=2.2, range=10–17]). As expected, patients attained less education (mean=12.9 years [SD=2.7, range=9–18]) than did the comparison subjects (mean=15.6 years [SD=2.6, range=10–18]) (t=2.64, df=26, p<0.02). After complete description of the study to the subjects, written informed consent was obtained.

Facial Stimuli

The stimuli were color photographs of actors and actresses who participated in a study on emotion. The procedure for generating these stimuli has been previously described (28). Briefly, professional actors were coached by theater directors to express five emotions (happiness, sadness, anger, fear, disgust), each at three levels of intensity and under posed and evoked conditions. A subset of stimuli was selected for the fMRI study based on a high degree (>80%) of identification accuracy by raters. Stimuli consisted of 90 faces (50% male, 50% female). The actors were Caucasian (58.9%, N=53), African American (23.3%, N=21), Asian (6.7%, N=6), or Hispanic (11.1%, N=10). For each of the five emotions there were 12 expressions (two examples of posed and evoked emotion at three levels of intensity), and there were 30 neutral faces.

Procedures

Participants were placed supine in the scanner, wearing earplugs to muffle noise. Head fixation was assured through a foam-rubber device mounted on the head coil. Subjects viewed target stimuli through a mirror mounted inside the gantry. The stimuli were rear projected onto a screen by using a computer-activated LCD projection system. Scanning triggered the presentation of a crosshair (resting baseline condition) for 60 seconds before the first block. Another 40 seconds of the resting baseline condition was presented after each block. The initial resting baseline was followed by a 120-second block of emotional valence discrimination, a 120-second block of age discrimination, and then a second 120-second block of emotional valence discrimination (boxcar design). Each block consisted of 30 faces presented at 4-second intervals: 20 expressed emotions (four from each of the five emotions) and 10 neutral faces. A crosshair block of 40 seconds indicated termination of the task, as illustrated in Figure 1.

Task administration was triggered by the scanner and coupled to image acquisition by using the PowerLaboratory platform (29) on a Macintosh Powerbook laptop computer. A color-coded response keypad made of nonferromagnetic components (the fORP [Fiber-Optic Response Pad], Current Designs Inc., Philadelphia) was placed in the participant’s lap. In the emotional valence discrimination condition, participants were asked to determine whether the emotion was positive (left button press) or negative (right button press). In the age discrimination condition, participants were asked to determine whether the face was younger (14 faces, left button press) or older (16 faces, right button press) than 30. Hand used for responding alternated between subjects. A practice session with different faces was conducted to ensure optimal performance (not available for two participants [one patient, one comparison subject]).

fMRI Measurement

Data were acquired by using BOLD contrasts in a 4-T GE Signa Scanner (Milwaukee, Wis.), employing a whole-head coil. Structural images consisted of a sagittal T₁-weighted localizer followed by a T₁-weighted acquisition of the entire brain in the axial plane (24-cm field of view, 256×256 matrix, resulting in voxel size of 0.9375×0.9375×4 mm). This sequence was used for spatial normalization to a standard atlas (30) and for anatomic overlays of the functional data. Functional imaging was performed in the axial plane by using multislice gradient echo-echo planar imaging with a field of view of 24 (frequency) × 15 (phase), and an acquisition matrix of 64×40 (22 slices, 4-mm thickness, no skip, TR=2000 msec, TE=21 msec). The 22 slices were acquired from the superior cerebellum up through the frontal lobe. This corresponded inferiorly to a level just below the inferior aspect of the temporal lobes and superiorly to approximately the level of the hand-motor area in the primary motor cortex. This sequence delivered voxel resolution of 3.75×3.75×4 mm.

Since the gradient echo-echo planar images used for fMRI can be degraded in the presence of nonuniform magnetic fields, we paid special attention to the image quality in the anterior medial temporal lobes. First, we used an automated shimming program to adjust the background gradients in the field to make it more uniform within a manually chosen region of interest containing the anterior medial temporal lobes (31). After the shimming, pilot echo planar images were obtained, which were visually inspected before fMRI acquisition to ensure good image quality in the amygdala region. Second, the images were corrected for residual geometric distortion (32) on the basis of a magnetic field map acquired with a 1-minute reference scan.

Image Processing and Statistical Analysis

Functional data were preprocessed by using MedX 3.3 (Sensor Systems, Inc., Sterling, Va.). Images within each run were motion corrected (33) to the image in the middle of the run (intrarun realignment). After global intensity scaling (34), the images were band-pass filtered (Butterworth filter, 6–80 seconds) and smoothed (8-mm full width at half maximum, isotropic) with a kernel based on twice the in-plane resolution at which the data were acquired. The kernel size was chosen to optimize sensitivity (35) and to account for anatomic variability. Transformation to Talairach space (30) occurred in two steps. The first used a surface registration method (36), with contours drawn on the realignment reference image and on the T₁ axial localizer. A least squares fitting algorithm then registered the raw functional image to the localizer. This step accounted for movement between acquisition of the localizer image and the functional data. The second transformation was created by selecting commissural landmarks on the T₁ localizer and a polynomial Talairach transform with trilinear interpolation.

A multisubject analysis was performed by using a two-stage random-effects approach, which takes into account intersubject variance permitting population-level inferences. First, statistical parametric maps were generated by using the general linear model within SPM 99 (37). A 6-second time-shifted boxcar waveform was used as the reference paradigm, which modeled for emotional valence and age discrimination blocks relative to the crosshair resting baseline condition. For the group analysis, paired within-group t tests were performed on subtractions of beta weights contrasting two conditions at each voxel. Subtractions were emotional valence discrimination minus crosshair resting baseline condition (each block separately), age discrimination minus crosshair resting baseline condition, and emotional valence discrimination minus age discrimination. Voxels showing significantly greater response to emotional valence than to age discrimination conditions were examined in the group contrasts between patients and comparison subjects. Both “patients minus comparison subjects” and the opposite contrasts were performed.

Results

As expected, participants did not have difficulty performing the emotional valence discrimination task. Healthy participants averaged 35.1 (SD=3.1) out of 40 possible correct responses (87.8%), and patients averaged 33.3 (SD=4.2) (83.3%), which was not a significant difference (t=1.23, df=24, p<0.24). Reaction time was slightly faster for healthy participants (mean=1489.4 msec, SD=329.2) than for patients (mean=1530.6 msec, SD=346.3), but the difference was not significant (t=1.81, df=24, p<0.09). The age discrimination task was more difficult for both groups, with healthy participants averaging 18.1 (SD=2.8) out of 30 possible correct responses (60.3%) and patients averaging 15.2 (SD=3.4) (50.7%), a nonsignificant difference (t=0.94, df=24, p<0.36). Corresponding mean reaction times were 1175.0 msec (SD=275.4) for healthy participants and 1384.4 msec (SD=325.5) for patients, a nonsignificant difference (t=1.77, df=24, p<0.09).

In healthy participants, z maps indicated several voxels with significant increases for tasks relative to the crosshair resting baseline condition. For both emotional valence discrimination blocks, responsive voxels were observed in the amygdala and hippocampus (Table 1, Figure 2). The age discrimination task also showed some amygdala and hippocampal response but suggested greater response in cortical and posterior regions. All tasks showed activation in the fusiform gyrus, occipital lobe, and inferior frontal cortex. In the schizophrenia patients, minimal focal response was observed for all tasks relative to the baseline condition. Lack of focal response could reflect either overall diminished activity or failure of activity to be distributed in specific regions. To test whether the lack of focal activation in the schizophrenia patients was related to overall diminished activation, we counted the number of activated voxels throughout the imaged brain. Patients and comparison subjects activated a similar number of voxels at all levels above a beta of 0.1.

To identify voxels uniquely activated by the emotional valence discrimination task, we subtracted the age discrimination block from each of the emotional valence discrimination blocks. The subtraction images in healthy participants (Figure 3) showed responsive voxels in the amygdala and hippocampus for both blocks. For the first block, peaks were observed in the amygdala bilaterally and in the right hippocampus, whereas for the second block the amygdala activation remained significant only in the left while hippocampal activation became bilateral (Table 2). Patients showed voxels responsive to this contrast only in the hippocampus in the first emotional valence discrimination block and minimal amygdala response in the second block.

Contrasting patients and comparison subjects on the emotional valence discrimination task (both blocks combined) revealed voxels in the left amygdala and bilateral hippocampus that activated significantly more in comparison subjects (Table 3). A group (patients, comparison subjects) by hemisphere (left, right) 2×2 multivariate analysis of variance with repeated measures was performed on the beta contrast values and revealed a significant diagnosis-by-hemisphere interaction (F=6.75, df=1, 26, p<0.02). Healthy participants showed greater left hemispheric amygdala response for both blocks, while patients showed minimal left hemispheric activation for either block. Individual differences in activation did not correlate with symptom severity measures in patients.

Discussion

Patients with schizophrenia showed reduced activation of the left amygdala and bilateral hippocampus in response to a task requiring discrimination of positive from negative facial affect valence. The finding of left amygdala activation in healthy participants is consistent with reports of greater involvement of the left amygdala in evaluative aspects of emotion processing (17). The failure of clinically stable patients to activate limbic regions for a simple valence discrimination task, on which they perform at par with their healthy counterparts, is noteworthy. It suggests that the circuitry recruited to process the task is not adequately mobilized in individuals with schizophrenia. While the lack of such recruitment did not affect performance on the simple valence discrimination task, it could be speculated that such failure of recruitment would cause difficulties with more challenging affect discrimination tasks (2–6). Furthermore, activation of hippocampal regions during emotional valence discrimination may relate to memory facilitation (38), and patients’ failure to activate the hippocampus could hypothetically contribute to memory deficits (39).

The present study did not find correlations between activation and clinical features. Such correlations could require a wider range of symptom severity and perhaps more difficult tasks. Greater task difficulty would also permit establishment of correlations between activation and performance (40). Interpretation of such correlations could be facilitated by the present results, since increasing task difficulty will also produce group differences in performance. A limitation of the present study is that the design combined several emotions as negative. Future studies can use blocks of specific emotions, or event-related designs (35), to help identify whether amygdala response to emotional valence discrimination varies for the different emotions. Alternative designs would also be needed to establish the specificity of the reported effect. For example, amygdala activation has been noted in response to emotional faces in the absence of any particular task requirements (16). In the present study we purposefully equated the number of emotional faces for the two tasks and included neutral faces in both conditions. Furthermore, since only one emotion (happy) is positive, there was an imbalance between the number of positive and negative emotions. However, this limitation reflects on the endemic lack of distinct positive emotions that can be reliably expressed by humans (41). Specific designs are needed to establish whether the diminished limbic response of patients occurs already at the stage of confronting an emotional face. Another limitation of the study is that the age discrimination task was more difficult than the emotional valence discrimination task. Manipulating task difficulty for the age discrimination condition can be achieved by creating a gap in the age range of the stimuli. Our study is also limited in that the small number of women did not permit evaluation of sex differences. Finally, the patients were medicated at the time of study, and possible effects of medication merit further scrutiny. Thus far, fMRI studies in schizophrenia have not examined medication effects. However, studies with other methods for measuring cerebral blood flow suggest that neuroleptics do affect activation patterns (42).

Notwithstanding its limitations, the present study establishes diminished limbic response in patients with schizophrenia for an emotion processing task and thereby encourages further examination of this area. Abnormal activation in response to emotion processing demands may help explain aspects of the pathophysiology of schizophrenia, where deficits are prominent in interpersonal relatedness and social functioning. Emotion processing deficits in schizophrenia are receiving greater attention (2–6), and linking such deficits to neural activation may help place them in the context of cognitive deficits and symptoms. Understanding activation of brain circuitry during emotion processing can help in designing methods of intervention, and activation measures can be used to evaluate their effects.

TABLE 1

Enlarge table

TABLE 2

Enlarge table

TABLE 3

Enlarge table