Disturbed Facial Affect Recognition in Patients With Schizophrenia Associated With Hypoactivity in Distributed Brain Regions: A Magnetoencephalographic Study

Deficits in the recognition of facial expressions of emotion in schizophrenic patients are well known from several studies (1) across different cultures (2). Longitudinal studies have shown that the deficits remain stable over the course of the illness (3, 4) and that they are relatively unaffected by severity of acute symptoms. Furthermore, patients suffering from other psychiatric disorders (4–7) do not show these deficits. These findings are consistent with the assumption that facial affect recognition deficits might be critically associated with pathophysiological processes of the illness and that the neurophysiology underlying facial affect recognition and the neurophysiology underlying schizophrenic disorders might share a common basis. Since it is assumed that mental processes such as facial affect recognition are known to be based on a fast network of distributed brain regions (8), magnetoencephalography—with its combination of a high resolution in time and a relatively good resolution in space—offers a powerful method to study the cerebral substrates of such processes (9).

A number of studies on cerebral processing of facial stimuli in healthy subjects (10, 11) have shown that several cortical and subcortical structures are involved in evaluating specific information faces contain, such as emotional expression (12–15). Magnetoencephalography and EEG studies have provided evidence that information processing specifically related to faces might begin as early as about 110–130 msec after stimulus onset in occipital brain structures (16, 17). Evaluation of more complex facial information like recognition of a person’s identity (18) or recognition of facial affect (19) seems to begin after about 150 msec. Recently, Streit et al. (20) were able to show that schizophrenic patients, as compared to healthy subjects, exhibited diminished event-related potentials during facial affect recognition; the amplitudes peaked at about 230 msec. These potentials recorded at frontal electrode sites were associated with poorer task performance.

The main objectives of the present study were to identify localization and temporal characteristics of brain activations associated with facial affect recognition deficits in schizophrenic patients, which is expected to contribute to a better understanding of the pathophysiology of the disorder.

Method

Subjects

Fifteen inpatients with partly remitted DSM-IV schizophrenia (six women and nine men; mean age=34.5 years, SD=9.8) and a comparison group of 12 healthy subjects (seven women and five men; mean age=35.8 years, SD=11.6) participated in the study. Only right-handed subjects were included. At the time of the study all patients were receiving antipsychotic medication (mean dose=675 mg/day [SD=414] in chlorpromazine equivalents). Exclusion criteria for patients were substance abuse, neurological disorders, and treatment with additional anticholinergic medication or benzodiazepines. Healthy subjects were excluded for CNS-affecting disorders and substance abuse. After complete description of the study to the subjects, written informed consent was obtained. Symptoms were rated in patients by means of the Scale for the Assessment of Negative Symptoms (SANS) (21) and the Scale for the Assessment of Positive Symptoms (SAPS) (22). Positive symptoms were in the mild range (SAPS global score: mean=2.33, SD=2.55).

Magnetoencephalography and magnetic resonance imaging (MRI) were performed at the Institute of Medicine of the Research Center Jülich.

Task Design

Subjects performed three tasks. In the first task, Ekman and Friesen’s “pictures of facial affect” (23) were used to test categorization of facial expressions of emotion. We selected five faces (three male and two female) that each displayed the six basic emotions: happiness, fear, anger, surprise, disgust, and sadness. In the second task, subjects were asked to categorize complex objects, which included faces. Since this task was used to study processing of nonemotional items, only neutral faces from the aforementioned series were used. Photographs of five other categories of inanimate and natural objects served as additional stimuli (front views of motor trucks, chairs, and horse heads together with birds and flowers). Thirty different images per object category were selected. In the third task, subjects had to categorize blurred objects, which also included faces (for examples, see reference 20). This was used to study basic perception of faces while subjects were unable to recognize more complex information like emotional expressions. However, to guarantee that subjects were still able to recognize the faces and objects in the majority of cases, the stimuli were blurred only to a degree that healthy subjects in pilot studies recognized the faces and objects with an accuracy rate of about 80%. This procedure provided also a condition to control for higher demands on decision making, comparable to those in the affect recognition task.

All photographs were digitized and reworked to ensure uniformity of the general visual qualities. They were adjusted for luminance and mounted into the center of the same mid-gray background. Stimuli subtended a visual angle of about 11° × 15°.

Stimuli were presented in random order for 500 msec on a screen in a magnetically shielded room. Subjects were instructed to give their response verbally by selecting from a multiple-choice list containing the six facial emotions (task 1) or six object categories (tasks 2 and 3). The list appeared on the screen after a delay of 1 second to avoid confounding motor effects and was presented for a maximum of 8 seconds. A random interval of 1.6 to 2.2 seconds was used between the subject’s response and the onset of the next stimulus.

Tasks were conducted consecutively. Since the investigation of neurophysiological correlates of facial affect recognition was the focus of the study, a good reliability of the neuromagnetic response to facial expressions of emotion had to be particularly ensured. Therefore, subjects performed the affect recognition task twice in two separate runs.

Magnetoencephalographic Recording and Analysis

Data acquisition and signal processing

Magnetic fields were recorded continuously with a BTi 148-channel whole head magnetoencephalography system. Head movement was calculated with the BTi Magnes sensor position coil system. Therefore, head position was measured before and after each of the runs. The signals were sampled at 500 Hz and offline digitally high pass filtered above 1 Hz. Heartbeat and eye-blink artifacts were corrected with an independent component analysis algorithm (24). Intervals containing other types of artifacts were rejected from the analysis.

Consecutively, the signals were averaged for each run and category and filtered low pass 45 Hz, leading to a bandwidth of 1–45 Hz. Length of the analyzed epochs was 800 msec, starting 200 msec before stimulus onset. The 200-msec prestimulus interval was chosen as a baseline and subtracted from the data.

A 1.5-T Siemens vision system was used for measurements of the MRIs.

Source localization and data analysis

For the localization of the current sources, we used the magnetic field tomography package (25, 26). The magnetic field tomography solutions provide the spatial and temporal distribution of the primary current density within a subject’s brain. In particular, the magnetic field tomography algorithm is able to identify multiple distributed sources, which had to be expected in this experiment. The computations are performed within a source space defined directly from the MRI of each subject. To achieve this the head shape of the subject is digitized before each experiment together with the locations of three head-shape coils attached to the subject’s forehead and left and right pre-auricular points. The digital head shape is coregistered with the outline of the head defined from the MRI of the subject, and hence the head-shape coils are defined in the MRI system and the source space of the subject.

The magnetic field tomography solutions belonging to the same categories of stimuli (emotional faces, neutral faces, blurred faces, blurred trucks, etc.) were averaged. Thus, a maximum of 30 trials per average was achieved. The number of sufficient trials were not different between groups during the first affect recognition run (patients: mean=27.1 [SD=2.2], comparison group: mean=28.3 [SD=2.1]; t=–1.5, df=25, p=0.14) or the second (patients: mean=27.2 [SD=1.9], comparison group: mean=28.2 [SD=1.8]; t=–1.33, df=25, p=0.20). Patients and comparison subjects were also not different in head movement during the first affect recognition run (patients: mean=0.68 cm [SD=0.36], comparison group: mean=0.54 cm [SD=0.27]; t=1.11, df=25, p=0.28) or the second (patients: mean=0.47 cm [SD=0.19], comparison group: mean=0.53 cm [SD=0.13], t=–0.96, df=25, p=0.35).

To identify foci of high cerebral activity, current densities were displayed color-coded on the subject’s MRI for each of the time slices in the time range of 0–500 msec. Initially, the MR overlays were visually inspected for activations. (For each of the subjects the sagittal MRI planes were inspected stepwise with a spacing of 1 cm.) Thereafter, the strengths of the activations were calculated within a sphere of 1-cm radius around the center of the activations. The strength was defined as the modulus of the current density vector minus the prestimulus current density. An activation was defined as “strong” if the amplitude was ≥3 standard deviations above the prestimulus baseline. Finally, an area was specified as a region of interest when strong and reproducible (occurring in both affect recognition runs) activations were found at equivalent sites for at least 50% of the subjects. The same procedure was done with the responses to the neutral and blurred faces (except the use of the two-run criterion, since the corresponding tasks were only conducted once). However, additional regions of interest were not found. Time windows were defined around strong activations on the basis of a comparison of the latencies of these activations across each of the subjects. For each of the regions of interest we calculated the mean activation(s) within the specified time windows and the latencies of the activation maxima.

Statistics

Strength and latencies of activations at the different regions of interest were compared between diagnostic groups by means of separate analyses of variance (ANOVAs). Data obtained during the performance of the affect recognition task were computed by repeated measures ANOVAs with group and repetition (run) as factors. For the two control tasks, univariate ANOVAs were used. Independent Student’s t tests were conducted between healthy subjects and patients to determine whether the groups differed in behavioral task performance. Associations between magnetoencephalography data, task performance, symptoms, and medication were studied by Pearson’s correlation coefficients. All statistical analyses were two-tailed and evaluated for significance at the 0.05 alpha level.

Results

Behavioral Results

In terms of the number of correct responses, schizophrenic patients performed worse than healthy subjects in categorization of facial expressions of emotion in the first affect recognition run (mean=17.60 [SD=4.00] versus 20.25 [SD=2.56], respectively; t=1.99, df=25, p=0.06) and performed significantly worse in the second affect recognition run (mean=17.40 [SD=4.72] versus 22.17 [SD=2.25]: t=3.21, df=25, p=0.01). Performance in categorization of blurred faces among objects was not significantly different between groups (t=1.04, df=25, p=0.31). Accuracy in categorization of neutral faces was 100% in both groups.

Magnetoencephalographic Results

Subjects showed reliable regional activations at circumscribed latencies during categorization of facial expressions of emotion (an example is given in Figure 1). We found strong activations in 16 regions of interest within a total of 23 time windows (examples are shown in Figure 2). As shown in Table 1, 10 out of these 23 regional activations (in eight out of 16 regions of interest) were significantly stronger in the comparison group than in the patient group.

Group differences in peak latencies between the comparison subjects and patients were observed only in two regions of interest: the left orbitofrontal cortex within time window 1 (first affect recognition run: mean=127 msec [SD=24] versus 149 msec [SD=25], respectively; second affect recognition run: mean=135 msec [SD=27] versus 147 msec [SD=21]) (F=4.40, df=1, 25, p<0.05) and in the right inferior prefrontal cortex within time window 3 (first affect recognition run: mean=288 msec [SD=44] versus 332 msec [SD=52], respectively; second affect recognition run: mean=295 msec [SD=48] versus 325 msec [SD=58]) (F=5.34, df=1, 25, p<0.03). We are aware of the statistical weakness of finding differences in only two out of 23 comparisons, but we decided to mention this observation because in both cases the differences occurred in both of the affect recognition runs in the same regions of interest.

Correlations between current density and affect recognition performance (number of correct responses) were found in six out of 16 regions of interest (Table 2). Correlations between latencies and task performance were not found.

The activations that differentiated groups by strength showed a relative consistent evolution in time across subjects: left cuneus (second activation), right posterior fusiform, right middle occipital, left inferior parietal, left cuneus (third activation), right inferior prefrontal, left anterior cingulate, left inferior prefrontal (second activation), left orbitofrontal (second activation).

Within the face and object categorization tasks, healthy subjects showed stronger neuromagnetic responses to neutral faces in four regions: left cuneus within time window 3 (F=5.09, df=1, 25, p<0.04), left inferior parietal cortex within time window 1 (F=7.75, df=1, 25, p=0.01), right middle occipital cortex within time window 2 (F=6.71, df=1, 25, p<0.02), and right middle fusiform gyrus within time window 2 (160–300 msec) (F=4.85, df=1, 25, p<0.04). Differences in four out of 23 comparisons are statistically weak but nevertheless are important to be mentioned. Correlations were not computed for this task, since behavioral performance reached 100% in both groups.

Differences in responses to blurred faces were only detected for one regional activation (out of 23 regions of interest/time windows), which is at chance level. Significant correlations between activations and task performance were not found.

We did not find an association between medication and behavioral performance in any of the face-processing tasks (df=13). Furthermore, chlorpromazine-equivalent doses were not associated with the regional activations (df=13).

Only two correlations between SANS global scores and regional activations were significant, which is at chance level (df=13). Positive symptoms were not associated with current density at all (df=13).

Discussion

This study replicates previous reports of disturbed facial affect recognition in schizophrenic patients. However, much more interesting is the fact that our findings might provide important information about the functional neuroanatomy underlying these disturbances. Since a number of areas showed hypoactivity in patients, the results suggest that the facial affect recognition deficit in schizophrenia might be related to a dysfunctional network of distributed brain regions. All of the dysfunctional areas revealed hypoactivation at circumscribed latencies during affect recognition in schizophrenic patients, whereas none of them was hyperactive. Some of these areas have been previously reported to be important to the pathology of schizophrenia. Particularly interesting is the inferior prefrontal cortex, in which hypoactivation in patients was observed and strength of activation was related to behavioral performance. This region is known to be critically involved in the evaluation of facial expressions (12, 27, 28). Furthermore, it is well established that prefrontal regions play an important role in the pathophysiology of schizophrenia (29). This has been recently confirmed through positron emission tomography studies in neuroleptic-naive first-episode patients and in neuroleptic-free patients with chronic schizophrenia (30). Additional evidence for the importance of these regions has been provided by a study showing selective gray matter volume deficits in inferior prefrontal cortex in schizophrenic patients (31).

Anterior cingulate is another area often related to pathology in schizophrenia (32, 33). Since this region is known to have integrative functions in affect regulation and selective attention (33, 34), hypoactivity in patients in response to facial expressions of emotion seems plausible.

More recent studies on cerebral blood flow (35) and regional brain volume (36) add more evidence to the assumption that the orbitofrontal cortex also plays a major role in the disorder. We also found hypoactivity there in patients during facial affect recognition tasks. Furthermore, the significance of this region in processing of affective stimuli has been recently shown by a magnetoencephalography study (37).

The right posterior fusiform area was the second region in which group differences in activation and associations between activation and task performance occurred. This area is well known to be a central site in face processing (38). However, there is little literature addressing the significance of this region in schizophrenia. Therefore, it will be necessary to determine whether patients’ diminished activity during this task is a manifestation of a fundamental abnormality of this region in schizophrenia or rather secondary to dysfunctions in other brain regions within the face processing network.

Abnormalities in the left cuneus, left inferior parietal cortex, and right middle occipital cortex have also rarely been reported in schizophrenia. Therefore, similar to the situation with the right fusiform area, it remains to be clarified whether the hypoactivity found in these regions is more related to schizophrenia pathology or rather functionally disturbed during affect recognition or face processing.

Basic visual areas were consistently activated between 60 and 120 msec after stimulus onset, without differences between groups. This suggests that the facial affect recognition deficit in patients might not be caused by a basic sensory disturbance. Furthermore, the fact that we did not find brain dysfunctions in response to blurred faces in patients argues against an assumption of a disturbance at the level of basic face perception. However, at later latencies some of the visual areas were also hypoactive in schizophrenic patients in response to neutral faces. This finding needs discussion, since it leads to the assumption that specificity has not been shown for recognition of facial expressions of emotion as compared to recognition of neutral faces (the specificity for facial expressions of emotion as compared to blurred faces seems to be, however, obvious). An argument for at least a relative specificity is that patients did not have any problems at the behavioral level with the neutral face (and object) task. Furthermore, the patients’ brain dysfunctions only occurred in the more basic areas, not in the higher areas (prefrontal, orbitofrontal, anterior cingulate), in response to neutral faces. The dysfunctions in the higher areas were restricted to the emotional faces. An explanation for this pattern might be that naturalistic faces, even if they do not display an emotional expression, in general evoke affect evaluation processes in some visual brain regions, and if these processes happen, one finds differences between schizophrenic patients and healthy subjects there. Nevertheless, this needs to be tested through specific studies.

Since activations occur repeatedly at the same cortical sites during stimulus presentation, while only some of these activations are weaker in patients, we assume that the time course of interactions between areas might be important for the dysfunction of the network process. The assumption that the network might be disturbed because of a dysfunctional connectivity or rather because of a dysfunction of certain regions has yet to be demonstrated conclusively. In doing this it will be necessary to determine if there are areas or steps in the network process where the dysfunction begins across patients. However, in this context it is necessary to remember that the results are based on analyses of averaged trials. A definite specification of the significance of dysconnectivity for the network disturbance will only be provided through an attempt of single-trial analyses of such data (for an example, see reference 39). This item is also related to the issue of a global versus focal local deficit underlying disturbed facial affect recognition in schizophrenic patients. Our results point to the assumption that the disturbance is not caused by a local focal deficit but also not by an absolutely global deficit. We found eight out of 16 brain regions that showed dysfunctions during categorization of facial expressions of emotion in patients, but the other eight regions did not show dysfunctions. Therefore, we propose a disturbed circuitry of a number of brain regions but not of the whole brain. Furthermore, single-trial analyses will show whether specific emotions are particularly affected by neuroanatomical disturbances and whether top-down processes play a role, in particular, with respect to the dysfunctional visual areas.

In summary, our results suggest that the facial affect recognition deficit in patients with schizophrenia might be a result of hypoactivity of a system of distributed brain regions, some of them previously related to schizophrenia pathophysiology as determined by other methods. The fact that the regional hypoactivities occur at circumscribed latencies during the task and that the affected areas are active in a relatively consistent chronology are conclusive indicators that these areas are working in a connected circuitry. Further studies are needed to specify the exact mechanisms of the network disturbance and to relate them specifically to macro- and microstructural abnormalities.

TABLE 1

Enlarge table

TABLE 2

Enlarge table