Facial Expressiveness in Patients With Schizophrenia Compared to Depressed Patients and Nonpatient Comparison Subjects

In the last 25 years, research interest in “flat affect” as a symptom of schizophrenia has been renewed. Flat affect is present in 66% of patients with schizophrenia and is the only universally accepted negative symptom. Affective deficits in schizophrenia have longitudinal stability, predict poor outcome, and may have a genetic etiology (see references 1–3 for review).

Investigators have developed reliable operationalized multi-item scales for blunted affect: the Scale for Emotional Blunting (4), the Affective Flattening Scale (5), and the affective flattening subscale of the Scale for the Assessment of Negative Symptoms (SANS) (6). The underlying concept of affect in these scales is a rather broad one, and blunted affect is defined as an expressiveness deficit in multiple channels. For this reason, many items (such as “poor eye contact,” “decreased spontaneous movement,” “avoids social contacts,” “lacks spontaneity”) not only refer to affective deficits but also to psychomotor and social deficits. In other words, these scales combine items for two kinds of expression: direct expression of emotions (that are actually rare events during a clinical interview, see reference 7) and other and more frequent types of expression/behavior that may reflect the underlying emotional state but that may also have other determinants (8).

Emotion research studies have shown that blunted affect in schizophrenia is an “encoding” deficit, a deficit in affective expression (2) that does not correspond to a diminished emotional experience in most (9, 10) but not all (11) studies and is independent of affect perception and recognition (10). Most of these studies integrated the methodology used in basic emotion research: they looked at facial expressions, they selected emotions among the six commonly cited cross-cultural emotions (anger, disgust, fear, happiness, sadness, and surprise), and they used specific rating scales or computerized analyses. Patients with schizophrenia consistently showed less expression in their faces than nonpatient comparison subjects (9, 12–16). It is surprising that several studies could not find any correlation between facial emotion expressiveness and clinical ratings of blunted affect (7).

Another line of research on facial expressiveness comes from ethological studies. These studies observe subjects’ interactions with their environment. Behavioral patterns are then grouped into different categories, such as social behaviors (smiles are often placed in this category), coverbal behavior (hand, head, and face movements that accompany and are tied to speech), and body-to-body gestures (also called adaptors). In relation to nonpatient comparison subjects, subjects with schizophrenia showed fewer body movements (17), fewer facial movements (18), less social and coverbal behavior (19), and fewer smiles (20). The relationship between these measures and clinical ratings of blunted affect has rarely been studied (19).

Another issue concerns the differentiation of affective deficits seen in schizophrenia from those seen in depression, which are part of the psychomotor retardation syndrome described in depression (21). On one hand, clinical rating scales for affective deficits do not differentiate schizophrenia patients from depressed patients (5), but on the other hand, clinicians rarely use the terms “flat” or “blunted” affect to describe depressed patients. Few research studies (14–17, 22–24) have compared a depressed group to a schizophrenic group, and mixed results were reported. These discrepancies among studies might be the result of methodological differences, population selection, or small group sizes.

The facial expressions of emotion are regulated by pyramidal and extrapyramidal tracts. While posed expressions use cortical and pyramidal circuits, spontaneous emotional expressions are essentially of subcortical limbic and extrapyramidal origins (25). Although there is a large body of evidence for a neuroanatomical distinction between volitional and spontaneous emotion facial movements, it seems that in everyday life situations and in laboratory conditions, a complete distinction cannot be made, and all facial emotion expressions combine a volitional and a spontaneous component to different degrees. The connection between these two structures—cortical and subcortical—is much less known. The cortical tracts can interact with the subcortical tracts at the final stage (the facial nerve nucleus), but there is also evidence that cortical pathways involved during posed facial emotional expressions interact with subcortical structures at an earlier stage (26), and this interaction should occur before the neuronal activation reaches the primary motor areas (27). Expressions of most emotions may share similar neuronal pathways, but of interest, Rinn (28) suggested that the neural substrates of smiles are different from other emotion expressions. In a similar vein, coverbal gestures seem to rely on cortical structures (for example, when they replace a word) or on subcortical structures (28), however much less is known regarding their exact neural substrates.

It thus appears that affective deficits in schizophrenia are clinically conceived as deficits in expressiveness and are assessed through two kinds of behaviors: direct expressions of emotions and behaviors indirectly linked to emotional states (such as social smiles and coverbal gestures). Patients with schizophrenia are impaired in both types of behaviors, but no studies have looked at these deficits simultaneously. Affective deficits have also been described in depression, but it is not clear whether they are similar to or different from the affective deficits seen in schizophrenia. For these reasons, we aimed to look at various aspects of facial expressions (posed and spontaneous expressions of emotions, coverbal gestures, and smiles) in patients with schizophrenia compared to nonpatient comparison subjects and depressed subjects.

Method

Subjects

Subjects were inpatients at New York State Psychiatric Institute; Creedmoor Psychiatric Center, New York; or Centre Hospitalier de Rouffach, France (this study was started in the United States and continued in France). They were between 18 and 60 years of age, and all had the capacity to give consent. Fifty-eight patients with schizophrenia, 25 patients with unipolar depression, and 25 nonpatient comparison subjects (hospital employees or medical students) were enrolled. Subjects with neurological conditions were excluded. After complete description of the study to the subjects, written informed consent was obtained. The study was approved by each hospital’s institutional review board.

Diagnoses were made with the Diagnostic Interview for Genetic Studies (29) for the patients in research wards (N=18). For other patients (N=65), diagnoses were obtained with the use of the Schedule for Affective Disorders and Schizophrenia (30) according to the DSM-IV diagnostic criteria.

In the schizophrenia group, the subjects were not depressed (Brief Psychiatric Rating Scale [31] depression item score of 0). The dosages of their antipsychotic medications could not have been changed in the preceding 2 weeks, and they also had to be taking anticholinergic therapy if they were taking typical antipsychotics. Fifteen subjects were tested twice: while taking and not taking antipsychotics.

Patients with unipolar depression were nonpsychotic, currently depressed (measured with the 21-item Hamilton Depression Rating Scale [32]) inpatients who were not taking antipsychotics.

All patients were first administered the modified Scale for the Assessment of Negative Symptoms (SANS) (33) and the Brief Psychiatric Rating Scale (score: range=0–6). All patients had to have had a score of 0 on the Abnormal Involuntary Movement Scale (34) for the oral and facial areas, and they also could not have scored more than 1 on any item of the Simpson-Angus Rating Scale (35). For the French subjects, previously validated translations of the clinical rating scales were used.

Procedures

Three tests were conducted: a visually cued task, a verbally cued task, and a narrative task. The first two tasks tested posed facial expressions and differed on the sensory requirement—visual versus verbal; the third test explored spontaneous expressions during an interview.

In test 1, the subjects were shown six pictures of facial expression and were asked to imitate each expression. Twelve pictures (PE2-21, WF3-11, PE3-21, GS1-8, PE5-7, JJ4-13; NR2-7, MO2-18, MO1-23, JM1-4, JM3-1, PF1-16) were taken from the Pictures of Facial Affect (36); an independent group of 10 male and 10 female nonpatient comparison subjects selected the same-gender pictures that “best” expressed the corresponding emotions among the pictures whose apparent validity had been well studied in a previous work (13). Each picture reflected one of six emotions—anger, disgust, fear, joy, sadness, and surprise—and was matched by gender (male faces for male subjects). In test 2, the subjects were asked to show anger, disgust, fear, happiness, sadness, and surprise in their faces.

In test 3, one emotion was named and the subjects were asked to relate past events or to imagine and describe events that caused or would cause the highest level of this emotion for 2 minutes. Each of the six emotions was tested successively; the order of emotions was randomly assigned.

During the three tasks, the subjects were videotaped, with a particular focus on their faces. All clinical ratings and tests were administered by one of us (F.T.). One judge certified in the Facial Action Coding System who was blind to the diagnoses (M.H.-B.) rated videotaped emotional expressions. Another blind rater (L.C.-B.) counted the number of facial coverbal gestures in test 3.

Data Analysis

Ratings and variables

For tests 1 and 2, facial muscle movements were rated with the Facial Action Coding System (37). The Facial Action Coding System is a rating scale that indicates each specific part of the face that is moving (called an action unit). All action units of each expression were recorded. This pattern of action unit was then compared to the combinations of action units that are known to correspond to a specific emotion and reported in the Facial Action Coding System Dictionary (also called the Emotional Facial Action Coding System). A score of 1 was given if the subject’s facial expression corresponded to the given emotion and a score of 0 if it did not. Expressions of blended emotions or expressions containing an “extra” action unit received a score of 0. Consequently, each subject had a score in the range of 0 to 6 for test 1 and test 2.

In test 3, seven variables were defined for each subject:

Interrater reliability for test 1, test 2, and the first five test-3 variables was evaluated between two judges certified in the Facial Action Coding System for 11 patients: kappa and intraclass correlation (ICC) scores were all above 0.95. The interrater reliability for facial coverbal gestures was tested between two judges with 20 video clips: the ICC reached 0.92.

We differentiated smiles from other emotional expressions because in an interview situation, the value of smiles is primarily social (see Discussion section).

From the clinical ratings, the major variable of interest was the score on the SANS affective flattening subscale (item 7). Variables of secondary interest were the SANS global and alogia (item 12) scores and the Brief Psychiatric Rating Scale total, negative subscale (items 3, 13, and 16), and positive subscale scores (items 4, 11, 12, and 15).

Statistics

Clinical ratings were compared between the two patient groups with the nonparametric Kruskal-Wallis test. Each performance score was analyzed separately. Performance scores were compared between groups with the use of an analysis of variance for normally distributed variables and with the Kruskal-Wallis test for nonparametric variables. Pairwise comparisons were conducted with the Bonferroni correction, which was applied by multiplying the p value by 3 (because we conducted three comparisons for each variable).

In order to determine the influence of additional variables on each performance score, regression analyses were conducted with generalized linear model analysis (SAS procedure GENMOD, SAS Institute, Cary, N.C.). This method is an extension of traditional linear models to handle abnormally distributed observations. For each study performance, a separate regression analysis was conducted. “Diagnostic group” was kept as the principal independent variable, and the following variables were considered: gender, education, language (English or French), and words. In order to test the influence of the variable “emotion type,” a new set of regression analyses was conducted with performance scores obtained for each emotion (six scores by subject) as the dependent variable. A backward selection method was used; control variables and interaction terms were entered as independent variables and were kept in the model if the type 1 error probability was below p=0.05. Language was kept in the model for the dependent variable “words” because speech rate can differ between these two languages (38). A Poisson distribution was assumed for the variable “number of other-than-joy emotions expressed in test 3” because it was a rare event.

In order to compare the scores of patients with schizophrenia when they were and were not taking antipsychotics, a paired t test was computed for each score. No correction for multiple testing was applied.

Spearman nonparametric correlations between study performances and clinical ratings were calculated for the schizophrenia group. Significance is reported without correction for multiple testing. All tests were two-tailed, and the alpha level was set at 0.05.

Results

Demographics and Clinical Ratings

Patient groups did not differ on the degree of blunted affect, as measured by the SANS affective flattening subscale score. The patients with schizophrenia had higher total scores on the SANS and the Brief Psychiatric Rating Scale (Table 1).

Study Performances

The patients with schizophrenia and the patients with depression had lower scores than the nonpatient comparison subjects for all emotion variables (Table 2). Compared to the depressed group, the schizophrenia group showed no difference for posed emotion expression and smiles, better scores for spontaneous expression of the selected five emotions, and worse scores for overall facial expressiveness and “words,” while the difference for facial coverbal gestures approached significance.

Effects of Additional Variables

Regression analyses showed that emotion type was a significant independent variable for most performances— test 1 score (χ²=144, df=5, p<0.0001), test 2 score (χ²=188, df=5, p<0.0001), number of other-than-happiness emotion expressions (χ²=46, df=4, p<0.0001), time expressing the given emotion (χ²=15, df=4, p=0.005), number of smiles (χ²=48, df=5, p<0.0001), and time spent smiling (χ²=53, df=5, p<0.0001)—but did not significantly influence overall facial expressiveness and facial coverbal gestures. During the posed emotion tests, happiness and surprise were the emotions most correctly expressed, and sadness, anger, and fear were the least. In test 3, disgust and sadness were more frequently expressed than surprise, anger, and fear. In test 3, the two positive valence emotions—surprise and happiness (all subjects except one reported pleasant experiences when they discussed surprise)—elicited the most smiles, although smiles were the least frequent when they discussed sadness and fear.

Gender influenced some study scores: total number of smiles (for men: mean=8.8, SD=7.9; for women: mean=15.6, SD=9.7) (χ²=12, df=1, p=0.0006), time spent smiling (for men: mean=83 seconds, SD=103; for women, mean=157, SD=118) (χ²=10, df=1, p=0.001), and overall facial expressiveness (for men, mean=11.7, SD=5; for women, mean=15.3, SD=4.7) (χ²=8, df=1, p=0.004).

Education was a predicting factor for performance score in test 1 (χ²=7, df=1, p=0.008) and test 2 (χ²=11, df=1, p=0.0008). The influence of rate of speech was significant for overall facial expressiveness (χ²=4.4, df=1, p=0.04) and for facial coverbal gestures (χ²=24, df=1, p<0.0001). Language and interaction terms were not significant factors for any analysis.

In these regression analyses, “diagnostic group” remained a significant independent variable. Least-squares pairwise comparisons yielded changes for one variable only: “words.” Nonpatient subjects uttered more words than schizophrenia patients (χ²=26.3, df=1, p<0.0001) and depressed subjects (χ²=5.1, df=1, p=0.02), while the two patient groups did not differ (χ²=1.8, df=1, p=0.17). After we controlled for “words,” facial coverbal gestures did not differ between the two patient groups (χ²=1.4, df=1, p=0.24).

We also compared American patients with schizophrenia (N=46) with French patients with schizophrenia (N=12). No significant differences in clinical ratings and study performances were found, although a type II error could not be ruled out.

Effect of Medications

Fifteen subjects with schizophrenia were tested twice: while not taking antipsychotics (recently admitted patients or patients in a drug-free phase of another study) and while taking antipsychotics. Four of them were not taking antipsychotics for 2 weeks, two for 3 weeks, three for 4 weeks, and six for more than 6 months. None was taking depot medications before discontinuation. Among the clinical ratings, only the total score on the Brief Psychiatric Rating Scale (while not taking medications: mean=23, SD=6; while taking medications: mean=17, SD=8) (t=2.79, df=14, p=0.01) and the positive symptom subscale of the Brief Psychiatric Rating Scale (items 4, 11, 12, and 15) (while not taking medications: mean=7.8, SD=4.4; while taking medications: mean=4.2, SD=4.6) (t=2.55, df=14, p=0.02) changed significantly. No study variable changed significantly.

Correlations With Clinical Ratings in Schizophrenia

All study performances were significantly related to the primary clinical rating, the SANS affective flattening subscale (Table 3).

Discussion

Our study showed that patients with schizophrenia and patients with depression are impaired in all studied types of facial expression.

Volitional Emotion Expression

The patients with schizophrenia appeared quite impaired in posed emotion expression. This is consistent with the literature (13, 15, 16). The depressed patients were equally impaired. Three other studies (14, 15, 24) could not find any differences between the depressed and the schizophrenic groups, whereas another study (16) reported more impairment in the schizophrenia patients. However, in this latter study, the intensity of facial muscle involvement but not the accuracy of facial expressions was measured, only schizophrenia patients with blunted affect were selected, and no clinical rating for the depressed group was reported.

Spontaneous Emotion Expression (for Emotions Other Than Happiness)

Both patient groups had less spontaneous emotional expression than the nonpatient comparison subjects, and the patients with schizophrenia performed better than the depressed patients. Previous studies have found a lower degree of emotional expressiveness in patients with schizophrenia than in nonpatient comparison subjects (9, 12, 15). Comparisons between depression and schizophrenia have brought contrasting results. In one study (9), patients with schizophrenia with blunted affect were less expressive than depressed patients, while patients with nonblunted affect showed more emotional response than depressed patients for positive stimuli. In another study (15), no differences could be found between schizophrenia and depression. A third study (22) reported that patients with depression had fewer facial expressions on a visual rating scale but not on a computerized analysis.

It is surprising that no depressed patients ever expressed anger, disgust, fear, sadness, or surprise while they were talking about these emotions. Tentative explanations could be advanced. First, it is possible that their facial expressions were not “pure” (did not correspond to the Facial Action Coding System emotion predictions). Second, their current depressive mood (as assessed by the score for the first item of the Hamilton Depression Rating Scale: mean=3, SD=1) may have prevented them from experiencing any other feelings and had a ceiling effect for sadness expression, and thus they did not express any emotion. It will certainly be interesting to know when in the course of their illness and in which circumstances depressed patients express sadness in their faces.

In the three groups, when the subjects were asked to speak about an emotional experience other than joy for 2 minutes, they rarely expressed this emotion in their faces. This is in accord with Ekman’s work (39). Some authors have questioned the capacity of a clinical interview to elicit any emotional reaction (7, 8). Our study tends to show that techniques to easily elicit emotional expressions other than smiles during a psychiatric interview remain to be discovered. In a German study (12), patients with schizophrenia and nonpatient comparison subjects had a discussion on politics during a one-to-one interview and were rated with the Facial Action Coding System. For the whole group, smiles and raising of the eyebrows (which correspond best to our facial coverbal gestures) accounted for more than 50% of all facial expressions, while expressions of disgust, anger, sadness, fear, and surprise accounted for 11%. From their data, it seems that schizophrenia inpatients expressed any of these five emotions with a frequency of 0.25 expressions per minute (0.65 for nonpatient comparison subjects).

It can be concluded that spontaneous expression of these five emotions is quite infrequent for patients and nonpatient comparison subjects in an interview situation, and its contribution to the assessment of blunted affect might be minimal.

Smiles

As previously reported (12, 20), both patient groups smiled less frequently than the nonpatient comparison subjects. The schizophrenia group did not differ from the depression group by the number of smiles and by the time spent smiling. Two studies used a comparison depression group, one (17) found a similar number of smiles between the two groups, and the other (20) did not report on the significance of the difference.

Coverbal Gestures and Speech

The patients with schizophrenia spoke less than the nonpatient comparison subjects and the depressed patients and used fewer facial coverbal gestures than the nonpatient comparison subjects. However, differences between the two patient groups became nonsignificant after we adjusted for language spoken (English versus French). The fact that “number of words” was a significant independent variable for overall facial expressiveness and facial coverbal gestures underlies the interaction between alogia and facial expressiveness and, consequently, the assessment of blunted affect. Other studies (1, 19, 23) have looked at hand coverbal gestures and reported comparable results.

The nature and communicative function of coverbal gestures (40) are still the subject of ongoing debates, psycholinguists against neurocognitivists. Research in psychiatry will certainly benefit from these approaches and ongoing research.

Additional Variables and Effect of Neuroleptics

First, the schizophrenia patients’ performances did not change when they were and were not taking antipsychotics. This is consistent with the literature (13–15, 22, 41). One study (42) reported a decrease in facial action units and in smiling after patients were given antipsychotics. In this study, there was no report on the degree of extrapyramidal symptoms, and not all patients were receiving anticholinergic medications. In our study, the patients were enrolled only if they showed no sign of extrapyramidal symptoms, and half of them (N=7) were taking atypical antipsychotics after their medication-free phase.

Female subjects smiled more and appeared more expressive than male subjects, and no differences between groups were found. Similar gender differences have been reported for the general population (43).

Education was a predictive factor for tests 1 and 2. The role of education on facial expressiveness has rarely been studied. The predictive value of education might be explained by the fact that tests 1 and 2 were more cognitively demanding than test 3. Others have reported that positive emotions are less cognitively demanding than negative emotions. This is partially in accord with our study: education was a significant variable in tests 1 and 2 for anger, disgust, and sadness but not for happiness, surprise, or fear (results not shown).

Language did not influence any performance variables. Although our study did not intend to look at differences across nations and cultures, our results show that the use of patients from two different countries in our study did not introduce a significant bias.

Emotion type influenced various performances. Happiness and surprise were the most easily reproduced emotions. Sadness was the worst performed posed expression and the emotion that elicited the fewest smiles in test 3. Compared to expression of the other five emotions, smiles were more frequent in any subtest of test 3 and more frequent when subjects were discussing happy experiences. This indicates that smiling plays a central role during social interaction and that it is influenced by emotional context.

After we controlled for these variables, differences between nonpatient comparison subjects and the two patient groups persisted, while only two differences between patient groups persisted: the patients with schizophrenia expressed the intended emotion more frequently during test 3, yet they had lower overall facial expressiveness scores. Several explanations might be advanced: the schizophrenia subjects’ facial expressions might be inappropriate or less intense, or the subjects with depression may have used other facial expressions more often (such as body-to-body gestures or facial expressions of anger while discussing sad experiences). Future research should clarify this apparent discrepancy.

Relationship With Clinical Ratings in Schizophrenia

In our study, all performance scores were significantly correlated with the SANS affective flattening subscale. Correlation coefficients varied between 0.30 and 0.60, and thus, each study variable independently explained from 9% to 36% of the variance of the SANS affective flattening subscale. This finding is consistent with the notion that the SANS affective flattening subscale items refer to direct and indirect expressions of emotions. Correlations were not higher than 0.60 for several reasons: 1) the SANS ratings were performed before the study tests, 2) performance scores considered facial expressions only, and 3) some items of the SANS affective flattening subscale may not be well related to facial expressiveness.

Significant correlations were also found between performance scores and the SANS total score and the Brief Psychiatric Rating Scale negative scale score. This is consistent with the idea that blunted affect is a major—if not the major—component of negative symptoms.

Are Affective Deficits Emotional or Social?

This distinction is theoretical, and opposed discrete-emotional theorists (39) against behavioral ecologists (44). One reason for these disagreements came from the fact that the relationship and interaction between emotional experiences (feelings) and facial expressions are quite complex (45), and nowadays, most emotion researchers agree that emotional expressions can serve both an emotional and a social function. Applied to our study, this means that when subjects showed an angry face while they were relating anger experiences, it could not be deduced that they were effectively feeling anger at that time; they could have used an angry face as a symbolic expression, a facial coverbal gesture. On the other hand, even if depressed subjects were not showing a sad face while they were talking about sad experiences, it did not mean that they were not feeling sad, and indeed, their clinical ratings proved the opposite.

In our study, even depressed patients expressed smiles but not sadness during their telling of sad experiences, which can clearly be related to the social value of smiles. The fact that patients who were hospitalized for depression did not express sadness at all during psychiatric interviews cannot be easily explained by the societal display rules hypothesized in the neurocultural model. Another finding was that the verbal emotional content affected smiling, which could reflect a happy feeling or facilitate the expression of social smiles.

Further Implications

It thus appears that facial expressions of emotions are better considered as nonverbal expressions with various potential values: emotional, social, linguistic, or communicative. Consequently, blunted affect should not be conceived as a pure affective/emotional deficit but as part of a broader deficit in expressiveness (see also reference 46). We hypothesize that this akinesia in expressive gestures represents or is part of a syndrome that can be found in various psychiatric disorders (the same way that psychosis is a transnosological psychiatric syndrome). This hypothesis goes against the too often held view that the psychomotor deficits of depression and negative schizophrenia symptoms, such as blunted affect and alogia, are different symptoms, a view that has never been clearly confirmed by research studies.

Basic research has brought a clear description and a distinctive typology to these expressive gestures, and their incorporation into psychiatric clinical rating scales should be developed.

The expressive deficits found in our study involved posed and spontaneous emotional expressions, smiles, verbal output, and coverbal gestures. The most conservative explanation regarding the neurobiology of these deficits would be that their common neuroanatomical substrate involves a premotor region connected to cortical and subcortical structures and involved in emotional and social behavior, such as the anterior cingulate (47) and the anterior cingulate circuit (48). Moreover, the anterior cingulate has been repeatedly linked to the pathophysiology of depression and of schizophrenia.

Limitations

Conclusions

In an interindividual situation, patients with schizophrenia and patients with depression present similar deficits in facial expressiveness. These expressive deficits involve different modalities (posed and spontaneous emotional expressions, smiling, coverbal gestures, verbal output) and have previously been classified as affective deficits, blunted affect, psychomotor retardation symptoms, negative symptoms, or abulia. These deficits can be grouped into a common syndrome, an “expressivity” deficit, which reflects motor, affective, and social impairments. Further direction for research should replicate that this syndrome is transnosological and develop a reliable and convenient measurement method for this syndrome. It will also be of great interest to determine if this “expressivity” deficit has a unique neurobiology that cuts across DSM-IV diagnostic categories. Neuroimaging and genetic studies can now be used in this regard.

TABLE 1

Enlarge table

TABLE 2

Enlarge table

TABLE 3

Enlarge table