Emotion processing is composed of evaluative, experiential, and expressive components (1). The evaluation of affect may be correct or incorrect depending on an individual’s ability to identify the emotional valence carried by an event or an object and is influenced by mental illness. For example, for patients with depression, positive life events are often considered negative or harmful. Patients with schizophrenia instead seem to be unable to extract the emotional content from a situation or experience, whether the experience is pleasurable or unpleasant. Because people with psychiatric disorders lose their capacity to distinguish between pleasant and unpleasant experiences and the ability to assign the appropriate emotional valence to these experiences, developing cognitive models of emotional processing in healthy humans will assist in identifying the neural mechanisms of serious mental illnesses such as schizophrenia, depression, anxiety, and posttraumatic stress disorder (2).

Neuroimaging techniques permit identification of the in vivo neural substrates of many domains of brain function such as memory, attention, and sensation. Although several research reports have been published (3–10), the study of emotions through use of neuroimaging technology has lagged behind studies of cognition.

We report here on a study examining the functional neuroanatomy associated with the attribution of affective valence to visual stimuli carrying positive, negative, and neutral content. The study, which used the [¹⁵O] water positron emission tomography (PET) method in healthy young volunteers, focused on one component of the process of affective evaluation, i.e., the degree to which experiences or objects are judged as pleasant or unpleasant (11).

Subjects

Subjects were 17 healthy right-handed individuals (10 women and seven men) recruited from the community. The group had a mean age of 31.2 years (SD=8.7), mean education of 14.5 years (SD=1.6, range=12–18), and no history of psychiatric/neurological disorder, alcohol/substance abuse, or current use of psychotropic medications. No gross brain abnormalities were found on magnetic resonance (MR) scans. Mean full-scale IQ (12) was 110.17 (SD=12.42, range=83–133), verbal IQ was 106.83 (SD=8.73, range=88–122), and performance IQ was 112.28 (SD=16.12, range=80–138). Subjects performed within the high end of the normative curve on the Benton Facial Recognition test, short form (mean score=23.67, SD=1.97, range=20–27) (13). All subjects gave written informed consent to protocols approved by the University of Iowa Human Subjects Institutional Review Board.

Activation Stimuli

Three sets of activation stimuli were used in the present study: pleasant images, unpleasant images, and neutral images. Each set of stimuli was shown once. In each condition subjects viewed a set of 18 complex images displayed on an 11-by-8-inch computer monitor. Images were displayed individually for 2 seconds; thus, each image set was presented for 36 seconds. The order of picture presentation was neutral-pleasant-unpleasant in order to minimize possible intensity carryover effects (14). The screen was positioned approximately 14 inches from the subject. Images were 7.75 inches wide and 7.5 inches high and subtended 29° of visual angle in height and 28° in width. Timing of the activation was such that image displays began 10 seconds before the arrival of the [¹⁵O] water bolus in the brain, assessed individually for each subject.

Images were chosen from The International Affective Picture System (15) through a multistep process that achieved a wide range of pleasant images/emotions such as happiness, appetite, satisfaction, beauty, and success. Unpleasant emotions included fear, disgust, sorrow, and disappointment. The International Affective Picture System (15) provides a large data set of standardized, emotionally evocative color pictures associated with highly reliable affective judgments and psychophysiological responses (11, 15). Pictures with high, low, and neutral normative valence ratings were first identified. Normative valence and arousal ratings were available from a large sample of healthy subjects on a scale ranging from 0 to 9 (0=very negative and 9=very positive valence or arousal) (15). Images with high arousal, sexual content, or interpersonal violence were removed. Next, the selected pictures were shown to a large group of healthy individuals and rated for valence. The pictures with the smallest standard deviations were retained in order to ensure that all subjects would have a similar emotional response during the PET experiment. The slides were rated again for valence by another group of healthy individuals. Finally, the 18 pictures with the highest and lowest ratings were selected, along with 18 neutral pictures. The three picture sets in this study were equated in terms of content (i.e., number of pictures containing people, faces, objects, and scenes). Mean normative valence scores were 2.25 (SD=0.65) for the unpleasant picture sequence, 7.68 (SD=0.36) for the pleasant picture sequence, and 5.49 (SD=0.75) for the neutral sequence. Mean normative arousal scores were 5.73 (SD=0.74) for the unpleasant sequence, 4.69 (SD=1.03) for the pleasant sequence, and 3.00 (SD=0.64) for the neutral sequence. Mean picture luminance was 12.37 feet/candles (SD=0.49) for the unpleasant picture set, 12.47 (SD=0.49) for the pleasant set, and 12.36 (SD=0.35) for the neutral set. A description of the pictures is in A1.

Before viewing each set, subjects were instructed to look at the pictures and were told that they would be rating the pictures as to how pleasant or unpleasant they were. Subjects were not told that they needed to correctly identify the emotional valence of stimuli. Immediately after acquisition of the blood flow image, subjects were asked to rate the valence of the entire image set on a verbal analog scale ranging from –7 to 7 (–7=extremely unpleasant, 0=neutral, and 7=extremely pleasant).

PET Data Acquisition

Regional cerebral blood flow (rCBF) was measured by using the bolus [¹⁵O] water method (16–18) with a GE-4096 PLUS Scanner. The subjects were oriented in the PET scanner with laser light guides aligned at the orbitomeatal line. The center of the most rostral slice was indicated by the laser guides. Fifteen slices (6.5 mm center to center), with an intrinsic in-plane resolution of 6.5 mm full width at half maximum and a 10-cm axial field-of-view, were acquired. Images were reconstructed by using a Butterworth filter (cutoff frequency=0.35 Nyquist interval). CBF was determined by using the [¹⁵O] water (50 mCi/injection) method and methods previously described (19). For each injection, arterial blood was sampled from time 0 (injection) to 100 seconds. Imaging, initiated at injection, consisted of 20 frames at 5 seconds per frame for a total of 100 seconds. The parametric image (i.e., blood flow image) was created by using a 40-second summed image (initial 40 seconds immediately after bolus transit) and the arterial input function. A preliminary injection (sham), to determine bolus arrival time, was employed to establish stimulus timing (20).

MR Image Acquisition and Processing

MR images consisted of contiguous coronal slices (1.5 mm thick) acquired on a 1.5-Tesla GE Signa scanner. Technical measures of the MR image acquisition were as follows: spoiled gradient recalled sequence, flip angle=40˚, TE=5 msec, TR=24 msec, number of excitations=2. MR images were transferred to the Image Processing Laboratory of the University of Iowa Mental Health Clinical Research Center for analysis through use of Silicon Graphics workstations and locally developed software (BRAINS) (21–27).

The initial step of postacquisition processing involved "removing" the brain from the skull by using a combination of automated edge detection techniques and manual tracing. Pixels representing surface CSF are classified through a thresholding procedure and removed from the display. All brains are realigned parallel to the anterior commissure/posterior commissure line and the interhemispheric fissure to ensure comparability of head position across subjects. Alignment also places the images in standard Talairach-Tournoux space (28). At this point, images from multiple subjects or multiple scans from a single subject can be coregistered. Finally, images are resliced in three orthogonal planes to produce a three-dimensional data set that is used for visualization and analysis.

PET Image Processing

The quantitative PET blood flow images were transferred to the Image Processing Laboratory for further analysis (21–27). The first step in image analysis involved registration of each individual’s PET and MR images. The co-registration used a two-stage process. Initially, a coarse fit based on surface matching of the MR and PET images was done (29) Then, with the surface fit data used as input, a variance minimization program was employed for the final co-registration (30). Brain landmarks identified on the MR image were used to place each co-registered image into standardized coordinate space (28). An 18-mm Hanning filter was applied to the PET images to eliminate residual anatomical variability.

Statistical Analysis

Statistical analysis of the blood flow images used a modification of the Worsley method (24, 31, 32). A within-subject subtraction of relevant conditions was performed (e.g., unpleasant pictures minus neutral pictures), followed by across-subject averaging of the subtraction images and computation of voxel-by-voxel t tests of blood flow differences.

This study examines the results from three subtractions: pleasant minus neutral pictures, unpleasant minus neutral pictures, and unpleasant minus pleasant pictures. It is not possible to determine from these subtractions whether decreased flow in a region reflects an actual decrease in blood flow in the region or relatively higher flow in that region in the comparison condition. In such analyses, conditions are measured relative to one another and are referred to as activations. Comparison of one emotional condition with both the opposite valence and neutral conditions will give insight into the neural circuitry involved.

Data reported in tables t1, t2, and t3 show the location of peaks (with anatomical localization based on visual inspection of co-registered MR and PET images and Talairach-Tournoux coordinates); the x, y, z Talairach-Tournoux coordinates, the t_max (highest t value identified in the peak), and the volume of the peak (in cubic centimeters) that exceeds the t value of 3.61 (df≈3872) threshold. This threshold, which has been consistently used by our center, corresponds to an uncorrected significance level of <0.0005 per voxel. Regions of significant activation were identified on the t-map images and corrected for the large number of t tests performed, the lack of independence between voxels, and the resolution of the processed images (24, 31, 32). There were about 300,000 gray matter voxels in our images, representing approximately 242 resolution elements (31). After filtering, the three-dimensional image resolution is 2.5 cc. The degrees of freedom were extremely large for the t tests: df≈3872=number of resolution elements x (number of subjects –1). Only areas that exceeded 50 contiguous voxels were tabled, in order to omit isolated outlying values.

Subjects’ ratings of the unpleasant (mean score=­–6.12, SD=1.4), pleasant (mean=6.10, SD=1.1), and neutral (mean=2.38, SD=1.6) picture sets were consistent with the intended valence (F=358, df=2, 15, p<0.0001). Ratings of the neutral and pleasant stimuli were statistically different (F=57, df=1, 16, p<0.0001), as were the neutral/unpleasant (F=457, df=1, 16, p<0.0001) and the unpleasant/pleasant (F=680, df=1, 16, p<0.0001) comparisons.

Evaluation of unpleasant visual stimuli relative to pleasant stimuli produced activations in primary and secondary visual cortex and in the superior semilunar lobule of the cerebellum. The left amygdala was relatively more active during evaluation of unpleasant stimuli (t1 and F1).

Evaluation of pleasant stimuli produced activations bilaterally in the medial, orbital, and dorsal lateral frontal cortex (t1 and F1).

Subjects showed relatively higher blood flow in the primary visual cortex, left retrosplenial cingulate, and right precuneus in the pleasant picture condition. Bilaterally, the posterior cingulate gyrus was also more active (t2 and F2) in the pleasant than in the neutral condition.

Evaluating neutral visual stimuli produced relatively higher blood flow, compared to the evaluation of pleasant pictures, in the visual association cortex, right frontal operculum, and entorhinal cortex. Several cerebellar locations, including the superior and inferior semilunar lobules, the pyramid of the vermis, and the nodulus, were also relatively more active (t2 and F2).

Evaluation of unpleasant stimuli produced activations in the primary and secondary visual cortex including bilateral fusiform gyri. The left retrosplenial cingulate gyrus and the right precuneus were relatively more active in a pattern very similar to that seen during evaluation of pleasant stimuli t3 and F3). In addition, there was increased flow in the left nucleus accumbens. The neutral condition produced relative activations bilaterally in the dorsal lateral and orbital frontal cortex and in the left frontal operculum, bilaterally in the anterior cingulate, and in several locations in the cerebellum including the bilateral superior and right inferior semilunar lobules.

This study examined the functional neuroanatomy associated with the attribution of emotional valence to visual stimuli. Subcortical limbic regions were activated during attribution of valence to unpleasant stimuli, whereas cortical limbic areas were activated during attribution of valence to pleasant stimuli regardless of the comparison condition used. How may these findings be interpreted? Humans and nonhuman animals have very efficient neural mechanisms to detect danger that developed much earlier than the massive enlargement of the frontal cortex observed in primates. The danger recognition system, which is crucial for the survival of the species, evolved in order to function effectively with least cortical appraisal and thus is largely subcortical (33). Human and nonhuman animals respond to a sudden danger in a somewhat stereotypic and universal way that initially does not involve complex cognitive processing and achieves promptness to escape. The ability to appreciate the positive aspect of events and situations, however, requires more sophisticated processing of the stimuli that is individually personalized and has the characteristic of a "higher" cortical process. It is thus arguable that the detection of pleasant features relies on a rather phylogenetically newer circuit that involves largely the prefrontal cortex and the cortical executive system.

A way of isolating a differential neural response to attribution of emotional valence in pleasant and unpleasant pictorial stimuli that is least influenced by arousal is to subtract these two conditions from one another. This subtraction showed increased activity in the amygdala associated with the attribution of emotional valence to the unpleasant stimuli, whereas emotional evaluation of pleasant stimuli was associated with several areas in the orbital medial and dorsal lateral prefrontal cortex.

The role of the amygdala in emotional behavior has been recognized (34–37). Studies using affective manipulations of sensory/cognitive tasks or drugs have associated the amygdala with emotions in humans (4, 6, 8, 9, 14, 38–41). Results from this study are consistent with research showing amygdala involvement in emotion evaluation, specifically in extraction of affective content from visual stimuli (8, 9, 14, 42–46).

Whether the amygdala may specifically process only one of several possible negative emotions (9, 42) or play a very general role in all affective behavior (47–49) is one of the open questions in emotional neuroscience (14). In contrast to the view positing a role of the amygdala in the appraisal of only one negative emotion, e.g., fear (42), the present study found that increased amygdala activity was associated with evaluation of a wide range of unpleasant stimuli grouped in a pictorial sequence.

Consistent with the majority of the literature (8, 9, 42, 45), the present study found no evidence of human amygdala involvement in the attribution of positive valence to visual stimuli with pleasant content. Using picture sets derived from the same source (50) and a similar study design, Lane et al. (45) have shown left amygdala activation in response to unpleasant but not pleasant visual stimuli. The higher amygdala blood flow in response to unpleasant relative to pleasant stimuli, but not relative to neutral stimuli, is consistent with the observations of Morris et al. (9). They reported left amygdala blood flow increases with increasing fearfulness and decreases with increasing happiness in the visual stimuli (9). Rapid habituation to happy faces (14) may also account for the absence of amygdala differential activity observed in response to visually presented pleasant stimuli. Furthermore, the human amygdala may be involved in the mental processes necessary to detect affective valence for both negative and positive stimuli in other cognitive domains (47). In nonhumans, the amygdala participates in emotion processing for both negative and positive stimuli regardless of the delivery of the stimuli (33, 48, 51).

Results from the present study do suggest, however, that in humans affective evaluation of pleasant stimuli in the visual domain is carried on with a substantial contribution from the frontal lobe. This is consistent with findings from a number of research areas that suggest that the orbital frontal cortex plays an important role in representing information about reinforcing stimuli. Specifically, lesions of the orbital frontal cortex in animals interfere with short-term memory of reward information. The animal no longer can discriminate between good and bad (52). Neuronal cells in the orbital frontal cortex respond to information about reward or punishment (53–55).

The present study is also consistent with previous research in healthy humans and individuals with clinical depression. Human inferior, medial, and orbital prefrontal cortices play an important role in emotional cognitive processes (3, 4, 40, 56) including recognition of facial emotion (57). Frontal regions are also engaged in the assessment of facial attractiveness (58). Patients with full-blown depression, a condition that affects the patient’s ability to detect and take pleasure in joyful events and situations, show decreased perfusion in the left dorsal lateral and bilateral medial prefrontal regions (59, 60). In patients with brain injury, damage to the left dorsal lateral frontal cortex is associated with clinical depression (61–65). Damage to the orbital frontal cortex is also associated with diminished ability in social decision making and detection of emotional clues (66, 67), leading to changes in personality (68). Taken together, these findings support the notion that the prefrontal cortex is part of a neural system engaged in detection of positive features in objects, events, and mental states.

The relative increased blood flow in the frontal lobe during the neutral/unpleasant comparison is consistent with the above hypothesis and may be explained by the different degree of pleasant valence detected by subjects.

Increased activity of primary and associative visual cortex was found in the unpleasant/pleasant comparison, as well as in the pleasant and unpleasant versus neutral comparisons. The observation of increased blood flow in the visual cortex during processing of emotional pictorial stimuli is frequent (4, 7, 9, 45, 69) but poorly understood. Since the experimental and neutral stimuli in our and other studies (4, 9) were matched for content complexity and luminance, it is arguable that some of the visual areas in the circuit may have been activated as a result of subcortical limbic back-signaling to visual cortex for a secondary assessment of the visual stimuli (33). Consistent with previous studies (45), visual association areas were relatively more active in the unpleasant/pleasant and unpleasant/neutral comparisons. Visual association areas may be preferentially involved in the evaluation of unpleasant visual stimuli in young individuals (7, 46).

In the pleasant versus neutral subtraction, evaluation of pleasant pictures increased blood flow bilaterally in the posterior cingulate (Brodmann’s area 23). Functions attributed to the posterior cingulate include monitoring sensory events and the organism’s own behavior with respect to spatial orientation and memory (70–74). However, the role of the posterior cingulate as a component of the limbic system (75) (and therefore in emotion) should probably be reevaluated in light of recent data. The posterior cingulate cortex of the macaque is strongly connected with orbital medial prefrontal cortex area 11m (76) and participates in instrumental avoidance learning in the rabbit (77). In humans, increased blood flow in the posterior cingulate is associated with classic conditioning (78), comprehension of narratives necessitating the attribution of mental states (79) or metaphors (80), and recall of emotionally laden personal memories (81). The left posterior cingulate showed increased activity during implicit recognition of emotion (9). These and our findings suggest that the posterior cingulate gyrus may be involved in implicit and/or explicit evaluation of past and present contextual and emotional experiences.

Attribution of unpleasant valence relative to neutral showed relative increased activity in a nucleus of the basal forebrain: the nucleus accumbens. This subcortical nucleus, which is part of the limbic striatum (82), receives major input from the amygdala (83, 84) and may influence the anterior cingulate via the ventral pallidum (85). Whereas the classic view has seen the nucleus accumbens playing a role in appetitive motivation and positive reinforcement (86–90), a variety of aversive and stressful situations (e.g., active avoidance behavior) increase dopamine release within the accumbens (91–95). Recently, the function of the accumbens has been conceptualized as linking motor and motivational processes that characterize goal-directed behavior (96).

Relative to the neutral condition, evaluation of either pleasant or unpleasant stimuli activated the left retrosplenial cingulate (Brodmann’s area 29) and the right precuneus (Brodmann’s area 31). These shared neural substrates may represent memory component necessary during attribution of pleasant or unpleasant character to emotionally charged stimuli. Consistent with this hypothesis, the precuneus has been shown to be involved in episodic retrieval (97) dependent on visual imagery (98), as well as in recall, planning, and associative thinking (99–101). We posit that in order for humans to be consciously aware of a stimulus (and therefore examine its affective valence), the sensory representations of the stimulus need to be compared with past experiences of that stimulus and associated emotions (33). This evaluation and its encoding in episodic memory, which involves the retrosplenial cingulate cortex (97), are simultaneous processes.

Understanding the physiological meaning of areas of relative decreased blood flow is particularly challenging. Suspension of activity during the passive task or rest state (102) or true neural inhibitions are proposed interpretations (103). In the present study there was no "rest" state in that subjects attributed valence during all tasks. Therefore, neural inhibition or relative increased blood flow during the neutral condition (7, 24, 99) may apply. These hypotheses help in the interpretation of two main findings in the present study that were not expected: the large cerebellar activation in the neutral condition (relative to both pleasant and unpleasant conditions) and the several frontal regions activated during neutral relative to unpleasant stimuli evaluation.

No verbal response or other movements were required during the PET experiment. Eye movement-related increased cerebellar flow during the most arousing tasks might have been expected in areas such as the vermis, fastigial nuclei, and floccular lobe (104–106). Instead, relative increased neocerebellar cortex blood flow was observed during the neutral condition. Consistent with a cerebellar role in cognition (24, 107–110) including attention (111), this finding may be explained by a relatively more sustained attentional demand during the neutral condition in light of a decisional process evidently less straightforward compared to the pleasant or unpleasant condition. The ­increased blood flow in the anterior cingulate in the neutral/unpleasant comparison also points to a greater attentional load.

Whereas it is convenient under scholarly principles to divide emotion into evaluative, experiential, and expressive components, in real-life situations they may, albeit to a different degree, occur in some combination. Similarly to an ecological situation, the subjects in this study may have experienced "some" emotional arousal during attribution of valence. Experimental investigations using PET are currently under way in our laboratory to dissociate the functional neuroanatomy of evaluation and experience of emotion.

In conclusion, the present study has traced the functional neural anatomy associated with attribution of emotional valence in visual stimuli. Cortical limbic regions were associated with pleasant and subcortical regions with unpleasant valence regardless of the comparisons between the different experimental conditions. In the direct comparison, detection of pleasant stimuli displayed increased prefrontal activity, whereas detection of negative stimuli exhibited increased activity in the amygdala. Understanding the normal neural circuitry that supports the affective evaluation of events will help to uncover the neuroanatomical basis of psychiatric disorders with prominent emotional disturbances such as depression, anxiety disorders, posttraumatic stress disorder, and schizophrenia.

Presented in part at the Third International Conference on Functional Brain Mapping of the Human Brain, Copenhagen, Denmark, June 19–23, 1997. Received July 14, 1998; revision received March 12, 1999; accepted April 19, 1999. From the Department of Psychiatry and the Department of Radiology PET Center, Mental Health Clinical Research Center, University of Iowa College of Medicine. Address reprint requests to Dr. Paradiso, Psychiatry Research-MEB, Department of Psychiatry, University of Iowa, Iowa City, IA 52242-1000. Supported in part by grants MH-31593, MH-40856, Clinical Research Center grant MH-43271, and Research Scientist Award MH-00625 from NIMH and an Established Investigator Award from the National Alliance for Research on Schizophrenia and Depression.