Obsessive-compulsive disorder (OCD) is a highly heritable neuropsychiatric disorder, with risk to first-degree relatives much greater than for the general population (1, 2). OCD may constitute part of a spectrum of disorders characterized by overlapping comorbidity, familiarity, and difficulties suppressing inappropriate repetitive behaviors (2, 3). So far, attempts to identify contributory genes have met with limited success (4). This may be because OCD comprises a heterogeneous illness that cannot easily be differentiated into genetically homogenous subgroups by using outward symptoms or signs (phenotypic markers). There is an ongoing search in psychiatry for intermediate markers of brain dysfunction (endophenotypes) that represent vulnerability markers for disease development and lie closer to the genetic origins of the disorder (5). In the context of OCD, it has been hypothesized that impairments in motor inhibition and cognitive flexibility may be useful in this regard (2, 6). Impairments in these domains have been identified in OCD patients with a variety of objective tests dependent upon integrity of frontal-striatal circuitry (2, 7, 8). To date, studies have not assessed the performance of unaffected first-degree relatives of OCD patients—a vital step in the identification of putative endophenotypes (5).

We assessed cognitive flexibility, motor inhibition, and decision making in unaffected first-degree relatives of OCD patients, patient probands, and matched healthy comparison subjects without a family history of OCD. Cognitive flexibility was assessed with the Intradimensional/Extradimensional Shift task, originally developed from the Wisconsin Card Sorting Test of frontal lobe integrity. This paradigm examines different components of attentional flexibility, including reversal learning, set formation, and the ability to inhibit and shift attention between stimulus dimensions (extradimensional shifting) (9). Prior studies have demonstrated that extradimensional shifting is impaired in OCD but not in trichotillomania (7, 8). Motor inhibition was assessed by using the Stop-Signal task, which provides a sensitive estimate of the time taken to internally suppress motor responses. This paradigm has been shown to be sensitive to motor impulsivity associated with attention deficit hyperactivity disorder, OCD, trichotillomania, and damage to the right inferior frontal gyrus (8, 10). Decision making was assessed by using the Cambridge Gamble task, which is sensitive to abnormal decision making in substance abuse, mania, and frontal lesions (7, 11–13). Prior work has identified intact decision making in OCD using this task (7). It was predicted that unaffected first-degree relatives may, like OCD patients themselves, show impaired cognitive flexibility and motor inhibition, thereby supporting such measures as trait markers.

The study group comprised 20 pairs of unaffected first-degree relatives and OCD patient probands and 20 comparison subjects without a known family history of OCD. Before enrollment, all potential participants undertook a clinical interview, including the Mini International Neuropsychiatric Inventory, the Montgomery-Åsberg Depression Rating Scale (MADRS), and an obsessive-compulsive personality disorder score card. The Mini International Neuropsychiatric Inventory is a well-validated screening instrument for axis I disorders (14), the MADRS is a well-established measure of mood status (15), and obsessive-compulsive personality disorder scores were assessed by using an eight-item checklist adapted from DSM-IV criteria for obsessive-compulsive personality disorder. Verbal IQ estimates were calculated with the National Adult Reading Test (16). First-degree relatives and comparison subjects were excluded if they had DSM-IV axis I disorders (including major depressive disorder, Tourette’s syndrome, eating disorders, OCD itself, and other anxiety disorders). OCD patients were recruited from an outpatient clinic by a board-certified psychiatrist on the basis of DSM-IV OCD criteria, archetypal washing/checking symptoms (without significant hoarding), and freedom from axis I comorbidities. The patients gave consent for a first-degree relative (preferentially a similarly aged sibling) to be contacted. The comparison subjects were recruited through advertisements in the local community. All subjects gave written informed consent, and the study was approved by the Cambridge Local Research Ethics Committee. Cognitive assessment was undertaken by an experienced neuropsychologist in a quiet testing environment.

The Intradimensional/Extradimensional Shift task is a nine-stage visual discrimination task that uses multidimensional stimuli. Two stimuli are displayed at a time, and feedback is provided so that the subject can learn which stimulus is correct. To pass each stage, six consecutive correct responses are required within 50 trials; otherwise, the task ends. The rule for correct responding is modified at the start of each task stage to dissociate different aspects of cognitive flexibility. For example, one rule would be that the correct stimulus is the one with three (rather than two) white lines. (See reference 9 for full task description.) The intradimensional shift stage examines rule generalization when novel stimuli are introduced, whereas the extradimensional shift stage examines the ability to inhibit and shift attention away from a previously relevant stimulus dimension (akin to a category shift on the Wisconsin Card Sorting Test). Performance is assessed in terms of the number of trials required to achieve learning criterion (six consecutive correct responses) at each stage. The subjects who fail to pass a stage are assigned 50 trials to criterion and excluded from analysis for subsequent stages not attempted. On the computerized Stop-Signal task, the subjects respond rapidly to left- or right-facing arrows onscreen with corresponding motor responses and attempt to inhibit responses when an auditory stop signal sounds. With a tracking algorithm, this task estimates the time taken to internally suppress prepotent motor responses (stop-signal reaction time; see reference 10). The Cambridge Gamble task (11) assesses different components of decision making by requiring volunteers to gamble points over a range of probabilities of winning. Decision making is quantified in terms of the percentage of rational decisions made and the overall mean percentage of points gambled.

Data were analyzed by using one-way analysis of variance (ANOVA) in the first instance. Where significant group differences were detected according to ANOVA, post hoc least significant difference tests were conducted in order to compare study groups in a pairwise fashion. If assumptions of homogeneity of variance and normality were not met, standard transformations were performed or data were subjected to nonparametric tests, as indicated. Significance threshold was p<0.05.

The groups were matched for age, verbal IQ, and male:female ratio (Table 1). As expected, the groups differed significantly on Yale-Brown Obsessive Compulsive Scale scores (obsession, compulsion, and total) because OCD patients scored higher than both other groups (least significant difference tests following significant ANOVAs in Table 1) (p<0.01) (all tests are post hoc least significant difference tests). The relatives did not differ significantly from the comparison subjects on Yale-Brown Obsessive Compulsive Scale scores (obsessions, compulsions, and total scores) (all p>0.30). Although mean MADRS scores for all groups were well beneath the cutoff for clinically significant depression, the groups differed on MADRS scores overall. Post hoc analysis revealed that this was because the OCD group showed higher scores than the relatives (p<0.01) and the comparison subjects (p<0.01). MADRS scores did not differ significantly between the relatives and the comparison subjects (p>0.20). The groups also differed overall for total obsessive-compulsive personality disorder scores. Post hoc analysis revealed that OCD patients had significantly higher obsessive-compulsive personality disorder scores than the relatives (p<0.01), who had significantly higher scores than the comparison subjects (p<0.01).

In terms of neuropsychological performance, the groups differed on trials to criterion for the extradimensional shift (Intradimensional/Extradimensional Shift task) and on stop-signal reaction times for the Stop-Signal task (Table 1). The groups did not differ on performance on the Cambridge Gamble task. For the Intradimensional/Extradimensional Shift task, post hoc analysis revealed that the first-degree relatives (p<0.01) and the OCD patients (p<0.05) required significantly more trials to attain criterion for the extradimensional shift stage in relation to comparison subjects. The relatives did not differ significantly from the patients on this measure (p>0.20). On the Stop-Signal task, post hoc analysis revealed that first-degree relatives (p<0.01) and OCD patients (p<0.05) showed significantly longer stop-signal reaction times (i.e., impaired motor inhibition) in relation to comparison subjects. Again, the performance of relatives did not differ significantly from that of patients (p>0.20).

Correlation analyses were conducted between clinical indices (Yale-Brown Obsessive Compulsive Scale obsession/compulsion subscores and total scores, MADRS total scores, and obsessive-compulsive personality disorder total scores) and neuropsychological indices. No significant correlations were found within groups (all p>0.20, Pearson’s r).

Impaired inhibition processes are implicated in the search for OCD endophenotypes (2). To the authors’ knowledge, this is the first study ever to assess cognition in unaffected first-degree relatives of OCD patients. In relation to matched comparison subjects without a family history of OCD, the relatives showed deficits in cognitive flexibility and motor inhibition but showed intact performance on a test of decision making. The profile of dysfunction on the neuropsychological tasks in unaffected relatives was indistinguishable from that of OCD patient probands and was comparable to prior findings in OCD patients (2, 7, 8).

These data support the utility of specific objective indices of inhibitory control (extradimensional shifting, stop-signal reaction times) as relatively easily measurable familial candidate endophenotypic markers for archetypal OCD. The relatives did not differ significantly from the comparison subjects on Yale-Brown Obsessive Compulsive Scale scores, and no significant correlations between Yale-Brown Obsessive Compulsive Scale scores and cognitive performance were identified. Therefore, these impairments appear to reflect trait markers that can exist in the absence of clinically significant symptoms and medication confounds. We included mainly female participants, which may have influenced the results. It remains possible that the present findings may not generalize to other OCD symptom clusters, such as hoarding, because we included only patients with archetypal washing/checking symptoms (17). Also, it would be of interest in future work to examine the differences between patients with early- and late-onset OCD and their relatives (18) and to examine the effects of family history, such as the frequency of psychiatric illnesses in the extended family. Future studies should seek out not only trait but also state markers of OCD and spectrum disorders by using a range of tests and examining links with genetic polymorphisms and frontal-striatal brain abnormalities (structural, functional, and neurochemical). They should also evaluate the clinical consequences of these cognitive abnormalities, and the effects of selective serotonin reuptake inhibitors and other drug treatments on cognition (19). This will help to elucidate etiological contributions and lead to improved models and treatment algorithms for OCD and obsessive-compulsive spectrum disorders.