Cognitive Correlates of Job Tenure Among Patients With Severe Mental Illness

There is a sizable literature documenting the importance of multiple measures of cognitive performance for functional outcomes in schizophrenia. There is replicated evidence that cognition is related to current level of functioning in the community as well as the ability to benefit from rehabilitation interventions (1, 2). Thus, in broad terms, cognitive strengths appear to facilitate adaptive functioning. Conversely, cognitive deficits may be important obstacles to psychosocial rehabilitation. One clear implication of these data is that patients most in need of rehabilitation may be the least able to benefit from such interventions, highlighting the challenge of reducing the disability associated with schizophrenia.

The literature on vocational performance in schizophrenia as an outcome dimension is surprisingly small. The literature is split between studies examining cognitive correlates of performance in vocational rehabilitation settings and those examining competitive employment status. Bell and colleagues (3–6) have conducted several studies of cognitive performance and vocational functioning in rehabilitation settings. They have documented that aspects of cognitive performance at baseline, including scores on verbal memory and the Wisconsin Card Sorting Test, are related to ratings of work behavior/performance. In addition, baseline performance on the Wisconsin Card Sorting Test, Digit Symbol, Hopkins Verbal Learning Test, and the Continuous Performance Test predicted improvement in patient work performance over 6 months in a work rehabilitation program. These measures have also emerged in the broader outcome literature (1, 2), suggesting that the predictors of vocational performance may overlap with predictors of other aspects of instrumental functioning.

A number of positive findings have been reported in studies examining cognitive correlates of competitive employment. These findings range from measures of verbal/academic competence to overall accuracy of smooth-pursuit eye movements to verbal memory to block design (7–12). McGurk and Meltzer (13) reported that patients holding competitive employment or enrolled in school full-time demonstrated superior performance across measures of sustained attention, working memory, problem solving, and episodic memory when compared to unemployed patients, with the scores of part-time workers falling between the other two groups.

These data are consistent with our prior reports that competitively employed patients demonstrate clear cognitive strengths relative to patients disabled for the long term. In a study of archival data (14), employed patients performed better than unemployed patients across multiple measures, with the largest effect sizes on measures of social reasoning, verbal fluency, visual memory, and Stroop word reading. In a second study (15), patients currently working 20 or more hours a week in competitive settings differed from all other clinic patients on four of the five subscales as well as on total score on the Repeatable Battery for the Assessment of Neuropsychological Status, a brief screening measure of attention, memory, language, and visuoconstructional abilities. In a prior report on the group used in the current study (16), we found that poorer performance on the Repeatable Battery for the Assessment of Neuropsychological Status and measures of general intellectual competence was associated with continuous unemployment over the 5 years preceding study entry. Thus, it appears that cognition is reliably associated with concurrent and past history of vocational functioning, but there is little compelling evidence that specific aspects of cognition (i.e., memory, attention, problem solving, etc.) are differentially predictive of this outcome. Our prior studies, like others in the literature, have a number of methods limitations, including small groups, limited test data, and reliance on patient self-reporting of distant events. Thus, the issue of specific relationships between cognition and vocational functioning remains largely unexplored in the literature.

This study examined the role of baseline neurocognitive performance as a predictor of outcome in a large clinical trial comparing the efficacy of two different approaches to vocational rehabilitation with outcomes assessed over a 2-year follow-up period. This study design facilitated evaluating two clear predictions of whether cognitive functioning is a mediator of the ability to benefit from rehabilitative interventions as well as a correlate of vocational status. First, we expected that patients who were able to obtain competitive employment would demonstrate superior cognitive performance relative to patients who remained unemployed throughout the follow-up interval. Second, we expected that the degree of success in maintaining employment should also bear a relationship to cognitive performance. Third, an exploratory examination of the pattern of associations between cognitive measures and vocational success would potentially shed light on the possible differential role of specific cognitive functions.

Method

Overview of Study Design

These data were collected at the Baltimore site of the Employment Intervention Demonstration Project. This eight-site project was designed to examine the efficacy of several approaches to vocational rehabilitation for patients with severe mental illness. The Baltimore site investigated the comparative efficacy of the individual placement and support model, developed by Drake et al. (17), versus the current service model, which emphasizes a continuum of transitional and supported employment services. The study involved random assignment of patients to rehabilitation conditions, with long-term assessment of vocational and nonvocational outcomes.

The individual placement and support model emphasizes the integration of vocational and clinical services with the goal of rapidly achieving competitive employment, bypassing many of the prevocational activities common in rehabilitation programs. Unlike the traditional models of vocational rehabilitation, individual placement and support require that the employment specialist, psychiatrist, and case manager be part of the same service team and share in decision making. The employment specialist, client, and treatment team then develop an individualized approach geared directly toward rapidly achieving competitive employment. The individual placement and support model seeks employment opportunities that are consistent with the clients’ preferences, skills, and abilities. Working with small caseloads, the individual placement and support specialist is available to focus on rapid job search and placement and provide continued follow-up support. (For a complete description of the individual placement and support model, see Bond et al. and Drake et al. [18, 19]. A complete description of study methods and results from the Baltimore site is found in Lehman and colleagues [20].)

Study Group

The study group consisted of 150 patients in the Employment Intervention Demonstration Project who completed the neuropsychological assessment and received psychiatric care from one of three continuous-care teams or a general outpatient clinic within a university-run community mental health agency that serves inner-city Baltimore. The population of this catchment area is predominantly African American and lower income. The total Baltimore Employment Intervention Demonstration Project study group of 219 patients was randomly selected from the roster of all patients currently receiving psychiatric care in this treatment system. These care teams provide a comprehensive array of outpatient/inpatient services, including substance abuse, mobile, and assertive community treatment. The majority of the subjects, 68% (N=102), were drawn from the assertive community treatment and mobile treatment teams that provide care for the most severely disabled patients.

Study inclusion criteria required that subjects be receiving psychiatric-disability-based income support or have a diagnosis of schizophrenia or schizoaffective disorder. Patients could also qualify if they had an extensive prior hospitalization history (requiring two or more psychiatric hospitalizations of >21 days for any psychotic or mood disorder over the previous 3 years at least 42 days prior to a current hospitalization or 90 or more total days in a psychiatric or nursing home during the past 3 years). In addition, patients not meeting either of the first two criteria could also enter the study if they had a history of mental disorder lasting for at least the past year, during which they were unable to spend at least 75% of their time in some gainful activity because of a mental disorder. All patients had to have been unemployed for a minimum of 3 months before study entry.

As part of study participation, subjects underwent an extensive baseline clinical psychiatric evaluation, including the Structured Clinical Interview for DSM-IV Axis I Disorders, Patient Edition (21), that was combined with medical records and clinical reports to establish a research diagnosis. Participants were approached to participate in the neuropsychological testing after completion of the baseline assessment; 150 of 219 patients consented to participate. Patient decision, failure to keep multiple scheduled appointments, and the fact that the Employment Intervention Demonstration Project recruitment period extended slightly beyond the period when testing personnel were readily available explain the difference between the number of subjects with cognitive data versus the entire Baltimore Employment Intervention Demonstration Project study group. Since the tested (N=150) versus nontested (N=69) groups did not differ significantly on multiple baseline demographic characteristics (including age, years of education, distribution of diagnosis, or past work history), it appears likely the results from the tested group generalized to the entire study group.

After study entry, the vocational status (job start dates, job end dates, salary, hours worked, benefits, level of mainstream integration) of the subjects was documented on a weekly basis throughout the 24-month follow-up period. In the event that participants withdrew from the study or were lost to follow-up, their last vocational observation (which in every instance was unemployment) was carried forward. The analyses reported were based on the total number of hours worked by the 12-month and 24-month follow-up time points.

Diagnostic and Demographic Features

The majority of the study group, 74% (N=111), had a diagnosis of psychotic disorder, 91% (N=101) of whom were diagnosed with a schizophrenia spectrum disorder. The remaining 26% (N=39) of the study group received a mood disorder diagnosis (most often bipolar disorder). The mean age of study participants was 39.8 years (SD=8.2). Approximately half (48%, N=72) of the study group did not finish high school or receive a General Equivalency Diploma. Only 7% of the study group attended 1 or more years of college.

A minority of the subjects, 25% (N=38), were Caucasian, 65% (N=98) were African American, and 9% (N=14) were of another racial group. There were more men (N=86, 57%) than women (N=64, 43%). The majority of the subjects (N=114, 76%) had a lifetime substance use diagnosis, with problematic use in the last year evident in 53% (N=79) of the study group. The study group also demonstrated significant long-term functional disability. The majority of the subjects, 90% (N=135), were receiving Supplemental Security Income, Social Security Disability Insurance, or both. Participants had been hospitalized on average 12.2 times (SD=9.9). Although most subjects, 95% (N=143), had a lifetime work history, only 68 (45%) had held a job in the last 5 years.

All participation was voluntary, and patients were reimbursed for participating in the Employment Intervention Demonstration Project study and for the cognitive testing. Written informed consent was obtained from all subjects after complete description of the study.

Neuropsychological Procedures

Subjects received a test battery, which typically took 2.5–3.0 hours to complete. The test battery was designed to provide an estimate of general intellectual ability (a WAIS-III [22] estimated IQ was generated from four subtests), language/academic ability (the Wide-Range Achievement Test 3 [WRAT-3] reading subtest [23] and the Woodcock-Johnson listening comprehension subtest [24]), aspects of attention (the Stroop Color-Word Test [25], the Trail Making Test [26], and the Continuous Performance Test [27]), memory (logical memory and visual reproduction subtests drawn from the Wechsler Memory Scale, 3rd ed. [28]), executive functioning and working memory (a 64-card version of the Wisconsin Card Sorting Test [29], verbal fluency, and letter-number span from the WAIS-III), and motor functioning (Purdue Pegboard [30] and Finger Tapping [26] tests). Subjects also completed the Repeatable Battery for the Assessment of Neuropsychological Status (31). Tests were administered and scored as specified in test manuals, with conversions to age-scaled scores or demographically corrected T scores whenever possible. Only 64 cards were administered for the Wisconsin Card Sorting Test, and scoring was modified to give partial credit for number of categories completed if subjects finished the test with three or more items correct in a row.

Data Analysis

The data analyses addressed two major questions: 1) are there cognitive differences between patients who did and patients who did not achieve employment? and 2) is cognitive performance related to the extent of employment (defined as number of hours worked) over the follow-up period? To address the first question, we compared the performance of the 40 patients who obtained employment by the 24-month follow-up to the 110 who did not using t tests for individual tests and composite cognitive continuous measures. We did not conduct post hoc corrections for multiple tests since such correction was not necessary.

The analyses for the second question were quite complex because of the distribution of the hours-worked score—the critical study outcome measure. The majority of the subjects worked very few hours (75% [N=30] had worked 415 hours or fewer by the 24-month follow-up), but a small minority worked substantially more. This positive skew resulted in violation of the assumptions of hypothesis tests for Pearson’s correlations and multiple regression techniques. After exploring multiple options, we decided to treat the data in two ways. First, although the data violated the distributional assumptions for Pearson’s correlations, such an analytic approach actually captured the underlying clinical significance of the association. That is, the outliers in outcome were also likely to be the cognitive outliers. Thus, Pearson’s correlations reflected this relationship. However, this approach is open to the criticism that results driven by such observations of outliers may be less likely to generalize to other study groups. Second, in order to attenuate the impact of the outliers, we also performed Spearman’s correlations. This analytic approach was based on the rank ordering of the cognitive test results and the total hours worked and treated both the cognitive scores and the total hours worked as though they were equally spaced from one another. Therefore, Spearman’s results were much less susceptible than Pearson’s correlations to the impact of extreme or outlier values. Insofar as such outliers were present in the observed distribution and that these extreme values were clinically meaningful, Spearman’s results failed to reflect the role of such observations. We present both analytic approaches for the 12- and 24-month follow-up time points to provide a comprehensive description of the data.

The skewed distribution of the outcome variable precluded a multivariate approach to examining the relationship between cognitive functioning and outcome. Specifically, multiple regression models of the relationship of cognition and hours worked resulted in the prediction that certain subjects would work “negative hours”—an impossibility—which suggests that such models could not be legitimately applied to the data. In order to simplify data presentation, we created several composite scores that were derived from tests with multiple conditions after first determining that the composite score behaved similarly to the original tests in terms of relationship to the outcome variables. The three Stroop conditions were averaged, the three Purdue Pegboard conditions were averaged, as were the two Finger Tapping scores, and we averaged the scaled scores from the immediate and delayed recall conditions of the logical memory and visual reproduction subtests to generate overall measures of auditory and visual memory.

Results

Work Outcomes

A total of 40 study participants (27%) obtained employment over the 24-month follow-up interval, and 33 obtained employment during the first 12-month follow-up. The vast majority of jobs were in the services industry (restaurant, janitorial, etc.). Of these 40 participants, 35 (88%) had been randomly assigned to the individual placement and support condition, and five (13%) had been randomly assigned to standard rehabilitation services. Thus, there was a clear effect of treatment condition, consistent with the findings in the total group, as presented by Lehman et al. (20). The 40 subjects who obtained employment did not differ from the 110 who did not obtain employment on multiple demographic features, including age, distribution of ethnicity, gender distribution, education history, psychotic diagnosis, age of first hospitalization, or substance abuse (history or current). A complete analysis of the role of demographic factors is presented in the main report for the total study group and was not undertaken here in this subgroup.

The participants who obtained employment worked an average of 214.79 hours (SD=268.73) by the 12-month follow-up; the average had increased to 388.37 hours (SD=486.93) by the 24-month follow-up. As noted, the distribution of hours worked was skewed (similarly at both follow-ups; only the 24-month data are described in detail). Quartiles for total hours worked at the 24-month follow-up were as follows: first quartile: total=1–36 hours, second quartile: total=37–135 hours, third quartile: total=136–415 hours, fourth quartile: total=416–2,425 hours. Patients in these quartiles worked the following average number of hours: first quartile: mean=19.50 hours, second quartile: mean=87.00 hours, third quartile: mean=260.10 hours, fourth quartile: mean=942.70 hours. Thus, the bottom 75% of the group all worked less than 415 hours, and an average of 122.20 hours was worked by the group. The average number of hours worked by the top quartile was nearly eight times higher than the mean worked by the bottom 75% of the workers.

Cognitive Features

The mean performance of the 40 patients who obtained employment on the major cognitive variables is shown in Table 1. The overall group demonstrated marked cognitive impairment relative to the performance of test standardization samples, with multiple scores approximately two standard deviations below the normal mean. It appears that much of the impairment in this group stemmed from early in development, as seen in the poor performance on measures of academic skill (WRAT-3 reading), expressive vocabulary (WAIS-III vocabulary), and receptive vocabulary (Woodcock-Johnson listening comprehension). Measures of basic academic and language skills are typically considered to be markers of premorbid competence (32). Thus, it is likely that the overall level of impairment documented in this group represents a combination of premorbid and illness-onset-related deficits.

Baseline cognitive performance

We compared the baseline cognitive performance of subjects who did and those who did not achieve employment on the measures shown in Table 1. The groups performed nearly identically, without a single statistically significant difference. For example, employed versus unemployed groups obtained the following mean scores, respectively, on measures of full-scale IQ (mean=72.0, SD=11.5, versus mean=73.7, SD=11.4), scaled scores of immediate and delayed story recall (mean=5.99, SD=2.8, versus mean=6.17, SD=2.8), T score on the Trail Making Test Part B (mean=30.42, SD=12.8, versus mean=30.95, SD=11.5), and the Wisconsin Card Sorting Test percent of perseverative errors (mean=24.49, SD=12.3, versus mean=25.21, SD=11.4). These results clearly contradict the prediction that patients with cognitive advantages were more likely to obtain employment in the context of this clinical trial.

Cognitive performance and duration of employment

The results from the Pearson’s and the Spearman’s correlations for the 12- and 24-month follow-up intervals are shown in Table 1. Cohen has suggested that r coefficients at or above 0.30 should be considered as having a medium effect size and r values at or above 0.50 should be considered as a large effect size (33). At the 12-month follow-up, 21 of 26 Pearson’s correlations met or exceeded the medium-effect-size criteria, and four of 21 met the large-effect-size standard. The Spearman’s results demonstrate a somewhat less robust degree of association at 12 months: data for 14 of 26 cognitive measures met or exceeded the 0.30 criteria for a medium effect size, but none of the values met the criteria for a large effect size. There was a high degree of concordance between the two types of correlations at the 12-month follow-up: 12 measures met the criteria for a medium effect with both Spearman’s and Pearson’s approaches.

The 24-month results were less robust overall, and there was a clear divergence between the Pearson’s and Spearman’s results. The majority of the Pearson’s coefficients (16 of 26) continued to meet or exceed the criteria for a medium effect size. However, only four of the Spearman’s values reached the medium-effect-size criteria, and scores for only two tests (the Continuous Performance Test’s distractibility and the verbal fluency test) demonstrated medium effect sizes in both approaches at the 24-month follow-up. The small number of subjects who worked the most hours drove the Pearson’s results; these subjects also demonstrated better cognitive performance. However, this relationship was much less evident throughout the distribution of the patients with the bottom 75% of the hours worked, resulting in a marked attenuation of the Spearman’s coefficients relative to the Pearson’s values. Thus, the cognition-hours worked relationship was much less orderly at the 24-month follow-up than at 12 months, when the Pearson’s and Spearman’s results were fairly consistent.

Cognitive specificity and outcome

The fact that more than half of the test variables demonstrated a medium effect size in association with job tenure at the 12-month follow-up (and at 24 months with the Pearson’s coefficients) argues against any strong claims that specific aspects of cognition bear a differential relationship to employment outcome. We have previously presented a principal-components analyses of this cognitive battery (34); six factors were identified, described as representing general intellectual ability, attention, visual memory, vigilance, executive, and motor functions. In this study, medium-effect-size Pearson’s correlations were obtained for at least one of the main factor markers for all six of the factors. This broad pattern of association is consistent with the fact that the most robust, consistent relationships were observed with multiple measures from the first factor (general intellectual ability), including estimated IQ, which had the highest Pearson’s correlation of any measure at the 12-month follow-up (r=0.59).

Of interest, only two tests (the verbal fluency test and the Continuous Performance Test’s distractibility subtest) met the medium-effect-size criteria for both time points with both correlational techniques. These two variables could be interpreted as highlighting the importance of attention and volitional response generation as mediators of functional outcome. In light of the fact that deficits in these functions are thought to be central to schizophrenia, this could lead to a conclusion that the specific cognitive pathology of the disorder mediates outcome. Although this simplified interpretation of the data is attractive, the results suggest a more general relationship, given the breadth of associations of similar magnitude across multiple cognitive functions observed at the 12-month follow-up with both Pearson’s and Spearman’s approaches. That is, the pattern of relationships between cognition and hours worked is clearly general, with only modest differences in the extent of association observed with specific variables, particularly at the 12-month follow-up.

Verbal memory has previously been reported to be a significant correlate of functional outcome (1, 2, 12). Our primary verbal memory measure, logical memory, correlated significantly with hours worked at the 12-month follow-up with both Pearson’s and Spearman’s coefficients, but it fell below the criteria for a medium-effect-size at 24 months. Given this unexpected result, we performed additional analyses on the two verbal memory measures from the Repeatable Battery for the Assessment of Neuropsychological Status. Medium-effect-size correlations were observed with both measures in relationship to total hours worked: list learning total score (r=0.30, p=0.06) and story memory score (second-trial recall) (r=0.33, p<0.05). Thus, we did find a relationship between measures of verbal memory and outcome. However, the memory-outcome relationship does not appear to be unusually selective or robust.

Comment

These data suggest that cognitive performance is an important predictor of job tenure among patients with severe mental illness. Although baseline cognitive performance failed to differentiate patients who obtained employment versus those who remained unemployed, better baseline cognitive performance was associated with the number of hours worked at the 12- and 24-month follow-ups. The cognition-outcome relationship appears to be mediated by general level of cognitive ability, with little evidence that specific aspects of cognitive performance bear a clear differential relationship to total hours worked. This conclusion is consistent with the large literature on personnel selection that has demonstrated the importance of general mental ability as a predictor of job performance across a wide range of job types (35).

In other study groups, we have observed large cognitive differences between patients who held sustained competitive employment versus persistently unemployed patient groups (14, 15). Those findings are broadly consistent with the present results concerning duration of employment. In our prior studies, patients had to have held employment over prolonged intervals to be considered employed and thus would have worked more hours over 24 months than nearly all of the current subjects. Although it is difficult to directly compare findings from prospective and retrospective study designs, it appears that cognition is broadly associated with competitive employment. This relationship is most easily observed when study groups include subjects who have held competitive employment for sustained periods of time.

In contrast, we did not observe any hint of a difference in baseline cognitive performance in the subjects who achieved employment versus those who failed to do so. However, the vast majority of subjects who achieved employment worked quite briefly, suggesting that cognition may be a rate-limiting factor for vocational rehabilitation if competitive employment is the outcome of interest. In essence, many of the patients who obtained employment in this group—nearly all with the individual placement and support intervention—appeared to lack the cognitive competencies needed for sustained employment success. We previously reported a mean total scaled score of 86.8 (SD=12.6) on the Repeatable Battery for the Assessment of Neuropsychological Status among patients working 20 or more hours a week in competitive settings (without receiving vocational services), while unemployed patients had a mean score of 70.8 (SD=19.3) in a cohort drawn from the Maryland Psychiatric Research Center. The mean score on the Repeatable Battery for the Assessment of Neuropsychological Status for the patients who obtained employment in the current study (66.43, SD=17.53) was slightly below that observed in our previous unemployed group and far below the level observed among employed patients. This level of impairment, however, may not be a barrier to obtaining competitive employment if supportive vocational services are available. This interpretation of the results suggests that patients with marked cognitive deficits may need additional forms of support to sustain employment and highlights the need for novel treatment approaches that target cognitive functioning.

It should be made clear that this is not a criticism of the individual placement and support model; the intervention clearly led to a large increase in job starts, with a smaller number of examples of sustained employment. The observed low overall rate of sustained employment should not be surprising in an inner-city study group that was poorly educated, has substantial substance abuse comorbidity, marked premorbid and illness-associated cognitive deficits, as well as a history of severe and persistent mental illness. In short, this study group had multiple serious disadvantages and handicaps that likely reduced the likelihood of employment success.

It could be argued that these characteristics of the study group were a limitation, since it is difficult to untangle the contribution of these multiple factors and the results may not generalize to other, more typical, research populations. We argue that the present study group might be more representative of the clinical norm than many research groups for which comorbid substance abuse is often an exclusion criterion. Furthermore, the fact that we were able to detect a cognition-outcome “signal” in such a “noisy” context increases confidence that the results would generalize to other settings.

One practical implication of these data is that cognitive assessment may be helpful in identifying patients who are most and least likely to succeed in vocational rehabilitation settings. Such data may be helpful in individualizing treatment goals to actual competencies and limitations. It appears that a 2–3-hour neuropsychological battery is more than adequate to provide this information, since many of the most robust correlations were observed on measures that take only minutes to administer. Additional data from different study groups are needed before it is possible to suggest a specific set of brief instruments for these purposes. It is quite likely that a limited set of cross-validated, user-friendly cognitive procedures could be identified with limited additional research if this goal were considered part of the study design.

In summary, these data demonstrate that cognitive performance may be an important predictor of long-term vocational rehabilitation outcomes among patients with severe mental illness. Baseline cognitive performance was related to the number of hours that patients worked over a 2-year follow-up interval. Cognitive performance did not distinguish patients who worked versus those who remained unemployed. It appears that the impact of the rehabilitation intervention was to facilitate the achievement of employment among subjects, many of whom may have lacked the cognitive competencies needed for long-term job success. More research is needed to understand treatment strategies that enhance the capacity to sustain employment.

TABLE 1

Enlarge table