Cognitive Training for Supported Employment: 2-3 Year Outcomes of a Randomized Controlled Trial

Supported employment has been repeatedly demonstrated to improve competitive work outcomes in people with severe mental illnesses such as schizophrenia and bipolar disorder (1) . Although strong research supports the effectiveness of supported employment, and efforts are underway to increase access to these programs (2 , 3) , not all participants benefit from supported employment. Specifically, across most studies between 20% and 40% of patients do not find jobs (4 – 8) , and of those who do, many have brief job tenures that end unsuccessfully, such as being fired for poor work performance (9 , 10) . These findings have stimulated efforts to address illness-related impairments that may limit the effectiveness of supported employment for some individuals with severe mental illness.

Impaired cognitive functioning is a potent clinical predictor of response to a variety of psychiatric rehabilitation interventions (11 – 15) . Furthermore, cognitive functioning is predictive of work in persons with severe mental illness, including in patients receiving supported employment (16) . These findings have led to the development of cognitive rehabilitation programs aimed at improving cognitive functioning and the effects of vocational rehabilitation programs (17 – 19) .

This article describes the results of a controlled evaluation of a cognitive training program for supported employment: the Thinking Skills for Work Program (19) . In this program, cognitive training is provided by a cognitive specialist who works in collaboration with the supported employment specialist to provide cognitive remediation and to develop compensatory strategies for cognitive impairments that interfere with work functioning. The program includes four components: 1) Cognitive assessment and job loss analysis aimed at understanding how cognitive problems have interfered with past job performance and to motivate the patient to participate in the cognitive training program. 2) Computer-based cognitive training sessions (N=24) delivered over 12 weeks using a commercially available program shown to improve cognitive functioning in schizophrenia (20) . 3) Review and discussion of gains made following cognitive training, and collaborative planning with the patient and employment specialist about preferred jobs and coping strategies for any persistent cognitive impairments. 4) Ongoing consultation with the employment specialist and the patient to develop additional compensatory strategies to manage any cognitive impairments that interfere with job performance.

Preliminary results from a randomized controlled trial comparing supported employment with this cognitive training program to supported employment alone indicated that participation in cognitive training was associated with significantly greater improvements at 3 months in performance on a neuropsychological battery and in autistic preoccupation and depression symptoms per the Positive and Negative Syndrome Scale, as well as better competitive work outcomes at 1 year (19) . The present report provides the 2–3 year employment and hospitalization outcomes for this study.

Method

The study took place at two community-based rehabilitation centers in Brooklyn, N.Y. Both agencies serve predominantly minority patients and provide a comprehensive range of services, including housing, psychiatric, community support, day treatment, psychosocial, and integrated vocational services (including supported employment) to adults, children, and families with severe mental illness, mental retardation, or substance abuse.

Participants

The study was approved by the institutional review board of the Mount Sinai School of Medicine. All participants provided written informed consent. Eligibility criteria for participation included severe mental illness as defined by the State of New York Office of Mental Health, current unemployment, desire for employment, enrollment in supported employment, history of at least one unsatisfactory job ending (getting fired from a job held less than 3 months or walking off a job without another job in place), and willingness and capacity to provide informed consent.

Forty-four patients (20 women and 24 men; mean age=37.6 years [SD=9.9]; mean education=11.2 years [SD=1.68]; African American: N=30, Hispanic: N=7, Caucasian: N=6, Asian: N=1) with severe mental illness (schizophrenia [N=32], schizoaffective disorder [N=2], or mood disorder [N=10]) were randomly assigned to supported employment (N=21) or supported employment with cognitive training (N=23). At study entry, 38 (86%) patients were prescribed antipsychotic medications, which included risperidone (N=11), haloperidol (N=9), olanzapine (N=8), quetiapine (N=7), aripiprazole (N=4), clozapine (N=3), ziprasidone (N=2), fluphenazine (N=2), and thorazine (N=1). The majority of patients were receiving one antipsychotic medication (N=29); nine patients were receiving two or more antipsychotic medications. Of the 38 patients receiving an antipsychotic, 16 had also been prescribed an antiparkinsonian agent, 14 a mood stabilizer, 17 an antidepressant, and three a benzodiazepine. Of the six patients who had not been prescribed an antipsychotic, four had been prescribed a mood stabilizer, and three had been prescribed an antidepressant.

Previous comparisons of the supported employment with cognitive training and supported employment alone groups on demographic, diagnostic, and baseline clinical and neuropsychological characteristics using t tests and chi square analyses indicated no significant differences between the two groups (19) .

Computer Cognitive Training

Patients were engaged in approximately 24 hours of computer-based cognitive exercises (Cogpack, version 6.0; Marker Software, Mannheim, Germany), which involved practice across the broad range of cognitive functions that are impaired in persons with severe mental illness, including attention and concentration, psychomotor speed, learning and memory, and executive functions. Exercises practicing all of these areas of cognitive functioning are included within the first six cognitive training sessions, with additional sessions then focusing on further practice. Sessions required 45–60 minutes to complete, with patients usually completing two to three sessions per week for a total duration of about 12 weeks. Patients receive performance scores reflecting accuracy and speed following completion of each exercise, which are recorded and referred to in order to reinforce performance progress. The computer exercises are designed to be enjoyable and reinforcing to complete, with difficulty gradually increasing over time.

Measures

Competitive work was defined, according to the Substance Abuse and Mental Health Services Administration Employment Intervention Demonstration Project, as jobs paying minimum wage or higher, “owned” by the individual (and not the agency), not set aside for a person with a disability, and integrated in the community (7) . All paid work obtained by study participants met these criteria. Work outcomes were tracked continuously throughout the follow-up period by regular contacts with patients and vocational staff members. Dates and duration of psychiatric hospitalizations were tracked prospectively by the cognitive specialists, employment specialists, or direct subject contact.

The average length of follow-up was 26.8 months (SD=8.4, range=5–35) for the supported employment with cognitive training group and 24.3 months (SD=9.5, range=6–35) for the supported employment alone group, which did not differ significantly.

Fidelity to the principles of supported employment was rated with the Supported Employment Fidelity Scale (7) , a 15-item, 5-point anchored Likert scale tapping domains of staffing, organization, and services. Possible total scores range from 15 to 75, with high numbers representing better fidelity. The first two authors conducted the fidelity assessments, with sites receiving ratings of “fair” (total score=60) and “good” (total score=66) implementation.

Statistical Analyses

Employment outcomes were aggregated by computing the total number of jobs, hours worked, and wages earned over the entire follow-up period. Potential interactions between site, treatment group, and work were evaluated by performing an analysis of variance (ANOVA), with site and treatment group (supported employment with cognitive training or supported employment alone) as the independent variables and the total number of jobs worked as the dependent variable. A chi-square analysis was conducted to compare the two treatment groups on percentage of patients who obtained work over the course of the follow-up period. As other employment data on hours worked and wages earned were severely skewed, Mann-Whitney U tests were conducted to compare employment outcomes between the two treatment groups. Site and group differences in hospitalizations and numbers of days in the hospital over the follow-up period were evaluated by performing two ANOVAs, as described earlier.

In order to explore the relationship between hospitalization and employment, a chi-square analysis was performed comparing work during the follow-up period (yes/no) with hospitalization (yes/no). In addition, we computed Spearman’s correlations between number of days in the hospital and total jobs held, hours worked, and wages earned. All statistical tests were two-tailed with alpha set at 0.05.

Results

Competitive work outcomes for the two treatment groups are summarized in Table 1 . The ANOVA comparing the group and site effects on total number of jobs obtained resulted in a significant site effect (F=31.05, df=1, 39, p<0.0001) and significant treatment group effect (F=9.47, df=1, 39, p=0.004) but a nonsignificant treatment group-by-site interaction. Patients at site 1 worked more jobs over the 2–3 year follow-up period than patients at site 2 (1.2 versus 0.6, respectively), and patients in the supported employment with cognitive training program worked more jobs than patients in the supported employment alone program. Similarly, significantly more patients worked in the supported employment with cognitive training program than the supported employment alone program (χ ² =18.0, df=1, p<0.001). Last, patients in the supported employment with cognitive training program held more jobs (Z=–4.4, p<0.001) and over the follow-up period worked more weeks (Z=–3.9, p<0.001), worked more hours (Z=–4.1, p<0.001), and earned more wages (Z=–4.1, p<0.001) than patients in the supported employment alone program. Figure 1 depicts the percentage of people working per month in each treatment group through month 26 of follow up (the month for which work data were available for at least 50% of the participants in each group).

Enlarge table

ANOVAs indicated significant site effects for number of hospitalizations (F=12.35, df=1, 40, p=0.001) and days in the hospital over the follow-up period (F=9.86, df=1, 40, p=0.003) but no significant effects for either treatment group or the site-by-treatment group interaction. Patients at site 1 had significantly fewer hospitalizations than patients at site 2 (mean=0 versus 1.7) and spent significantly fewer days in the hospital (mean=0 versus 29.7 [SD=35.2, range= 0–119]).

The patients who had been hospitalized were less likely to have worked over the follow-up period (21%, N=4 of 19) than patients who had not been hospitalized (60%, N=15 of 25), a significant difference (χ ² =6.7, df=1, p=0.01). Similarly, the number of days in the hospital was significantly correlated with the number of jobs (r _s =–0.34, p=0.02), hours worked (r _s =–0.38, p=0.01), and wages earned (r _s =–0.39, p=0.01), indicating that patients who spent more time in the hospital worked less.

Discussion

Patients who were randomly assigned to the Thinking Skills for Work program in addition to supported employment had significantly better employment outcomes than those who only received supported employment, including obtaining more jobs, working more hours, and earning more wages over the 2–3 year follow-up period. These findings extend results from the 1-year outcome study (19) and suggest that the cognitive training program may have contributed to improved work outcomes for patients with job failures who were enrolled in supported employment. However, cognitive functioning and symptoms were not evaluated at the 2–3 year follow-up, so it is unclear whether improved cognitive functioning or symptoms, or other factors, contributed to the better vocational outcomes.

The study design did not include an “attention control group,” and thus it is possible that the superior work outcomes of patients in the Thinking Skills for Work Program could have been due to higher levels of staff attention. Aside from the question of the nonspecific effects of this program, the role of the cognitive training component versus the other components of the program (e.g., collaborative assessment and job planning, development of coping strategies for persistent cognitive impairment) in improving work outcomes cannot be determined. Cognitive functioning has prospectively been linked to work outcomes (16) , and patients in the Thinking Skills for Work Program demonstrated greater improvements in cognitive functioning at the 3-month assessment. In addition, the pattern of work over the study period indicated a steady rise in employment rates for participants in the Thinking Skills for Work Program during the 5–7 months after program enrollment, or 2–4 months after completion of computer cognitive training, which was sustained throughout the remainder of the follow-up period. Considering the time required to get a job in supported employment (5 , 7 , 8) , the time course of these gains in work is consistent with what might be expected following an improvement in cognitive functioning from cognitive training. However, other program components could have been equally or more important in improving vocational outcomes. For example, positive learning experiences during cognitive training could have bolstered self-esteem, self-confidence, and motivation (22) , contributing to better employment outcomes. The fact that patients in the Thinking Skills for Work program also showed significant improvements in depression over the 3-month cognitive training component suggests that the impact of the program is not limited to cognitive functioning. It is also possible that learning coping strategies to deal with persistent cognitive difficulties may have contributed to more successful job experiences and better work outcomes (23) .

Work outcomes differed significantly between the sites. The site with poorer work outcomes (site 2) also had lower fidelity to the supported employment model. Higher fidelity is associated with better work outcomes (24) . Thus, relatively poorer adherence to principles of supported employment may have contributed to more modest work outcomes at this site.

Patients at site 2 also had higher rates of psychiatric hospitalization over the course of the study. Furthermore, patients who were hospitalized worked fewer hours and made less wages. Hospitalization can have disruptive effects on employment in patients with schizophrenia and other severe mental illnesses, both because it removes them from the community and because time is required to recover from a symptom relapse (25 – 29) . Employment outcomes of the patients at site 2 might have been better if more effective relapse prevention services had been in place, such as training in illness self-management (30) , family psychoeducation (31) , or assertive community treatment (32) .

Limitations of this study included the small study group size, the lack of cognitive and symptom ratings at follow-up, and the suboptimal fidelity of the supported employment programs. Strengths of the study were that it was conducted at two typical treatment settings serving predominantly disadvantaged, minority patients with low educational levels and that it focused on individuals who had experienced job failures. When combined with supported employment, the Thinking Skills for Work Program demonstrated high rates of retention and better work outcomes than supported employment alone. Further research is warranted to replicate the effects of this program, including research with larger group sizes and higher fidelity supported employment programs.