Long-Term Effectiveness of Supported Employment: 5-Year Follow-Up of a Randomized Controlled Trial

Over the past 20 years, the superiority of supported employment over alternative approaches to vocational rehabilitation for persons with severe mental illness has been shown in multiple randomized controlled trials, both in the United States (1–5) and around the world (5–7). Among supported employment approaches, the individual placement and support model (8) has been the most extensively studied because of the standardization of the approach and the development of fidelity scales (9, 10). However, the long-term effectiveness of supported employment is less established.

With the exception of one study from Hong Kong (11), which reported that individual placement and support was more effective than traditional vocational rehabilitation over a 3-year period, all other controlled trials of the model have shown better work outcomes than control programs over follow-up periods ranging from 6 to 24 months (5). There is some evidence from two longer-term naturalistic follow-up studies in the United States that supported employment may continue to have a beneficial impact on vocational functioning after the initial years of the program. A 10-year follow-up study (12) found that 75% of participants who had received supported employment worked beyond the initial 18-month study period, and approximately one-third had worked for 5 years or more. An 8- to 12-year follow-up study (13) reported that 47% of the participants were working at the time of the interview and that 71% indicated that they had worked for more than half of the follow-up time, with most working less than 20 hours per week. Among the participants who were working at the follow-up, half were still receiving support from the supported employment program. These two studies were limited by relatively small sample sizes (N=36 and N=38, respectively) and a lack of control conditions.

Thus, there is strong evidence that supported employment is effective in improving the competitive work outcomes of people with a severe mental illness over 1 to 2 years, but its longer-term impact is less clear, and no controlled studies have evaluated outcomes beyond 3 years. We previously reported (7) the 2-year results of a randomized controlled trial in Switzerland, which found that a supported employment program for people with severe mental illness led to higher competitive employment rates (58% compared with 26%) and more weeks of competitive work (25 compared with 10) than traditional vocational rehabilitation. At the 2-year follow-up, 45% of the supported employment group were competitively employed, compared with 17% of the traditional rehabilitation group. Here we describe the 5-year outcomes of this study.

Method

The study was conducted at the University Hospital of Psychiatry in Bern. The hospital provides a wide range of both inpatient and outpatient services to people with severe mental illness. The study protocol and the consent forms were approved by the Canton of Bern Ethics Committee.

Participants

All study participants had a severe mental illness, defined as an ICD-10 psychiatric diagnosis combined with persistent impairment in the areas of role functioning, social functioning, or independent living/self-care skills. After obtaining written informed consent, we confirmed eligibility at a 2-week assessment.

Inclusion criteria for the study were age between 18 and 64 years; referral to vocational rehabilitation by the Swiss Invalidity Insurance State Office; desire for competitive work; and not currently competitively employed. Exclusion criteria were intellectual disability (IQ <70); a primary substance use disorder over the past year; a physical or organic disability or disorder that could seriously impede ability to work; unwillingness to receive regular outpatient treatment; performance <50% of normal work performance during the assessment phase; and attendance at the program <15 hours per week during the assessment phase. The last two exclusion criteria were required by the Swiss Invalidity Insurance State Office for participants to start a vocational rehabilitation program focused on obtaining competitive employment.

Of 143 persons who were screened for the study, 100 met the inclusion and exclusion criteria and were randomly assigned to the two vocational programs—46 to the supported employment program and 54 to traditional vocational rehabilitation. Figure 1 provides the CONSORT diagram showing the flow of participants through the study. The participants’ baseline demographic and clinical characteristics are summarized in Table 1.

Supported Employment Program

The Job Coach Project was modeled after the individual placement and support model of supported employment (7). The Job Coach Project is staffed by employment specialists (job coaches) who have a maximum caseload of 15 participants. Just after enrollment, each participant is assisted by an employment specialist in a rapid job search based the participant’s educational background, work preferences, and previous work experience. Once employed, participants receive on-the-job training and follow-along support to facilitate job maintenance. In the event of job loss, support and assistance are provided in securing a new job. Employment specialists had contact at least once every 2 weeks with participants and at least monthly with work supervisors, the treatment team, or other relevant persons. Some incentives were given to employers of participants in the Job Coach Project.

To evaluate the extent to which the Job Coach Project adhered to the model of supported employment, the Individual Placement and Support Fidelity Scale (9) was administered by the research team annually over the course of the study. Overall summary scores ranged from 66 to 68 (out of 75), indicating consistently good fidelity to the model. The only criterion on which the program consistently scored low was the “zero-exclusion criterion” (i.e., not excluding any client who wants to work), as the law on Swiss Invalidity Insurance requires that participants meet a minimal work performance standard on an assessment conducted prior to enrollment in vocational rehabilitation.

Traditional Vocational Rehabilitation

Traditional vocational rehabilitation programs in Switzerland are certified by the Federal Social Insurance Office and are based on a “train-place” approach. Each program participating in this study was deemed by the Federal Social Insurance Office to be the best locally available alternative for the prospective participant. All participants receive prevocational training in sheltered workshops for 6 to 12 months. Employable participants then complete 3 to 6 months of training in a competitive job. At the end of the traditional rehabilitation program, all support stops, regardless of whether the person has obtained a competitive job.

Measures

Participants were followed up for 5 years, with interviews at baseline and 1, 2, and 5 years later. Data were collected by interview. Vocational outcomes were defined according to the recommendations of Bond et al. (14), including rates of competitive employment; length of employment at least 50% (130 weeks) in a competitive job; total and annualized weeks competitively employed; job tenure in longest competitive job held; mean hours worked per year in competitive jobs; average yearly income; average hourly wage in competitive employment; and cumulative duration of competitive employment. The primary outcome was competitive employment, defined as holding a job paying at least minimum wage (set at CHF10.00 [about US$10] for this study) for at least 2 weeks on the open labor market (i.e., excluding jobs protected for people with a disability, such as transitional employment).

Primary psychiatric diagnosis based on ICD-10 criteria was drawn from participant charts and reviewed by the second author. Interview-based and self-report assessments were conducted at baseline, 2 years after enrollment, and 5 years after enrollment. Background information, such as demographic characteristics and work experience, was obtained during the baseline interview. At intake, symptoms were assessed by interview with the Positive and Negative Syndrome Scale (15), followed by evaluation of overall level of functioning on the Global Assessment of Functioning Scale. Self-report questionnaires included the Stress Management Questionnaire (16, 17) and the Wisconsin Quality of Life Index (18). At the 5-year follow-up, self-perceived quality of life was reported and the Recovery Process Inventory (19) was administered. The number of hospital admissions, time spent in psychiatric hospitals, and day treatment were tracked over the whole course of the study.

For the cost-benefit analysis over the 5-year period, we calculated social return on investment, a measure designed to assess the value of the social benefits created by an organization in relation to the relative cost of achieving those benefits. It is a technique for measuring outcomes and impact of programs in the tradition of economic evaluation. Unlike the return on investment back to the organization or company, social return on investment is not the profit-and-loss ratio commonly used in economics (20). Social benefits include the client’s earnings benefits that return to society by increasing purchasing power, reaching partial independence from social benefit programs, and creating tax revenue. Social return on investment was computed as the ratio of benefits (clients’ work earnings) to total investment for each client, expressed as a percentage (21) (benefits divided by investments, multiplied by 100; the minimum value would be 0%, indicating no return on investment).

The benefits were measured as the total of each client’s earnings (in both competitive and noncompetitive jobs) over the 5-year period, and the investments were measured as the sum of 1) the total vocational program costs per client over the 5-year period; 2) the total of the mental health service costs (in psychiatric hospitals and day treatment, outpatient mental health services, and medication) for each client over the 5-year period; and 3) the combined vocational program and mental health treatment costs for each client over the 5-year period (22). All data were collected monthly for each client. Social return on investment was calculated both with vocational program costs alone as the denominator and with total vocational and mental health treatment costs as the denominator.

Procedures

Randomization was conducted following a procedure meeting Cochrane criteria (23). Randomization was based on a random list generated by a computer algorithm, with assignments to treatment conditions placed in sequentially numbered sealed envelopes. These steps were performed by an administrative office unrelated to the research team.

Statistical Analysis

Intent-to-treat analyses were conducted including all allocated participants, regardless of exposure to the vocational programs. For the primary outcome of employment status, missing data were counted as unemployment. Analyses of participants with other missing follow-up data were conducted with the last observation carried forward. Groups were compared on measures regarding competitive or sheltered employment. Independent t tests were used to compare differences between group means.

For nonvocational outcomes, proportions of categorical variables were compared according to initial group assignment using chi-square tests. The two groups were compared on continuous variables using either t tests or, for nonnormally distributed variables (e.g., hospitalizations, day treatment, and the social return on investment analysis), the (nonparametric) Wilcoxon rank-sum test. All statistical tests were two-tailed with the significance threshold set at 0.05. Data were analyzed with JMP, version 10.0 (SAS Institute, Cary, N.C.).

Results

Follow-Up Rates and Attrition

There were no statistically significant differences in follow-up rates between the supported employment and traditional rehabilitation groups at either the 2-year or the 5-year follow-up, with high rates across both groups, as shown in Figure 1.

Vocational Outcomes

All competitively employed participants in both groups worked for at least 2 weeks. Table 2 summarizes the differences in the vocational outcomes between the two groups. The supported employment group had significantly better competitive work outcomes than the traditional rehabilitation group on all measures. Among participants who obtained competitive work, those in supported employment worked significantly more weeks and had longer job tenures than those in traditional rehabilitation. Among competitively employed participants, there were no significant differences between the groups in the average number of hours worked per year, wages earned, or yearly income.

Random-effects logistic regression was used to assess the overall differences between the supported employment and traditional rehabilitation groups in year-by-year competitive employment rates for the 5-year study period. The type III tests of fixed effects revealed a significant group effect (F=15.63, df=1, 98, p<0.001) and a significant time effect (F=6.73, df=5, 490, p<0.001), but no significant group-by-time interaction. As shown in Figure 2, these results indicated that participants in supported employment had higher monthly competitive employment rates over the study period than those in traditional rehabilitation, with the rate of employment in both groups increasing over time.

Job stability.

As shown in Table 3, the percentage of participants in the supported employment group who were competitively employed during almost the entire follow-up period and who were still employed at the end of follow-up (37%) was more than four times the percentage in the traditional rehabilitation group (9%), while only 35% of the supported employment group were never competitively employed, compared with 67% of the traditional rehabilitation group. The proportions of those who got a competitive job later during the follow-up period and those who lost their job did not differ significantly between groups. Participants in supported employment who obtained competitive work were significantly more likely to sustain their employment to the end of the 5-year follow-up than those in the traditional rehabilitation group (p<0.005).

Nonvocational Outcomes

Participants in supported employment had significantly fewer psychiatric hospital admissions than those in traditional rehabilitation (mean=0.4 [SD=0.9], compared with mean=1.1 [SD=2.1]; p=0.026), and fewer of them had hospital admissions at all (N=8 [21%], compared with N=21 [46.7%]; p=0.015). They also spent fewer days in the hospital and day hospital (mean=38.6 days [SD=83.4], compared with mean=96.8 [SD=178.9]; p=0.027). There were no differences in psychiatric outpatient service utilization or participants’ recovery attitudes as measured by the Recovery Process Inventory. Self-rated quality of life at the 5-year follow-up was somewhat better for the supported employment group (mean=6.9, SD=1.3) than for the traditional rehabilitation group (mean=6.2, SD=2.0), but the difference did not reach statistical significance (p=0.06).

The results of the cost-benefit analysis are presented in Table 4. Neither the vocational program nor the mental health costs were significantly different between the two groups, and the total costs (both program and mental health treatment costs) were practically the same for the two groups. However, total client earnings and the social return on investment were both significantly higher for the supported employment group.

Discussion

To our knowledge, this is the first randomized controlled trial of supported employment to report 5-year outcomes. Approximately twice as many participants in supported employment obtained competitive work over the 5-year period than in the traditional rehabilitation program. Participants in supported employment remained in their longest competitive job approximately three times longer than those in traditional vocational rehabilitation, and they worked more hours and earned more wages from competitive employment over the 5 years.

The average percentage of participants working competitively at the end of 2 years was relatively stable for both groups over the next 3 years—about 45% for supported employment and only 15%−17% for traditional vocational rehabilitation. The rate remained stable for the supported employment group despite the fact that the proportion of participants who received support from an employment specialist decreased from 67% at the 2-year follow-up to 35% at the 5-year follow-up (7). The findings suggest that the beneficial effects of supported employment on competitive work were sustained over the 5-year follow-up period and that sustained employment with vocational supports enabled many participants to eventually work independently. Interestingly, the rate of competitive work at 5 years for the supported employment group in this study is similar to the 47% rate in the naturalistic 8- to 12-year follow-up study of supported employment reported by Becker et al. (13).

Supported employment was also associated with significantly lower levels of inpatient psychiatric treatment over the 5-year study period, a difference that was not significant at the 2-year follow-up. Participants in supported employment were less likely to be admitted to a psychiatric hospital, had fewer hospital admissions, and spent fewer days in the hospital than participants in traditional vocational rehabilitation. While Burns et al. (6) found that participants in supported employment were significantly less likely to be hospitalized over 2 years than those receiving traditional vocational services, most other randomized controlled trials of supported employment have not reported such differences in hospital utilization (24). It is possible that the longer follow-up period in our study resulted in more statistical sensitivity to the cumulative benefits of competitive work on reducing vulnerability to relapse and hospitalization.

In addition, although the vocational program costs of supported employment were slightly higher than those of traditional vocational rehabilitation, the supported employment program produced higher earnings and thus a significantly greater social return on investment, measured in terms of both the ratio of work earnings to vocational program costs and the ratio of work earnings to total vocational program and mental health treatment costs. The superior social benefits of supported employment appear to be due to a combination of higher earnings and lower mental health service costs. The greater cost-effectiveness of supported employment in our study is in line with findings from previous studies (25, 26).

Study Limitations

As in other non-U.S. studies of supported employment, modifications in the individual placement and support model had to be made in order to fit the local context, in this case to meet the standards of the Swiss social insurance system. However, program fidelity was high and was consistent with the individual placement and support model, except that the latter has no exclusion criteria for participation other than desire for competitive work. Swiss social insurance requires that in order to receive vocational rehabilitation, participants must be able to work at a rate at least 50% of the normal rate on a standardized work task and must actively attend the program for at least 15 hours per week during the assessment phase. Thus, the study excluded more-impaired participants who were unable to meet the work criterion and less-motivated participants who did not attend the program with sufficient regularity. This may partly account for the relatively high numbers of hours and weeks of competitive work per year achieved by the supported employment group compared with most other randomized controlled trials of individual placement and support.

Future Directions

A growing body of international research shows that the individual placement and support model of supported employment is more effective in improving competitive work outcomes than alternative or traditional approaches to vocational rehabilitation. This study suggests that the beneficial effects of supported employment on competitive work are sustained beyond 2 years, for up to 5 years. Furthermore, there appear to be longer-term benefits of supported employment and competitive employment on reducing the utilization of high-cost inpatient services over 5 years that are not evident over shorter follow-up periods. These findings, if replicated, could provide additional motivation to the funders of mental health services to increase access to supported employment, as well as incentives to employers to offer jobs to individuals with psychiatric disabilities.

Supported employment appears to be a good means for improving inclusion of people with severe mental illness in the community. Although the importance of inclusion is emphasized in the United Nations Convention on the Rights of Persons with Disabilities (27), the broader effects of supported employment on social inclusion have yet to be investigated. Future research should evaluate whether supported employment can promote and sustain the inclusion of people with severe mental illness in their communities and assess the impact of such inclusion on mental health care utilization and costs.

In summary, the results of this study show that the superiority of supported employment over traditional vocational rehabilitation programs becomes even stronger over time and thus is consistent with the principle of time-unlimited support in the model. Furthermore, the results show that participants who received supported employment had fewer hospitalizations, which, combined with higher earnings from competitive employment, resulted in a higher social return on investment than traditional vocational rehabilitation. This study is a further contribution to the evidence of the effectiveness and economic efficiency of supported employment not only in the long term, but also outside the United States.