Service Intensity as a Predictor of Competitive Employment in an Individual Placement and Support Model

People with severe mental illness consistently identify employment as an important goal (1). The individual placement and support (IPS) (2) model of supported employment helps people with severe mental illness obtain and maintain competitive employment (3–5). Variability in service intensity—the total time or number of service contacts (6)—was found in several studies of supported employment (7–15). Intensity of services may relate directly to their impact on outcomes.

MacDonald-Wilson and colleagues (7) examined supported employment hours by using a state billing database and found that people with severe mental illness averaged about eight hours per month. Services were most intense at first, then rapidly decreased, and then were followed by periodic spikes. Bond and Dietzen (8) examined service logs completed by supported employment specialists (that is, staff providing services). Participants in the condition most like supported employment received about seven hours of employment services per month over a 12-month period. Mowbray and colleagues (9,10) found supported employment specialists attached to case management teams gave an average of less than two hours of employment services per month to each service recipient. In contrast to the previously mentioned studies (7,8), which included participants receiving regular services, Mowbray and colleagues' (10) analyses included any client receiving at least one hour of service spanning more than a month. Rogers and colleagues (11) found an average of about seven contacts per month provided to supported employment clients; the three-year total number of service contacts per participant varied widely (11 to 1,268).

Recently, Bond and Kukla (12) found that participants who received IPS services and were followed for two years after obtaining a competitive job received an average of about two follow-along contacts per month. The greatest intensity of services was soon after participants obtained employment and decreased sharply after the first few months on the job. After this period, follow-along support leveled off to one contact per month.

Some supported employment studies have shown a relationship between service intensity and vocational outcomes. Bond and Dietzen (8) found that service intensity before job placement was associated with more positive employment outcomes. Jones and colleagues (13) found that clients who obtained employment received more total services than their nonworking counterparts. Cook and colleagues (14) examined various vocational programs and found a relationship between the intensity of vocational services and employment. In a reanalysis of the same data set, Leff and colleagues (15) found a relationship between job development and job attainment, but they found no association between service intensity and weeks worked once a client obtained a job. In contrast, Bond and Kukla (12) found that service intensity received after attainment of a job had a positive relationship with weeks worked.

Little research has focused on predictors of service intensity or on variables that moderate the relationship between service intensity and vocational outcomes. McGurk and colleagues (16) found that most individual characteristics were not predictive of service intensity, except for positive and negative symptoms and cognitive factors. Leff and colleagues (15) found that services were associated more strongly with success for people with little previous work experience. Data from the Employment Intervention Development Project (14,15) demonstrated that vocational services were more effective when integrated with clinical services.

The study presented here examined four aspects of service intensity in a supported employment program. First, what is the average level of service intensity? Second, what is the association between service intensity and weeks worked? Third, what individual demographic and clinical variables are associated with service intensity? Finally, what individual characteristics moderate the relationship between service intensity and weeks worked? The study used data from the IPS arm of a longitudinal randomized controlled trial comparing IPS to a diversified placement approach (17).

Methods

Data used in the study presented here originated from the Thresholds' billing database and interviews conducted with participants in the parent study (17), a two-year prospective study in which clients in the IPS condition received IPS services for that duration. The study was conducted at two sites of Thresholds (north and south sites), a comprehensive psychosocial rehabilitation organization in Chicago. It was approved by the Indiana University-Purdue University Indianapolis Institutional Review Board.

Sampling

Participants of the parent study had to be admitted to one of two day programs operated by Thresholds and be 18 or older. Participants met Illinois criteria for severe mental illness: a DSM-IV diagnosis of schizophrenia, other psychotic disorders, bipolar disorder, or obsessive-compulsive disorder and significant treatment history or significant functional impairments. Furthermore, participants were enrolled in Thresholds services for at least 30 days, had a goal of paid employment, and had not been competitively employment during the prior three months. Participants did not have a physical illness that would prevent their participation, agreed to exclusion from the nonassigned vocational program for two years, and attended two sessions of a research information group during the first 60 days after admission to Thresholds, after which they gave written consent to participate in the study.

A total of 194 persons were entered into the parent study (96 in the IPS arm and 98 in the diversified placement approach arm). The study presented here included participants who were randomly assigned to the IPS arm of the parent study, excluding five participants who discontinued IPS services within the first quarter, for a final sample of 91. The excluded group did not differ from the included sample on any individual characteristics (for example, demographic characteristics or clinical factors), did not receive any IPS contacts during the study period, and did not obtain employment before dropping out, but they did receive fewer mental health service hours during the two-year study period than the sample included in the study presented here (mean±SD=22.6±15.9 versus mean±SD=375.9±348.1; t=9.50, df=93, p<.001).

Measures

Service variables.

The sample was enrolled between 1999 and 2002, with service data collection continuing until 2004. Service data were obtained from Thresholds's PsychServe system, an in-house billing database. IPS contacts were defined as contacts that were billed by an IPS employment specialist and coded as “individual vocational support.” Billable hours for all other classes of services provided by Thresholds (mental health services), excluding residential services, were collected for each of the eight quarters.

Vocational outcome.

Information on weeks worked in competitive employment was collected from participants in all eight quarters.

Background variables.

Participants in the parent study were administered a questionnaire at the baseline interview that assessed, among other factors, age, race, gender, marital status, educational level, Social Security Disability Insurance or Supplemental Security Income benefits, and Thresholds site. Prior work experience was measured through a self-report of the number of weeks worked in the past five years and by the number of years since the participant was last competitively employed.

Illness variables.

The Positive and Negative Syndrome Scale (PANSS) (18) was administered at baseline. The PANSS is a semistructured clinical interview that has shown good reliability and validity in the assessment of a range of symptoms. The five-factor model (positive, negative, cognitive, hostility, and emotional) differentiated types of symptoms (19). Number of lifetime hospitalizations and diagnosis, assessed using the Structured Clinical Interview for the DSM-IV (20), were also collected at baseline.

Data analysis

Data were entered into a software database, were double-checked for accuracy, and were analyzed using SPSS, version 11.0, for Windows and Hierarchical Linear Modeling, version 5.00, for Windows (21,22). We recorded zeros for IPS contacts for quarters after which participants dropped out of IPS services.

All hypotheses were examined in two manners. First, hypotheses were tested in a simple, aggregated fashion by totaling variables across the entire study and conducting hierarchical linear regressions. Second, data were examined longitudinally using hierarchical linear modeling (HLM) (23). HLM controls for the correlated nature of multiple observations made on the same individual and thus is appropriate for longitudinal data. In order to test for a potentially delayed effect of IPS services, a longitudinal, time-lag analysis was conducted. This analysis was identical to the other HLM analyses, but this analysis paired IPS contacts for a given quarter with the weeks worked in the following quarter. Zeros were imputed for IPS service contacts in the quarter before baseline.

Results

Sample characteristics

In the parent study (17), 72 of 96 (75%) IPS participants obtained competitive employment; 45 of 96 (47%) worked at least 20 hours per week. Characteristics of the 91 participants in the study presented here can be found in Table 1. Within our study sample (N=91), 44 (48%) participants continued IPS services throughout the two-year study period (program continuers) but may not have had an IPS contact in each quarter. Dropouts were roughly evenly distributed across the study period. Program continuers did not differ from dropouts on demographic or clinical variables. We repeated all of the following analyses with participants enrolled in IPS throughout the study period program; results did not differ substantially, so results from the full study sample are reported below.

Sample characteristics, service intensity, and weeks worked

Table 2 shows service data, including IPS contacts for both the full sample and for those enrolled in services, for the entire follow-up period. Averages for quarters reflect the mean number of IPS contacts for participants still enrolled in IPS services; therefore averages exclude dropouts and participants with no data recorded. Participants averaged about six contacts per quarter in quarters in which they received at least one contact. In general, the number of IPS contacts declined across time (F=17.6, df=7 and 322, p<.001). IPS service receipt fell steadily from 86% of the sample in the first quarter to 42% in the final quarter. Over the two-year period, clients averaged 56.9 hours of mental health services per quarter. The hours of mental health services also declined with time (F=5.23, df=7 and 357, p<.001). Mental health service receipt fell from 100% in the first quarter to 59% in the final quarter. Intensity of IPS services was modestly positively correlated with mental health services (N=91, r=.25, p=.02).

Client background characteristics were generally unrelated to IPS service intensity (Table 1). (Continuous variables are not reported in Table 1, but those tested can be seen in Table 3.) The only exception was that fewer psychiatric hospitalizations in the year before the study were associated with more IPS contacts (N=91, r=–.20, p=.05). Length of hospital stay during the follow-up period was not associated with the number of IPS contacts.

Service intensity and weeks worked

Total IPS contacts for the full sample were positively associated with weeks worked (N=88, r=.30, p<.01). This correlation was not maintained when it was calculated for only the participants who obtained employment. A longitudinal analysis using HLM did not support an association between the number of IPS contacts in a given quarter and the weeks worked in the same quarter. However, the number of IPS contacts in a given quarter was positively associated with the number of weeks worked in the following quarter (ν₁₀=.11, SE=.04, df=89, p<.01). This effect remained (ν₂₀=.15, SE=.04, df=87, p<.05) when entering two covariates: weeks worked in the previous five years and negative symptoms. We also evaluated the relationship between service intensity required per week of work and found similar results; therefore, the results are not reported.

Moderation effects

The interactions between individual characteristics and IPS contacts on weeks worked were also explored (Tables 3,4). One interaction of interest was that of mental health service hours. When IPS contacts and mental health hours were entered into the model simultaneously, IPS contacts remained significantly associated with weeks worked, whereas mental health hours did not. There was no significant interaction effect. When entered into the same model, IPS contacts were associated with weeks worked (β=.40). The interaction between cognitive symptoms and vocational contacts was significant; the relationship between IPS contacts and weeks worked decreased as cognitive symptoms increased (Table 3).

Discussion

This study supported the relationship between service intensity and vocational outcomes. An important question for service providers is, “What intensity is related to continued employment?” In this sample, IPS clients actively engaged in services received slightly more than two IPS contacts per month. This rate is lower than might be expected of an “intensive” service model and is substantially lower than that reported in earlier studies, including a major multisite study (14). However, the intensity is comparable to that found by Bybee and colleagues (10) and Bond and Kukla (12). The rate in the study presented here is restricted to services documented in a Thresholds billing database and excludes vocational service contacts outside of the IPS model (for example, activities without the client present). Services include assistance both on and off the job, but they must include face-to-face contact with the consumer.

Service intensity and vocational outcomes

The study presented here further supported a connection between service intensity and vocational outcomes (8,13,14,24). This study suggests a possible “time-lag effect” in which the effect of employment services may not be realized until several months later. These time-lag findings suggest a temporal precedence of IPS service and are consistent with IPS services requiring intensive front-end job development and initial job placement, followed by a tapering off of employment specialist contact. The identification of factors related to vocational success within IPS is of particular importance because participants receiving IPS services still worked only a portion of the total study period.

Predicting service intensity

In general, client background characteristics did not predict service intensity. However, one relationship indicated that participants who were higher functioning (that is, fewer previous hospitalizations) received more services. In contrast, McGurk and colleagues (16,25) found that having more severe symptoms was associated with more intense services per hour of work. It may be that recent hospitalizations affect services differently than symptoms; moreover, McGurk examined services to currently employed people whereas the study presented here examined both employed and unemployed participants.

Moderation of the effect of service intensity on weeks worked

Two types of symptoms moderated the relationship between service intensity and weeks worked. Participants with more severe hostility or cognitive symptoms did not benefit from increased service intensity to the extent that people with less severe symptoms did. One possibility is that people with more severe symptoms may not be able to absorb feedback at the rate that other people can. Conversely, the results of the time-lag analysis suggest the importance of job placement services for participants. However, participants with more severe cognitive symptoms may benefit more from follow-along services, which may be deemphasized in routine practice. Another confounding factor that was not explored is overall model fidelity—these results were in the context of a high-fidelity supported employment program and may not generalize to low-fidelity sites.

It is important to note that mental health services were not directly associated with weeks worked and did not moderate the effect of IPS services on weeks worked. This result is particularly notable for two reasons. First, only one other study (14) has explored the additive effects of mental health and vocational services on weeks worked. Second, mental health services were approximately nine times as intense as vocational services (assuming an average of one hour per vocational contact). Although these results argue against the additive effects of mental health services in promoting vocational outcomes, one should note the variability in types of mental health service hours and that the IPS model prescribes the integration of IPS and mental health services (26). Moreover, it may be that adequate mental health services control the effects of mental health symptoms and therefore mask the effects of both mental health services and symptoms on employment.

Limitations

A substantial proportion of participants had no record of IPS contacts in some quarters; these observations were treated as zero contacts, but some may have been instances of underreporting. Missing data are attributable to two main sources: either employment specialists provided services but failed to document services in the database or participants did not receive services. Much of the missing data is likely a result of underreporting by employment specialists, who were initially unfamiliar with the database and billing codes. Also, we know anecdotally that clients sometime pause in the vocational process, resulting in service gaps. Regardless of the reasons, the study's missing data present a substantial threat to internal validity. Finally, billing databases do not account for nonbillable services—that is, times during which the service recipient was not present. Therefore, services such as calling potential employers and conferring with service providers or job supervisors were systematically excluded. Because of the missing data and nonbillable services, the average service intensity documented in this study should be considered an underestimation of actual services in an IPS program.

The large proportion of program dropouts is another limitation. These dropouts likely differed in unknown ways from program continuers. This systematic difference would again bias the sample; most likely this bias would eliminate the less motivated or work-ready participants, thus leaving a more successful sample. However, most participants received services for at least one year, and attrition was lower for IPS services than for the comparison program (27). Furthermore, it should be noted that these data were collected several years ago and current employment data may differ.

The study presented here tested several study hypotheses, as well as numerous exploratory analyses, there by increasing the threat of alpha inflation. Results from exploratory analyses should be viewed with particular skepticism given the lack of a priori theoretical support. Finally, cognitive functioning was measured by a symptom measure rather than by more thorough neurocognitive measures.

Future directions

More effective methods for collecting service data are essential for the further study of psychiatric rehabilitation services. One promising approach is the use of service logs paired with regular reminders to employment specialists. Also, new technologies would make the recording of service data less time-intensive for employment specialists and more timely and accurate (28). Future research should also focus on the interplay between the service recipient and the various factors involved in his or her employment search, such as the relationship between the employment specialist and the client.

Conclusions

Intensity of IPS services was modestly associated with weeks worked, whereas intensity of mental health treatment (although extensive) was unrelated to weeks worked. Cognitive symptoms attenuated the relationship between service intensity and weeks worked. This study provides a baseline for administrators, service providers, and program evaluators regarding optimal levels of service intensity. Moreover, it emphasizes the importance of service intensity as a critical element in effective IPS services.

Figures and Tables