Implementing Computer-Based Psychotherapy Among Veterans in Outpatient Treatment for Substance Use Disorders

Internet-based technology is increasingly used to deliver a broad spectrum of health interventions. One of the fastest growing segments of this technology involves computer-based psychotherapy interventions (CBPIs), consisting primarily of self-guided, Internet-based programs that use cognitive-behavioral therapy (CBT) techniques to improve knowledge, manage symptoms, or induce behavior change (1). CBPIs present standardized yet individualized material in rich formats, including audio, video, and text.

The potential health service benefits of CBPIs are considerable. They can reduce transportation barriers, reduce barriers associated with the stigma of seeking mental health care, accommodate deficiencies in the number of trained therapists, and promote self-efficacy (2–4). CBPIs may also increase clinic productivity and reduce costs (5–7).

Such interventions have the potential to improve care for common mental disorders such as depression, anxiety, and especially insomnia. Insomnia is one of the most common complaints of veterans from recent conflicts and is strongly associated with trauma, problematic alcohol use, decreased quality of life, and use of significantly more health services (8–13). Sedative-hypnotics, many of which are associated with abuse potential and a host of side effects, are the current mainstay of treatment (14,15).

Face-to-face CBT has been shown in multiple studies, including meta-analyses, to be superior to both placebo and active pharmacologic agents for the treatment of insomnia, prompting the American Academy of Sleep Medicine to recommend its use as a first-line treatment (16,17). This therapy is being actively disseminated throughout the Veterans Health Administration (VHA) but requires extensive provider training and in-person client attendance (18). These barriers can be overcome through the use of self-help CBPI versions as the first-line treatment, with referral to face-to-face therapy if the CBPI fails. Such CBPIs for insomnia have shown efficacy (19).

As the development and use of evidence-based self-help CBPIs grow, it will be important to understand how current outpatient treatment programs can benefit from implementing this mode of service delivery. To date, work in this area has focused on efficacy testing through controlled trials in selected populations. Strategies for implementing self-help CBPIs in outpatient treatment settings serving populations with extensive comorbidity and that include provider referral and support have not been developed or evaluated. The objective of this study was to assess the feasibility of a six-session self-help CBPI for insomnia as well as the strategy used to implement it among users of VHA outpatient substance use treatment services, including an exploration of the level of assistance required to support participants in the program.

Methods

Study Design and Intervention

This study explored the feasibility of implementing a self-help CBPI for insomnia in a VHA substance use disorder outpatient care site using an uncontrolled pre-post intervention format evaluating program engagement, completion, acceptability, and clinical response. The CBPI, called Restore, is a six-session, Internet-based, self-administered therapy consisting of interactive instruction on sleep hygiene, stimulus control, relaxation, sleep restriction, and cognitive distortions. Therapeutic tools include videos, questions, a sleep log, and homework. Providers can follow treatment progress electronically. The program has shown efficacy in a controlled trial involving 118 persons with chronic insomnia (20).

The Replicating Effective Programs (REP) framework for implementation, developed by the Centers for Disease Control and Prevention, provides extensive guidance in all phases of implementation, from the identification of quality gaps in mental health services to the maintenance of newly implemented interventions. This study was patterned after the REP concept of piloting the intervention before full implementation efforts. The four core components of the implementation strategy used in this study were drawn from the many concepts described in the preimplementation phase of REP (training, facilitation, technical assistance, and the identification of a “program champion”).

In this study, the core components were as follows. First, all clinic providers attended an education session about Restore and its implementation in which they were introduced to recommended treatments for insomnia and the efficacy of CBT for treating it, as well as to the format, content, and efficacy of the program. Second, engagement and completion were facilitated through the use of an optional Internet-connected computer set up in a private part of the clinic. Third, prior to using the program, enrollees participated in a 45-minute face-to-face education session and discussion covering insomnia, its behavioral components, and treatment, as well as the content, format, and technical aspects of Restore. Fourth, support was provided through facilitation and technical assistance from a “program champion,” who contacted patients once a week, primarily over the phone, and provided face-to-face help sessions when requested by the participant. Goals of support were to provide encouragement for engagement and completion and to answer clinical and technical questions regarding either insomnia treatment or the program. For this pilot study, the amount, format, and content of these contact sessions were exploratory and performed by the principal investigator (EH), who was not a provider at the clinic. The goal was to use data from these interactions to modify the implementation strategy for future implementation efforts to where the support role would be accomplished by individuals within the clinic. The institutional review boards of the Veterans Affairs Connecticut Health Care System and the Yale University School of Medicine approved this study.

Sample

The target population was veterans with chronic insomnia who were receiving outpatient substance abuse treatment. Entry criteria were consistent with general diagnostic criteria for insomnia disorder: a delay in sleep onset, delay in return to sleep after awakening, or early morning awakening longer than 30 minutes for three or more nights a week for three months or longer with at least one symptom of daytime impairment (including fatigue and difficulty concentrating) (21). Participants were English speaking and had no evidence of diagnosed sleep apnea or restless leg syndrome by self-report or medical record review; no past-month psychiatric emergency department or inpatient admissions, nor active suicidal or homicidal ideation; no concurrent treatment with buprenorphine, methadone, or CBT for insomnia; and no engagement in evening or night shift work. All medications were allowed, including sedative-hypnotics.

Measures

Feasibility was operationalized as participant engagement in or completion of the program; acceptability, as assessed in a series of follow-up survey questions; and positive clinical response, measured by improvement on the Insomnia Severity Index (ISI), an assessment of sleep dissatisfaction and daytime impairment (22,23). Secondary outcomes included the number of sedative-hypnotic medication doses taken weekly and the 12-Item Short-Form (SF-12) health-related quality-of-life measure (24). Additional measures included sociodemographic characteristics, Internet use, psychiatric diagnoses, profile of current substance use, and the number and content of CBPI support contacts. Participants completed measures before program initiation, after completion or after a decision either to discontinue or to not engage, and three months after the follow-up assessment for those who completed the program.

Analysis

Using the provider supervisory function of the program, participants were categorized as “completers” if they completed all six active modules, “partial completers” if they completed fewer than six, and “nonengagers” if they did not complete any modules. Bivariate analysis of differences between the engagement and completion groups and baseline variables were tested with chi square tests and analysis of variance.

Acceptability questions used a 7-point Likert scale, and responses were analyzed with chi square tests in three categories according to engagement or completion status. For general CBPI acceptability questions, the entire sample was used. For Restore-specific questions, only persons who fully or partially completed the program were used. For questions regarding early dropout, responses of only those who partially completed or did not engage in the program were analyzed.

Pre- and postintervention clinical outcomes, including ISI and SF-12 scores (24) and the total weekly number of sedative-hypnotic doses, were compared by using paired t tests and chi square. The follow-up ISI score was compared in linear regression models. This analysis was by completion or engagement status, with adjustment for baseline ISI score, days between baseline and follow-up measurement (in order to adjust for spontaneous improvement), as well as the number of sedative-hypnotic doses used in the week before assessment at baseline and follow-up. Effect sizes for the ISI were calculated with dichotomous dummy codes for intervention completers and partial completers compared with those who did not engage.

Results

Engagement, Completion, and Sample Characteristics

One hundred individuals were screened, and 51 were enrolled. [Reasons for exclusion are described in a CONSORT diagram provided as an online supplement to this article.] Of enrollees, 43% completed the program, 26% completed part of the program, and 31% did not engage (Table 1). The population enrolled was largely male (88%) and older (mean age of 52.2±10.4 years) with 98% reporting a history of an alcohol or a drug use disorder or combination, whereas 49% reported current problematic alcohol or drug use and 46 (90%) reported a psychiatric diagnosis. The mean insomnia level for the sample represented moderate to severe insomnia (ISI score=20 out of 28, with higher scores indicating more severe insomnia), and 39 (77%) veterans reported using at least one sedative-hypnotic regularly. There were no statistically significant differences between groups of completers, partial completers, and nonengagers for baseline characteristics (Table 1).

Acceptability

When asked about the acceptability of CBPI, 67% of enrolled veterans agreed that they would participate in another CBPI, whereas only 11% disagreed (Table 2). When asked if they would have preferred meeting in person, only 16 participants (36%) agreed (Table 2). For Restore-specific questions, 68% agreed that the program was helpful in reducing insomnia, with a higher, but not statistically significant, proportion of completers (74%) agreeing compared with partial completers (26%) (Table 3). On questions regarding nonengagement and noncompletion, agreement with any of the presented reasons was not strong (>50% agreeing or strongly agreeing), and differences between partial completers and nonengagers were not statistically significant (Table 4). Specifically, for the statement, “Access to a computer . . . was too difficult,” only 35% of the total sample agreed, 63% of whom were nonengagers.

Clinical Outcomes

In the total sample, total ISI scores decreased by a mean of 6.3 points (32% decrease) from baseline to follow-up (t=6.82, df=44, p<.001). Among completers, ISI scores decreased from 20.1 to 10.2 (49% decrease; t=8.14, df=20, p<.001) and further, to 9.9 (51%) at three-month follow-up (t=7.40, df=20, p<.001). Partial completers also experienced a statistically significant decrease in the ISI score of 27%, from 19.9 to 14.6 (t=2.64, df=12, p=.022). Those who did not engage showed only a nonsignificant decrease of 3%, from 19.9 to 14.6. In adjusted linear regression analysis, statistically significant differences were observed in mean follow-up ISI scores between engagement and completion groups (F=3.87, df=4 and 40, p≤.004). Differences corresponded to an effect size of 1.2 for completers and .7 for partial completers, compared with nonengagers. No clinically meaningful or statistically significant differences were observed in either component (physical or mental health) of the SF-12.

There were no statistically significant changes in the number of sedative-hypnotic doses ingested per week for the entire sample or in any of the engagement and completion groups. Likewise, there were no statistically significant differences in the number of new sedative-hypnotic starts or stops between engagement and completion groups.

In analysis of follow-up clinical support contacts, completers and partial completers had 9.0±4.5 and 9.6±4.2 contacts, respectively, whereas those who did not engage had 7.8±4.6 contacts. Among these contacts, 36 (8%) dealt with computer- and Internet-related questions such as username and password loss, and only six (1%) were related to program content or clinical questions. A majority (N=410, 91%) were either phone messages or short conversations of a motivational nature. In the sample, 20 individuals (39%) accessed the program from the clinic, and 31 (61%) used a computer in a different location. Neither engagement and completion status nor the number of clinical support contacts varied significantly by location of computer use.

Discussion

This study evaluated the feasibility of implementing a self-help CBPI for insomnia among individuals in outpatient VHA substance abuse treatment using an implementation strategy that offers provider education, optional on-site Internet availability, a patient education session, and facilitation through weekly telephone support. Results suggest that implementation of the Restore program using this strategy is feasible, which is supported by evidence in three areas: reasonable engagement and completion rates in comparison with those found in other studies of CBPIs, positive self-report of satisfaction, and a dose-response type of association between engagement in the program and insomnia improvement, independent of sedative-hypnotic use. Moreover, an exploration of the amount and content of support provided indicates that in future CBPI implementation efforts, support can be easily provided by clinicians or potentially support personnel within a practice setting.

Although there have been many efforts within the VHA to develop and test CBPIs for a number of common clinical problems, efforts to implement them in current VHA care models have not been documented. Similar efforts outside the VHA have focused on implementing CBPIs for depression or anxiety by using similar care management strategies in the United Kingdom’s National Health Service, with evidence that they have improved access and outcomes and decreased costs (6,7). Self-help CBPIs, it should be noted, are first-line treatments that can be followed by face-to-face intervention if they do not achieve the desired goals.

The acceptability of CBPIs, as measured by program engagement and completion and participant feedback, is a useful indicator of the platform’s feasibility (25). In this study, 69% of veterans engaged in the intervention and 43% completed it, compared with 68% who completed Restore in a university-based controlled trial and rates of between 57% and 95% in trials of other CBPIs for insomnia (20,26–29). These comparisons are with completion rates in controlled trials, which typically recruited homogeneous uncomplicated samples and used rigorous research protocols with staff to facilitate engagement. In contrast, this sample was drawn from a population seeking treatment for drug and alcohol use in a public mental health setting, and participants had high levels of psychiatric comorbidity and concomitant sedative-hypnotic use. Reduced engagement and completion in such treatment settings may be the rule rather than an exception (30). For example, trials of clinician-assisted computerized interventions for depression have reported dropout rates of between 30% and 50% (31). This relatively low engagement and completion rate may also be associated with the relatively high proportion of individuals using the computer station within the clinic (40%), requiring them to come to the clinic to participate. Similarly, a recent study of the implementation within the VHA of face-to-face CBT for insomnia reported that only 52% of veterans who engaged completed six sessions (32).

Participant self-report data from this study suggest that both CBPIs in general and Restore specifically were acceptable to between one-half to two-thirds of veterans in outpatient substance abuse treatment. These findings mirror other literature on research that used patient self-report data, which showed general acceptability of Internet-based self-care resources for veterans (33–35). Moreover, our finding that no baseline factors predicted engagement and completion is similar to that of other studies showing few such predictors (36,37).

The program was associated with clinically and statistically significant changes in insomnia severity among individuals who engaged in or completed the program. Although most efficacy studies have reported data from sleep logs, including sleep onset latency and sleep efficiency, such data were not available in our study for those who did not engage or who dropped out. Prior studies of CBPIs for insomnia have shown small to medium effect sizes for patient report of insomnia severity, disrupted sleep, and sleep efficiency (20,26–29). This study showed large effect sizes on the ISI among both completers and partial completers compared with those who did not engage. The strong effect for partial completers may indicate that insomnia levels of those individuals either improved on their own or after only minimal engagement with modules addressing the subjects of sleep hygiene or behavioral anxiolytic techniques, perhaps leading individuals to drop out after what they regarded as sufficient clinical improvement. An alternative explanation would be that a placebo effect of trying the program was responsible for improved sleep. Studies of sedative-hypnotic medication treatment for insomnia have shown strong placebo effects (38).

A secondary objective was to evaluate the program’s association with a decrease in sedative-hypnotic medication use. Results showed only small changes in such use. Three prior studies of CBPIs for insomnia have found similarly small changes in sedative-hypnotic use (26–28). These findings underscore the difficulty that patients with insomnia face in stopping such dependence-inducing medications. Additional explanations for this limited impact include the brief duration of the study and the fact that the education component of the strategy did not include an explicit discussion of decreasing sedative-hypnotic use during the program.

Additional limitations of this study deserve comment. This was an open pilot trial to evaluate initial feasibility of the program and implementation strategy. The untreated comparison group was self-selected and not identified by random assignment; thus causal inferences are not conclusive. However, the evaluation of baseline characteristics and engagement and completion status did not reveal any factors that may have accounted for these differences. Moreover, the clinical site for this study, like most outpatient mental health clinics, does not screen for the presence of insomnia among patients, and the penetration of this program’s use among potential insomnia sufferers is unknown. However, a small survey for a different project at the same clinic suggested that over 50% of patients suffer from clinically significant sleep problems (39), signifying that only 10% of potentially eligible patients were referred. Future studies investigating these findings are planned. Similarly, the sample size of 51 participants was relatively small, especially given the number of individuals who did not engage. Also, three-month follow-up measures were accomplished only by those who competed the program, which limited statistical power, making results susceptible to type II statistical error. Finally, although we collected information from participants on acceptability of treatment, the validity of these assessments has not been evaluated, and we do not have systematic data on provider response to the program. However, we feel that the acceptability questions have strong face validity given that this was a pilot project, and semistructured interviews of providers were conducted for qualitative analysis, which will be reported in future articles.

Conclusions

Whereas multiple past studies have documented the development and testing of CBPI programs for mental health problems, a strategy for their implementation in large public health care systems treating numerous patients facing multiple chronic conditions had not been devised or piloted, which is the major accomplishment of this study (40). Results suggest that the implementation of a CBPI for insomnia using a strategy consisting of provider and patient education sessions, clinic-based Internet access, and telephone-based support is feasible in VHA outpatient mental health and substance abuse treatment settings. Future studies of CBPI effectiveness, implementation, and use should focus on the integration of such programs into current clinical care, strategies for increasing engagement and completion, and the recruitment and training of individuals within the practice setting to educate and support participants.