Improving the Efficiency of Psychotherapy for Depression: Computer-Assisted Versus Standard CBT

Cognitive-behavioral therapy (CBT) is one of the best established nonpharmacological treatments for major depressive disorder (1, 2). Meta-analyses of a large number of randomized controlled trials have shown that a 12–16 week course of individual CBT has efficacy comparable to antidepressant pharmacotherapy (3–5), with fewer relapses after treatment is stopped (6, 7). CBT also may significantly improve treatment outcomes when used in combination with pharmacotherapy (8, 9), especially for patients with more severe or treatment-resistant depressive disorders (10, 11). Despite compelling justification for widespread use of CBT, there are significant barriers to providing this form of therapy in everyday practice. One barrier to broader dissemination is an insufficient number of trained therapists, particularly in rural and public mental health settings (12). Other constraints are the cost of treatment and difficulties in scheduling and attending a large number of 50-minute therapy sessions across 3–4 months. These limitations help explain why antidepressant pharmacotherapy, not CBT, continues to be the most commonly used treatment for depressive disorders (13).

Computer-assisted CBT (CCBT) is a strategy that could make therapy more widely available and reduce cost, as well as provide learning experiences and data tracking features that might enhance standard CBT (14). Although the development of CCBT can be traced back more than 20 years (15, 16), activity in this area has increased over the past decade, particularly after the introduction of multimedia programs that can be accessed online (for example, beatingthebluesus.com, empower-interactive.com, and moodgym.anu.edu.au). Meta-analyses of a steadily growing body of research have documented the efficacy and efficiency of CCBT (17–21). However, evidence to date indicates that stand-alone programs that do not provide therapeutic support—representing about 40% of controlled studies—typically have much smaller effects than those that include at least a modest amount of clinician time (20). For example, in a recent primary care study (22) that provided participants with a small amount of technical support and no support from a clinician, neither of the two forms of CCBT that were studied was more effective than treatment as usual. Although it appears that some amount of clinical support is important, it is unclear what amount of therapist contact is optimal. In studies that do provide therapeutic support, the average amount of support typically has ranged from 1 to 5 hours (17, 20).

The method of CCBT used in the present study was designed to integrate computer-delivered training with therapist support to substantially reduce the amount of therapist time and effort required to deliver an effective course of therapy. It is a nine-module multimedia program (Good Days Ahead [GDA]; http://www.empower-interactive.com/solutions/good-days-ahead) that has shown promise in two open studies (23, 24) and a small randomized controlled trial (25). In the latter study, 45 medication-free outpatients with major depressive disorder were randomly assigned to 8 weeks of treatment with conventional CBT (nine 50-minute sessions of individual therapy), CCBT plus about 4 hours of therapist support, or a waiting list control condition. At the posttreatment assessment, both treatment groups showed clinically meaningful improvements compared with the waiting list group; the substantial reductions in depressive symptoms observed in the CBT and CCBT groups (Cohen’s d was 1.04 for CBT and 1.14 for CCBT) were maintained across a 6-month follow-up (25). Although that preliminary study was not designed to test noninferiority, the results suggested that this approach to CCBT might produce results that are comparable to those of individual CBT.

The present study was designed to conduct a more rigorous test of this method of therapist-supported CCBT. To ensure that the design was sensitive to any potential advantage for conventional therapy, patients treated with individual CBT received up to 20 sessions across 16 weeks of acute-phase therapy. The CCBT group, by contrast, received only about one-third the amount of therapist contact as those in the CBT group. To minimize potential allegiance effects (J.H.W. led the development of GDA at the University of Louisville site), investigators with high allegiance to conventional CBT (M.E.T., G.K.B.) led the University of Pennsylvania site (where Beck developed his model of CBT). Study therapists were selected for their expertise in conventional CBT and were trained in the CCBT method so that they could provide both interventions. Finally, the study’s sample size was determined to conduct a formal statistical test of noninferiority. The main hypothesis was that CCBT would be noninferior to a full course of CBT on the primary study outcome measure.

Method

The study was conducted at the Departments of Psychiatry of the medical schools of the University of Louisville and the University of Pennsylvania. All study methods were specified in advance of participant recruitment and were approved by the institutional review boards of the two sites. Prospective participants were recruited through advertisements and from the clinical services of the participating sites; online recruitment was not used in this study. After providing written informed consent, potentially eligible individuals were diagnosed with the Structured Clinical Interview for DSM-IV Axis I Disorders (SCID-I). Prospective participants were excluded if they had severe or poorly controlled concurrent medical disorders that would interfere with participation; met DSM-IV criteria for a psychotic or organic mental disorder, bipolar disorder, active alcohol or drug dependence, a primary anxiety or eating disorder (primary refers to the diagnosis associated with the most functional impairment), attention deficit hyperactivity disorder, a learning disorder, or borderline, antisocial, or paranoid personality disorder; scored less than 14 on the first 17 items of the Hamilton Depression Rating Scale (HAM-D) (26) at either the screening or the baseline (week 0) visit; could not complete questionnaires written in English; had not completed at least a 10th grade education or earned a General Equivalency Diploma; or scored below the 9th grade reading level on the Wide-Range Achievement Test (27).

Prospective participants were also excluded if they were considered to be an immediate active suicide risk (e.g., a score of 3 or 4 on HAM-D item 3), if they had previously failed to respond to a trial of at least 8 weeks of CBT, or if they were currently taking antidepressant medications and were either unable or unwilling to discontinue these medications. For the several consenting patients who wanted to stop taking a psychotropic medication, study physicians supervised a tapering and washout such that all participants were free of psychotropic medications for at least 1 week prior to randomization. Thus, patients who could not be quickly and safely withdrawn from psychotropic medications were not eligible for randomization.

Concurrent with the SCID-I, patients’ medical histories were reviewed by a study physician and, when clinically indicated, a physical examination and appropriate laboratory tests were obtained to ensure that patients were medically eligible.

Patients were allocated at each site to the CCBT or CBT arms using a web-based randomization procedure overseen by the University of Pittsburgh Epidemiology Data Center. Because symptom severity and chronicity have been found to moderate response to CBT in some studies (10, 28, 29), randomization was stratified for these variables in addition to site.

Study Therapists and Therapies

Cognitive-behavioral therapists.

A total of 10 therapists (six at the University of Pennsylvania and four at the University of Louisville) participated in the study. All therapists were experienced in delivery of standard CBT and demonstrated competence by scoring at least 40 on the Cognitive Therapy Scale (30) throughout the study. Two of the therapists at the University of Louisville had previous experience with the GDA software, but none of the other therapists had treated patients with CCBT before the present study. After attending a training workshop conducted by one of the investigators (J.H.W.), they each successfully treated at least one patient with CCBT before beginning to treat study patients. During the course of the study, ongoing consultation was provided for all therapists at both sites, jointly, during weekly conference calls conducted by experts in each particular method (consultation on CBT was provided by G.K.B., and on CCBT by J.H.W.). Specific feedback was provided whenever a therapist’s total score on the Cognitive Therapy Scale fell below 40 and whenever a therapist deviated from the session time limits for each mode of therapy.

Computer-assisted CBT (CCBT).

The 16-week CCBT protocol consisted of the nine Internet-delivered modules of GDA and 12 sessions with a therapist (see below). The modules provided a sequential grounding in the methods of CBT, ranging from behavioral activation to recognizing and addressing dysfunctional thoughts and beliefs. The final module covered relapse prevention. The modules used a blend of video illustrations, psychoeducation from a psychiatrist-narrator, feedback to users, mood graphs to measure progress, interactive skill-building exercises that help users apply CBT methods in daily life, and quizzes to assess comprehension and promote learning. Each module took about 25 minutes to complete, although patients were encouraged to work through the program at their own pace. A clinician dashboard allowed therapists to assess progress, view learning exercises, and facilitate coordination of the human and computer elements of treatment. The first 8 weeks of the protocol delivered the same “dose” of CCBT provided in the earlier study of Wright et al. (25). The initial session with the therapist lasted 50 minutes and provided both an overview of CBT and an introduction to using GDA. The next seven weekly therapy sessions were 25 minutes long. Therapists reviewed the material covered in the module and self-help assignments as a springboard for applying CBT methods to specific problem areas identified by the patient. During the second 8 weeks, patients received four 25-minute booster sessions with their therapist and could use the CCBT modules ad libitum to facilitate mastery of material. Thus, across 16 weeks, a patient could receive up to 325 minutes of contact with a therapist.

Standard CBT.

CBT was conducted according to the methods of Beck et al. (31) as updated by J. Beck (32) and by Wright et al. (33). The relatively intensive 16-week, 20-session protocol used in the earlier studies of Thase et al. (28, 29) was chosen to maximize the contrast between the two forms of therapy. Fifty-minute sessions were held twice weekly for the first 4 weeks and then weekly for the next 12 weeks. The standard CBT protocol thus consisted of up to 1,000 minutes of therapist contact, approximately three times the therapist contact provided in CCBT.

Therapy completion was defined as having attended at least two-thirds of treatment sessions (in the CBT arm, 14 of 20 sessions; in the CCBT arm, eight of 12 with the therapist and six of nine computer modules).

Dependent Measures

Depression symptom severity.

The principal symptom severity measure was the 17-item HAM-D, with the week-16 score serving as the primary outcome measure. HAM-D ratings were conducted by independent clinical evaluators blind to treatment assignment. Rater reliability was established before the study and was calibrated annually (via an exchange of video recordings) to prevent rater “drift.” All raters achieved and maintained intraclass correlation coefficients >0.8. Two well-validated self-report measures, the Beck Depression Inventory–II (BDI-II) and the Inventory of Depressive Symptomatology–Self-Rated (34), were used to obtain patients’ perspectives of symptomatic outcome. These assessments were obtained at baseline (week 0), at weeks 4, 8, 12, and 16 of the treatment protocol, and at the two follow-up assessments, 3 months and 6 months after completion of treatment (see below).

Maladaptive cognitions.

The self-report battery included the Automatic Thoughts Questionnaire (35) and the Dysfunctional Attitude Scale (36), which were completed at baseline and at weeks 8 and 16 of the treatment protocol and repeated at months 3 and 6 of the follow-up.

Interpersonal functioning.

The Inventory of Interpersonal Problems (37) was completed at the pre- and posttreatment assessments and at both follow-up evaluations. This self-report scale provides a global rating of patients’ difficulties in their relationships.

Global functioning.

The Global Assessment of Functioning Scale (GAF) was completed at baseline, at week 16, and at months 3 and 6 of the follow-up. Although this assessment does not fully uncouple functioning from the impact of depressive symptoms, it is widely used to gauge patients’ adaptation at work and at home.

Knowledge about CBT.

Patients completed the Cognitive Therapy Awareness Scale (25) at the pre- and posttreatment assessments and at months 3 and 6 of the follow-up. This 40-item true-false test measures knowledge about the principles and methods of CBT.

Follow-Up

Follow-up evaluations were conducted 3 and 6 months after completion of treatment. At the start of the follow-up, participants who had not responded to study treatment received referrals for alternative therapies. Participants who desired to continue CBT were not permitted to continue with their study therapists but received referrals to other therapists.

Statistical Analysis

The target sample size (N=172) was chosen to yield adequate statistical power to test for noninferiority of CCBT compared with conventional CBT at week 16. Noninferiority was based on the assumption that in treatment studies contrasting CBT with both an active antidepressant therapy and a placebo, the expected difference is typically a small to moderate effect size, on the order of 4 points on the HAM-D, which can represent a meaningful effect at the patient level (38). With a noninferiority design, a type I error rate at 0.025 (reflecting tests at week 16 and month 6), a standard deviation of 8 on HAM-D score, and the assumption that less than a 4-point difference in the means (±4 points) would indicate equivalence, 86 participants per cell were needed to achieve 80% power. Our final enrollment (N=154) was about 10% below this goal, which reduced statistical power to 76%.

All analyses of continuous outcome variables were based on the intent-to-treat principle. To account for the impact of the participants who dropped out of the study, a multiple imputation method was used (39). Five complete data sets were generated using the Markov chain Monte Carlo method with a single chain to create five different imputations of missing data. The results from the five complete data sets were then combined and averages were calculated to estimate the missing scores in the analyses of continuous measures.

To test the equivalence of CCBT and CBT, a confidence interval approach was used to make the desired outcome of the trial clear. A 95% confidence interval was estimated for the difference in the mean HAM-D scores of the two groups at the 16-week assessment point. CCBT would be considered to be noninferior to CBT if the upper (or lower) confidence limit was less than (or greater than) the size of the difference of the means. If significant treatment effects were identified, post hoc pairwise comparisons adjusting the type I error rate for multiplicity were planned.

We also tested the moderating effects of site, pretreatment symptom severity, and chronicity on the HAM-D, both as main effects and in interaction with treatment. We next compared the two treatments on the self-report measures of depressive symptoms, automatic negative thoughts, global functioning, and interpersonal problems. Mixed-effects regression models were used to examine the effects of CCBT compared with CBT. Separate models were fitted for each outcome and each model included random effects for intercept and time and fixed effects for treatment. As a second step for each outcome, a time-by-treatment interaction was investigated by adding the two-way interaction between time and treatment into the model.

Remission was defined as a score ≤7 on the HAM-D at week 16. During follow-up, a relapse was declared if a formerly remitted patient again met criteria for major depressive disorder. Remission and relapse rates were compared in the intent-to-treat and observed-cases samples with two-sided Fisher’s exact tests.

Results

Patient Characteristics

The sample included 154 randomized patients (see the CONSORT figure in data supplement that accompanies the online edition of this article). There were no significant differences in baseline patient characteristics between the two treatment groups. Overall, the average patient was about 45 years old, two-thirds of the participants were female, three-quarters identified themselves as white, and a little more than half had attended at least some college (Table 1). Slightly more than half of the participants had chronic depression, and nearly half scored 20 or higher on the HAM-D at the pretreatment assessment.

Treatment Completion Rates

Treatment completion rates did not differ significantly between the two intervention groups (79.2% for the CBT group and 81.8% for the CCBT group). The CCBT group completed an average of 8.1 (SD=2.1) computer modules and received an average of 11.0 (SD=3.0) sessions with their therapist (5.0 hours total). Patients in the CBT group attended an average of 16.0 (SD=5.0) therapy sessions (13.3 hours total). The CCBT group thus received 37% of the amount of therapist contact the CBT group received.

There were two serious adverse events in the CBT arm that led to premature study termination (hospitalization after a panic attack and death following an emergency hospitalization for open heart surgery). There were also two serious adverse events among patients allocated to the CCBT arm that did not result in study termination (one patient was a victim of domestic violence and another took an overdose of a small number of acetaminophen tablets during the follow-up phase of the study).

Test of Primary Hypothesis

At week 16, the mean HAM-D score was 8.9 (SD=5.6; 95% CI=7.5–10.3) for the patients in the CCBT group and 9.2 (SD=6.3; 95% CI=7.6–10.8) for those in the CBT group. CCBT thus met the criterion for noninferiority on the primary dependent measure. The observed between-group effect size difference (Cohen’s d) was 0.05, underscoring the equivalence of the two interventions.

Speed of Improvement and Remission

Both groups experienced large improvements across the 16 weeks of therapy: within-subject effect sizes were 2.4 and 2.0 for the CCBT and CBT groups, respectively. Likewise, the two treatment groups experienced a similar rate of symptom reduction across the 16 weeks of treatment (Figure 1). For the intent-to-treat sample (N=77 per arm), remission rates were 42.9% and 41.6% for the CCBT and CBT groups, respectively. Among those who completed treatment, remission rates at week 16 were 46.9% (30/64) for the CCBT group and 48.4% (30/62) for the CBT group.

Outcomes on Secondary Dependent Measures

Scores on the BDI-II, the Inventory of Depressive Symptomatology–Self-Rated, the GAF, the Automatic Thoughts Questionnaire, the Dysfunctional Attitude Scale, the Inventory of Interpersonal Problems, and the Cognitive Therapy Awareness Scale are summarized in Table 2. For the symptom and function measures, both treatment groups experienced considerable improvement, and the outcomes of the two groups did not differ significantly. There was, by contrast, a significant difference on the Cognitive Therapy Awareness Scale, which indicated that patients in the CCBT group gained significantly more knowledge about the methods of CBT than patients who received conventional CBT.

Influence of Site, Chronicity, and Severity of Depression

The two treatments were comparably effective at the two sites, suggesting that investigator allegiance and greater familiarity with CCBT did not affect the results (data available upon request). Similarly, duration of index episode was not significantly associated with outcome as a main effect or as an interaction with treatment group (data available upon request). Higher HAM-D scores at the pretreatment assessment were associated with higher scores at week 16 (F=4.74, df=1, 149, p=0.03), although this effect was similar for the CBT and CCBT groups (F=0.01, df=1, 149, p=0.92).

Results at Follow-Up Visits

Results at posttreatment months 3 and 6 are summarized in Figure 1 and Table 3. Improvements were maintained in both groups at both follow-up visits. There were no statistically significant differences between the two treatment groups on symptom and function measures. The advantage in knowledge gained about CBT observed in the CCBT arm was sustained across the follow-up period. Among the 55 participants who had remitted at week 16 and completed the follow-up, only six relapsed (11%), two in the CBT group (7%) and four in the CCBT group (16%) (Fisher’s exact test, two sided, p=0.39).

Discussion

We designed this study to provide a rigorous test of a clinician-supported method of CCBT compared with standard CBT in medication-free patients with major depressive disorder. We found noninferiority based on the primary outcome measure (HAM-D score at week 16), with comparable study completion and remission rates. It is likewise noteworthy that the CCBT and CBT groups showed similar gains on other measures of depressive symptoms, negative cognitions, global functioning, and interpersonal problems. The CCBT group gained greater knowledge about CBT compared with the CBT group, replicating the earlier finding of Wright et al. (25).

Although the great majority of previous trials of CCBT compared the computer-assisted delivery method with treatment as usual or a waiting list control condition (17–21), six studies of depressed patients compared CCBT with conventional CBT. The results of a recent meta-analysis of five of these studies suggested that comparable outcomes are possible (17). However, these studies were relatively small (25, 40–44), often did not control for use of antidepressants (40–44), and used attenuated courses of individual or group therapy as the comparator (25, 40–44). In the present study, we addressed these shortcomings by comparing CCBT to a full 16-week, 20-session course of CBT, and the primary hypothesis of equivalence was examined with a planned test of noninferiority. Moreover, we designed the study to ensure that there was a meaningful difference in the amount of clinical contact (the CBT group received 8.3 more hours of therapist contact than the CCBT group, which corresponds to 10 fewer 50-minute visits across 16 weeks). We are therefore confident that our finding of noninferiority provides strong, prospective support for the conclusion of the meta-analysis of earlier studies.

Because CCBT reduces the “dose” of therapist time, it is possible that this form of treatment would be less effective than standard CBT for patients with higher symptom severity or more complex, long-standing depressions. We also did not enroll patients who wanted to receive concomitant antidepressant therapy, which may have skewed sampling toward a subset of patients who were more highly motivated for psychotherapy. It would be worthwhile in future research to study a wider range of depressed patients, including those who prefer combined treatment with antidepressants and those have not obtained an adequate response to pharmacotherapy.

Another limitation to the generalizability of our results is that CCBT was conducted by well-trained therapists who demonstrated competence on the Cognitive Therapy Scale and received ongoing supervision. Therapists with similar expertise or support may not be available to assist with CCBT in some settings, and in other settings it may not be possible to provide up to 5 hours of therapist support. Further research is needed to determine whether comparable outcomes can be obtained with therapists and counselors who have less training or who spend less time in treatment delivery. There is some evidence from meta-analyses to suggest that therapist support times in the range of 1–3 hours may be sufficient to facilitate CCBT (17–20).

A third limitation is that our study’s noninferiority design assumes that the standard of comparison, CBT, was indeed efficacious. The finding of noninferiority does not rule out the possibility that the standard therapy was ineffective and that in fact neither treatment actually worked. To minimize the probability of this occurrence, we provided a full course of individual CBT, and the observed outcomes at both sites were comparable to those observed in controlled studies that also included antidepressants and/or pill placebo (45–47). Although we cannot rule out the possibility that CBT was no more effective than an attention placebo condition, we are confident that study participants obtained clinically meaningful improvements that are similar to the outcomes of other studies of efficacious treatments.

Additional limitations that could be addressed in future studies include the following:

1. Independent replication of the value of this model of CCBT is still needed. It is possible that the enthusiasm of learning a new approach from the developer of the technology gave the CCBT-treated patients an advantage that could not be controlled by the research design.

2. It is possible that CCBT might not work as well in a less well educated population. Other CCBT versus CBT studies that reported educational levels had similar demographic profiles (25, 48–50), possibly as a result of greater Internet access and computer experience in such populations. Further research on CCBT is needed in settings where economically disadvantaged persons are treated, particularly with people who may have had limited previous online access or computer experience.

3. The study was not large enough to permit powerful analyses of potential moderators of CCBT response. We did not find significant effects for site, severity, or chronicity, but it is possible that there are other, yet to be identified patient characteristics that are associated with CCBT response.

4. We provided only face-to-face therapist support in this investigation. Other methods of providing support (e.g., telephone, e-mail, text messaging) (48–51) may be useful, and greater use of these methods could further reduce barriers to receiving CCBT.

5. The method of CCBT used in this study required use of a personal computer or tablet and is not currently available as a mobile app. We think that the learning experience is facilitated by the personal computer/tablet environment (e.g., larger screen for viewing video, keyboard for data entry, full-screen display of learning exercises and checklists, and the attention required to set up and use the device for a concentrated period of time) as compared with the typical “on-the-go” use of mobile apps. Nevertheless, research is needed to determine whether a mobile delivery method (or a hybrid of personal computer/tablet plus mobile app) would improve the utilization and effectiveness of this treatment approach. Two recent studies with mobile apps containing CBT-oriented content found either no sustained impact on depressive symptoms (52) or less robust effects (53) than those found in our study.

Despite these limitations, we think that further development and implementation of CCBT is warranted. With increasing utilization of computers in society, improvements in broadband speed and access, and continued work on enhancing the quality of online CCBT programs, computer-assisted methods that reduce costs and improve the efficiency of psychotherapy offer a valuable means to make treatment available to larger numbers of people with depression.