Complex evidence-based practices are difficult to implement with fidelity in routine care. One such evidence-based practice, practice-based collaborative care, involves co-located primary care providers, mental health specialists, and care managers working together on-site in a primary care setting. Practice-based collaborative care has been shown to improve depression outcomes in numerous randomized effectiveness trials (1–7). However, implementation in smaller primary care practices presents challenges where it is typically not feasible to employ on-site mental health specialists or full-time care managers. Only 25% of U.S. primary care practices have on-site mental health specialists (8). Two studies (3, 9) have demonstrated that depression outcomes can be improved in small primary care clinics lacking on-site mental health specialists by training on-site nurses to be depression care managers; however, the effect sizes have been small to medium. Moreover, based on the results of a meta-analysis of randomized trials of practice-based collaborative care, Gilbody et al. (10) concluded that collaborative care interventions with more mental health specialist involvement have larger effect sizes.

Telemedicine technologies now make possible the virtual co-location of mental health specialists and primary care providers. Telemedicine-based collaborative care involves an off-site team of mental health specialists collaborating with on-site primary care providers, from a centralized location, using telephones, videoconferencing, and electronic health records. A multisite randomized trial conducted in the Department of Veterans Affairs (VA) (11, 12) demonstrated that telemedicine-based collaborative care is more effective than usual care in small satellite primary care clinics. Telemedicine-based collaborative care has also been adopted and sustained as part of routine care in small satellite primary care clinics within the VA (13). Compared with practice-based collaborative care, telemedicine-based collaborative care has the potential to be implemented with higher fidelity in smaller primary care clinics because the off-site team can include an array of mental health specialists, and full-time care managers can practice under more intensive clinical supervision and dedicate their time to care coordination activities, resulting in higher fidelity to care manager protocols. However, there are also potential disadvantages to telemedicine-based collaborative care. Off-site care managers cannot build on established relationships with patients and may have difficulty establishing therapeutic alliances from a distance. Likewise, if off-site care managers lack access to on-site medical records, they may have difficulty integrating behavioral and physical aspects of care and face barriers communicating with on-site primary care providers. In addition, the stepped care elements of the collaborative care model (e.g., psychiatric consultation) may be less effective when delivered via telemedicine. However, patients and providers uniformly report high levels of satisfaction with videoconferencing (14–17), and there is good evidence for the clinical equivalency of psychiatric (18–21) and psychological (22–26) treatments delivered via videoconferencing compared with face-to-face.

Given that 1) practice-based collaborative care without the involvement of on-site mental health specialists improves outcomes, 2) practice-based collaborative care with involvement from on-site mental health specialists improves outcomes more, and 3) the potential disadvantages of telemedicine-based collaborative care, an important policy-relevant question is whether it is more effective for small, remote primary care clinics to implement practice-based collaborative care without the involvement of on-site mental health specialists or to implement telemedicine-based collaborative care in partnership with an off-site team of mental health specialists.

Our objective in this pragmatic (27) comparative effectiveness trial (28) was to compare the process outcomes and clinical outcomes of patients randomly assigned to telemedicine-based collaborative care or practice-based collaborative care in small, remote primary care clinics that lacked on-site mental health specialists, thereby comparing two organizational approaches (with clinical equipoise) to delivering collaborative care for depression (29). We chose to conduct a pragmatic trial design in order to compare two viable competing strategies to delivering an evidence-based practice rather than an explanatory trial designed to determine why one approach was superior to the other. We hypothesized that compared with patients assigned to practice-based collaborative care, those assigned to telemedicine-based collaborative care would receive higher-fidelity care management, more specialty mental health services (e.g., telepsychiatry and telepsychology), and higher-quality pharmacotherapy and therefore would experience greater symptom improvement. Secondary outcomes examined included health status, quality of life, and satisfaction with care.

We partnered with five federally qualified health centers serving medically underserved populations in Arkansas’ Mississippi Delta region and the Ozark Highlands. With federal oversight from the Health Resources and Services Administration (HRSA), federally qualified health centers make up the nation’s largest and fastest growing network of primary care providers, with 8,000 clinics providing services to 20 million Americans. Three-quarters of federally qualified health center patients live in poverty, half live in rural areas, one-third are uninsured, and one-third are from minority populations. Mental health problems are the most commonly reported reasons for visits to federally qualified health centers (30), yet only 5.5% of encounters are with on-site mental health specialists (31). Research conducted in partnership with federally qualified health centers has direct applicability to a large segment of the U.S. population at risk for experiencing health disparities.

Design Overview

This multisite randomized trial employed a comparative effectiveness research design (28). Patients were randomly assigned to one of two active treatment arms, both of which represent potentially feasible approaches to adapting the evidence-based collaborative care model for routine delivery in medically underserved areas. The study design also contained many elements of pragmatic trials (27), including the use of relatively few exclusion criteria, enrollment of a diverse sample of patients, monitoring of but not controlling intervention fidelity, a primary outcome measure defined in terms of changes in patient-reported symptoms, and use of intent-to-treat analyses to examine group differences.

Setting and Participants

We approached six federally qualified health centers, and five agreed to participate. Participating centers employed between 1.3 and 9.7 primary care physician full-time equivalents, served between 5,362 and 13,050 unique primary care patients, and operated one to six clinics across multiple locations. None of the participating clinic locations had an on-site mental health specialist. Between November 2007 and June 2009, 19,285 patients were screened for depression by health center staff using the nine-item Patient Health Questionnaire (see the CONSORT diagram in the data supplement that accompanies the online edition of this article). Fifteen percent of patients (N=2,863) screened positive (a score ≥10 on the Patient Health Questionnaire); health center staff obtained informed consent from 62% of potentially eligible patients (N=829), and 55% (N=364) were found to be eligible and completed the baseline telephone interview. We excluded patients that primary care providers would not be comfortable treating; exclusion criteria included pregnancy, schizophrenia, acute suicidal ideation, substance dependence, bipolar disorder, recent bereavement, and current specialty mental health treatment. We also excluded patients who were unable to participate in research because of cognitive impairment, having a court-appointed guardian, not speaking English, having no telephone, or having a life event preventing participation. Screening and eligibility results were entered into the medical record.

Randomization and Interventions

Randomization.

Eligible patients were assigned by computer randomization using a 2×2 Latin square design (stratified by clinic) to either telemedicine-based collaborative care or practice-based collaborative care. It was not feasible to blind patients or providers to randomization status.

Practice-based collaborative care.

The practice-based collaborative care intervention was designed to be the same as the model of care supported by HRSA, known as the Depression Health Disparities Collaborative (32, 33), which represents more intensive treatment than usual care. Practice-based collaborative care involved two types of providers: on-site primary care providers and on-site nurse depression care managers. Each clinic location employed a half-time depression care manager funded by the study. Depression care managers were nurses (R.N. or L.P.N.) who had no prior mental health experience. They received a care manager training manual, 1 day of training in depression care management, and access to a web-based patient registry and depression care manager decision support system (34). Patients could choose either watchful waiting or antidepressant treatment. Depression care manager encounters were conducted either face-to-face or by telephone, depending on the patient’s preference. The initial depression care manager encounter included 1) symptom monitoring with the Patient Health Questionnaire, 2) education/activation, 3) barrier assessment/resolution, and 4) establishing self-management goals, including planning physical, rewarding, and social activities. Follow-up encounters, scheduled every 2 weeks during acute treatment and every 4 weeks during continuation treatment, included the monitoring of symptoms, medication adherence, side effects, and engagement in planned self-management activities. Depression care managers received no clinical supervision from a mental health specialist. Progress notes were entered into the patients’ paper medical record. A trial was considered to have failed in the acute phase if the patient did not respond to treatment (response was defined as a decrease of ≥50% in Patient Health Questionnaire score) after 8 weeks of treatment. No additional on-site mental health support was available for patients who did not respond to treatment, although patients could be referred to off-site mental health providers (e.g., community mental health centers). Patients received the intervention for up to 12 months.

Telemedicine-based collaborative care.

Telemedicine-based collaborative care involved five types of providers: on-site primary care providers and off-site depression care managers (at the R.N. level), pharmacists (at the Pharm.D. level), psychologists (at the Ph.D. level), and psychiatrists. The off-site team, located at the University of Arkansas for Medical Sciences, was funded by the study. The full-time depression care manager was a nurse who had no prior mental health experience and received the same training and tools as the on-site depression care managers. All depression care manager encounters were conducted by telephone and followed the protocol described above. Progress notes were faxed to the clinic. During weekly meetings, the depression care manager received clinical supervision and the off-site team discussed new patients and patients who were not responding to treatment, and they offered treatment recommendations to primary care providers via the depression care manager progress notes. Patients received stepped care, whereby treatment intensity was increased for patients who were not responding to treatment. If the patient did not respond to the initial antidepressant, the pharmacist conducted a medication history and provided medication management by telephone as needed. If the patient did not respond to two antidepressant trials, a psychiatry consultation via videoconferencing was scheduled. At any time, patients had access to cognitive-behavioral therapy (CBT) delivered via videoconferencing. Patients who did not respond to an antidepressant trial were specifically encouraged to initiate and complete CBT.

Outcomes and Follow-Up

Fidelity assessment.

Fidelity to the depression care manager protocol was measured through chart review. Five of the charts (1.4%) could not be located during the site visits. The following fidelity measures were abstracted: number of depression care manager encounters with documented Patient Health Questionnaire scores; self-management activities; antidepressant adherence assessments and side effect assessments for those treated with antidepressants; and counseling adherence assessments for those referred to psychotherapy. Depression care manager fidelity was also measured from patient self-report during the 6- and 12-month telephone interviews using items that addressed education, self-management, symptom monitoring, adherence monitoring, and collaboration among providers. To measure fidelity to the stepped care protocol, we examined what proportion of patients in the telemedicine group who failed to respond to at least one medication trial had a telephone pharmacist encounter and what proportion of those who failed to respond to two trials had a telepsychiatry consultation. We also examined what proportion of patients in the telemedicine-based group attended at least one telepsychotherapy session and completed at least eight sessions.

Baseline interviews.

Data were collected at baseline in a blinded telephone interview. At baseline, sociodemographic and clinical case-mix data were collected using the Depression Outcomes Module (35, 36), the Mini International Neuropsychiatric Interview (37, 38), the Duke Social Support and Stress Scale (39, 40), the Quality Improvement for Depression Treatment Acceptability scale (3, 4), and the Depression Health Beliefs Inventory (41). Zip codes were used to categorize patients’ residence as rural or urban according to Rural Urban Commuting Area classification scheme C.

Follow-up interviews.

Blinded follow-up telephone interviews were completed for 87% of participants (N=318) at 6 months, 79% (N=287) at 12 months, and 78% (N=283) at 18 months. In addition to the fidelity measures described above, the other primary process measures included self-reported mental health service utilization, antidepressant prescriptions, antidepressant dosage (categorized as starting, usual, or high [42]), and antidepressant adherence (taking the full prescribed dosage at least 80% of days in the previous month). The primary clinical outcome measures were changes in depression severity and treatment response and remission. Depression severity was measured continuously using the 20-item Hopkins Symptom Checklist (HSCL) (43, 44). Response, measured dichotomously, was defined as a decrease of ≥50% in HSCL score between baseline and follow-up. Remission, measured dichotomously, was defined as an HSCL score <0.5. Secondary outcome measures included health status (based on the physical and mental health composite scale scores from the 12-item Short-Form Health Survey) (45, 46), quality of life (based on the Quality of Well-Being scale) (47–50), and satisfaction with care (based on the Consumer Assessment of Healthcare Providers and Systems) (51).

Statistical Analysis

Patients were the unit of the intent-to-treat analysis. Clinic-level intraclass correlation coefficients for HSCL change scores were not significant. Provider-level intraclass correlation could not be calculated because patients could have multiple primary care providers. For the hypotheses examining fidelity, separate logistic regressions were specified for the first 6 months and the second 6 months. For the hypotheses examining other outcomes, we used mixed models and included data from all completed research assessments (52). Case-mix variables were selected using the method of purposeful selection (53). Case-mix variables with missing values were imputed using the PROC MI procedure in SAS, version 9.3 (SAS Institute, Cary, N.C.). PROC MIXED and PROC GLIMMIX were used with the PROC MIANALYZE procedure to model outcomes with linear, binomial, negative binomial, or ordinal distributions. All models specified clinic as a random effect to control for intraclass correlation coefficient. Time was included as a fixed effect. The model specifications included main effects for group and time (with practice-based collaborative care assigned as the reference group) and interaction effects for group by time. The main group effect was used to test the hypotheses for dependent variables not measured at baseline (e.g., treatment response). For dependent variables that were measured at baseline (e.g., HSCL score), group-by-time interaction effects were used to test the hypothesis that the rate of improvement differed across the two groups. Because there were multiple group-by-time interaction terms, an omnibus test was used to determine whether these variables collectively explained a significant amount of the variance in the dependent variable (54). If the omnibus test was significant at the conservative alpha level of 0.10, we report group differences and significance tests for each time period. Otherwise, we report group differences averaged across the time periods and one significance test. The analysis had an 85% power to detect a 15% difference (e.g., 30% versus 45%) in response rates.

The study participants’ socioeconomic and clinical characteristics are summarized in Table 1. Two-thirds (64.5%) were unemployed, 69.7% had an annual household income <$20,000, 50.8% were uninsured, and 68.1% lived in a rural area. At baseline, 83.2% met diagnostic criteria for major depressive disorder, and the mean HSCL score was 1.9, indicating moderately severe depression. The mean number of chronic physical health disorders was 4.6 (SD=2.6), and psychiatric comorbidity was common. Mean physical and mental health composite scale scores were nearly two standard deviations below those for the general population. Nearly half (48.4%) were already receiving depression treatment at enrollment, indicating treatment resistance.

Care Manager Fidelity

Table 2 describes care manager fidelity. At the 6- and 12-month follow-ups, significantly more patients in the telemedicine-based group than in the practice-based group reported that a health care professional other than their primary care provider 1) gave them helpful information about depression or depression treatment (odds ratios, 2.77 at 6 months and 2.32 at 12 months); 2) made helpful suggestions about things they could do to help depression, such as exercise or becoming more socially active (odds ratios, 3.47 at 6 months and 2.50 at 12 months); 3) asked them about their depression symptoms (odds ratios, 3.60 at 6 months and 2.63 at 12 months); and 4) asked them if they were taking antidepressant medications as prescribed or attending scheduled counseling sessions (odds ratios, 4.70 at 6 months and 3.96 at 12 months). More patients in the telemedicine-based group also reported that their primary care provider worked collaboratively with a mental health specialist (odds ratios, 4.63 at 6 months and 9.05 at 12 months).

Compared with patients in the practice-based group, those in the telemedicine-based group had significantly more depression care manager encounters in which Patient Health Questionnaire depression severity scores were documented in the medical record during both the first 6 months and the second 6 months (incidence rate ratios, 4.10 and 4.64, respectively). Patients in the telemedicine-based group also had significantly more depression care manager encounters in which self-management goals were documented during the first 6 months (incidence rate ratio=5.62). Among patients who received a prescription for an antidepressant, those in the telemedicine-based group had significantly more depression care manager encounters in which medication adherence was documented during both the first 6 months and the second 6 months (incidence rate ratios, 2.69 and 2.49, respectively). Likewise, patients in the telemedicine-based group who received a prescription for an antidepressant had significantly more depression care manager encounters in which the presence or absence of side effects was documented during both the first 6 months and the second 6 months (incidence rate ratios, 4.22 and 4.46, respectively). Among patients who were referred to counseling, there were no significant group differences in the number of depression care manager encounters in which session attendance was documented in the medical record.

Stepped Care Fidelity

Among patients in the telemedicine-based group who failed to respond to at least one medication trial (N=73), 8.2% (N=6) had a telephone encounter with the pharmacist. Among those who had failed to respond to two trials (N=29), 48.3% (N=14) had a telepsychiatry consultation. Another seven patients had an ad hoc telepsychiatry consultation. Also, 16.6% of patients (N=30) attended at least one telepsychotherapy session, and 7.8% (N=14) completed eight or more sessions.

Treatment Process Outcomes

Table 3 summarizes the treatment provided to the two groups. There were no significant group differences at baseline, and no significant group-by-time interactions for number of primary care visits or number of depression-related primary care visits, or mental health visits. With respect to antidepressant treatment, there were no significant group main effects for receiving an antidepressant prescription, number of prescribed antidepressants, dosage level (starting, usual, high), or adherence.

Clinical Outcomes

Clinical outcomes are presented in Table 4. A significant group main effect was observed for both response (odds ratio=7.74) and remission (odds ratio=12.69), with patients in the telemedicine-based group having better outcomes. We also observed a significant overall group-by-time interaction effect for HSCL score (χ²=40.51, df=3, p<0.001), with greater reductions in severity for patients in the telemedicine-based group (Figure 1). There was also a significant overall group-by-time interaction effect for mental health composite scale score (χ²=11.46, df=3, p=0.01) and Quality of Well-Being score (χ²=6.55, df=3, p=0.09), with greater improvements among patients in the telemedicine-based group. There were no significant overall group-by-time interaction effects for physical health composite scale score, although this finding is likely an artifact resulting from the orthogonal factor rotation and negative weights used to score the physical and mental health composite scales (55). There were no group differences in satisfaction at baseline, but the omnibus test of the group-by-time interactions was significant at the 0.10 level (χ²=6.69, df=3, p=0.08), with the telemedicine-based group having higher satisfaction.

Study participants were recruited from small, remote primary care clinics associated with the largest publicly funded health care system in the country. The sample was predominantly rural, unemployed, and uninsured. Patients had numerous comorbidities, and their depression was treatment resistant. The high degree of treatment resistance likely contributed to the low response and remission rates among those assigned to the practice-based collaborative care group. Compared with patients assigned to the practice-based collaborative care group, those in the telemedicine-based group had significantly and substantially greater treatment response rates, remission rates, reductions in depression severity, and increases in mental health status and quality of life. Improved outcomes were achieved in the telemedicine-based group without increasing the number of primary care visits. Improved outcomes appear to be due to higher fidelity to the care manager protocol in the telemedicine-based collaborative care model, despite the fact that off-site and on-site depression care managers had similar levels of clinical experience at baseline and underwent identical training. The higher fidelity to the care manager protocol in the telemedicine-based model may have been due to the fact that the off-site depression care manager practiced under more intensive clinical supervision and dedicated 100% of her time to depression care manager activities.

The greater fidelity to the care manager protocol did not translate into improvements in the quality of pharmacotherapy for the telemedicine-based group. The majority of patients in both groups initiated antidepressant treatment at therapeutic dosages, and adherence was high. Likewise, the psychotherapy available to patients in the telemedicine-based group via videoconferencing was not highly utilized and was not likely to have contributed substantially to improved outcomes. Our findings were essentially unchanged when patients who received telepsychotherapy were excluded from the analytical sample. Because group differences in outcomes are not likely to be attributable to either pharmacotherapy or psychotherapy, we hypothesize that patients assigned to telemedicine-based collaborative care were more likely to engage in self-management activities, such as physical, rewarding, and social activities. This hypothesis is based on the finding that patients in the telemedicine-based group received more encouragement from the depression care manager to engage in self-management activities. Previous research has demonstrated that behavioral activation is a clinically effective stand-alone treatment for depression (56, 57). Also, patients in the practice-based collaborative care intervention in Project Impact (58) were found to have significantly better treatment response rates if the depression care manager documented in the medical record that the patient scheduled physical, rewarding, and social activities. This hypothesis needs to be tested using an experimental dismantling study specifically designed to estimate the incremental treatment effect of scheduling self-management activities. Another possibility is that the more frequent depression care manager encounters provided to the telemedicine-based group resulted in greater social support, which in turn reduced depressive symptoms. This was the conclusion of Hunkeler et al. (59), who similarly reported that a depression care manager program at Kaiser Permanente did not improve antidepressant management, but did reduce depression symptoms.

An inherent limitation of this pragmatic trial is that the results are not conclusive with respect to identifying treatment mechanisms. Explanatory trial designs are needed to determine why patients assigned to receive telemedicine-based collaborative care had better outcomes than those assigned to receive practice-based collaborative care. However, for policy makers at HRSA and federally qualified health centers lacking on-site mental health personnel, these results clearly indicate that contracting with an off-site depression care team yields better depression outcomes than implementing collaborative care with staff available on-site. Future research should also examine whether having on-site nurse care managers supported by off-site psychiatrists, psychologists, and pharmacists via telephone and videoconferencing is an effective organizational approach to delivering collaborative care.

The authors gratefully acknowledge the patients and staff at the Boston Mountain Rural Health Center, Community Clinic NWA, Corning Area Healthcare, East Arkansas Family Health Center, and Jefferson Comprehensive Care Systems, as well as staff at the Community Health Centers of Arkansas. They also acknowledge the important contributions of project staff, including Amanda Davis, Loretta Ducker, Debbie Hodges, Choi Lai, Liya Lu, Michael McCarther, Camille Mack, Jennifer Stephens, and Vera Tate, as well as the contributions of the anonymous manuscript reviewers.