A Comparison of Collaborative Care Outcomes in Two Health Care Systems: VA Clinics and Federally Qualified Health Centers

Collaborative care is a structured, stepped-care intervention designed to treat common mental health conditions such as depression and anxiety (1). In collaborative care interventions, care managers (for example, nurses and social workers) conduct outreach to the patient in between visits with their primary care provider and conduct case reviews with a consulting psychiatrist. Care managers contact the patient at prespecified intervals to conduct care coordination and management, monitor symptoms by using standardized assessment tools, and address medication adherence and side effects (1). In a meta-analysis of collaborative care trials for depression, patients receiving collaborative care exhibited greater symptom reduction than patients receiving usual care (2–4).

Prior research has indicated that collaborative care is effective in a variety of publicly and privately funded health care settings and with a variety of diagnoses (depression, anxiety disorder, and posttraumatic stress disorder [PTSD]). Five randomized controlled trials conducted with veterans seeking care for depression through the U.S. Department of Veterans Affairs (VA) health care system have shown larger symptom reductions for veterans receiving collaborative care than for veterans receiving usual care (5–9). Similarly, patients receiving collaborative care for depression in non-VA settings have exhibited substantial clinical improvements compared with patients receiving usual care (4). Although there are positive results in studies of collaborative care in both VA and non-VA publicly funded primary care clinics, a nonstatistical comparison across studies suggests that the impact of treatment varies by population, with veterans exhibiting lower response rates (5,10).

Moderation analysis is a statistical technique that can help identify specific demographic and clinical factors that are associated with treatment response heterogeneity. Veteran status is an important dimension to explore because emerging research suggests that veterans may experience more treatment-resistant mental health problems (11,12) and a poorer response to psychotherapy (13,14). Identifying differential responses to specific interventions creates an opportunity to tailor the treatment to meet the needs of specific patient populations.

To date, no studies have systematically compared collaborative care outcomes between veterans treated at VA primary care clinics and civilians treated at publicly funded non-VA primary care clinics, such as federally qualified health centers (FQHCs). Although there are many similarities among patients treated in publicly funded health care systems, there are likely to be important demographic differences between VA and FQHC patients that could confound a direct comparison of outcomes if not controlled for statistically. Patients at VA community-based outpatient clinics (CBOCs) are predominantly male, whereas FQHC patients are predominantly female. This finding is potentially problematic because prior studies of collaborative care for depression focusing on pharmacotherapy suggest that gender has a positive impact on treatment outcome, with women experiencing higher remission rates (15) and quality of life (16). Additionally, men in general are less likely to engage in and benefit from collaborative care interventions (7,17).

CBOCs and FQHCs also have important similarities and differences in the number and types of available services. Both CBOCs and FQHCs mainly offer primary care services, and most facilities do not offer on-site psychiatrists or psychologists. If a specialty mental health care service is indicated but unavailable at a CBOC, patients have the option to receive services via interactive video or to seek care in person at a larger VA health care affiliate in the region. FQHCs are not typically part of an integrated system of care, and referrals to off-site mental health providers are often problematic (18).

The primary aim of the current study is to combine data from two different collaborative care trials and to determine whether there are statistically significant differences in outcomes between veterans with depression receiving collaborative care in CBOCs versus civilians receiving collaborative care in FQHCs. One trial, Telemedicine-Enhanced Antidepressant Management (TEAM), evaluated collaborative care at CBOCs; the other trial, Outreach Using Telemedicine for Rural Enhanced Access in Community Health (OUTREACH), evaluated collaborative care at FQHCs. In post hoc analyses, we tested whether clinic type moderated the effect of the intervention after the analyses were controlled for baseline demographic and clinical characteristics.

Methods

We merged data from the collaborative care and enhanced usual care arms of two separate randomized controlled trials that tested similar interventions using similar evaluation methodologies. The TEAM study took place in seven CBOCs from April 2003 to September 2004, and the OUTREACH study took place in five FQHCs from November 2007 to June 2009. Both studies were conducted in similar geographic areas of the rural south. These studies are described in detail elsewhere (5,10). The total sample comprised 759 participants from the TEAM (N=395) and OUTREACH (N=364) trials. A review of the VA electronic health record confirmed that none of the OUTREACH study participants were receiving care at the VA. Randomization strategies in TEAM and OUTREACH differed slightly. In TEAM, CBOC sites were randomly assigned in yoked pairs to provide either collaborative care or usual care. In OUTREACH, FQHC patients were randomly assigned to receive collaborative care or usual care. This analysis was conducted under the purview of the University of Arkansas for Medical Sciences Institutional Review Board. Both trials were registered at ClinicalTrials.gov.

Collaborative Care Intervention

In both trials, collaborative care was a stepped-care intervention. In both trials, off-site telephone-based care management included depression screening with the Patient Health Questionnaire–9 (PHQ-9), education and patient activation, assessment and resolution of barriers to medication use, scheduling and monitoring of self-management goals, monitoring and promotion of adherence to medication, monitoring and management of medication side effects, and monitoring and promotion of medication adherence. Participants also had access to their on-site primary care provider and to an off-site telepsychiatrist and offsite clinical pharmacist. Participants in the TEAM trial had access to counseling at the VA medical center, and participants in the OUTREACH trial had access to a study telepsychologist. However, few participants in either study received a therapeutic dose of evidence-based psychotherapy.

Usual Care

Participants randomly assigned to receive usual care in the TEAM study received enhanced usual care in the CBOC, which included depression screening and medication management by a primary care provider. Participants randomly assigned to the usual care arm in the OUTREACH study received enhanced usual care, which included depression screening, medication management by a primary care provider, and infrequent contact (one encounter in six months) with an on-site care coordinator.

Measures

Research assistants who were blind to patient randomization conducted telephone interviews with participants at baseline and at six-month follow-ups. Demographic information included age, race, gender, income marital status, educational level, and employment. In addition, participants in both studies reported their level of social support, perceived barriers to treatment, perceived need for treatment, and perceived effectiveness of treatment. Participants also gave information on depression history and experiences with prior treatment. At baseline, participants completed several modules from the Mini-International Neuropsychiatric Interview (major depressive disorder, dysthymia, panic disorder, generalized anxiety disorder, and PTSD) (19) and the Alcohol Use Disorders Inventory Test (20). Health-related quality of life was assessed by using the Medical Outcomes Study 12-Item Short Form, which includes the Physical Component Summary and the Mental Component Summary (21). The primary outcome (depression severity) was measured by using the Symptom Checklist-20 (22).

Planned Post Hoc Analysis

A bivariate analysis was conducted to compare baseline demographic and clinical variables among participants in each trial. The explanatory variable (veteran status) represents receipt of care at the VA versus a FQHC; thus, patient type and clinic type are confounded. Therefore, the patient effect and system-of-care effect cannot be disentangled. In the multivariate analysis, we controlled for any clinical or demographic characteristics that differed significantly (p<.20) between groups. A multivariate logistic regression was used to determine whether veteran status moderated the relationship between intervention and clinical outcome (response defined as a ≥50% reduction in depression symptom severity between baseline and six-month follow-up). First, we tested a model that included all baseline characteristics and a main intervention effect as well as demographic and clinic characteristics. Next, we tested a model that added an interaction between the intervention and gender. Last, we tested a model that added an interaction between intervention and veteran status.

We also conducted sensitivity analyses to determine whether results differed by gender. Specifically, we stratified the analysis by gender to determine whether results were consistent with results from the main analysis in which both men and women were combined in the analytical sample. To determine whether differences between veterans and civilians was due to intensity-of-care differences between the two types of clinics, we added variables representing receipt of adequate antidepressant medication (full dosage on ≥80% of days in the previous month), number of mental health visits (number of visits to a specialty mental health care provider in the previous month), and number of general health visits to examine whether the addition of these treatment variables accounted for any differences detected between veterans and civilians. All analyses were conducted using SAS, version 9.3 (23).

Last, because gender and veteran status are highly correlated, we tested for multicollinearity by examining the variance inflation factor (VIF) for each predictor included in an ordinary least squares version of the model. We used the generally accepted VIF threshold of 10, which is considered to be an acceptable level of variance inflation (24).

Results

A comparison of baseline characteristics revealed a few differences between veteran and civilian samples (Table 1). Most (92%) CBOC patients were male compared with 18% of the FQHC patients. CBOC patients were also considerably older (mean±SD age=59.2±12.2) compared with FQHC patients (mean age=47.2±12.6; t=–13.33, df=757, p≤.001). Several clinical characteristics were also different between the two groups. Significantly more FQHC patients reported prior (76% vs. 66%; z=3.02, df=757, p≤.001) and current (48% vs. 41%; z=2.03, df=757, p=.04) depression treatment and rated depression treatment as acceptable (85% vs. 80%; z=2.04, df=757, p=.04).

In the full sample, 35% of the patients randomly assigned to the intervention responded to treatment compared with 15% of the usual care group (z=33.74, df=757, p<.001) (Figure 1). Overall, 19% of veterans responded to treatment compared with 30% of civilians (z=10.56, df=757, p<.01), and 20% of men responded to treatment compared with 29% of women (z=7.24, df=757, p<.01). Table 2 reports the results of model 1 (main effects only), model 2 (main effects and intervention × gender interaction), and model 3 (main effects, intervention × gender interaction, and intervention × veterans interaction).

In model 1, there were significant main effects for intervention (t=5.75, df=726, odds ratio [OR]=3.15, p<.001) but not for veteran status (t=−1.55, df=726, OR=.63, p=.12) or gender (t=−1.23, df=726, OR=.70, p=.22). In model 2, there were significant main effects for intervention (t=5.09, df=725, OR=4.56, p<.001) but not for veteran status, gender, or gender × intervention. In model 3, there were significant main effects for intervention (t=5.43, df=724, OR=5.41, p<.001) but not for veteran status or gender, and there was a significant interaction effect for intervention × veteran (t=−2.50, df=724, OR=.22, p=.01) but not for intervention × gender.

Table 3 reports the gender-stratified analysis yielded similar results for men as for the full sample. In the male subsample, there was a significant main effect for intervention (t=3.15, df=408, OR=33.24, p=.002) but not for veteran status. The moderating effect for veteran status on intervention was significant (t=−2.59, df=408, OR=.05, p=.01). For women, the intervention effect was significant (t=4.90, df=309, OR=1.15, p<.001), but veteran status was not a significant moderator.

When we added the receipt of adequate antidepressant medication and number of visits as covariates (to account for differences in treatment quality and intensity at CBOCs and FQHCs), they were not significant predictors of treatment response. Moreover, the veteran status × intervention interaction term was not affected by the addition of adequate receipt of antidepressant medication (t=−3.86, df=724, OR=.51, p<.001) or number of visits (t=−2.60, df=724, OR=.20, p=.009). Last, all predictors in the model were within the acceptable range for VIF (<10; range 1.11–7.05), including veteran status (VIF=5.65) and gender (VIF=4.07). This indicates that the large imbalance in these variables between groups did not significantly affect findings.

Discussion

Veteran status significantly moderated the positive effect of being randomly assigned to the collaborative care intervention versus usual care, according to regression analyses that controlled for demographic, clinical, and service use variables. Although the main effect for collaborative care was significantly positive, veterans who were randomly assigned to collaborative care in CBOCs reported a significantly and substantially lower response rate at the six-month follow-up compared with civilians randomly assigned to collaborative care in FQHCs. This observed treatment heterogeneity suggests that although veterans receiving care in CBOCs respond better to collaborative care compared with usual care, there are additional, unmeasured patient characteristics that contribute to poorer response when compared with civilians receiving care in FQHCs.

We performed several sensitivity analyses to evaluate the validity of the detected differences between veteran and civilian outcomes, including stratifying the analysis by gender. For men, veteran status moderated the relationship between collaborative care and depression outcomes. Combing (multiplying) the main intervention effect and the intervention × veteran interaction effect reported in Table 3, the overall treatment effect for male veterans (OR=1.66) was similar to the main treatment effect reported in the TEAM study (OR=1.94, p=.02), which was 92% male (5).

For women, veteran status did not moderate the relationship between collaborative care and depression outcomes, indicating that male but not female veterans have worse collaborative care outcomes than civilians. The main intervention effect for female civilians (OR=5.31) was similar to the main treatment effect reported in the OUTREACH study (OR=4.72, p<.001), which was 82% female (5). Evaluating outcomes for men and women separately was essential because prior research has demonstrated that women and men respond differently to collaborative care (7,17), and there was a substantially higher proportion of men in the CBOC sample than the FQHC sample. Acceptable VIFs suggested that the detected differences between groups were not unduly inflated by highly correlated independent variables.

Differences in the intensity of treatment used by patients across settings did not influence the treatment response, even though CBOC patients received a larger number of mental health encounters, had fewer primary care encounters, and were more likely to have adequate antidepressant treatment. This finding suggests that the influence of these process variables did not contribute significantly to the observed outcomes and that observed differences were not related to intensity of care received in CBOCs compared with FQHCs. However, it is still possible that unmeasured system of care characteristics contributed to the observed treatment heterogeneity.

These findings suggest that factors unique to veterans who seek care in CBOCs put veterans at increased risk for poor collaborative care outcomes, especially among men. This observed treatment heterogeneity supports prior research that suggests that veterans may have poorer mental health outcomes overall (25). The exact factors that contribute to poorer outcomes is unclear but may be due to unmeasured culture influences of military service on mental health that persist across the life span (25,26). Qualitative research on the help-seeking behaviors among veterans shows that veteran and military culture holds negative attributions for experiencing and expressing emotionality and values stoicism and self-reliance, which could interfere with treatment seeking and engagement and could lead to poorer outcomes (27,28).

A study examining differences in help-seeking patterns among men and women found that men scored higher than women on measures of stoicism and self-stigma and that lower rates of help seeking were related to lower receptivity to feelings, lower willingness to try new things, lower importance of aesthetics, and less willingness to explore values (29). Veterans may also be burdened by unmeasured general health problems resulting from military service. There may also be unmeasured clinical characteristics of patients who seek care in the VA health care system (treatment refractory) (30). Others have also argued that there may be a disincentive to report improvements in mental health status because of the structure of the VA disability compensation system (29).

In spite of multiple tests of sensitivity, some limitations should be considered. The explanatory variable reflected differences in both the patient populations and the health care systems, and these two effects could not be disentangled in the analyses. It is possible that some of the FQHC patients were veterans (although none of them were using care at the VA). Differences in randomization between studies could have affected results, although the potential cross-arm contamination associated with patient-level randomization would have biased the FQHC intervention effect toward zero, suggesting that the observed results could underestimate the differences between veteran and civilian outcomes.

Conclusions

This study included an evaluation of the influence of veteran status on treatment outcomes in collaborative care and found significant treatment heterogeneity. Efforts to tailor collaborative care to meet the needs of specific populations may increase the overall efficacy and effectiveness of collaborative care interventions, for example, increasing access to care for mental health comorbid conditions (alcohol use and PTSD) and age-related medical conditions or including spouses and families in treatment decision making. Future studies should routinely evaluate veteran and civilian differences in outcomes. Additionally, before a mental health intervention is widely recommended for implementation in the VA, it should be tested with veterans to identify ways to tailor the therapy to more effectively meet the unique needs of patients.