The Quality of Medication Treatment for Mental Disorders in the Department of Veterans Affairs and in Private-Sector Plans

The quality of health care provided by the U.S. Department of Veterans Affairs (VA) has been the subject of controversy. Some reports suggest that the quality of care is as good as or better than the private sector or Medicare, whereas others suggest that VA care is characterized by “unchecked incompetence” (1–4). No recent studies have compared the quality of outpatient care provided to veterans with mental disorders with the care provided to a comparable population treated in the private sector across multiple mental health diagnoses, although one study published in 2000 examined the quality of inpatient care episodes from 1993 to 1997 and found that during that time VA care improved “markedly” compared with the private sector (5). The lack of research is important, because many individuals with mental disorders have complex conditions that are costly to care for (whether in the VA or private sector); because mental health conditions are among the principal sources of disability in the veteran population and use of appropriate care processes has important consequences for outcomes of these conditions; and because comparison of the quality of care processes across systems can provide important insight into the effectiveness of alternative approaches to the organization, financing, and management of services for this large and growing population and inform efforts to improve care processes.

U.S. veterans are a vulnerable population, with higher rates of serious mental disorders than found in the civilian population (6). Among veterans of the Afghanistan and Iraq conflicts, prolonged and repeated deployments have magnified these problems (7). The prevalence of mental health problems, especially posttraumatic stress disorder (PTSD), is also high among veterans of earlier conflicts. Meeting the health care needs of this vulnerable population is the responsibility of the VA, which has the nation’s largest integrated health care system. In recent years, the VA has made improving mental health care for veterans an institutional priority.

In 2006, the VA Office of Policy and Planning contracted with Altarum Institute and the RAND Corporation to conduct a formal, independent evaluation of the quality of VA mental health and substance use care. The evaluation focused on veterans who had a diagnosis of one of five conditions: schizophrenia, bipolar disorder, PTSD, major depressive disorder, and substance use disorders—conditions that are the most prevalent in this population, are associated with high levels of disability, and are costly to treat. Results of this comprehensive evaluation have been reported elsewhere (8). In the study reported here, we compared the quality of VA care to that received by comparable individuals in the private sector, in an analysis conducted in collaboration with a team of researchers at Rutgers University.

Watkins and colleagues (8) described the VA’s performance measured against its own institutional goals and standards and in terms of improvements over time in providing access to services and evidence-based care. Another way to evaluate quality is to calibrate results by placing them in a broader context. Using administrative data, we used selected quality indicators to assess the performance of the VA and the performance of private-sector plans for a large, commercially insured population. We conducted the private-plan analysis in partnership with researchers at the Rutgers University Center for Education and Research on Therapeutics.

Methods

Data Sources

The research team used administrative data on utilization from the VA National Patient Care Database, which includes patient treatment files of all VA inpatient treatment discharges and outpatient care files. Laboratory and pharmacy data were obtained from the National Data Extract files. Administrative data, from the Central Fee data sets, also included care delivered by non-VA providers but paid for by the VA. The private-plan cohort was a sample of privately insured individuals developed by using administrative claims data from the Thomson-Reuters MarketScan Commercial Claims and Encounter Database (9). The database describes health care utilization of non-Medicare active employees, early retirees, individuals receiving COBRA benefits, and their dependents insured by more than 150 employer-sponsored health insurance plans and enrolled in more than 100 health plans. Pharmacy and laboratory claims are included in the database, even for beneficiaries covered through a capitated behavioral health carve out.

Defining Study Cohorts

The analysis used two study populations. The veteran cohort consisted of all veterans under age 65 who in fiscal year (FY) 2007 had at least one inpatient episode with a qualifying primary or secondary diagnosis or two outpatient encounters, at least one of which was for a qualifying primary or secondary diagnosis. Qualifying diagnoses included 43 diagnoses associated with schizophrenia, bipolar I disorder, PTSD, major depression, and substance use disorders. Each veteran was assigned to only one mental disorder diagnostic cohort on the basis of the modal frequency of appearance of diagnosis codes in the veteran’s medical utilization files, except when veterans had both a mental and a substance use disorder diagnosis. In those instances, veterans were also placed into the substance use disorder cohort in any fiscal year if their utilization records contained ICD-9-CM diagnosis codes for a substance use disorder.

The same methodology used to define the veteran cohort was used to define the private-plan cohort. We excluded individuals age 65 and older because MarketScan data do not include Medicare claims data; incomplete data for seniors would reduce the reliability and validity of the claims-based performance indicators and decrease the generalizability of the comparison of VA and private plans for these individuals. We also excluded from the MarketScan data any care provided to children under age 18, because this age group is not represented among veterans. To the extent possible, we used MarketScan data from the same period used for the VA analysis—FY 2007, with a follow-up period extending into 2008. However, the MarketScan data were available only for the first half of FY 2008; thus for two indicators (maintenance treatment with antipsychotics or mood stabilizers), the MarketScan data included only individuals who filled prescriptions for related medications in the first half of FY 2007 to allow for the full 12 months of follow-up to determine whether the numerator criteria were met for the indicator. The VA data reflected the veteran population for all who were eligible for the measure in FY 2007.

Selecting Performance Indicators

We selected seven performance indicators for the comparison with private plans from the set of administrative data indicators used in the VA evaluation (8,10) that could be operationalized by using a database of paid insurance claims. The indicators, shown in Table 1, reflect VA clinical practice guidelines or are standard best practices. To make the indicators comparable, we mapped VA pharmacy and laboratory codes to National Drug Codes and Current Procedure Terminology codes and excluded inpatient medications from the VA data to match the level of information available in the MarketScan data.

Analysis

We computed VA and MarketScan national means for each performance indicator and estimated national-level performance by age and gender. Apart from age and gender, we were unable to risk adjust for differences that exist between the two systems. We note, however, that unlike outcomes, which are strongly related to initial illness severity, the provision of services is largely under the control of providers. All our performance indicators measure processes of care for individuals for whom such care is recommended.

Results

Comparability of the Study Cohorts

Table 2 presents basic demographic and diagnostic information for the VA and MarketScan cohorts. The VA diagnostic cohorts were systematically more likely to be male and older than the privately insured cohorts drawn from the MarketScan data. For example, for major depressive disorder, 83% of the VA cohort was male, and 43% of individuals were age 55 to 64; in the privately insured cohort, the comparable percentages were 30% and 22%, respectively.

Quality of Care

We computed the VA national average and the MarketScan average for each indicator, and we report both overall results and results by gender, summarized in Table 3. [Figures presenting results of each comparison by age and gender are included in an online supplement to this article.] The proportions for each indicator when stratified by age or gender were similar to the respective national average.

VA performance was superior to the MarketScan national average by more than 30% in every case. In some cases, the margin of superiority was very large. For example, the rate at which the medication lab tests indicator was met in the VA (77%) was 13 times the rate at which it was met in the private plans (6%). This measure applies to veterans who received antipsychotic medications, lithium, valproic acid, or carbamazepine and reflects the proportion of these patients who received all recommended blood level–monitoring tests during the study period. Compared with individuals in private plans, veterans with schizophrenia or major depression were more than twice as likely to receive appropriate initial medication treatment, and veterans with depression were more than twice as likely to receive appropriate long-term treatment.

Discussion

We found that the quality of care provided by the VA to veterans with mental and substance use disorders consistently exceeded the quality of care provided by the private sector for the performance indicators examined, sometimes by large margins. The findings presented here are consistent with prior reports that VA performance consistently exceeds that of non-VA comparison groups for process-based quality measures (2,11).

It is likely that the superior performance observed in the VA system is in part the result of the additional structures that the VA has put in place to support and encourage high-quality care. These structures influence both the provider’s ability to deliver care and the patient’s ability to access and adhere to recommended treatment. For example, the colocation of pharmacy and laboratory services near specialty and primary care clinics facilitates patient access to these services, and the integrated electronic medical record means that all providers can instantly review and address patient laboratory results. Colocation of laboratory services may be particularly important for monitoring metabolic parameters among patients receiving medications that can have significant metabolic impact, as reflected in the VA’s superior performance in this area. VA providers also have access to decision support tools, and the electronic medical record supports best practices through automated clinical reminders. Network leadership provides systematic oversight of performance, and the salary model of care provides more flexibility in how resources and personnel are organized. Finally, best practices are encouraged through the dissemination of clinical practice guidelines, performance metrics, and financial performance incentives for network leaders.

Some of the differences may have stemmed from differences in patient populations. For example, the VA cohort was older, and it is established in the literature that medication adherence (which influences five of these measures) is positively correlated with age. However, VA performance was superior within each age category, suggesting that population differences were not a primary reason for the observed differences.

Our results are unlikely to be affected by missing information on dual coverage by Medicare for some individuals in the private-plan population. Because the MarketScan data set we used is derived from an employed population, it should contain very few individuals covered by Medicare in addition to their private insurance, because only dependents are eligible for additional coverage by Medicare. We note that because the private plan is the first payer for the services received by such dependents (12), the service and pharmacy use measures based on MarketScan should be valid and should not be biased downward because of dual enrollment in Medicare.

Our findings indicate much lower rates of both acute and maintenance antidepressant treatment than rates reported by Busch and colleagues (11), who compared the quality of VA and private-sector treatment in 2000 by using MarketScan data. In their study, 84.7% of VA patients versus 81% of MarketScan patients received appropriate antidepressant treatment for an acute episode, and 53.9% versus 50.9%, respectively, received appropriate continuation phase treatment. Our differing results are likely attributable to the different way our cohorts were constructed and the indicators defined and underline how even small variations in the ways that an indicator and eligible population are operationalized can have a substantial impact on results. Busch and colleagues required only one outpatient visit with a diagnosis of major depression, rather than two visits, and included individuals who may have had an additional modal diagnosis of bipolar disorder, schizophrenia, or PTSD.

Our analysis had several limitations. First, the number of indicators compared was relatively small; thus the results may not be generalizable to the care delivered for these conditions more broadly. Second, we do not know the extent of missing data. For example, medication data may have been missing because individuals were paying out of pocket for medications. Although some of the private-plan individuals were covered under behavioral health carve-out arrangements, laboratory and pharmacy claims are typically not included in the carve out and so should have been present in our data. However, data about laboratory tests may have been missing if the laboratory or physician received bundled or capitated payments for medical care on a per episode or per patient basis, which does not encourage filing of claims for individual tests. This may be more of a problem in private plans than in the VA and may explain some of the observed difference. We do not know the extent of bundled or capitated payments.

Third, the privately insured individuals did not have uniform coverage and benefit levels. The MarketScan data represent many different types of health plans, including fee-for-service, fully capitated, and partially capitated arrangements, and we did not have information on the generosity of coverage provided by different plans. Access to specialty care, particularly mental health services, probably varied from plan to plan, because those services may be carved out to behavioral health services companies, with varying screening and preauthorization algorithms.

However, because the contributors to the MarketScan databases tend to be large employers, it is likely that the health care coverage provided is more comprehensive than among the privately insured population in general, suggesting that lack of coverage was not the reason for differences in performance. Differences in out-of-pocket expenses for services (for example, copayments) may also have been larger in private insurance plans than in the VA.

Finally, the two study populations may have differed on dimensions that we could not observe and measure. Apart from age and gender, we were unable to risk adjust for unmeasured differences. Although we present national-level estimates of performance by age and gender, there may be other systematic differences between the cohorts—for example, in race-ethnicity, socioeconomic status, or general medical or mental health status—that would be useful in understanding performance results and that could not be included, because administrative data, to which the MarketScan database is limited, do not contain this information. To the extent differences existed in socioeconomic and general medical or mental health status, the veteran population was likely to be more economically disadvantaged and sicker.

The direction of any bias related to the chronicity and severity of mental disorders is unclear. Because of the stigma associated with a psychiatric diagnosis, providers may record a psychiatric diagnosis only for the sickest individuals. It is also possible that the VA may be more likely than private plans to identify mental disorders at lower levels of severity because of the VA’s extensive screening procedures, particularly for depression. There may have been differences in diagnostic coding practices between VA and private providers, and it is possible that the severity of mental disorders may have been confounded with willingness to take medications on a long-term basis. Finally, we had no information on medication possession ratios and were unable to definitively show that paid claims in the MarketScan data were equivalent to prescriptions filled in the VA.

Conclusions

These findings have important clinical and policy implications. Among veterans who received a new diagnosis of depression, suicide attempt rates have been found to be lower among patients who were appropriately treated with antidepressants than among those who were not (13,14). Among individuals with bipolar disorder, maintenance treatment with mood-stabilizing drugs was found to be associated with a decreased rate of completed suicide, compared with brief or interrupted treatment with these medications, and the rate of suicide decreased consistently with the number of additional prescriptions (15). Consistent use of antipsychotic and mood-stabilizing medication for individuals with either schizophrenia or bipolar disorder has been found to be associated with lower rates of relapse and hospitalization (16). Laboratory monitoring is essential for the safe and effective use of second-generation antipsychotics. Clinical guidelines, such as the American Psychiatric Association–American Diabetes Association guidelines, call for glucose and lipid testing for all patients starting to receive second-generation antipsychotic medication, but these guidelines and U.S. Food and Drug Administration warnings have had only limited impact on testing rates in general treatment populations. The much higher performance of the VA on these measures has important implications for the safe management of these powerful medications that can have a significant impact on clinical outcomes.