The American Psychiatric Association (APA) has updated its Privacy Policy and Terms of Use, including with new information specifically addressed to individuals in the European Economic Area. As described in the Privacy Policy and Terms of Use, this website utilizes cookies, including for the purpose of offering an optimal online experience and services tailored to your preferences.

Please read the entire Privacy Policy and Terms of Use. By closing this message, browsing this website, continuing the navigation, or otherwise continuing to use the APA's websites, you confirm that you understand and accept the terms of the Privacy Policy and Terms of Use, including the utilization of cookies.

×

Abstract

Objective: Managed care financing strategies that involve financial risk to insurers can reduce budgeted health expenditures. However, resource substitution may occur and negate apparent savings in budgeted expenditures. These substitutions may be important for individuals with disabling illnesses. The distribution of societal costs for adults with mental illnesses enrolled in plans that differ in their financial risk is examined to evaluate the degree to which risk-based financing strategies result in net savings or in the differential distribution of costs across public or private payers. Method: Six hundred twenty-eight adults with severe mental illnesses enrolled in three Medicaid plans that differ in financial risk arrangements were followed for 1 year to determine the distribution of resource use across Medicaid and other payers. Self-reported service use was obtained through interviews. Cost data were derived from self-reported expenditure, administrative, or agency data. Statistical procedures were used to control for preexisting group differences. Results: Managed care was associated with a tendency toward reduced overall costs to Medicaid. However, private expenditures for managed care enrollees offset decreased Medicaid expenditures, resulting in no net difference in societal costs associated with managed care. Conclusions: Understanding the distribution of societal costs is essential in evaluating health care financing strategies. For adults with mental illnesses, efforts to manage Medicaid expenditures may result in substituting individual and family resources for Medicaid services. Government must focus on the distribution of societal costs since risk-based financing strategies may redistribute costs across the fragmented human services sector and result in unintended system inefficiencies.

Managed care strategies that place plans at financial risk for services have been effective mechanisms for containing covered costs when contrasted with fee-for-service reimbursement (13) . Numerous studies (411) of managed care plans that have carved out mental health services support this claim. Cost containment in the health sector, however, can result in substituting resources from remaining fee-for-service payers (1214) and/or other sectors, such as the state-financed mental health service system (15) , criminal justice system (16 , 17) , other human services sectors, and/or individuals, their families, and friends (1821) .

Societal costs involve resource use across all payers and sectors (22 , 23) . Understanding the distribution of costs across these sectors is essential when evaluating managed care cost-containment strategies as a social policy (24) . This may be particularly true when considering individuals with long-term disabling illnesses (25) who have multiple needs that cross payers and sectors. The administrative data sets that are frequently used in calculating costs are not adequate to estimate societal costs since no comprehensive existing data system captures consumption of all private and governmental resources.

In this study, the societal costs for adults with severe mental illness who were enrolled in one of three common Medicaid plans were estimated (26) . These plans differ in their financial risk arrangements, thereby providing differing incentives for cost substitution within Medicaid and to other payers both inside and outside of the health sector. By contrasting the distribution of societal costs for enrollees of the three plans, the degree to which cost substitution may exist was investigated. Earlier analyses of these data identified the effects of risk arrangements on cost distributions within the health sector, particularly with regard to the use of non-Medicaid services (18) . In this article, the distribution of costs to Medicaid and other public and private payers for persons with severe mental illness who were enrolled in different plans was examined. We hypothesized that individuals enrolled in managed care plans that were at financial risk for service provision would have higher non-Medicaid costs than individuals enrolled in a fee-for-service plan without such risk.

Method

Setting

The research capitalized on a natural experiment resulting from Florida’s inaugural attempts to manage community mental health care through a 1915(b) Medicaid waiver in the Tampa Bay area (27) . The waiver established mandatory enrollment for Medicaid recipients either in a health maintenance organization (HMO) with a fully integrated premium (general health, mental health, and pharmacy) or in a behavioral health carve-out. In the carve-out, a partnership of a private, for-profit, national behavioral health care firm and local community mental health centers was at financial risk for community mental health services. Carve-out enrollees’ general health and pharmacy services were reimbursed on a fee-for-service basis by Medicaid. Since HMO and carve-out enrollees were served by the same community mental health center providers, differences between the HMO and the carve-out financing conditions should reflect plan effects rather than provider effects (27) .

In other areas of Florida, Medicaid mental health services continued to be reimbursed on a fee-for-service basis. Statewide, substance abuse services were reimbursed by Medicaid on a fee-for-service basis. State hospitalization remained under the financial auspices of the Florida mental health authority. For the purposes of the contrasts here, individuals whose mental health, general health, and pharmacy services were paid through a Medicaid fee-for-service mechanism were compared with persons in managed care. The Jacksonville area was selected as a contrast site because, of the 15 Medicaid service regions in Florida, it most resembled Tampa Bay in its health care delivery system (e.g., a similar population proportion enrolled in Medicaid, per capita Medicaid expenditures, and HMO market penetration) and demographic characteristics (e.g., sex, ethnicity, education and age distributions, poverty and unemployment rates, and median income) in census and Florida Medicaid eligibility and claims data.

Sample

Persons with severe mental illness were identified through a mail screening procedure in which a random sample of 7,658 adult (i.e., ages 21–65) Medicaid enrollees receiving supplemental security income were asked to identify their disabling condition(s). Individuals who reported disability for mental illness were contacted until the desired sample size was obtained. A total of 688 Medicaid enrollees participated in the study (80% of the individuals contacted). There were no discernible differences in age, sex, or race/ethnicity between the participants and the nonparticipants. Adults who were dually enrolled in Medicare were excluded from the waiver and this study.

Data Collection

Before the initiation of subject recruitment and data collection, all study procedures and informed consent disclosures were reviewed and approved by the University of South Florida’s Social/Behavioral Institutional Review Board. Written informed consent was obtained during face-to-face contact with potential participants. Prospective participants were given copies of the consent disclosures, which were then read to them aloud. Individuals were then given an opportunity to have questions about the study answered. Once this process was completed, those expressing interest in participating completed a six-item recall assessment about the study. Questions answered incorrectly by prospective participants were reviewed with them before enrollment in the study to ensure that they understood the content.

The participants were interviewed face-to-face by trained field interviewers bimonthly for 12 months between October 1997 and November 1999. Six hundred thirty participants (92% of the original sample of 688) were successfully contacted at least once following their initial interview.

Although differential attrition occurred across financing conditions (carve-out, 12.3%; fee for service, 8.6%; and HMO, 5.7%; χ 2 =7.01, df=2, p=0.03), no significant differences were found on any initial general health, mental health, or functional measures between individuals retained in the analysis and those lost to follow-up. Attrition is unlikely to have any effect on the validity of the study findings. Two individuals were excluded from the analysis because of extremely high service costs associated with end-of-life services. The 628 individuals included in this analysis averaged five interviews spanning 450 days. A total of 3,102 interviews were completed.

Bimonthly cost interviews used a structured calendar follow-back procedure (28) to prompt recall of resource use. The information queried included service use, residential history, legal involvement, and income. Services included medical (e.g., general health, mental health, substance abuse, pharmacy, dental, vision) and other social services (e.g., vocational rehabilitation, food stamps). Information on service type, frequency, duration, location, provider (if applicable), out-of-pocket expenses, and payments made by friends or family was collected. Persons reported their living situations over the interview period (e.g., days resided in group home, own residence, jail, etc.). Legal involvement included time in jail, prison, on probation, or performing community service, frequency of police contact/involvement, court appearances, and attorney service use. In addition to the bimonthly cost interviews, lengthy interviews at baseline, 6 months, and 12 months captured demographic information; a comprehensive review of health, mental health, and functional status; and a summary of service use.

The reliability and validity of the use data were examined. High-cost services (e.g., hospital and crisis unit stays, arrests, and incarcerations) were verified with administrative data or by contacting providers. Reported medication use was compared to pharmacy paid claims. Interview data and administrative records were found generally to agree (e.g., Cohen’s kappa coefficient [29] for use of atypical antipsychotic medications ranged from 0.66 to 0.73).

Cost Estimation

Societal costs are intended to reflect total resource consumption. Here they are operationalized as the sum of all service costs, housing subsidies, legal system costs, and support from family and friends. Income is included in societal costs since we assume that this impoverished sample does not save.

Service costs were calculated by using unit cost estimates and self-reported service use. Medicaid fee-for-service reimbursement rates for 1997 were used to estimate Medicaid unit costs, whereas per-contact cost estimates for other services were calculated by using 1997 facility financial documents. Income was calculated from self-reported wages, alimony, pensions, and 1997 entitlement amounts for public transfer income (mostly supplemental security income and food stamps). Time contributions by family and friends were valued at the minimum wage rate. Costs for each component of legal involvement (i.e., both criminal and civil involvement) were derived from estimates provided by the courts and criminal justice and legal service agencies. A detailed explanation of classification and costing procedures is available as an online data supplement at http://ajp.psychiatryonline.org.

Costs were grouped into three broad categories based on payer: Medicaid-financed, other publicly financed, and privately financed costs. Medicaid costs included all on-plan mental health, general health, pharmacy, and transportation services. Other public costs included all other government costs encompassing non-Medicaid mental health, general health, pharmacy, and criminal justice costs, public housing subsidies, volunteer costs, and public transfer income (supplemental security income, food stamps). Private costs consisted of earned transfer income (social security disability insurance, Veterans Administration benefits, pension benefits), earned income, other private transfer income (e.g., alimony), and resources (money transfers and time providing support) provided by families and friends.

Statistical Analyses

The major tests of interest involve contrasts between the three financing conditions on societal costs and its components, composed of Medicaid and other public and private costs. The nonequivalent comparison group design required correction for preexisting, between-condition differences. Linear regression and classification and regression tree (CART) analyses (30) were used to evaluate these differences. Log transformations were used to accommodate skewed cost distributions. Demographic, clinical, and method variables were included as control variables and are listed in Table 1 .

Results

On average, individuals in an HMO were more likely to be younger, Hispanic, female, and living in a private home than fee-for-service enrollees. HMO enrollees had poorer mental health functioning and more psychiatric symptoms at study enrollment than persons in the carve-out and fee-for-service conditions. Fee-for-service enrollees were more likely to be black and male, were least likely to live independently, and reported the highest level of mental health functioning. Carve-out enrollees were generally intermediate on these variables but more like HMO than fee-for-service enrollees.

Table 2 presents a summary of the unadjusted annualized total societal costs, as well as Medicaid, other public costs, and private costs and their respective components. Given the skewed cost data, analyses of variance were performed with a log-transformed cost variable. Significant differences were found among the three financing conditions with respect to societal, Medicaid, and private costs. With regard to societal costs, persons in the fee-for-service sector had significantly lower costs than persons in the HMO or carve-out condition. HMO enrollees had significantly lower Medicaid costs than persons in the carve-out or fee-for-service condition. Although the differences in other public costs did not reach conventional levels of significance, a tendency (F=2.9, df=2,625, p=0.056) toward greater costs in the HMO condition was noted. Individuals enrolled in either managed care condition had significantly greater private costs than individuals in the fee-for-service condition, with HMO enrollees having the highest private costs.

Several significant differences in the comparison of the cost subcomponents should be briefly noted. Patterns in Medicaid subcomponent expenditures were generally consistent with expectations from the financial risk arrangements, with lower Medicaid costs associated with financial risk. Outside of Medicaid, HMO enrollees reported relatively higher other public pharmacy costs and lower non-Medicaid mental health costs compared with fee-for-service enrollees. Enrollees in both managed care conditions had greater legal costs than fee-for-service enrollees. Aggregate differences in private costs were driven primarily by the contributions of family members.

Four multiple regression analyses using log (cost) as the dependent measure were conducted ( Table 3 ) to statistically control for demographic, clinical, and method differences among the financing conditions. Demographic, clinical, and method variables were significantly related to all of the cost variables. Owing to differences between Jacksonville and Tampa in the use of supported living, the interaction of housing and financing condition was included as a control for nonlinear effects. The interaction was related to all of the societal cost subcomponents except Medicaid costs.

The regression controls, and particularly the introduction of the interaction term, eliminated the financing condition effect on overall societal cost and reduced the difference in Medicaid costs to marginal significance (F=2.89, df=2, 607, p=0.06). The adjustment procedures did not affect the results for other public costs. Although private costs continued to have a strong relationship to financing condition following the regression adjustment (F=33.65, df=2, 607, p<0.01), only the coefficient contrasting the carve-out condition with fee for service achieved traditional levels of significance.

Living in the community (versus in supported housing) was negatively related to societal, Medicaid, and other public costs and positively related to private costs. An incarceration during the study was positively related to other public and societal costs. Among the clinical variables, only physical functioning had a significant unique relationship to societal and Medicaid costs.

The interaction of financing condition and housing status was strongly related to societal and other public and private costs. The fee-for-service condition had a significantly higher proportion of persons living in supported settings than either of the two managed care conditions. Fee-for-service enrollees living in supported settings had significantly higher societal and other public costs than managed care enrollees who lived in supported settings. Fee-for-service enrollees living independently had dramatically lower costs than their fee-for-service counterparts living in supported settings or managed care enrollees living independently. In fact, persons in the HMO and carve-out conditions living independently had modestly higher societal costs than their counterparts in supported settings. When these area-specific differences in the distribution of costs were controlled, the main effect of financing condition on societal costs disappeared.

To further examine the potential biases from the nonequivalence of the financing conditions, a CART analysis (30) was conducted. In the CART procedure, demographically homogeneous subgroups were hierarchically formed based upon the best predictors of financing condition group membership. Differences between the financing conditions were then tested on these homogeneous subgroups. In the CART analysis, six homogeneous subgroups were identified. Comparisons among the financing conditions for each subgroup generally paralleled those for the regression-adjusted findings. These analyses indicated that the financing condition effects are likely not the result of preexisting differences between the nonequivalent groups employed in this natural experiment.

Discussion

These results, along with those from our earlier analyses (18) , indicate that for persons with severe mental illnesses, Medicaid managed care strategies result in cost savings within the Medicaid budget. However, when broader societal costs are considered, the apparent cost savings associated with managed care are greatly diminished or eliminated. Personal expenditures as well as contributions of family and friends to managed care enrollees negate whatever savings accrued to the Medicaid budget, even when we controlled for respondent characteristics, method variables, and the interaction of living situation and plan assignment with both regression and CART control statistical procedures.

Of particular interest is the explanation for the apparently large differences in private costs for those enrolled in the HMO and carve-out plans compared with those enrolled in fee for service. The majority of private costs were time costs associated with informal care provided by family members. These differences were not explained by the differential composition of the comparison groups, since the CART analysis showed that these time costs for individuals in HMO and carve-out plans were significantly higher than for individuals in fee-for-service plans across all homogeneous subgroups. The findings suggest that there was substantially greater informal caregiver burden for persons in the managed care plans than for persons whose service providers were reimbursed by fee for service.

Managed care was not associated with significantly increased costs in other governmental programs in the aggregate. However, several differences were obtained on other public subcomponents with mixed relationships to financial risk arrangements. Counter to risk-based expectations, persons in the fee-for-service condition reported greater use of non-Medicaid mental health and substance abuse services. In contrast, HMO enrollees whose plan was at risk for pharmacy costs reported significantly more use of non-Medicaid pharmacies than either the carve-out or fee-for-service condition that was not at risk for pharmacy use. Consistent with other research (16) , persons in the managed care conditions did have greater legal involvement than their fee-for-service counterparts in the unadjusted cost data.

The study has several limitations. The most important of these relates to the nonequivalent comparison group design. The effects of the managed care interventions are confounded with the areas in which the interventions occurred. To address this potential bias, the Florida area that most resembled Tampa in the characteristics of its health care market (e.g., proportion of managed care, average state expenditures on mental health, etc.) was selected. Additionally, two different statistical control procedures (regression and CART) were used to further examine the confounding variables. Although this adjustment for between-group differences moderated several effects of financing condition, it did not substantially change interpretation, particularly with regard to private costs. Preexisting area variation, however, cannot be ruled out as a confounding factor. However, there were no reasons, a priori, to believe that between-area differences would invalidate comparisons or that methods to control these differences were not sufficient.

Although the study was quite comprehensive in documenting service use, the costing procedures relied on preexisting data sources. As such, they reflect unit costs derived from price sheets or accounting documents, as contrasted with the value of resources consumed. Relatedly, families’ and friends’ time contributions were valued at minimum wage and were therefore likely to understate the true value of family contributions (31) , as were average jail costs likely to understate the true cost of incarcerating a person with severe mental illness. Generally, we have relied on self-reported service use to determine units of services, checking high-cost and pharmacy service use against existing institutional data. Given inaccuracies in recall, this technique likely introduces some unreliability in the estimation of service volume. However, none of these differences in costing procedures should bias the between-group comparisons, since the cost estimates were uniformly applied to the three financing conditions. Also, the actual costs summarized in this article will differ in today’s market owing to the age of these data. However, the validity and relevance of the differential distribution of costs are current given the continued use of risk-based financing mechanisms.

Finally, although these differential patterns of resource use may be indicative of cost substitution in response to financing incentives, this study does not allow an unambiguous conclusion that cost substitution has occurred. For example, it is uncertain whether the involvement of families, friends, and personal resource use reflects substitutions of informal care for formal care. Furthermore, it is not clear if these substitutions are desirable or undesirable relative to community integration outcomes.

To the degree to which these cost patterns reflect cost substitution of private for public resources, they underscore the importance of broad perspectives in evaluating health care financing policy. Of importance, in other components of this research, no differences were found between the financing conditions in clinical outcomes, whereas large differences were observed in Medicaid mental health service use estimated from administrative and later self-report data (32) . Without a societal cost perspective, one might incorrectly conclude that the cost reductions in the health care sector are not associated with adverse outcomes. But in this study, informal care by family and friends was significant and may have ameliorated negative clinical and functional outcomes. Appropriately measuring and valuing informal care when evaluating these financing strategies is critical to understanding the outcomes associated with differing financial risk arrangements.

In a fragmented human services system, the distribution of costs and benefits is difficult to determine. Agency managers focus on their budgets generally without regard to the effects of their decisions on other payers. Leadership from governors, presidents, and other executive authority is needed to manage and value the broad societal consequences of health care financing decisions. To do otherwise is tantamount to pushing costs around the human service system, adversely affecting certain cost centers and perhaps increasing system inefficiency.

Summary

Evaluating the overall impact of health care financing policy requires a societal perspective. For adults with severe mental illness, it appears that efforts to contain Medicaid mental health costs may result in deflecting costs back to these vulnerable persons and onto their families and friends. Such cost substitution leads to further impoverishment of already destitute individuals. To the extent that this substitute support is not as effective as formal treatments and medications, the cost substitution may increase inefficiency and ultimately compromise outcomes. Although Medicaid authorities recognize cost savings from these managed care strategies, societal savings may be more illusory than real. A public health perspective is required to account for the array of societal costs and incorporate them in health and human services policy.

Received Dec. 20, 2006; revision received July 24, 2007; accepted Sept. 11, 2007 (doi: 10.1176/appi.ajp.2007.06122089). From the Louis de la Parte Florida Mental Health Institute, University of South Florida, Tampa, Fla.; Mental Health America; Statistics and Services Research Division, Nathan Kline Institute for Psychiatric Research, Orangeburg, N.Y.; Byrdine F. Lewis School of Nursing, Georgia State University, Atlanta; the Department of Economics, University of the Andes, Merida, Venezuela; the Mental Health Services and Policy Program, Department of Psychiatry and Behavioral Sciences, Feinberg School of Medicine, Northwestern University, Chicago; and the Center for Management of Complex Chronic Care, Hines VA Hospital, Hines, Ill. Address correspondence and reprint requests to Dr. Shern, Mental Health America, 2000 N. Beauregard St., 6th Fl., Alexandria, VA 22311; [email protected] (e-mail).

All authors report no competing interests.

Funded in part under a contract with the Florida Agency for Health Care Administration (AHCA contract number M0107) and a grant from the U.S. Substance Abuse and Mental Health Services Administration (SAMHSA grant number 5 UR TI1127-03).

The authors thank Dr. Steven Banks for statistical assistance.

References

1. Mechanic D: Managing behavioral health in Medicaid. N Engl J Med 2003; 348:1914–1916Google Scholar

2. Bloom JR, Hu TW, Wallace N, Cuffel B, Hausman J, Scheffler R: Mental health costs and outcomes under alternative capitation systems in Colorado: early results. J Ment Health Policy Econ 1998; 1:3–13Google Scholar

3. Iglehart J: Reform of the Veterans Affairs health care system. N Engl J Med 1996; 335:1407–1411Google Scholar

4. Frank RG, Garfield RL: Managed behavioral health care carve-outs: past performance and future prospects. Annu Rev Public Health 2007; 28:303–320Google Scholar

5. Rothbard AB, Kuno E, Hadley TR, Dogin J: Psychiatric service utilization and cost for persons with schizophrenia in a Medicaid managed care program. J Behav Health Serv Res 2004; 31:1–12Google Scholar

6. Bouchery E, Harwood H: The Nebraska Medicaid managed behavioral health care initiative: impacts on utilization, expenditures, and quality of care for mental health. J Behav Health Serv Res 2003; 30:93–108Google Scholar

7. Morrissey JP, Stroup TS, Ellis AR, Merwin E: Service use and health status of persons with severe mental illness in full-risk and no-risk Medicaid programs. Psychiatr Serv 2002; 53:293–298Google Scholar

8. Huskamp HA: Episodes of mental health and substance abuse treatment under a managed behavioral health care carve-out. Inquiry 1999; 36:147–161Google Scholar

9. Goldman W, McCulloch J, Sturm R: Costs and use of mental health services before and after managed care. Health Aff (Millwood) 1998; 17:40–52Google Scholar

10. Frank RG, McGuire TG: Savings from a Medicaid carve-out for mental health and substance abuse services in Massachusetts. Psychiatr Serv 1997; 48:1147–1152Google Scholar

11. Ma CA, McGuire TG: Costs and incentives in a behavioral health carve-out. Health Aff (Millwood) 1998; 17:53–69Google Scholar

12. Cuffel BJ, Goldman W, Schlesinger H: Does managing behavioral health care services increase the cost of providing medical care? J Behav Health Serv Res 1999; 26:372–380Google Scholar

13. Libby AM, Cuellar A, Snowden LR, Orion HD: Substitution in a Medicaid mental health carve-out: services and costs. J Health Care Finance 2002; 28:11–23Google Scholar

14. Norton E, Lindrooth R, Dickey B: Cost shifting in managed care. Ment Health Serv Res 1999; 1:185–196Google Scholar

15. Heflinger CA, Northrup D: Community level changes in behavioral health care following capitated contracting. Child Youth Serv Rev 2000; 22:175–193Google Scholar

16. Domino ME, Norton EC, Morrissey JP, Thakur N: Cost shifting to jails after a change to managed mental health care. Health Serv Res 2004; 39:1379–1402Google Scholar

17. Fisher WH, Packer IK, Banks SM, Smith D, Simon LJ, Roy-Bujnowski K: Self-reported lifetime psychiatric hospitalization histories of jail detainees with mental disorders: comparison with a non-incarcerated sample. J Behav Health Serv Res 2002; 29:458–465Google Scholar

18. Jones K, Chen HJ, Jordan N, Boothroyd RA, Ramoni-Perazzi J, Shern DL: Examination of formal treatment costs in the provision of managed care for a Medicaid population with psychiatric disabilities. Med Care 2006; 44:320–327Google Scholar

19. Stein BD, Reardon E, Sturm R: Substance abuse service utilization under managed care: HMOs versus carve-out plans. J Behav Health Serv Res 1999; 26:451–456Google Scholar

20. Chandler D, Hu TW, Meisel J, McGowen M, Madison K: Mental health costs, other public costs and family burden among mental health clients in capitated integrated service agencies. J Ment Health Adm 1997; 24:178–188Google Scholar

21. Stroup TS, Dorwart R: Impact of a managed mental health program on Medicaid patients with severe mental illness. Psychiatr Serv 1995; 46:885–889Google Scholar

22. Drummond MF, O’Brien B, Stoddart GL, Torrance GW: Methods for the Economic Evaluation of Health Care Programmes, 2nd ed. New York, Oxford University Press, 1997Google Scholar

23. Hargreaves WA, Shumway M, Hu T, Cuffel B: Cost-Outcome Methods for Mental Health. San Diego, Academic Press, 1998Google Scholar

24. Schlesinger M, Gray BA: A broader vision for managed care, part 1: measuring the benefit to communities. Health Aff (Millwood) 1998; 17:152–168Google Scholar

25. Grazier KL, Pollack H: Translating behavioral health services research into benefits policy. Med Care Res Rev 2000; 57(suppl 2):53–71Google Scholar

26. Hanson KW, Huskamp HA: State health care reform: behavioral health services under Medicaid managed care: the uncertain implications of state variation. Psychiatr Serv 2001; 52:447–450Google Scholar

27. Ridgely MS, Giard J, Shern D: Florida’s Medicaid mental health carve out: lessons from the first years of implementation. J Behav Health Serv Res 1999; 26:400–415Google Scholar

28. Sobell LC, Sobell MB, Maisto SA, Cooper AM: Time-line follow-back assessment method, in Alcoholism Treatment Assessment Research Instruments. National Institute on Alcohol Abuse and Alcoholism Treatment Handbook Series, vol 2: DHHS publication number 85-1380. Edited by Lettieri DJ, Sayers MA, Nelson JE. Washington, DC, NIAAA, 1985, pp 530–534Google Scholar

29. Cohen JA: A coefficient of agreement for nominal scales. Educ Psychol Measure 1960; 37–46Google Scholar

30. Breiman L, Friedman JH, Olshen RA, Stone CI: Classification and Regression Trees. Belmont, Calif, Wadsworth, 1984Google Scholar

31. Arno PS, Levine C, Memmott MM: The economic value of informal caregiving. Health Aff (Millwood) 1999; 18:182–188Google Scholar

32. Shern DL, Giard J, Robinson P, Stiles P, Boothroyd R, Murrin MR, Chen H, Boaz T, Dow M, Ward J: Evaluation of Florida’s Prepaid Mental Health Plan: Year 4, Report. Tampa, Fla, University of South Florida, Louis de la Parte Florida Mental Health Institute, 2001Google Scholar