Patients, health care providers, and patient advocacy organizations have expressed increasing concern about the confidentiality of clinical information stored in large computerized databases (1–6). Both technological advances and the increasing consolidation of health care have contributed to the growth of electronic data systems including clinical data on millions of individuals. In some hospitals and outpatient practices, computerized medical records have replaced the traditional paper record. True electronic medical records include all the details of the traditional paper chart—including narrative histories and descriptions of all clinical encounters. Most electronic medical databases, however, are designed for administrative purposes such as billing or processing of insurance claims. While these administrative systems include much less detailed information than a complete medical record, they do include information that many regard as sensitive (e.g., specific diagnoses for visits or hospitalizations, specific medications received). As local or regional health care organizations merge into national conglomerates, clinical data move farther from the control of individual providers.

The accumulation of ever-larger stockpiles of sensitive information raises reasonable concerns about inappropriate access and unauthorized disclosure. Given the stigma often attached to psychiatric disorders and psychiatric treatment, confidentiality of information on mental health and substance abuse treatment is especially critical. A few occurrences of inappropriate use or disclosure of clinical information have been well publicized (2–4). Federal legislation restricting access to large medical databases was considered during the most recent congressional session (7–9). Unfortunately, such concerns about inappropriate use of medical records data have overshadowed some long-standing and legitimate uses of records for epidemiologic and health services research (10, 11). The very characteristics of these databases that raise privacy concerns (large size, speed of access, portability) are the characteristics that make them so valuable for population-based research. As we will discuss, population-based research using large clinical databases has helped to demonstrate the impact of psychiatric disorders, the effectiveness of psychiatric treatment, and the adverse consequences of discriminatory insurance coverage. In fact, population-based research has been one of the more effective tools in the battle against stigmatization of psychiatric disorders and psychiatric treatment. Continued population-based research will be essential to address key policy questions over the next decade. We believe that appropriate safeguards can protect patient privacy while permitting legitimate research to improve the availability and quality of treatment for people with psychiatric and other medical disorders.

Before the advent of computerized record systems, hospital-based or community-based case registers were an essential component of epidemiologic and genetic research. Research using population-based case registers has helped to identify both genetic (12) and environmental (13) contributions to the etiology of psychiatric disorders. Research based on hospital case registers has made critical contributions to our knowledge of long-term prognosis (14, 15). Community case registers also supported some of the earliest research on patterns of mental health service use (16).

Some of the earliest examples of research using large computerized databases were studies examining decreases in the use of general medical services following mental health treatment (i.e., cost-offset effects) (17–21). These studies illustrate the unique value of large administrative data systems as a research resource. Using insurance claims records, investigators were able to identify individuals beginning episodes of mental health care as well as (in some cases) comparison samples of individuals not receiving mental health treatment. Identification of large and representative samples was possible only by using population-based electronic databases. Because the records of mental health care were linked to records of other health care utilization, investigators could examine changes in the use of general medical services following mental health treatment.

Access to population-based data has also been critical to recent research demonstrating the large societal burden of psychiatric illness and the potential societal benefits of improved treatment. Several studies using large clinical databases have shown that depressive disorders are associated with a tremendous burden of "excess" medical utilization that is not explained by differences in severity of medical illness (22–24). Pharmacoeconomic studies (also dependent on large clinical databases) have examined the potential economic benefits of newer psychotropic medications (25, 26). Large clinical databases also provide evidence for the benefits of appropriate psychiatric treatment in nonacademic settings (27).

Clinical research using large computerized databases has made valuable contributions to our knowledge of the effectiveness and risks of psychotropic medications in everyday practice. Studies demonstrating deficiencies in the real-world prescribing of antidepressant (28, 29) and mood-stabilizing (30) medications have helped focus attention on the large gaps between potential treatment efficacy and actual treatment effectiveness. These large observational studies prompted the development and testing of several programs to improve the prescribing of psychotropic medications in everyday practice (31–33). Data documenting adverse effects of older antidepressant drugs and long-acting sedative-hypnotic drugs have informed efforts to improve the quality of psychotropic drug prescribing for older adults (34, 35). These examples illustrate the invaluable characteristics of large clinical databases for research on treatment effectiveness: inclusion of large and representative samples of patients treated by community providers. We should caution, however, that computerized databases rarely contain sufficient clinical data to support definitive decisions regarding appropriateness of care or relative benefits of alternative treatments. Records-based research may only suggest questions and hypotheses to be addressed in later prospective studies.

Large databases of insurance claims have also allowed researchers to examine the negative consequences of restricting insurance coverage for mental health treatment. These studies have demonstrated that decreasing insurance coverage or increasing out-of-pocket expenditures for mental health care is associated with reduced access to treatment and increased rates of treatment termination (36–38). Furthermore, these reductions in use of services do not seem to be related to clinical need (i.e., the effect of changes in insurance coverage does not vary with the apparent severity of illness) (37). Research demonstrating the modest cost of parity coverage for mental health treatment has also depended on access to large insurance claims databases (39).

Over the last decade, a focus on cost reduction and profit seeking has led to both dramatic reductions in resources devoted to mental health treatment and increasing restrictions on insurance coverage of mental disorders (40). Insurance coverage is increasingly concentrated in a small number of for-profit managed behavioral health firms. Competition between firms is based almost completely on price, resulting in continued decreases in the resources devoted to mental health treatment. Overall funding levels previously considered unthinkable (e.g., $2 per person per month) have become commonplace in some regions of the United States.

Shifting the attention of health insurers and purchasers from cost reduction to quality improvement will require the development and dissemination of meaningful quality measures, or "report cards." Absent accurate data regarding access to and quality of mental health care, purchasers will continue to make choices based solely on price—and insurers will continue to compete on that basis alone. The organizations currently attempting to develop and disseminate such quality measures include the Joint Commission on Accreditation of Healthcare Organizations, the National Committee on Quality Assurance, and the American Psychiatric Association (41, 42). All of these efforts are in the earliest stages, and some of the first proposed measures have been questioned (43, 44). Providers and patients will be poorly served if officially sanctioned quality measures are either biased or unrelated to true quality of care. Population-based research will be necessary to develop meaningful quality measures and to make such measures credible to providers, patients, and purchasers. In fact, some of the criticisms of the first-generation quality measures (43, 44) have also depended on the research use of medical records data. Identifying and measuring the quality "failures" of greatest concern (e.g., restricted access to treatment, premature treatment termination) will require population-based data on the treatment actually provided.

Improving insurance coverage for mental health treatment will also depend on valid and widely applicable techniques for comparing and adjusting risk across health plans. If more generous insurance coverage attracts sicker (and more expensive) enrollees, the marketplace will punish insurers offering less restrictive coverage. Rather than competing on the basis of access and service quality, insurers are encouraged to compete by "cherry picking"—selectively enrolling only those who are unlikely to need treatment. Financial incentives for "cherry picking" can be eliminated by effective risk adjustment, the strategy of varying payments to insurers on the basis of the predicted need for treatment (45–47). Accurate methods for risk adjustment reward insurers or providers who accept responsibility for those most in need of treatment. Without such risk adjustment, insurers have no financial incentives to improve psychiatric care and strong financial incentives to maintain discriminatory coverage practices. Developing and validating risk-adjustment measures absolutely depends on data from very large populations. The accuracy of measures must be monitored over time to guard against "gaming," i.e., manipulation of risk-adjustment formulas to increase payments received. Computerized medical databases are the only viable source for such data.

Continued research access to population-based records data is essential to protecting the rights and interests of people with psychiatric illness. Investigators conducting any such research should take all possible steps to limit access to confidential information, minimize risks of disclosure, and (when possible) obtain informed consent for research use of clinical data. The most effective strategy for preventing disclosure of confidential information is to remove all identifying information from medical records data before any research use. Whenever practical, research use should be limited to such "de-identified" information. In some cases, however, identifying information is essential. Examples of situations in which there is a clear need for identifying information include linkage of data from clinical sources to other records (such as birth or death records), linkage across different clinical data systems (such as linkage of outpatient visits with hospital admissions or pharmacy data), and contacting patients to invite participation in follow-up studies (such as long-term follow-up of patients discharged from psychiatric hospitals).

Whenever practical, patients should be asked to provide specific informed consent before release of clinical data for research use. When potential research uses are anticipated at the time of data collection, those collecting clinical information should be obligated to advise patients regarding possible research use. We will discuss some specific proposals for such a process. For most current records-based research, however, obtaining individual informed consent for each specific research use is impossible or extremely impractical (11). Many studies depend on information collected years or decades previously, when no research use was anticipated. In these cases, attempts to recontact patients (or heirs of deceased patients) would often be unsuccessful. Limiting research to those contacted would often lead to biased conclusions (48). Research using computerized administrative data often includes samples in the tens (or hundreds) of thousands. In these cases, a requirement for individual informed consent before review of existing computerized records would make such research impossible (49). For these reasons, most records-based research depends on having access to identifiable clinical information without obtaining specific informed consent. In these cases, investigators must demonstrate the need for research access to records data and must develop appropriate procedures to prevent inappropriate use of or disclosure of confidential information. As we will discuss, appropriate safeguards need not be expensive or unduly burdensome for researchers.

Applicable Regulations and Statutes

The confidentiality of medical records is protected by a variety of statutes and regulations. State laws governing the privacy of medical records specify how and when medical records data may be released. These statutes, however, typically place fewer restrictions (or none at all) on the transfer of information within health systems (such as large managed care organizations). Federal regulations also govern the use of medical records data or other clinical information in research that is either federally funded or intended to support an application to the Food and Drug Administration for approval of a drug or device. Privately funded or proprietary uses of clinical data by insurers or pharmacy benefits managers remain largely unregulated. We suspect that patients (the true "owners" of clinical data) would prefer the opposite scenario: stricter control on proprietary use of clinical data than on use of data in peer-reviewed public domain research.

Research intended to increase public domain medical knowledge should be clearly differentiated from proprietary activities. Examples of such commercial or profit-oriented uses of clinical data include development of proprietary utilization management and utilization review programs, provider profiling activities, and the sale of clinical data for marketing purposes. Many of the most alarming uses of large clinical databases (e.g., selling names of patients with specific diagnoses to manufacturers of health products) fall into this category of unregulated commercial use. If legitimate public domain research activities were clearly distinguished from other uses of large clinical databases, efforts to regulate storage and disclosure of clinical data could concentrate on the activities that are now largely unregulated.

Federal regulations (50) governing the protection of human subjects clearly specify the conditions under which the normal requirements for individual informed consent can be waived or modified: "(1) The research involves no more than minimal risk to the subjects; (2) the waiver or alteration [of individual informed consent] will not adversely affect the rights and welfare of the subjects; (3) the research could not be practicably carried out without the waiver or alteration; and (4) whenever appropriate, the subjects will be provided with additional pertinent information after participation." These regulations define "minimal risk" as "the probability and magnitude of harm or discomfort anticipated in the research are not greater in and of themselves than those ordinarily encountered in daily life or during the performance of routine physical or psychological examinations or tests." Any investigator proposing to use clinical data without obtaining individual consent must demonstrate that all of the preceding criteria are satisfied. By federal law, the authority to evaluate these criteria and to grant or deny research access to clinical data rests with an institutional review board (IRB)—not with investigators or health plan administrators.

Institutional Review Boards

An independent and energetic IRB is the most important safeguard of patients’ rights. Federal regulations recognize the value of independence by specifying that all IRBs include at least one member not directly affiliated with the research institution (50). Unfortunately, some IRB reviews may not be sufficiently energetic or independent. Reports by the General Accounting Office (51) and the Inspector General of the Department of Health and Human Services (52) have questioned the effectiveness of the existing IRB system. Both reports cited increasing workloads and inadequate resources that led to inadequate oversight. Increasing the resources and personnel available to IRBs is one step toward improving the quality of initial review and periodic monitoring (53). A recent Institute of Medicine report (54) describes model policies and procedures developed by different IRBs to facilitate legitimate research while protecting individual privacy. Academic institutions and health care organizations can also bolster the independence of IRBs by increasing the level of patient or consumer participation in IRB reviews. Patient or consumer representation can help to assure that research both minimizes potential risks and maximizes potential benefits to current and future users of mental health services. Inclusion of patients or consumers can also increase the likelihood that risks and benefits are evaluated according to community standards rather than the standards of researchers or health professionals.

Federal guidelines for the protection of human subjects also identify beneficence as a fundamental consideration in the evaluation of research proposals (50, 55). IRBs are advised to weigh potential risks to research participants against potential benefits—both specific benefits to participants as well as eventual benefits to others. In the case of records-based research, the primary risk to participants is breach of confidentiality. By design, records-based research typically carries no direct health risks related to testing or treatment. For the same reason, records-based research typically yields no direct benefits to those whose records are examined. Any benefits will be indirect and will most likely accrue to other patients treated years in the future. When weighing requests for access to medical records data, IRBs must determine whether the potential benefits to future patients justify the potential threat to privacy. If significant benefit is not likely (e.g., if a research proposal indicates faulty methods or insufficient statistical power), even the slightest risk is not justified.

Minimizing Risk of Disclosure

Once granted access to medical records data, researchers must take any and all reasonable steps to minimize disclosure of identifiable clinical information. As already discussed, researchers should request access to identifiable information only when such identifiers are essential for the specific research proposed. Even when identifiers are necessary, the identifying information should be promptly removed when no longer necessary (e.g., when linkages between different data sources are complete). As a general principle, access to identifiable information should be limited to people who require such access to provide clinical care—unless access by research staff is necessary for scientific reasons. Any clinical data held by research staff (whether identifiable or not) should be held in strictest confidence, and the need for confidentiality must take precedence over considerations of economy or convenience. The additional time or expense required to "anonymize" clinical data (e.g., creation of research-specific identifiers distinct from those used in clinical care) is well justified. Identifiable data should be transferred between organizations only when absolutely necessary to accomplish scientific objectives. All identifiable data should be transferred and stored in encrypted form. The research staff must be bound by specific policies and procedures regarding confidentiality of clinical information. At a minimum, such policies and procedures should include formal training of all staff regarding confidentiality of clinical information and provisions for disciplinary action (including termination of employment) in case of unauthorized disclosure.

Communicating With Patients

Health care organizations allowing research access to records data should communicate clearly and frequently with patients or subscribers regarding research activities. While many patients sign broad waivers of confidentiality at the time of enrollment in an insurance plan, we suspect few are aware of the possible uses of large clinical databases. Health insurers have numerous opportunities to communicate with members regarding research uses of health care data (e.g., enrollment materials, patient or subscriber newsletters, benefits descriptions). Governmental agencies (such as the Department of Veterans Affairs and the Health Care Financing Administration) have similar opportunities. At each of these opportunities, insurers and other payers should clearly describe the past and expected future use of clinical data in research. These communications should describe procedures in place to safeguard confidentiality and protect patients’ rights and should identify representatives to contact regarding the privacy and rights of research participants.

In some cases, abbreviated or simplified consent procedures can be incorporated into routine practice. Such procedures can inform patients regarding the possible future research uses of clinical information and allow each patient the opportunity to exclude her or his information from research use. Such procedures need not be unduly burdensome for patients, providers, or health care organizations. Federal regulations governing informed consent allow such abbreviated procedures when the criteria for waiver of consent (listed earlier) are met. When the possibility of research use is known at the time of data collection, an abbreviated consent procedure is preferable to complete waiver of consent. The breast cancer screening risk questionnaire used by Group Health Cooperative (56) is one working example of such an approach. When research use is anticipated, similar abbreviated consent procedures could be included in any questionnaires routinely administered by health care providers or facilities (e.g., standardized assessments administered to all first-time visitors to an outpatient clinic).

The Health Insurance Portability and Accountability Act of 1996 included a mandate that, failing Congressional action, the Department of Health and Human Services issue regulations regarding the privacy of individually identifiable health information. Draft standards were issued by the Department of Health and Human Services in November 1999 (57). While the regulations create a specific exception for psychotherapy case notes, information regarding mental health treatment (or other more sensitive information) is not given special treatment. The proposed rules acknowledge the value of records-based research and explicitly address appropriate privacy protections.

The draft standards address several gaps in the existing privacy safeguards we have described. First, the requirement for review by an IRB or privacy board would be extended to all research activities (not just those supported by federal funds or those concerning an investigational drug or device). Second, commercial uses of health information (e.g., sale of patient lists for marketing purposes) are explicitly identified as an activity requiring additional consent. Third, health plans or other holders of health information would be required to disclose arrangements for sale or other commercial use. Finally, the regulations include explicit guidance for limiting access to identifiable information (e.g., encryption of identifiers, destruction of identifying linkages).

The proposed regulations also strengthen the requirements that must be satisfied for "waiver of consent" (use of individually identifiable health information without individual consent). In addition to the four criteria currently required (50), the new regulations would require the following:

• The research would be impracticable to conduct without identifiable health information.

• The importance of the research outweighs loss of individual privacy.

• There is an adequate plan to protect identifiers and prevent disclosure.

• Identifying information will be destroyed at the earliest opportunity.

The proposal by the Department of Health and Human Services allows, but does not require, mechanisms for individuals to request higher levels of privacy protection than generally provided. The draft regulations argue that requiring such a process might create conflicts with other legal and ethical obligations of health plans and health care providers. While this argument may apply to clinical and business uses of health information, we do not believe it applies to research uses. Mechanisms for patients to "opt out" of records-based research are neither administratively nor scientifically burdensome.

Heightened concern about the confidentiality of inormation on mental health treatment reflects an unfortunate reality: psychiatric disorders and psychiatric treatments are generally more stigmatized than are other health conditions. While inappropriate use of any medical information may affect future employability or insurability, psychiatric disorders are often subject to an additional burden of prejudice. This prejudice is grounded in outdated beliefs regarding the nature of psychiatric illness and the effectiveness of psychiatric treatment.

The stigmatization of mental disorders has far-reaching negative consequences. People who suffer from psychiatric disorders may not seek treatment because of shame and misunderstanding. Concern about confidentiality may also prompt some to pay "out of pocket" for treatment already paid for through insurance premiums. Discriminatory insurance coverage policies are permitted (or even codified) by law. Purchasers of health insurance (both employers and employees) do not demand accurate information regarding the availability and quality of mental health treatment because of embarrassment or a belief that "that won’t happen to me." All of these factors contribute to the current profit-driven "race to the bottom" in coverage and provision of mental health treatment.

Reducing the stigmatization of psychiatric illness will require more energetic education of business leaders, policy makers, and the general public. Effective education will require evidence regarding the impact of psychiatric disorders, the public health benefits of psychiatric treatment, and the negative consequences of denying access to necessary care. As we have argued, the most compelling evidence depends on population based-research—research that often relies on access to large clinical databases.

The situation presents a dilemma: stigmatization of psychiatric illness results in heightened concern about confidentiality of clinical information. An indiscriminate response to confidentiality concerns may restrict or preclude the population-based research necessary to combat stigma. We do not believe, however, that this dilemma is insoluble. Specific steps can permit records-based research that protects patient privacy. Public domain, peer-reviewed research should be clearly distinguished from marketing and other proprietary uses of clinical data. Existing privacy safeguards should be vigorously enforced. Organizations entrusted with clinical data should communicate and educate regarding appropriate research uses of clinical information.

In an ideal world, psychiatric diagnoses and psychiatric treatment would carry no greater stigma than do other health conditions or treatments. Until that ideal world arrives, mental health researchers must be doubly careful that records-based research protects patient confidentiality. In our efforts to protect patient privacy, however, we should take care not to endorse or reinforce prejudices against psychiatric treatment and people who suffer from psychiatric disorders. Neither should we ignore important opportunities to improve the quality of care and influence public policy through population-based research.

Received Sept. 14, 1999; revisions received Jan. 19 and May 24, 2000; accepted July 5, 2000. From the Center for Health Studies, Group Health Cooperative; the UCLA Neuropsychiatric Institute; and the Department of Psychiatry, University of Pittsburgh School of Medicine and RAND–University of Pittsburgh Health Institute. Address reprint requests to Dr. Simon, Center for Health Studies, Group Health Cooperative, #1600, 1730 Minor Ave., Seattle, WA 98101-1448; simon.g@ghc.org (e-mail). Supported in part by NIMH grant MH-51338.