Dr. Goldman and Colleagues Reply
TO THE EDITOR: Dr. Steen questions our use of expert physician panels for testing the validity of utilization management criteria by pointing out that the InterQual criteria were developed to challenge practitioners toward more efficient resource use than is currently practiced. We have several major disagreements with Dr. Steen's letter.
First, we disagree with Dr. Steen's statement that our study defined validity as the matching of current practice. Current practice in our study was represented by the actual care provided according to the medical records studied. We did not compare the judgments of the criteria sets to a standard based on this care; instead, we compared the criteria to the consensus judgments of a panel of psychiatrists, selected by their peers as having special expertise in making judgments regarding the need for acute care. The panelists were instructed to ignore the constraints of actual practice by assuming that all treatment options existed at each Veterans Affairs facility studied and that private-sector psychiatric practice applied to the VA. The panel's consensus judgments were more stringent than the actual practice for 45% of the admissions and continued-stay days we studied; i.e., the panel decided that the patient should be treated on an ambulatory basis rather than as an inpatient. Furthermore, the panel's judgments were frequently more stringent than the assessments based on the InterQual criteria. The criteria indicated that inpatient care was needed while the panel judged that ambulatory care was appropriate for 36% of the discrepancies between the 1993 InterQual criteria and the panel and for 43% of the discrepancies between the 1992 InterQual criteria and the panel.
We also disagree with Dr. Steen's suggestion that to “push the frontier,” developers should strive for utilization criteria that are more stringent than expert clinicians' assessments based on the available scientific evidence. His view of criteria development (and, implicitly, of validity testing) is in conflict with the great bulk of the literature on the development of clinical guidelines (e.g., 1–3) and assessment criteria (e.g., 4, 5) and with all prior studies of the validity of utilization criteria of which we are aware (e.g., 6–8). This literature emphasizes expert clinician assessment of scientific evidence in the development and testing of criteria and guidelines. It is our belief that Dr. Steen's more subjective approach to criteria development places too much emphasis on cost reduction and too little on expert clinical judgment and scientific evidence relating to the needs of patients for appropriate care.
Finally, Dr. Steen indicates that the InterQual criteria, even if invalid, will do little harm because they are designed to flag cases for discussion with practitioners and review by physician advisers. However, it is clear from accounts of the utilization management process that the criteria significantly influence these processes (9, 10). Even if the criteria were used only as warning signals, it clearly would be more conducive to appropriate patient care if they were valid warning signals.
1 Institute of Medicine: Clinical Practice Guidelines: Directions for a New Program. Edited by Field MJ, Lohr KN. Washington, DC, National Academy Press, 1990Google Scholar
2 Woolf SH: Practice guidelines: A new reality in medicine, II: methods of developing guidelines. Arch Intern Med 1992; 152:946–952Crossref, Medline, Google Scholar
3 Hadorn DC, Baker D: Development of the AHCPR-sponsored heart failure guideline: methodological and procedural issues. Jt Comm J Qual Improv 1994; 20:539–547Medline, Google Scholar
4 Institute of Medicine: Medicare, vol 1: A Strategy for Quality Assurance. Edited by Lohr KN. Washington, DC, National Academy Press, 1990, chapter 10Google Scholar
5 Ellerbeck EF, Jencks SF, Radford MJ, Kresowik TF, Craig AS, Gold JA, Krumholz HM, Vogel RA: Quality of care for Medicare patients with acute myocardial infarction: a four-state pilot study from the Cooperative Cardiovascular Project. JAMA 1995; 2732:1509–1514Crossref, Google Scholar
6 Sui AL, Sonnenberg FA, Manning WG, Goldberg CA, Bloomfield ES, Newhouse JP: Inappropriate use of hospitals in a randomized trial of health insurance plans. N Engl J Med 1986; 315:1259–1266Crossref, Medline, Google Scholar
7 Strumwasser I, Paranjpe NV, Ronis DL, Share D, Sell MJ: Reliability and validity of utilization review criteria. Med Care 1990; 28:95–109Crossref, Medline, Google Scholar
8 Inglis AL, Coast J, Gray SF, Peters TJ, Frankel SJ: Appropriateness of hospital utilization: the validity and reliability of the Intensity-Severity-Discharge review system in a United Kingdom acute hospital setting. Med Care 1995; 33:952–957Crossref, Medline, Google Scholar
9 Institute of Medicine: Controlling Costs and Changing Patient Care: The Role of Utilization Management. Edited by Field MJ, Gray BH. Washington, DC, National Academy Press, 1989Google Scholar
10 Kongstvedt PR: The Managed Health Care Handbook, 3rd ed. Gaithersburg, Md, Aspen, 1996Google Scholar
