Dr. Bauer Replies
To the Editor: Dr. Mattes’s comments are extremely important. He correctly points out that evidence-based medicine analyses are limited by the availability of relevant data—typically published studies. Even the Cochrane Collaboration, the premier evidence-based medicine review group (1), is limited primarily to published or soon-to-be published data. Should we worry that the peer-reviewed, published data on which we based our evidence-based decisions are biased? The answer is overwhelmingly, “Yes, worry!” For instance, meta-analyses of antidepressant and anxiolytic efficacy, based on predominantly published data, are regularly positive (1). However, analysis of the Food and Drug Administration’s (FDA’s) Summary Basis of Approval Reports for approved antidepressants and anxiolytics—which reflect all studies, published and unpublished, submitted to the FDA—indicates that the majority of such studies are actually negative (2). Concern is not limited to clinical trials but extends to pharmacoeconomic analyses as well: industry-sponsored studies are significantly more likely than non-industry-sponsored studies to favor their proprietary drugs over comparators (3). Potential bias in the published, peer-reviewed evidence base is not limited to the mental health literature. This issue has in fact been investigated much more extensively in medicine (e.g., references 3–6). Of additional concern, it has been alleged that the pharmaceutical corporations have frequently been in violation of a 1997 law mandating reporting to the FDA of all clinical trials of a treatment when seeking FDA approval (7), thus compromising even the unpublished data set in the public domain. The impact of such practices on psychiatric research, practice, and public health has simply yet to be delineated. Regardless of how even-handed evidence-based reviews of the peer-reviewed literature are, they are still limited by what is actually published. Dr. Mattes’s message is appropriate: Caveat lector!
Reprints are not available; however, Letters to the Editor can be downloaded at http://ajp.psychiatryonline.org.
1. Cochrane Collaboration. www.cochrane.orgGoogle Scholar
2. Khan A, Khan S, Brown WA: Are placebo controls necessary to test new antidepressants and anxiolytics? Int J Neuropsychopharmacol 2002; 5:193–197Crossref, Medline, Google Scholar
3. Baker CB, Johnsrud MT, Crismon ML, Rosenheck RA, Woods SW: Quantitative analysis of sponsorship bias in economic studies of antidepressants. Br J Psychiatry 2003; 183:498–506Crossref, Medline, Google Scholar
4. Davidson RA: Source of funding and outcome of clinical trials. J Gen Intern Med 1986; 1:155–158Crossref, Medline, Google Scholar
5. Rochon PA, Gurwitz JH, Simms RW, Fortin PR, Felson DT, Minaker KL, Chalmers TW: A study of manufacturer-supported trials of nonsteroidal anti-inflammatory drugs in the treatment of arthritis. Arch Intern Med 1994; 154:157–163Crossref, Medline, Google Scholar
6. Djulbegovic B, Lacevic M, Cantor A, Fields KK, Bennett CL, Adams JR, Kuderer NM, Lyman GH: The uncertainty principle and industry-sponsored research. Lancet 2000; 356:635–638Crossref, Medline, Google Scholar
7. Vedamtam S: Drugmakers prefer silence on test data. Washington Post, July 9, 2004Google Scholar