Assessment of psychotherapy skills: the problem of interrater agreement
Abstract
Thirteen experienced supervising psychiatrists independently rated the psychotherapy interviews of six residents, using a detailed assessment instrument. Although interrater agreement was significantly greater than would be expected by chance, the extent of agreement was uniformly low. Agreement was significantly poorer for judgments about technique skills than about communication skills and for second-year than for first-year residents. Other factors, such as the faculty rank and training of the rater, did not covary significantly with observed ratings. The findings suggest that rating methods which purport to assess psychotherapy skills should undergo careful methodological evaluation and testing before they are accepted as valid assessment techniques.
Access content
To read the fulltext, please use one of the options below to sign in or purchase access.- Personal login
- Institutional Login
- Sign in via OpenAthens
- Register for access
-
Please login/register if you wish to pair your device and check access availability.
Not a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5 library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).
