OBJECTIVE: The aim of the study was to examine the reliability
(interexaminer consistency) of the American Board of Psychiatry and
Neurology (ABPN) Part II (oral) examination in psychiatry. METHOD: Grades
were assigned independently by two examiners who observed the same
examination in a 1-year cycle (1,422 candidates, two examinations each).
The consistency between these pairs of grades (pass, condition, fail) was
analyzed using a weighted kappa statistic. RESULTS: There was perfect
agreement between examiners in 67% of examinations, minor disagreement in
26%, and major disagreement in 7% (weighted kappa = 0.54-0.56).
CONCLUSIONS: The Part II ABPN examination demonstrates fair to good
reliability as measured by interexaminer consistency. Development of more
explicit grading criteria should further improve examiner agreement in
future examinations.
Abstract Teaser