OBJECTIVE: The aim of the study was to examine the reliability
(interexaminer consistency) of the American Board of Psychiatry and
Neurology (ABPN) Part II (oral) examination in psychiatry. METHOD: Grades
were assigned independently by two examiners who observed the same
examination in a 1-year cycle (1,422 candidates, two examinations each).
The consistency between these pairs of grades (pass, condition, fail) was
analyzed using a weighted kappa statistic. RESULTS: There was perfect
agreement between examiners in 67% of examinations, minor disagreement in
26%, and major disagreement in 7% (weighted kappa = 0.54-0.56).
CONCLUSIONS: The Part II ABPN examination demonstrates fair to good
reliability as measured by interexaminer consistency. Development of more
explicit grading criteria should further improve examiner agreement in