OBJECTIVE: The purpose of this study was to identify specific competencies that differentiated passing from failing performance on the American Board of Psychiatry and Neurology Part II oral board certification examination. METHOD: A total of 363 candidates were rated in 18 areas of competence defined by behavioral criteria. Data were analyzed to determine which areas best discriminated between passing and failing candidates. RESULTS: In both parts of the examination, the ability to develop a comprehensive treatment plan was most influential. After that, specific interview and observational skills discriminated best. Failing candidates had most difficulty detecting and following informational and affective cues in their interview of a patient and presenting a biopsychosocial formulation and comprehensive treatment plan. CONCLUSIONS: Specific candidate competencies related to passing and failing performances can be identified and assessed by a rating scale of observable behaviors on the board certification examination in psychiatry.