The construct validity of the standardized‐patient (SP) examination used at Southern Illinois University School of Medicine was assessed by comparing second‐year and fourth‐year medical students on five SP cases. The results showed sizable differences between the groups. In this article, we demonstrate the usefulness of passing rates and effect‐size measures as a means of enhancing the typically weak evidence for validity provided by group‐differences studies of construct validity. The results obtained with these approaches show that the clerkships are having a considerable effect on clinical competence, as would be expected, and that the examination is sensitive to these changes in the clinical‐competence construct as a valid measure of clinical competence should be.