Homer, M orcid.org/0000-0002-1161-5938, Fuller, R, Hallam, J orcid.org/0000-0002-1044-0515 et al. (1 more author) (2020) Setting defensible standards in small cohort OSCEs: Understanding better when borderline regression can ‘work’. Medical Teacher, 42 (3). pp. 306-315. ISSN 0142-159X
Abstract
Introduction: Borderline regression (BRM) is considered problematic in small cohort OSCEs (e.g. n < 50), with institutions often relying on item-centred standard setting approaches which can be resource intensive and lack defensibility in performance tests.
Methods: Through an analysis of post-hoc station- and test-level metrics, we investigate the application of BRM in three different small-cohort OSCE contexts: the exam for international medical graduates wanting to practice in the UK, senior sequential undergraduate exams, and Physician associates exams in a large UK medical school.
Results: We find that BRM provides robust metrics and concomitantly defensible cut scores in the majority of stations (percentage of problematic stations 5, 14, and 12%, respectively across our three contexts). Where problems occur, this is generally due to an insufficiently strong relationship between global grades and checklist scores to be confident in the standard set by BRM in these stations.
Conclusion: This work challenges previous assumptions about the application of BRM in small test cohorts. Where there is sufficient spread of ability, BRM will generally provide defensible standards, assuming careful design of station-level scoring instruments. However, extant station cut-scores are preferred as a substitute where BRM standard setting problems do occur.
Metadata
Item Type: | Article |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2019 Informa UK Limited, trading as Taylor & Francis Group. This is an Accepted Manuscript of an article published by Taylor & Francis in Medical Teacher on 26 Oct 2019, available online: http://www.tandfonline.com/10.1080/0142159X.2019.1681388. Uploaded in accordance with the publisher's self-archiving policy. |
Keywords: | small cohorts; standard setting; OSCE; borderline regression |
Dates: |
|
Institution: | The University of Leeds |
Academic Units: | The University of Leeds > Faculty of Education, Social Sciences and Law (Leeds) > School of Education (Leeds) The University of Leeds > Faculty of Medicine and Health (Leeds) > School of Medicine (Leeds) > Leeds Institute of Medical Education > Medical Education Unit (Leeds) |
Depositing User: | Symplectic Publications |
Date Deposited: | 16 Oct 2019 13:01 |
Last Modified: | 26 Oct 2020 01:41 |
Status: | Published |
Publisher: | Taylor & Francis |
Identification Number: | 10.1080/0142159X.2019.1681388 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:152089 |