Pell, G, Fuller, R, Homer, M orcid.org/0000-0002-1161-5938 et al. (1 more author) (2013) Advancing the objective structured clinical examination: sequential testing in theory and practice. Medical Education, 47 (6). pp. 569-577. ISSN 0308-0110
Abstract
Introduction
Models of short term remediation for failing students are typically associated with improvement in candidate performance at retest, yet are costly to deliver (particularly for performance retest with Objective Structured Clinical Examinations (OSCEs)). There is increasing evidence that these traditional models are associated with longitudinal underperformance of candidates.
Methods
Rather than a traditional OSCE model, sequential testing involves a shorter ‘screening’ test format, with an additional ‘sequential’ test for candidates who fail to meet the standards of the screening test. For those tested twice, overall pass/fail decisions are then made on the full sequence of tests. The impact of sequential assessment on student performance cost of assessment delivery and overall reliability was modelled using prior data from the final, graduating OSCE of an undergraduate medical degree programme.
Results
The initial modelling predicted significant improvements to reliability in the critical area, reflected in pilot results (with 14% of students, n=228, required to sit the sequential OSCE). One student (0.4%) was identified as a false positive, i.e. under the old system would have passed the OSCE, but failed on extended testing. Nine students (4%) who would have required OSCE retests under the prior system passed the full sequence and were therefore able to graduate at the normal time without loss of earnings. Overall reliability was estimated as 0.79 for the full test sequence, with significant cost saving realised.
Discussion
Introducing sequential testing for OSCEs increases reliability for borderline students since the increased number of observations implies that ‘observed’ student marks are closer to the ‘true’ marks. However, the station level quality of the assessment needs to be sufficiently high for the full benefits in terms of reliability to be achieved. Introduction of such a system has financial benefits, good validity inferences and has proved acceptable to students and other stakeholders.
Metadata
Item Type: | Article |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | (c) 2013, Wiley-Blackwell. This is an author produced version of a paper published in Medical Education. Uploaded in accordance with the publisher's self-archiving policy (This is the pre-peer reviewed version of the following article: Pell, G, Fuller, R, Homer, M and Roberts, T (2013) Advancing the OSCE: Sequential testing in theory and practice. Medical Education, 47 (6). 569 - 577. ISSN 0308-0110, which has been published in final form at http://dx.doi.org/10.1111/medu.12136). |
Dates: |
|
Institution: | The University of Leeds |
Academic Units: | The University of Leeds > Faculty of Education, Social Sciences and Law (Leeds) > School of Education (Leeds) The University of Leeds > Faculty of Medicine and Health (Leeds) > School of Medicine (Leeds) > Leeds Institute of Medical Education > Medical Education Unit (Leeds) |
Depositing User: | Symplectic Publications |
Date Deposited: | 03 Jun 2013 11:14 |
Last Modified: | 13 Jan 2021 16:07 |
Published Version: | http://dx.doi.org/10.1111/medu.12136 |
Status: | Published |
Publisher: | Wiley |
Identification Number: | 10.1111/medu.12136 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:75610 |