Fuller, R, Homer, MS orcid.org/0000-0002-1161-5938, Pell, G et al. (1 more author) (2017) Managing extremes of assessor judgement within the OSCE. Medical Teacher, 39 (1). pp. 58-66. ISSN 0142-159X
Abstract
Context: There is a growing body of research investigating assessor judgments in complex performance environments such as OSCE examinations. Post hoc analysis can be employed to identify some elements of “unwanted” assessor variance. However, the impact of individual, apparently “extreme” assessors on OSCE quality, assessment outcomes and pass/fail decisions has not been previously explored. This paper uses a range of “case studies” as examples to illustrate the impact that “extreme” examiners can have in OSCEs, and gives pragmatic suggestions to successfully alleviating problems. Method and results: We used real OSCE assessment data from a number of examinations where at station level, a single examiner assesses student performance using a global grade and a key features checklist. Three exemplar case studies where initial post hoc analysis has indicated problematic individual assessor behavior are considered and discussed in detail, highlighting both the impact of individual examiner behavior and station design on subsequent judgments. Conclusions: In complex assessment environments, institutions have a duty to maximize the defensibility, quality and validity of the assessment process. A key element of this involves critical analysis, through a range of approaches, of assessor judgments. However, care must be taken when assuming that apparent aberrant examiner behavior is automatically just that.
Metadata
Item Type: | Article |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2016, Informa UK Limited, trading as Taylor & Francis Group. This is an Accepted Manuscript of an article published by Taylor & Francis in Medical Teacher on 27 Sep 2016, available online: https://doi.org/10.1080/0142159X.2016.1230189. Uploaded in accordance with the publisher's self-archiving policy. |
Keywords: | OSCE; Assessor judgement; Assessment quality |
Dates: |
|
Institution: | The University of Leeds |
Academic Units: | The University of Leeds > Faculty of Education, Social Sciences and Law (Leeds) > School of Education (Leeds) The University of Leeds > Faculty of Medicine and Health (Leeds) > School of Medicine (Leeds) > Leeds Institute of Medical Education > Medical Education Unit (Leeds) |
Depositing User: | Symplectic Publications |
Date Deposited: | 07 Sep 2016 10:53 |
Last Modified: | 26 Sep 2018 12:09 |
Published Version: | https://doi.org/10.1080/0142159X.2016.1230189 |
Status: | Published |
Publisher: | Taylor & Francis |
Identification Number: | 10.1080/0142159X.2016.1230189 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:104385 |