Crossley, J.G.M., Groves, J., Croke, D. et al. (1 more author) (2019) Examiner training: A study of examiners making sense of norm-referenced feedback. Medical Teacher, 41 (7). pp. 787-794. ISSN 0142-159X
Abstract
PURPOSE: Examiner training has an inconsistent impact on subsequent performance. To understand this variation, we explored how examiners think about changing the way they assess. METHOD: We provided comparative data to 17 experienced examiners about their assessments, captured their sense-making processes using a modified think-aloud protocol, and identified patterns by inductive thematic analysis. RESULTS: We observed five sense-making processes: (1) testing personal relevance (2) interpretation (3) attribution (4) considering the need for change, and (5) considering the nature of change. Three observed meta-themes describe the manner of examiners' thinking: Guarded curiosity - where examiners expressed curiosity over how their judgments compared with others', but they also expressed guardedness about the relevance of the comparisons; Dysfunctional assimilation - where examiners' interpretation and attribution exhibited cognitive anchoring, personalization, and affective bias; Moderated conservatism - where examiners expressed openness to change, but also loyalty to their judgment-framing values and aphorisms. CONCLUSIONS: Our examiners engaged in complex processes as they considered changing their assessments. The 'stabilising' mechanisms some used resembled learners assimilating educational feedback. If these are typical examiner responses, they may well explain the variable impact of examiner training, and have significant implications for the pursuit of meaningful and defensible judgment-based assessment.
Metadata
Item Type: | Article |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2019 Informa UK Limited, trading as Taylor & Francis Group. This is an author produced version of a paper subsequently published in Medical Teacher. Uploaded in accordance with the publisher's self-archiving policy. |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Medicine, Dentistry and Health (Sheffield) > The Medical School (Sheffield) |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 02 Apr 2019 14:25 |
Last Modified: | 22 Nov 2021 14:00 |
Status: | Published |
Publisher: | Taylor & Francis |
Refereed: | Yes |
Identification Number: | 10.1080/0142159X.2019.1579902 |
Related URLs: | |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:144291 |