Mang Shou, X., Sanderson, M. and Tuffs, N. (2004) The relationship of word error rate to document ranking. In: Proceedings of the AAAI Spring Symposium Intelligent Multimedia Knowledge Management Workshop. AAAI Technical Report SS-03-08. AAAI-IMKM, March 24–26, 2003, Stanford University, California. , pp. 28-33. ISBN 1-57735-190-8Full text available as:
This paper describes two experiments that examine the relationship of Word Error Rate (WER) of retrieved spoken documents returned by a spoken document retrieval system. Previous work has demonstrated that recognition errors do not significantly affect retrieval effectiveness but whether they will adversely affect relevance judgement remains unclear. A user-based experiment measuring ability to judge relevance from the recognised text presented in a retrieved result list was conducted. The results indicated that users were capable of judging relevance accurately despite transcription errors. This lead an examination of the relationship of WER in retrieved audio documents to their rank position when retrieved for a particular query. Here it was shown that WER was somewhat lower for top ranked documents than it was for documents retrieved further down the ranking, thereby indicating a possible explanation for the success of the user experiment.
|Item Type:||Proceedings Paper|
|Academic Units:||The University of Sheffield > Faculty of Social Sciences (Sheffield) > Information School (Sheffield)|
|Depositing User:||Repository Officer|
|Date Deposited:||23 Sep 2008 15:20|
|Last Modified:||08 Feb 2013 16:56|
Actions (login required)