Reference bias in monolingual machine translation evaluation

Fomicheva, M. and Specia, L. orcid.org/0000-0002-5495-3128 (2016) Reference bias in monolingual machine translation evaluation. In: 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016 - Short Papers. The 54th Annual Meeting of the Association for Computational Linguistics, 07-12 Aug 2016, Berlin, Germany. ACL Home Association for Computational Linguistics , pp. 77-82. ISBN 9781510827592

Abstract

In the translation industry, human translations are assessed by comparison with the source texts. In the Machine Translation (MT) research community, however, it is a common practice to perform quality assessment using a reference translation instead of the source text. In this paper we show that this practice has a serious issue - annotators are strongly biased by the reference translation provided, and this can have a negative impact on the assessment of MT quality.

Metadata

Item Type:	Proceedings Paper
Authors/Creators:	Fomicheva, M. Specia, L. https://orcid.org/0000-0002-5495-3128
Copyright, Publisher and Additional Information:	© 2016 ACL. Available under a Creative Commons Attribution 4.0 License.
Dates:	Published: 1 January 2016
Institution:	The University of Sheffield
Academic Units:	The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Computer Science (Sheffield)
Depositing User:	Symplectic Sheffield
Date Deposited:	26 Jun 2018 09:05
Last Modified:	19 Dec 2022 13:49
Published Version:	http://www.aclweb.org/anthology/P16-2013
Status:	Published
Publisher:	ACL Home Association for Computational Linguistics
Refereed:	Yes
Identification Number:	10.18653/v1/P16-2013
Open Archives Initiative ID (OAI ID):	oai:eprints.whiterose.ac.uk:132083

Download

Published Version

Filename: P16-2013

Licence: CC-BY 4.0

CLICK TO DOWNLOAD

CORE (COnnecting REpositories)

Reference bias in monolingual machine translation evaluation

Abstract

Metadata

Download

Published Version

Export

Statistics