Rationale for a multilingual corpus for machine translation evaluation

An overview of research to date in human and automated machine translation (MT) evaluation (Elliott 2002) points to a growing interest in the investigation of new automated methods, allowing for the quick and inexpensive evaluation of MT output. It is clear, however, that corpora designed for this purpose are lacking. Our own research in automated evaluation methods will require not only a corpus of source texts with machine translations that represent actual MT use, but also the detailed scores for these translations given by human evaluators. These scores will allow us to test the reliability of new automated evaluation methods. It is our intention, therefore, to compile a multilingual corpus specifically for MT evaluation, to meet not only our own research requirements, but the needs of the MT community at large.