TY - GEN
T1 - Climbing Mont BLEU: The Strange World of Reachable High-BLEU Translations
AU - Smith, Aaron
AU - Hardmeier, Christian
AU - Tiedemann, Jörg
PY - 2016/6/1
Y1 - 2016/6/1
N2 - We present a method for finding oracle BLEU translations in phrase-based statistical machine translation using exact document-level scores. Experiments are presented where the BLEU score of a candidate translation is directly optimised in order to examine the properties of reachable translations with very high BLEU scores. This is achieved by running the document-level decoder Docent in BLEU-decoding mode, where proposed changes to the translation of a document are only accepted if they increase BLEU. The results confirm that the reference translation cannot in most cases be reached by the decoder, which is limited by the set of phrases in the phrase table, and demonstrate that high-BLEU translations are often of poor quality.
AB - We present a method for finding oracle BLEU translations in phrase-based statistical machine translation using exact document-level scores. Experiments are presented where the BLEU score of a candidate translation is directly optimised in order to examine the properties of reachable translations with very high BLEU scores. This is achieved by running the document-level decoder Docent in BLEU-decoding mode, where proposed changes to the translation of a document are only accepted if they increase BLEU. The results confirm that the reference translation cannot in most cases be reached by the decoder, which is limited by the set of phrases in the phrase table, and demonstrate that high-BLEU translations are often of poor quality.
KW - phrase-based statistical machine translation
KW - BLEU score optimization
KW - document-level decoding
KW - oracle translations
KW - translation quality assessment
UR - https://www.scopus.com/pages/publications/85000916347
M3 - Article in proceedings
BT - Proceedings of the 19th Annual Conference of the European Association for Machine Translation, EAMT 2016
ER -