Title: Character-based Neural Machine Translation

Abstract: Neural Machine Translation (MT) has reached state-of-the-art results. However, one of the main challenges that neural MT still faces is dealing with very large vocabularies and morphologically rich languages. In this paper, we propose a neural MT system using character-based embeddings in combination with convolutional and highway layers to replace the standard lookup-based word representations. The resulting unlimited-vocabulary and affix aware source word embeddings are tested in a state-of-the-art neural MT based on an attention-based bidirectional recurrent neural network. The proposed MT scheme completely avoids the problem of unknown source words and provides improved results even when the source language is not morphologically rich. The number of target words is still limited by the standard word-based softmax output layer. However the number of unknowns at the output of the translation network is dramatically reduced (by a relative 66%) with a significant overall improvement over both neural and phrase-based baselines. Improvements up to 3 BLEU points are obtained in the German-English WMT task.