Neural machine translation has been lately established as the new state of the art in machine translation, especially with the Transformer model. This model emphasized the importance of self-attention mechanism and sug- gested that it could capture some linguistic phenomena. However, this claim has not been examined thoroughly, so we propose two main groups of meth- ods to examine the relation between these two. Our methods aim to im- prove the translation performance by directly manipulating the self-attention layer. The first group focuses on enriching the encoder with source-side syn- tax with tree-related position embeddings or our novel specialized attention heads. The second group is a joint translation and parsing model leveraging self-attention weight for the parsing task. It is clear from the results that enriching the Transformer with sentence structure can help. More impor- tantly, the Transformer model is in fact able to capture this type of linguistic information with guidance in the context of multi-task learning at nearly no increase in training costs. 1
Identifer | oai:union.ndltd.org:nusl.cz/oai:invenio.nusl.cz:387905 |
Date | January 2018 |
Creators | Pham, Thuong-Hai |
Contributors | Bojar, Ondřej, Helcl, Jindřich |
Source Sets | Czech ETDs |
Language | English |
Detected Language | English |
Type | info:eu-repo/semantics/masterThesis |
Rights | info:eu-repo/semantics/restrictedAccess |
Page generated in 0.0022 seconds