Institutional Repository of Vilnius University
Teksto žinių perkėlimas natūralios kalbos apdorojimo uždaviniams /
Abstract
dc:descriptionThis thesis analyses transformer architecture deep neural networks for natural language processing tasks and text knowledge transfer for Lithuanian language using these networks. During the experiment two transformer architecture models were created and trained using knowledge transfer methods. In order to test the performance of created models, they were fine-tuned to recognize authors of Lithuanian texts. The development of natural language processing is briefly reviewed then the literature about transformer deep neural networks is analysed. Thesis describes the features of the original transformer architecture: encoder, decoder, attention mechanism, input encodings. Then BERT architecture is described, similarities and differences with original transformer architecture. Knowledge transfer methods are also described. Experiments start by choosing the model architecture. The data needed for knowledge transfer is described. Data was collected from Directorate-General for Translation of the European Commission translation memory. Models were trained using knowledge transfer method. Created models were fine-tuned on author recognition task and their result were compared to a model trained from scratch. Also it’s shown how created models can be applied in semantic search. This work demonstrates that Lithuanian language model can be created by transferring knowledge from pretrained English language model without large corpus of data.
Degree
thesis:*- Grantor dc:publisher
- Institutional Repository of Vilnius University
- Year dc:date
- 2020
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Grotuzas, Domantas,
- Contributors dc:contributor
-
- Petkevičius, Linas
Rights
dc:rights- Statement dc:rights
-
- info:eu-repo/semantics/openAccess
- Language dc:language
- lit
Identifiers
dc:identifier.*- Repository record dc:identifier
- https://repository.vu.lt/VU:ELABAETD81706922&prefLang=en_US