Global ETD Search

1	Extractive Text Summarization of Norwegian News Articles Using BERT Biniam, Thomas Indrias, Morén, Adam January 2021 (has links) Extractive text summarization has over the years been an important research area in Natural Language Processing. Numerous methods have been proposed for extracting information from text documents. Recent works have shown great success for English summarization tasks by fine-tuning the language model BERT using large summarization datasets. However, less research has been made for low-resource languages. This work contributes by investigating how BERT can be used for Norwegian text summarization. Two models are developed by applying a modified BERT architecture, called BERTSum, on pre-trained Norwegian and Multilingual BERT. The results are models able to predict key sentences from articles to generate bullet-point summaries. These models are evaluated with the automatic metric ROUGE and in this evaluation, the Multilingual BERT model outperforms the Norwegian model. The multilingual model is further evaluated in a human evaluation by journalists, revealing that the generated summaries are not entirely satisfactory in some aspects. With some improvements, the model shows to be a valuable tool for journalists to edit and rewrite generated summaries, saving time and workload. / <p>Examensarbetet är utfört vid Institutionen för teknik och naturvetenskap (ITN) vid Tekniska fakulteten, Linköpings universitet</p> extractive text summarization NLP deep learning BERT BERTSum Multilingual BERT Norwegian BERT transformer Norwegian news articles Computer Sciences Datavetenskap (datalogi)
2	Large-Context Question Answering with Cross-Lingual Transfer Sagen, Markus January 2021 (has links) Models based around the transformer architecture have become one of the most prominent for solving a multitude of natural language processing (NLP)tasks since its introduction in 2017. However, much research related to the transformer model has focused primarily on achieving high performance and many problems remain unsolved. Two of the most prominent currently are the lack of high performing non-English pre-trained models, and the limited number of words most trained models can incorporate for their context. Solving these problems would make NLP models more suitable for real-world applications, improving information retrieval, reading comprehension, and more. All previous research has focused on incorporating long-context for English language models. This thesis investigates the cross-lingual transferability between languages when only training for long-context in English. Training long-context models in English only could make long-context in low-resource languages, such as Swedish, more accessible since it is hard to find such data in most languages and costly to train for each language. This could become an efficient method for creating long-context models in other languages without the need for such data in all languages or pre-training from scratch. We extend the models’ context using the training scheme of the Longformer architecture and fine-tune on a question-answering task in several languages. Our evaluation could not satisfactorily confirm nor deny if transferring long-term context is possible for low-resource languages. We believe that using datasets that require long-context reasoning, such as a multilingual TriviaQAdataset, could demonstrate our hypothesis’s validity. Long-Context Multilingual Model Longformer XLM-R Longformer Long-term Context Extending Context Extend Context Large-Context Long-Context Large Context Long Context Cross-Lingual Multi-Lingual Cross Lingual Multi Lingual QA Question-Answering Question Answering Transformer model Machine Learning Transfer Learning SQuAD Memory Transfer Learning Long-Context Long Context Efficient Monolingual Multilingual QA model Language Model Huggingface BERT RoBERTa XLM-R mBERT Multilingual BERT Efficient Transformers Reformer Linformer Performer Transformer-XL Wikitext-103 TriviaQA HotpotQA WikiHopQA VINNOVA Peltarion AI LM MLM Deep Learning Natural Language Processing NLP Attention Transformers Transfer Learning Datasets Computer and Information Sciences Data- och informationsvetenskap

Search results

Extractive Text Summarization of Norwegian News Articles Using BERT

Large-Context Question Answering with Cross-Lingual Transfer