This research presents an innovative approach to extracting information from Positive Energy Districts (PEDs), urban areas generating surplus energy. PEDs are integral to the European Commission's SET Plan, tackling housing challenges arising from population growth. The study refines BERT to categorize PED-related entities, producing a cutting-edge NER model and an integrated pipeline of diverse NER tools and data sources. The model achieves an accuracy of 0.81 and an F1 Score of 0.55 with notably high confidence scores through pipeline evaluations, confirming its practical applicability. While the F1 score falls short of expectations, this pioneering exploration in PED information extraction sets the stage for future refinements and studies, promising enhanced methodologies and impactful outcomes in this dynamic field. This research advances NER processes for Positive Energy Districts, supporting their development and implementation.

The recent advancements of Natural Language Processing have cleared the path for many new applications. This is primarily a consequence of the transformer model and the transfer-learning capabilities provided by models like BERT. However, task-specific labeled data is required to fine-tune these models. To alleviate the expensive process of labeling data, Active Learning (AL) aims to maximize the information gained from each label. By including a model in the annotation process, the informativeness of each unlabeled sample can be estimated and hence allow human annotators to focus on vital samples and avoid redundancy. This thesis investigates to what extent AL can accelerate model training with respect to the number of labels required. In particular, the focus is on pre- trained Swedish language models in the context of Named Entity Recognition. The data annotation process is simulated using existing labeled datasets to evaluate multiple AL strategies. Experiments are evaluated by analyzing the F1 score achieved by models trained on the data selected by each strategy. The results show that AL can significantly accelerate the model training and hence reduce the manual annotation effort. The state-of-the-art strategy for sentence classification, ALPS, shows no sign of accelerating the model training. However, uncertainty-based strategies consistently outperform random selection. Under certain conditions, these strategies can reduce the number of labels required by more than a factor of two. / Framstegen som nyligen har gjorts inom naturlig språkbehandling har möjliggjort många nya applikationer. Det är mestadels till följd av transformer-modellerna och lärandeöverföringsmöjligheterna som kommer med modeller som BERT. Däremot behövs det fortfarande uppgiftsspecifik annoterad data för att finjustera dessa modeller. För att lindra den dyra processen att annotera data, strävar aktiv inlärning efter att maximera informationen som utvinns i varje annotering. Genom att inkludera modellen i annoteringsprocessen, kan man estimera hur informationsrikt varje träningsexempel är, och på så sätt låta mänskilga annoterare fokusera på viktiga datapunkter. Detta examensarbete utforskar hur väl aktiv inlärning kan accelerera modellträningen med avseende på hur många annoterade träningsexempel som behövs. Fokus ligger på förtränade svenska språkmodeller och uppgiften namnigenkänning. Dataannoteringsprocessen simuleras med färdigannoterade dataset för att evaluera flera olika strategier för aktiv inlärning. Experimenten evalueras genom att analysera den uppnådda F1-poängen av modeller som är tränade på datapunkterna som varje strategi har valt. Resultaten visar att aktiv inlärning har en signifikant förmåga att accelerera modellträningen och reducera de manuella annoteringskostnaderna. Den toppmoderna strategin för meningsklassificering, ALPS, visar inget tecken på att kunna accelerera modellträningen. Däremot är osäkerhetsbaserade strategier är konsekvent bättre än att slumpmässigt välja datapunkter. I vissa förhållanden kan dessa strategier reducera antalet annoteringar med mer än en faktor 2.

Named entity recognition (NER) is a task that concerns detecting and categorising certain information in text. A promising approach for NER that recently has emerged is fine-tuning Transformer-based language models for this specific task. However, these models may require a relatively large quantity of labelled data to perform well. This can limit NER models applicability in real-world applications as manual annotation often is costly and time-consuming. In this thesis, we investigate the learning curve of human annotation and of a NER model during a semi-supervised bootstrapping process. Special emphasis is given the dependence of the number of classes and the amount of training data used in the process. We first annotate a set of collected job advertisements and then apply bootstrapping using both annotated and unannotated data and continuously fine-tune a pre-trained Swedish BERT model. The initial class system is simplified during the bootstrapping process according to model performance and inter-annotator agreement. The model performance increased as the training set grew larger with a final micro F1-score of 54%. This result provides a good baseline, and we point out several improvements that can be made to further enhance performance. We further identify classes handled differently by the annotators and potential factors as to why this is. Suggestions for future work include adjusting the current class system further by removing classes that were identified as low-performing in this thesis. / Namngiven entitetsigenkänning (eng. named entity recognition) innebär att identifiera och kategorisera nyckelord i text. En ny lovande teknik för identifiering av namngivna enheter är att finjustera Transformerbaserade språkmodeller för denna specifika uppgift. Dessa modeller kräver dock stora mängder märkt data för att prestera väl. Detta kan begränsa antal områden i vilka de kan användas då manuell märkning av data ofta är kostsamt och tidskrävande. I denna avhandling undersöker vi inlärningskurvan för manuell annotering och för en språkmodell under en halvövervakad bootstrapping process. Särskild vikt läggs på hur modellens och annoterarnas inlärning påverkas av antal klasser och mängden träningsdata som används i processen. Vi annoterar först en samling jobbannonser och tillämpar sedan en bootstrapping process med både märkt och omärkt data i vilken en förtränad svensk BERT-modell kontinuerligt finjusteras. Det första klasssystemet förenklas under processens gång beroende på modellprestation och interannoterar-överenskommelse. Modellen presterade bättre med mer träningsdata och uppnådde en slutlig micro F1-score på 54%. Detta resultat ger en bra baslinje, och vi föreslår flera förbättringar som kan göras för att ytterligare förbättra modellprestationen. Vidare identifierar vi även klasser som hanteras olika av annoterare och potentiella faktorer till vad detta beror på. Förslag för framtida arbete inkluderar att justera det nuvarande klasssystemet ytterligare genom att ta bort klasser som identifierades som lågpresterande i denna avhandling.

No description available.

To label words of interest into a predefined set of named entities have traditionally required a large amount of labeled in-domain data. Recently, the availability of pre-trained transformer-based language models have enabled multiple natural language processing problems to utilize transfer learning techniques to construct machine learning models with less task-specific labeled data. In this thesis, the impact of data augmentation when training a pre-trained transformer-based model to adapt to a named entity recognition task with few labeled sentences is explored. The experimental results indicate that data augmentation increases performance of the trained models, however the data augmentation is shown to have less impact when more labeled data is available. In conclusion, data augmentation has been shown to improve performance of pre-trained named entity recognition models when few labeled sentences are available for training. / Att kategorisera ord som tillhör någon av en mängd förangivna entiteter har traditionellt krävt stora mängder förkategoriserad områdesspecifik data. På senare år har det tillgängliggjorts förtränade språkmodeller som möjliggjort för språkprocesseringsproblem att lösas med en mindre mängd områdesspecifik kategoriserad data. I den här uppsatsen utforskas datautöknings påverkan på en maskininlärningsmodell för identifiering av namngivna entiteter. De experimentella resultaten indikerar att datautökning förbättrar modellerna, men att inverkan blir mindre när mer kategoriserad data är tillgänglig. Sammanfattningsvis så kan datautökning förbättra modeller för identifiering av namngivna entiteter när få förkategoriserade meningar finns tillgängliga för träning.

Das Finden von zuverlässigen Informationen über gegebene Ereignisse aus großen und dynamischen Textsammlungen, wie dem Web, ist ein wichtiges Thema. Zum Beispiel sind Rettungsteams und Versicherungsunternehmen an prägnanten Fakten über Schäden nach Katastrophen interessiert, die heutzutage online in Web-Blogs, Zeitungsartikeln, Social Media etc. zu finden sind. Solche Fakten helfen, die erforderlichen Hilfsmaßnahmen zu bestimmen und unterstützen deren Koordination. Allerdings ist das Finden, Extrahieren und Aggregieren nützlicher Informationen ein hochkomplexes Unterfangen: Es erfordert die Ermittlung geeigneter Textquellen und deren zeitliche Einordung, die Extraktion relevanter Fakten in diesen Texten und deren Aggregation zu einer verdichteten Sicht auf die Ereignisse, trotz Inkonsistenzen, vagen Angaben und Veränderungen über die Zeit. In dieser Arbeit präsentieren und evaluieren wir Techniken und Lösungen für jedes dieser Probleme, eingebettet in ein vierstufiges Framework. Die angewandten Methoden beruhen auf Verfahren des Musterabgleichs, der Verarbeitung natürlicher Sprache und des maschinellen Lernens. Zusätzlich berichten wir über die Ergebnisse zweier Fallstudien, basierend auf dem Einsatz des gesamten Frameworks: Die Ermittlung von Daten über Erdbeben und Überschwemmungen aus Webdokumenten. Unsere Ergebnisse zeigen, dass es unter bestimmten Umständen möglich ist, automatisch zuverlässige und zeitgerechte Daten aus dem Internet zu erhalten. / Finding reliable information about given events from large and dynamic text collections, such as the web, is a topic of great interest. For instance, rescue teams and insurance companies are interested in concise facts about damages after disasters, which can be found today in web blogs, online newspaper articles, social media, etc. Knowing these facts helps to determine the required scale of relief operations and supports their coordination. However, finding, extracting, and condensing specific facts is a highly complex undertaking: It requires identifying appropriate textual sources and their temporal alignment, recognizing relevant facts within these texts, and aggregating extracted facts into a condensed answer despite inconsistencies, uncertainty, and changes over time. In this thesis, we present and evaluate techniques and solutions for each of these problems, embedded in a four-step framework. Applied methods are pattern matching, natural language processing, and machine learning. We also report the results for two case studies applying our entire framework: gathering data on earthquakes and floods from web documents. Our results show that it is, under certain circumstances, possible to automatically obtain reliable and timely data from the web.

Contemporary approaches to natural language processing are predominantly based on statistical machine learning from large amounts of text, which has been manually annotated with the linguistic structure of interest. However, such complete supervision is currently only available for the world's major languages, in a limited number of domains and for a limited range of tasks. As an alternative, this dissertation considers methods for linguistic structure prediction that can make use of incomplete and cross-lingual supervision, with the prospect of making linguistic processing tools more widely available at a lower cost. An overarching theme of this work is the use of structured discriminative latent variable models for learning with indirect and ambiguous supervision; as instantiated, these models admit rich model features while retaining efficient learning and inference properties. The first contribution to this end is a latent-variable model for fine-grained sentiment analysis with coarse-grained indirect supervision. The second is a model for cross-lingual word-cluster induction and the application thereof to cross-lingual model transfer. The third is a method for adapting multi-source discriminative cross-lingual transfer models to target languages, by means of typologically informed selective parameter sharing. The fourth is an ambiguity-aware self- and ensemble-training algorithm, which is applied to target language adaptation and relexicalization of delexicalized cross-lingual transfer parsers. The fifth is a set of sequence-labeling models that combine constraints at the level of tokens and types, and an instantiation of these models for part-of-speech tagging with incomplete cross-lingual and crowdsourced supervision. In addition to these contributions, comprehensive overviews are provided of structured prediction with no or incomplete supervision, as well as of learning in the multilingual and cross-lingual settings. Through careful empirical evaluation, it is established that the proposed methods can be used to create substantially more accurate tools for linguistic processing, compared to both unsupervised methods and to recently proposed cross-lingual methods. The empirical support for this claim is particularly strong in the latter case; our models for syntactic dependency parsing and part-of-speech tagging achieve the hitherto best published results for a wide number of target languages, in the setting where no annotated training data is available in the target language.

Cette thèse présente une approche d'analyse des textes non-standardisé qui consiste à modéliser une chaine de traitement permettant l’annotation automatique de textes à savoir l’annotation grammaticale en utilisant une méthode d’étiquetage morphosyntaxique et l’annotation sémantique en mettant en œuvre un système de reconnaissance des entités nommées. Dans ce contexte, nous présentons un système d'analyse du Moyen Français qui est une langue en pleine évolution dont l’orthographe, le système flexionnel et la syntaxe ne sont pas stables. Les textes en Moyen Français se singularisent principalement par l’absence d’orthographe normalisée et par la variabilité tant géographique que chronologique des lexiques médiévaux.L’objectif est de mettre en évidence un système dédié à la construction de ressources linguistiques, notamment la construction des dictionnaires électroniques, se basant sur des règles de morphologie. Ensuite, nous présenterons les instructions que nous avons établies pour construire un étiqueteur morphosyntaxique qui vise à produire automatiquement des analyses contextuelles à l’aide de grammaires de désambiguïsation. Finalement, nous retracerons le chemin qui nous a conduits à mettre en place des grammaires locales permettant de retrouver les entités nommées. De ce fait, nous avons été amenés à constituer un corpus MEDITEXT regroupant des textes en Moyen Français apparus entre le fin du XIIIème et XVème siècle. / This thesis presents a non-standardized text analysis approach which consists a chain process modeling allowing the automatic annotation of texts: grammar annotation using a morphosyntactic tagging method and semantic annotation by putting in operates a system of named-entity recognition. In this context, we present a system analysis of the Middle French which is a language in the course of evolution including: spelling, the flexional system and the syntax are not stable. The texts in Middle French are mainly distinguished by the absence of normalized orthography and the geographical and chronological variability of medieval lexicons.The main objective is to highlight a system dedicated to the construction of linguistic resources, in particular the construction of electronic dictionaries, based on rules of morphology. Then, we will present the instructions that we have carried out to construct a morphosyntactic tagging which aims at automatically producing contextual analyzes using the disambiguation grammars. Finally, we will retrace the path that led us to set up local grammars to find the named entities. Hence, we were asked to create a MEDITEXT corpus of texts in Middle French between the end of the thirteenth and fifteenth centuries.

O objetivo deste trabalho é desenvolver uma plataforma para anotação e classificação automática de entidades nomeadas para notícias escritas em português do Brasil. Para restringir um pouco o escopo do treinamento e análise foram utilizadas notícias esportivas do Campeonato Paulista de 2011 do portal UOL (Universo Online). O primeiro artefato desenvolvido desta plataforma foi a ferramenta WebCorpus. Esta tem como principal intuito facilitar o processo de adição de metainformações a palavras através do uso de uma interface rica web, elaborada para deixar o trabalho ágil e simples. Desta forma as entidades nomeadas das notícias são anotadas e classificadas manualmente. A base de dados foi alimentada pela ferramenta de aquisição e extração de conteúdo desenvolvida também para esta plataforma. O segundo artefato desenvolvido foi o córpus UOLCP2011 (UOL Campeonato Paulista 2011). Este córpus foi anotado e classificado manualmente através do uso da ferramenta WebCorpus utilizando sete tipos de entidades: pessoa, lugar, organização, time, campeonato, estádio e torcida. Para o desenvolvimento do motor de anotação e classificação automática de entidades nomeadas foram utilizadas três diferentes técnicas: maximização de entropia, índices invertidos e métodos de mesclagem das duas técnicas anteriores. Para cada uma destas foram executados três passos: desenvolvimento do algoritmo, treinamento utilizando técnicas de aprendizado de máquina e análise dos melhores resultados. / The main target of this research is to develop an automatic named entity classification tool to sport news written in Brazilian Portuguese. To reduce this scope, during training and analysis only sport news about São Paulo Championship of 2011 written by UOL2 (Universo Online) was used. The first artefact developed was the WebCorpus tool, which aims to make easier the process of add meta informations to words, through a rich web interface. Using this, all the corpora news are tagged manually. The database used by this tool was fed by the crawler tool, also developed during this research. The second artefact developed was the corpora UOLCP2011 (UOL Campeonato Paulista 2011). This corpora was manually tagged using the WebCorpus tool. During this process, seven classification concepts were used: person, place, organization, team, championship, stadium and fans. To develop the automatic named entity classification tool, three different approaches were analysed: maximum entropy, inverted index and merge tecniques using both. Each approach had three steps: algorithm development, training using machine learning tecniques and best score analysis.

A family of Natural Language Processing (NLP) tasks such as part-of- speech (PoS) tagging, Named Entity Recognition (NER), and Multiword Expression (MWE) identification all involve assigning labels to sequences of words in text (sequence labeling). Most modern machine learning approaches to sequence labeling utilize word embeddings, learned representations of text, in which words with similar meanings have similar representations. Quite recently, contextualized word embeddings have garnered much attention because, unlike pretrained context- insensitive embeddings such as word2vec, they are able to capture word meaning in context. In this thesis, I evaluate the performance of different embedding setups (context-sensitive, context-insensitive word, as well as task-specific word, character, lemma, and PoS) on the three abovementioned sequence labeling tasks using a deep learning model (BiLSTM) and Portuguese datasets. v