Back to search

Aristotle University Of Thessaloniki (AUTH)

Improving language model representations for biomedical relation extraction and low-resource languages

Abstract

dc:description

This thesis presents significant contributions to the fields of Natural Language Processing (NLP) and Machine Learning (ML) in three distinct areas. The thesis begins with an introductory part, comprised of two chapters, followed by three core chapters, each addressing specific research problems and providing novel solutions. The introductory part lays the groundwork for the entire thesis by introducing key concepts in ML and NLP. It covers the fundamentals of Machine Learning paradigms, including Supervised, Unsupervised, and Reinforcement Learning. Additionally, it delves into NLP concepts, such as Language Modeling (LM), Relation Extraction (RE), and the challenges of Low-Resource Languages (LRLs). The chapter also introduces critical architectural elements like the Transformer and the significance of Knowledge Graphs (KGs) in structured knowledge representation. The third chapter focuses on advancing the field of biomedical NLP by proposing novel Language Model (LM)-based techniques for Drug-Drug Interaction (DDI) extraction. This chapter addresses the complex task of identifying interactions between drugs in biomedical literature. It breaks down the DDI extraction process into two subtasks: classifying interactions based on given drug entities and extracting drug entities from text before classifying their interactions. The models developed in this chapter outperform existing state-of-the-art approaches in terms of performance, generalization capabilities, and attempt to provide interpretability. Benchmark datasets are used to validate their superiority experimentally. The fourth chapter explores methods for combining multiple sources of information or tasks within a single architecture, particularly in the context of low-resource languages and tasks in NLP. The chapter introduces novel LM-based techniques tailored for Human Values Identification and Affective Text Analysis (ATA) tasks. It presents innovative second-stage pre-training strategies and a unified framework with shared parameters for multi-task classification. The research includes the creation of a new low-resource language dataset and demonstrates the superior performance of the models in various problems through experimental evaluations. The fifth chapter delves into improving LM representations for low-resource languages, with a specific emphasis on the Greek language. The chapter introduces diverse Greek datasets and addresses challenges like imbalanced data in Aspect-Based Sentiment Analysis. It explores domain-adapted contrastive learning and multi-stage LM approaches to enhance the quality of LM representations. The research demonstrates substantial performance improvements across a range of classification tasks in the Greek language. In the concluding chapter, the thesis summarizes the key findings and contributions from each chapter. It also discusses potential directions for future research aligned with the objectives and insights gained throughout the thesis. This thesis aims to advance the fields of biomedical NLP, information synergy in low-resource settings, and LM representations for LRLs, with implications for healthcare, multilingual NLP, and low-resource language understanding.

Degree

thesis:*
Grantor dc:publisher
Aristotle University Of Thessaloniki (AUTH)
Year dc:date
2023

Author and committee

dc:creator, dc:contributor.*
Authors dc:creator
  • Zaikis, Dimitrios
  • Ζαΐκης, Δημήτριος

Subjects

dc:subject × 17

Rights

Language dc:language
eng

Identifiers

dc:identifier.*
Identifier
10.12681/eadd/55364

Chain of custody

source
Harvested from
Greek National Archive of PhD Theses
Base URL
phdtheses.ekt.gr/eadd_oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Zaikis, Dimitrios; Ζαΐκης, Δημήτριος. Improving language model representations for biomedical relation extraction and low-resource languages. Aristotle University Of Thessaloniki (AUTH), 2023. http://hdl.handle.net/10442/hedi/55364