Back to search

Aristotle University Of Thessaloniki (AUTH)

Techniques for evolving label sets and interpretability in multi-label text classification

Abstract

dc:description

Natural Language Processing (NLP) is a field of Artificial Intelligence (AI), tasked with enabling machines to understand, interpret, and generate human language, bridging the gap between human communication and computational systems. It encompasses a wide range of techniques and algorithms that enable computers to process and analyze natural language data. Machine Learning (ML) is another field of AI, which facilitates machines to learn patterns from data. ML plays a major role in the advancement of technology and its use in a plethora of domains. One of the most fundamental tasks of NLP and ML is text classification, which involves automatically assigning predefined categories or labels to textual data and plays a significant role in organizing, analyzing, and extracting insights from large volumes of textual data. Text classification finds use in a plethora of domains, with one of the most prominent ones being the biomedical domain, where it facilitates the classification of medical documents, clinical notes or research articles, into fine-grained categories, something that would otherwise require expert supervision, ultimately aiding not only in the treatment outcomes of patients but also in the advancement of the biomedical field. Despite its prominence text classification also faces significant challenges. Notably, training effective models to classify texts from evolving domains is a not a straightfor- ward task, especially when this evolution causes changes in the label space or shifts the relationships between the input and the output space of the data. Additionally, in Multi Label Learning (MLL) scenarios where labels are closely associated with each other, and therefore changes in one label may indirectly affect other ones related to it, designing techniques for effectively detecting and dealing with these changes becomes even more strenuous. To exacerbate these issues, new labels are frequently introduced in evolving domains, without annotated texts accompanying them, prompting data shortage issues as well. Aside from the complications arising from the evolving domains, another pronounced hurdle faced in text classification is interpretability. Most of the models achieving state-of-the-art performance in text classification tasks are notoriously uninterpretable, making it impossible to derive reasoning behind their decisions. Interpretability is crucial in certain domains such as biomedicine discussed earlier, since the decisions of models in these domains can directly affect human livelihood. The goal of this thesis is to introduce novel techniques to help alleviate those challenges. Specifically, for the biomedical domain and more precisely biomedical article classification, where the ever evolving Medical Subject Headings (MeSH) thesaurus is used we designed techniques to deal with the emergence of new labels or descriptors as they are called in the thesaurus, in addition to other more complex changes. With that goal in mind a Zero-Shot Learning (ZSL) approach was developed, making use of semantic similarity between biomedical articles and descriptors for which we have no train data, in order to perform classification. This approach made use of a state-of-the-art embedding method to obtain per sentence article representations in contrast to the n-gram based approaches commonly used for ZSL. MeSH, provides provenance information about each new descriptor introduced in the thesaurus, which can be used to aid the classification process. Utilizing this provenance information we introduced a Weakly-Supervised Learning (WSL) technique that creates a high quality weakly-supervised train set for new MeSH descriptors before training a typical MLL classification model. This technique leverages both the abstracts and titles of biomedical articles from previous iterations of MeSH and in conjunction to the provenance information of each new descriptor, assigns these new descriptors to the articles to create the weakly-supervised train set. We additionally, studied the evolution of the thesaurus outside the introduction of new descriptors, instead focusing on the how the concept drift phenomenon affects the vocabulary, as a consequence of the yearly MeSH revisions. Due to the multi-label nature of MeSH and the close relationships between descriptors in the tree-hierarchy of the thesaurus, changes in one descriptor can indirectly affect other ones related to it. We called this phenomenon indirect concept drift and performed a two-fold analysis, focusing on the performance changes of descriptors, in conjunction to shifts in their co-occurrence in order to detect it. Concerning interpretability in the context of text-classification, novel techniques were developed, to provide interpretations for the decision of transformer models, which are considered as state-of-the-art for the task of text-classification. In particular, the use of the readily available attention module present in transformer models as means of extracting interpretations was explored, leading to the creation of the Optimus family of techniques. The Optimus family selects the most faithful interpretation, among multiple attention-based ones, according to an unsupervised faithfulness-based metric. This selection, depending on the sub-method of the Optimus family, is performed per instance, per label per instance or for a set of instances. Optimus was found to provide more faithful interpretations compared to other state-of-the-art techniques, while also having lower execution time for certain techniques in the family. Additionally, a novel faithfulness-based metric was introduced to aid in the inter- pretability process of Optimus and to offer a more complete evaluation of feature- importance based interpretability techniques. This metric, contrary to the most well- known faithfulness-based metric, examines all the tokens of a textual input, when evaluating the interpretations provided by a feature-importance technique. Further- more, the importance of each token is evaluated not only based on its score, but in regards to its rank when compared to other tokens in the same input. This process of the metric brings its evaluation closer to supervised metrics, making it more suitable for interpretability evaluation. Besides the use of attention, keyphrase-based of interpretability of text classification was explored in this thesis, with the goal of providing smaller and more concise interpretations that are easier for the human reader to understand. Keyphrase- based interpretations have the additional advantage of taking into account word interactions when interpreting a model’s decision a topic that is scarcely explored in the literature. To that end, several state-of-the-art keyphrase extraction techniques were used, to provide the relevant keyphrases to an input text, before a scoring and a filtering procedure take place to provide the final keyphrases along with their feature-importance as interpretations. Finally, a method for interpreting dimensionality reduction techniques was devised, towards providing more interpretable document representations. The method in question, can interprete the dimensionality reduction process of any non-linear dimen- sionality reduction technique, and was found to be applicable in a plethora of different domains including topic representation and extreme multi-label learning, aiding in the generation of higher quality topics in the former and increased performance in the latter.

Degree

thesis:*
Grantor dc:publisher
Aristotle University Of Thessaloniki (AUTH)
Year dc:date
2024

Author and committee

dc:creator, dc:contributor.*
Authors dc:creator
  • Mylonas, Nikolaos
  • Μυλωνάς, Νικόλαος

Subjects

dc:subject × 16

Rights

Language dc:language
eng

Identifiers

dc:identifier.*
Identifier
10.12681/eadd/57175

Chain of custody

source
Harvested from
Greek National Archive of PhD Theses
Base URL
phdtheses.ekt.gr/eadd_oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Mylonas, Nikolaos; Μυλωνάς, Νικόλαος. Techniques for evolving label sets and interpretability in multi-label text classification. Aristotle University Of Thessaloniki (AUTH), 2024. http://hdl.handle.net/10442/hedi/57175