National Technical University of Athens (NTUA)
Δημιουργία αντιπαραδειγμάτων και εμπλουτισμός πολυτροπικών συστημάτων μέσω εξωτερικής γνώσης
Abstract
dc:descriptionIn recent years, natural language models and by extension multimodal systems havedemonstrated remarkable success in a wide range of tasks, exploiting common representations of visual and textualinputs. Despite often impressive results, these models are opaque, making the interpretation ofdecision making quite fuzzy, while their potential in specific tasks is limited,leading to limited generalizability. These two limitations can be resolved by exploiting externalknowledge sources, which traditionally include knowledge graphs, and more recently are based on largelanguage models. This thesis examines the contribution of external knowledge in addressing theaforementioned constraints by exploiting counterexamples: replacing concepts in textual descriptions withwell-defined ones can reveal the decision making of models without the need topenetrate inside them. Specifically, we propose the application of graph-drivenknowledge counterexamples to a variety of models involving both image retrieval from text, image generation from textand the generation of textual descriptions from images. In addition, the use of large linguistic models asknowledge bases provides enrichment of their existing knowledge, allowing them to perform new tasks with betterresults. In this thesis, knowledge enrichment is performed on multimodal models for retrievingimages from text, demonstrating the functionality of the method. Finally, an analysis of the limitations of usinglarge language models as knowledge bases in terms of reasoning abilities and incorrect generation ofanswers is provided.
Degree
thesis:*- Grantor dc:publisher
- National Technical University of Athens (NTUA)
- Year dc:date
- 2025
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Lymperaiou, Maria
- Λυμπεραίου, Μαρία
Subjects
dc:subject × 16- Επεξηγήσιμη τεχνητή νοημοσύνη
- Επεξηγήσεις μέσω Αντιπαραδειγμάτων
- Πολυτροπικά δεδομένα
- Βάσεις γνώσεων
- Μεγάλα γλωσσικά μοντέλα
- Explainable artificial intelligence
- Counterfactual Explanations
- Multimodal data
- Knowledge bases
- Large language models
- Επιστήμη Ηλεκτρολόγου Μηχανικού, Ηλεκτρονικού Μηχανικού, Μηχανικού Η/Υ
- Επιστήμες Μηχανικού και Τεχνολογία
- Υπολογιστές, Υλικό (hardware) και Αρχιτεκτονική
- Electrical Engineering, Electronic Engineering, Information Engineering
- Engineering and Technology
- Computer science, Hardware and Architecture
Rights
- Language dc:language
- eng
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/60505