University of Crete (UOC)
Combining structured knowledge and data-driven representations for object states classification
Abstract
dc:descriptionThis dissertation investigates the task of Object State Classification (OSC) addressing the challenges of recognizing object states in images. We integrate symbolic knowledge from Knowledge Graphs (KGs) with data-driven Graph Neural Networks (GNNs) to enhance OSC performance and generalization capabilities, particularly in zero-shot learning scenarios. Our core contribution is a novel approach to Object-agnostic Zero-Shot State Classification that leverages KGs to encode commonsense knowledge and employs GNNs to learn expressive embeddings capturing rich semantic information. Crucially, our method operates without relying on explicit object recognition, increasing robustness to appearance variations and applicability in scenarios where object recognition may be unreliable. Through extensive experiments on four benchmark datasets, we demonstrate our method's effectiveness in recognizing unseen state classes, achieving significant improvements over baseline approaches and state-of-the-art zero-shot learning techniques. We further investigate the use of Large Language Models (LLMs) to enhance KG construction and utilization for OSC, exploring their capacity to generate semantically rich corpora for training visual models and to construct more informative knowledge representations. Additionally, we explore metapath learning to capture complex semantic relationships in KGs, enhancing both performance and interpretability of the proposed method. In conclusion, this dissertation advances OSC through the development of novel methodologies that synergistically combine KGs, GNNs, and LLMs, the creation of new datasets, and the introduction of innovative research objectives. Our findings underscore the potential of neurosymbolic AI integration for achieving more robust, interpretable, and accurate OSC methods, laying the groundwork for future research in this domain.
Degree
thesis:*- Grantor dc:publisher
- University of Crete (UOC)
- Year dc:date
- 2025
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Gouidis, Filippos
- Γουΐδης, Φίλιππος
Subjects
dc:subject × 20- Ταξινόμηση Καταστάσεων Αντικειμένων
- Μηδενική Μάθηση
- Γράφοι γνώσης
- Νευρωνικά ∆ίκτυα Γράφων
- Νευροσυμβολική απαρτίωση
- Μεγάλα γλωσσικά μοντέλα
- Εκμάθηση Μεταμονοπατιών
- Object State Classification
- Zero Shot Learning
- Knowledge graphs
- Graph neural networks
- Neurosymbolic Integration
- Large language models
- Metapath Learning
- Επιστήμη Ηλεκτρονικών Υπολογιστών και Πληροφορική
- Φυσικές Επιστήμες
- Τεχνητή νοημοσύνη
- Computer and Information Sciences
- Natural Sciences
- Artificial Intelligence
Rights
- Language dc:language
- eng
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/58987