Democritus University of Thrace (DUTH)
Exploring the generalization capabilities of deep learning architectures for universal image feature learning
Abstract
dc:descriptionIn recent years, there has been a rapid growth in the field of Artificial Intelligence, capturing the attention not only of the scientific community but also the entire world. This surge is primarily attributed to the increasing volume of available data and the advancement of computer hardware, such as graphics cards and processors, enabling effective handling of this data. Since 2012, with the introduction of AlexNet by Alex Krizhevsky, demonstrating that neural networks could be trained exceptionally fast using graphics cards, a new era began in the field of Artificial Intelligence. This milestone paved the way for the development of thousands of techniques based on various architectures such as Fully Connected Networks (FNNs), Convolutional Neural Networks (CNNs), Long Short-Term Memory Networks (LSTMs), and recently developed Transformers and Vision Transformers. Particularly in the field of computer vision, the landscape has completely transformed in the last decade. The scientific community has predominantly turned to AI techniques for various applications, diminishing the use of classical methods. With the availability of open-source code and pre-trained networks, users can leverage a plethora of architectures trained on massive databases. While the progress of these techniques is remarkable, there are still significant challenges regarding the generalization and universality of the information extracted from them. These challenges mainly revolve around knowledge transfer, a popular and powerful method where a network trained on one dataset is applied to another dataset or application with similar or different data. Experimentally, networks perform exceptionally well when the data distribution or domain is similar. However, as the data or application diverges, the network's performance deviates from desired results. The goal of the doctoral research initiated in 2020 was to develop a powerful network capable of extracting high-quality and generalized features from any image. The foundation for this project was the use of pre-trained networks on much larger databases than ImageNet, as training from scratch was prohibitive due to computational resource constraints. The research conducted can be divided into two main parts that are interconnected. The first part involves the evaluation and development of deep learning techniques for image feature extraction. The evaluation of the extracted features utilized the Image Retrieval process from specific benchmark databases. This process aims to approximate human visual understanding concerning the similarity of images. The finer and more essential the features extracted by the network, the richer the information descriptor for each image. Initially, an extensive exploration and study were conducted on deep learning techniques for feature extraction, employing the most modern architectures for both CNNs and Vision Transformers pre-trained on the ImageNet database. In both cases, two Content-based Image Retrieval approaches were proposed. The first approach used the entire image from the pre-trained network to extract a global descriptor, while the second provided various parts of the image to extract multiple local descriptors. Additionally, various hyperparameters were tested, such as descriptor normalization, image size when entering the network, and various distance/similarity metrics. At the end of the first part, the system takes a step further. The objective is to create a universal encoder independent of the domain. In other words, the network should extract robust features regardless of the content of the image, even if those images have no relation to the images presented in the training dataset. In this case, the CLIP architecture was adopted, pre-trained on a much larger database than ImageNet. The network was retrained using a new approach that achieved the 6th position in Google's "Universal Image Embedding" competition. This approach combines various learning techniques and a plethora of data from different fields to approach the universality of extracted features. The second part of the thesis involves the application of deep learning models to real-world problems and how their generalization capabilities make them the preferred choice for vision applications. The first application combines LoRa packet transmission protocol with CNNs to identify vine diseases from images under conditions of limited bandwidth (significant packet/image loss). The second utilized CNNs as CBIR systems for automating the similarity process based on GESTALT for educational purposes, interacting with a humanoid robot NAO. The goal was to increase participants' attention and creativity through interaction.
Degree
thesis:*- Grantor dc:publisher
- Democritus University of Thrace (DUTH)
- Year dc:date
- 2024
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Gkelios, Sokratis
- Γκέλιος, Σωκράτης
Subjects
dc:subject × 16- Βαθιά μάθηση
- Καθολική μάθηση χαρακτηριστικών
- Ανάκτηση εικόνας
- Καθολικός κωδικοποιητής
- Γενίκευση
- Deep learning
- Universal feature learning
- Image retrieval
- Universal image encoder
- Generalization
- Επιστήμη Ηλεκτρονικών Υπολογιστών και Πληροφορική
- Φυσικές Επιστήμες
- Τεχνητή νοημοσύνη
- Computer and Information Sciences
- Natural Sciences
- Artificial Intelligence
Rights
- Language dc:language
- eng
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/56482