Abstract
dc:descriptionIn recent years, Deep Learning (DL) has made tremendous contributions to both academic research and industry, leading to state-of-the-art performance in a wide range of applications. However, DL models increasingly require more powerful hardware, raising concerns regarding their sustainability. To address the demanding nature of DL and meet the modern challenges, a growing effort has been made to develop advanced methodologies for both efficient training and inference, as well as for designing specialized hardware targeting increased speed and reduced energy consumption. Motivated by the fact that the advent of DL became feasible only when the appropriate hardware was designed to support the emerging needs arising from developed methodologies, this Thesis attempts to establish a more holistic approach to DL efficiency, incorporating software-hardware co-design. Thus, it aims to improve the efficiency of hardware, training, and inference, taking into account cutting-edge photonic hardware architectures and making them practically deployable. To this end, it introduces methodologies ranging from noise-aware and optics-informed training methods to advanced quantization-aware training approaches for both uniform and mixed-precision low-bit resolution neural networks. More specifically, the proposed hardware-aware training method takes into consideration the limitations that arise from the unique nature of optical components, including noise sources and highly saturated transfer functions, it introduces an adaptive data-driven initialization method. Then, the novel quantization-aware method reduces quantization error during inference, which enables lower bit resolution models without significant performance degradation, by quantizing parameters based on their statistical distribution. In turn, the proposed mixed precision quantization-aware training method formalizes a stochastic reduction in bit resolution for each layer based on its position, progressively lowering the bit resolution of layers that are more tolerant to quantization error. Furthermore, the Thesis introduces a novel isomorphic non-negative transformation that allows the transformation of any regularly trained model into its non-negative equivalent, ensuring that all parameters involved in the decision of the network are non-negative. In this way, the proposed isomorphic transformation unlocks the capability to perform non-negative inference without any performance degradation in a wide range of neural network architectures. In turn, the Thesis provides evidence for improved explainability and hardware integration when non-negative neural networks are applied, while proposing a generic optimization framework that enables the introduction of a novel multiplicative update rule. The multiplicative update rule can be integrated into any typical DL optimization method enabling non-negative training, robust inference and accelerated convergence. As demonstrated, the proposed methodologies unlock numerous applications that require high-speed inference and low energy consumption and enables the effective experimental use of optical hardware layouts in DL accelerators.
Degree
thesis:*- Grantor dc:publisher
- Aristotle University Of Thessaloniki (AUTH)
- Year dc:date
- 2025
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Kirtas, Emmanouil
- Κίρτας, Εμμανουήλ
Subjects
dc:subject × 16- Βαθιά μάθηση
- Νευρωνικά δίκτυα
- Αποδοτική Βαθιά Μάθηση
- Φωτονικά Νευρωνικά Δίκτυα
- Νευρομορφικά Φωτονικά
- Deep learning
- Artificial neural networks
- Efficient deep learning
- Photonic neuromorphic computing
- Photonic Neural Networks
- Επιστήμη Ηλεκτρονικών Υπολογιστών και Πληροφορική
- Φυσικές Επιστήμες
- Τεχνητή νοημοσύνη
- Computer and Information Sciences
- Natural Sciences
- Artificial Intelligence
Rights
- Language dc:language
- eng
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/59495