University of Crete (UOC)
Multi-domain inference and generative modeling for hand image synthesis
Abstract
dc:descriptionThe exponential growth of computer vision methodologies, particularly deep learning architectures, has significantly increased the demand for high-quality data across multiple domains, to serve as input or output modalities. While synthetic data generation has seen numerous developments in many areas, unfortunately the same does not apply to the field of human hand-related tasks. The few available synthesis approaches fail to achieve satisfactory realism and multi-domain variety. This dissertation aims to address this gap by proposing novel methodologies for hand image synthesis in the RGB and depth domains, focusing specifically on depth estimation, efficient probabilistic generative modeling, and controllable data generation. To tackle the hand depth estimation problem, we aim at balancing accuracy and computational complexity. In that direction, we present a lightweight model based on the stacked-hourglass architecture for monocular RGB-to-depth estimation of hand images. Employing intermediate supervision and a staged learning approach, the proposed model achieves an accuracy of 22mm that other general-purpose depth estimating methods cannot achieve. To facilitate training and evaluation, we introduce HandRGBD, a new dataset of over 20,000 aligned hand image pairs of RGB and depth. With an accuracy comparable to that of low-cost depth cameras, this work bridges the gap between RGB and RGBD domains, enabling RGBD-based hand pose estimation methods to be applicable to RGB input. To enhance generative methods with the imminent goal of enabling more accurate and controllable generation of images, we require direct manipulation of the latent representation of our data. Variational Autoencoders (VAE) possess that property. For this reason, we augment this family of models by introducing Random Variable Variational Autoencoders (RV-VAE), a novel formulation that treats artificial neural network activations as continuous Random Variables (RV). We achieve this by integrating specially designed Artificial Neural Network modules, that are applied to RV operands using rules defined by the algebra of random variables. By optimizing over full distributions in the latent space rather than discrete samples, RV-VAE incorporate mathematical priors, reducing computational load and improving reconstruction and generative performance. Finally, we propose the Supervised Random Variable Variational Autoencoder (SRV-VAE), a novel framework for generating photorealistic hand images conditioned on precise pose annotations. The SRV-VAE manages to disentangle the pose and appearance of the input hand image, ensuring robust control over synthesis while maintaining realism during inference. The model demonstrates efficacy in generating diverse hand images by utilizing the capabilities of RV-aware modalities. Moreover, using this framework, we are able to augment and diversify hand training sets with the purpose of enhancing pose estimation tasks. Collectively, these contributions establish a foundation for advancing hand-related tasks in computer vision by offering solutions to multi-domain hand image synthesis. The presented methodologies bridge critical gaps in data generation and have the potential to meet the growing demand for high-quality hand data while addressing challenges in efficiency and realism, laying the foundation for future advancements in generative systems and hand analysis.
Degree
thesis:*- Grantor dc:publisher
- University of Crete (UOC)
- Year dc:date
- 2025
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Nikodimou, Vassilios-Clitos
- Νικοδήμου, Βασίλειος-Κλείτος
Subjects
dc:subject × 24- Συνθετικά Δεδομένα
- Εκτίμηση βάθους χεριού
- Πιθανολογική μοντελοποίηση
- Τυχαίες μεταβλητές
- Μοντελοποίηση υπό όρους
- ∆ιαχωρισμός δεδομένων χεριού
- Synthetic data
- Hand depth estimation
- Probabilistic generative modeling
- Random variables
- Conditional generative modeling
- Hand data disentanglement
- HandRGBD
- VAE
- RV-VAE
- SRV-VAE
- Επιστήμη Ηλεκτρονικών Υπολογιστών και Πληροφορική
- Φυσικές Επιστήμες
- Τεχνητή νοημοσύνη
- Επιστήμη ηλεκτρονικών υπολογιστών και Πληροφορική, άλλοι τομείς
- Computer and Information Sciences
- Natural Sciences
- Artificial Intelligence
- Computer and Information sciences, miscellaneous
Rights
- Language dc:language
- eng
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/58574