Institutes outside Greece
Artistic and photorealistic image style transfer using deep learning
Abstract
dc:descriptionStyle transfer is a popular technique in computer vision and image processing that involves generating an image that combines the content of one image with the style of another image or of a collection of images. In this thesis we address the two style transfer types of this problem namely artistic and photorealistic, which have found applications in a variety of fields, including video game design, mobile application development, and architecture. Artistic style transfer involves the complex challenge of transferring the style of a collection of images to a given image. Our proposed method involves an extension of the Cycle-Consistent Adversarial Network architecture that incorporates Fast and Adaptive Bidimensional Empirical Mode Decomposition (FABEMD) in the optimization problem of loss minimisation. This modification allows for the analysis of images into their key texture elements and provides a more qualitative definition of the loss function. We also propose an adaptive selection mechanism that involves different amounts of spectral information (Bidimensional Intrinsic Mode Functions), depending on the specific characteristics of the image being stylised. Contrary to existing approaches, our methods transfer the style effectively and efficiently without content distortion. Similar to the artistic, photorealistic style transfer has been defined as the process of transferring the style of one image to another, while maintaining visual realism. A popular application of photorealistic style transfer is re-creating a low-light image, aiming to generate the same content image with normal light conditions. However, existing photorealistic style transfer methods often suffer from spatial distortions, unrealistic artifacts, or underrepresented colours. This limitation hinders the use of style transfer in applications that require photorealistic results, such as image enhancement in low-light conditions. To address this problem, we propose a novel approach to photorealistic style transfer that involves a modified U-Net-based architecture with dense blocks and wavelet-based pooling. This architecture is designed to better preserve texture regions in image enhancement and has been shown to be effective and robust in comparison to other state-of-the-art style transfer methods. Finally, we propose a late fusion approach of two parallel UNet architectures, namely HDD-Unet, to further enrich the colourisation of the final generated image. To evaluate our proposed methods, for both artistic and photorealistic style transfer problem, we compare quantitatively and qualitatively our methods to other state of-the-art techniques and demonstrate their effectiveness in generating images while preserving important visual information.
Degree
thesis:*- Grantor dc:publisher
- Institutes outside Greece
- Year dc:date
- 2023
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Batziou, Elissavet
- Μπάτζιου, Ελισσάβετ
Subjects
dc:subject × 20- Μεταφορά στυλ
- Επεξεργασία εικόνας
- Καλλιτεχνικό στυλ
- Φωτορεαλιστικό
- Νευρωνικά δίκτυα
- Βαθιά μάθηση
- Υπολογιστική όραση και επεξεργασία εικόνας
- Style transfer
- Image processing
- Artistic style transfer
- Photorealistic style transfer
- Neural networks
- Deep learning
- Computer vision and image processing
- Επιστήμη Ηλεκτρολόγου Μηχανικού, Ηλεκτρονικού Μηχανικού, Μηχανικού Η/Υ
- Επιστήμες Μηχανικού και Τεχνολογία
- Ηλεκτρική και Ηλεκτρονική μηχανική
- Electrical Engineering, Electronic Engineering, Information Engineering
- Engineering and Technology
- Electrical and Electronic Engineering
Rights
- Language dc:language
- eng
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/58022