This thesis was done in collaboration with Ericsson AB with the goal of researching the possibility of creating a machine learning model that can transfer the style of a text into another arbitrary style depending on the data used. This had the purpose of making their technical documentation appear to have been written with one cohesive style for a better reading experience. Two approaches to solve this task were tested, the first one was to implement an encoder-decoder model from scratch, and the second was to use the pre-trained GPT-2 model created by a team from OpenAI and fine-tune the model on the specific task. Both of these models were trained on data provided by Ericsson, sentences were extracted from their documentation. To evaluate the models training loss, test sentences, and BLEU scores were used and these were compared to each other and with other state-of-the-art models. The models did not succeed in transforming text into a general technical documentation style but a good understanding of what would need to be improved and adjusted to improve the results were obtained. / <p>This thesis was presented on June 22, 2021, the presentation was done online on Microsoft teams. </p>

Humans have always used sketches to explain the visual world. It is a simple and straight- forward mean to communicate new ideas and designs. Consequently, as in almost every aspect of our modern life, the relatively recent major developments in computer science have highly contributed to enhancing individual sketching experience. The literature of sketch related research has witnessed seminal advancements and a large body of interest- ing work. Following up with this rich literature, this dissertation provides a holistic study on sketches through three proposed novel models including sketch analysis, transfer, and geometric representation. The first part of the dissertation targets sketch authorship recognition and analysis of sketches. It provides answers to the following questions: Are simple strokes unique to the artist or designer who renders them? If so, can this idea be used to identify authorship or to classify artistic drawings? The proposed stroke authorship recognition approach is a novel method that distinguishes the authorship of 2D digitized drawings. This method converts a drawing into a histogram of stroke attributes that is discriminative of authorship. Extensive classification experiments on a large variety of datasets are conducted to validate the ability of the proposed techniques to distinguish unique authorship of artists and designers. The second part of the dissertation is concerned with sketch style transfer from one free- hand drawing to another. The proposed method exploits techniques from multi-disciplinary areas including geometrical modeling and image processing. It consists of two methods of transfer: stroke-style and brush-style transfer. (1) Stroke-style transfer aims to transfer the style of the input sketch at the stroke level to the style encountered in other sketches by other artists. This is done by modifying all the parametric stroke segments in the input, so as to minimize a global stroke-level distance between the input and target styles. (2) Brush-style transfer, on the other hand, focuses on transferring a unique brush look of a line drawing to the input sketch. In this transfer stage, we use an automatically constructed input brush dictionary to infer which sparse set of input brush elements are used at each location of the input sketch. Then, a one-to-one mapping between input and target brush elements is learned by sparsely encoding the target sketch with the input brush dictionary. The last part of the dissertation targets a geometric representation of sketches, which is vital in enabling automatic sketch analysis, synthesis and manipulation. It is based on utilizing the well known convolutional sparse coding (CSC) model. We observe that CSC is closely related to how line sketches are drawn. This process can be approximated as the sparse spatial localization of a number of typical basic strokes, which in turn can be cast as a non-standard CSC model that forms a line drawing from parametric curves. These curves are learned to optimize the fit between the model and a specific set of line drawings. Each part of the dissertation shows the utility of the proposed methods through a variety of experiments, user studies, and proposed applications.

Die Verwendung von ikonographischen, bildhaften Kartenzeichen ist sehr beliebt bei der Darstellung von Sehenswürdigkeiten in touristischen Karten sowie bei Kartendarstellungen für Kinder und Jugendliche. Der Begriff des Non-Photorealistic Rendering (NPR) beschreibt einen zentralen Bereich in der Computergrafik, der sich mit der Erzeugung von Bildern auseinandersetzt, die scheinbar handgemacht sind und bewusst nicht dem physikalisch korrekten Abbild eines Modells entsprechen. Ein weiteres Trendthema zur Nachahmung eines bestimmten Stils eines Kunstwerks stellt der Neural Style Transfer (NST) dar. Hierbei werden künstlerische Bilder durch Trennung und Rekombination von Bildinhalt und Stil erzeugt. Im Rahmen der vorliegenden Arbeit ist ein semi-automatischer Workflow zur Erzeugung ikonographischer Gebäudedarstellungen für die Nutzung in zoombaren Webkarten entwickelt und in drei künstlerischen Stilvarianten unter Nutzung von Bildverarbeitungswerkzeugen in dem rasterbasierten Open Source Bildbearbeitungsprogramm GIMP, speziell mit der Filtersammlung G'MIC technisch umgesetzt worden. Außerdem zeigt die Masterarbeit das Potential der Ableitung von ikonographischen Signaturen durch den Style-Transfer mittels neuronaler Netze.:Selbstständigkeitserklärung III Inhaltsverzeichnis 5 Abbildungsverzeichnis 7 Tabellenverzeichnis 8 Abkürzungsverzeichnis 9 1 Einleitung 10 1.1 Motivation 10 1.2 Gliederung der Arbeit 10 2 Literaturstudium 11 2.1 Computergrafik 11 2.2 Non-Photorealistic Rendering 11 2.3 Neural Style Transfer 14 2.3.1 Einleitung 14 2.3.2 Convolutional Neural Network 15 2.3.3 Beschreibung des Algorithmus 17 3 Methodik 19 3.1 Technische Komponenten 19 3.2 Kriterien der Bildauswahl 19 3.3 Workflow „Ölmalerei“ 21 3.4 Workflow „Tuschezeichnung 22 3.5 Workflow „Silhouette“ 22 4 Praktischer Teil 23 4.1 Konkrete Umsetzung 23 4.1.1 Workflow „Ölmalerei“ 24 4.1.2 Workflow „Tuschezeichnung“ 32 4.1.3 Workflow „Silhouette“ 32 4.2 Implementierung eines Automatisierungsprozesses 35 4.3 Anwendung: Karte Dresden 39 4.4 Neural Style Transfer 43 4.4.1 Online-Anwendungen 43 4.4.2 Offline-Implementierung 45 5 Diskussion 51 5.1 Resultate 51 5.1.1 Bildverarbeitung 51 5.1.2 Neural Style Transfer 51 5.2 Ausblick 52 6 Zusammenfassung 52 Literaturverzeichnis 53 / The use of iconographic, pictorial map symbols is very popular for the representation of places of interest in tourist maps as well as for map presentations for children and young people. The term Non-Photorealistic Rendering (NPR) describes a prominent field in computer graphics that deals with the generation of images that are apparently handmade and deliberately do not correspond to the physically correct image of a model. Neural Style Transfer (NST) is another trend topic for imitating a certain style of an artwork. Here, artistic images are created by separating and recombining image content and style. In the context of the present work, a semi-automatic workflow for the creation of iconographic building representations for use in zoomable web maps has been developed and technically implemented in three artistic style variants using image processing tools in the raster-based open source image processing program GIMP, especially with the filter collection G'MIC. In addition, the master thesis demonstrates the potential of deriving iconographic signatures through style transfer using neural networks.:Selbstständigkeitserklärung III Inhaltsverzeichnis 5 Abbildungsverzeichnis 7 Tabellenverzeichnis 8 Abkürzungsverzeichnis 9 1 Einleitung 10 1.1 Motivation 10 1.2 Gliederung der Arbeit 10 2 Literaturstudium 11 2.1 Computergrafik 11 2.2 Non-Photorealistic Rendering 11 2.3 Neural Style Transfer 14 2.3.1 Einleitung 14 2.3.2 Convolutional Neural Network 15 2.3.3 Beschreibung des Algorithmus 17 3 Methodik 19 3.1 Technische Komponenten 19 3.2 Kriterien der Bildauswahl 19 3.3 Workflow „Ölmalerei“ 21 3.4 Workflow „Tuschezeichnung 22 3.5 Workflow „Silhouette“ 22 4 Praktischer Teil 23 4.1 Konkrete Umsetzung 23 4.1.1 Workflow „Ölmalerei“ 24 4.1.2 Workflow „Tuschezeichnung“ 32 4.1.3 Workflow „Silhouette“ 32 4.2 Implementierung eines Automatisierungsprozesses 35 4.3 Anwendung: Karte Dresden 39 4.4 Neural Style Transfer 43 4.4.1 Online-Anwendungen 43 4.4.2 Offline-Implementierung 45 5 Diskussion 51 5.1 Resultate 51 5.1.1 Bildverarbeitung 51 5.1.2 Neural Style Transfer 51 5.2 Ausblick 52 6 Zusammenfassung 52 Literaturverzeichnis 53

The field of Computer Vision continues to be revolutionized by advances in Convolutional Neural Networks. These networks are well suited for the regular grid structure of image data. However, there are many irregular image types that do not fit within such a framework, such as multi-view images, spherical images, superpixel representations, and texture maps for 3D meshes. These kinds of representations usually have specially designed networks that only operate and train on that unique form of data, thus requiring large datasets for each data domain. This dissertation aims to bridge the gap between standard convolutional networks and specialized ones. It proposes selection-based convolution. This technique operates on graph representations, giving it the flexibility to represent many irregular image domains, but maintains the spatially-oriented nature of an image convolution. Thus, it is possible to train a network on standard images, then use those same network weights for any kind graph-based representation. The effectiveness of this technique is evaluated on image types such as spherical images and 3D meshes for tasks such as segmentation and style transfer. Improvements to the selection mechanism through various forms of interpolation are also presented. Finally, this work demonstrates the generality of selection and its ability to be applied to various forms of graph networks and graph data, not just those specific to the image domain.

In Sweden, women of age between of 40 and 74 go through regular screening of their breasts every 18-24 months. The screening mainly involves obtaining a mammogram and having radiologists analyze them to detect any sign of breast cancer. However reading a mammography image requires experienced radiologist, and the lack of radiologist reduces the hospital's operating efficiency. What's more, mammography from different facilities increases the difficulty of diagnosis. Our work proposed a deep learning segmentation system which could adapt to mammography from various facilities and locate the position of the tumor. We train and test our method on two public mammography datasets and do several experiments to find the best parameter setting for our system. The test segmentation results suggest that our system could play as an auxiliary diagnosis tool for breast cancer diagnosis and improves diagnostic accuracy and efficiency. / I Sverige går kvinnor i åldrarna mellan 40 och 74 igenom regelbunden screening av sina bröst med 18-24 månaders mellanrum. Screeningen innbär huvudsakligen att ta mammogram och att låta radiologer analysera dem för att upptäcka tecken på bröstcancer. Emellertid krävs det en erfaren radiolog för att tyda en mammografibild, och bristen på radiologer reducerar sjukhusets operativa effektivitet. Dessutom, att mammografin kommer från olika anläggningar ökar svårigheten att diagnostisera. Vårt arbete föreslår ett djuplärande segmenteringssystem som kan anpassa sig till mammografi från olika anläggningar och lokalisera tumörens position. Vi tränar och testar vår metod på två offentliga mammografidataset och gör flera experiment för att hitta den bästa parameterinställningen för vårt system. Testsegmenteringsresultaten tyder på att vårt system kan fungera som ett hjälpdiagnosverktyg vid diagnos av bröstcancer och förbättra diagnostisk noggrannhet och effektivitet.

For many years, light fields have been a unique way of capturing a scene. By using a particular set of optics, a light field camera is able to, in a single moment, take images of the same scene from multiple perspectives. These perspectives can be used to calculate the scene geometry and allow for effects not possible with standard photographs, such as refocus and the creation of novel views.Neural style transfer is the process of training a neural network to render photographs in the style of a particular painting or piece of art. This is a simple process for a single photograph, but naively applying style transfer to each view in a light field generates inconsistencies in coloring between views. Because of these inconsistencies, common light field effects break down.We propose a style transfer method for light fields that maintains consistencies between different views of the scene. This is done by using warping techniques based on the depth estimation of the scene. These warped images are then used to compare areas of similarity between views and incorporate differences into the loss function of the style transfer network. Additionally, this is done in a post-training fashion, which removes the need for a light field training set.

Musical timbre transfer is the task of re-rendering the musical content of a given source using the rendering style of a target sound. The source keeps its musical content, e.g., pitch, microtiming, orchestration, and syncopation. I specifically focus on the task of transferring the style of percussive patterns extracted from polyphonic audio using a MelGAN-VC model [57] by training acoustic properties for each genre. Evaluating audio style transfer is challenging and typically requires user studies. An analytical methodology based on supervised and unsupervised learning including visualization for evaluating musical timbre transfer is proposed. The proposed methodology is used to evaluate the MelGAN-VC model for musical timbre transfer of drum tracks. The method uses audio features to analyze results of the timbre transfer based on classification probability from Random Forest classifier. And K-means algorithm can classify unlabeled instances using audio features and style-transformed results are visualized by t-SNE dimensionality reduction technique, which is helpful for interpreting relations between musical genres and comparing results from the Random Forest classifier. / Graduate

Digitization of physical artwork is usually done using image scanning devices in order to ensure that the output is accurate in terms of color and is of sufficiently high resolution, usually over 300 pixels per inch, however the usage of such a device is in some cases unfeasible due to medium or size constraints. Photography of the artwork is another method of artwork digitization, however such methods often produce results containing camera artifacts such as shadows, reflections or low resolution. This thesis project explores the possibility of creating an alternative to image scanners using smartphone photography and machine learning-based methods. Due to the very high memory requirement for enhancing images at very high resolutions, this is done in a two-stage process. The first stage uses an unpaired image style transfer model to remove shadows and highlights. The second stage uses a superresolution model to increase the resolution of the image. The results are evaluated on a small set of paired images using objective metrics and subjective metrics in the form of a user study. In some cases the method removed camera artifacts in the form of reflection and color accuracy, however the best results were achieved when the input data did not contain any major camera artifacts. Based on this it seems likely that style transfer models are not applicable for problems with a wide range of expected input and output. The use of super-resolution seems to be a crucial component of high-resolution image enhancement and the current state-of-the-art methods are able to convincingly increase the resolution of images provided that the input is of a sufficiently high resolution. The subjective evaluation shows that commonly used metrics such as structural similarity and Fréchet Inception Distance are applicable for this type of problem when analyzing the full image, however for smaller details other evaluation methods are required. / Digitalisering av fysiska konstverk görs vanligtvis med bildskannrar för att försäkra att den digitaliserade bilden är färgnoggrann och att upplösningen är tillräckligt hög, vanligtvis över 300 pixlar per tum. Dock är användandet av bildskannrar ibland svårt på grund av konstverkets material eller storlek. Fotografi av konstverk är en annan metod för digitalisering, men denna metod producerar ofta kameraartefakter i form av skuggor, reflektioner och låg upplösning. Detta examensarbete utforskar möjligheten att skapa ett alternativ till bildskannrar genom att använda smartphonefotografi och maskininlärningsbaserade metoder. På grund av de höga minneskraven för bildförbättring med mycket höga upplösningar görs detta i en tvåstegsprocess. Det första steget använder oparad bildstilöversättning för att eliminera skuggor och ljuspunkter. Det andra steget använder en superupplösningsmodell för att öka bildens upplösning. Resultaten utvärderas på en liten mängd parade bilder med objektiva jämförelser och subjektiva jämförelser i form av en användarstudie. I vissa fall reducerade metoden kameraartefakter i form av reflektioner och förbättrade färgexakthet, dock skedde dessa resultat i fall där indatan inte innehöll några större kameraartefakter. Baserat på detta är det sannolikt att stilöversättningsmodeller inte är applicerbara för problem med ett brett omfång av möjliga indata och utdata. Användandet av superupplösning verkar vara en viktig komponent av högupplöst bildförbättring och de bäst presenterande metoderna kan övertygande öka upplösningen av bilder i fall där indatan är av tillräckligt hög upplösning. Den subjektiva utvärderingen visar att vanligt använda utvärderingsmetoder som Fréchet-Inception-avstånd och strukturell likhet är applicerbara för denna typ av problem när de används för att analysera en hel bild, men för mindre detaljer behövs alternativa utvärderingsmetoder.

Histopathological evaluation and Gleason grading on Hematoxylin and Eosin(H&amp;E) stained specimens is the clinical standard in grading prostate cancer. Recently, deep learning models have been trained to assist pathologists in detecting prostate cancer. However, these predictions could be improved further regarding variations in morphology, staining and differences across scanners. An approach to tackle such problems is to employ conditional GANs for style transfer. A total of 52 prostatectomies from 48 patients were scanned with two different scanners. Data was split into 40 images for training and 12 images for testing and all images were divided into overlapping 256x256 patches. A segmentation model was trained using images from scanner A, and the model was tested on images from both scanner A and B. Next, GANs were trained to perform style transfer from scanner A to scanner B. The training was performed using unpaired training images and different types of Unsupervised Image to Image Translation GANs (CycleGAN and UNIT). Beside the common CycleGAN architecture, a modified version was also tested, adding Kullback Leibler (KL) divergence in the loss function. Then, the segmentation model was tested on the augmented images from scanner B.The models were evaluated on 2,000 randomly selected patches of 256x256 pixels from 10 prostatectomies. The resulting predictions were evaluated both qualitatively and quantitatively. All proposed methods outperformed in AUC, in the best case the improvement was of 16%. However, only CycleGAN trained on a large dataset demonstrated to be capable to improve the segmentation tool performance, preserving tissue morphology and obtaining higher results in all the evaluation measurements. All the models were analyzed and, finally, the significance of the difference between the segmentation model performance on style transferred images and on untransferred images was assessed, using statistical tests.

Background: Generative Adversarial Networks (Goodfellow et al., 2014) (GANs)are the current state of the art machine learning data generating systems. Designed with two neural networks in the initial architecture proposal, generator and discriminator. These neural networks compete in a zero-sum game technique, to generate data having realistic properties inseparable to that of original datasets. GANs have interesting applications in various domains like Image synthesis, 3D object generation in gaming industry, fake music generation(Dong et al.), text to image synthesis and many more. Despite having a widespread application domains, GANs are popular for image data synthesis. Various architectures have been developed for image synthesis evolving from fuzzy images of digits to photorealistic images. Objectives: In this research work, we study various literature on different GAN architectures. To understand significant works done essentially to improve the GAN architectures. The primary objective of this research work is synthesis of plant images using Style GAN (Karras, Laine and Aila, 2018) variant of GAN using style transfer. The research also focuses on identifying various machine learning performance evaluation metrics that can be used to measure Style GAN model for the generated image datasets. Methods: A mixed method approach is used in this research. We review various literature work on GANs and elaborate in detail how each GAN networks are designed and how they evolved over the base architecture. We then study the style GAN (Karras, Laine and Aila, 2018a) design details. We then study related literature works on GAN model performance evaluation and measure the quality of generated image datasets. We conduct an experiment to implement the Style based GAN on leaf dataset(Kumar et al., 2012) to generate leaf images that are similar to the ground truth. We describe in detail various steps in the experiment like data collection, preprocessing, training and configuration. Also, we evaluate the performance of Style GAN training model on the leaf dataset. Results: We present the results of literature review and the conducted experiment to address the research questions. We review and elaborate various GAN architecture and their key contributions. We also review numerous qualitative and quantitative evaluation metrics to measure the performance of a GAN architecture. We then present the generated synthetic data samples from the Style based GAN learning model at various training GPU hours and the latest synthetic data sample after training for around ~8 GPU days on leafsnap dataset (Kumar et al., 2012). The results we present have a decent quality to expand the dataset for most of the tested samples. We then visualize the model performance by tensorboard graphs and an overall computational graph for the learning model. We calculate the Fréchet Inception Distance score for our leaf Style GAN and is observed to be 26.4268 (the lower the better). Conclusion: We conclude the research work with an overall review of sections in the paper. The generated fake samples are much similar to the input ground truth and appear to be convincingly realistic for a human visual judgement. However, the calculated FID score to measure the performance of the leaf StyleGAN accumulates a large value compared to that of Style GANs original celebrity HD faces image data set. We attempted to analyze the reasons for this large score.