Development of deep learning methods is dependent majorly on availability of large amounts of high quality data. To tackle the problem of data scarcity one of the workarounds is to generate synthetic data using deep learning methods. Especially, when dealing with trajectory data there are added challenges that come in to the picture such as high dependencies of the spatial and temporal component, geographical context sensitivity, privacy laws that protect an individual from being traced back to them based on their mobility patterns etc. This project is an attempt to overcome these challenges by exploring the capabilities of Generative Adversarial Networks (GANs) to generate synthetic trajectories which have characteristics close to the original trajectories. A naive model is designed as a baseline in comparison with a Long Short Term Memorys (LSTMs) based GAN. GANs are generally associated with image data and that is why Convolutional Neural Network (CNN) based GANs are very popular in recent studies. However, in this project an LSTM-based GAN was chosen to work with in order to explore its capabilities and strength of handling long-term dependencies sequential data well. The methods are evaluated using qualitative metrics of visually inspecting the trajectories on a real-world map as well as quantitative metrics by calculating the statistical distance between the underlying data distributions of the original and synthetic trajectories. Results indicate that the baseline method implemented performed better than the GAN model. The baseline model generated trajectories that had feasible spatial and temporal components, whereas the GAN model was able to learn the spatial component of the data well and not the temporal component. Conditional map information could be added as part of training the networks and this can be a research question for future work. / Utveckling av metoder för djupinlärning är till stor del beroende av tillgången på stora mängder data av hög kvalitet. För att ta itu med problemet med databrist är en av lösningarna att generera syntetisk data med hjälp av djupinlärning. Speciellt när man hanterar bana data finns det ytterligare utmaningar som kommer in i bilden såsom starka beroenden av den rumsliga och tidsmässiga komponenten, geografiska känsliga sammanhang, samt integritetslagar som skyddar en individ från att spåras tillbaka till dem baserat på deras mobilitetsmönster etc. Detta projekt är ett försök att överkomma dessa utmaningar genom att utforska kapaciteten hos generativa motståndsnätverk (GAN) för att generera syntetiska banor som har egenskaper nära de ursprungliga banorna. En naiv modell är utformad som en baslinje i jämförelse med en LSTM-baserad GAN. GAN:er är i allmänhet förknippade med bilddata och det är därför som CNN-baserade GAN:er är mycket populära i nya studier. I det här projektet valdes dock en LSTM-baserad GAN att arbeta med för att utforska dess förmåga och styrka att hantera långsiktiga beroenden och sekventiella data på ett bra sätt. Metoderna utvärderas med hjälp av kvalitativa mått för att visuellt inspektera banorna på en verklig världskarta samt kvantitativa mått genom att beräkna det statistiska avståndet mellan de underliggande datafördelningarna för de ursprungliga och syntetiska banorna. Resultaten indikerar att den implementerade baslinjemetoden fungerade bättre än GAN-modellen. Baslinjemodellen genererade banor som hade genomförbara rumsliga och tidsmässiga komponenter, medan GAN-modellen kunde lära sig den rumsliga komponenten av data väl men inte den tidsmässiga komponenten. Villkorskarta skulle kunna läggas till som en del av träningen av nätverken och detta kan vara en forskningsfråga för framtida arbete.

<p dir="ltr">Generating images from training samples solves the challenge of imbalanced data. It provides the necessary data to run machine learning algorithms for image classification, anomaly detection, and pattern recognition tasks. In medical settings, having imbalanced data results in higher false negatives due to a lack of positive samples. Generative Adversarial Networks (GANs) have been widely adopted for image generation. GANs allow models to train without computing intractable probability while producing high-quality images. However, evaluating GANs has been challenging for the researchers due to a need for an objective function. Most studies assess the quality of generated images and the variety of classes those images cover. Overfitting of training images, however, has received less attention from researchers. When the generated images are mere copies of the training data, GAN models will overfit and will not generalize well. This study examines the ability to detect overfitting of popular metrics: Maximum Mean Discrepancy (MMD) and Fréchet Inception Distance (FID). We investigate the metrics on two types of data: handwritten digits and chest x-ray images using Analysis of Variance (ANOVA) models.</p>

Generative models are broadly used in many subfields of DL. DNNs have recently developed a core approach to solving data-centric problems in image classification, translation, etc. The latest developments in parameterizing these models using DNNs and stochastic optimization algorithms have allowed scalable modeling of complex, high-dimensional data, including speech, text, and image. This dissertation proposal presents our state-the-art probabilistic bases and DL algorithms for generative models, including VAEs, GANs, and RNN-based encoder-decoder. The proposal also discusses application areas that may benefit from deep generative models in both NLP and computer vision. In NLP, we proposed an Arabic poetry generation model with extended phonetic and semantic embeddings (Phonetic CNN_subword embeddings). Extensive quantitative experiments using BLEU scores and Hamming distance show notable enhancements over strong baselines. Additionally, a comprehensive human evaluation confirms that the poems generated by our model outperform the base models in criteria including meaning, coherence, fluency, and poeticness. We proposed a generative video model using a hybrid VAE-GAN model in computer vision. Besides, we integrate two attentional mechanisms with GAN to get the essential regions of interest in a video, focused on enhancing the visual implementation of the human motion in the generated output. We have considered quantitative and qualitative experiments, including comparisons with other state-of-the-arts for evaluation. Our results indicate that our model enhances performance compared with other models and performs favorably under different quantitive metrics PSNR, SSIM, LPIPS, and FVD.Recently, mimicking biologically inspired learning in generative models based on SNNs has been shown their effectiveness in different applications. SNNs are the third generation of neural networks, in which neurons communicate through binary signals known as spikes. Since SNNs are more energy-efficient than DNNs. Moreover, DNN models have been vulnerable to small adversarial perturbations that cause misclassification of legitimate images. This dissertation shows the proposed ``VAE-Sleep'' that combines ideas from VAE and the sleep mechanism leveraging the advantages of deep and spiking neural networks (DNN--SNN).On top of that, we present ``Defense–VAE–Sleep'' that extended work of ``VAE-Sleep'' model used to purge adversarial perturbations from contaminated images. We demonstrate the benefit of sleep in improving the generalization performance of the traditional VAE when the testing data differ in specific ways even by a small amount from the training data. We conduct extensive experiments, including comparisons with the state–of–the–art on different datasets.

Satellite remote sensing has revolutionized the acquisition of large amounts of data, employing both active and passive sensors to capture critical information about our planet. These data can be analysed by using deep learning methodologies that demonstrate excellent capabilities in extracting the semantics from the data. However, one of the main challenges in exploiting the power of deep learning for remote sensing applications is the lack of labeled training data. Deep learning architectures, typically demand substantial quantities of training samples to achieve optimal performance. Motivated by the above-mentioned challenges, this thesis focuses on the limited availability of labeled datasets. These challenges include issues such as ambiguous labels in case of large-scale remote sensing datasets, particularly when dealing with the analysis of multi-source satellite remote sensing images. By employing novel deep learning techniques and cutting-edge methodologies, this thesis endeavors to contribute to advancements in the field of remote sensing. In this thesis, the problems related to limited labels are solved in several ways by developing (i) a novel spectral index generative adversarial network to augment real training samples for generating class-specific remote sensing data to provide a large number of labeled samples to train a neural-network classifier; (ii) a mono- and dual-regulated contractive-expansive-contractive convolutional neural network architecture to incorporate spatial-spectral information of multispectral data and minimize the loss in the feature maps and extends this approach to the analysis of hyperspectral images; (iii) a hybrid deep learning architecture with a discrete wavelet transform and attention mechanism to deal with few labeled samples for scene-based classification of multispectral images; and (iv) a weak supervised semantic learning technique that utilises weak or low-resolution labeled samples with multisource remote sensing images for predicting pixel-wise land-use-land-cover maps. The experiments show that the proposed approaches perform better than the state-of-the-art methods on different benchmark datasets and in different conditions.

Multi-temporal image analysis has been widely used in many applications such as urban monitoring, disaster management, and agriculture. With the development of the remote sensing technology, the new generation remote sensing satellite images with High/ Very High spatial resolution (HR/VHR) are now available. Compared to the traditional low/medium spatial resolution images, the detailed information of ground objects can be clearly analyzed in the HR/VHR images. Classical methods of multi-temporal image analysis deal with the images at pixel level and have worked well on low/medium resolution images. However, they provide sub-optimal results on new generation images due to their limited capability of modeling complex spatial and spectral information in the new generation products. Although significant number of object-based methods have been proposed in the last decade, they depend on suitable segmentation scale for diverse kinds of objects present in each temporal image. Thus their capability to express contextual information is limited. Typical spatial properties of last generation images emphasize the need of having more flexible models for object representation. Another drawback of the traditional methods is the difficulty in transferring knowledge learned from one specific problem to another. In the last few years, an interesting development is observed in the machine learning/computer vision field. Deep learning, especially Convolution Neural Networks (CNNs) have shown excellent capability to capture object level information and in transfer learning. By 2015, deep learning achieved state-of-the-art performance in most computer vision tasks. Inspite of its success in computer vision fields, the application of deep learning in multi-temporal image analysis saw slow progress due to the requirement of large labeled datasets to train deep learning models. However, by the start of this PhD activity, few works in the computer vision literature showed that deep learning possesses capability of transfer learning and training without labeled data. Thus, inspired by the success of deep learning, this thesis focuses on developing deep learning based methods for unsupervised/semi-supervised multi-temporal image analysis. This thesis is aimed towards developing methods that combine the benefits of deep learning with the traditional methods of multi-temporal image analysis. Towards this direction, the thesis first explores the research challenges that incorporates deep learning into the popular unsupervised change detection (CD) method - Change Vector Analysis (CVA) and further investigates the possibility of using deep learning for multi-temporal information extraction. The thesis specifically: i) extends the paradigm of unsupervised CVA to novel Deep CVA (DCVA) by using a pre-trained network as deep feature extractor; ii) extends DCVA by exploiting Generative Adversarial Network (GAN) to remove necessity of having a pre-trained deep network; iii) revisits the problem of semi-supervised CD by exploiting Graph Convolutional Network (GCN) for label propagation from the labeled pixels to the unlabeled ones; and iv) extends the problem statement of semantic segmentation to multi-temporal domain via unsupervised deep clustering. The effectiveness of the proposed novel approaches and related techniques is demonstrated on several experiments involving passive VHR (including Pleiades), passive HR (Sentinel-2), and active VHR (COSMO-SkyMed) datasets. A substantial improvement is observed over the state-of-the-art shallow methods.

Background: Generative Adversarial Networks (Goodfellow et al., 2014) (GANs)are the current state of the art machine learning data generating systems. Designed with two neural networks in the initial architecture proposal, generator and discriminator. These neural networks compete in a zero-sum game technique, to generate data having realistic properties inseparable to that of original datasets. GANs have interesting applications in various domains like Image synthesis, 3D object generation in gaming industry, fake music generation(Dong et al.), text to image synthesis and many more. Despite having a widespread application domains, GANs are popular for image data synthesis. Various architectures have been developed for image synthesis evolving from fuzzy images of digits to photorealistic images. Objectives: In this research work, we study various literature on different GAN architectures. To understand significant works done essentially to improve the GAN architectures. The primary objective of this research work is synthesis of plant images using Style GAN (Karras, Laine and Aila, 2018) variant of GAN using style transfer. The research also focuses on identifying various machine learning performance evaluation metrics that can be used to measure Style GAN model for the generated image datasets. Methods: A mixed method approach is used in this research. We review various literature work on GANs and elaborate in detail how each GAN networks are designed and how they evolved over the base architecture. We then study the style GAN (Karras, Laine and Aila, 2018a) design details. We then study related literature works on GAN model performance evaluation and measure the quality of generated image datasets. We conduct an experiment to implement the Style based GAN on leaf dataset(Kumar et al., 2012) to generate leaf images that are similar to the ground truth. We describe in detail various steps in the experiment like data collection, preprocessing, training and configuration. Also, we evaluate the performance of Style GAN training model on the leaf dataset. Results: We present the results of literature review and the conducted experiment to address the research questions. We review and elaborate various GAN architecture and their key contributions. We also review numerous qualitative and quantitative evaluation metrics to measure the performance of a GAN architecture. We then present the generated synthetic data samples from the Style based GAN learning model at various training GPU hours and the latest synthetic data sample after training for around ~8 GPU days on leafsnap dataset (Kumar et al., 2012). The results we present have a decent quality to expand the dataset for most of the tested samples. We then visualize the model performance by tensorboard graphs and an overall computational graph for the learning model. We calculate the Fréchet Inception Distance score for our leaf Style GAN and is observed to be 26.4268 (the lower the better). Conclusion: We conclude the research work with an overall review of sections in the paper. The generated fake samples are much similar to the input ground truth and appear to be convincingly realistic for a human visual judgement. However, the calculated FID score to measure the performance of the leaf StyleGAN accumulates a large value compared to that of Style GANs original celebrity HD faces image data set. We attempted to analyze the reasons for this large score.

In this master’s thesis we explore the idea of using artificial intelligence in the product design process and seek to develop a conceptual framework for how it can be incorporated to make user customized products more accessible and affordable for everyone. We show how generative deep learning models such as Variational Auto Encoders and Generative Adversarial Networks can be implemented to generate design variations of windows and clarify the general implementation process along with insights from recent research in the field. The proposed framework consists of three parts: (1) A morphological matrix connecting several identified possibilities of implementation to specific parts of the product design process. (2) A general step-by-step process on how to incorporate generative deep learning. (3) A description of common challenges, strategies andsolutions related to the implementation process. Together with the framework we also provide a system for automatic gathering and cleaning of image data as well as a dataset containing 4564 images of windows in a front view perspective.

No description available.

<div><div><div><p>Face redaction is needed by law enforcement and mass media outlets to guarantee privacy. In this thesis, a performance analysis of several face redaction/obscuration methods, such as blurring and pixelation is presented. The analysis is based on various threat models and obscuration attackers to achieve a comprehensive evaluation. We show that the traditional blurring and pixelation methods cannot guarantee privacy. To provide a more secured privacy protection, we propose two novel obscuration methods that are based on the generative adversarial networks. The proposed methods not only remove the identifiable information, but also preserve the non-identifiable facial information (as known as the utility information), such as expression, age, skin tone and gender.</p><p>We also propose methods for change detection in satellite imagery. In this thesis, we consider two types of building changes: 2D appearance change and 3D height change. We first present a model with an attention mechanism to detect the building appearance changes that are caused by natural disasters. Furthermore, to detect the changes of building height, we present a height estimation model that is based on building shadows and solar angles without relying on height annotation. Both change detection methods require good building segmentation performance, which might be hard to achieve for the low-quality images, such as off-nadir images. To solve this issue, we use uncertainty modeling and satellite imagery metadata to achieve accurate building segmentation for the noisy images that are taken from large off-nadir angles.</p></div></div></div>

Generation of realistic skydome images We aim to generate realistic images of the sky with clouds using generative adversarial networks (GANs). We explore two GAN architectures, ProGAN and StyleGAN, and find that StyleGAN produces significantly better results. We also propose a novel architecture SuperGAN which aims to generate images at very high resolutions, which cannot be efficiently handled using state-of-art architectures. 1