<div>Brain tumor sub region segmentation is a challenging problem in Magnetic Resonance imaging. The tumor regions tend to suffer from lack of homogeneity, textural differences, variable location, and their ability to proliferate into surrounding tissue. </div><div> The segmentation task thus requires an algorithm which can be indifferent to such influences and robust to external interference. In this work we propose a conditional generative adversarial network which learns off multiple planes of reference. Using this learning, we evaluate the quality of the segmentation and back propagate the loss for improving the learning. The results produced by the network show competitive quality in both the training and the testing data-set.</div><div><br></div>

Deep learning models heavily rely on an abundance of data, and their performance is directly affected by data availability. In mobility pattern modeling, problems, such as next location prediction or flow prediction, are commonly solved using deep learning approaches. Despite advances in modeling techniques, complications arise when acquiring mobility data is limited by geographic factors and data protection laws. Generating highquality synthetic data is one of the solutions to get around at times when information is scarce. Trajectory generation is concerned with generating trajectories that can reproduce the spatial and temporal characteristics of the underlying original mobility patterns. The task of this project was to evaluate Generative Adversarial Network (GAN) capabilities to generate synthetic vehicle trajectory data. We extend the methodology of previous research on trajectory generation by introducing conditional trajectory duration labels and a model pretraining mechanism. The evaluation of generated trajectories consisted of a two-fold analysis. We perform qualitative analysis by visually inspecting generated trajectories and quantitative analysis by calculating the statistical distance between synthetic and original data distributions. The results indicate that extending the previous GAN methodology allows the novel model to generate trajectories statistically closer to the original data distribution. Nevertheless, a statistical base model has the best generative performance and is the only model to generate visually plausible results. We accredit the superior performance of the statistical base model to the highly predictive nature of vehicle trajectories, which must follow the road network and have the tendency to follow minimum distance routes. This research considered only one type of GAN-based model, and further research should explore other architecture alternatives to understand the potential of GAN-based models fully / Modeller för djupinlärning är starkt beroende av ett överflöd av data, och derasprestanda påverkas direkt av datatillgänglighet. I mobilitetsmönstermodellering löses problem, såsom nästa platsförutsägelse eller flödesprediktion,vanligtvis med hjälp av djupinlärningsmetoder. Trots framsteg inommodelleringsteknik uppstår komplikationer när inhämtning av mobilitetsdatabegränsas av geografiska faktorer och dataskyddslagar. Att generera syntetiskdata av hög kvalitet är en av lösningarna för att ta sig runt i tider dåinformationen är knapp. Bangenerering handlar om att generera banorsom kan reproducera de rumsliga och tidsmässiga egenskaperna hos deunderliggande ursprungliga rörlighetsmönstren. Uppgiften för detta projektvar att utvärdera GAN-kapaciteten för att generera syntetiska fordonsbanor. Viutökar metodiken för tidigare forskning om banagenerering genom att introducera villkorliga etiketter för banalängd och en modellförträningsmekanism.Utvärderingen av genererade banor bestod av en tvåfaldig analys. Viutför kvalitativ analys genom att visuellt inspektera genererade banor ochkvantitativ analys genom att beräkna det statistiska avståndet mellan syntetiskaoch ursprungliga datafördelningar. Resultaten indikerar att en utvidgningav den tidigare GAN-metoden tillåter den nya modellen att generera banorstatistiskt närmare den ursprungliga datadistributionen. Ändå har en statistiskbasmodell den bästa generativa prestandan och är den enda modellen somgenererar visuellt rimliga resultat. Vi ackrediterar den statistiska basmodellensöverlägsna prestanda till den mycket prediktiva karaktären hos fordonsbanor,som måste följa vägnätet och ha en tendens att följa minimiavståndsrutter.Denna forskning övervägde endast en typ av GAN-baserad modell, ochytterligare forskning bör utforska andra arkitekturalternativ för att förståpotentialen hos GAN-baserade modeller fullt ut

Internal models are believed to be crucial components in human motor control. It has been suggested that the central nervous system (CNS) uses forward and inverse models as internal representations of the motor systems. However, it is still unclear how the CNS implements the high-dimensional control of our movements. In this project, generative adversarial networks (GAN) are studied as a generative model of movement data. It is shown that, for a relatively small number of effectors, it is possible to train a GAN which produces new movement samples that are plausible given a simulator environment. It is believed that these models can be extended to generate high-dimensional movement data. Furthermore, this project investigates the possibility to use a trained GAN as an integrated forward and inverse model for motor control. / Interna modeller tros vara en viktig del av mänsklig rörelsekontroll. Det har föreslagits att det centrala nervsystemet (CNS) använder sig av framåtriktade modeller och inversa modeller för intern representation av motorsystemen. Dock är det fortfarande okänt hur det centrala nervsystemet implementerar denna högdimensionella kontroll. Detta examensarbete undersöker användningen av generativa konkurrerande nätverk som generativ modell av rörelsedata. Experiment visar att dessa nätverk kan tränas till att generera ny rörelsedata av en tvådelad arm och att den genererade datan efterliknar träningsdatan. Vi tror att nätverken även kan modellera mer högdimensionell rörelsedata. I projektet undersöks även användningen av dessa nätverk som en integrerad framåtriktad och invers modell.

The recent advance in generative modeling in particular the unsupervised learning of data distribution is attributed to the invention of models with new learning algorithms. Among the methods proposed, generative adversarial networks (GANs) have shown to be the most efficient approaches to estimate data distributions. The core idea of GANs is an adversarial training of two deep neural networks, called generator and discriminator, to learn an implicit approximation of the true data distribution. The distribution is approximated through the weights of the generator network, and interaction with the distribution is through the process of sampling. GANs have found to be useful in applications such as image-to-image translation, in-painting, and text-to-image synthesis. In this thesis, we propose to capitalize on the power of GANs for different remote sensing problems. The first problem is a new research track to the remote sensing community that aims to generate remote sensing images from text descriptions. More specifically, we focus on exploiting ancient text descriptions of geographical areas, inherited from previous civilizations, and convert them the equivalent remote sensing images. The proposed method is composed of a text encoder and an image synthesis module. The text encoder is tasked with converting a text description into a vector. To this end, we explore two encoding schemes: a multilabel encoder and a doc2vec encoder. The multilabel encoder takes into account the presence or absence of objects in the encoding process whereas the doc2vec method encodes additional information available in the text. The encoded vectors are then used as conditional information to a GAN network and guide the synthesis process. We collected satellite images and ancient text descriptions for training in order to evaluate the efficacy of the proposed method. The qualitative and quantitative results obtained suggest that the doc2vec encoder-based model yields better images in terms of the semantic agreement with the input description. In addition, we present open research areas that we believe are important to further advance this new research area. The second problem we want to address is the issue of semi-supervised domain adaptation. The goal of domain adaptation is to learn a generic classifier for multiple related problems, thereby reducing the cost of labeling. To that end, we propose two methods. The first method uses GANs in the context of image-to-image translation to adapt source domain images into target domain images and train a classifier using the adapted images. We evaluated the proposed method on two remote sensing datasets. Though we have not explored this avenue extensively due to computational challenges, the results obtained show that the proposed method is promising and worth exploring in the future. The second domain adaptation strategy borrows the adversarial property of GANs to learn a new representation space where the domain discrepancy is negligible, and the new features are discriminative enough. The method is composed of a feature extractor, class predictor, and domain classifier blocks. Contrary to the traditional methods that perform representation and classifier learning in separate stages, this method combines both into a single-stage thereby learning a new representation of the input data that is domain invariant and discriminative. After training, the classifier is used to predict both source and target domain labels. We apply this method for large-scale land cover classification and cross-sensor hyperspectral classification problems. Experimental results obtained show that the proposed method provides a performance gain of up to 40%, and thus indicates the efficacy of the method.

This study is dedicated to enhancing the effectiveness of Optical Character Recognition (OCR) systems, with a special emphasis on Arabic handwritten digit recognition. The choice to focus on Arabic handwritten digits is twofold: first, there has been relatively less research conducted in this area compared to its English counterparts; second, the recognition of Arabic handwritten digits presents more challenges due to the inherent similarities between different Arabic digits.OCR systems, engineered to decipher both printed and handwritten text, often face difficulties in accurately identifying low-quality or distorted handwritten text. The quality of the input image and the complexity of the text significantly influence their performance. However, data augmentation strategies can notably improve these systems' performance. These strategies generate new images that closely resemble the original ones, albeit with minor variations, thereby enriching the model's learning and enhancing its adaptability. The research found Conditional Variational Autoencoders (C-VAE) and Conditional Generative Adversarial Networks (C-GAN) to be particularly effective in this context. These two generative models stand out due to their superior image generation and feature extraction capabilities. A significant contribution of the study has been the formulation of the Synthetic Image Evaluation Procedure, a systematic approach designed to evaluate and amplify the generative models' image generation abilities. This procedure facilitates the extraction of meaningful features, computation of the Fréchet Inception Distance (LFID) score, and supports hyper-parameter optimization and model modifications.

In modern society, cameras on intelligent devices can generate a huge amount of natural images, including images of the human body and face. Therefore, there is a huge social demand for more efficient editing of images to meet human production and life needs, including entertainment, such as image beauty. In recent years, Generative Models with Deep Learning techniques have attracted lots of attention in the Artificial Intelligence field, and some powerful methods, such as Variational Autoencoder and Generative Adversarial Networks, can generate very high-resolution and realistic images, especially for facial images, human body image. In this thesis, we follow the powerful generative model to achieve image generation and editing tasks, and we focus on human image generation and editing tasks, including local eye and face generation and editing, global human body generation, and editing. We introduce different methods to improve previous baselines based on different human regions. 1) Eye region of human image: Gaze correction and redirection aim to manipulate the eye gaze to a desired direction. Previous common gaze correction methods require annotating training data with precise gaze and head pose information. To address this issue, we proposed the new datasets as training data and formulated the gaze correction task as a generative inpainting problem, addressed using two new modules. 2) Face region of human image: Based on a powerful generative model for face region, many papers have learned to control the latent space to manipulate face attributes. However, they need more precise controls on 3d factors such as camera pose because they tend to ignore the underlying 3D scene rendering process. Thus, we take the pre-trained 3D-Aware generative model as the backbone and learn to manipulate the latent space using the attribute labels as conditional information to achieve the 3D-Aware face generation and editing task. 3) Human Body region of human image: 3D-Aware generative models have been shown to produce realistic images representing rigid/semi-rigid objects, such as facial regions. However, they usually struggle to generate high-quality images representing non-rigid objects, such as the human body, which greatly interests many computer graphics applications. Thus, we introduce semantic segmentation into the model. We split the entire generation pipeline into two stages and use intermediate segmentation masks to bridge these two stages. Furthermore, our model can control pose, semantic, and appearance codes by using multiple latent codes to achieve human image editing.

Structural health monitoring (SHM) is an anomaly detection process. Data-driven SHM has gained much attention compared to the model-based strategy, specifically with the current state-of-the-art machine learning routines. Model-based methods require structural information, time-consuming model updating, and may fail with noisy data, a persistent condition in real-time SHM problems. However, there are several hindrances in supervised and unsupervised settings in machine learning-based SHM. This study identifies and addresses such hindrances with the versatility of state-of-the-art deep learning strategies. While managing those complications, we aim at proposing a general, structure-independent (ie requires no prior information) SHM framework. Developing such techniques plays a crucial role in the SHM of smart cities. In the supervised SHM and sensor output validation (SOV) category, data class imbalance results from the lack of data from nuanced structural states. Employing Long Short-Term Memory (LSTM) units, we developed a general technique that manages both SHM and SOV. The developed architecture accepts high-dimensional features, enabling the train of Generative Adversarial Networks for data generation, addressing the complications of data imbalance. GAN-generated SHM data improved accuracy for low-sampled classes from 44.77% to 64.58% and from 73.39% to 90.84% in two SOV and SHM case studies, respectively. Arguing the unsupervised SHM as a practical category since it identifies novelties (ie unseen states), the current application of dimensionality reduction (DR) in unsupervised SHM is investigated. Due to the curse of dimensionality, classical unsupervised techniques cannot function with high-dimensional features, driving the use of DR techniques. Investigations highlighted the importance of avoiding DR in unsupervised SHM, as data dimensions that DR suppresses may contain damage-sensitive features for novelties. With DR, novelty detection accuracy declined up to 60% in two benchmark SHM datasets. Other obstacles in the unsupervised SHM area are case-dependent features, lack of dynamic-class novelty detection, and the impact of user-defined detection parameters on novelty detection accuracy. We chose the fast Fourier transform-based (FFT) of raw signals with no dimensionality reduction to develop the SHM framework. A deep neural network scheme is developed to perform the pattern recognition of that high-dimensional data. The framework does not require prior information, with GAN models implemented, offering robustness to sensor placement in structures. These characteristics make the framework suitable for developing general unsupervised SHM techniques. / Doctor of Philosophy / Detecting abnormal behaviors in structures from the input signals of sensors is called Structural health monitoring (SHM). The vibrational characteristics of signals or direct pattern recognition techniques can be applied to detect anomalies in a data-driven scheme. Machine learning (ML) tools are suitable for data-driven methods; However, challenges exist on both supervised and unsupervised ML-based SHM. Recent improvements in deep learning are employed in this study to address such obstacles after their identification. In supervised learning, the data points for the normal state of structures are abundant, and datasets are usually imbalanced, which is the same issue for the sensor output validation (SOV). SOV must be present before SHM takes place to remove anomalous sensor outputs. First, a unified decision-making system for SHM and SOV problems is proposed, and then data imbalance is alleviated by generating new data objects from low-sampled classes. The proposed unified method is based on the recurrent neural networks, and the generation mechanism is Generative Adversarial Network (GAN). Results indicate improvements in accuracy metrics for data classes in the minority. For the unsupervised SHM, four major issues are identified, including data loss during feature extraction, case-dependency of such extraction schemes. These two issues are solved with the fast Fourier transform (FFT) of signals to be the features with no reduction in their dimensionality. The other obstacles are the lack of discrimination between different novel classes (ie only two classes of damage and undamaged) and the effect of the detection parameters, defined by users, on the SHM analysis. The latter two predicaments are also addressed by online generating new data objects from the incoming signal stream with GAN and tuning the detection system to have the same performance regarding user-defined parameters regarding GAN-generated data. The proposed unsupervised technique is further improved to be insensitive to the sensor placement on structures by employing recurrent neural network layers in the GAN architecture, with the GAN that has overfitted discriminator.

Dimensionality reduction techniques (DR) enhance data interpretability and reduce space complexity, though at the cost of information loss. Such methods have been prevalent in the Structural Health Monitoring (SHM) anomaly detection literature. While DR is favorable in supervised anomaly detection, where possible novelties are known a priori, the efficacy is less clear in unsupervised detection. In this work, we perform a detailed assessment of the DR performance trade-offs to determine whether the information loss imposed by DR can impact SHM performance for previously unseen novelties. As a basis for our analysis, we rely on an SHM anomaly detection method operating on input signals' fast Fourier transform (FFT). FFT is regarded as a raw, frequency-domain feature that allows studying various DR techniques. We design extensive experiments comparing various DR techniques, including neural autoencoder models, to capture the impact on two SHM benchmark datasets exclusively. Results imply the loss of information to be more detrimental, reducing the novelty detection accuracy by up to 60\% with autoencoder-based DR. Regularization can alleviate some of the challenges though unpredictable. Dimensions of substantial vibrational information mostly survive DR; thus, the regularization impact suggests that these dimensions are not reliable damage-sensitive features regarding unseen faults. Consequently, we argue that designing new SHM anomaly detection methods that can work with high-dimensional raw features is a necessary research direction and present open challenges and future directions. / M.S. / Structural health monitoring (SHM) aids the timely maintenance of infrastructures, saving human lives and natural resources. Infrastructure will undergo unseen damages in the future. Thus, data-driven SHM techniques for handling unlabeled data (i.e., unsupervised learning) are suitable for real-world usage. Lacking labels and defined data classes, data instances are categorized through similarities, i.e., distances. Still, distance metrics in high-dimensional spaces can become meaningless. As a result, applying methods to reduce data dimensions is currently practiced, yet, at the cost of information loss. Naturally, a trade-off exists between the loss of information and the increased interpretability of low-dimensional spaces induced by dimensionality reduction procedures. This study proposes an unsupervised SHM technique that works with low and high-dimensional data to assess that trade-off. Results show the negative impacts of dimensionality reduction to be more severe than its benefits. Developing unsupervised SHM methods with raw data is thus encouraged for real-world applications.

Generative Adversarial Networks have been used to obtain state-of-the-art results for low-level computer vision tasks like single image super-resolution, however, they are notoriously difficult to train due to the instability related to the competing minimax framework. Additionally, traditional ensembling mechanisms cannot be effectively applied with these types of networks due to the resources they require at inference time and the complexity of their architectures. In this thesis an alternative method to create ensembles of individual, more stable and easier to train, models by using interpolations in the parameter space of the models is found to produce better results than those of the initial individual models when evaluated using perceptual metrics as a proxy of human judges. This method can be used as a framework to train GANs with competitive perceptual results in comparison to state-of-the-art alternatives. / Generative Adversarial Networks (GANs) har använts för att uppnå state-of-the- art resultat för grundläggande bildanalys uppgifter, som generering av högupplösta bilder från bilder med låg upplösning, men de är notoriskt svåra att träna på grund av instabiliteten relaterad till det konkurrerande minimax-ramverket. Dessutom kan traditionella mekanismer för att generera ensembler inte tillämpas effektivt med dessa typer av nätverk på grund av de resurser de behöver vid inferenstid och deras arkitekturs komplexitet. I det här projektet har en alternativ metod för att samla enskilda, mer stabila och modeller som är lättare att träna genom interpolation i parameterrymden visat sig ge bättre perceptuella resultat än de ursprungliga enskilda modellerna och denna metod kan användas som ett ramverk för att träna GAN med konkurrenskraftig perceptuell prestanda jämfört med toppmodern teknik.

Architectural design is a complex subject involving many different aspects that need to be considered. Drafting a floor plan from a blank slate can require iterating over several designs in the early phases of planning, and it is likely an even more daunting task for non-architects to tackle. This thesis investigates the opportunities of using conditional generative adversarial networks to generate floor plans for living spaces. The pix2pixHD method is used to learn a mapping between building boundaries and color-mapped floor plan layouts from the RPLAN dataset consisting of over 80k images. Previous work has mainly focused on either preserving an input boundary or generating layouts based on a set of conditions. To give potential users more control over the generation process, it would be useful to generate floor plans that respect both an input boundary and some high-level client requirements. By encoding requirements about desired room types and their locations in colored centroids, and stacking this image with the boundary input, we are able to train a model to synthesize visually plausible floor plan images that adapt to the given conditions. This model is compared to another model trained on only the building boundary images that acts as a baseline. Results from visual inspection, image properties, and expert evaluation show that the model trained with centroid conditions generates samples with superior image quality to the baseline model. Feeding additional information to the networks is therefore not only a way to involve the user in the design process, but it also has positive effects on the model training. The results from this thesis demonstrate that floor plan generation with generative adversarial networks can respect different kinds of conditions simultaneously, and can be a source of inspiration for future work seeking to make computer-aided design a more collaborative process between users and models. / Arkitektur och design är komplexa områden som behöver ta hänsyn till ett flertal olika aspekter. Att skissera en planritning helt från början kan kräva flera iterationer av olika idéer i de tidiga stadierna av planering, och det är troligtvis en ännu mer utmanande uppgift för en icke-arkitekt att angripa. Detta examensarbete syftar till att undersöka möjligheterna till att använda conditional generative adversarial networks för att generera planritningar för boendeytor. Pix2pixHD-metoden används för att lära en modell ett samband mellan gränsen av en byggnad och en färgkodad planritning från datasamlingen RPLAN bestående av över 80 tusen bilder. Tidigare arbeten har främst fokuserat på att antingen bevara en given byggnadsgräns eller att generera layouts baserat på en mängd av villkor. För att ge potentiella slutanvändare mer kontroll över genereringsprocessen skulle det vara användbart att generera planritningar som respekterar både en given byggnadsgräns och några klientbehov på en hög nivå. Genom att koda krav relaterade till önskade rumstyper och deras placering som färgade centroider, och sedan kombinera denna bild med byggnadsgränsen, kan vi träna en modell som kan framställa visuellt rimliga bilder på planritningar som kan anpassa sig till de givna villkoren. Denna modell jämförs med en annan modell som tränas endast på byggnadsgränser och som kan agera som en baslinje. Resultat från inspektion av genererade bilder och deras egenskaper, samt expertevaluering visar att modellen som tränas med centroidvillkor genererar bilder med högre bildkvalitet jämfört med baslinjen. Att ge mer information till modellen kan därmed både involvera användaren mer i designprocessen och bidra till positiva effekter på träningen av modellen. Resultaten från detta examensarbete visar att generering av planritningar med generative adversarial networks kan respektera olika typer av villkor samtidigt, och kan vara en källa till inspiration för framtida arbete som syftar till att göra datorstödd design en mer kollaborativ process mellan användare och modeller.