This thesis describes design and implementation of an approach to the mobile robot localization. The proposed method is based purely on images taken by a monocular camera. The described solution handles localization as an association problem and, therefore, falls in the category of topological localization methods. The method is based on a generative probabilistic model of the environment appearance. The proposed solution is capable to eliminate some of the difficulties which are common in traditional localization approaches.

Nous argumentons que l'estimation de l'information mutuelle entre des ensembles de variables aléatoires continues de hautes dimensionnalités peut être réalisée par descente de gradient sur des réseaux de neurones. Nous présentons un estimateur neuronal de l'information mutuelle (MINE) dont la complexité croît linéairement avec la dimensionnalité des variables et la taille de l'échantillon, entrainable par retro-propagation, et fortement consistant au sens statistique. Nous présentons aussi une poignée d'application ou MINE peut être utilisé pour minimiser ou maximiser l'information mutuelle. Nous appliquons MINE pour améliorer les modèles génératifs adversariaux. Nous utilisons aussi MINE pour implémenter la méthode du goulot d'étranglement de l'information dans un cadre de classification supervisé. Nos résultats montrent un gain substantiel en flexibilité et performance. / We argue that the estimation of mutual information between high dimensional continuous random variables can be achieved by gradient descent over neural networks. We present a Mutual Information Neural Estimator (MINE) that is linearly scalable in dimensionality as well as in sample size, trainable through back-prop, and strongly consistent. We present a handful of applications on which MINE can be used to minimize or maximize mutual information. We apply MINE to improve adversarially trained generative models. We also use MINE to implement the Information Bottleneck, applying it to supervised classification; our results demonstrate substantial improvement in flexibility and performance in the settings.

The thesis proposes an unsupervised representation learning method to predict 3D human pose from a 2D skeleton via a VAEGAN (Variational Autoencoder Generative Adversarial Network) hybrid network. The method learns to lift poses from 2D to 3D using selfsupervision and adversarial learning techniques. The method does not use images, heatmaps, 3D pose annotations, paired/unpaired 2Dto3D skeletons, 3D priors, synthetic 2D skeletons, multiview or temporal information in any shape or form. The 2D skeleton input is taken by a VAE that encodes it in a latent space and then decodes that latent representation to a 3D pose. The 3D pose is then reprojected to 2D for a constrained, selfsupervised optimization using the input 2D pose. Parallelly, the 3D pose is also randomly rotated and reprojected to 2D to generate a ’novel’ 2D view for unconstrained adversarial optimization using a discriminator network. The combination of the optimizations of the original and the novel 2D views of the predicted 3D pose results in a ’realistic’ 3D pose generation. The thesis shows that the encoding and decoding process of the VAE addresses the major challenge of erroneous and incomplete skeletons from 2D detection networks as inputs and that the variance of the VAE can be altered to get various plausible 3D poses for a given 2D input. Additionally, the latent representation could be used for crossmodal training and many downstream applications. The results on Human3.6M datasets outperform previous unsupervised approaches with less model complexity while addressing more hurdles in scaling the task to the real world. / Uppsatsen föreslår en oövervakad metod för representationslärande för att förutsäga en 3Dpose från ett 2D skelett med hjälp av ett VAE GAN (Variationellt Autoenkodande Generativt Adversariellt Nätverk) hybrid neuralt nätverk. Metoden lär sig att utvidga poser från 2D till 3D genom att använda självövervakning och adversariella inlärningstekniker. Metoden använder sig vare sig av bilder, värmekartor, 3D poseannotationer, parade/oparade 2D till 3D skelett, a priori information i 3D, syntetiska 2Dskelett, flera vyer, eller tidsinformation. 2Dskelettindata tas från ett VAE som kodar det i en latent rymd och sedan avkodar den latenta representationen till en 3Dpose. 3D posen är sedan återprojicerad till 2D för att genomgå begränsad, självövervakad optimering med hjälp av den tvådimensionella posen. Parallellt roteras dessutom 3Dposen slumpmässigt och återprojiceras till 2D för att generera en ny 2D vy för obegränsad adversariell optimering med hjälp av ett diskriminatornätverk. Kombinationen av optimeringarna av den ursprungliga och den nya 2Dvyn av den förutsagda 3Dposen resulterar i en realistisk 3Dposegenerering. Resultaten i uppsatsen visar att kodningsoch avkodningsprocessen av VAE adresserar utmaningen med felaktiga och ofullständiga skelett från 2D detekteringsnätverk som indata och att variansen av VAE kan modifieras för att få flera troliga 3D poser för givna 2D indata. Dessutom kan den latenta representationen användas för crossmodal träning och flera nedströmsapplikationer. Resultaten på datamängder från Human3.6M är bättre än tidigare oövervakade metoder med mindre modellkomplexitet samtidigt som de adresserar flera hinder för att skala upp uppgiften till verkliga tillämpningar.

Nous pouvons trouver des propriétés structurelles similaires dans presque tous les réseaux trophiques (ensemble d’interactions de prédation). L'existence de ces invariants suggère qu’il serait possible, pour chaque réseau trophique, de déterminer des paramètres généraux qui décrivent sa structure. Il serait également possible de faire le cheminement inverse, soit à partir de paramètres généraux, d’obtenir une structure de réseau qui respecte ces invariants. C’est ainsi que fonctionnent les modèles génératifs, qui prédisent une structure à partir de paramètres généraux. Cependant, les modèles génératifs peuvent générer des structures de réseau qui diffèrent des données empiriques, parce qu'ils intègrent différentes hypothèses sur les mécanismes qui façonnent les réseaux trophiques, et donc sur les paramètres généraux qui doivent être utilisés. Dans ce mémoire, j’étudie l'effet de la pondération de la stabilité à l'aide du paramètre sigma (écart-type maximum des forces d’interactions qu’il ne faut pas dépasser si l’on veut que le réseau d’espèces reste stable) sur la distribution des propriétés de réseau obtenues par différents modèles génératifs. En effet, en donnant une plus grande importance aux réseaux dont la structure est a priori stable (potentiellement plus proche de celles retrouvées dans la nature) on pourrait corriger les prédictions des modèles en rapprochant leurs résultats des données empiriques. Le principe de correction fait ici référence à l’utilisation des probabilités par les modèles génératifs : la correction est la modification de ces probabilités en faveur des réseaux stables afin qu’ils soient sur-représentés dans les données générées. Notre hypothèse est donc que la pondération de la stabilité pourrait améliorer les prédictions des modèles génératifs. Les modèles génératifs étudiés ici sont les modèles de cascade, de niche et de hiérarchie emboîtée. Notre principale conclusion est que, de manière contre-intuitive, la pondération de la stabilité n’améliore pas la différence entre les structures de réseaux empiriques et celles des réseaux générés par les mo-dèles. Nos résultats montrent que pour les réseaux étudiés, la plus grande différence entre les réseaux trophiques modélisés par les modèles génératifs et les réseaux empiriques est la nature du modèle et non la correction par la pondération de la stabilité. Cela suggère que ces modèles prédisent la structure à partir d’un nombre de paramètres insuffisants, où de paramètres ne représentant qu’une fraction de la structure du réseau. Le modèle de niche présente la prédiction la plus proche des données empiriques, mais seulement pour les réseaux comptant jusqu'à 20 espèces. Cette étude souligne donc le long chemin qu'il nous reste à parcourir avant de pouvoir représenter les réseaux trophiques de façon réaliste à partir de modèles génératifs simples. / We can find similar structural properties in almost every food web. The existence of these invariants suggests that it could be possible for each food web to determine general parameters. The reverse case also works, i.e. from general parameters, to obtain a network structure. This is how generative models work, they predict a structure from general parameters. However, the network structures obtained from generative models differ from empirical data, because they incorporate different assumptions about the mechanisms that shape food webs and thus the gen-eral parameters used. In this study, I’ll investigate the effect of weighting stability using the sigma parameter (maximum standard deviation of interaction forces that should not be exceeded if the species network is to remain stable). I’m studying its effect on the distribution of network prop-erties obtained by different generative models. Indeed, by giving greater importance to networks whose structure is stable, one could correct the predictions of the models by bringing their results closer to the empirical data. The correction is the modification of these probabilities in favor of stable networks so that they are more easily chosen by the model. Our hypothesis is therefore that weighting stability could improve the predictions of the cascade, niche and nested hierarchy models. Our main conclusion is that stability weighting does not improve the difference between empirical and model-generated network structures. Our results show that for the networks stud-ied, the biggest difference between food webs modeled by generative models and empirical net-works is the nature of the model and not the correction by stability weighting. This suggests that these models predict structure from an insufficient number of parameters or from parameters that represent only a fraction of the network structure. The niche model shows the closest pre-diction to the empirical data, but only for networks with up to 20 species. This study highlights the long way to go before we can realistically represent food webs using generative models.

Forecasting of highway traffic is a common practice for real traffic information system, and is of vital importance to traffic management and control on highways. As a typical time-series forecasting task, we want to propose a deep learning model to map the historical sensory traffic values (e.g., speed, flow) to future traffic forecasts. Prevailing traffic forecasting methods focus on the graph representation of the urban road. However, compared to the dense connectivity of urban road networks, highway traffic flows normally run on road segments of serial topology. This indicates that the highway traffic flows do not have the same type of spatial interaction, therefore motivating us to resort to a new forecasting paradigm. While traffic patterns can be intuitively represented by spatial-temporal (ST) images, this study transforms the traffic forecasting task into the conditional image generation task. Our approach explores the inherent properties of ST-images from the perspectives of physical meaning and traffic dynamics. An innovative deep learning based architecture is designed to process the ST-image, and a diffusion model is trained to obtain traffic forecasts by generating future ST-image based on the historical STimages. We demonstrate the effectiveness of the architecture in processing ST-image through ablation studies and the effectiveness of the model through comparison with popular baseline models, i.e., LSTM and T-GCN. / Prognos av vägtrafik är en vanlig praxis för riktiga trafikinformationssystem och är av vital betydelse för trafikhantering och kontroll på motorvägar. Som en typisk tidsserieförutsägelseuppgift vill vi föreslå en djupinlärningsmodell för att kartlägga historiska sensoriska trafikvärden (t.ex. hastighet, flöde) till framtida trafikprognoser. Rådande trafikprognosmetoder fokuserar på grafrepresentationen av stadsvägar. Jämfört med den täta anslutningen av stadsvägnät, löper motorvägstrafik normalt på vägsegment med seriell topologi. Detta indikerar att motorvägstrafikflöden inte har samma typ av rumslig interaktion, vilket motiverar oss att använda en ny prognosparadigm. Medan trafikmönster intuitivt kan representeras av spatial-temporala (ST) bilder, omvandlar denna studie trafikprognosuppgiften till en uppgift för betingad bildgenerering. Vår metod utforskar de inneboende egenskaperna hos ST-bilder från perspektiven fysisk betydelse och trafikdynamik. En innovativ djupinlärningsbaserad arkitektur är utformad för att behandla STbilden, och en diffusionsmodell tränas för att erhålla trafikprognoser genom att generera framtida ST-bilder baserat på historiska ST-bilder. Vi demonstrerar effektiviteten hos arkitekturen genom avbränningsstudier och modellens effektivitet genom jämförelse med populära baslinjemodeller, dvs. LSTM och T-GCN.

Data augmentation is a technique that can be performed in various ways to improve the training of discriminative models. The recent developments in deep generative models offer new ways of augmenting existing data sets. In this thesis, a framework for augmenting annotated data sets with deep generative models is proposed together with a method for quantitatively evaluating the quality of the generated data sets. Using this framework, two data sets for pupil localization was generated with different generative models, including both well-established models and a novel model proposed for this purpose. The unique model was shown both qualitatively and quantitatively to generate the best data sets. A set of smaller experiments on standard data sets also revealed cases where this generative model could improve the performance of an existing discriminative model. The results indicate that generative models can be used to augment or replace existing data sets when training discriminative models. / Dataaugmentering är en teknik som kan utföras på flera sätt för att förbättra träningen av diskriminativa modeller. De senaste framgångarna inom djupa generativa modeller har öppnat upp nya sätt att augmentera existerande dataset. I detta arbete har ett ramverk för augmentering av annoterade dataset med hjälp av djupa generativa modeller föreslagits. Utöver detta så har en metod för kvantitativ evaulering av kvaliteten hos genererade data set tagits fram. Med hjälp av detta ramverk har två dataset för pupillokalisering genererats med olika generativa modeller. Både väletablerade modeller och en ny modell utvecklad för detta syfte har testats. Den unika modellen visades både kvalitativt och kvantitativt att den genererade de bästa dataseten. Ett antal mindre experiment på standardiserade dataset visade exempel på fall där denna generativa modell kunde förbättra prestandan hos en existerande diskriminativ modell. Resultaten indikerar att generativa modeller kan användas för att augmentera eller ersätta existerande dataset vid träning av diskriminativa modeller.

L'increment de bases de dades que cada vegada contenen imatges més difícils i amb un nombre més elevat de categories, està forçant el desenvolupament de tècniques de representació d'imatges que siguin discriminatives quan es vol treballar amb múltiples classes i d'algorismes que siguin eficients en l'aprenentatge i classificació. Aquesta tesi explora el problema de classificar les imatges segons l'objecte que contenen quan es disposa d'un gran nombre de categories. Primerament s'investiga com un sistema híbrid format per un model generatiu i un model discriminatiu pot beneficiar la tasca de classificació d'imatges on el nivell d'anotació humà sigui mínim. Per aquesta tasca introduïm un nou vocabulari utilitzant una representació densa de descriptors color-SIFT, i desprès s'investiga com els diferents paràmetres afecten la classificació final. Tot seguit es proposa un mètode par tal d'incorporar informació espacial amb el sistema híbrid, mostrant que la informació de context es de gran ajuda per la classificació d'imatges. Desprès introduïm un nou descriptor de forma que representa la imatge segons la seva forma local i la seva forma espacial, tot junt amb un kernel que incorpora aquesta informació espacial en forma piramidal. La forma es representada per un vector compacte obtenint un descriptor molt adequat per ésser utilitzat amb algorismes d'aprenentatge amb kernels. Els experiments realitzats postren que aquesta informació de forma te uns resultats semblants (i a vegades millors) als descriptors basats en aparença. També s'investiga com diferents característiques es poden combinar per ésser utilitzades en la classificació d'imatges i es mostra com el descriptor de forma proposat juntament amb un descriptor d'aparença millora substancialment la classificació. Finalment es descriu un algoritme que detecta les regions d'interès automàticament durant l'entrenament i la classificació. Això proporciona un mètode per inhibir el fons de la imatge i afegeix invariança a la posició dels objectes dins les imatges. S'ensenya que la forma i l'aparença sobre aquesta regió d'interès i utilitzant els classificadors random forests millora la classificació i el temps computacional. Es comparen els postres resultats amb resultats de la literatura utilitzant les mateixes bases de dades que els autors Aixa com els mateixos protocols d'aprenentatge i classificació. Es veu com totes les innovacions introduïdes incrementen la classificació final de les imatges. / The release of challenging data sets with ever increasing numbers of object categories isforcing the development of image representations that can cope with multiple classes andof algorithms that are efficient in training and testing. This thesis explores the problem ofclassifying images by the object they contain in the case of a large number of categories. We first investigate weather the hybrid combination of a latent generative model with a discriminative classifier is beneficial for the task of weakly supervised image classification.We introduce a novel vocabulary using dense color SIFT descriptors, and then investigate classification performances by optimizing different parameters. A new way to incorporate spatial information within the hybrid system is also proposed showing that contextual information provides a strong support for image classification. We then introduce a new shape descriptor that represents local image shape and its spatial layout, together with a spatial pyramid kernel. Shape is represented as a compactvector descriptor suitable for use in standard learning algorithms with kernels. Experimentalresults show that shape information has similar classification performances and sometimes outperforms those methods using only appearance information. We also investigate how different cues of image information can be used together. Wewill see that shape and appearance kernels may be combined and that additional informationcues increase classification performance. Finally we provide an algorithm to automatically select the regions of interest in training. This provides a method of inhibiting background clutter and adding invariance to the object instance's position. We show that shape and appearance representation over the regions of interest together with a random forest classifier which automatically selects the best cues increases on performance and speed. We compare our classification performance to that of previous methods using the authors'own datasets and testing protocols. We will see that the set of innovations introduced here lead for an impressive increase on performance.

Cette thèse s'intéresse au développement de méthodes automatiques pour la segmentation et la synthèse d'images par résonance magnétique de tumeurs cérébrales. La principale perspective clinique de la segmentation des gliomes est le suivi de la vitesse d'expansion diamétrique dans le but d'adapter les solutions thérapeutiques. A cette fin, la thèse formalise au moyen de modèles graphiques probabilistes des approches de segmentation multi-atlas fondées sur l'appariement de blocs de voxels. Un premier modèle probabiliste prolonge à la segmentation automatique de régions cérébrales pathologiques les approches multi-atlas classiques de segmentation de structures anatomiques. Une approximation de l'étape de marginalisation remplace la notion de fenêtre de recherche locale par un tamisage par atlas et par étiquette. Un modèle de détection de gliomes fondé sur un a priori spatial et des critères de pré-sélection de blocs de voxels permettent d'obtenir des temps de calcul compétitifs malgré un appariement non local. Ce travail est validé et comparé à l'état de l'art sur des bases de données publiques. Un second modèle probabiliste, symétrique au modèle de segmentation, simule des images par résonance magnétique de cas pathologiques, à partir d'une unique segmentation. Une heuristique permet d'estimer le maximum a posteriori et l'incertitude du modèle de synthèse d'image. Un appariement itératif des blocs de voxels renforce la cohérence spatiale des images simulées. Le réalisme des images simulées est évalué avec de vraies IRM et des simulations de l'état de l'art. Le raccordement d'un modèle de croissance de tumeur permet de créer des bases d'images annotées synthétiques. / This thesis focuses on the development of automatic methods for the segmentation and synthesis of brain tumor Magnetic Resonance images. The main clinical perspective of glioma segmentation is growth velocity monitoring for patient therapy management. To this end, the thesis builds on the formalization of multi-atlas patch-based segmentation with probabilistic graphical models. A probabilistic model first extends classical multi-atlas approaches used for the segmentation of healthy brains structures to the automatic segmentation of pathological cerebral regions. An approximation of the marginalization step replaces the concept of local search windows with a stratification with respect to both atlases and labels. A glioma detection model based on a spatially-varying prior and patch pre-selection criteria are introduced to obtain competitive running times despite patch matching being non local. This work is validated and compared to state-of-the-art algorithms on publicly available datasets. A second probabilistic model mirrors the segmentation model in order to synthesize realistic MRI of pathological cases, based on a single label map. A heuristic method allows to solve for the maximum a posteriori and to estimate uncertainty of the image synthesis model. Iterating patch matching reinforces the spatial coherence of synthetic images. The realism of our synthetic images is assessed against real MRI, and against outputs of the state-of-the-art method. The junction of a tumor growth model to the proposed synthesis approach allows to generate databases of annotated synthetic cases.

Neutrino astronomy is expanding into the ultra-high energy (>1017eV) frontier with the use of in-ice detection of Askaryan radio emission from neutrino-induced particle showers. There are already pilot arrays for validating the technology and the next few years will see the planning and construction of IceCube-Gen2, an upgrade to the current neutrino telescope IceCube. This thesis aims to facilitate that planning by providing faster simulations using deep learning surrogate models. Faster simulations could enable proper optimisation of the antenna stations providing better sensitivity and reconstruction of neutrino properties. The surrogates are made for two parts of the end-to-end simulations: the signal generation and the signal propagation. These two steps are the most time-consuming parts of the simulations. The signal propagation is modelled with a standard fully connected neural network whereas for the signal generation a conditional Wasserstein generative adversarial network is used. There are multiple reasons for using these types of models. For both problems the neural networks provide the speed necessary as well as being differentiable -both important factors for optimisation. Generative adversarial networks are used in the signal generation because of the inherent stochasticity in the particle shower development that leads to the Askaryan radio signal. A more standard neural network is used for the signal propagation as it is a regression task. Promising results are obtained for both tasks. The signal propagation surrogate model can predict the parameters of interest at the desired accuracy, except for the travel time which needs further optimisation to reduce the uncertainty from 0.5 ns to 0.1 ns. The signal generation surrogate model predicts the Askaryan emission well for the limited parameter space of hadronic showers and within 5° of the Cherenkov cone. The two models provide a first step and a proof of concept. It is believed that the models can reach the required accuracies with more work.

<p dir="ltr">Both diet and physical activity are known to be risk factors for obesity and chronic diseases such as diabetes and metabolic syndrome. We explore a distance-based approach for clustering daily physical activity time series to find temporal physical activity patterns among U.S. adults (ages 20-65). We further extend this approach to integrate both diet and physical activity, and find joint temporal diet and physical activity patterns. Our experiments indicate that the integration of diet, physical activity, and time has the potential to discover joint patterns with association to health. </p><p dir="ltr">Unsupervised domain adaptive (UDA) person re-identification (re-ID) aims to learn identity information from labeled images in source domains and apply it to unlabeled images in a target domain. We propose a deep learning architecture called Synthesis Model Bank (SMB) to deal with illumination variation in unsupervised person re-ID. From our experiments, the proposed SMB outperforms other synthesis methods on several re-ID benchmarks. </p><p dir="ltr">Recent technology advancement introduced modern high-throughput methodologies such as Unmanned Aerial Vehicles (UAVs) to replace the traditional, labor-intensive phenotyping. For many UAV phenotyping analysis, the first step is to extract the smallest groups of plants called “plots” that have the same genotype. We propose an optimization-based, rotation-adaptive approach for extracting plots in a UAV RGB orthomosaic image. From our experiments, the proposed method achieves better plot extraction accuracy compared to existing approaches, and does not require training data.</p>