In this thesis, we consider the planning problem for first-order mean-field games (MFG). These games degenerate into optimal transport when there is no coupling between players. Our aim is to extend the concept of displacement convexity from optimal transport to MFGs. This extension gives new estimates for solutions of MFGs. First, we introduce the Monge-Kantorovich problem and examine related results on rearrangement maps. Next, we present the concept of displacement convexity. Then, we derive first-order MFGs, which are given by a system of a Hamilton-Jacobi equation coupled with a transport equation. Finally, we identify a large class of functions, that depend on solutions of MFGs, which are convex in time. Among these, we find several norms. This convexity gives bounds for the density of solutions of the planning problem.

Measure of distance between two probability distributions plays a fundamental role in statistics and machine learning. Optimal Transport (OT) theory provides such distance. Recent advance in OT theory is a generalization of classical OT with entropy regularized, called entropic OT. Despite its convenience in computation, it still lacks theoretical support. In this thesis, we study the connection between entropic OT and Entropy Power Inequality (EPI). First, we prove an HWI-type inequality making use of the infinitesimal displacement convexity of OT map. Second, we derive two Talagrand-type inequalities using the saturation of EPI that corresponds to a numerical term in our expression. We evaluate for a wide variety of distributions this term whereas for Gaussian and i.i.d. Cauchy distributions this term is found in explicit form. We show that our results extend previous results of Gaussian Talagrand inequality for Sinkhorn distance to the strongly log-concave case. Furthermore, we observe a dimensional measure concentration phenomenon using the new Talagrand-type inequality. / Mått på avstånd mellan två sannolikhetsfördelningar spelar en grundläggande roll i statistik och maskininlärning. Optimal transport (OT) teori ger ett sådant avstånd. Nyligen framsteg inom OT-teorin är en generalisering av klassisk OT med entropi-reglerad, kallad entropisk OT. Trots dess bekvämlighet i beräkning saknar det fortfarande teoretiskt stöd. I denna avhandling studerar vi sambandet mellan entropisk OT och Entropy Power Inequality (EPI). Först bevisar vi en ojämlikhet av HWI-typ med användning av OT-kartans oändliga förskjutningskonvexitet. För det andra härleder vi två Talagrand-typkvaliteter med mättnaden av EPI som motsvarar ett numeriskt uttryck vårt uttryck. Vi utvärderar för ett brett utbud av distributioner den här termen för Gauss och i.i.d. Cauchy-distributioner denna term finns oförklarlig form. Vi visar att våra resultat utökar tidigare resultat av GaussianTalagrand-ojämlikhet för Sinkhorn-avstånd till det starkt log-konkava fallet. Dessutom observerar vi ett dimensionellt mått koncentrationsfenomen mot den nya Talagrand-typen ojämlikhet.

Inom reglerteknik har integrationen av maskininlärningsmetoder framträtt som en central strategi för att förbättra prestanda och adaptivitet hos styrsystem. Betydande framsteg har gjorts inom flera viktiga aspekter av reglerkretsen, såsom inlärningsbaserade metoder för systemidentifiering och parameterskattning, filtrering och brusreducering samt reglersyntes. Denna avhandling fördjupar sig i området inlärning för reglerteknik med särskild betoning på inlärningsbaserade regulatorer och identifieringsmetoder.  Avhandlingens första del behandlar undersökningen av neuronnätsbaserad Modellprediktiv Reglering (MPC). Olika nätstrukturer studeras, både generella black box-nät och nät som väver in MPC-specifik information i sin struktur. Dessa nät jämförs och utvärderas med avseende på två prestandamått genom experiment på realistiska två- och fyrdimensionella system. Den huvudsakliga nyskapande aspekten är inkluderingen av gradientdata i träningsprocessen, vilket visar sig förbättra noggrannheten av de genererade styrsignalerna. Vidare påvisar de experimentella resultaten att en MPC-informerad nätstruktur leder till förbättrad prestanda när mängden träningsdata är begränsad.  Med insikt om vikten av noggranna matematiska modeller av styrsystemet, riktar den andra delen av avhandlingen sitt fokus mot inlärningsbaserade identifieringsmetoder. Denna forskningsgren behandlar karakterisering och modellering av dynamiska system med hjälp av maskininlärning. Avhandlingen bidrar till området genom att introducera kooperativa systemidentifieringsmetoder för att förbättra parameterskattningen. Specifikt utnyttjas verktyg från Optimal Transport för att introducera en ny och mer generell formulering av ramverket Correctional Learning. Detta ramverk är baserat på en mästare-lärlingsmodell, där en expertagent (mästare) observerar och modifierar den insamlade data som används av en lärande agent (lärling), med syftet att förbättra lärlingens skattningsprocess. Genom att formulera correctional learning som ett optimal transport-problem erhålls ett mer flexibelt ramverk, bättre lämpat för skattning av komplexa systemegenskaper samt anpassning till alternativa handlingsstrategier. / In the context of control systems, the integration of machine learning mechanisms has emerged as a key approach for improving performance and adaptability. Notable progress has been made across several aspects of the control loop, including learning-based techniques for system identification and estimation, filtering and denoising, and controller design. This thesis delves into the rapidly expanding domain of learning in control, with a particular focus placed on learning-based controllers and learning-based identification methods. The first part of this thesis is devoted to the investigation of Neural Network approximations of Model Predictive Control (MPC). Model-agnostic neural network structures are compared to networks employing MPC-specific information, and evaluated in terms of two performance metrics. The main novel aspect lies in the incorporation of gradient data in the training process, which is shown to enhance the accuracy of the network generated control inputs. Furthermore, experimental results reveal that MPC-informed networks outperform the agnostic counterparts in scenarios when training data is limited. In acknowledgement of the crucial role accurate system models play in in the control loop, the second part of this thesis lends its focus to learning-based identification methods. This line of work addresses the important task of characterizing and modeling dynamical systems, by introducing cooperative system identification techniques to enhance estimation performance. Specifically, it presents a novel and generalized formulation of the Correctional Learning framework, leveraging tools from Optimal Transport. The correctional learning framework centers around a teacher-student model, where an expert agent (teacher) modifies the sampled data used by the learner agent (student), to improve the student's estimation process. By formulating correctional learning as an optimal transport problem, a more adaptable framework is achieved, better suited for estimating complex system characteristics and accommodating alternative intervention strategies. / VR 2018-03438 projekt 3224

Cell differentiation is the process of a cell developing from one cell type to another. It is of interest to analyse the differentiation from stem cells to different types of mature cells, and discover what genes are involved in regulating the differentiation to specific cells, for instance to get insights to what is causing certain diseases and find potential treatments.  In this project, two mathematical models are developed for analysing blood cell differentiation (haematopoiesis) with methods based on optimal transportation. Optimal transportation is about moving one mass distribution to another at minimal cost. Modelling a sample of cells as point masses placed in a space based on the cells' gene expressions, accessed by single-cell RNA sequencing, optimal transportation is used to find transitions between cells that costs the least in terms of changes in gene expression. With this, cell-to-cell trajectories, from haematopoietic stem cells to mature blood cells, are obtained. With the first model, cells are divided into groups based on their maturity, which is determined by using diffusion pseudotime, and optimal transportation is preformed between groups. The resulting trajectories suggest that haematopoietic stem cells possibly can develop into the same mature cell type in different ways, and that the cell fate for some cell types is decided late on in development. In future work, the gene regulation along the obtained trajectories can be analysed. The second model is developed to be more general than the first, and not be dependent on a group division before preforming optimal transportation. / Celldifferentiering är processen då en cell utvecklas från en celltyp till en annan. Det är av intresse att analysera differentieringen från stamcell till olika typer av mogna celler, och undersöka vilka gener som har betydelse i regleringen av differentieringen till specifika celler, bland annat för att få en inblick i vad som orsakar vissa sjukdomar och hitta potentiella botemedel. I detta projekt utvecklas två matematiska modeller för att analysera blodcellsutveckling (hematopoes) med metoder som är baserade på optimal transport. Optimal transport handlar om att förflytta en massfördelning till en annan till lägst kostnad. Genom att modellera celler som punktmassor, placerade i ett rum baserat på cellernas genuttryck som fås genom singel-cell RNA-sekvensering, används optimal transport för att hitta förflyttningar mellan celler som kostar minst i termer av förändringar i genuttryck. Från detta skapas vägar mellan celler, från hematopoetiska stamceller till mogna celler. I den första modellen delas cellerna upp i grupper baserat på deras mognadsgrad, som bestäms genom att använda pseudotid baserad på en diffusionsavbildning, och optimal transport används sedan mellan grupperna. De resulterande vägarna visar på att hematopoetiska stamceller möjligen kan utvecklas till samma typ av mogen cell på olika sätt, och att cellödet för vissa typer av celler bestäms sent i utvecklingen. I framtida arbete kan genregleringen längs de funna vägarna analyseras. Den andra modellen utvecklas för att vara mer generell än den första, och inte bero på en gruppuppdelning innan optimal transport används.

Almost every company around the world were aected by the COVID-19 crisis and the government measures that were taken to slow the spread of the virus. The impact the crisis had on the economy caused the appearance of anomalies in the data collected by companies : such as abnormal trend, seasonality etc. Traditional methods of forecasting were then questioned when trying to predict business indicators such as sales in a post COVID19 world, as we saw performance like forecast accuracy decreased. So how can data scientists increase the performance of their forecasting models in a post COVID-19 world knowing that the training data contains COVID-19, an event never observed before? What methods can be used to overcome this problem? The goal of this project was to provide a guideline for dealing with COVID-19 data points for forecasters. We first dedicated this thesis to data analysis and finding a clear methodology to better understand and quantify the impact of COVID-19 crisis on business indicators. Then, we compared multiple methods to overcome the forecasting issues that are faced when training datasets influenced by the phenomenon of COVID-19 and improved forecast accuracy and reduce bias. Each method had its pros and cons. Among the methods changing the training data, imputation is the easiest method and can give very good results. Multiplicative coecients also can be used, and give also good results. Finally, optimal transport was tested as an alternative to the two first methods. This method changes less the original the time series compared to imputation. Among methods consisting in adding external features to the model, a boolean feature is the most simple way to flag a COVID-19 period and works surprisingly well. Adding more complex features describing COVID-19 impact on the time series is challenging since we need to find a feature that describes well the phenomenon and be able to use another model to predict its future values if we want to use it for our first model. Adding Google mobility features to the model as external regressors seem to increase the most forecast accuracy, but its performance depends on how well we can estimate their future values. This applies also to stringency index, but predicting stringency index future values is even harder as we are trying to estimate government measures. However, with the Stringency index we can simulate scenarios if we make a hypothesis on future government measures: we can estimate COVID-19 impact on the time series in a worst case scenario with lockdowns by setting the Stringency index high for instance. / Nästan alla företag runt om i världen drabbades av covid-19-krisen och de statliga åtgärder som har vidtagits för att bromsa spridningen av viruset. Krisens inverkan på ekonomin orsakade uppkomsten av anomalier i data som samlats in av företag: onormal trend, säsongsvariationer ... etc. Traditionella metoder för prognoser ifrågasattes sedan när man försökte förutsäga aärsindikatorer som försäljning i en värld efter covid-19, eftersom vi såg att prestanda som prognosnoggrannhet minskade. Så hur kan dataforskare öka prestandan för sina prognosmodeller i en värld efter covid-19 med vetskapen om att träningsdata innehåller covid-19, en händelse som aldrig tidigare observerats? Vilka metoder kan användas för att övervinna detta problem? Målet med detta projekt var att ge en riktlinje för hantering av covid-19-datapunkter för prognosmakare. Vi dedikerade först denna avhandling till dataanalys och att hitta en tydlig metod för att bättre förstå och kvantifiera eekten av covid-19- krisen på aärsindikatorer. Sedan jämförde vi flera metoder för att övervinna problemet med den COVID-19-påverkade träningsdatauppsättningen och förbättrad prognosnoggrannhet och minskad bias. Varje metod hade sina för- och nackdelar. Bland metoderna för att ändra träningsdata är imputering den enklaste metoden och kan ge mycket goda resultat. Multiplikativa koecienter kan också användas och ger också bra resultat. Slutligen undersöktes en ny metod: optimal transport, och kan vara ett alternativ till imputering. Med denna metod är den ursprungliga formen på tidsseriekurvan lite mer bevarad, så viss information i originaldata är fortfarande användbar för modellen. Bland de externa funktioner som lagts till i modellen är den booleska funktionen det enklaste sättet att flagga en covid-19-period och fungerar förvånansvärt bra. Googles mobilitetsfunktioner är de externa regressorer som verkar öka mest prognosnoggrannhet, men det beror på hur väl vi kan uppskatta deras framtida värden. Detta gäller även stringensindex, men ännu svårare då vi försöker skatta statliga åtgärder. Stringensindex kan användas för att simulera scenarier (värsta scenario med låsningar, bästa fall där allt är öppet).

Fokker-Planck equations, along with stochastic differential equations, play vital roles in physics, population modeling, game theory and optimization (finite or infinite dimensional). In this thesis, we study three topics, both theoretically and computationally, centered around them. In part one, we consider the optimal transport for finite discrete states, which are on a finite but arbitrary graph. By defining a discrete 2-Wasserstein metric, we derive Fokker-Planck equations on finite graphs as gradient flows of free energies. By using dynamical viewpoint, we obtain an exponential convergence result to equilibrium. This derivation provides tools for many applications, including numerics for nonlinear partial differential equations and evolutionary game theory. In part two, we introduce a new stochastic differential equation based framework for optimal control with constraints. The framework can efficiently solve several real world problems in differential games and Robotics, including the path-planning problem. In part three, we introduce a new noise model for stochastic oscillators. With this model, we prove global boundedness of trajectories. In addition, we derive a pair of associated Fokker-Planck equations.

Lors de l’apprentissage d’un modèle de classification pour un nouveau domaine cible avec seulement une petite quantité d’échantillons de formation, l’application des algorithmes d’apprentissage automatiques conduit généralement à des classifieurs surdimensionnés avec de mauvaises compétences de généralisation. D’autre part, recueillir un nombre suffisant d’échantillons de formation étiquetés manuellement peut s’avérer très coûteux. Les méthodes de transfert d’apprentissage visent à résoudre ce type de problèmes en transférant des connaissances provenant d’un domaine source associé qui contient beaucoup plus de données pour faciliter la classification dans le domaine cible. Selon les différentes hypothèses sur le domaine cible et le domaine source, l’apprentissage par transfert peut être classé en trois catégories: apprentissage par transfert inductif, apprentissage par transfert transducteur (adaptation du domaine) et apprentissage par transfert non surveillé. Nous nous concentrons sur le premier qui suppose que la tâche cible et la tâche source sont différentes mais liées. Plus précisément, nous supposons que la tâche cible et la tâche source sont des tâches de classification, tandis que les catégories cible et les catégories source sont différentes mais liées. Nous proposons deux méthodes différentes pour aborder ce problème. Dans le premier travail, nous proposons une nouvelle méthode d’apprentissage par transfert discriminatif, à savoir DTL(Discriminative Transfer Learning), combinant une série d’hypothèses faites à la fois par le modèle appris avec les échantillons de cible et les modèles supplémentaires appris avec des échantillons des catégories sources. Plus précisément, nous utilisons le résidu de reconstruction creuse comme discriminant de base et améliore son pouvoir discriminatif en comparant deux résidus d’un dictionnaire positif et d’un dictionnaire négatif. Sur cette base, nous utilisons des similitudes et des dissemblances en choisissant des catégories sources positivement corrélées et négativement corrélées pour former des dictionnaires supplémentaires. Une nouvelle fonction de coût basée sur la statistique de Wilcoxon-Mann-Whitney est proposée pour choisir les dictionnaires supplémentaires avec des données non équilibrées. En outre, deux processus de Boosting parallèles sont appliqués à la fois aux distributions de données positives et négatives pour améliorer encore les performances du classificateur. Sur deux bases de données de classification d’images différentes, la DTL proposée surpasse de manière constante les autres méthodes de l’état de l’art du transfert de connaissances, tout en maintenant un temps d’exécution très efficace. Dans le deuxième travail, nous combinons le pouvoir du transport optimal (OT) et des réseaux de neurones profond (DNN) pour résoudre le problème ITL. Plus précisément, nous proposons une nouvelle méthode pour affiner conjointement un réseau de neurones avec des données source et des données cibles. En ajoutant une fonction de perte du transfert optimal (OT loss) entre les prédictions du classificateur source et cible comme une contrainte sur le classificateur source, le réseau JTLN (Joint Transfer Learning Network) proposé peut effectivement apprendre des connaissances utiles pour la classification cible à partir des données source. En outre, en utilisant différents métriques comme matrice de coût pour la fonction de perte du transfert optimal, JTLN peut intégrer différentes connaissances antérieures sur la relation entre les catégories cibles et les catégories sources. Nous avons effectué des expérimentations avec JTLN basées sur Alexnet sur les jeux de données de classification d’image et les résultats vérifient l’efficacité du JTLN proposé. A notre connaissances, ce JTLN proposé est le premier travail à aborder ITL avec des réseaux de neurones profond (DNN) tout en intégrant des connaissances antérieures sur la relation entre les catégories cible et source. / When learning a classification model for a new target domain with only a small amount of training samples, brute force application of machine learning algorithms generally leads to over-fitted classifiers with poor generalization skills. On the other hand, collecting a sufficient number of manually labeled training samples may prove very expensive. Transfer Learning methods aim to solve this kind of problems by transferring knowledge from related source domain which has much more data to help classification in the target domain. Depending on different assumptions about target domain and source domain, transfer learning can be further categorized into three categories: Inductive Transfer Learning, Transductive Transfer Learning (Domain Adaptation) and Unsupervised Transfer Learning. We focus on the first one which assumes that the target task and source task are different but related. More specifically, we assume that both target task and source task are classification tasks, while the target categories and source categories are different but related. We propose two different methods to approach this ITL problem. In the first work we propose a new discriminative transfer learning method, namely DTL, combining a series of hypotheses made by both the model learned with target training samples, and the additional models learned with source category samples. Specifically, we use the sparse reconstruction residual as a basic discriminant, and enhance its discriminative power by comparing two residuals from a positive and a negative dictionary. On this basis, we make use of similarities and dissimilarities by choosing both positively correlated and negatively correlated source categories to form additional dictionaries. A new Wilcoxon-Mann-Whitney statistic based cost function is proposed to choose the additional dictionaries with unbalanced training data. Also, two parallel boosting processes are applied to both the positive and negative data distributions to further improve classifier performance. On two different image classification databases, the proposed DTL consistently out performs other state-of-the-art transfer learning methods, while at the same time maintaining very efficient runtime. In the second work we combine the power of Optimal Transport and Deep Neural Networks to tackle the ITL problem. Specifically, we propose a novel method to jointly fine-tune a Deep Neural Network with source data and target data. By adding an Optimal Transport loss (OT loss) between source and target classifier predictions as a constraint on the source classifier, the proposed Joint Transfer Learning Network (JTLN) can effectively learn useful knowledge for target classification from source data. Furthermore, by using different kind of metric as cost matrix for the OT loss, JTLN can incorporate different prior knowledge about the relatedness between target categories and source categories. We carried out experiments with JTLN based on Alexnet on image classification datasets and the results verify the effectiveness of the proposed JTLN in comparison with standard consecutive fine-tuning. To the best of our knowledge, the proposed JTLN is the first work to tackle ITL with Deep Neural Networks while incorporating prior knowledge on relatedness between target and source categories. This Joint Transfer Learning with OT loss is general and can also be applied to other kind of Neural Networks.

Gradient flows of energy functionals on the space of probability measures with Wasserstein metric has proved to be a strong tool in studying certain mass conserving evolution equations. Such gradient flows provide an alternate formulation for the solutions of the corresponding evolution equations. An important condition, which is known to guarantees existence, uniqueness, and continuous dependence on initial data is that the corresponding energy functional be displacement convex. We introduce a relaxed notion of displacement convexity and we show that it still guarantees short time existence and uniqueness of Wasserstein gradient flows for higher order energy functionals which are not displacement convex in the standard sense. This extends the applicability of the gradient flow approach to larger family of energies. As an application, local and global well-posedness of different higher order non-linear evolution equations are derived. Examples include the thin-film equation and the quantum drift diffusion equation in one spatial variable.

We present a numerical method to solve the optimal transport problem with a quadratic cost when the source and target measures are periodic probability densities. This method relies on a numerical resolution of the corresponding Monge-Ampère equation. We use an existing Newton-like algorithm that we generalize to the case of a non uniform final density. The main idea consists of designing an iterative scheme where the fully nonlinear equation is approximated by a non-constant coefficient linear elliptic PDE that we discretize and solve at each iteration, in two different ways: a second order finite difference scheme and a Fourier transform (FT) method. The FT method, made possible thanks to a preconditioning step based on the coefficient-averaged equation, results in an overall O(P LogP )-operations algorithm, where P is the number of discretization points. We prove that the generalized algorithm converges to the solution of the optimal transport problem, under suitable conditions on the initial and final densities. Numerical experiments demonstrating the robustness and efficiency of the method on several examples of image processing, including an application to multiple sclerosis disease detection, are shown. We also demonstrate by numerical tests that the method is competitive against some other methods available.

Gradient flows of energy functionals on the space of probability measures with Wasserstein metric has proved to be a strong tool in studying certain mass conserving evolution equations. Such gradient flows provide an alternate formulation for the solutions of the corresponding evolution equations. An important condition, which is known to guarantees existence, uniqueness, and continuous dependence on initial data is that the corresponding energy functional be displacement convex. We introduce a relaxed notion of displacement convexity and we show that it still guarantees short time existence and uniqueness of Wasserstein gradient flows for higher order energy functionals which are not displacement convex in the standard sense. This extends the applicability of the gradient flow approach to larger family of energies. As an application, local and global well-posedness of different higher order non-linear evolution equations are derived. Examples include the thin-film equation and the quantum drift diffusion equation in one spatial variable.