Video retrieval represents a challenging problem with many caveats and sub-problems. This thesis focuses on two of these sub-problems, namely shot transition detection and text-based search. In the case of shot detection, many solutions have been proposed over the last decades. Recently, deep learning-based approaches improved the accuracy of shot transition detection using 3D convolutional architectures and artificially created training data, but one hundred percent accuracy is still an unreachable ideal. In this thesis we present a deep network for shot transition detection TransNet V2 that reaches state-of- the-art performance on respected benchmarks. In the second case of text-based search, deep learning models projecting textual query and video frames into a joint space proved to be effective for text-based video retrieval. We investigate these query representation learning models in a setting of known-item search and propose improvements for the text encoding part of the model. 1

Dans ce manuscrit, nous étudions la diffusion d'information dans les réseaux sociaux en ligne. Des sites comme Facebook ou Twitter sont en effet devenus aujourd'hui des media d'information à part entière, sur lesquels les utilisateurs échangent de grandes quantités de données. La plupart des modèles existant pour expliquer ce phénomène de diffusion sont des modèles génératifs, basés sur des hypothèses fortes concernant la structure et la dynamique temporelle de la diffusion d'information. Nous considérerons dans ce manuscrit le problème de la prédiction de diffusion dans le cas où le graphe social est inconnu, et où seules les actions des utilisateurs peuvent être observées. - Nous proposons, dans un premier temps, une méthode d'apprentissage du modèle independent cascade consistant à ne pas prendre en compte la dimension temporelle de la diffusion. Des résultats expérimentaux obtenus sur des données réelles montrent que cette approche permet d'obtenir un modèle plus performant et plus robuste. - Nous proposons ensuite plusieurs méthodes de prédiction de diffusion reposant sur des technique d'apprentissage de représentations. Celles-ci nous permettent de définir des modèles plus compacts, et plus robustes à la parcimonie des données. - Enfin, nous terminons en appliquant une approche similaire au problème de détection de source, consistant à retrouver l'utilisateur ayant lancé une rumeur sur un réseau social. En utilisant des méthodes d'apprentissage de représentations, nous obtenons pour cette tâche un modèle beaucoup plus rapide et performant que ceux de l'état de l'art. / In this thesis, we study information diffusion in online social networks. Websites like Facebook or Twitter have indeed become information medias, on which users create and share a lot of data. Most existing models of the information diffusion phenomenon relies on strong hypothesis about the structure and dynamics of diffusion. In this document, we study the problem of diffusion prediction in the context where the social graph is unknown and only user actions are observed. - We propose a learning algorithm for the independant cascades model that does not take time into account. Experimental results show that this approach obtains better results than time-based learning schemes. - We then propose several representations learning methods for this task of diffusion prediction. This let us define more compact and faster models. - Finally, we apply our representation learning approach to the source detection task, where it obtains much better results than graph-based approaches.

En raison de ses enjeux sociétaux, économiques et culturels, l’intelligence artificielle (dénotée IA) est aujourd’hui un sujet d’actualité très populaire. L’un de ses principaux objectifs est de développer des systèmes qui facilitent la vie quotidienne de l’homme, par le biais d’applications telles que les robots domestiques, les robots industriels, les véhicules autonomes et bien plus encore. La montée en popularité de l’IA est fortement due à l’émergence d’outils basés sur des réseaux de neurones profonds qui permettent d’apprendre simultanément, la représentation des données (qui était traditionnellement conçue à la main), et la tâche à résoudre (qui était traditionnellement apprise à l’aide de modèles d’apprentissage automatique). Ceci résulte de la conjonction des avancées théoriques, de la capacité de calcul croissante ainsi que de la disponibilité de nombreuses données annotées. Un objectif de longue date de l’IA est de concevoir des machines inspirées des humains, capables de percevoir le monde, d’interagir avec les humains, et tout ceci de manière évolutive (c’est `a dire en améliorant constamment la capacité de perception du monde et d’interaction avec les humains). Bien que l’IA soit un domaine beaucoup plus vaste, nous nous intéressons dans cette thèse, uniquement à l’IA basée apprentissage (qui est l’une des plus performante, à ce jour). Celle-ci consiste `a l’apprentissage d’un modèle qui une fois appris résoud une certaine tâche, et est généralement composée de deux sous-modules, l’un représentant la donnée (nommé ”représentation”) et l’autre prenant des décisions (nommé ”résolution de tâche”). Nous catégorisons, dans cette thèse, les travaux autour de l’IA, dans les deux approches d’apprentissage suivantes : (i) Spécialisation : apprendre des représentations à partir de quelques tâches spécifiques dans le but de pouvoir effectuer des tâches très spécifiques (spécialisées dans un certain domaine) avec un très bon niveau de performance; ii) Universalité : apprendre des représentations à partir de plusieurs tâches générales dans le but d’accomplir autant de tâches que possible dansdifférents contextes. Alors que la spécialisation a été largement explorée par la communauté de l’apprentissage profond, seules quelques tentatives implicites ont été réalisée vers la seconde catégorie, à savoir, l’universalité. Ainsi, le but de cette thèse est d’aborder explicitement le problème de l’amélioration de l’universalité des représentations avec des méthodes d’apprentissage profond, pour les données d’image et de texte. [...] / Because of its key societal, economic and cultural stakes, Artificial Intelligence (AI) is a hot topic. One of its main goal, is to develop systems that facilitates the daily life of humans, with applications such as household robots, industrial robots, autonomous vehicle and much more. The rise of AI is highly due to the emergence of tools based on deep neural-networks which make it possible to simultaneously learn, the representation of the data (which were traditionally hand-crafted), and the task to solve (traditionally learned with statistical models). This resulted from the conjunction of theoretical advances, the growing computational capacity as well as the availability of many annotated data. A long standing goal of AI is to design machines inspired humans, capable of perceiving the world, interacting with humans, in an evolutionary way. We categorize, in this Thesis, the works around AI, in the two following learning-approaches: (i) Specialization: learn representations from few specific tasks with the goal to be able to carry out very specific tasks (specialized in a certain field) with a very good level of performance; (ii) Universality: learn representations from several general tasks with the goal to perform as many tasks as possible in different contexts. While specialization was extensively explored by the deep-learning community, only a few implicit attempts were made towards universality. Thus, the goal of this Thesis is to explicitly address the problem of improving universality with deep-learning methods, for image and text data. We have addressed this topic of universality in two different forms: through the implementation of methods to improve universality (“universalizing methods”); and through the establishment of a protocol to quantify its universality. Concerning universalizing methods, we proposed three technical contributions: (i) in a context of large semantic representations, we proposed a method to reduce redundancy between the detectors through, an adaptive thresholding and the relations between concepts; (ii) in the context of neural-network representations, we proposed an approach that increases the number of detectors without increasing the amount of annotated data; (iii) in a context of multimodal representations, we proposed a method to preserve the semantics of unimodal representations in multimodal ones. Regarding the quantification of universality, we proposed to evaluate universalizing methods in a Transferlearning scheme. Indeed, this technical scheme is relevant to assess the universal ability of representations. This also led us to propose a new framework as well as new quantitative evaluation criteria for universalizing methods.

No description available.

In the last decade, machine learning models have increased in size and amount of data they are using, which has led to improved performance on many tasks. Most notably, there has been a significant development in end-to-end deep learning and reinforcement learning models with new learning algorithms and architectures proposed frequently. Furthermore, while previous methods were focused on supervised learning, in the last five years, many models were proposed that learn in semi-supervised or self-supervised ways. The model is then fine-tuned to a specific task or different data domain. Adapting machine learning models learned on one type of data to similar but different data is called domain adaptation. This thesis discusses various challenges in the domain adaptation of machine learning models to specific tasks and real-world applications and proposes solutions for those challenges. Data in real-world applications have different properties than clean machine-learning datasets commonly used for the experimental evaluation of proposed models. Learning appropriate representations from high-dimensional complex data with internal dependencies is arduous due to the curse of dimensionality and spurious correlation. However, most real-world data have these properties in addition to a small number of labeled samples since labeling is expensive and tedious. Additionally, accuracy drops drastically if models are applied to domain-specific datasets and unbalanced problems. Moreover, state-of-the-art models are not able to handle missing data. In this thesis, I strive to create frameworks that can learn a good representation of high-dimensional small data with correlations between variables. The first chapter of this thesis describes the motivation, background, and research objectives. It also gives an overview of contributions and publications. A background needed to understand this thesis is provided in the second chapter and an introduction to domain adaptation is described in chapter three. The fourth chapter discusses domain adaptation with small target data. It describes the algorithm for semi-supervised learning over domain-specific short texts such as reviews or tweets. The proposed framework achieves up to 12.6% improvement when only 5000 labeled examples are available. The fifth chapter explores the influence of unanticipated bias in fine-tuning data. This chapter outlines how the bias in news data influences the classification performance of domain-specific text, where the domain is U.S. politics. It is shown that fine-tuning with domain-specific data is not always beneficial, especially if bias towards one label is present. The sixth chapter examines domain adaptation on datasets with high missing rates. It reviews a system created to learn from high-dimensional small data from psychological studies, which have up to 70% missingness. The proposed framework is achieving 9.3% smaller imputation and 33% lower prediction errors. The seventh chapter discusses the curse of dimensionality problem in domain adaptation. It presents a methodology for discovering research articles containing evolutionary timetrees. That system can search for, download, and filter research articles in which timetrees are imported. It scans 5 million articles in a few days. The proposed method also decreases the error of finding research papers by 21% compared to the baseline, which cannot work with high-dimensional data properly. The last, eighth chapter, summarizes the findings of this thesis and suggests future prospects. / Computer and Information Science

<p>Representation learning algorithms, as the cutting edge of modern AIs, has shown their ability to automatically solve complex tasks in diverse fields including computer vision, speech recognition, autonomous driving, biology. Unsurprisingly, representation learning applications in computer networking domains, such as network management, video streaming, traffic forecasting, are enjoying increasing interests in recent years. However, the success of representation learning algorithms is based on consistency between training and test data distribution, which can not be guaranteed in some scenario due to resource limitation, privacy or other infrastructure reasons. Caused by distribution shift in training and test data, representation learning algorithms have to apply tuned models into environments whose data distribution are solidly different from the model training. This issue is addressed as Out-Of-Distribution (OOD) Generalization, and is still an open topic in machine learning. In this dissertation, I present solutions for OOD cases found in cloud services which will be beneficial to improve user experience. First, I implement Infinity SGD which can extrapolate from light-load server log to predict server performance under heavy-load. Infinity SGD builds the bridge between light-load and heavy-load server status through modeling server status under different loads by an unified Continuous Time Markov Chain (CTMC) of same parameters. I show that Infinity SGD can perform extrapolations that no precedent works can do on real-world testbed and synthetic experiments. Next, I propose Veritas, a framework to answer what will be the user experience if a different ABR, a kind of video streaming data transfer algorithm, was used with the same server, client and connection status. Veritas strictly follows Structural Causal Model (SCM) which guarantees its power to answer what-if counterfactual and interventional questions for video streaming. I showcase that Veritas can accurately answer confounders for what-if questions on real-world emulations where on existing works can. Finally, I propose time-then-graph, a provable more expressive temporal graph neural network (TGNN) than precedent works. We empirically show that time-then-graph is a more efficient and accurate framework on forecasting traffic on network data which will serve as an essential input data for Infinity SGD. Besides, paralleling with this dissertation, I formalize Knowledge Graph (KG) as doubly exchangeable attributed graph. I propose a doubly exchangeable representation blueprint based on the formalization which enables a complex logical reasoning task with no precedent works. This work may also find potential traffic classification applications in networking field.</p>

This thesis is divided into two distinct chapters. In the first chapter, we apply network representation learning to the field of materials science in order to predict aluminum grain boundaries' properties and locate the most influential atoms and subgraphs within each grain boundary. We create fixed-length representations of the aluminum grain boundaries that successfully capture grain boundary structure and allow us to accurately predict grain boundary energy. We do this through two distinct methods. The first method we use is a graph convolutional neural network, a semi-supervised deep learning algorithm, and the second method is graph2vec, an unsupervised representation learning algorithm. The second chapter presents our dynamic global value chain network, the combination of the dynamic global supply chain network and the dynamic global strategic alliance network. Our global value chain network provides a level of scope and accessibility not found in any other global value chain network, commercial or academic. Through applications of network theory, we discover business applications that would increase the robustness and resilience of the global value chain. We accomplish this through an analysis of the static, dynamic, and community structure of our global value chain network.

Enterprises use live video streaming as a mean of communication. Streaming high-quality video to thousands of devices in a corporate network is not an easy task; the bandwidth requirements often exceed the network capacity. For that matter, Peer-To-Peer (P2P) networks have been proven beneficial, as peers can exchange content efficiently by utilizing the topology of the corporate network. However, such networks are dynamic and their topology might not always be known. In this project we propose ABD, a new dynamic graph representation learning approach, which aims to estimate the bandwidth capacity between peers in a corporate network. The architecture of ABDis adapted to the properties of corporate networks. The model is composed of an attention mechanism and a decoder. The attention mechanism produces node embeddings, while the decoder converts those embeddings into bandwidth predictions. The model aims to capture both the dynamicity and the structure of the dynamic network, using an advanced training process. The performance of ABD is tested with two dynamic graphs which were produced by real corporate networks. Our results show that ABD achieves better results when compared to existing state-of-the-art dynamic graph representation learning models. / Företag använder live video streaming för både intern och extern kommunikation. Strömmning av hög kvalitet video till tusentals tittare i ett företagsnätverk är inte enkelt eftersom bandbreddskraven ofta överstiger kapaciteten på nätverket. För att minska lasten på nätverket har Peer-to-Peer (P2P) nätverk visat sig vara en lösning. Här anpassar sig P2P nätverket efter företagsnätverkets struktur och kan därigenom utbyta video data på ett effektivt sätt. Anpassning till ett företagsnätverk är ett utmanande problem eftersom dom är dynamiska med förändring över tid och kännedom över topologin är inte alltid tillgänglig. I det här projektet föreslår vi en ny lösning, ABD, en dynamisk approach baserat på inlärning av grafrepresentationer. Vi försöker estimera den bandbreddskapacitet som finns mellan två peers eller tittare. Architekturen av ABD anpassar sig till egenskaperna av företagsnätverket. Själva modellen bakom ABD använder en koncentrationsmekanism och en avkodare. Attention mekanismen producerar node embeddings, medan avkodaren konverterar embeddings till estimeringar av bandbredden. Modellen fångar upp dynamiken och strukturen av nätverket med hjälp av en avancerad träningsprocess. Effektiviteten av ABD är testad på två dynamiska nätverksgrafer baserat på data från riktiga företagsnätverk. Enligt våra experiment har ABD bättre resultat när man jämför med andra state-of the-art modeller för inlärning av dynamisk grafrepresentation.

Neural networks trained using gradient-based optimization methods exhibit a surprising phenomenon known as mode connectivity, where two independently trained network weights are not isolated low loss minima in the parameter space. Instead, they can be connected by simple curves along which the loss remains low. In case of linear mode connectivity up to permutation, even linear interpolations of the trained weights incur low loss when networks that differ by permutation of their hidden neurons are considered equivalent. While some recent research suggest that this implies existence of a single near-convex loss basin to which the parameters converge, others have empirically shown distinct basins corresponding to different strategies to solve the task. In some settings, averaging multiple network weights naively, without explicitly accounting for permutation invariance still results in a network with improved generalization. In this thesis, linear mode connectivity among a set of neural networks independently trained on labelled datasets, both naively and upon reparameterization to account for permutation invariance is studied. Specifically, the effect of hidden layer width on the connectivity is empirically evaluated. The experiments are conducted on a two dimensional toy classification problem, and the insights are extended to deeper networks trained on handwritten digits and images. It is argued that accounting for permutation of hidden neurons either explicitly or implicitly is necessary for weight averaging to improve test performance. Furthermore, the results indicate that the training dynamics induced by the optimization plays a significant role, and large model width alone may not be a sufficient condition for linear model connectivity. / Neurala nätverk som tränats med gradientbaserade optimeringsmetoder uppvisar ett överraskande fenomen som kallas modeconnectivity, där två oberoende tränade nätverksvikter inte är isolerade lågförlustminima i parameterutrymmet. Istället kan de kopplas samman med enkla kurvor längs vilka förlusten förblir låg. I händelse av linjär mode-anslutning upp till permutation medför även linjära interpolationer av de tränade vikterna låga förluster när nätverk som skiljer sig åt genom permutation av deras dolda neuroner anses vara likvärdiga. Medan en del nyare undersökningar tyder på att detta innebär att det finns en enda nära-konvex förlustbassäng till vilken parametrarna konvergerar, har andra empiriskt visat distinkta bassänger som motsvarar olika strategier för att lösa uppgiften. I vissa inställningar resulterar ett naivt medelvärde av flera nätverksvikter, utan att uttryckligen ta hänsyn till permutationsinvarians, fortfarande i ett nätverk med förbättrad generalisering. I den här avhandlingen studeras linjärmodsanslutningar mellan en uppsättning neurala nätverk som är oberoende tränade på märkta datamängder, både naivt och vid omparameterisering för att ta hänsyn till permutationsinvarians. Specifikt utvärderas effekten av dold lagerbredd på anslutningen empiriskt. Experimenten utförs på ett tvådimensionellt leksaksklassificeringsproblem, och insikterna utökas till djupare nätverk som tränas på handskrivna siffror och bilder. Det hävdas att redogörelse för permutation av dolda neuroner antingen explicit eller implicit är nödvändigt för viktgenomsnitt för att förbättra testprestanda. Dessutom indikerar resultaten att träningsdynamiken som induceras av optimeringen spelar en betydande roll, och enbart stor modellbredd kanske inte är ett tillräckligt villkor för linjär modellanslutning.

Dans l'apprentissage par renforcement profond basé sur la valeur avec des mémoires de relecture, le paramètre de taille de lot joue un rôle crucial en déterminant le nombre de transitions échantillonnées pour chaque mise à jour de gradient. Étonnamment, malgré son importance, ce paramètre n'est généralement pas ajusté lors de la proposition de nouveaux algorithmes. Dans ce travail, nous menons une vaste étude empirique qui suggère que la réduction de la taille des lots peut entraîner un certain nombre de gains de performances significatifs ; ceci est surprenant et contraire à la pratique courante consistant à utiliser de plus grandes tailles de lots pour améliorer la formation du réseau neuronal. Ce résultat inattendu défie la sagesse conventionnelle et appelle à une compréhension plus approfondie des gains de performances observés associés à des tailles de lots plus petites. Pour faire la lumière sur les facteurs sous-jacents, nous complétons nos résultats expérimentaux par une série d'analyses empiriques. Ces analyses approfondissent divers aspects du processus d'apprentissage, tels que l'analyse de la dynamique d'optimisation du réseau, la vitesse de convergence, la stabilité et les capacités d'exploration. Le chapitre 1 présente les concepts nécessaires pour comprendre le travail présenté, notamment des aperçus de l'Apprentissage Profond (Deep Learning) et de l'Apprentissage par Renforcement (Reinforcement Learning). Le chapitre 2 contient une description détaillée de nos contributions visant à comprendre les gains de performance observés associés à des tailles de lots plus petites lors de l'utilisation d'algorithmes d'apprentissage par renforcement profond basés sur la valeur. À la fin, des conclusions tirées de ce travail sont fournies, incluant des suggestions pour des travaux futurs. Le chapitre 3 aborde ce travail dans le contexte plus large de la recherche en apprentissage par renforcement. / In value-based deep reinforcement learning with replay memories, the batch size parameter plays a crucial role by determining the number of transitions sampled for each gradient update. Surprisingly, despite its importance, this parameter is typically not adjusted when proposing new algorithms. In this work, we conduct a broad empirical study that suggests {\em reducing} the batch size can result in a number of significant performance gains; this is surprising and contrary to the prevailing practice of using larger batch sizes to enhance neural network training. This unexpected result challenges the conventional wisdom and calls for a deeper understanding of the observed performance gains associated with smaller batch sizes. To shed light on the underlying factors, we complement our experimental findings with a series of empirical analyses such as analysis of network optimization dynamics, convergence speed, stability, and exploration capabilities. Chapter 1 introduces concepts necessary to understand the work presented, including overviews of Deep Learning and Reinforcement Learning. Chapter 2 contains a detailed description of our contributions towards understanding the observed performance gains associated with smaller batch sizes when using value based deep reinforcement learning algorithms. At the end, some conclusions drawn from this work are provided, including some exciting suggestion as future work. Chapter 3 talks about this work in the broader context of reinforcement learning research.