<p dir="ltr">Continuous optimization is a fundamental topic both in theoretical computer science and applications of machine learning. Meanwhile, an important idea in the development modern algorithms it the use of randomness to achieve empirical speedup and improved theoretical runtimes. Stochastic gradient descent (SGD) and matrix-multiplication time linear program solvers [1] are two important examples of such achievements. Matrix sketching and related ideas provide a theoretical framework for the behavior of random matrices and vectors that arise in these algorithms, thereby provide a natural way to better understand the behavior of such randomized algorithms. In this dissertation, we consider three general problems in this area.</p>

Cette thèse s'inscrit dans la théorie des matrices aléatoires, à l'intersection avec la théorie des probabilités libres et des algèbres d'opérateurs. Elle s'insère dans une démarche générale qui a fait ses preuves ces dernières décennies : importer les techniques et les concepts de la théorie des probabilités non commutatives pour l'étude du spectre de grandes matrices aléatoires. On s'intéresse ici à des généralisations du théorème de liberté asymptotique de Voiculescu. Dans les Chapitres 1 et 2, nous montrons des résultats de liberté asymptotique forte pour des matrices gaussiennes, unitaires aléatoires et déterministes. Dans les Chapitres 3 et 4, nous introduisons la notion de fausse liberté asymptotique pour des matrices déterministes et certaines matrices hermitiennes à entrées sous diagonales indépendantes, interpolant les modèles de matrices de Wigner et de Lévy. / The thesis fits into the random matrix theory, in intersection with free probability and operator algebra. It is part of a general approach which is common since the last decades: using tools and concepts of non commutative probability in order to get general results about the spectrum of large random matrices. Where are interested here in generalization of Voiculescu's asymptotic freeness theorem. In Chapter 1 and 2, we show some results of strong asymptotic freeness for gaussian, random unitary and deterministic matrices. In Chapter 3 and 4, we introduce the notion of asymptotic false freeness for deterministic matrices and certain random matrices, Hermitian with independent sub-diagonal entries, interpolating Wigner and Lévy models.

Dans cette thèse, nous considérons le problème de la localisation de source dans les grands réseaux de capteurs, quand le nombre d'antennes du réseau et le nombre d'échantillons du signal observé sont grands et du même ordre de grandeur. Nous considérons le cas où les signaux source émis sont déterministes, et nous développons un algorithme de localisation amélioré, basé sur la méthode MUSIC. Pour ce faire, nous montrons de nouveaux résultats concernant la localisation des valeurs propres des grandes matrices aléatoires gaussiennes complexes de type information plus bruit / In this thesis, we consider the problem of source localization in large sensor networks, when the number of antennas of the network and the number of samples of the observed signal are large and of the same order of magnitude. We also consider the case where the source signals are deterministic, and we develop an improved algorithm for source localization, based on the MUSIC method. For this, we fist show new results concerning the position of the eigen values of large information plus noise complex gaussian random matrices

Três ensembles, respectivamente relacionados com as distribuições Gaussiana, Lognormal e de Levy, são abordados neste trabalho primordialmente do ponto de vista da termodinâmica de seus espectros. Novas expressões para as grandezas termodinâmicas sao encontradas para os ensembles de Stieltjes e de Bertuola-Pato, e a conexão destes com os ensembles Gaussianos e estabelecida. Esta tese também se compromete com a continuação do desenvolvimento e aprimorarão do ensemble generalizado de Bertuola-Pato, estendendo alguns resultados para os ensembles simplifico e unitário generalizados, alem do ortogonal generalizado já introduzido anteriormente por A. C. Bertuola e M. P. Pato. / Three ensembles, related to the Gaussian, the Lognormal and the L´evy distributions respectively, have been studied in this work and were investigated most of all in what concerns their spectral thermodynamics. New expressions for the thermodynamics quantities were found for the Stieltjes and the Bertuola-Pato ensembles, and the connection with the gaussian ensembles is established. This work concerned with the development continuity and with the improvement of Bertuola-Pato generalized ensemble, extending some of the results to the simplectic and unitary generalized ensembles, besides the orthogonal generalized ensemble introduced before by A. C. Bertuola and M. P. Pato.

Simetrias compõe parte integral da análise na Teoria das Matrizes Aleatórias (RMT). As simetrias de inversão temporal e rotacional são aspectos-chave do Ensemble Gaussiano Ortogonal (GOE), enquanto esta última é quebrada no Ensemble Gaussiano Simplético (GSE) e ambas são quebradas no Conjunto Unitário Gaussiano (GUE). Desde o final da década de 1990, o crescente interesse no campo dos sistemas quânticos PT-simétricos levou os pesquisadores a considerar o efeito, em matrizes aleatórias, dessa classe de simetrias, bem como simetrias pseudo-hermitianas. A principal questão a ser respondida pela pesquisa apresentada nesta tese é se a simetria PT ou, de forma mais geral, a pseudo-Hermiticidade implica alguma distribuição de probabilidade específica para os autovalores. Ou, em outras palavras, se há um aspecto comum transmitido por tal simetria que pode ser usada para modelar alguma classe particular de sistemas físicos. A abordagem inicial considerada consistiu na introdução de um conjunto pseudo-hermitiano, isospectral ao conjunto -Hermite, que apresentaria o tipo de quebra de realidade típico dos sistemas PT-simétricos. Nesse modelo, a primeira abordagem adotada foi a introdução de perturbações que quebraram a realidade dos espectros. Os resultados obtidos permitem concluir que a transformação em seu similar pseudo-hermitiano conduz a um sistema assintoticamente instável. Esse modelo foi extendido ao considerar um pseudo-hermitiano não positivo, que leva a uma quebra similar na realidade dos espectros. Este caso apresenta um comportamento mais próximo do típico dos sistemas PT-simétricos presentes na literatura. Um modelo denso geral baseado em projetores foi proposto, e duas realizações particulares deste modelo receberam atenção mais detalhada. O comportamento espectral também foi similar àquele típico da simetria PT para as duas realizações consideradas, e seus limites assintóticos foram conectados a conjuntos clássicos de teoria de matriz aleatória. Além disso, as propriedades de seus polinômios característicos médios foram obtidas e os limites assintóticos desses polinômios também foram considerados e relacionados a polinômios clássicos. O comportamento estatístico deste conjunto foi estudado e comparado com o destes polinômios. Impor a pseudo-Hermiticidade não parece implicar qualquer distribuição particular de autovalores, sendo a característica comum a quebra da realidade dos autovalores comumente encontrados na literatura de simetria PT. O resultado mais notável dos estudos apresentados nesta tese é o fato de que uma interação pseudo-hermitiana pode ser construída de tal forma que o comportamento espectral médio possa ser controlado calibrando-se o mecanismo de interação, bem como sua intensidade. / The role of symmetries is an integral part of the analysis in Random Matrix Theory (RMT). Time reversal and rotational symmetries are key aspects of the Gaussian Orthogonal Ensemble (GOE), whereas the latter is broken in the Gaussian Sympletic Ensemble (GSE) and both are broken in the Gaussian Unitary Ensemble (GUE). Since the late 1990s, growing interest in the field of PT symmetric quantum systems has led researchers to consider the effect, in random matrices, of this class of symmetries, as well as that of pseudo-Hermitian symmetries. The primary question to be answered by the research presented in this thesis is whether PT-symmetry or, more generally, pseudo-Hermiticity implies some specific probability distribution for the eigenvalues. Or, in other words, whether there is a common aspect imparted by such a symmetry which may be used to model some particular class of physical systems. The initial approach considered consisted of introducing an pseudo-Hermitian ensemble, isospectral to the -Hermite ensemble, which would present the type of reality-breaking typical of PT-symmetrical systems. In this model, the first approach taken was to introduce perturbation which broke the reality of the spectra. The results obtained allow the conclusion that the transformation into its pseudo-Hermitian similar leads into a system which is asymptotically unstable. An extension of this model was to consider a non-positive pseudo-Hermitian , which lead to similar breaking in the reality of the spectra. This case displays behavior closer to that typical of the PT-symmetric systems present in the literature. A general dense projector model was proposed, and two particular realizations of this model were given more detailed attention. The spectral behavior was also similar to that typical of PT-symmetry for the two realizations considered, and their asymptotic limits were shown to connect to classical ensembles of random matrix theory. Furthermore, the properties of their average characteristic polynomials were obtained and the asymptotic limits of these polynomials were also considered and were related to classical polynomials. The statistical behavior of this ensemble was studied and compared to that of these polynomials. Imposing the pseudo-Hermitian does seem not imply any particular eigenvalue distribution, the common feature being the breaking of the reality of the eigenvalues commonly found in PT-symmetry literature. The most notable result of the studies presented herein is the fact that a pseudo-Hermitian interaction may be constructed such that the average spectral behavior may be controlled by calibrating the mechanism of interaction as well as its intensity.

Le sujet principal de cette thèse est la physique des verres de spin. Les verres de spin ont été introduits au début des années 70 pour décrire alliages magnétiques diluées. Ils ont désormais été considerés pour comprendre le comportement de liquides sousrefroidis. Parmis les systèmes qui peuvent être décrits par le langage des systèmes desordonnés, on trouve les problèmes d'optimisation combinatoire. Dans la première partie de cette thèse, nous considérons les modèles de verre de spin avec intéraction de Kac pour investiguer la phase de basse température des liquides sous-refroidis. Dans les chapitres qui suivent, nous montrons comment certaines caractéristiques des modèles de verre de spin peuvent être obtenues à partir de résultats de la théorie des matrices aléatoires en connection avec la statistique des valeurs extrêmes. Dans la dernière partie de la thèse, nous considérons la connexion entre la théorie desverres de spin et la science computationnelle, et présentons un nouvel algorithme qui peut être appliqué à certains problèmes dans le domaine des finances.

Cette thèse, constituée de trois papiers de recherche, est organisée autour de la construction d’indicateurs de crises financières dont les signaux sont ensuite utilisés pour l’élaboration de stratégies de trading algorithmique. Le premier papier traite de l’établissement d’un cadre de travail permettant la construction des indicateurs de crises financière. Le pouvoir de prédiction de nos indicateurs est ensuite démontré en utilisant l’un d’eux pour construire une stratégie de type protective-put active qui est capable de faire mieux en termes de performances qu’une stratégie passive ou, la plupart du temps, que de multiples réalisations d’une stratégie aléatoire. Le second papier va plus loin dans l’application de nos indicateurs de crises à la création de stratégies de trading algorithmique en utilisant le signal combiné d’un grand nombre de nos indicateurs pour gouverner la composition d’un portefeuille constitué d’un mélange de cash et de titres d’un ETF répliquant un indice equity comme le SP500. Enfin, dans le troisième papier, nous construisons des indicateurs de crises financières en utilisant une approche complètement différente. En étudiant l’évolution dynamique de la distribution des spreads des composantes d’un indice CDS tel que l’ITRAXXX Europe 125, une bande de Bollinger est construite autour de la fonction de répartition de la distribution empirique des spreads, exprimée sur une base de deux distributions log-normales choisies à l’avance. Le passage par la fonction de répartition empirique de la frontière haute ou de la frontière basse de cette bande de Bollinger est interprétée en termes de risque et permet de produire un signal de trading. / This thesis is constituted of three research papers and is articulated around the construction of financial crisis indicators, which produce signals, which are then applied to devise successful systematic trading strategies. The first paper deals with the establishment of a framework for the construction of our financial crisis indicators. Their predictive power is then demonstrated by using one of them to build an active protective-put strategy, which is able to beat in terms of performance a passive strategy as well as, most of the time, multiple paths of a random strategy. The second paper goes further in the application of our financial crisis indicators to the elaboration of systematic treading strategies by using the aggregated signal produce by many of our indicators to govern a portfolio constituted of a mix of cash and ETF shares, replicating an equity index like the SP500. Finally, in the third paper, we build financial crisis indicators by using a completely different approach. By studying the dynamics of the evolution of the distribution of the spreads of the components of a CDS index like the ITRAXX Europe 125, a Bollinger band is built around the empirical cumulative distribution function of the distribution of the spreads, fitted on a basis constituted of two lognormal distributions, which have been chosen beforehand. The crossing by the empirical cumulative distribution function of either the upper or lower boundary of this Bollinger band is then interpreted in terms of risk and enables us to construct a trading signal.

L'une des stratégies utilisée pour augmenter l'efficacité spectrale (ES) des réseaux cellulaires est de réutiliser la bande de fréquences sur des zones relativement petites. Le problème majeur dans ce cas est un plus grand niveau d'interférence, diminuant l'efficacité énergétique (EE). En plus d'une plus grande largeur de bande, la densification des réseaux (cellules de petite taille ou multi-utilisateur à entrées multiples et sortie unique, MU-EMSO), peut augmenter l'efficacité spectrale par unité de surface (ESuS). La consommation totale d'énergie des réseaux sans fil augmente en raison de la grande quantité de puissance de circuit consommée par les structures de réseau denses, réduisant l'EE. Dans cette thèse, la région EE-SE est caractérisé dans un réseau cellulaire hexagonal en considérant plusieurs facteurs de réutilisation de fréquences (FRF), ainsi que l'effet de masquage. La région EE-ESuS est étudiée avec des processus de Poisson ponctuels (PPP) pour modéliser un réseau MU-EMSO avec un précodeur à rapport signal sur fuite plus bruit (RSFB). Différentes densités de station de base (SB) et nombre d'antennes aux SB avec une consommation d'énergie statique sont considérées.Nous caractérisons d'abord la région EE-SE dans le réseau cellulaire hexagonal pour différentes FRF, avec et sans masquage. Avec le masquage en plus de la perte de propagation, la mesure de coupure ε-EE-ES est proposée pour évaluer les performances. Les courbes EE-ES présentent une grande partie linéaire, due à la consommation de puissance statique, suivie d'une forte diminution de l'EE, puisque le réseau est homogène et limité par les interférences. Les résultats montrent qu'un FRF de 1 pour les régions proches de la SB et des FRF plus élevés dans la région plus proche du bord de la cellule améliorent le point optimal du EE-ES. De plus, un meilleur compromis EE-ES peut être obtenu avec une valeur plus élevée de coupure. En outre, un FRF de 1 est le meilleur choix pour une valeur élevée de coupure en raison d'une réduction du rapport signal sur interférence plus bruit (RSIB).Les précodeurs sont utilisés en liaison descendante des réseaux cellulaires MU-EMSO à accès multiple par division spatiale (AMDS) pour améliorer le RSIB. La géométrie stochastique a été utilisée intensivement pour analyser de tels systèmes complexes. Nous obtenons une expression analytique de l'ESuS en régime asymptotique, c.-à-d. nombre d'antennes et d'utilisateurs infinis, en utilisant des résultats de matrices aléatoires et de géométrie stochastique. Les SBs et les utilisateurs sont modélisés par deux PPP indépendants et le précodage RSFB est utilisé. L'EE est dérivée d'un modèle de consommation de puissance linéaire. Les simulations de Monte Carlo montrent que les expressions analytiques sont précises même pour un nombre faible d'antennes et d'utilisateurs. De plus, les courbes d'EE-ESuS ont une grande partie linéaire avant une forte décroissante de l'EE, comme pour les réseaux hexagonaux. Les résultats montrent également que le précodeur RSFB offre de meilleurs performances que le précodeur forçage à zéro (FZ), qui est typiquement utilisé dans la literature. Les résultats numériques pour le précodeur RSFB montrent que déployer plus de SBs ou d'antennes aux BSs augmente l'ESuS, mais que le gain dépend du rapport des densités SB-utilisateurs et du nombre d'antennes lorsque la densité de l'utilisateur est fixe. L'EE augmente seulement lorsque l'augmentation de l'ESuS est plus importante que l'augmentation de la consommation d'énergie par unité de surface. D'autre part, lorsque la densité d'utilisateur augmente, l'ESuS dans la région limitée par les interférences peut être améliorée en déployant davantage de SB sans sacrifier l'EE et le débit ergodique des utilisateurs. / One of the used strategies to increase the spectral efficiency (SE) of cellular network is to reuse the frequency bandwidth over relatively small areas. The major issue in this case is higher interference, decreasing the energy efficiency (EE). In addition to the higher bandwidth, densification of the networks (e.g. small cells or multi-user multiple input single output, MU-MISO) potentially increases the area spectral efficiency (ASE). The total energy consumption of the wireless networks increases due to the large amount of circuit power consumed by the dense network structures, leading to the decrease of EE. In this thesis, the EE-SE achievable region is characterized in a hexagonal cellular network considering several frequency reuse factors (FRF), as well as shadowing. The EE-ASE region is also studied using Poisson point processes (PPP) to model the MU-MISO network with signal-to-leakage-and-noise ratio (SLNR) precoder. Different base station (BS) densities and different number of BS antennas with static power consumption are considered.The EE-SE region in a hexagonal cellular network for different FRF, both with and without shadowing is first characterized. When shadowing is considered in addition to the path loss, the ε-SE-EE tradeoff is proposed as an outage measure for performance evaluation. The EE-SE curves have a large linear part, due to the static power consumption, followed by a sharp decreasing EE, since the network is homogeneous and interference-limited. The results show that FRF of 1 for regions close to BS and higher FRF for regions closer to the cell edge improve the EE-SE optimal point. Moreover, better EE-SE tradeoff can be achieved with higher outage values. Besides, FRF of 1 is the best choice for very high outage value due to the significant signal-to-interference-plus-noise ratio (SINR) decrease.In downlink, precoders are used in space division multiple access (SDMA) MU-MISO cellular networks to improve the SINR. Stochastic geometry has been intensively used to analyse such a complex system. A closed-form expression for ASE in asymptotic regime, i.e. number of antennas and number of users grow to infinity, has been derived using random matrix theory and stochastic geometry. BSs and users are modeled by two independent PPP and SLNR precoder is used at BS. EE is then derived from a linear power consumption model. Monte Carlo simulations show that the analytical expressions are tight even for moderate number of antennas and users. Moreover, the EE-ASE curves have a large linear part before a sharply decreasing EE, as observed for hexagonal network. The results also show that SLNR outperforms the zero-foring (ZF) precoder, which is typically used in literature. Numerical results for SLNR show that deploying more BS or a large number of BS antennas increase ASE, but the gain depends on the BS-user density ratio and on the number of antennas when user density is fixed. EE increases only when the increase in ASE dominates the increase of the power consumption per unit area. On the other hand, when the user density increases, ASE in interference-limited region can be improved by deploying more BS without sacrificing EE and the ergodic rate of the users.

Três ensembles, respectivamente relacionados com as distribuições Gaussiana, Lognormal e de Levy, são abordados neste trabalho primordialmente do ponto de vista da termodinâmica de seus espectros. Novas expressões para as grandezas termodinâmicas sao encontradas para os ensembles de Stieltjes e de Bertuola-Pato, e a conexão destes com os ensembles Gaussianos e estabelecida. Esta tese também se compromete com a continuação do desenvolvimento e aprimorarão do ensemble generalizado de Bertuola-Pato, estendendo alguns resultados para os ensembles simplifico e unitário generalizados, alem do ortogonal generalizado já introduzido anteriormente por A. C. Bertuola e M. P. Pato. / Three ensembles, related to the Gaussian, the Lognormal and the L´evy distributions respectively, have been studied in this work and were investigated most of all in what concerns their spectral thermodynamics. New expressions for the thermodynamics quantities were found for the Stieltjes and the Bertuola-Pato ensembles, and the connection with the gaussian ensembles is established. This work concerned with the development continuity and with the improvement of Bertuola-Pato generalized ensemble, extending some of the results to the simplectic and unitary generalized ensembles, besides the orthogonal generalized ensemble introduced before by A. C. Bertuola and M. P. Pato.

At a first glance, artificial neural networks, with engineered learning algorithms and carefully chosen nonlinearities, are nothing like the complicated self-organized spiking neural networks studied by theoretical neuroscientists. Yet, both adapt to their inputs, keep information from the past in their state space and are able of learning, implying that some information processing principles should be common to both. In this thesis we study those principles by incorporating notions of systems theory, statistical physics and graph theory into artificial neural networks and theoretical neuroscience models. % TO DO: What is different in this thesis? -> classical signal processing with complex systems on top The starting point for this thesis is \ac{RC}, a learning paradigm used both in machine learning\cite{jaeger2004harnessing} and in theoretical neuroscience\cite{maass2002real}. A neural network in \ac{RC} consists of two parts, a reservoir – a directed and weighted network of neurons that projects the input time series onto a high dimensional space – and a readout which is trained to read the state of the neurons in the reservoir and combine them linearly to give the desired output. In classical \ac{RC}, the reservoir is randomly initialized and left untrained, which alleviates the training costs in comparison to other recurrent neural networks. However, this lack of training implies that reservoirs are not adapted to specific tasks and thus their performance is often lower than that of other neural networks. Our contribution has been to show how knowledge about a task can be integrated into the reservoir architecture, so that reservoirs can be tailored to specific problems without training. We do this design by identifying two features that are useful for machine learning: the memory of the reservoir and its power spectra. First we show that the correlations between neurons limit the capacity of the reservoir to retain traces of previous inputs, and demonstrate that those correlations are controlled by moduli of the eigenvalues of the adjacency matrix of the reservoir. Second, we prove that when the reservoir resonates at the frequencies that are present on the desired output signal, the performance of the readout increases. Knowing the features of the reservoir dynamics that we need, the next question is how to impose them. The simplest way to design a network with that resonates at a certain frequency is by adding cycles, which act as feedback loops, but this also induces correlations and hence memory modifications. To disentangle the frequencies and the memory design, we studied how the addition of cycles modifies the eigenvalues in the adjacency matrix of the network. Surprisingly, the shape of the eigenvalues is quite beautiful \cite{aceituno2019universal} and can be characterized using random matrix theory tools. Combining this knowledge with our result relating eigenvalues and correlations, we designed an heuristic that tailors reservoirs to specific tasks and showed that it improves upon state of the art \ac{RC} in three different machine learning tasks. Although this idea works in the machine learning version of \ac{RC}, there is one fundamental problem when we try to translate to the world of theoretical neuroscience: the proposed frequency adaptation requires prior knowledge of the task, which might not be plausible in a biological neural network. Therefore the following questions are whether those resonances can emerge by unsupervised learning, and which kind of learning rules would be required. Remarkably, these resonances can be induced by the well-known Spike Time-Dependent Plasticity (STDP) combined with homeostatic mechanisms. We show this by deriving two self-consistent equations: one where the activity of every neuron can be calculated from its synaptic weights and its external inputs and a second one where the synaptic weights can be obtained from the neural activity. By considering spatio-temporal symmetries in our inputs we obtained two families of solutions to those equations where a periodic input is enhanced by the neural network after STDP. This approach shows that periodic and quasiperiodic inputs can induce resonances that agree with the aforementioned \ac{RC} theory. Those results, although rigorous, are expressed on a language of statistical physics and cannot be easily tested or verified in real, scarce data. To make them more accessible to the neuroscience community we showed that latency reduction, a well-known effect of STDP\cite{song2000competitive} which has been experimentally observed \cite{mehta2000experience}, generates neural codes that agree with the self-consistency equations and their solutions. In particular, this analysis shows that metabolic efficiency, synchronization and predictions can emerge from that same phenomena of latency reduction, thus closing the loop with our original machine learning problem. To summarize, this thesis exposes principles of learning recurrent neural networks that are consistent with adaptation in the nervous system and also improve current machine learning methods. This is done by leveraging features of the dynamics of recurrent neural networks such as resonances and correlations in machine learning problems, then imposing the required dynamics into reservoir computing through control theory notions such as feedback loops and spectral analysis. Then we assessed the plausibility of such adaptation in biological networks, deriving solutions from self-organizing processes that are biologically plausible and align with the machine learning prescriptions. Finally, we relate those processes to learning rules in biological neurons, showing how small local adaptations of the spike times can lead to neural codes that are efficient and can be interpreted in machine learning terms.