Ce travail a été développé dans le cadre du projet européen FACETS-ITN, dans le domaine des Neurosciences Computationnelles. Son but est d’améliorer la compréhension des réseaux de neurones stochastiques de taille finie, pour des sources corrélées à caractère aléatoire et pour des matrices de connectivité biologiquement réalistes. Ce résultat est obtenu par l’analyse de la matrice de corrélation du réseau et la quantification de la capacité de codage du système en termes de son information de Fisher. Les méthodes comprennent diverses techniques mathématiques, statistiques et numériques, dont certaines ont été importés d’autres domaines scientifiques, comme la physique et la théorie de l’estimation. Ce travail étend de précédents résultats fondées sur des hypothèses simplifiées qui ne sont pas réaliste d’un point de vue biologique et qui peuvent être pertinents pour la compréhension des principes de travail liés cerveau. De plus, ce travail fournit les outils nécessaires à une analyse complète de la capacité de traitement de l’information des réseaux de neurones, qui sont toujours manquante dans la communauté scientifique. / The brain is the most complex system in the known universe. Its nested structure with small-world properties determines its function and behavior. The analysis of its structure requires sophisticated mathematical and statistical techniques. In this thesis we shed new light on neural networks, attacking the problem from different points of view, in the spirit of the Theory of Complexity and in terms of their information processing capabilities. In particular, we quantify the Fisher information of the system, which is a measure of its encoding capability. The first technique developed in this work is the mean-field theory of rate and FitzHugh-Nagumo networks without correlations in the thermodynamic limit, through both mathematical and numerical analysis. The second technique, the Mayer’s cluster expansion, is taken from the physics of plasma, and allows us to determine numerically the finite size effects of rate neurons, as well as the relationship of the Fisher information to the size of the network for independent Brownian motions. The third technique is a perturbative expansion, which allows us to determine the correlation structure of the rate network for a variety of different types of connectivity matrices and for different values of the correlation between the sources of randomness in the system. With this method we can also quantify numerically the Fisher information not only as a function of the network size, but also for different correlation structures of the system. The fourth technique is a slightly different type of perturbative expansion, with which we can study the behavior of completely generic connectivity matrices with random topologies. Moreover this method provides an analytic formula for the Fisher information, which is in qualitative agreement with the other results in this thesis. Finally, the fifth technique is purely numerical, and uses an Expectation-Maximization algorithm and Monte Carlo integration in order to evaluate the Fisher information of the FitzHugh-Nagumo network. In summary, this thesis provides an analysis of the dynamics and the correlation structure of the neural networks, confirms this through numerical simulation and makes two key counterintuitive predictions. The first is the formation of a perfect correlation between the neurons for particular values of the parameters of the system, a phenomenon that we term stochastic synchronization. The second, which is somewhat contrary to received opinion, is the explosion of the Fisher information and therefore of the encoding capability of the network for highly correlated neurons. The techniques developed in this thesis can be used also for a complete quantification of the information processing capabilities of the network in terms of information storage, transmission and modification, but this would need to be performed in the future.

Artificial Neural Networks (ANNs) can be viewed as a mathematical model to simulate natural and biological systems on the basis of mimicking the information processing methods in the human brain. The capability of current ANNs only focuses on approximating arbitrary deterministic input-output mappings. However, these ANNs do not adequately represent the variability which is observed in the systems’ natural settings as well as capture the complexity of the whole system behaviour. This thesis addresses the development of a new class of neural networks called Stochastic Neural Networks (SNNs) in order to simulate internal stochastic properties of systems. Developing a suitable mathematical model for SNNs is based on canonical representation of stochastic processes or systems by means of Karhunen-Loève Theorem. Some successful real examples, such as analysis of full displacement field of wood in compression, confirm the validity of the proposed neural networks. Furthermore, analysis of internal workings of SNNs provides an in-depth view on the operation of SNNs that help to gain a better understanding of the simulation of stochastic processes by SNNs.

Artificial Neural Networks (ANNs) can be viewed as a mathematical model to simulate natural and biological systems on the basis of mimicking the information processing methods in the human brain. The capability of current ANNs only focuses on approximating arbitrary deterministic input-output mappings. However, these ANNs do not adequately represent the variability which is observed in the systems' natural settings as well as capture the complexity of the whole system behaviour. This thesis addresses the development of a new class of neural networks called Stochastic Neural Networks (SNNs) in order to simulate internal stochastic properties of systems. Developing a suitable mathematical model for SNNs is based on canonical representation of stochastic processes or systems by means of Karhunen-Loéve Theorem. Some successful real examples, such as analysis of full displacement field of wood in compression, confirm the validity of the proposed neural networks. Furthermore, analysis of internal workings of SNNs provides an in-depth view on the operation of SNNs that help to gain a better understanding of the simulation of stochastic processes by SNNs.