No description available.

Feature selection or variable selection is an important step in different machine learning tasks. In a traditional approach, users specify the amount of features, which shall be selected. Afterwards, algorithm select features by using scores like the Joint Mutual Information (JMI). If users do not know the exact amount of features to select, they need to evaluate the full learning chain for different feature counts in order to determine, which amount leads to the lowest training error. To overcome this drawback, we extend the JMI score and mitigate the flaw by introducing a stopping criterion to the selection algorithm that can be specified depending on the learning task. With this, we enable developers to carry out the feature selection task before the actual learning is done. We call our new score Historical Joint Mutual Information (HJMI). Additionally, we compare our new algorithm, using the novel HJMI score, against traditional algorithms, which use the JMI score. With this, we demonstrate that the HJMI-based algorithm is able to automatically select a reasonable amount of features: Our approach delivers results as good as traditional approaches and sometimes even outperforms them, as it is not limited to a certain step size for feature evaluation.

Nous explorons l’intrication d’états mixtes, particulièrement d’états de Hall quantiques, par le biais de l’information mutuelle (MI) et de la négativité logarithmique (LN) fermionique. Cette dernière est une bonne mesure d’intrication pour des états mixtes quantiques car elle ne capture pas de corrélations classiques comme la MI. Nous étudions des géométries tripartites qui contiennent des coins où, en plus de la loi du périmètre standard, l’intrication reçoit une contribution angulaire : le terme de coin. Avec l’entropie d’intrication, ce terme a été étudié pour divers états purs, y compris les états de Hall quantiques entier (IQH), et il a été constaté que la fonction angulaire est presque universelle ; elle ne dépend pas des détails microscopiques de l’état en considération. Nous faisons des prédictions sur la forme du terme de coin de la LN et de la MI en utilisant des propriétés générales. Nous testons numériquement nos prédictions sur des états IQH à différents remplissages et sur différentes géométries en utilisant deux méthodes, une dans l’espace des impulsions et une dans l’espace réel. Dans les états fondamentaux, nous trouvons que les termes de coin de la MI et de la LN suivent également le comportement quasi-universel. À température finie et des coins d’angle π/2, le coefficient de la loi du périmètre et les termes de coin atteignent d’abord un plateau puis décroissent rapidement avec la température. Les effets de température finie sont étudiés davantage en travaillant dans les limites de faibles et fortes températures. / We explore the entanglement of quantum mixed states, with an emphasis on quantum Hall states, via the mutual information (MI) and the fermionic logarithmic negativity (LN). The latter is a good measure of entanglement for quantum mixed states as it does not capture classical correlations, unlike the MI. We study tripartite geometries with corners where in addition to the standard boundary law, the entanglement receives an angle-dependent contribution : the corner term. Using the entanglement entropy, this corner term has been studied for various pure states, including integer quantum Hall (IQH) states, and it was found that the angle-dependent function is almost super-universal; it does not depend on the microscopic details of the state under consideration. First, we make predictions on the form of the corner term for the LN and MI using general properties. Then, we test our predictions numerically on IQH states at different fillings and on different geometries, using two approaches, one in momentum space and one in real space. In groundstates, we find that the corner terms of the MI and LN also follow the quasi-universal behaviour. At finite temperatures and angle π/2, we find that the boundary law coefficient and corner terms first plateau then decay rapidly with temperature. The finite-temperature effects are studied in more details by working in low and high temperature limits.

Diffusion spectrum imaging is a type of diffusion magnetic resonance imaging, capable of capturing very complex tissue structures, but requiring a very large amount of samples in q-space and therefore time.  The purpose of this project was to create and evaluate a new sampling scheme in q-space for diffusion MRI, trying to recreate the ensemble averaged propagator (EAP) with fewer samples without significant loss of quality. The sampling scheme was created by greedily selecting the measurements contributing with the most mutual information. The EAP was then recreated using the sampling scheme and interpolation. The mutual information was approximated using the kernel from a Gaussian process machine learning model.  The project showed limited but promising results on synthetic data, but was highly restricted by the amount of available computational power. Having to resolve to using a lower resolution mesh when calculating the optimal sampling scheme significantly reduced the overall performance.

Les remarquables performances des réseaux de neurones profonds dans de nombreux domaines de l'apprentissage automatique au cours de la dernière décennie soulèvent un certain nombre de questions théoriques. Par exemple, quels mecanismes permettent à ces reseaux, qui ont largement la capacité de mémoriser entièrement les exemples d'entrainement, de généraliser correctement à de nouvelles données, même en l'absence de régularisation explicite ? De telles questions ont fait l'objet d'intenses efforts de recherche ces dernières années, combinant analyses de systèmes simplifiés et études empiriques de propriétés qui semblent être corrélées à la performance de généralisation. Les deux premiers articles présentés dans cette thèse contribuent à cette ligne de recherche. Leur but est de mettre en évidence et d'etudier des mécanismes de biais implicites permettant à de larges modèles de prioriser l'apprentissage de fonctions "simples" et d'adapter leur capacité à la complexité du problème. Le troisième article aborde le problème de l'estimation de information mutuelle en haute, en mettant à profit l'expressivité et la scalabilité des reseaux de neurones profonds. Il introduit et étudie une nouvelle classe d'estimateurs, dont il présente plusieurs applications en apprentissage non supervisé, notamment à l'amélioration des modèles neuronaux génératifs. / The striking empirical success of deep neural networks in machine learning raises a number of theoretical puzzles. For example, why can they generalize to unseen data despite their capacity to fully memorize the training examples? Such puzzles have been the subject of intense research efforts in the past few years, which combine rigorous analysis of simplified systems with empirical studies of phenomenological properties shown to correlate with generalization. The first two articles presented in these thesis contribute to this line of work. They highlight and discuss mechanisms that allow large models to prioritize learning `simple' functions during training and to adapt their capacity to the complexity of the problem. The third article of this thesis addresses the long standing problem of estimating mutual information in high dimension, by leveraging the scalability of neural networks. It introduces and studies a new class of estimators and present several applications in unsupervised learning, especially on enhancing generative models.

No description available.

No description available.

Neurone sind anregbare Zellen, die mit Hilfe von elektrischen Signalen miteinander kommunizieren. Im allgemeinen werden eingehende Signale von den Nervenzellen in einer nichtlinearen Art und Weise verarbeitet. Wie diese Verarbeitung in einer umfassenden und exakten Art und Weise mathematisch beschrieben werden kann, ist bis heute nicht geklärt und ist Gegenstand aktueller Forschung. In dieser Arbeit untersuchen wir die nichtlineare Übertragung und Verarbeitung von Signalen durch stochastische Nervenzellen und wenden dabei zwei unterschiedliche Herangehensweisen an. Im ersten Teil der Arbeit befassen wir uns mit der Frage, auf welche Art und Weise ein Signal mit einer bekannten Zeitabhängigkeit die Rate der neuronalen Aktivität beeinflusst. Im zweiten Teil der Arbeit widmen wir uns der Rekonstruktion eingehender Signale aus der durch sie hervorgerufenen neuronalen Aktivität und beschäftigen uns mit der Abschätzung der übertragenen Informationsmenge. Die Ergebnisse dieser Arbeit demonstrieren, wie die etablierten linearen Theorien, die die Modellierung der neuronalen Aktivitätsrate bzw. die Rekonstruktion von Signalen beschreiben, um Beiträge höherer Ordnung erweitert werden können. Einen wichtigen Beitrag dieser Arbeit stellt allerdings auch die Darstellung der Signifikanz der nichtlinearen Theorien dar. Die nichtlinearen Beiträge erweisen sich nicht nur als schwache Korrekturen zu den etablierten linearen Theorien, sondern beschreiben neuartige Effekte, die durch die linearen Theorien nicht erfasst werden können. Zu diesen Effekten gehört zum Beispiel die Anregung von harmonischen Oszillationen der neuronalen Aktivitätsrate und die Kodierung von Signalen in der signalabhängigen Varianz einer Antwortvariablen. / Neurons are excitable cells which communicate with each other via electrical signals. In general, these signals are processed by the Neurons in a nonlinear fashion, the exact mathematical description of which is still an open problem in neuroscience. In this thesis, the broad topic of nonlinear signal processing is approached from two directions. The first part of the thesis is devoted to the question how input signals modulate the neural response. The second part of the thesis is concerned with the nonlinear reconstruction of input signals from the neural output and with the estimation of the amount of the transmitted information. The results of this thesis demonstrate how existing linear theories can be extended to capture nonlinear contributions of the signal to the neural response or to incorporate nonlinear correlations into the estimation of the transmitted information. More importantly, however, our analysis demonstrates that these extensions do not merely provide small corrections to the existing linear theories but can account for qualitatively novel effects which are completely missed by the linear theories. These effects include, for example, the excitation of harmonic oscillations in the neural firing rate or the estimation of information for systems with a signal-dependent output variance.

Many individuals with severe motor impairments are cognitively capable, but because of their physical impairments, unable to express their intention through conventional means of communication. Access technologies are devices that attempt to translate the intention of these individuals into functional activity by harnessing their residual physical or physiological abilities. The primary objective of this thesis was to design and develop a novel non-invasive and non-contact access technology based on infrared thermal imaging. This access technology translates the local temperature change associated with voluntary mouth opening to activation of a binary switch such as a mouse click or key press. To this end, an algorithm based on motion and temperature analyses, and morphological and anthropometric filters was designed to detect mouth opening activity in thermal video in real-time. The secondary objective of this thesis was to introduce a mutual information measure for objective assessment of binary switch users’ performance. A model was suggested, in which combination of cognitive and physical abilities of the human user of a binary access switch constitute a communication channel. The proposed mutual information measure estimates the rate of information transmission in the ‘human communication channel’ during stimulus response tasks. Using this measure, in a study with ten able-bodied participants, the infrared thermal switch was validated against a conventional chin switch. Impairments in body functions and structures that may contraindicate the use of the infrared thermal switch were explored in a study with seven clients, with severe disabilities. Potential hard and soft technological solutions to mitigate the effect of these impairments on infrared thermal switch use were recommended. Finally the infrared thermal switch was tailored to meet the needs of a young man with severe spastic quadriplegic cerebral palsy, who had no other means of physical access.

Many individuals with severe motor impairments are cognitively capable, but because of their physical impairments, unable to express their intention through conventional means of communication. Access technologies are devices that attempt to translate the intention of these individuals into functional activity by harnessing their residual physical or physiological abilities. The primary objective of this thesis was to design and develop a novel non-invasive and non-contact access technology based on infrared thermal imaging. This access technology translates the local temperature change associated with voluntary mouth opening to activation of a binary switch such as a mouse click or key press. To this end, an algorithm based on motion and temperature analyses, and morphological and anthropometric filters was designed to detect mouth opening activity in thermal video in real-time. The secondary objective of this thesis was to introduce a mutual information measure for objective assessment of binary switch users’ performance. A model was suggested, in which combination of cognitive and physical abilities of the human user of a binary access switch constitute a communication channel. The proposed mutual information measure estimates the rate of information transmission in the ‘human communication channel’ during stimulus response tasks. Using this measure, in a study with ten able-bodied participants, the infrared thermal switch was validated against a conventional chin switch. Impairments in body functions and structures that may contraindicate the use of the infrared thermal switch were explored in a study with seven clients, with severe disabilities. Potential hard and soft technological solutions to mitigate the effect of these impairments on infrared thermal switch use were recommended. Finally the infrared thermal switch was tailored to meet the needs of a young man with severe spastic quadriplegic cerebral palsy, who had no other means of physical access.