No description available.

Internal Symmetry Networks are a recently developed class of Cellular Neural Network inspired by the phenomenon of internal symmetry in quantum physics. Their hidden unit activations are acted on non-trivially by the dihedral group of symmetries of the square. Here, we extend Internal Symmetry Networks to include recurrent connections, and train them by backpropagation to perform a variety of image processing tasks, smoothing, sharpening, edge detection, synthetic image segmentation, texture segmentation and object recognition. By a large number of experiments, we find some guidelines to construct appropriate configurations of the net for different tasks.

Internal Symmetry Networks are a recently developed class of Cellular Neural Network inspired by the phenomenon of internal symmetry in quantum physics. Their hidden unit activations are acted on non-trivially by the dihedral group of symmetries of the square. Here, we extend Internal Symmetry Networks to include recurrent connections, and train them by backpropagation to perform a variety of image processing tasks, smoothing, sharpening, edge detection, synthetic image segmentation, texture segmentation and object recognition. By a large number of experiments, we find some guidelines to construct appropriate configurations of the net for different tasks.

Artificial neural networks are widely used for parallel processing of data analysis and visual information. The most prominent example of artificial neural networks is a cellular neural network (CNN), composed from two-dimensional arrays of simple first-order dynamical systems (“cells”) that are interconnected by wires. The information, to be processed by a CNN, represents the initial state of the network, and the parallel information processing is performed by converging to one of the stable spatial equilibrium states of the multi-stable CNN. This thesis studies a specific type of CNNs designed to perform the winner-take-all function of finding the largest among the n numbers, using the network dynamics. In a wider context, this amounts to automatically detecting a target spot in the given visual picture. The research, reported in this thesis, demonstrates that the addition of fast on-off switching (blinking) connections significantly improves the functionality of winner-take-all CNNs. Numerical calculations are performed to reveal the dependence of the probability, that the CNN correctly classifies the largest number, on the switching frequency.

Les réseaux de neurones artificiels sont nés, il y a presque cinquante ans, de la volonté de modéliser les capacités de mémorisation et de traitement du cerveau biologique. Aujourd'hui encore, les nombreux modèles obtenus brillent par leur simplicité de mise en œuvre, leur puissance de traitement, leur polyvalence, mais aussi par la complexité des méthodes de programmation disponibles. En réalité, très peu d'entre-elles sont capables d'aboutir analytiquement à un réseau de neurones correctement configuré. Bien au contraire, la plupart se " contentent " d'ajuster, petit à petit, une ébauche de réseau de neurones, jusqu'à ce qu'il fonctionne avec suffisamment d'exemples de la tâche à accomplir. Au travers de ces méthodes, dites " d'apprentissages ", les réseaux de neurones sont devenus des boîtes noires, que seuls quelques experts sont effectivement capables de programmer. Chaque traitement demande en effet de choisir convenablement une configuration initiale, la nature des exemples, leur nombre, l'ordre d'utilisation, ... Pourtant, la tâche finalement apprise n'en reste pas moins le résultat d'une stratégie algorithmique implémentée par le réseau de neurones. Une stratégie qui peut donc être identifiée par le biais de l'analyse, et surtout réutilisée lors de la conception d'un réseau de neurones réalisant une tâche similaire, court-circuitant ainsi les nombreux aléas liés à ces méthodes d'apprentissage. Les bénéfices de l'analyse sont encore plus évidents dans le cas de réseaux de neurones à sortie binaire. En effet, le caractère discret des signaux traités simplifie grandement l'identification des mécanismes mis en jeu, ainsi que leur contribution au traitement global. De ce type d'analyse systématique naît un formalisme original, qui décrit la stratégie implémentée par les réseaux de neurones à sortie binaire de façon particulièrement efficace. Schématiquement, ce formalisme tient lieu d'" état intermédiaire " entre la forme boîte noire d'un réseau de neurones et sa description mathématique brute. En étant plus proche des modèles de réseaux de neurones que ne l'est cette dernière, il permet de retrouver, par synthèse analytique, un réseau de neurones effectuant la même opération que celui de départ, mais de façon optimisée selon un ou plusieurs critères : nombre de neurones, nombre de connexions, dynamique de calcul, etc. Cette approche analyse-formalisation-synthèse constitue la contribution de ces travaux de thèse.

In this thesis, the geometric margin is used for the first time as the robustness indicator of an uncoupled cellular neural network implementing a given Boolean function. First, robust template design for uncoupled cellular neural networks implementing linearly separable Boolean functions by support vector machines is proposed. A fast sequential minimal optimization algorithm is presented to find maximal margin classifiers, which in turn determine the robust templates. Some general properties of robust templates are investigated. An improved CFC algorithm implementing an arbitrarily given Boolean function is proposed. Two illustrative examples are provided to demonstrate the validity of the proposed method.

<p>Deep ‘Analog Artificial Neural Networks’ (AANNs) perform complex classification problems with high accuracy. However, they rely on humongous amount of power to perform the calculations, veiling the accuracy benefits. The biological brain on the other hand is significantly more powerful than such networks and consumes orders of magnitude less power, indicating some conceptual mismatch. Given that the biological neurons are locally connected, communicate using energy efficient trains of spikes, and the behavior is non-deterministic, incorporating these effects in Artificial Neural Networks (ANNs) may drive us few steps towards a more realistic neural networks. </p> <p> </p> <p>Emerging devices can offer a plethora of benefits including power efficiency, faster operation, low area in a vast array of applications. For example, memristors and Magnetic Tunnel Junctions (MTJs) are suitable for high density, non-volatile Random Access Memories when compared with CMOS implementations. In this work, we analyze the possibility of harnessing the characteristics of such emerging devices, to achieve neuro-inspired solutions to intricate problems.</p> <p> </p> <p>We propose how the inherent stochasticity of nano-scale resistive devices can be utilized to realize the functionality of spiking neurons and synapses that can be incorporated in deep stochastic Spiking Neural Networks (SNN) for image classification problems. While ANNs mainly dwell in the aforementioned classification problem solving domain, they can be adapted for a variety of other applications. One such neuro-inspired solution is the Cellular Neural Network (CNN) based Boolean satisfiability solver. Boolean satisfiability (k-SAT) is an NP-complete (k≥3) problem that constitute one of the hardest classes of constraint satisfaction problems. We provide a proof of concept hardware based analog k-SAT solver that is built using MTJs. The inherent physics of MTJs, enhanced by device level modifications, is harnessed here to emulate the intricate dynamics of an analog, CNN based, satisfiability (SAT) solver. </p> <p> </p> <p>Furthermore, in the effort of reaching human level performance in terms of accuracy, increasing the complexity and size of ANNs is crucial. Efficient algorithms for evaluating neural network performance is of significant importance to improve the scalability of networks, in addition to designing hardware accelerators. We propose a scalable approach for evaluating Liquid State Machines: a bio-inspired computing model where the inputs are sparsely connected to a randomly interlinked reservoir (or liquid). It has been shown that biological neurons are more likely to be connected to other neurons in the close proximity, and tend to be disconnected as the neurons are spatially far apart. Inspired by this, we propose a group of locally connected neuron reservoirs, or an ensemble of liquids approach, for LSMs. We analyze how the segmentation of a single large liquid to create an ensemble of multiple smaller liquids affects the latency and accuracy of an LSM. In our analysis, we quantify the ability of the proposed ensemble approach to provide an improved representation of the input using the Separation Property (SP) and Approximation Property (AP). Our results illustrate that the ensemble approach enhances class discrimination (quantified as the ratio between the SP and AP), leading to improved accuracy in speech and image recognition tasks, when compared to a single large liquid. Furthermore, we obtain performance benefits in terms of improved inference time and reduced memory requirements, due to lower number of connections and the freedom to parallelize the liquid evaluation process.</p>