A Chip Multiprocessor for a Large-scale Neural SimulatorEustace PainkrasA thesis submitted to The University of Manchesterfor the degree of Doctor of Philosophy, 17 December 2012The modelling and simulation of large-scale spiking neural networks in biologicalreal-time places very high demands on computational processing capabilities andcommunications infrastructure. These demands are difficult to satisfy even with powerfulgeneral-purpose high-performance computers. Taking advantage of the remarkableprogress in semiconductor technologies it is now possible to design and buildan application-driven platform to support large-scale spiking neural network simulations.This research investigates the design and implementation of a power-efficientchip multiprocessor (CMP) which constitutes the basic building block of a spikingneural network modelling and simulation platform. The neural modelling requirementsof many processing elements, high-fanout communications and local memoryare addressed in the design and implementation of the low-level modules in the designhierarchy as well as in the CMP. By focusing on a power-efficient design, the energyconsumption and related cost of SpiNNaker, the massively-parallel computation engine,are kept low compared with other state-of-the-art hardware neural simulators.The SpiNNaker CMP is composed of many simple power-efficient processors withsmall local memories, asynchronous networks-on-chip and numerous bespoke modulesspecifically designed to serve the demands of neural computation with a globallyasynchronous, locally synchronous (GALS) architecture.The SpiNNaker CMP, realised as part of this research, fulfills the demands of neuralsimulation in a power-efficient and scalable manner, with added fault-tolerancefeatures. The CMPs have, to date, been incorporated into three versions of SpiNNakersystem PCBs with up to 48 chips onboard. All chips on the PCBs are performing successfully, during both functional testing and their targeted role of neural simulation.

No description available.

<p>QC 20160428</p>

Online monitoring applications requiring advanced pattern recognition capabilities implemented in resource-constrained wireless sensor systems are challenging to construct using standard digital computers. An interesting alternative solution is to use a low-power neuromorphic processor like the ROLLS, with subthreshold mixed analog/digital circuits and online learning capabilities that approximate the behavior of real neurons and synapses. This requires that the monitoring algorithm is implemented with spiking neural networks, which in principle are efficient computational models for tasks such as pattern recognition. In this work, I investigate how spiking neural networks can be used as a pre-processing and feature learning system in a condition monitoring application where the vibration of a machine with healthy and faulty rolling-element bearings is considered. Pattern recognition with spiking neural networks is investigated using simulations with Brian -- a Python-based open source toolbox -- and an implementation is developed for the ROLLS neuromorphic processor. I analyze the learned feature-response properties of individual neurons. When pre-processing the input signals with a neuromorphic cochlea known as the AER-EAR system, the ROLLS chip learns to classify the resulting spike patterns with a training error of less than 1 %, at a combined power consumption of approximately 30 mW. Thus, the neuromorphic hardware system can potentially be realized in a resource-constrained wireless sensor for online monitoring applications.However, further work is needed for testing and cross validation of the feature learning and pattern recognition networks.i

Artificial neural network simulators are a research field which attracts the interest of researchers from various fields, from biology to computer science. The final objectives are the understanding of the mechanisms underlying the human brain, how to reproduce them in an artificial environment, and how drugs interact with them. Multiple neural models have been proposed, each with their peculiarities, from the very complex and biologically realistic Hodgkin-Huxley neuron model to the very simple 'leaky integrate-and-fire' neuron. However, despite numerous attempts to understand the learning behaviour of the synapses, few models have been proposed. Spike-Timing-Dependent Plasticity (STDP) is one of the most relevant and biologically plausible models, and some variants (such as the triplet-based STDP rule) have been proposed to accommodate all biological observations. The research presented in this thesis focuses on a novel learning rule, based on the spike-pair STDP algorithm, which provides a statistical approach with the advantage of being less computationally expensive than the standard STDP rule, and is therefore suitable for its implementation on stand-alone computational units. The environment in which this research work has been carried out is the SpiNNaker project, which aims to provide a massively parallel computational substrate for neural simulation. To support such research, two other topics have been addressed: the first is a way to inject spikes into the SpiNNaker system through a non-real-time channel such as the Ethernet link, synchronising with the timing of the SpiNNaker system. The second research topic is focused on a way to route spikes in the SpiNNaker system based on populations of neurons. The three topics are presented in sequence after a brief introduction to the SpiNNaker project. Future work could include structural plasticity (also known as synaptic rewiring); here, during the simulation of neural networks on the SpiNNaker system, axons, dendrites and synapses may be grown or pruned according to biological observations.

Inférence et apprentissage dans les réseaux de neurones profonds nécessitent une grande quantité de calculs qui, dans beaucoup de cas, limite leur intégration dans les environnements limités en ressources. Les réseaux de neurones évènementiels de type « spike » présentent une alternative aux réseaux de neurones artificiels classiques, et promettent une meilleure efficacité énergétique. Cependant, entraîner les réseaux spike demeure un défi important, particulièrement dans le cas où l’apprentissage doit être exécuté sur du matériel de calcul bio-inspiré, dit matériel neuromorphique. Cette thèse constitue une étude sur les algorithmes d’apprentissage et le codage de l’information dans les réseaux de neurones spike.A partir d’une règle d’apprentissage bio-inspirée, nous analysons quelles propriétés sont nécessaires dans les réseaux spike pour rendre possible un apprentissage embarqué dans un scénario d’apprentissage continu. Nous montrons qu’une règle basée sur le temps de déclenchement des neurones (type « spike-timing dependent plasticity ») est capable d’extraire des caractéristiques pertinentes pour permettre une classification d’objets simples comme ceux des bases de données MNIST et N-MNIST.Pour dépasser certaines limites de cette approche, nous élaborons un nouvel outil pour l’apprentissage dans les réseaux spike, SpikeGrad, qui représente une implémentation entièrement évènementielle de la rétro-propagation du gradient. Nous montrons comment cette approche peut être utilisée pour l’entrainement d’un réseau spike qui est capable d’inférer des relations entre valeurs numériques et des images MNIST. Nous démontrons que cet outil est capable d’entrainer un réseau convolutif profond, qui donne des taux de reconnaissance d’image compétitifs avec l’état de l’art sur les bases de données MNIST et CIFAR10. De plus, SpikeGrad permet de formaliser la réponse d’un réseau spike comme celle d’un réseau de neurones artificiels classique, permettant un entraînement plus rapide.Nos travaux introduisent ainsi plusieurs mécanismes d’apprentissage puissants pour les réseaux évènementiels, contribuant à rendre l’apprentissage des réseaux spike plus adaptés à des problèmes réels. / Inference and training in deep neural networks require large amounts of computation, which in many cases prevents the integration of deep networks in resource constrained environments. Event-based spiking neural networks represent an alternative to standard artificial neural networks that holds the promise of being capable of more energy efficient processing. However, training spiking neural networks to achieve high inference performance is still challenging, in particular when learning is also required to be compatible with neuromorphic constraints. This thesis studies training algorithms and information encoding in such deep networks of spiking neurons. Starting from a biologically inspired learning rule, we analyze which properties of learning rules are necessary in deep spiking neural networks to enable embedded learning in a continuous learning scenario. We show that a time scale invariant learning rule based on spike-timing dependent plasticity is able to perform hierarchical feature extraction and classification of simple objects of the MNIST and N-MNIST dataset. To overcome certain limitations of this approach we design a novel framework for spike-based learning, SpikeGrad, which represents a fully event-based implementation of the gradient backpropagation algorithm. We show how this algorithm can be used to train a spiking network that performs inference of relations between numbers and MNIST images. Additionally, we demonstrate that the framework is able to train large-scale convolutional spiking networks to competitive recognition rates on the MNIST and CIFAR10 datasets. In addition to being an effective and precise learning mechanism, SpikeGrad allows the description of the response of the spiking neural network in terms of a standard artificial neural network, which allows a faster simulation of spiking neural network training. Our work therefore introduces several powerful training concepts for on-chip learning in neuromorphic devices, that could help to scale spiking neural networks to real-world problems.

Driving is one of the most common and preferred forms of transport used in our actual society. However, according to studies, it is also one of the most dangerous. One solution to increase safety on the road is applying technology to automate and prevent avoidable human errors. Nevertheless, despite the efforts to obtain reliable systems, we have yet to find a reliable and safe enough solution for solving autonomous driving. One of the reasons is that many drives are done in conditions far from the ideal, with variable lighting conditions and fast-paced, unpredictable environments. This project develops and evaluates an algorithm that takes the input of dynamic vision sensors (DVS) and runs on neuromorphic spiking neural networks (SNN) to obtain a robust road lane tracking system. We present quantitative and qualitative metrics that evaluate the performance of lane recognition in low light conditions against conventional algorithms. This project is motivated by the main advantages of neuromorphic vision sensors: recognizing a high dynamic range and allowing a high-speed image capture. Another improvement of this system is the computational speed and power efficiency that characterize neuromorphic hardware based on spiking neural networks. The results obtained show a similar accuracy of this new algorithm compared to previous implementations on conventional hardware platforms. Most importantly, it accomplishes the proposed task with lower latency and computing power requirements than previous algorithms. / Att köra bil är ett av de vanligaste och mest populära transportsätten i vårt samhälle. Enligt forskningen är det också ett av de farligaste. En lösning för att öka säkerheten på vägarna är att med teknikens hjälp automatisera bilkörningen och på så sätt förebygga misstag som beror på den mänskliga faktorn. Trots ansträngningarna för att få fram tillförlitliga system har man dock ännu inte hittat en tillräckligt tillförlitlig och säker lösning för självkörande bilar. En av orsakerna till det är att många körningar sker under förhållanden som är långt ifrån idealiska, med varierande ljusförhållanden och oförutsägbara miljöer i höga hastigheter. I det här projektet utvecklar och utvärderar vi en algoritm som tar emot indata från dynamiska synsensorer (Dynamic Vision Sensors, DVS) och kör datan på neuromorfiska pulserande neuronnät (Spiking Neural Networks, SNN) för att skapa ett robust system för att läsa av vägbanan. Vi presenterar en kvantitativ och kvalitativ utvärdering av hur väl systemet läser av körbanans linjer i svagt ljus, och jämför därefter resultaten med dem för tidigare algoritmer. Detta projekt motiveras av de viktigaste fördelarna med neuromorfiska synsensorer: brett dynamiskt omfång och hög bildtagningshastighet. En annan fördel hos detta system är den korta beräkningstiden och den energieffektivitet som kännetecknar neuromorfisk hårdvara baserad på pulserande neuronnät. De resultat som erhållits visar att den nya algoritmen har en liknande noggrannhet som tidigare algoritmer på traditionella hårdvaruplattformar. I jämförelse med den traditionella tekniken, utför algoritmen i den föreliggande studien sin uppgift med kortare latenstid och lägre krav på processorkraft. / La conducción es una de las formas de transporte más comunes y preferidas en la actualidad. Sin embargo, diferentes estudios muestran que también es una de las más peligrosas. Una solución para aumentar la seguridad en la carretera es aplicar la tecnología para automatizar y prevenir los evitables errores humanos. No obstante, a pesar de los esfuerzos por conseguir sistemas fiables, todavía no hemos encontrado una solución suficientemente fiable y segura para resolver este reto. Una de las razones es el entorno de la conducción, en situaciones que distan mucho de las ideales, con condiciones de iluminación variables y entornos rápidos e imprevisibles. Este proyecto desarrolla y evalúa un algoritmo que toma la entrada de sensores de visión dinámicos (DVS) y ejecuta su computación en redes neuronales neuromórficas (SNN) para obtener un sistema robusto de seguimiento de carriles en carretera. Presentamos métricas cuantitativas y cualitativas que evalúan el rendimiento del reconocimiento de carriles en condiciones de poca luz, frente a algoritmos convencionales. Este proyecto está motivado por la validación de las ventajas de los sensores de visión neuromórficos: el reconocimiento de un alto rango dinámico y la captura de imágenes de alta velocidad. Otra de las mejoras que se espera de este sistema es la velocidad de procesamiento y la eficiencia energética que caracterizan al hardware neuromórfico basado en redes neuronales de impulsos. Los resultados obtenidos muestran una precisión similar entre el nuevo algoritmo en comparación con implementaciones anteriores en plataformas convencionales. Y lo que es más importante, realiza la tarea propuesta con menor latencia y requisitos de potencia de cálculo.