Von Neumann Bottleneck, which refers to the limited throughput between the CPU and memory, has already become the major factor hindering the technical advances of computing systems. In recent years, neuromorphic systems started to gain increasing attention as compact and energy-efficient computing platforms. Spike based-neuromorphic computing systems require high performance and low power neural encoder and decoder to emulate the spiking behavior of neurons. These two spike-analog signals converting interface determine the whole spiking neuromorphic computing system's performance, especially the highest performance. Many state-of-the-art neuromorphic systems typically operate in the frequency range between 〖10〗^0KHz and 〖10〗^2KHz due to the limitation of encoding/decoding speed. In this dissertation, all these popular encoding and decoding schemes, i.e. rate encoding, latency encoding, ISI encoding, together with related hardware implementations have been discussed and analyzed. The contributions included in this dissertation can be classified into three main parts: neuron improvement, three kinds of ISI encoder design, two types of ISI decoder design. Two-path leakage LIF neuron has been fabricated and modular design methodology is invented. Three kinds of ISI encoding schemes including parallel signal encoding, full signal iteration encoding, and partial signal encoding are discussed. The first two types ISI encoders have been fabricated successfully and the last ISI encoder will be taped out by the end of 2019. Two types of ISI decoders adopted different techniques which are sample-and-hold based mixed-signal design and spike-timing-dependent-plasticity (STDP) based analog design respectively. Both these two ISI encoders have been evaluated through post-layout simulations successfully. The STDP based ISI encoder will be taped out by the end of 2019. A test bench based on correlation inspection has been built to evaluate the information recovery capability of the proposed spiking processing link. / Doctor of Philosophy / Neuromorphic computing is a kind of specific electronic system that could mimic biological bodies’ behavior. In most cases, neuromorphic computing system is built with analog circuits which have benefits in power efficient and low thermal radiation. Among neuromorphic computing system, one of the most important components is the signal processing interface, i.e. encoder/decoder. To increase the whole system’s performance, novel encoders and decoders have been proposed in this dissertation. In this dissertation, three kinds of temporal encoders, one rate encoder, one latency encoder, one temporal decoder, and one general spike decoder have been proposed. These designs could be combined together to build high efficient spike-based data link which guarantee the processing performance of whole neuromorphic computing system.

In recent years neuromorphic computing systems have achieved a lot of success due to its ability to process data much faster and using much less power compared to traditional Von Neumann computing architectures. There are two main types of Artificial Neural Networks (ANNs), Feedforward Neural Network (FNN) and Recurrent Neural Network (RNN). In this thesis we first study the types of RNNs and then move on to Spiking Neural Networks (SNNs). SNNs are an improved version of ANNs that mimic biological neurons closely through the emission of spikes. This shows significant advantages in terms of power and energy when carrying out data intensive applications by allowing spatio-temporal information processing. On the other hand, emerging non-volatile memory (eNVM) technology is key to emulate neurons and synapses for in-memory computations for neuromorphic hardware. A particular eNVM technology, memristors, have received wide attention due to their scalability, compatibility with CMOS technology and low power consumption properties. In this work we develop a spiking neural network by incorporating an inter-spike interval encoding scheme to convert the incoming input signal to spikes and use a memristive crossbar to carry out in-memory computing operations. We develop a novel input and output processing engine for our network and demonstrate the spatio-temporal information processing capability. We demonstrate an accuracy of a 100% with our design through a small-scale hardware simulation for digit recognition and demonstrate an accuracy of 87% in software through MNIST simulations. / M.S. / In recent years neuromorphic computing systems have achieved a lot of success due to its ability to process data much faster and using much less power compared to traditional Von Neumann computing architectures. Artificial Neural Networks (ANNs) are models that mimic biological neurons where artificial neurons or neurodes are connected together via synapses, similar to the nervous system in the human body. here are two main types of Artificial Neural Networks (ANNs), Feedforward Neural Network (FNN) and Recurrent Neural Network (RNN). In this thesis we first study the types of RNNs and then move on to Spiking Neural Networks (SNNs). SNNs are an improved version of ANNs that mimic biological neurons closely through the emission of spikes. This shows significant advantages in terms of power and energy when carrying out data intensive applications by allowing spatio-temporal information processing capability. On the other hand, emerging non-volatile memory (eNVM) technology is key to emulate neurons and synapses for in-memory computations for neuromorphic hardware. A particular eNVM technology, memristors, have received wide attention due to their scalability, compatibility with CMOS technology and low power consumption properties. In this work we develop a spiking neural network by incorporating an inter-spike interval encoding scheme to convert the incoming input signal to spikes and use a memristive crossbar to carry out in-memory computing operations. We demonstrate the accuracy of our design through a small-scale hardware simulation for digit recognition and demonstrate an accuracy of 87% in software through MNIST simulations.

This thesis integrates biologically-inspired mechanisms into machine learning to develop novel tuning algorithms, gradient abstractions for depth-wise parallelism, and an original bias neuron design. We introduce neuromodulatory tuning, which uses neurotransmitter-inspired bias adjustments to enhance transfer learning in spiking and non-spiking neural networks, significantly reducing parameter usage while maintaining performance. Additionally, we propose a novel approach that decouples the backward pass of backpropagation using layer abstractions, inspired by feedback loops in biological systems, enabling depth-wise training parallelization. We further extend neuromodulatory tuning by designing spiking bias neurons that mimic dopamine neuron mechanisms, leading to the development of volumetric tuning. This method enhances the fine-tuning of a small spiking neural network for EEG emotion classification, outperforming previous bias tuning methods. Overall, this thesis demonstrates the potential of leveraging neuroscience discoveries to improve machine learning.

As complimentary metal-oxide semiconductor (CMOS) technologies scale and field-effect transistor (FET) architectures change, the factors in deciding to utilize analog or digital transistor behaviors evolve. This thesis examines three case studies where traditionally analog or digital circuitry has dominated published works but I show that the opposite regime has significant benefits in scaled CMOS technologies. I present a highly digital operational amplifier (traditionally analog) and two artificial neurons (traditionally digital). In Chapters 2 and 3 I present a highly-digital five-stage zero-crossing-based amplifier which breaks the trade-off between slew rate and settling accuracy. I investigate the optimal charge pump design by analyzing the effects of the current scaling factor, number of current sources, maximum current value, and input amplitude on the settling performance including overshoot and settling time. I find that there exists an optimal number of stages that yields the fastest settling for a given total current and load capacitance. The proposed amplifier achieves a signal-to-noise ratio of 57 dB at a sampling rate of 40 MHz and consumes 1.45 mW under a 1V supply. In Chapters 4 and 5, I propose two novel analog artificial spiking neurons, operating in the voltage domain and phase domain respectively. The voltage domain neuron presented in Chapter 4 implements a novel fine-tuning method called neuromodulatory tuning which reduced the number of parameters to be tuned by four orders of magnitude as compared with traditional fine-tuning methods. Chapter 5 presents the design of a novel phase-domain neuron. Voltage domain neurons mimic biological neurons by integrating charge on a capacitor. I instead integrate phase in a voltage-controlled ring oscillator (VCO). I also propose a novel bidirectional switched-capacitor synapse which saves significant area compared to bidirectional current based synapses. The proposed neuron, synapse and weight memory occupy only 21x27um, and consume 134fJ/spike under a 0.35V supply.

Yes / Structural and behavioural properties of models are very important in development of complex systems and applications. In this paper, we investigate such properties for some classes of SN P systems. First, a class of SN P systems associated to a set of routing problems are investigated through their matrix representation. This allows to make certain connections amongst some of these problems. Secondly, the behavioural properties of these SN P systems are formally verified through a natural and direct mapping of these models into kP systems which are equipped with adequate formal verification methods and tools. Some examples are used to prove the effectiveness of the verification approach. / EPSRC research grant EP/R043787/1; DOST-ERDT research grants; Semirara Mining Corp; UPD-OVCRD;

Networks of spiking neurons constitute analog systems capable of effective and resilient computing. Recent work has shown that networks of symmetrically connected inhibitory neurons may implement basic computations such that they are resilient to system disruption. For instance, if the functionality of one neuron is lost (e.g., the neuron, along with its connections, is removed), the system may be robustly reconfigured by adapting only one global system parameter. How to effectively adapt network parameters to robustly perform a given computation is still unclear. Here, we present an analytical approach to derive such parameters. Specifically, we analyze k-winners-takes-all (k-WTA) computations, basic computational tasks of identifying the k largest signals from a total of N input signals from which one can construct any computation. We identify and characterize different dynamical regimes and provide analytical expressions for the transitions between different numbers k of winners as a function of both input and network parameters. Our results thereby provide analytical insights about the dynamics underlying k-winner-takes-all functionality as well as an effective way of designing spiking neural network computing systems implementing disruption-resilient dynamics.

Cortical malformations as a result of altered development are a common cause of human epilepsy. The cellular mechanisms that render neurons of malformed cortex epileptogenic remain unclear. Using a rat model of the malformation of microgyria, a previous study showed an alteration in the number of immunocytochemically-identified parvalbumin cells, a GABAergic inhibitory interneurons subtype (Rosen et al., 1998). A second study showed no change in the total number of GABAergic neurons (Schwarz et al., 2000). Consequently, we hypothesize that interneuron subtypes are differentially affected by maldevelopment. The present study investigated (1) whether interneuron subtype identity is retained in malformed cortex, based on chemical content, and (2) whether the proportion of three chemical subtypes is altered in malformed cortex. Here we demonstrate that three non-overlapping subtype markers remain non-overlapping in malformed cortex, but show altered distributions. These findings suggest that an increase in one subpopulation of interneurons may compensate for a corresponding decrease in a second subset.

<p>Neuromorphic computing endeavors to develop computational systems capable of emulating the brain’s capacity to execute intricate tasks concurrently and with remarkable energy efficiency. By utilizing new bioinspired computing architectures, these systems have the potential to revolutionize high-performance computing and enable local, low-energy computing for sensors and robots. Organic and soft materials are particularly attractive for neuromorphic computing as they offer biocompatibility, low-energy switching, and excellent tunability at a relatively low cost. Additionally, organic materials provide physical flexibility, large-area fabrication, and printability.</p><p>This doctoral dissertation showcases the research conducted in fabricating a comprehensive spiking organic neuron, which serves as the fundamental constituent of a circuit system for neuromorphic computing. The major contribution of this dissertation is the development of the organic, flexible neuron composed of spiking synapses and somas utilizing ultra-low voltage organic field-effect transistors (OFETs) for information processing. The synaptic and somatic circuits are implemented using physically flexible and biocompatible organic electronics necessary to realize the Polymer Neuromorphic Circuitry. An Axon-Hillock (AH) somatic circuit was fabricated and analyzed, followed by the adaptation of a log-domain integrator (LDI) synaptic circuit and the fabrication and analysis of a differential-pair integrator (DPI). Finally, a spiking organic neuron was formed by combining two LDI synaptic circuits and one AH synaptic circuit, and its characteristics were thoroughly examined. This is the first demonstration of the fabrication of an entire neuron using solid-state organic materials over a flexible substrate with integrated complementary OFETs and capacitors.</p>

Hebbian learning based neural network learning rules when implemented on hardware, store their synaptic weights in the form of a two-dimensional matrix. The storage of synaptic weights demands large memory bandwidth and storage. While memory units are optimized for only row-wise memory access, Hebbian learning rules, like the spike-timing dependent plasticity, demand both row and column-wise access of memory. This dual pattern of memory access accounts for the dominant cost in terms of latency as well as energy for realization of large scale spiking neural networks in hardware. In order to reduce the memory access cost in Hebbian learning rules, a Column Update Elimination optimization has been previously implemented, with great efficacy, on the Bayesian Confidence Propagation neural network, that faces a similar challenge of dual pattern memory access. This thesis explores the possibility of extending the column update elimination optimization to spike-timing dependent plasticity, by simulating the learning rule on a two layer network of leaky integrate-and-fire neurons on an image classification task. The spike times are recorded for each neuron in the network, to derive a suitable probability distribution function for spike rates per neuron. This is then used to derive an ideal postsynaptic spike history buffer size for the given algorithm. The associated memory access reductions are analysed based on data to assess feasibility of the optimization to the learning rule. / Hebbiansk inlärning baserat på neural nätverks inlärnings regler används vid implementering på hårdvara, de lagrar deras synaptiska vikter i form av en tvådimensionell matris. Lagringen av synaptiska vikter kräver stor bandbredds minne och lagring. Medan minnesenheter endast är optimerade för radvis minnesåtkomst. Hebbianska inlärnings regler kräver som spike-timing-beroende plasticitet, både rad- och kolumnvis åtkomst av minnet. Det dubbla mönstret av minnes åtkomsten står för den dominerande kostnaden i form av fördröjning såväl som energi för realiseringen av storskaliga spikande neurala nätverk i hårdvara. För att minska kostnaden för minnesåtkomst i hebbianska inlärnings regler har en Column Update Elimination-optimering tidigare implementerats, med god effektivitet på Bayesian Confidence Propagation neurala nätverket, som står inför en liknande utmaning med dubbel mönster minnesåtkomst. Denna avhandling undersöker möjligheten att utöka ColumnUpdate Elimination-optimeringen till spike-timing-beroende plasticitet. Detta genom att simulera inlärnings regeln på ett tvålagers nätverk av läckande integrera-och-avfyra neuroner på en bild klassificerings uppgift. Spike tiderna registreras för varje neuron i nätverket för att erhålla en lämplig sannolikhetsfördelning funktion för frekvensen av toppar per neuron. Detta används sedan för att erhålla en idealisk postsynaptisk spike historisk buffertstorlek för den angivna algoritmen. De associerade minnesåtkomst minskningarna analyseras baserat på data för att bedöma genomförbarheten av optimeringen av inlärnings regeln.

O sistema olfatório é notável por sua capacidade de discriminar odores muito similares, mesmo que estejam misturados. Essa capacidade de discriminação é, em parte, devida a padrões de atividade espaço-temporais gerados nas células mitrais, as células principais do bulbo olfatório, durante a apresentação de um odor. Tais padrões dinâmicos decorrem de interações sinápticas recíprocas entre as células mitrais e interneurônios inibitórios do bulbo olfatório, por exemplo, as células granulares. Nesta tese, apresenta-se um modelo do bulbo olfatório baseado em modelos pulsados das células mitrais e granulares e avalia-se o seu desempenho como sistema reconhecedor de padrões usando-se bases de dados de padrões artificiais e reais. Os resultados dos testes mostram que o modelo possui a capacidade de separar padrões em diferentes classes. Essa capacidade pode ser explorada na construção de sistemas reconhecedores de padrões. Apresenta-se também a ferramenta denominada Nemos, desenvolvida para a implementação do modelo, que é uma plataforma para simulação de neurônios e redes de neurônios pulsados com interface gráfica amigável com o usuário. / The olfactory system is a remarkable system capable of discriminating very similar odorant mixtures. This is in part achieved via spatio-temporal activity patterns generated in mitral cells, the principal cells of the olfactory bulb, during odor presentation. Here, we present a spiking neural network model of the olfactory bulb and evaluate its performance as a pattern recognition system with datasets taken from both artificial and real pattern databases. Our results show that the dynamic activity patterns produced in the mitral cells of the olfactory bulb model by pattern attributes presented to it have a pattern separation capability. This capability can be explored in the construction of high-performance pattern recognition systems. Besides, we proposed Nemos a framework for simulation spiking neural networks through graphical user interface and has extensible models for neurons, synapses and networks.