Deep neural networks (DNNs) are receiving immense attention because of their ability to solve complex problems. However, running a DNN requires a very large number of computations. Hence, dedicated hardware optimized for running deep learning algorithms known as neuromorphic architectures is often utilized. This dissertation focuses on evaluating andenhancing the accuracy of these neuromorphic architectures considering the designs of components, process variations, and adversarial attacks. The first contribution of the dissertation (Chapter 2) proposes design enhancements in analog Memristive Crossbar Array(MCA)-based neuromorphic architectures to improve classification accuracy. It introduces an analog Winner-Take-All (WTA) architecture and an on-chip training architecture. WTA ensures that the classification of the analog MCA is correct at the final selection level and the highest probability is selected. In particular, this dissertation presents a design of a highly scalable and precise current-mode WTA circuit with digital address generation. The design is based on current mirrors and comparators that use the cross-coupled latch structure. A post-silicon calibration circuit is also presented to handle process variations. On-chip training ensures that there is consistency in classification accuracy among different all analog MCA-based neuromorphic chips. Finally, an enhancement to the analog on-chip training architecture by implementing the Convolutional Neural Network (CNN) on MCA and software considerations to accelerate the training is presented.The second focus of the dissertation (Chapter 3) is on producing correct classification in the presence of malicious inputs known as adversarial attacks. This dissertation shows that MCA-based neuromorphic architectures ensure correct classification when the input is compromised using existing adversarial attack models. Furthermore, it shows that adversarialrobustness can be further improved by compression-based preprocessing steps that can be implemented on MCAs. It also evaluates the impact of the architecture in Chapter 2 under adversarial attacks. It shows that adversarial attacks do not uniformly affect the classification accuracy of different MCA-based chips. Experimental evidence using a variety of datasets and attack models supports the impact of MCA-based neuromorphic architectures and compression-based preprocessing implemented on MCAs to mitigate adversarial attacks. It is also experimentally shown that the on-chip training improves consistency in mitigating adversarial attacks among different chips. The final contribution (Chapter 4) of this dissertation introduces an enhancement of the method in Chapter 3. It consists of input preprocessing using compression and subsequent rescale and rearrange operations that are implemented using MCAs. This approach further improves the robustness against adversarial attacks. The rescale and rearrange operations are implemented using a DNN consisting of fully connected and convolutional layers. Experimental results show improved defense compared to similar input preprocessing techniques on MCAs.

No description available.

Durant ces trois années de doctorat, financées par le projet ANR MHANN (MemristiveHardware Analog Neural Network), nous nous sommes intéressés au développement d’une nouvelle architecture de calculateur à l’aide de réseaux de neurones. Les réseaux de neurones artificiels sont particulièrement bien adaptés à la reconnaissance d’images et peuvent être utilisés en complément des processeurs séquentiels. En 2008, une nouvelle technologie de composant a vu le jour : le memristor. Classé comme étant le quatrième élément passif, il est possible de modifier sa résistance en fonction de la densité de courant qui le traverse et de garder en mémoire ces changements. Grâce à leurs propriétés, les composants memristifs sont des candidats idéaux pour jouer le rôle des synapses au sein des réseaux de neurones artificiels. En effectuant des mesures sur la technologie des memristors ferroélectriques de l’UMjCNRS/Thalès de l’équipe de Julie Grollier, nous avons pu démontrer qu’il était possible d’obtenir un apprentissage de type STDP (Spike Timing Dependant Plasticity) classiquement utilisé avec les réseaux de neurones impulsionnels. Cette forme d’apprentissage, inspirée de la biologie, impose une variation des poids synaptiques en fonction des évènements neuronaux. En s’appuyant sur les mesures réalisées sur ces memristors et sur des simulations provenant d’un programme élaboré avec nos partenaires de l’INRIA Saclay, nous avons conçu successivement deux puces en silicium pour deux technologies de memristors ferroélectriques. La première technologie (BTO), moins performante, a été mise de côté au profit d’une seconde technologie (BFO). La seconde puce a été élaborée avec les retours d’expérience de la première puce. Elle contient deux couches d’un réseau de neurones impulsionnels dédié à l’apprentissage d’images de 81 pixels. En la connectant à un boitier contenant un crossbar de memristors, nous pourrons réaliser un démonstrateur d’un réseau de neurones hybride réalisé avec des synapses memristives ferroélectriques. / Supported financially by ANR MHANN project, this work proposes an architecture ofspiking neural network in order to recognize pictures, where traditional processing units are inefficient regarding this. In 2008, a new passive electrical component had been discovered : the memristor. Its resistance can be adjusted by applying a potential between its terminals. Behaving intrinsically as artificial synapses, memristives devices can be used inside artificial neural networks.We measure the variation in resistance of a ferroelectric memristor (obtained from UMjCNRS/Thalès) similar to the biological law STDP (Spike Timing Dependant Plasticity) used with spiking neurons. With our measurements on the memristor and our network simulation (aided by INRIASaclay) we designed successively two versions of the IC. The second IC design is driven by specifications of the first IC with additional functionalists. The second IC contains two layers of a spiking neural network dedicated to learn a picture of 81 pixels. A demonstrator of hybrid neural networks will be achieved by integrating a chip of memristive crossbar interfaced with thesecond IC.

No description available.

The multidisciplinary field of memristors calls for the necessity for theoreticallyinclined researchers and experimenters to join forces, merging complementary expertise and technical know-how, to develop and implement rigorous and systematic techniques to design variability-aware memristor-based circuits and systems. The availability of a predictive physics-based model for a memristor is a necessary requirement before commencing these investigations. An interesting dynamic phenomenon, occurring ubiquitously in non-volatile memristors, is fading memory. The latter may be defined as the appearance of a unique steady-state behavior, irrespective of the choice of the initial condition from an admissible range of values, for each stimulus from a certain family, for example, the DC or the purely-AC periodic input class. This paper first provides experimental evidence for the emergence of fading memory effects in the response of a TaOₓ redox-based random access memory cell to inputs from both of these classes. Leveraging the predictive capability of a physics-based device model, called JART VCM v1, a thorough system-theoretic analysis, revolving around the Dynamic Route Map graphic tool, is presented. This analysis allows to gain a better understanding of the mechanisms, underlying the emergence of history erase effects, and to identify the main factors, that modulate this nonlinear phenomenon, toward future potential applications.

Orientador: Prof. Dr. Gustavo Martini Dalpian / Tese (doutorado) - Universidade Federal do ABC, Programa de Pós-Graduação em Nanociências e Materiais Avançados, 2015. / A propriedade de chaveamento da resitência ou memoristiva é a habilidade de um material de alterar seu estado de resistência elétrica devido a um campo elétrico. O memoristor é um dispositivo de dois terminais com tal propriedade capaz de armazenar informação através de sua resistência, constituído de uma estrutura metal/isolante/metal. Este dispositivo pode revolucionar a indústria de memórias por apresentar tempos de chaveamento rápidos e de retenção longos, assim como altas densidades. Entretanto, seu princípio de funcionamento não é totalmente entendido a nível atômico, logo sua aplicação é impedida. Dois mecanismos são propostos: o mecanismo de difusão-deriva de íons afirma que campos elétricos e gradientes de temperatura formam e dissolvem canais condutores, alterando a resistividade. Por outro lado, modelos eletrônicos consideram o aprisionamento e liberação de cargas como causa da mudança da resistividade. Neste trabalho utilizamos uma abordagem heurística¿cálculos de teoria do funcional da densidade e soluções numéricas¿para entender os processos ocorrendo em escala atômica no interior de dispositivos baseados em TiO2. Os resultados mostram que a dificência em oxigênio neste caso leva à formação de fases TinO2n..1 que apresentam uma banda intermediária, a qual pode se tornar carregada quando propriamente interfaceada. A resolução numérica da equação de Poisson apresenta múltiplas soluções relacionadas a diferentes estados de resistência, estas soluções são usadas em um código de transmissão que fornece curvas teóricas i X V para o memoristor. / The resistive switching or memristive property is the ability of a material to change its electrical resistance due to the application of an electric field. The memristor is a two-terminal device with this property that is capable of storing information as its resistance state, being architectured in a metal/insulator/metal stacking. This device can revolutionize the memory industry by providing fast switching and large retention times as well as high-density capabilities. However, its working principle is not completely understood at an atomic level, thus its application as next-generation resistive memories is hindered. Two mechanisms are proposed: ion drift mechanisms claim that the electric field and temperature gradients inside the device can form and dissolve a conducting filament, changing the electrical resistivity. On the other hand, electronic models consider charge trapping and de-trapping inside the insulator layer as the cause of the resistivity change. In this work we use a heuristic computational approach¿density functional theory calculations and other numerical solutions¿to understand the processes developing at the atomic scale inside TiO2-based devices. Our results show that the oxygen deficiency in this material leads to the formation of a series of phases TinO2n..1 that present an intermediate band which can become charged when properly interfaced. The self-consistent-numerical solver of the Poisson equation shows multiple solutions that are related to the resistance states, and finally the potential is used in a transmission code that results in theoretical i X V curves for the memristor.

<p>U radu je razmatran problem dizajna i minimizacije rekurzivne<br />Bulove formule konstruisane za proizvoljnu Bulovu funkciju y:BN<br />&rarr;B.<br />U cilju rešavanja ovog problema, predstavljene su dve algoritamske<br />heuristike za minimizaciju rekurzivne Bulove formule. Minimizacija<br />rekurzivne Bulove formule vrši se korišćenjem regularnih poredaka<br />pozitivnih proizvod termova. U disertaciji je dokazano kako je ova<br />regularnost poredaka zapravo potreban i dovoljan uslov da željena<br />Bulova funcija y bude korektno predstavljena rekurzivnom Bulovom<br />formulom konstruisanom na osnovu tih poredaka. Pokazano je i kako<br />predstavljeni algoritmi daju bolje rezultate za veći broj instanci<br />problema u poređenju sa algoritmima dostupnim u literaturi.</p> / <p>In this thesis, the problem of design and minimization of recursive Boolean<br />formula, based on an arbitrary Boolean function y:BN<br />&rarr;B , is considered. As a<br />solution of a problem, two heuristic algorithms that minimize the length of<br />recursive Boolean formula, were presented. Minimization, itself, is done by<br />using regular orders of positive product terms. In the thesis it was proved that<br />the regularity of orders represents necessary and sufficient condition for<br />correct representation of Boolean function y by recursive Boolean formula<br />based on such regular order. Developed algorithms are compared with other<br />heuristic algorithms for recursive Boolean formula minimization, available in<br />the literature, and it is shown how algorithms proposed in this thesis provide<br />better results for more problem instances.</p>

Dans cette thèse, nous étudions les applications potentielles des nano-dispositifs mémoires émergents dans les architectures de calcul. Nous montrons que des architectures neuro-inspirées pourraient apporter l'efficacité et l'adaptabilité nécessaires à des applications de traitement et de classification complexes pour la perception visuelle et sonore. Cela, à un cout moindre en termes de consommation énergétique et de surface silicium que les architectures de type Von Neumann, grâce à une utilisation synaptique de ces nano-dispositifs. Ces travaux se focalisent sur les dispositifs dit «memristifs», récemment (ré)-introduits avec la découverte du memristor en 2008 et leur utilisation comme synapse dans des réseaux de neurones impulsionnels. Cela concerne la plupart des technologies mémoire émergentes : mémoire à changement de phase – «Phase-Change Memory» (PCM), «Conductive-Bridging RAM» (CBRAM), mémoire résistive – «Resistive RAM» (RRAM)... Ces dispositifs sont bien adaptés pour l'implémentation d'algorithmes d'apprentissage non supervisés issus des neurosciences, comme «Spike-Timing-Dependent Plasticity» (STDP), ne nécessitant que peu de circuit de contrôle. L'intégration de dispositifs memristifs dans des matrices, ou «crossbar», pourrait en outre permettre d'atteindre l'énorme densité d'intégration nécessaire pour ce type d'implémentation (plusieurs milliers de synapses par neurone), qui reste hors de portée d'une technologie purement en «Complementary Metal Oxide Semiconductor» (CMOS). C'est l'une des raisons majeures pour lesquelles les réseaux de neurones basés sur la technologie CMOS n'ont pas eu le succès escompté dans les années 1990. A cela s'ajoute la relative complexité et inefficacité de l'algorithme d'apprentissage de rétro-propagation du gradient, et ce malgré tous les aspects prometteurs des architectures neuro-inspirées, tels que l'adaptabilité et la tolérance aux fautes. Dans ces travaux, nous proposons des modèles synaptiques de dispositifs memristifs et des méthodologies de simulation pour des architectures les exploitant. Des architectures neuro-inspirées de nouvelle génération sont introduites et simulées pour le traitement de données naturelles. Celles-ci tirent profit des caractéristiques synaptiques des nano-dispositifs memristifs, combinées avec les dernières avancées dans les neurosciences. Nous proposons enfin des implémentations matérielles adaptées pour plusieurs types de dispositifs. Nous évaluons leur potentiel en termes d'intégration, d'efficacité énergétique et également leur tolérance à la variabilité et aux défauts inhérents à l'échelle nano-métrique de ces dispositifs. Ce dernier point est d'une importance capitale, puisqu'il constitue aujourd'hui encore la principale difficulté pour l'intégration de ces technologies émergentes dans des mémoires numériques. / In this thesis, we study the potential applications of emerging memory nano-devices in computing architecture. More precisely, we show that neuro-inspired architectural paradigms could provide the efficiency and adaptability required in some complex image/audio processing and classification applications. This, at a much lower cost in terms of power consumption and silicon area than current Von Neumann-derived architectures, thanks to a synaptic-like usage of these memory nano-devices. This work is focusing on memristive nano-devices, recently (re-)introduced by the discovery of the memristor in 2008 and their use as synapses in spiking neural network. In fact, this includes most of the emerging memory technologies: Phase-Change Memory (PCM), Conductive-Bridging RAM (CBRAM), Resistive RAM (RRAM)... These devices are particularly suitable for the implementation of natural unsupervised learning algorithms like Spike-Timing-Dependent Plasticity (STDP), requiring very little control circuitry.The integration of memristive devices in crossbar array could provide the huge density required by this type of architecture (several thousand synapses per neuron), which is impossible to match with a CMOS-only implementation. This can be seen as one of the main factors that hindered the rise of CMOS-based neural network computing architectures in the nineties, among the relative complexity and inefficiency of the back-propagation learning algorithm, despite all the promising aspects of such neuro-inspired architectures, like adaptability and fault-tolerance. In this work, we propose synaptic models for memristive devices and simulation methodologies for architectural design exploiting them. Novel neuro-inspired architectures are introduced and simulated for natural data processing. They exploit the synaptic characteristics of memristives nano-devices, along with the latest progresses in neurosciences. Finally, we propose hardware implementations for several device types. We assess their scalability and power efficiency potential, and their robustness to variability and faults, which are unavoidable at the nanometric scale of these devices. This last point is of prime importance, as it constitutes today the main difficulty for the integration of these emerging technologies in digital memories.

En utilisant les méthodes d’apprentissages tirées des récentes découvertes en neuroscience, les réseaux de neurones impulsionnels ont démontrés leurs capacités à analyser efficacement les grandes quantités d’informations provenant de notre environnement. L’implémentation de ces circuits à l’aide de processeurs classiques ne permet pas d’exploiter efficacement leur parallélisme. L’utilisation de mémoire numérique pour implémenter les poids synaptique ne permet pas la lecture ou la programmation parallèle des synapses et est limité par la bande passante reliant la mémoire à l’unité de calcul. Les technologies mémoire de type memristive pourrait permettre l’implémentation de ce parallélisme au coeur de la mémoire.Dans cette thèse, nous envisageons le développement d’un réseau de neurones impulsionnels dédié au monde de l’embarqué à base de dispositif mémoire émergents. Dans un premier temps, nous avons analysé un réseau impulsionnel afin d’optimiser ses différentes composantes : neurone, synapse et méthode d’apprentissage STDP en vue d’une implémentation numérique. Dans un second temps, nous envisageons l’implémentation de la mémoire synaptique par des dispositifs memristifs. Enfin, nous présentons le développement d’une puce co-intégrant des neurones implémentés en CMOS avec des synapses en technologie CBRAM. / By using learning mechanisms extracted from recent discoveries in neuroscience, spiking neural networks have demonstrated their ability to efficiently analyze the large amount of data from our environment. The implementation of such circuits on conventional processors does not allow the efficient exploitation of their parallelism. The use of digital memory to implement the synaptic weight does not allow the parallel reading or the parallel programming of the synapses and it is limited by the bandwidth of the connection between the memory and the processing unit. Emergent memristive memory technologies could allow implementing this parallelism directly in the heart of the memory.In this thesis, we consider the development of an embedded spiking neural network based on emerging memory devices. First, we analyze a spiking network to optimize its different components: the neuron, the synapse and the STDP learning mechanism for digital implementation. Then, we consider implementing the synaptic memory with emergent memristive devices. Finally, we present the development of a neuromorphic chip co-integrating CMOS neurons with CBRAM synapses.

The performance of a modern computing system is improving with technology scaling due to advancements in the modern semiconductor industry. However, the power efficiency along with reliability does not scale linearly with performance efficiency. High leakage and standby power in sub 100 nm technology are critical challenges faced by circuit designers. Recent developments in device physics have shown that emerging non-volatile memories are very effective in reducing power dissipation because they eliminate stand by power and exhibit almost zero leakage powerThis dissertation studies the use of emerging non-volatile memory devices in designing circuit architecture for improving power dissipation and the performance of the computing system. More specically, it proposes a novel spintronic Ternary Content AddressableMemory (TCAM), a novel memristive TCAM with improved power and performance efficiency. Our experimental evaluation on 45 nm technology for a 256-bit word-size spintronic TCAM at a supply voltage of 1 V with a sense margin of 50 mV show that the delay is lessthan 200 ps and the per-bit search energy is approximately 3 fJ. The proposed spintronic TCAM consumes at least 30% less energy when compared to state-of-the-art TCAM designs. The search delay on a 144-bit proposed memristive TCAM at a supply voltage of 1 V and a sense margin of 140 mV is 175 ps with per bit search energy of 1.2 fJ on a 45 nm technology. It is 1.12 x times faster and dissipates 67% less search energy per bit than the fastest existing 144-bit MTCAM design.Emerging non-volatile memories are well known for their ability to perform fast analog multiplication and addition when they are arranged in crossbar fashion and are especially suited for neural network applications. However, such systems require the on-chip implementation of the backpropagation algorithm to accommodate process variations. This dissertation studies the impact of process variation in training memristive neural network architecture. It proposes a low hardware overhead on-chip implementation of the backpropagation algorithm that utilizes effectively the very dense memristive cross-bar arrayand is resilient to process variations.Another important issue that needs a careful study due to shrinking technology node is the impact of space or terrestrial radiation in Integrated Circuits (ICs) because the probability of a high energy particle causing an error increases with a decrease in thethreshold voltage and the noise margin. Moreover, single-event effects (SEEs) sensitivity depends on the set of input vectors used at the time of testing due to logical masking. This dissertation analyzes the impact of input test set on the cross section of the microprocessorand proposes a mechanism to derive a high-quality input test set using an automatic test pattern generation (ATPG) for radiation testing of microprocessors arithmetic and logical units..