The aim of this thesis is to describe how a statisticallybased neural network technology, here named BCPNN (BayesianConfidence Propagation Neural Network), which may be identifiedby rewriting Bayes' rule, can be used within a fewapplications, data mining and classification with credibilityintervals as well as unsupervised pattern recognition. BCPNN is a neural network model somewhat reminding aboutBayesian decision trees which are often used within artificialintelligence systems. It has previously been success- fullyapplied to classification tasks such as fault diagnosis,supervised pattern recognition, hiearchical clustering and alsoused as a model for cortical memory. The learning paradigm usedin BCPNN is rather different from many other neural networkarchitectures. The learning in, e.g. the popularbackpropagation (BP) network, is a gradient method on an errorsurface, but learning in BCPNN is based upon calculations ofmarginal and joint prob- abilities between attributes. This isa quite time efficient process compared to, for instance,gradient learning. The interpretation of the weight values inBCPNN is also easy compared to many other networkarchitechtures. The values of these weights and theiruncertainty is also what we are focusing on in our data miningapplication. The most important results and findings in thisthesis can be summarised in the following points:     We demonstrate how BCPNN (Bayesian Confidence PropagationNeural Network) can be extended to model the uncertainties incollected statistics to produce outcomes as distributionsfrom two different aspects: uncertainties induced by sparsesampling, which is useful for data mining; uncertainties dueto input data distributions, which is useful for processmodelling.     We indicate how classification with BCPNN gives highercertainty than an optimal Bayes classifier and betterprecision than a naïve Bayes classifier for limited datasets.     We show how these techniques have been turned into auseful tool for real world applications within the drugsafety area in particular.     We present a simple but working method for doingautomatic temporal segmentation of data sequences as well asindicate some aspects of temporal tasks for which a Bayesianneural network may be useful.     We present a method, based on recurrent BCPNN, whichperforms a similar task as an unsupervised clustering method,on a large database with noisy incomplete data, but muchquicker, with an efficiency in finding patterns comparablewith a well known (Autoclass) Bayesian clustering method,when we compare their performane on artificial data sets.Apart from BCPNN being able to deal with really large datasets, because it is a global method working on collectivestatistics, we also get good indications that the outcomefrom BCPNN seems to have higher clinical relevance thanAutoclass in our application on the WHO database of adversedrug reactions and therefore is a relevant data mining toolto use on the WHO database. Artificial neural network, Bayesian neural network, datamining, adverse drug reaction signalling, classification,learning.

<p>The aim of this thesis is to describe how a statisticallybased neural network technology, here named BCPNN (BayesianConfidence Propagation Neural Network), which may be identifiedby rewriting Bayes' rule, can be used within a fewapplications, data mining and classification with credibilityintervals as well as unsupervised pattern recognition.</p><p>BCPNN is a neural network model somewhat reminding aboutBayesian decision trees which are often used within artificialintelligence systems. It has previously been success- fullyapplied to classification tasks such as fault diagnosis,supervised pattern recognition, hiearchical clustering and alsoused as a model for cortical memory. The learning paradigm usedin BCPNN is rather different from many other neural networkarchitectures. The learning in, e.g. the popularbackpropagation (BP) network, is a gradient method on an errorsurface, but learning in BCPNN is based upon calculations ofmarginal and joint prob- abilities between attributes. This isa quite time efficient process compared to, for instance,gradient learning. The interpretation of the weight values inBCPNN is also easy compared to many other networkarchitechtures. The values of these weights and theiruncertainty is also what we are focusing on in our data miningapplication. The most important results and findings in thisthesis can be summarised in the following points:</p><p>    We demonstrate how BCPNN (Bayesian Confidence PropagationNeural Network) can be extended to model the uncertainties incollected statistics to produce outcomes as distributionsfrom two different aspects: uncertainties induced by sparsesampling, which is useful for data mining; uncertainties dueto input data distributions, which is useful for processmodelling.</p><p>    We indicate how classification with BCPNN gives highercertainty than an optimal Bayes classifier and betterprecision than a naïve Bayes classifier for limited datasets.</p><p>    We show how these techniques have been turned into auseful tool for real world applications within the drugsafety area in particular.</p><p>    We present a simple but working method for doingautomatic temporal segmentation of data sequences as well asindicate some aspects of temporal tasks for which a Bayesianneural network may be useful.</p><p>    We present a method, based on recurrent BCPNN, whichperforms a similar task as an unsupervised clustering method,on a large database with noisy incomplete data, but muchquicker, with an efficiency in finding patterns comparablewith a well known (Autoclass) Bayesian clustering method,when we compare their performane on artificial data sets.Apart from BCPNN being able to deal with really large datasets, because it is a global method working on collectivestatistics, we also get good indications that the outcomefrom BCPNN seems to have higher clinical relevance thanAutoclass in our application on the WHO database of adversedrug reactions and therefore is a relevant data mining toolto use on the WHO database.</p><p>Artificial neural network, Bayesian neural network, datamining, adverse drug reaction signalling, classification,learning.</p>

<p>The WHO database contains more than 2.8 million case reports of suspected adverse drug reactions reported from 70 countries worldwide since 1968. The Uppsala Monitoring Centre maintains and analyses this database for new signals on behalf of the WHO Programme for International Drug Monitoring. A goal of the Programme is to detect signals, where a signal is defined as "Reported information on a possible causal relationship between an adverse event and a drug, the relationship being unknown or incompletely documented previously."</p><p>The analysis of such a large amount of data on a case by case basis is impossible with the resources available. Therefore a quantitative, data mining procedure has been developed to improve the focus of the clinical signal detection process. The method used, is referred to as the BCPNN (Bayesian Confidence Propagation Neural Network). This not only assists in the early detection of adverse drug reactions (ADRs) but also further analysis of such signals. The method uses Bayesian statistical principles to quantify apparent dependencies in the data set. This quantifies the degree to which a specific drug- ADR combination is different from a background (in this case the WHO database). The measure of disproportionality used, is referred to as the Information Component (IC) because of its' origins in Information Theory. A confidence interval is calculated for the IC of each combination. A neural network approach allows all drug-ADR combinations in the database to be analysed in an automated manner. Evaluations of the effectiveness of the BCPNN in signal detection are described.</p><p>To compare how a drug association compares in unexpectedness to related drugs, which might be used for the same clinical indication, the method is extended to consideration of groups of drugs. The benefits and limitations of this approach are discussed with examples of known group effects (ACE inhibitors - coughing and antihistamines - heart rate and rhythm disorders.) An example of a clinically important, novel signal found using the BCPNN approach is also presented. The signal of antipsychotics linked with heart muscle disorder was detected using the BCPNN and reported.</p><p>The BCPNN is now routinely used in signal detection to search single drug - single ADR combinations. The extension of the BCPNN to discover 'unexpected' complex dependencies between groups of drugs and adverse reactions is described. A recurrent neural network method has been developed for finding complex patterns in incomplete and noisy data sets. The method is demonstrated on an artificial test set. Implementation on real data is demonstrated by examining the pattern of adverse reactions highlighted for the drug haloperidol. Clinically important, complex relationships in this kind of data are previously unexplored.</p><p>The BCPNN method has been shown and tested for use in routine signal detection, refining signals and in finding complex patterns. The usefulness of the output is influenced by the quality of the data in the database. Therefore, this method should be used to detect, rather than evaluate signals. The need for clinical analyses of case series remains crucial.</p>

The WHO database contains more than 2.8 million case reports of suspected adverse drug reactions reported from 70 countries worldwide since 1968. The Uppsala Monitoring Centre maintains and analyses this database for new signals on behalf of the WHO Programme for International Drug Monitoring. A goal of the Programme is to detect signals, where a signal is defined as "Reported information on a possible causal relationship between an adverse event and a drug, the relationship being unknown or incompletely documented previously." The analysis of such a large amount of data on a case by case basis is impossible with the resources available. Therefore a quantitative, data mining procedure has been developed to improve the focus of the clinical signal detection process. The method used, is referred to as the BCPNN (Bayesian Confidence Propagation Neural Network). This not only assists in the early detection of adverse drug reactions (ADRs) but also further analysis of such signals. The method uses Bayesian statistical principles to quantify apparent dependencies in the data set. This quantifies the degree to which a specific drug- ADR combination is different from a background (in this case the WHO database). The measure of disproportionality used, is referred to as the Information Component (IC) because of its' origins in Information Theory. A confidence interval is calculated for the IC of each combination. A neural network approach allows all drug-ADR combinations in the database to be analysed in an automated manner. Evaluations of the effectiveness of the BCPNN in signal detection are described. To compare how a drug association compares in unexpectedness to related drugs, which might be used for the same clinical indication, the method is extended to consideration of groups of drugs. The benefits and limitations of this approach are discussed with examples of known group effects (ACE inhibitors - coughing and antihistamines - heart rate and rhythm disorders.) An example of a clinically important, novel signal found using the BCPNN approach is also presented. The signal of antipsychotics linked with heart muscle disorder was detected using the BCPNN and reported. The BCPNN is now routinely used in signal detection to search single drug - single ADR combinations. The extension of the BCPNN to discover 'unexpected' complex dependencies between groups of drugs and adverse reactions is described. A recurrent neural network method has been developed for finding complex patterns in incomplete and noisy data sets. The method is demonstrated on an artificial test set. Implementation on real data is demonstrated by examining the pattern of adverse reactions highlighted for the drug haloperidol. Clinically important, complex relationships in this kind of data are previously unexplored. The BCPNN method has been shown and tested for use in routine signal detection, refining signals and in finding complex patterns. The usefulness of the output is influenced by the quality of the data in the database. Therefore, this method should be used to detect, rather than evaluate signals. The need for clinical analyses of case series remains crucial.

Human capability to recall episodic memories depends on how much time has passed since the memory was encoded. This dependency is described by a memory retrieval curve that reflects an interesting phenomenon referred to as a reminiscence bump - a tendency for older people to recall more memories formed during their young adulthood than in other periods of life. This phenomenon can be modelled with an attractor neural network, for example, the firing-rate Bayesian Confidence Propagation Neural Network (BCPNN) with incremental learning. In this work, the mechanisms underlying the reminiscence bump in the neural network model are systematically studied. The effects of synaptic plasticity, network architecture and other relevant parameters on the characteristics of the reminiscence bump are systematically investigated. The most influential factors turn out to be the magnitude of dopamine-linked plasticity at birth and the time constant of exponential plasticity decay with age that set the position of the bump. The other parameters mainly influence the general amplitude of the lifespan retrieval curve. Furthermore, the recency phenomenon, i.e. the tendency to remember the most recent memories, can also be parameterized by adding a constant to the exponentially decaying plasticity function representing the decrease in the level of dopamine neurotransmitters. / Människans förmåga att återkalla episodiska minnen beror på hur lång tid som gått sedan minnena inkodades. Detta beroende beskrivs av en sk glömskekurva vilken uppvisar ett intressant fenomen som kallas ”reminiscence bump”. Detta är en tendens hos äldre att återkalla fler minnen från ungdoms- och tidiga vuxenår än från andra perioder i livet. Detta fenomen kan modelleras med ett neuralt nätverk, sk attraktornät, t ex ett icke spikande Bayesian Confidence Propagation Neural Network (BCPNN) med inkrementell inlärning. I detta arbete studeras systematiskt mekanismerna bakom ”reminiscence bump” med hjälp av denna neuronnätsmodell. Exempelvis belyses betydelsen av synaptisk plasticitet, nätverksarkitektur och andra relavanta parameterar för uppkomsten av och karaktären hos detta fenomen. De mest inflytelserika faktorerna för bumpens position befanns var initial dopaminberoende plasticitet vid födseln samt tidskonstanten för plasticitetens avtagande med åldern. De andra parametrarna påverkade huvudsakligen den generella amplituden hos kurvan för ihågkomst under livet. Dessutom kan den s k nysseffekten (”recency effect”), dvs tendensen att bäst komma ihåg saker som hänt nyligen, också parametriseras av en konstant adderad till den annars exponentiellt avtagande plasticiteten, som kan representera densiteten av dopaminreceptorer.

This thesis presents an abstract model of the mammalian neocortex. The model was constructed by taking a top-down view on the cortex, where it is assumed that cortex to a first approximation works as a system with attractor dynamics. The model deals with the processing of static inputs from the perspectives of biological mapping, algorithmic, and physical implementation, but it does not consider the temporal aspects of these inputs. The purpose of the model is twofold: Firstly, it is an abstract model of the cortex and as such it can be used to evaluate hypotheses about cortical function and structure. Secondly, it forms the basis of a general information processing system that may be implemented in computers. The characteristics of this model are studied both analytically and by simulation experiments, and we also discuss its parallel implementation on cluster computers as well as in digital hardware. The basic design of the model is based on a thorough literature study of the mammalian cortex’s anatomy and physiology. We review both the layered and columnar structure of cortex and also the long- and short-range connectivity between neurons. Characteristics of cortex that defines its computational complexity such as the time-scales of cellular processes that transport ions in and out of neurons and give rise to electric signals are also investigated. In particular we study the size of cortex in terms of neuron and synapse numbers in five mammals; mouse, rat, cat, macaque, and human. The cortical model is implemented with a connectionist type of network where the functional units correspond to cortical minicolumns and these are in turn grouped into hypercolumn modules. The learning-rules used in the model are local in space and time, which make them biologically plausible and also allows for efficient parallel implementation. We study the implemented model both as a single- and multi-layered network. Instances of the model with sizes up to that of a rat-cortex equivalent are implemented and run on cluster computers in 23% of real time. We demonstrate on tasks involving image-data that the cortical model can be used for meaningful computations such as noise reduction, pattern completion, prototype extraction, hierarchical clustering, classification, and content addressable memory, and we show that also the largest cortex equivalent instances of the model can perform these types of computations. Important characteristics of the model are that it is insensitive to limited errors in the computational hardware and noise in the input data. Furthermore, it can learn from examples and is self-organizing to some extent. The proposed model contributes to the quest of understanding the cortex and it is also a first step towards a brain-inspired computing system that can be implemented in the molecular scale computers of tomorrow. The main contributions of this thesis are: (i) A review of the size, modularization, and computational structure of the mammalian neocortex. (ii) An abstract generic connectionist network model of the mammalian cortex. (iii) A framework for a brain-inspired self-organizing information processing system. (iv) Theoretical work on the properties of the model when used as an autoassociative memory. (v) Theoretical insights on the anatomy and physiology of the cortex. (vi) Efficient implementation techniques and simulations of cortical sized instances. (vii) A fixed-point arithmetic implementation of the model that can be used in digital hardware. / QC 20100903

A modular Recurrent Bayesian Confidence PropagatingNeural Networks (BCPNN) with two synaptic time tracesis a computational neural network that can serve as a modelof biological short term memory. The units in the network aregrouped into modules called hypercolumns within which there isa competitive winner-takes-all mechanism.In this work, the network’s capacity to store sequentialmemories is investigated while varying the size of and numberof hyperocolumns in the network. The network is trained on setsof temporal sequences where each sequence consist of a set ofsymbols represented as semi-stable attractor state patterns in thenetwork and evaluated by its ability to later recall the sequences.For a given distribution of training sequence the networks’ability to store and recall sequences was seen to significantlyincrease with the size of the hypercolumns. As the number ofhypercolumns was increased, the storage capacity increased upto a clear level in most cases. After this point it was observedto remain constant and did not improve by adding any morehypercolumns (for a given sequence distribution). The storagecapacity was also seen to depend a lot on the distribution of thesequences. / Ett modulärt Recurrent Bayesian Confidence Propagating Neural Network (BCPNN) med två synaptiskatidsspår är ett neuronnät som kan användas som en modell förbiologiskt korttidsminne. Enheterna i nätverket är grupperade imoduler kallade hyperkolumner inom vilka enheterna konkurrerarenligt en ”winner-takes-all”-mekanism.I det här arbetet undersöktes hur nätverkets förmåga attlagra sekventiella minnen beror på storleken och antalet hyperkolumner.Nätverket tränades på ett antal temporala följderdär varje följd bestod av en mängd symboler som representeradesom attraktor-tillstånd i nätverket och bedömdes baserat på dessförmåga att komma ihåg följder det lärt sig under träning.För en given fördelning av träningsföljder ökade nätverketsförmåga att lagra och återkalla följder med storleken på hyperkolumnerna.Då antalet hyperkolumner ökades ökade ocks i de flesta fall lagringsförmågan upp till en viss nivå varefterytterligare hyperkolumner inte gav några vidare förbättringar(för en given fördelning av sekvenser). Lagringskapacitetenberodde också mycket på fördelningen av följder. / Kandidatexjobb i elektroteknik 2021, KTH, Stockholm

Understanding the brain and its functions is achallenging undertaking. To facilitate this work, brain-inspiredtechnology may be used to examine cognitive phenomena to acertain extent, by replacing real biological brains with simulations.The aim of this project was to provide insights intohow different kinds of plasticity modulation affected long-termmemory recall through the use of a computational model. Aneural network was constructed based on the existing BayesianConfidence Propagation Neural Network (BCPNN) model andtrained with binary patterns representing memories acquiredover a lifetime. By varying network plasticity parameters forselected patterns and performing recall of “aging” memories,greater effects were observed in recall statistics for modulationearly in the lifetime in comparison with modulation of later ages.From the experiments conducted in this study it was possible toconclude that selective modulation of learning affected the longtermrecall of all memories in the simulation. / Att förstå hjärnan och alla dess funktionerär en stor utmaning. För att underlätta detta arbete kanhjärninspirerad teknologi i viss utsträckning användas för attstudera kognitiva fenomen, genom att ersätta biologiska hjärnormed simuleringar. Syftet med denna studie var att ge en insikt ihur olika typer av modulering av synaptisk plasticitet påverkadeett simulerat biologiskt långtidsminne genom användning av endatoriserad modell. Ett neuralt nätverk implementerat med eninlärningsregel av typen Bayesian Confidence Propagation NeuralNetwork (BCPNN) konstruerades och användes för att träna och återkalla binära mönster, representerande minnen förvärvadeunder en livstid. Nätverkets synaptiska plasticitet varierades underträning av utvalda mönster och därefter utfördes återkallningav “åldrade” minnen. Testerna påvisade effekt på nätverketsförmåga att korrekt återkalla lagrade minnen. Det visade sigäven att modulering utförd på tidiga simulerade åldrar jämförtmed modulering av senare åldrar under livstiden hade störrepåverkan på långtidsminnet. Från resultaten var det möjligtatt konstatera att selektiv plasticitetsmodulering under inlärningpåverkade nätverkets förmåga att korrekt återkalla samtligabinära mönster i simuleringen. / Kandidatexjobb i elektroteknik 2021, KTH, Stockholm