Dans ce travail de thèse, nous avons modélisé le rôle de la dopamine dans l'apprentissage et dans les processus de sélection de l'action en lien avec les ganglions de la base. L'activité des neurones dopaminergiques présente de nombreuses similarités avec l'erreur de prédiction de la récompense utilisée par les algorithmes d'apprentissage par renforcement. Ainsi, ces neurones sont supposés guider le processus de sélection de l'action.Dans une première partie, nous avons analysé l'information encodée par les neurones dopaminergiques dans une tâche à choix multiples en la comparant à différentes informations utilisées par les modèles d'apprentissage par renforcement. Nos résultats suggèrent que l'information encodée par les neurones dopaminergiques enregistrer dans la tâche n'est que partiellement compatible avec une erreur de prédiction et semble en partie dissociée du comportement.Dans une deuxième partie, nous avons simulé l'effet de la dopamine sur un modèle des ganglions de la base prenant en compte des connections existant chez le primate, souvent négligées dans la littérature. La plupart des modèles actuels font en effet l'hypothèse d'une séparation stricte de deux chemins dans les ganglions de la base : le chemin direct lié à la récompense et le chemin indirect lié à la punition. Cependant des études anatomiques remettent en question cette dissociation, en particulier chez le primate. Nous proposons ainsi d'étudier comment différents niveaux de dopamine, dans le contexte de la maladie de Parkinson, affectent l'apprentissage et la sélection de l'action dans ce modèle / In this thesis work, we modelled the role of dopamine in learning and in the processes of action selection through its interaction with the basal ganglia. During the 90’s, the work of Schultz and colleagues has led to major progress in understanding the neural mechanisms underlying the influence of feedback on learning. The activity of dopaminergic neurons exhibited properties of the reward prediction error signal used in so-called Temporal Difference (TD) machine learning algorithms. Thus, DA has been thought to be the neural signal that help us to adapt our behavior. In the first part of my PhD, we analyze the information encoded by dopaminergic neurons recorded during a multi-choice task. In this purpose, we modeled the task and simulated different TD learning algorithms to quantitatively compare their ability to reproduce dopamine neurons activity. Our results show that the information carried out by dopamine neurons is only partly consistent with a reward prediction error and seems to be dissociated from behavioral adaptation.In the second part of my PhD, we study the effect of different levels of dopamine in a biologically plausible model of primates basal ganglia that considers existing connections often neglected in the literature. Indeed, most of current models of basal ganglia assume the existence of two segregated pathway: the direct pathway associated with reward and the indirect pathway associated with punishment. However, anatomical studies in primates revealed that these two pathways are not dissociated. We study the ability of such a model to reproduce beta oscillations observed in Parkinsonian and the differences in reward and punishment sensitivity, with high or low-level of dopamine.

Abstract not available

PhD / When modeling a system as complex as the brain, considerable simplifications are inevitable. The nature of these simplifications depends on the available experimental evidence, and the desired form of model predictions. A focus on the former often inspires models of networks of individual neurons, since properties of single cells are more easily measured than those of entire populations. However, if the goal is to describe the processes responsible for the electroencephalogram (EEG), such models can become unmanageable due to the large numbers of neurons involved. Mean-field models in which assemblies of neurons are represented by their average properties allow activity underlying the EEG to be captured in a tractable manner. The starting point of the results presented here is a recent physiologically-based mean-field model of the corticothalamic system, which includes populations of excitatory and inhibitory cortical neurons, and an excitatory population representing the thalamic relay nuclei, reciprocally connected with the cortex and the inhibitory thalamic reticular nucleus. The average firing rates of these populations depend nonlinearly on their membrane potentials, which are determined by afferent inputs after axonal propagation and dendritic and synaptic delays. It has been found that neuronal activity spreads in an approximately wavelike fashion across the cortex, which is modeled as a two-dimensional surface. On the basis of the literature, the EEG signal is assumed to be roughly proportional to the activity of cortical excitatory neurons, allowing physiological parameters to be extracted by inverse modeling of empirical EEG spectra. One objective of the present work is to characterize the statistical distributions of fitted model parameters in the healthy population. Variability of model parameters within and between individuals is assessed over time scales of minutes to more than a year, and compared with the variability of classical quantitative EEG (qEEG) parameters. These parameters are generally not normally distributed, and transformations toward the normal distribution are often used to facilitate statistical analysis. However, no single optimal transformation exists to render data distributions approximately normal. A uniformly applicable solution that not only yields data following the normal distribution as closely as possible, but also increases test-retest reliability, is described in Chapter 2. Specialized versions of this transformation have been known for some time in the statistical literature, but it has not previously found its way to the empirical sciences. Chapter 3 contains the study of intra-individual and inter-individual variability in model parameters, also providing a comparison of test-retest reliability with that of commonly used EEG spectral measures such as band powers and the frequency of the alpha peak. It is found that the combined model parameters provide a reliable characterization of an individual's EEG spectrum, where some parameters are more informative than others. Classical quantitative EEG measures are found to be somewhat more reproducible than model parameters. However, the latter have the advantage of providing direct connections with the underlying physiology. In addition, model parameters are complementary to classical measures in that they capture more information about spectral structure. Another conclusion from this work was that a few minutes of alert eyes-closed EEG already contain most of the individual variability likely to occur in this state on the scale of years. In Chapter 4, age trends in model parameters are investigated for a large sample of healthy subjects aged 6-86 years. Sex differences in parameter distributions and trends are considered in three age ranges, and related to the relevant literature. We also look at changes in inter-individual variance across age, and find that subjects are in many respects maximally different around adolescence. This study forms the basis for prospective comparisons with age trends in evoked response potentials (ERPs) and alpha peak morphology, besides providing a standard for the assessment of clinical data. It is the first study to report physiologically-based parameters for such a large sample of EEG data. The second main thrust of this work is toward incorporating the thalamocortical system and the basal ganglia in a unified framework. The basal ganglia are a group of gray matter structures reciprocally connected with the thalamus and cortex, both significantly influencing, and influenced by, their activity. Abnormalities in the basal ganglia are associated with various disorders, including schizophrenia, Huntington's disease, and Parkinson's disease. A model of the basal ganglia-thalamocortical system is presented in Chapter 5, and used to investigate changes in average firing rates often measured in parkinsonian patients and animal models of Parkinson's disease. Modeling results support the hypothesis that two pathways through the basal ganglia (the so-called direct and indirect pathways) are differentially affected by the dopamine depletion that is the hallmark of Parkinson's disease. However, alterations in other components of the system are also suggested by matching model predictions to experimental data. The dynamics of the model are explored in detail in Chapter 6. Electrophysiological aspects of Parkinson's disease include frequency reduction of the alpha peak, increased relative power at lower frequencies, and abnormal synchronized fluctuations in firing rates. It is shown that the same parameter variations that reproduce realistic changes in mean firing rates can also account for EEG frequency reduction by increasing the strength of the indirect pathway, which exerts an inhibitory effect on the cortex. Furthermore, even more strongly connected subcircuits in the indirect pathway can sustain limit cycle oscillations around 5 Hz, in accord with oscillations at this frequency often observed in tremulous patients. Additionally, oscillations around 20 Hz that are normally present in corticothalamic circuits can spread to the basal ganglia when both corticothalamic and indirect circuits have large gains. The model also accounts for changes in the responsiveness of the components of the basal ganglia-thalamocortical system, and increased synchronization upon dopamine depletion, which plausibly reflect the loss of specificity of neuronal signaling pathways in the parkinsonian basal ganglia. Thus, a parsimonious explanation is provided for many electrophysiological correlates of Parkinson's disease using a single set of parameter changes with respect to the healthy state. Overall, we conclude that mean-field models of brain electrophysiology possess a versatility that allows them to be usefully applied in a variety of scenarios. Such models allow information about underlying physiology to be extracted from the experimental EEG, complementing traditional measures that may be more statistically robust but do not provide a direct link with physiology. Furthermore, there is ample opportunity for future developments, extending the basic model to encompass different neuronal systems, connections, and mechanisms. The basal ganglia are an important addition, not only leading to unified explanations for many hitherto disparate phenomena, but also contributing to the validation of this form of modeling.

Seit den Anfängen empirisch-neurowissenschaftlicher Forschung gilt Sprachkompetenz zuvorderst als eine Leistung der Hirnrinde (Kortex), jedoch wurden v. a. im Zuge sich verbessernder bildgebender Verfahren aphasische Syndrome auch nach Läsionen subkortikaler Hirnregionen, insbesondere der Basalganglien und des Thalamus nachgewiesen. Diese Strukturen liegen in der Tiefe des Gehirns und kommunizieren über weit gefächerte Faserverbindungen mit dem Kortex. In erster Linie werden den Basalganglien senso-motorische Kontrollfunktionen zugewiesen. Dementsprechend werden diverse Erkrankungen, die durch Störungen physiologischer Bewegungsabläufe gekennzeichnet sind (z. B. Morbus Parkinson, Chorea Huntington), auf Funktionsdefekte dieser Strukturen zurückgeführt. Der Thalamus wird häufig als Relaisstation des Informationsaustauschs zwischen anatomisch entfernten Arealen des Nervensystems aufgefasst. Basalganglien und Thalamus werden jedoch auch darüber hinausgehende Funktionen, z. B. zur Bereitstellung, Aufrechterhaltung und Auslenkung von Aufmerksamkeit bei der Bearbeitung kognitiver Aufgaben zugesprochen. In der vorliegenden Arbeit wurde mit elektrophysiologischen Methoden untersucht, ob auf der Ebene von Thalamus und Basalganglien kognitive Sprachleistungen, spezifisch der syntaktischen und semantischen Verarbeitung nachgewiesen werden können und inwieweit sich eventuell subkortikale von kortikaler Sprachverarbeitung unterscheidet. Die Untersuchung spezieller Sprachfunktionen der Basalganglien und des Thalamus ist im Rahmen der operativen Behandlung bewegungsgestörter Patienten mit der sog. Tiefenhirnstimulation (DBS = engl. Deep Brain Stimulation) möglich. Hierbei werden Patienten mit Morbus Parkinson Stimulationselektroden in den Nucleus subthalamicus (STN) implantiert. Bei Patienten mit generalisierten Dystonien erfolgt die Implantation in den Globus pallidus internus (GPI) und bei Patienten mit essentiellem Tremor in den Nucleus ventralis intermedius (VIM). STN und GPI sind Kernareale der Basalganglien, der VIM ist Teil des motorischen Systems. Nach der Implantation besteht die Möglichkeit, direkt von diesen Elektroden elektroenzephalographische (EEG)-Signale abzuleiten und diese mit simultan abgeleiteten Oberflächen-EEG zu vergleichen. In dieser Arbeit wurden DBS-Patienten aus allen genannten Gruppen in Bezug auf Sprachverständnisleistungen untersucht. Neben der Präsentation korrekter Sätze hörten die Patienten Sätze mit syntaktischen oder semantischen Fehlern. In verschiedenen Studien wurden an der Skalp-Oberfläche EKP-Komponenten (EKP = ereigniskorrelierte Potentiale) beschrieben, welche mit der Verarbeitung solcher Fehler in Verbindung gebracht werden. So verursachen syntaktische Phrasenstrukturverletzungen eine frühe links-anteriore Negativierung (ELAN). Dieser Komponente folgt eine späte Positivierung (P600), die mit Reanalyse und Reparaturmechanismen in Verbindung gebracht wird. Semantische Verletzungen evozieren eine breite Negativierung um 400ms (N400). In den thalamischen Ableitungen wurden zwei zusätzliche syntaktische fehlerbezogene Komponenten gefunden, die (i) ~ 80ms nach der Skalp-ELAN und (ii) ~ 70ms vor der Skalp-P600 auftraten. Bei semantischen Verletzungen wurde im Thalamus ein fehlerbezogenes Potential nachgewiesen, welches weitgehend parallel mit dem am Skalp gefundenen Muster verläuft. Aus den Ergebnissen der vorliegenden Studie folgt, dass der Thalamus spezifische Sprachfunktionen erfüllt. Komponenten, die Sprachverarbeitungsprozesse reflektieren, konnten in den Basalganglienstrukturen STN und GPI nicht identifiziert werden. Aufgrund der erhobenen Daten werden zwei getrennte Netzwerke für die Verarbeitung syntaktischer bzw. semantischer Fehler angenommen. In diesen Netzwerken scheint der Thalamus spezifische Aufgaben zu übernehmen. In einem ‚Syntaxnetzwerk’ kommunizieren frontale Hirnstrukturen unter Einbeziehung des Thalamus mit parietalen Hirnstrukturen. Dem Thalamus wurde eine Mediationsfunktion in der syntaktischen Reanalyse zugesprochen. In einem ‚Semantiknetzwerk’ waren keine eindeutig zuordenbaren Prozesse auf thalamischer Ebene nachweisbar. Es wurde eine unscharfe, jedoch aber spezifische Aktivierung des Thalamus über den gesamten Zeitraum der kortikalen semantischen Analyse gezeigt, welche als Integration verschiedener Analysemechanismen gewertet wurde. / Since the beginning of empirical neuroscientific research language competence has been primary localized at the brain cortex. Improved functional neuroimaging techniques were able to localize lesions in structures which caused aphasic syndromes. These syndromes were particularly found after lesions of the basal ganglia and the thalamus These structures are located in the depth of the brain and communicate over widespread fiber connections with the cortex. Sensori-motor control functions are primarily assigned to the basal ganglia. Various diseases, which are characterized by disturbances of physiological courses of motion (e.g. Parkinson’s disease, Chorea Huntington) are attributed to function defects of these structures. The thalamus was originally understood as a simple relay station of information exchange between various cortical areas of the nervous system. However, additional functions were assigned to the basal ganglia and thalamus, e.g. maintenance and deflection of attention while processing of cognitive tasks. Within the present investigation thalamic and basal ganglia achievements in language processing were studied with electro-physiological methods to investigate differences between cortical and subcortical processes. The investigation of special language functions of the basal ganglia and the thalamus is possible in the context of the surgical treatment of movement-disordered patients with "Deep Brain Stimulation (DBS)". Stimulation electrodes have been implanted to patients with Parkinson’s disease into the subthalamic nucleus (STN), patients with generalized dystonia into the globus pallidus internus (GPi), and patients suffering from essential tremor into the ventral intermediate nucleus of the thalamus (VIM). STN and GPi are core areas of the basal ganglia, the VIM is part of the motor system. EEG-signals may be derived directly from these implanted electrodes and be compared to the simultaneously derived signals of a surface EEG. In this thesis DBS patients from all groups mentioned above were examined regarding language understanding achievements. The patients listened to sentences, which were either correct or syntactical or semantical incorrect. Different studies described scalp ERP components (ERP = event related potentials) which occurred after different types of errors in sentences. Thus, syntactic phrase structure violations cause an early left anterior negativity (ELAN). A late positivity (P600) follows this component and was hypothesized as a reflection of syntactical reanalysis and/or repair. Semantic violations evoke a broad negativity around 400ms (N400). In the thalamic EEG two additional syntactic components were identified, which were seen (i) ~ 80ms after the scalp ELAN and (ii) ~ 70ms before the scalp-P600. At thalamic level semantic violations caused a negativity, which parallels largely with the negativity found at the scalp. The results of this study suggest, that the thalamus fulfills specific language functions. In the basal ganglia structures (GPI and STN) no language specific components were found. Due to the collected data two separate networks are suggested for the processing of syntactic and/or semantic errors. In these networks the thalamus seems to fulfill specific tasks. In a "syntax network” frontal brain structures communicate with parietale brain structures via the thalamus. The function of the thalamus in this network is the mediation of the syntactic reanalysis. In a "semantic network” no clearly classified processes were proved at thalamic level, because of a similarity of thalamic and scalp signals. However, during the entire period of the cortical analysis activation of the thalamus was apparent. This activation was rated as integration of different analysis mechanisms.

In this thesis, the learning of probabilistic relationships between stimulus-action pairs is investigated under the probability learning paradigm. The effect of reward is investigated in the first three experiments. Additionally, the effect of context and uncertainty is investigated in the second and third experiments, respectively. The fourth experiment is the replication of the second experiment with a group of Parkinson patients where the effect of dopamine medication on probability learning is studied. In Experiment 1, we replicate the classical probability learning task by comparing monetary and non-monetary reward feedback. Probability learning behavior is observed in both monetary and non-monetary rewarding feedback conditions. However, no significant difference between the monetary and non-monetary feedback conditions is observed. In Experiment 2, a variation of the probability learning task which includes irrelevant contextual information is applied. Probability learning behavior is observed, and a significant effect is found between monetary and non-monetary feedback conditions. In Experiment 3 / a probability learning task similar to that in Experiment 2 is applied, however, in this experiment, stimulus included relevant contextual information. As expected, due to the utilization of the relevant contextual information from the start of the experiment, no significant effect is found for probability learning behavior. The effect of uncertainty observed in this experiment is a replication of the reports in literature. Experiment 4 is identical to Experiment 2 / except that the subject population is a group of dopamine medicated Parkinson patients and a group of age matched controls. This experiment is introduced to test the suggestions in the literature regarding the enhancement effect of dopamine medication in probability learning based on positive feedback conditions. In Experiment 4, probability learning behavior is observed in both groups, but the difference in learning performance between Parkinson patients and controls was not significant, probably due to the low number of subject recruited in the experiment. In addition to these investigations, learning mechanisms are also examined in Experiments 1 and 4. Our results indicate that subjects initially search for patterns which lead to probability learning. At the end of Experiments 1 and 4, upon learning the winning frequencies, subjects change their behavior and demonstrate maximization behavior, which makes them prefer continuously one option over the other.

Neurological and psychiatric disorders reveal that the basal ganglia subserve diverse functional domains, including movement, reward, and cognitive disorders (e.g., Parkinson's disease, addiction, schizophrenia). Monkey anatomical studies show that the striatum, the input structure of the basal ganglia, receives projections from nearly the entire cerebral cortex with a broad topography of motor, limbic, and association zones. However, until recently, non-invasive methods have not been available to conduct the complete mapping of the cortex to the striatum in humans. The development of functional connectivity magnetic resonance imaging (fcMRI) now allows the identification of functional connections in humans. The present dissertation reports two studies that first create a complete map of corticostriatal connectivity and then more closely examine striatal connectivity with association networks underlying cognition.

Μελέτη της υπόθεσης ότι τα βασικά γάγγλια, μία κεντρική δομή του εγκεφάλου, συμμετέχει ενεργά στην επιλογή της κατάλληλης δράσης. Προσπάθεια μοντελοποίησης της λειτουργίας των βασικών γαγγλίων κια προσομοίωση παθολογικών καταστάσεων του ανθρώπινου εγκεφάλου. / Stydy of the hypothesis that the basal ganglia, a major structure in the mamalian brain, plays a crusial role in the selection of the appropriate action. In this direction we try to simulate the functionality of the basal ganglia as well the pathophysiological conditions (Parkinson, Huntington disease) related to them.

Μελετήθηκαν αλληλεπιδράσεις μεταξύ της παρεγκεφαλίδας και των βασικών γαγγλίων στον εγκέφαλο ενήλικων μυών. Για τη μελέτη της επίδρασης της παρεγκεφαλίδας στα βασικά γάγγλια εξετάστηκαν οι επιπτώσεις της εκφύλισης του παρεγκεφαλιδικού φλοιού στη νευροχημεία των βασικών γαγγλίων. Χρησιμοποιήθηκαν δύο μοντέλα εκφύλισης: το μεταλλαγμένο στέλεχος μυός PCD που εμφανίζει πλήρη εκφύλιση των κυττάρων Purkinje και μύες με μερική δεξιά εκφύλιση του παρεγκεφαλιδικού φλοιού μετά από ένεση καϊνικού οξέος στην περιοχή του παρασκώληκα. Ιn vitro ποσοτική αυτοραδιογραφία έδειξε αναβάθμιση των υποδοχέων ντοπαμίνης D1 στα βασικά γάγγλια και των δύο μοντέλων εκφύλισης. Για τη μελέτη της επίδρασης των βασικών γαγγλίων στην παρεγκεφαλίδα μελετήθηκε ο μεταφορέας της ντοπαμίνης στον παρεγκεφαλιδικό ιστό μυών με μεθόδους ποσοτικής αυτοραδιογραφίας, μεμβρανικής δέσμευσης και ανοσοϊστοχημείας. Τα αποτελέσματα έδειξαν την ύπαρξη, την ανατομική κατανομή και τις ιδιότητες του μεταφορέα της νοτπαμίνης στην παρεγκεφαλίδα. / The study of the interactions between the cerebellum and the basal ganglia was the aim of this thesis. To study cerebellar influences on the basal ganglia I investigated the effects of cerebellar cortical degeneration on basal ganglia neurochemistry. Two models of cerebellar degeneration were used: the mutant mouse strain \\\"Purkinje cell degenaration\\\" that completely lacks Purkinje cells, and mice with partial Purkije cell lesion after kainic acid injections in the right paravermis. In vitro quantitative receptor autoradiography showed an up-regulation of D1 dopamine receptors in the basal ganglia of both cerebellar degeneration models. This may reflect a compensatory adaptation. To study basal ganglia influences on the cerebellum I characterized the dopamine transporter of the mouse cerebellum by using receptor binding and determined its anatomical distribution by using receptor autoradiography and immunoistochemistry.

To survive, many biological organisms need to accurately infer which features of their environment predict future rewards and punishments. In machine learning terms, this is the problem of spatial credit assignment, for which many supervised learning algorithms have been developed. In this thesis, I mainly propose that a dual-pathway, regression-like strategy and associated biological implementations may be used to solve this problem. Using David Marr's (1982) three-level philosophy of computational neuroscience, the thesis and its contributions are organized as follows: - Computational Level: Here, the spatial credit assignment problem is formally defined and modeled using probability density functions. The specific challenges of the problem faced by organisms and machine learning algorithms alike are also identified. - Algorithmic Level: I present and evaluate the novel hypothesis that the general strategy used by animals is to perform a regression over past experiences. I also introduce an extension of a probabilistic model for regression that substantially improves generalization without resorting to regularization. This approach subdues residual associations to irrelevant features, as does regularization. - Physical Level: Here, the neuroscience of classical conditioning and of the basal ganglia is briefly reviewed. Then, two novel models of the basal ganglia are put forward: 1) an online-learning model that supports the regression hypothesis and 2) a biological implementation of the probabilistic model previously introduced. Finally, we compare these models to others in the literature. In short, this thesis establishes a theoretical framework for studying the spatial credit assignment problem, offers a simple hypothesis for how biological systems solve it, and implements basal ganglia-based algorithms in support. The thesis brings to light novel approaches for machine learning and several explanations for biological structures and classical conditioning phenomena. / Note: While the thesis contains content from two articles (one journal, one conference), their publishers do not require special permission for their use in dissertations (information confirming this is in an appendix of the thesis itself).

The motor nervous system is one of the main systems of the body and is our principle means ofbehavior. Some of the most debilitating and wide spread disorders are motor systempathologies. In particular the basal ganglia are complex networks of the brain that control someaspects of movement in all vertebrates. Although these networks have been extensively studied,lack of proper methods to study them on a system level has hindered the process ofunderstanding what they do and how they do it. In order to facilitate this process I have usedcomputational models as an approach that can faithfully take into account many aspects of ahigh dimensional multi faceted system.In order to minimize the complexity of the system, I first took agnathan fish and amphibians asmodeling animals. These animals have rather simple neuronal networks and have been wellstudied so that developing their biologically plausible models is more feasible. I developedmodels of sensory motor transformation centers that are capable of generating basic behaviorsof approach, avoidance and escape. The networks in these models used a similar layeredstructure having a sensory map in one layer and a motor map on other layers. The visualinformation was received as place coded information, but was converted into population codedand ultimately into rate coded signals usable for muscle contractions.In parallel to developing models of visuomotor centers, I developed a novel model of the basalganglia. The model suggests that a subsystem of the basal ganglia is in charge of resolvingconflicts between motor programs suggested by different motor centers in the nervous system.This subsystem that is composed of the subthalamic nucleus and pallidum is called thearbitration system. Another subsystem of the basal ganglia called the extension system which iscomposed of the striatum and pallidum can bias decisions made by an animal towards theactions leading to lower cost and higher outcome by learning to associate proper actions todifferent states. Such states are generally complex states and the novel hypothesis I developedsuggests that the extension system is capable of learning such complex states and linking themto appropriate actions. In this framework, striatal neurons play the role of conjunction (BooleanAND) neurons while pallidal neurons can be envisioned as disjunction (Boolean OR) neurons.In the next set of experiments I tried to take the idea of basal ganglia subsystems to a new levelby dividing the rodent arbitration system into two functional subunits. A rostral group of ratpallidal neurons form dense local inhibition among themselves and even send inhibitoryprojections to the caudal segment. The caudal segment does not project back to its rostralcounterpart, but both segments send inhibitory projections to the output nuclei of the rat basalganglia i.e. the entopeduncular nucleus and substantia nigra. The rostral subsystems is capableof precisely detecting one (or several) components of a rudimentary action and suppress othercomponents. The components that are reinforced are those which lead to rewarding stateswhereas those that are suppressed are those which do not. The hypothesis explains neuronalmechanisms involved in this process and suggests that this subsystem is a means of generatingsimple but precise movements (such as using a single digit) from innate crude actions that theanimal can perform even at birth (such as general movement of the whole limb). In this way, therostral subsystem may play important role in exploration based learning.In an attempt to more precisely describe the relation between the arbitration and extensionsystems, we investigated the effect of dynamic synapses between subthalamic, pallidal andstriatal neurons and output neurons of the basal ganglia. The results imply that output neuronsare sensitive to striatal bursts and pallidal irregular firing. They also suggest that few striatalneurons are enough to fully suppress output neurons. Finally the results show that the globuspallidus exerts its effect on output neurons by direct inhibition rather than indirect influence viathe subthalamic nucleus. / <p>QC 20131209</p>