Thesis (Ph.D.)--Mississippi State University. College of Veterinary Medicine. / Title from title screen. Includes bibliographical references.

Previous research has shown that orienting attention in time can help to improve behavioural outcomes. However, the extent to which temporal orienting can be preserved in ageing and in the context of neurological disorders remains unresolved. This thesis therefore explores temporal expectations in the healthy ageing and diseased brain by taking a neuropsychological approach. To begin, I provide an overview of the literature in <b>Chapter 1</b> most relevant for the following investigation. Four chapters of experiments then follow. To examine the effects of ageing on temporal expectation, the performance of healthy young adults and healthy older adults is presented and the results are discussed in <b>Chapter 2</b> and <b>3</b>. Though it had been previously shown that older adults seem to experience an expectation deficit on temporal expectation tasks, these chapters demonstrate the preservation of temporal expectation in ageing. On their own, these findings represent an important and novel contribution to the literature; however, this research is incapable of establishing the causal mechanisms involved in temporal expectation. To explore the causal role of relevant brain regions in temporal expectation, <b>Chapter 4</b> and <b>5</b> investigate the effects of temporal orienting in participants with damage to the basal ganglia - a brain region strongly implicated in temporal processing. In <b>Chapter 4</b>, the role of the basal ganglia in temporal expectations is examined using data collected from participants with Parkinson's disease and contrasts their performance with age-matched healthy controls. To complement this investigation, and to provide converging evidence for the basal ganglia's role in temporal expectations, Chapter 5 investigates the behavioural performance of individuals with focal lesions to the basal ganglia. The findings in this thesis are discussed in their wider context in Chapter 6, and directions for future research are proposed.

Les structures du système nerveux responsables des modalités de la prise de décision forment un circuit constitué par les ganglions de la base, le cortex, le thalamus et leurs nombreuses interconnexions. Ce circuit peut être décrit comme un ensemble de boucles fonctionnant en parallèle et interagissant en différents points. Des interactions entre ces boucles et de la plasticité de leurs connexions émergent les choix et donc les actions d’un individu. Ces comportements émergents et les phénomènes d’apprentissage qui en découlent sont abordés à travers une approche en boucle fermée dans laquelle le modèle théorique est en interaction constante avec l’environnement où se déroule la tâche comportementale étudiée. A cette fin, des outils de modélisation neuronale et d’analyse dédiés ont été développés dans le laboratoire d’accueil. Nous explorons donc ici la dynamique des flux d’information au sein de ce circuit à travers un modèle computationnel décrit à l’échelle du neurone et de la synapse. A partir d’observations expérimentales préalables réalisées sur le primate et de modèles computationnels antérieurs, nous avons développé de manière incrémentale un réseau capable d’apprendre à réaliser les tâches comportementales dans plusieurs protocoles et conditions. Le résultat obtenu ici est un modèle computationnel d’apprentissage et de prise de décision dans les ganglions de la base qui permet de tester des hypothèses expérimentales et d’effectuer des investigations physiopathologiques ou pharmacologiques in silico à l’échelle cellulaire. Le développement de ce modèle computationnel a été mené en parallèle avec l’étude expérimentale d’un protocole de prise de décision et la mise au point d’un modèle de maladie de Parkinson chez la salamandre (Pleurodeles waltlii). / The nervous system structures involved in decision making constitute a circuit formed by the basal ganglia, the cortex, the thalamus and their numerous interconnections. This circuit can be described as a set of loops operating in parallel and interacting at different points. The decisions and therefore the actions of an individual emerge from the interactions between these loops and the plasticity of their connections. These emerging behaviors and arising learning processes are addressed through a closed-loop approach in which the theoretical model is in constant interaction with the environment of the task. To this end, neural modeling and dedicated analysis software tools were developed in the laboratory. We explore here the dynamics of information flows within this circuit through a computational model described at the neuron and synapse level. Taking into account previous experimental observations from primates and earlier computational models, we incrementally developed a network capable of learning to perform behavioral tasks under several protocols and conditions. The result here is a computational model of learning and decision making in the basal ganglia that allows for the testing of experimental hypotheses and also to conduct in silico pathophysiological or pharmacological investigations at the cellular level. The development of this computational model was conducted in parallel with the development of an experimental protocol of decision making and with the adjustment of a model of Parkinson disease in the salamander (Pleurodeles waltlii).

Les noyaux gris centraux (ganglions de la base en anglais) sont un réseau de structures sous-corticales dont la persistance dans l'ensemble des vertébrés plaide en faveur d'une fonction clef au cours de l'évolution. Comme ce fut remarqué dès le 18ème siècle, ils ont l'unique particularité de concentrer des afférences de l'entièreté de la surface corticale. Cette position centrale et l'analyse de l'anatomie du réseau leur ont valu le rôle d'arbitre central du cerveau, réglant les conflits entre processus neuronaux concomitants bien qu'incompatibles. Au sein du réseau, le noyau subthalamique jouit d'une notoriété particulière. Ce noyau, sur la base de ses afférences corticales, et en vertu de ses projections sur le soma des neurones pallidaux, aurait pour fonction de filtrer les programmes comportementaux codés par le striatum et concourant pour leur expression. Rapporté aux théories de la prise de décision, le noyau subthalamique fixerait le seuil décisionnel, ou la quantité d'information à accumuler en faveur d'une option comportementale afin qu'elle soit exprimée. Mais si ce petit noyau est devenu si célèbre, c'est surtout qu'il est la cible d'une procédure chirurgicale spectaculaire: la stimulation cérébrale profonde. Cette opération du cerveau est le dernier recours pour les patients souffrant d'une maladie de Parkinson ou d'un trouble obsessionnel compulsif sévère. Elle parvient même parfois à faire disparaître leurs symptômes. Malgré cette efficacité remarquable, les mécanismes de la stimulation cérébrale profonde restent inconnus. Il faut, entre autres, blâmer l'obscurité qui règne encore sur le noyau subthalamique, car les fonctions mentionnées ci-dessus restent des conjectures théoriques en manque de validation expérimentale. La première étape de ce travail a été d'en valider les bases anatomiques. En effet, l'existence d'une voie fronto-subthalamique - nécessaire au modèle - n'était connue que sur la base d'études menées chez le rat. Nous avons démontré, par des méthodes de traçage axonal, l'existence de cette connexion chez le primate. En sus, cette connexion aura permis de redéfinir les frontières médiales du noyau subthalamique avec les conséquences cliniques qui peuvent en être tirées. Le deuxième objectif global de cette thèse était de tester la validité fonctionnelle du modèle, la stimulation cérébrale profonde offrant un accès rare aux activités du noyau subthalamique. Cependant, il était d'abord nécessaire de caractériser la population étudiée, à savoir des patients souffrants d'un trouble obsessionnel compulsif. Grâce à l'imagerie de diffusion nous démontrons une diminution ainsi qu'une désorganisation des connexions cortico-sous corticales, se traduisant probablement par un défaut de contrôle conscient sur le processus de sélection. Une étude de magnétoencéphalographie est en cours pour approfondir les changements d'activité corticale. Pour tester le rôle du noyau subthalamique dans l'établissement du seuil décisionnel nous avons enregistré son activité électrophysiologique pendant que les patients effectuaient une tâche de prise de décision perceptuelle. Nous démontrons que les neurones du noyau subthalamique ont une réponse multimodale, concordant en cela avec nos données anatomiques qui montrent une convergence d'informations au niveau du noyau subthalamique. De plus, une augmentation de l'activité est retrouvée dans les conditions attendues... / The basal ganglia are a network of subcortical structures of which the invariant architecture throughout vertebrate evolution suggests a key function in evolution. As was noted as early as the 18th century, they have the unique characteristic of concentrating afferences from the entire cortical surgace. Given this central position and the internal architecture of the network, they could provide a centralised selection mechanism in the brain, arbitrating between any two conflicting processes. Among the basal ganglia, the subthalamic nucleus has become of particular interest as it is the target of deep brain stimulation, a neurosurgical procedure used to treat severe Parkinson’s disease and obsessive-compulsive disorder. It would have for function to integrate contextual information from its cortical inputs to filter behavioural programs encoded by the striatum. Within the framework of decision-making models, this filtering function is akin to setting the decision threshold, or the amount of evidence required before selecting a program. However, this considerations remain hypothetical as they are lacking experimental support. The first objective of this work was to validate the anatomical basis of these assumptions. Indeed, the existence of a prefrontal-subthalamic pathway, necessary to expand the decision models to every type of decision, had only been demonstrated in rodents. We demonstrated its existence in the primate using anterograde axonal tracing. In addition, this projection will have allowed us to redefine the medial border of the subthalamic nucleus with the clinical consequences that that may have. The second objective of this thesis was to test the functional validity of the models, and specifically the role of the subthalamic nucleus in setting decision thresholds. Deep brain stimulation offers a rare access to the electrophysiology of this structure; however, it is a patient population, here obsessive-compulsive disorder patients. A first step was, therefore, to characterise this population, anatomically and behaviourally, to understand how it might be of use as a model of decision-making in the basal ganglia. We demonstrated a reduction in the strength of cortico-subcortical anatomical connections. We suggest that this prevents accurate conscious control over decision mechanisms. Behaviourally, patients displayed a pathologically low confidence levels in their decisions and we hypothesised that this would lead to an increase of the decision threshold and matching subthalamic activity. To test this, we recorded the activity of the subthalamic nucleus during a decision-making task. We demonstrate that subthalamic neurons have a multimodal activity, consistent with our demonstration of convergent cortical inputs. However, we were unable to demonstrate a link between subthalamic activity and decision threshold, although this may be due to technical considerations…

Neuro-computational models allow to study the brain mechanisms involved in intelligent behavior and extract essential computational principles which can be implemented in cognitive systems. They are a promising solution to achieve a brain-like artificial intelligence that can compete with natural intelligence on realistic behaviors. A crucial property of intelligent behavior is motivation, defined as the incentive to interact with the world in order to achieve specific goals, either extrinsic (obtaining rewards such as food or money, or avoiding pain) or intrinsic (satisfying one’s curiosity, fun). In the human brain, motivated or goal-directed behavior depends on a network of different structures, including the prefrontal cortex, the basal ganglia and the limbic system. Dopamine, a neurotransmitter associated with reward processing, plays a central role in coordinating the activity of this network. It structures processing in high-level cognitive areas along a limbic-associative-motor gradient and impacts the learning capabilities of the whole system. In this habilitation thesis, I present biologically-constrained neuro-computational models which investigate the role of dopamine in visual object categorization and memory retrieval (Vitay and Hamker, 2008), reinforcement learning and action selection (Vitay and Hamker, 2010), the updating, learning and maintenance of working memory (Schroll et al., 2012) and timing processes (Vitay and Hamker, 2014). These models outline the many mechanisms by which the dopaminergic system regulates cognitive and emotional behavior: bistable processing modes in the cerebral cortex, modulation of synaptic transmission and plasticity, allocation of cognitive resources and signaling of relevant events. Finally, I present a neural simulator able to simulate a variety of neuro-computational models efficiently on parallel architectures (Vitay et al., 2015). / Neuronale Modelle nach dem Vorbild des Gehirns bieten die Möglichkeit intelligente, kognitive Prozesse nicht nur besser zu verstehen, sondern sie stellen auch eine vielversprechende Lösung dar, um eine Gehirn-ähnliche künstliche Intelligenz für Wahrnehmung und Verhaltensweisen zu erreichen, die mit natürlicher Intelligenz konkurrieren kann. Eine entscheidende Eigenschaft von intelligentem Verhalten ist Motivation, definiert als der Anreiz mit der Welt zu interagieren, um bestimmte Ziele zu erreichen, sei es extrinsisch (Belohnungen wie Nahrung oder Geld zu erhalten oder die Vermeidung von Schmerzen) oder intrinsisch (die Neugier zu befriedigen, Spaß zu haben). Im menschlichen Gehirn basiert motiviertes oder zielgerichtetes Verhalten auf einem Netzwerk von verschiedenen Strukturen, einschließlich des präfrontalen Cortex, der Basalganglien und des limbischen Systems. Dopamin, ein Neurotransmitter, welcher der Belohnungsverarbeitung zugeordnet wird, spielt eine zentrale Rolle bei der Koordination der Aktivität in diesem Netzwerk. Es strukturiert die Verarbeitung in High-Level-kognitiven Bereichen entlang eines limbischen-assoziativ-motor Gradienten und beinflusst die Lernfähigkeit des gesamten Systems. In dieser Habilitation, präsentiere ich biologisch motivierte neuronale Modelle, die die Rolle von Dopamin in der visuellen Objektkategorisierung und Gedächtnisabruf (Vitay and Hamker, 2008), Reinforcement Lernen und Aktionsauswahl (Vitay and Hamker, 2010), Aktualisierung, Lernen und Aufrechterhaltung von Arbeitsgedächtnis (Schroll et al., 2012) und Timing Prozessen (Vitay and Hamker, 2014) untersuchen. Diese Modelle beschreiben Mechanismen, durch die das dopaminerge System kognitives und emotionales Verhalten reguliert: bistabile Verarbeitungsmodi in der Hirnrinde, Plastizität und Modulation der synaptischen Übertragung, Zuweisung von kognitiven Ressourcen und Signalisierung von relevanten Ereignissen. Schließlich beschreibe ich einen neuronalen Simulator, der in in der Lage ist, eine Vielzahl von neuronalen Modellen effizient auf parallelen Architekturen zu simulieren (Vitay et al., 2015).

Intrastriatal injections of L-Pyroglutamate (L-PGA) in mice produced behavioral and neuropathological effects that resemble in part the kainate-injected rat striatal model of Huntington's Disease (HD). The behavioral responses induced after unilateral injections of L-PGA included circling, postural asymmetry of head and trunk and possible dyskinesias. The neuropil in the injected striatum contained dilated profiles, degenerating neurons and oligodendroglia, and numerous phagocytic microglial-like cells. A dose response relation existed. The size of the lesion (expressed as a percent volume of the striatum destroyed) ranged from 1±0.18% at 0.02 μmoles to 20.2±3.97% at 200 μmoles L-PGA (pH=7.3). L-PGA is a weak neurotoxin when compared to kainic acid. Several factors raise interest in the possible role of L-PGA in HD, including the recently reported elevated plasma levels of L-PGA in some HD patients [51,52], and these are considered in the discussion.

In assessing and assimilating the neurodevelopmental basis of the so-called movement disorders it is probably useful to establish certain concepts that will modulate both the variation and selection of affliction, mechanisms-processes and diversity of disease states. Both genetic, developmental and degenerative aberrations are to be encompassed within such an approach, as well as all deviations from the necessary components of behaviour that are generally understood to incorporate "normal" functioning. In the present treatise, both conditions of hyperactivity/hypoactivity, akinesia and bradykinesia together with a constellation of other symptoms and syndromes are considered in conjunction with the neuropharmacological and brain morphological alterations that may or may not accompany them, e.g. following neonatal denervation. As a case in point, the neuroanatomical and neurochemical points of interaction in Attention Deficit and Hyperactivity disorder (ADHD) are examined with reference to both the perinatal metallic and organic environment and genetic backgrounds. The role of apoptosis, as opposed to necrosis, in cell death during grain development necessitates careful considerations of the current explosion of evidence for brain nerve growth factors, neurotrophins and cytokines, and the processes regulating their appearance, release and fate. Some of these processes may posses putative inherited characteristics, like asynuclein, others may to greater or lesser extents be endogenous or semi-endogenous (in food), like the tetrahydroisoquinolines, others exogenous until inhaled or injested through environmental accident, like heavy metals, e.g. mercury. Another central concept of neurodevelopment is cellular plasticity, thereby underlining the essential involvement of glutamate systems and N-methyl-D-aspartate receptor configurations. Finally, an essential assimilation of brain development in disease must delineate the relative merits of inherited as opposed to environmental risks not only for the commonly-regarded movement disorders, like Parkinson's disease, Huntington's disease and epilepsy, but also for afflictions bearing strong elements of psychosocial tragedy, like ADHD, autism and Savantism.

Decision-making is comprised of an incredibly varied set of behaviours. However, all vertebrates tend to repeat previously rewarding actions and avoid those that have led to loss, behaviours known collectively as the win-stay, lose-shift strategy. This response strategy is supported by the sensorimotor striatum and nucleus accumbens, structures also implicated in spatial processing and the integration of sensory information in order to guide motor action. Therefore, choices may be represented as spatial-motor actions whose value is determined by the rewards and punishments associated with that action. In this dissertation I demonstrate that the location of choices relative to previous rewards and punishments, rather than their identities, determines their value. Chapters 2 and 4 demonstrate that the location of rewards and punishments drives future decisions to win-stay or lose-shift towards that location. Even when choices differ in colour or shape, choice value is determined by location, not visual identity. Chapter 3 compares decision-making when two, six, twelve, or eighteen choices are present, finding that the value of a win or loss is not tied to a single location, but is distributed throughout the choice environment. Finally, Chapter 5 provides anatomical support for the spatial-motor basis of choice. Specifically, win-stay responses are associated with greater oscillatory activity than win-shift responses in the motor cortex corresponding to the hand used to make a choice, whereas lose-shift responses are accompanied by greater activation of frontal systems compared to lose-stay responses. The win-stay and lose-shift behaviours activate structures known to project to different regions of the striatum. Overall, this dissertation provides behavioural evidence that choice location, not visual identity, determines choice value. / Thesis / Doctor of Philosophy (PhD)

Long-term potentiation (LTP) and depression (LTD) are activity dependent long-lasting changes in synaptic efficacy and have been proposed as mechanisms for learning and memory. Although the exact relationship of LTP and LTD to memory is not known, they do share some properties and mechanisms that relate to memory, such as the strengthening and weakening of synapses. LTP and LTD have been studied extensively in hippocampal brain-slice preparations, due to its relatively organized structure, ease of induction, and its critical function in memory storage. Less work has been done in the neocortex despite the belief that it is heavily involved in the storage of long-term memories. Activity dependent plasticity has also been demonstrated in the basal ganglia in vivo and in vitro, but the results have been somewhat inconsistent. The experiments presented in this thesis explore a novel form of neural plasticity in two excitatory pathways (corticostriatal and thalamocortical) of the basal ganglia motor loop in the intact brain in awake, freely behaving rats. In thalamocortical slice preparations, simultaneous presynaptic stimulation and postsynaptic depolarization can induce L TP in animals prior to the critical period. However the results presented in this thesis show that applied stimulation to the thalamocortical pathway failed to produce either LTP or LTD in the awake freely moving animal.Corticostriatal LTD has been shown in slice preparations following direct tetanic stimulation of the striatum. In the current experiment, cortical stimulation failed to induce LTD although there was an observable decrease in the evoked potential following low-frequency stimulation. Corticostriatal L TP has been shown to depend on the type of stimulation applied. High-frequency and theta burst stimulation produced long-lasting changes in response amplitude in the corticostriatal pathway, with theta burst stimulation appearing to be the more effective stimulation protocol for inducing LTP in both the early and late components. Paired stimulation of the substantia nigra pars compacta and cortex indicated a modulatory action of dopamine on corticostriatal synaptic plasticity. Pairing led to a stable increase in the amplitude of LTP of both early and late components. We also report that a temporal relationship exists in the striatum with respect to the release of nigral dopamine and cortical glutamate. Simultaneous stimulation produced a more robust L TP compared to the two other conditions in which there was an applied stimulation delay to either the corticostriatal or nigrostriatal pathway. The results demonstrate the mechanistic differences, not only between the thalamocortical and corticostriatal pathways, but also slice and anesthetized preparations. The results also emphasize the need for further study on mechanisms of L TP and LTD in the various excitatory and inhibitory pathways of the basal ganglia motor loop. / Thesis / Doctor of Philosophy (PhD)

Accumulating evidence suggests that dopaminergic neurons show significant task-related diversity. Curiously, dopamine concentration and dopamine axon activity show spatio-temporal wave patterns in the dorsal striatum. What could be the function of this wave-like dynamics of dopamine in the striatum, particularly in Reinforcement Learning? This work introduces a novel Reinforcement Learning algorithm that exploits the wave-like dynamics of dopamine to increase speed, reliability and flexibility in decision-making. An agent can form a cognitive map by exploring the environment and obtaining the information about the expectation of time spent in each future state given a departing state (i.e. the Successor Representation). This map captures the temporal connections of the visited states and outlines several possible state transition trajectories leading to the reward. Using the cognitive map, following a single reward delivery, the reward prediction errors can be computed for each state. In the cognitive map, states leading to the reward possess a high positive error, while temporally distant states retain smaller errors. Thus, the dynamics of errors exhibit a wave front travelling in the cognitive map. Under the assumption of the neurons representing adjacent states in the cognitive map are also spatial neighbors, it automatically follows that the reward prediction error carrying signal will also show wave-like dynamics in space. By exploiting the dopamine waves, the proposed Reinforcement Learning approach outperforms three classical Reinforcement Learning algorithms: basic SARSA, the Successor Representation and SARSA with eligibility traces. Consequently, the algorithm suggests conditions under which wave-like dynamics of dopamine release in the striatum can have direct functional implications for learning. / En ökande mängd bevis pekar på att dopaminerga nervceller uppvisar en betydande uppgiftsrelaterad diversitet. Märkligt nog uppvisar såväl dopaminkoncentrationen som aktiviteten i dopaminerga axon i dorsala striatum en vågliknande dynamik. Vilken funktion kan dopaminets vågliknande dynamik tänkas fylla i striatum, särskillt vid förstärkningsinlärning? I detta arbete introduceras en ny förstärkningsinlärningsalgoritm som utnyttjar dopaminets vågliknande dynamik för att öka snabbheten, tillförlitligheten och flexbiliteten vid beslutsfattande. En agent kan skapa en kognitiv karta genom att utforska en miljö och tillgodogöra sig information om den förväntade tiden som kommer tillbringas i varje framtida tillstånd givet ett starttillstånd (en så kallad successionsrepresentation). Denna karta fångar upp de tidsmässiga förbindelserna mellan besökta tillstånd och ger en skiss för flera möjliga serier av tillståndsövergångar som leder till belöning. Genom att använda denna kognitiva karta efter en enskild belöning kan belöningsförutsägningsfel beräknas för varje tillstånd. I den kognitiva kartan har tillstånd som leder till belöning ett stort positivt fel, medan tidsmässigt avlägsna tillstånd har mindre fel. Detta ger upphov till att dynamiken för felen uppvisar en vågfront in den kognitiva kartan. Under antagandet att nervceller som representerar närliggande tillstånd i den kognitiva kartan också är fysiska grannar, följer det automatiskt att signalen för belöningsförutsägningsfel också uppvisar en våglikannde dynamik i rummet. Genom att utnyttja dopaminvågor överträffar den föreslagna förstärkningsinlärningsalgoritmen tre klassiska förstärkningsinlärningsalgoritmer: vanlig SARSA, successionsrepresentation, och SARSA med kvalificeringsspår. Algoritmen förslår därför betingelser under vilka en vågliknande dynamik av dopaminfrisättning i striatum kan ha direkta funktionella implikationer för inlärning.