The motor nervous system is one of the main systems of the body and is our principle means ofbehavior. Some of the most debilitating and wide spread disorders are motor systempathologies. In particular the basal ganglia are complex networks of the brain that control someaspects of movement in all vertebrates. Although these networks have been extensively studied,lack of proper methods to study them on a system level has hindered the process ofunderstanding what they do and how they do it. In order to facilitate this process I have usedcomputational models as an approach that can faithfully take into account many aspects of ahigh dimensional multi faceted system.In order to minimize the complexity of the system, I first took agnathan fish and amphibians asmodeling animals. These animals have rather simple neuronal networks and have been wellstudied so that developing their biologically plausible models is more feasible. I developedmodels of sensory motor transformation centers that are capable of generating basic behaviorsof approach, avoidance and escape. The networks in these models used a similar layeredstructure having a sensory map in one layer and a motor map on other layers. The visualinformation was received as place coded information, but was converted into population codedand ultimately into rate coded signals usable for muscle contractions.In parallel to developing models of visuomotor centers, I developed a novel model of the basalganglia. The model suggests that a subsystem of the basal ganglia is in charge of resolvingconflicts between motor programs suggested by different motor centers in the nervous system.This subsystem that is composed of the subthalamic nucleus and pallidum is called thearbitration system. Another subsystem of the basal ganglia called the extension system which iscomposed of the striatum and pallidum can bias decisions made by an animal towards theactions leading to lower cost and higher outcome by learning to associate proper actions todifferent states. Such states are generally complex states and the novel hypothesis I developedsuggests that the extension system is capable of learning such complex states and linking themto appropriate actions. In this framework, striatal neurons play the role of conjunction (BooleanAND) neurons while pallidal neurons can be envisioned as disjunction (Boolean OR) neurons.In the next set of experiments I tried to take the idea of basal ganglia subsystems to a new levelby dividing the rodent arbitration system into two functional subunits. A rostral group of ratpallidal neurons form dense local inhibition among themselves and even send inhibitoryprojections to the caudal segment. The caudal segment does not project back to its rostralcounterpart, but both segments send inhibitory projections to the output nuclei of the rat basalganglia i.e. the entopeduncular nucleus and substantia nigra. The rostral subsystems is capableof precisely detecting one (or several) components of a rudimentary action and suppress othercomponents. The components that are reinforced are those which lead to rewarding stateswhereas those that are suppressed are those which do not. The hypothesis explains neuronalmechanisms involved in this process and suggests that this subsystem is a means of generatingsimple but precise movements (such as using a single digit) from innate crude actions that theanimal can perform even at birth (such as general movement of the whole limb). In this way, therostral subsystem may play important role in exploration based learning.In an attempt to more precisely describe the relation between the arbitration and extensionsystems, we investigated the effect of dynamic synapses between subthalamic, pallidal andstriatal neurons and output neurons of the basal ganglia. The results imply that output neuronsare sensitive to striatal bursts and pallidal irregular firing. They also suggest that few striatalneurons are enough to fully suppress output neurons. Finally the results show that the globuspallidus exerts its effect on output neurons by direct inhibition rather than indirect influence viathe subthalamic nucleus. / <p>QC 20131209</p>

The interval programming model (IvP) is a mathematical programmingmodel for representing and solving multi-objective optimizationproblems. The central characteristic of the model is the use ofpiecewise linearly defined objective functions and a solution methodthat searches through the combination space of pieces rather thanthrough the actual decision space. The piecewise functions typicallyrepresent an approximation of some underlying function, but thisconcession is balanced on the positive side by relative freedom fromfunction form assumptions as well as the assurance of global optimality.In this paper the model and solution algorithms are described, and theapplicability of IvP to certain applications arediscussed.

L'apprentissage de geste est un processus complexe, impliquant de nombreux processus sous-jacents (psychomoteurs, cognitifs, biophysiques). Cet apprentissage peut être divisé en plusieurs grandes phases, définies par la capacité de l'apprenant à produire et à percevoir son geste. A l'instar d'un formateur adaptant son discours et son attitude à l'apprenant, un environnement d'apprentissage doit pouvoir adapter les aides qu'il fournit, d'une part en analysant l'état de l'apprenant, et d'autre part en prédisant les aides qui bénéficieront à son apprentissage. Nous proposons une approche modélisant les interactions entre l'apprenant et l'environnement d'apprentissage; c'est-à-dire l'évolution de l'état de l'apprenant en fonction des configurations successives de l'environnement, permettant une adaptation automatique et dynamique de la sélection d'aides. En s'appuyant sur des travaux portant sur la reconnaissance de gestes et les tuteurs intelligents, nous proposons un cadre formel permettant de représenter la qualité de l'activité gestuelle dans un espace métrique. Cette représentation interprète les résultats issus de modèles probabilistes comme des variables floues illustrant le niveau de l'apprenant sur chacune des dimensions du geste. Nous représentons l'environnement comme l'ensemble des actions possibles, chaque action étant représentée par un vecteur de paramètres estimant la situation dans laquelle l'action est la plus pertinente. Dans un premier temps, ces paramètres sont fixés par des experts, et nous considérons cette expertise comme parfaite. Dans un second temps, nous étudions une problématique omniprésente dans les domaines des environnements d'apprentissage, qui fait état de l'écart entre l’expertise et la réalité. Les domaines "mal définis" sont en effet répandus, du fait du coût et de la difficulté à réunir des experts, et de la complexité inhérente à la définition précise d'un domaine. Ne considérant plus le seul avis des experts, le processus de sélection d 'aides peut alors être vu comme une séquence de décisions dont l'objectif est de proposer à chaque itération l'action qui maximisera le gain en matière d'apprentissage. En s'appuyant sur des travaux récents portant sur les séquences de décisions, notre approche considère le processus de sélection d'aides comme un problème de bandit. Le problème de bandit vise à maximiser un gain lors d'une séquence de décisions, et modélise le compromis entre exploration (choisir une action dont l'influence est inconnue), et exploitation (choisir la meilleure action connue). Nous proposons dans ce cadre une extension des méthodes SoftMax. L'implémentation de notre modèle sur une plateforme d'apprentissage de la calligraphie a été réalisée dans le cadre d'une collaboration avec des experts de ce domaine. Nous montrons, au travers de deux cas d’étude, l'intérêt de notre modèle pour l'apprentissage de la calligraphie. Dans le premier cas d’étude, l'adaptation est construite depuis notre représentation du geste, et un ensemble de règles fixées par des experts. Nous y montrons l'avantage apporté par l'apport d'une diversité d'aides. Dans le second cas d'étude, nous comparons deux types d 'adaptation : une adaptation basée sur des règles, et une adaptation basée sur notre approche dynamique. Nous montrons les différences entre ces deux approches, et illustrons les avantages de l'approche dynamique lorsque les règles sont imprécises. / Gesture learning is a complex and multi-step process where trainees are supposed to improve several psychomotor and cognitive skills. This process can be divided into phases depending on trainees’ ability to perform and perceive their gestures. As human tutors adapt their behavior according to their perception and understanding of trainees learning situations' a learning environment should select an appropriate behavior from a representation of trainees’ learning states and a prediction of the potential influence of every possible behavior. The work presented in this document describes an approach modeling the interactions between a trainee and a learning environment: it represents trainees’ consecutive performances and the influence of the environment on these performances. This approach aims at permitting an adaptive selection of the pedagogical actions (i.e. behaviors) implemented in the environment. Relying on related works in the domains of gesture recognition and intelligent tutoring systems, we propose to represent the gestural activity in a metric space. This representation interprets results from a probabilistic mode as fuzzy variables highlighting trainees' level on every aspect of the gesture. We represent the environment as the set of actions it can select, every action being represented by a feature vector describing the learning situation maximizing the action's influence. As a first step, these features are given by a set of experts, and we consider the rules provided as perfect. As a second step, we study an ubiquitous issue in the field of learning environments, which is the difference between the rules provided by experts and the reality of trainees’ needs toward feedback. Ill-defined domains are indeed more and more common, as collecting expert knowledge is difficult and costly, and as studied learning domains are becoming more and more complex and difficult to define. In this second step, the selection process does not rely on expert knowledge, and this process can be seen as a sequence of decisions. At each iteration, the goal is thus to select the action which would maximize the reward in terms of benefits for trainees' learning. The action selection process is represented as a multi-armed bandit problem, where the goal is to compromise between exploration of unknown actions and exploitation of known actions. We present an extension of SoftMax methods which handles multi-dimensional contextual rewards. Taking advantage of the collaboration with calligraphy experts, a calligraphy training platform was implemented as part of this work. Two studies, where participants train on this platform, show the benefits of the proposed approach on calligraphy learning. In a first experiment, action selection is based on expert rules, and we show that providing a diversity of feedback improves skill acquisition. In a second experiment, we compare two configurations of the environment: a selection of actions based on expert rules, and a selection of actions based on the SoftMax method. We describe the difference between the influence of these two approaches on trainees' learning, and we point out the benefits of using dynamic rules.

In this thesis we pitted two views of action selection. On the one hand, a traditional view suggesting that action selection emerges from a sequential process whereby perception, cognition and action proceed serially and are subtended by distinct brain areas. On the other hand, an ecological view (formalized in the affordance competition hypothesis) advocating that action selection stems from the parallel implementation of potential action plans. In parallel, the competition between these action plans would be biased by relevant task factors. We first addressed the issue of the temporal dynamics of action selection processes in Chapter 2. We built a reaching task design that crucially gave equal opportunities for serial and parallel processing of cognitive and motor processes to occur. In our study, we first cued participants with probabilities associated to upcoming potential reaches. After several hundreds of milliseconds, participants were given a deterministic go signal indicating which target to reach for. They had to reach for the signaled target as fast as possible. Importantly, our design tries to cope with the biases involved in previous reaching tasks, allowing for a much more informative way to tackle the issue of serial versus parallel processing in action selection. We show that effects of action probability are not only present in the initiation time (i.e. the time it takes to initiate the movement), but crucially also in the movement time (i.e. the time interval between movement initiation and target reaching). Furthermore, an analysis of the movement trajectories showed that reach probability influenced the trajectories according to the predicted pattern. Thus, these results back up a system where cognitive and motor processes continuously interact with one another to come up with a decision. After clarifying the temporal dynamics, we concentrate our efforts on exposing the neural architecture of processes subtending action selection in Chapter 3. In a two-choice button press task, participants were first cued with predictive information regarding upcoming button presses. Crucially, we experimentally manipulated the amount of information in favor of specific button presses whilst adopting a design as similar as possible to those used in monkey neurophysiology (e.g. Cisek & Kalaska, 2005). Using fMRI, our results showed that as information in favor a button press increases, so does activity in the contralateral primary motor cortex, while activity in the ipsilateral primary motor cortex decreases. Moreover, we observed that primary motor regions are more tightly coupled with fronto-parietal areas in a condition involving a decision compared with a situation not implicating a decision between two button presses. Our results are compatible with an account predicting that decision-making emerges from motor areas, and therefore suggest that the architecture presented in the affordance competition hypothesis is not only valid in monkeys but also humans. In Chapter 4, we combine the findings acquired in the studies of chapter 2 and 3 with recent neurophysiological insights to develop a neuro-computational model capable of grasping the continuous interaction between cognitive and motor processes, responsible for the behavioral pattern in reach selection tasks. Our model functions on the principles of cascade forward models whereby activation at one stage of processing systematically spills to the next one, thereby substantially blurring the boundaries between perceptive, cognitive and motor processes. Contrary to most computational models confining action selection processes prior to action execution, our model allows for these processes to leak into action execution. Moreover, the threshold for action execution is not fixed, but rather dynamic and crucially depends on the activity pattern of the model’s primary motor neurons. We propose that the modification of the threshold is governed by the subthalamic nucleus, receiving direct input signals from the primary motor cortex and in turn imposing a dynamical brake on action execution. By including this dynamical threshold, our model has the advantage that it can release movement execution either rapidly or slowly depending on the context. Our model accounts not only for initiation times, but also movement times in reaching task studies. Furthermore, it can grasp the qualitative pattern of movement trajectories. This study suggests that to explain unfolding actions a classical fixed threshold is not sufficient, but rather an execution threshold level that is continuously being updated depending on the context is required. / Doctorat en Sciences psychologiques et de l'éducation / info:eu-repo/semantics/nonPublished

No description available.

Active movements are a key feature of human behavior. Even when we do not move our limbs we almost never stop guiding our eyes. As a minimal but omnipresent form of behavior, fast eye movements, called saccades, sample the visual world and determine to a large extent what we perceive. Despite being an integral part of visual perception, prevalent research practice treats the human subject as a passive observer who fixates a spot on the screen and is not allowed to move. Yet, learning sensorimotor interactions by active exploration in order to predict future changes and guide actions seems to be a fundamental principle of neural organization. This results in neural patterns of active behavior that can be fundamentally different from the neural processes revealed in movement-restricted laboratory settings questioning the transferability of results from experimental paradigms demanding fixation to real-world free viewing behavior. In this thesis, we aim at studying the neural mechanisms underlying free viewing behavior. In order to assess the fast, flexible and possibly distributed neural dynamics of active vision, we established a procedure for studying eye movements in magnetoencephalography (MEG) and investigated oscillatory signatures associated with sensorimotor processes of eye movements and saccade target selection, two fundamental processes of active vision. Electroencephalography (EEG) and MEG can non-invasively measure fast neural dynamics and hence seem ideally suited for studying active vision in humans. However, artifacts related to eye movements confound both EEG and MEG signals, and a thorough handling of these artifacts is crucial for investigating neural activities during active movements. Mostly, cleaning of ocular artifacts has been performed for occasional eye movements and blinks in fixation paradigms in EEG. Less is known about the impact of ocular artifacts and especially the saccadic spike on MEG. As a first step to enable active vision studies in MEG, we investigated ocular artifacts and possible ways of their separation from neural signals in MEG. We show that the saccadic spike seriously distorts the spatial and spectral characteristics of the MEG signal (Study 2). We further tested if electrooculogram (EOG) based regression is feasible for corneo-retinal artifact removal (Study 1). Due to an often-raised concern, we addressed if EOG regression eliminates neural activity when applied for MEG. Our results do not indicate such susceptibility and we conclude that EOG regression for removing the corneo-retinal artifact in MEG is suitable. Based on insights from both studies, we established an artifact handling procedure including EOG regression and independent component analysis (ICA) to assess the neural dynamics of active vision. In Study 3, we investigated spectral signatures of neuronal activity across cortex underlying saccade preparation, execution and re-fixation in a delayed saccade task. During preparation and execution, we found a dichotomic signature of gamma power increases and beta power decreases in widespread cortical areas related to saccadic control, including fronto-parietal structures. Saccade direction specific signatures resided in hemisphere lateralized changes in low gamma and alpha power in posterior parietal cortex during preparation extending to extrastriate areas during re-fixation. Real-world behavior implies the constant need to flexibly select actions between competing behavioral alternatives depending on both sensory input and internal states. In order to assess internally motivated viewing behavior, we compared neuronal activity of externally cued saccades with saccades to freely chosen, equally valuable targets. We found gamma band specific power increases in fronto-parietal areas that are likely to reflect a fast transient process of action guidance for sensory-guided saccades and a sustained process for internally selecting between competing behavioral alternatives. The sustained signature of internal action selection suggests that a decision between spatially oriented movements is mediated within sensorimotor structures by neural competition between assemblies encoding parallel evolving movement plans. Since our observations support the assumption that a decision emerges through the distributed consensus of neural activities within effector specific areas rather than in a distinct decision module, they argue for the importance of studying mental processes within their ecologically valid and active context. This thesis shows the feasibility of studying neural mechanisms of active vision in MEG and provides important steps for studying neurophysiological correlates of free viewing in the future. The observed spectrally specific, distributed signatures highlight the importance of assessing fast oscillatory dynamics across the cortex for understanding neural mechanisms mediating real-world active behavior.

Research in mobile robotics, unmanned systems, and autonomous man-portable vehicles has grown rapidly over the last decade. This push has taken the problems of robot cognition and behavioral control out of the lab and into the field. Two good examples of this are the DARPA Urban Challenge autonomous vehicle race and the RoboCup robot soccer competition. In these challenges, a mobile robot must be capable of completing complex, sophisticated tasks in a dynamic, partially observable and unpredictable environment. Such conditions necessitate a behavioral programming approach capable of performing high-level action selection in the presence of multiple goals of dynamically changing importance, and noisy, incomplete perception data. In this thesis, an approach to behavioral programming is presented that provides the designer with an intuitive method for building contextual intelligence while preserving the qualities of emergent behavior present in traditional behavior-based programming. This is done by using a modified hierarchical state machine for behavior arbitration in sequence with a command fusion mechanism for cooperative and competitive control. The presented approach is analyzed with respect to portability across platforms, missions, and functional requirements. Specifically, two landmark case-studies, the DARPA Urban Challenge and the International RoboCup Competition are examined. / Master of Science

No description available.

Cette thèse étudie les mécanismes de sélection de l'action et de choix de stratégie tels qu'ils apparaissent à travers des expériences animales et des enregistrements neurobiologiques. Nous proposons ensuite des modèles biologiquement plausibles de la sélection de l'action. L'objectif est de mieux comprendre le fonctionnement du cerveau chez les êtres vivants et de pouvoir endéduire des architectures de contrôle bio-inspirées, plus robustes et adaptées à l'environnement. Les modèles étudiés sont réalisés avec des réseaux de neurones artificiels, permettant de modéliser des régions cérébrales et ainsi pouvoir simuler le fonctionnement du cerveau, ce qui permet de tester nos hypothèses sur des robots et des agents virtuels.L'étude de la sélection de l'action pour des robots mobiles implique plusieurs approches. La sélection de l'action peut être étudiée du point de vue du choix entre plusieurs actions basiques, e.g. un choix binaire aller à gauche ou à droite. Ceci passe forcément par l'acquisition et la catégorisation d'instants et d'événements spéciaux, perçus ou effectués, qui représentent des contextes dans lesquels la perception change, le comportement est modifié ou bien la sélection est réalisée. Ainsi, la thèse traite aussi de l'acquisition, la catégorisation et l'encodage de ces événements importants dans la sélection del'action.Enfin, on s'intéressera à la sélection de l'action du point de vue de la sélection de stratégie. Les différents comportements peuvent être dirigés consciemment ou bien être des automatismes acquis avec l'habitude. Le but ici est d'explorer différentes approches pour que le robot puisse développer ces deux capacités, mais aussi d'étudier les interactions entre ces types de mécanismes dans la cadre de tâches de navigation.Les travaux de cette thèse se basent sur la modélisation du fonctionnement de différentes boucles hippocampo-cortico-basales impliquées dans des tâches de navigation, de sélection de l'action et de catégorisations multimodales. En particulier, nous avons un modèle de l'hippocampe permettant d'apprendre des associations spatio-temporelles et des conditionnements multimodaux entre des événements perceptifs. Il se base sur des associations sensorimotrices entre des cellules appelées cellules de lieu qui sont associées avec des actions pour définir des comportements cohérents. Le modèle fait aussi intervenir des cellules de transition hippocampiques, permettant de faire des prédictions temporelles sur les événements futurs. Celles-ci permettent l'apprentissagede séquences spatio-temporelles, notamment du fait qu'elles représentent le substrat neuronal à l'apprentissage d'une carte cognitive, située elle au niveau du cortex préfrontal et/ou pariétal.Ce type de carte permet de planifier des chemins à suivre en fonction des motivations du robot, ce qui permet de rejoindre différents buts précédemment découverts dans l'environnement. / This thesis aims at studying the different mechanisms involved in action selection and decision making processes, according to animal experiments and neurobiological recordings. For that matter, we propose several biologically plausible models for action selection. The goal is to achieve a better understanding of the animal's brain functions. This gives us the opportunity todevelop bioinspired control architectures for robots that are more robust and adaptative to a real environement. These models are based on Artificial Neural Networks, allowing us to test our hypotheses on simulations of different brain regions and function, implemented on robots and virtual agents.Action selection for mobile robots can be approached from different angles. This process can be seen as the selection between two possibilities, e.g. go left or go right. Those mechanisms involve the ability to learn and categorize specific events, encoding contexts where a change in the perception is perceived, a change in the behavior is noticed or the decision is made. There-fore, this thesis studies those capacities of acquisition, categorisation and coding of different events that can be relevant for action selection.We also, approach the action selection as a strategy selection. The different behaviors are guided consciously or through automated behavior learned as habits. We investigate different possibilities allowing a robot to develop those capacities. Also, we aim at studying interactions that can emerge between those mechanisms during navigational behaviors.The work presented in this these is based on the modelisation of the hippocampo-cotico-basal loops involved in the navigational behaviors, the action selection and the multimodal categorisation of events. We base our models on a previous model of the hippocampus for the learning of spatio-temporal associations and for multimodal conditionning of perceptive events. It is based on sensorimotor associations between place cells and actions to achieve navigational behaviors. The model involves also a specific type of hippocampic cells, named transition cells, for temporal prediction of future events. This capacity allows the model to learn spatio-temporal sequences, and it represents the neural substrate for the learning of a cognitive map, hypothesised to be localized in prefrontal and/or parietal areas. This kind of topological map allows to plan the behavior of the robot according to its motivations, which is used in goal orientedexperiments to achieve goals and capture rewards.

No description available.