Brain-Computer Interface (BCI) offers the opportunity to paralyzed patients to control their movements without any neuromuscular activity. Signal processing of neuronal activity enables to decode movement intentions. Ability for patient to control an effector is closely linked to this decoding performance. In this study, I tackle a recent way to decode neuronal activity: Deep learning. The study is based on public data extracted by Schalk et al. for BCI Competition IV. Electrocorticogram (ECoG) data from three epileptic patients were recorded. During the experiment setup, the team asked subjects to move their fingers and recorded finger movements thanks to a data glove. An artificial neural network (ANN) was built based on a common BCI feature extraction pipeline made of successive convolutional layers. This network firstly mimics a spatial filtering with a spatial reduction of sources. Then, it realizes a time-frequency analysis and performs a log power extraction of the band-pass filtered signals. The first investigation was on the optimization of the network. Then, the same architecture was used on each subject and the decoding performances were computed for a 6-class classification. I especially investigated the spatial and temporal filtering. Finally, a preliminary study was conducted on prediction of finger movement. This study demonstrated that deep learning could be an effective way to decode brain signal. For 6-class classification, results stressed similar performances as traditional decoding algorithm. As spatial or temporal weights after training are slightly described in the literature, we especially worked on interpretation of weights after training. The spatial weight study demonstrated that the network is able to select specific ECoG channels notified in the literature as the most informative. Moreover, the network is able to converge to the same spatial solution, independently to the initialization. Finally, a preliminary study was conducted on prediction of movement position and gives encouraging results.

No description available.

Brain computer interfaces (BCIs) are devices capable of relaying information directly from the brain to a digital device. BCIs have been proposed for a diverse range of clinical and commercial applications; for example, to allow paralyzed subjects to communicate, or to improve machine human interactions. At their core, BCIs need to predict the current state of the brain from variables measuring functional physiology. Functional near infrared spectroscopy (fNIRS) is a non-invasive optical technology able to measure hemodynamic changes in the brain. Along with electroencephalography (EEG), fNIRS is the only technique that allows non-invasive and portable sensing of brain signals. Portability and wearability are very desirable characteristics for BCIs, as they allow them to be used in contexts beyond the laboratory, extending their usability for clinical and commercial applications, as well as for ecologically valid research. Unfortunately, due to limited access to the brain, non-invasive BCIs tend to suffer from low accuracy in their estimation of the brain state. It has been suggested that feedback could increase BCI accuracy as the brain normally relies on sensory feedback to adjust its strategies. Despite this, presenting relevant and accurate feedback in a timely manner can be challenging when processing fNIRS signals, as they tend to be contaminated by physiological and motion artifacts. In this dissertation, I present the hardware and software solutions we proposed and developed to deal with these challenges. First, I will talk about ninjaNIRS, the wearable open source fNIRS device we developed in our laboratory, which could help fNIRS neuroscience and BCIs to become more accessible. Next, I will present an adaptive filter strategy to recover the neural responses from fNIRS signals in real-time, which could be used for feedback and classification in a BCI paradigm. We showed that our wearable fNIRS device can operate autonomously for up to three hours and can be easily carried in a backpack, while offering noise equivalent power comparable to commercial devices. Our adaptive multimodal Kalman filter strategy provided a six-fold increase in contrast to noise ratio of the brain signals compared to standard filtering while being able to process at least 24 channels at 400 samples per second using a standard computer. This filtering strategy, along with visual feedback during a left vs right motion imagery task, showed a relative increase of accuracy of 37.5% compared to not using feedback. With this, we show that it is possible to present relevant feedback for fNIRS BCI in real-time. The findings on this dissertation might help improve the design of future fNIRS BCIs, and thus increase the usability and reliability of this technology.

No description available.

Cette thèse s'inscrit dans le cadre du projet Européen intégré VERE (Virtual Embodiement and Robotics re-Embodiement). Il s'agit de proposer une architecture logicielle intégrant un ensemble de stratégies de contrôle et de retours informationnels basés sur la "fonction tâche" pour incorporer (embodiment) un opérateur humain dans un humanoïde robotique ou un avatar notamment par la pensée. Les problèmes sous-jacents peuvent se révéler par le démonstrateur suivant (auquel on souhaite aboutir à l'issue de cette thèse). Imaginons un opérateur doté d'une interface cerveau-ordinateur ; le but est d'arriver à extraire de ces signaux la pensée de l'opérateur humain, de la traduire en commandes robotique et de faire un retour sensoriel afin que l'opérateur s'approprie le "corps" robotique ou virtuel de son "avatar". Une illustration cinématographique de cet objectif est le film récent "Avatar" ou encore "Surrogates". Dans cette thèse, on s'intéressera tout d'abord à certains problèmes que l'on a rencontré en travaillant sur l'utilisation des interfaces cerveau-ordinateur pour le contrôle de robots ou d'avatars, par exemple, la nécessité de multiplier les comportements ou les particularités liées aux retours sensoriels du robot. Dans un second temps, nous aborderons le cœur de notre contribution en introduisant le concept d'interface cerveau-ordinateur orienté objet pour le contrôle de robots humanoïdes. Nous présenterons ensuite les résultats d'une étude concernant le rôle du son dans le processus d'embodiment. Enfin, nous montrerons les premières expériences concernant le contrôle d'un robot humanoïde en interface cerveau-ordinateur utilisant l'électrocorticographie, une technologie d'acquisition des signaux cérébraux implantée dans la boîte crânienne. / This thesis is part of the European project VERE (Virtual Embodiment and Robotics re-Embodiment). The goal is to propose a software framework integrating a set of control strategies and information feedback based on the "task function" in order to embody a human operator within a humanoid robot or a virtual avatar using his thoughts. The underlying problems can be shown by considering the following demonstrator. Let us imagine an operator equipped with a brain-computer interface; the goal is to extract the though of the human operator from these signals, then translate it into robotic commands and finally to give an appropriate sensory feedback to the operator so that he can appropriate the "body", robotic or virtual, of his avatar. A cinematographic illustration of this objective can be seen in recent movies such as "Avatar" or "Surrogates". In this thesis, we start by discussing specific problems that we encountered while using a brain-computer interface for the control of robots or avatars, e.g. the arising need for multiple behaviours or the specific problems induced by the sensory feedback provided by the robot. We will then introduce our main contribution which is the concept of object-oriented brain-computer interface for the control of humanoid robot. We will then present the results of a study regarding the role of sound in the embodiment process. Finally, we show some preliminary experiments where we used electrocorticography (ECoG)~--~a technology used to acquire signals from the brain that is implanted within the cranium~--~to control a humanoid robot.

Interface Cérebro-Computador é um meio que permite a comunicação do cérebro com dispositivos externos e tem como principal público-alvo as pessoas com problemas motores, incapazes de se comunicarem e/ou se locomoverem. Uma das principais aplicações são os soletradores baseados no P300 que fornecem um meio de indivíduos se comunicarem através de um teclado virtual. Devolver a capacidade de comunicação para uma pessoa é de extrema importância para a qualidade de vida das pessoas. Esse tipo de aplicação possui diversos desafios, um deles é a necessidade da BCI ser treinada especificamente para cada indivíduo. Esse treinamento pode levar horas e até mesmo dias. Uma das formas de diminuir esse tempo é utilizar um dos conjuntos de canais pré-definidos que são sugeridos na literatura, porém esses conjuntos não garantem um funcionamento adequado da BCI, o que pode frustar os indivíduos não desejar mais utilizar uma BCI. Para solucionar esse problema, é proposto no presente trabalho a seleção de canais a partir de um conjunto de canais para agilizar o processo de treinamento e atingir um ótimo desempenho com a BCI. / Brain-Computer Interface is a means that allows the communication of the brain with external devices and has as main target audience the people with motor problems, unable to communicate and/or move around. One of the main applications is the P300-based spellers that provide a means for individuals to communicate through a virtual keyboard. Recovering the ability to communicate to a person is of extreme importance to the quality of peoples lives. This type of application has several challenges, one of which is the need for BCI to be trained specifically for each individual. This training can take hours and even days. One of the ways to decrease this time is to use one of the predefined set of channels that are suggested in the literature, but these sets do not guarantee an adequate functioning of BCI, which can frustrate individuals no longer want to use a BCI. To solve this problem, it is proposed in the present work the selection of channels from a set of channels to accelerate the training process and achieve optimal performance with BCI.

<p style="text-align: justify">Cilj ove doktorske disertacije je da predstavi model i računarski sistem koji re&scaron;ava problem osoba sa invaliditetom koja nisu u mogućnosti da koriste ruke ili funkciju govora u ostvarivanju komunikacije. Disertacija se bavi problematikom mogućnosti primene ekonomski pristupačnih asistivnih tehnologija u domenu primene servisa elektronske uprave. Od asistivnih tehnologija disertacija istražuje mogućnosti primene neinvazivne BCI tehnologije u poređenju sa sistemima baziranih na HD kamerama. Razvijen je računarski sistem koji omogućava integraciju u operativni sistem i upotrebu računara za unos komandi upotrebom detekcije moždanih talasa.</p> / <p>The main objective of this doctoral thesis is to present the model and a computer system that solves the communication problem of people with disabilities (people who cannot use their hands or the function of speech communication). The dissertation researches the possibility to apply economic affordable assistive technologies in the field of application of e-government services. Thesis explores the possibilities of application of non-invasive BCI technology compared to systems based on HD<br />cameras. Has been developed a computer system that allows the integration into the<br />operating system that allow to enter commands by the detection of brain waves.</p>

Ces travaux de recherche présentent une modélisation/Simulation du Test BCI (Bulk Current Injection) sous contrainte RTCA – DO 160, test de qualification des équipements très contraignant en termes de coûts et délais. Lors de sa réalisation, il présente aussi beaucoup de disparités dont il est parfois difficile d’identifier les sources et de les maîtriser lors du test avec une maquette physique. La simulation présente tout son intérêt dans l’étude de ces phénomènes (qui peuvent avoir un impact non moins significatif sur les résultats de test) mais aussi la répétabilité des essais.Dans un premier temps, une méthode de modélisation du couplage pince d’injection de courant et câbles est établie qui tient compte de l’évolution en fonction de la fréquence du noyau de ferrite du transformateur RF (Pince de courant) et des paramètres linéiques des câbles. Deux modèles sont principalement proposés dans ces travaux :- Un modèle générique, modèle circuit constitué uniquement d’éléments passifs RLC et élaboré (sous SPICE) à partir de la mesure des paramètres S. Ce modèle fait apparaitre la zone de couplage entre pince et câbles au secondaire.- Un modèle magnétique, macro-modèle développé sous le logiciel Flux2D. Les paramètres géométriques du modèle sont renseignés à partir de la connaissance des dimensions de la pince (diamètres intérieur /extérieur, longueur) et des câbles (diamètres/longueurs). Les paramètres physiques de la pince de courant particulièrement la perméabilité magnétique complexe du noyau de ferrite est obtenue à partir de la mesure du coefficient de réflexion au port d’entrée de la pince et extraction en post-traitement.Les validations dans le domaine fréquentiel ont été effectuées avec une bonne corrélation entre simulations et mesures dans la bande BCI [10 kHz – 400 MHz]. Ces résultats obtenus ont permis l'élaboration d'un modèle complet du test BCI (sous l’outil logiciel PAM-CEM/CRIPTE) qui tient compte d’un toron aéronautique complexe et de l’EST (Équipement Sous Test modélisé au laboratoire Ampère de Lyon). Il est constitué du générateur de perturbation (qui fait office de pince d’injection de courant), du modèle du toron de câbles (constitué de paires torsadées blindées, de paires non-blindées, …) et de l’EST (Équipement Sous Test) dans la bande [10 kHz – 400 MHz]. La bonne concordance entre simulations et mesures laisse présager une utilisation par les avionneurs ou équipementiers pour des études paramétriques concernant le test BCI (influence de la disposition des câbles, queue de cochon, longueur toron, disposition de l’EST par rapport au plan de masse, …) et/ou pour une virtualisation dans une phase de pré-qualification des équipements.Mots clés : CEM (Compatibilité ElectroMagnétique), Test BCI (Bulk Current Injection), Modélisation/Simulation, Norme aéronautique DO 160. / This work presents a modeling/simulation approach of BCI (Bulk Current Injection) test under constraint RTCA - DO 160. This qualification test of equipment is very constraining in terms of cost and deadline. During the test, there are also many disparities for which it is difficult to identify sources (and control them) with a physical test setup. The simulation is of interest in the study of phenomena (which can have negative impacts on test results) but also the repeatability of tests.First, a method of modeling for the probe/cables coupling is established which takes into account the variation with frequency of the RF transformer (current probe) of the magnetic ferrite core and the linear parameters of cables (skin/ proximity effects). Two models are proposed in this work:- A generic model which is made up solely of passive elements RLC and elaborated (with SPICE software) from the measurement of S-parameters. It shows the coupling zone between probe and cables (secondary winding).- A magnetic macro-model developed with the Flux2D software. Its geometrical parameters are defined from dimensions of the probe (inner/outer diameter, length) and cables (diameters / length). Physical parameters of the current probe, particularly the complex magnetic permeability of the ferrite core, are obtained from measurement of the S-parameter at the input port of the probe and post-treatment extraction.Frequency domain validations were performed with a good correlation between simulations and measurements in the BCI band ([10 kHz - 400 MHz]).These results led to the development of a complete virtual BCI test (with PAM-CEM / CRIPTE software), which take into account an aeronautic complex harness and a DUT (Device Under Test which is modeled at Ampère laboratory). It consists of disturbance generator, harness model (consisting of shielding twisted cables, no shielding cables, etc.) and DUT (Device Under Test) in the band [10 kHz - 400 MHz].The good correlation between simulations and measurements suggests a use by the aircraft manufacturers or equipment manufacturers for parametric studies about BCI test (uncertainties related to cable positions, pigtail, cable length, DUT position with respect to the ground plane, ...) and /or for virtualization in a pre-qualification phase of the equipment.Keywords: EMC (ElectroMagnetic Compatibility), BCI (Bulk Current Injection) test, Modeling/Simulation, DO 160 aeronautic standard.

Restorer la faculté de parler chez des personnes paralysées et aphasiques pourrait être envisagée via l’utilisation d’une interface cerveau-machine permettant de contrôler un synthétiseur de parole en temps réel. L’objectif de cette thèse était de développer trois aspects nécessaires à la mise au point d’une telle preuve de concept.Premièrement, un synthétiseur permettant de produire en temps-réel de la parole intelligible et controlé par un nombre raisonable de paramètres est nécessaire. Nous avons choisi de synthétiser de la parole à partir des mouvements des articulateurs du conduit vocal. En effet, des études récentes ont suggéré que l’activité neuronale du cortex moteur de la parole pourrait contenir suffisamment d’information pour décoder la parole, et particulièrement ses propriété articulatoire (ex. l’ouverture des lèvres). Nous avons donc développé un synthétiseur produisant de la parole intelligible à partir de données articulatoires. Dans un premier temps, nous avons enregistré un large corpus de données articulatoire et acoustiques synchrones chez un locuteur. Ensuite, nous avons utilisé des techniques d’apprentissage automatique, en particulier des réseaux de neurones profonds, pour construire un modèle permettant de convertir des données articulatoires en parole. Ce synthétisuer a été construit pour fonctionner en temps réel. Enfin, comme première étape vers un contrôle neuronal de ce synthétiseur, nous avons testé qu’il pouvait être contrôlé en temps réel par plusieurs locuteurs, pour produire de la parole inetlligible à partir de leurs mouvements articulatoires dans un paradigme de boucle fermée.Deuxièmement, nous avons étudié le décodage de la parole et de ses propriétés articulatoires à partir d’activités neuronales essentiellement enregistrées dans le cortex moteur de la parole. Nous avons construit un outil permettant de localiser les aires corticales actives, en ligne pendant des chirurgies éveillées à l’hôpital de Grenoble, et nous avons testé ce système chez deux patients atteints d’un cancer du cerveau. Les résultats ont montré que le cortex moteur exhibe une activité spécifique pendant la production de parole dans les bandes beta et gamma du signal, y compris lors de l’imagination de la parole. Les données enregistrées ont ensuite pu être analysées pour décoder l’intention de parler du sujet (réelle ou imaginée), ainsi que la vibration des cordes vocales et les trajectoires des articulateurs principaux du conduit vocal significativement au dessus du niveau de la chance.Enfin, nous nous sommes intéressés aux questions éthiques qui accompagnent le développement et l’usage des interfaces cerveau-machine. Nous avons en particulier considéré trois niveaux de réflexion éthique concernant respectivement l’animal, l’humain et l’humanité. / Restoring natural speech in paralyzed and aphasic people could be achieved using a brain-computer interface controlling a speech synthesizer in real-time. The aim of this thesis was thus to develop three main steps toward such proof of concept.First, a prerequisite was to develop a speech synthesizer producing intelligible speech in real-time with a reasonable number of control parameters. Here we chose to synthesize speech from movements of the speech articulators since recent studies suggested that neural activity from the speech motor cortex contains relevant information to decode speech, and especially articulatory features of speech. We thus developed a speech synthesizer that produced intelligible speech from articulatory data. This was achieved by first recording a large dataset of synchronous articulatory and acoustic data in a single speaker. Then, we used machine learning techniques, especially deep neural networks, to build a model able to convert articulatory data into speech. This synthesizer was built to run in real time. Finally, as a first step toward future brain control of this synthesizer, we tested that it could be controlled in real-time by several speakers to produce intelligible speech from articulatory movements in a closed-loop paradigm.Second, we investigated the feasibility of decoding speech and articulatory features from neural activity essentially recorded in the speech motor cortex. We built a tool that allowed to localize active cortical speech areas online during awake brain surgery at the Grenoble Hospital and tested this system in two patients with brain cancer. Results show that the motor cortex exhibits specific activity during speech production in the beta and gamma bands, which are also present during speech imagination. The recorded data could be successfully analyzed to decode speech intention, voicing activity and the trajectories of the main articulators of the vocal tract above chance.Finally, we addressed ethical issues that arise with the development and use of brain-computer interfaces. We considered three levels of ethical questionings, dealing respectively with the animal, the human being, and the human species.

Syftet med uppsatsen är att ge en uppdaterad översikt av området BCI (Brain Computer Interface) och undersöka vad som hänt sedan begreppet introducerades i forskningssammanhang; vilka praktiska resultat forskningen lett till och vilka tillämpningar som tillkommit. Metoden som företrädesvis används är litteraturstudie som tecknar bakgrund och enkät. Därefter följer en diskussion där utmaningar för framtiden, potential och tillämpningar i BCI-tekniken behandlas utifrån ett neuropsykologiskt perspektiv. Kommer BCI-tekniken att implementeras på samma sätt som radio, TV och telekommunikationer i samhället och vilka etiska och tekniska problem finns idag. För att skildra allmänhetens uppfattning om BCI genomfördes en webbaserad enkätundersökning (survey) i form av pilotstudie (n=32) som syftar till att ge en indikation på attityder och hur allmänhetens opinion med avseende på tillämpningar i samtiden och jämförelser med avseende på teknisk bakgrund.