Cette thèse s'inscrit dans le cadre du projet Européen intégré VERE (Virtual Embodiement and Robotics re-Embodiement). Il s'agit de proposer une architecture logicielle intégrant un ensemble de stratégies de contrôle et de retours informationnels basés sur la "fonction tâche" pour incorporer (embodiment) un opérateur humain dans un humanoïde robotique ou un avatar notamment par la pensée. Les problèmes sous-jacents peuvent se révéler par le démonstrateur suivant (auquel on souhaite aboutir à l'issue de cette thèse). Imaginons un opérateur doté d'une interface cerveau-ordinateur ; le but est d'arriver à extraire de ces signaux la pensée de l'opérateur humain, de la traduire en commandes robotique et de faire un retour sensoriel afin que l'opérateur s'approprie le "corps" robotique ou virtuel de son "avatar". Une illustration cinématographique de cet objectif est le film récent "Avatar" ou encore "Surrogates". Dans cette thèse, on s'intéressera tout d'abord à certains problèmes que l'on a rencontré en travaillant sur l'utilisation des interfaces cerveau-ordinateur pour le contrôle de robots ou d'avatars, par exemple, la nécessité de multiplier les comportements ou les particularités liées aux retours sensoriels du robot. Dans un second temps, nous aborderons le cœur de notre contribution en introduisant le concept d'interface cerveau-ordinateur orienté objet pour le contrôle de robots humanoïdes. Nous présenterons ensuite les résultats d'une étude concernant le rôle du son dans le processus d'embodiment. Enfin, nous montrerons les premières expériences concernant le contrôle d'un robot humanoïde en interface cerveau-ordinateur utilisant l'électrocorticographie, une technologie d'acquisition des signaux cérébraux implantée dans la boîte crânienne. / This thesis is part of the European project VERE (Virtual Embodiment and Robotics re-Embodiment). The goal is to propose a software framework integrating a set of control strategies and information feedback based on the "task function" in order to embody a human operator within a humanoid robot or a virtual avatar using his thoughts. The underlying problems can be shown by considering the following demonstrator. Let us imagine an operator equipped with a brain-computer interface; the goal is to extract the though of the human operator from these signals, then translate it into robotic commands and finally to give an appropriate sensory feedback to the operator so that he can appropriate the "body", robotic or virtual, of his avatar. A cinematographic illustration of this objective can be seen in recent movies such as "Avatar" or "Surrogates". In this thesis, we start by discussing specific problems that we encountered while using a brain-computer interface for the control of robots or avatars, e.g. the arising need for multiple behaviours or the specific problems induced by the sensory feedback provided by the robot. We will then introduce our main contribution which is the concept of object-oriented brain-computer interface for the control of humanoid robot. We will then present the results of a study regarding the role of sound in the embodiment process. Finally, we show some preliminary experiments where we used electrocorticography (ECoG)~--~a technology used to acquire signals from the brain that is implanted within the cranium~--~to control a humanoid robot.

This thesis analyzes the market and products of brain-computer interface. Its main goal is evaluate the current market of these products and their use in information technologies and systems. The document is divided into three main parts. The first one focuses on familiarizing the reader with brain-computer interface technology. It mentions a brief history of development, technological principles, types of devices, their contemporary use, and the positives and negatives they bring. The second one focuses on market analysis. It summarizes the active companies on the market, and their products, and describes their customer targeting. It contains brief insight in market's future development. The third part focuses on practical testing of two selected brain-computer interface devices. The testing will evaluate applicability in information technologies and systems. For testing purpose will be developed its own methodology and selected appropriate evaluation criteria.

Functional neuroimaging techniques such as near-infrared spectroscopy (NIRS) have been studied in brain-computer interface (BCI) development. Previous research has suggested that the addition of a second brain-monitoring modality may improve the accuracy of a NIRS-BCI. The objective of this study was to determine whether the classification accuracies achievable by a multimodal BCI, which combines NIRS and transcranial Doppler ultrasonography (TCD) signals, can exceed those attainable using a unimodal NIRS-BCI or TCD-BCI. Nine able-bodied subjects participated in the study. Simultaneous measurements were made with NIRS and TCD instruments while participants were prompted to perform a verbal fluency task or to remain at rest, within the context of a block-stimulus paradigm. In five of nine (55.6%) participants, classification accuracies with the NIRS-TCD system were significantly higher (p<0.05) than with NIRS or TCD systems alone. Our results suggest that multimodal neuroimaging may be a promising approach towards improving the accuracy of future BCIs.

Functional neuroimaging techniques such as near-infrared spectroscopy (NIRS) have been studied in brain-computer interface (BCI) development. Previous research has suggested that the addition of a second brain-monitoring modality may improve the accuracy of a NIRS-BCI. The objective of this study was to determine whether the classification accuracies achievable by a multimodal BCI, which combines NIRS and transcranial Doppler ultrasonography (TCD) signals, can exceed those attainable using a unimodal NIRS-BCI or TCD-BCI. Nine able-bodied subjects participated in the study. Simultaneous measurements were made with NIRS and TCD instruments while participants were prompted to perform a verbal fluency task or to remain at rest, within the context of a block-stimulus paradigm. In five of nine (55.6%) participants, classification accuracies with the NIRS-TCD system were significantly higher (p<0.05) than with NIRS or TCD systems alone. Our results suggest that multimodal neuroimaging may be a promising approach towards improving the accuracy of future BCIs.

<p>Brain-machine interfaces (BMI) are systems which connect brains directly to machines or computers for communication. BMI-controlled prosthetic devices use algorithms to decode neuronal recordings into movement commands. These algorithms operate using models of how recorded neuronal signals relate to desired movements, called models of tuning. Models of tuning have typically been linear in prior work, due to the simplicity and speed of the algorithms used with them. Neuronal tuning has been shown to slowly change over time, but most prior work do not adapt tuning models to these changes. Furthermore, extracellular electrical recordings of neurons' action potentials slowly change over time, impairing the preprocessing step of spike-sorting, during which the neurons responsible for recorded action potentials are identified.</p> <p></p> <p>This dissertation presents a non-linear adaptive Bayesian filter and an adaptive spike-sorting method for BMI decoding. The adaptive filter consists of the n-th order unscented Kalman filter and Bayesian regression self-training updates. The unscented Kalman filter estimates desired prosthetic movements using a non-linear model of tuning as its observation model. The model is quadratic with terms for position, velocity, distance from center of workspace, and velocity magnitude. The tuning model relates neuronal activity to movements at multiple time offsets simultaneously, and the movement model of the filter is an order n autoregressive model.</p> <p>To adapt the tuning model parameters to changes in the brain, Bayesian regression self-training updates are performed periodically. Tuning model parameters are stored as probability distributions instead of point estimates. Bayesian regression uses the previous model parameters as priors and calculates the posteriors of the regression between filter outputs, which are assumed to be the desired movements, and neuronal recordings. Before each update, filter outputs are smoothed using a Kalman smoother, and tuning model parameters are passed through a transition model describing how parameters change over time. Two variants of Bayesian regression are presented: one uses a joint distribution for the model parameters which allows analytical inference, and the other uses a more flexible factorized distribution that requires approximate inference using variational Bayes.</p> <p>To adapt spike-sorting parameters to changes in spike waveforms, variational Bayesian Gaussian mixture clustering updates are used to update the waveform clustering used to calculate these parameters. This Bayesian extension of expectation-maximization clustering uses the previous clustering parameters as priors and computes the new parameters as posteriors. The use of priors allows tracking of clustering parameters over time and facilitates fast convergence.</p> <p>To evaluate the proposed methods, experiments were performed with 3 Rhesus monkeys implanted with micro-wire electrode arrays in arm-related areas of the cortex. Off-line reconstructions and on-line, closed-loop experiments with brain-control show that the n-th order unscented Kalman filter is more accurate than previous linear methods. Closed-loop experiments over 29 days show that Bayesian regression self-training helps maintain control accuracy. Experiments on synthetic data show that Bayesian regression self-training can be applied to other tracking problems with changing observation models. Bayesian clustering updates on synthetic and neuronal data demonstrate tracking of cluster and waveform changes. These results indicate the proposed methods improve the accuracy and robustness of BMIs for prosthetic devices, bringing BMI-controlled prosthetics closer to clinical use.</p> / Dissertation

This thesis describes a brain-computer interface (BCI) system that was developed to allow direct cortical control of 7 active degrees of freedom in a robotic arm. Two monkeys with chronic microelectrode implants in their motor cortices were able to use the arm to complete an oriented grasping task under brain control. This BCI system was created as a clinical prototype to exhibit (1) simultaneous decoding of cortical signals for control of the 3-D translation, 3-D rotation, and 1-D finger aperture of a robotic arm and hand, (2) methods for constructing cortical signal decoding models based on only observation of a moving robot, (3) a generalized method for training subjects to use complex BCI prosthetic robots using a novel form of operator-machine shared control, and (4) integrated kinematic and force control of a brain-controlled prosthetic robot through a novel impedance-based robot controller. This dissertation describes each of these features individually, how their integration enriched BCI control, and results from the monkeys operating the resulting system.

Les Interfaces Cerveau-Machine (ICM) sont des systèmes qui permettent à des patients souffrant d'un handicap moteur sévère d'utiliser leur activité cérébrale pour contrôler des effecteurs, par exemple des prothèses des membres supérieurs dans le cas d'ICM motrices. Les intentions de mouvement de l'utilisateur sont estimées en appliquant un décodeur sur des caractéristiques extraites de son activité cérébrale. Des challenges spécifiques au déploiement clinique d'ICMs motrices ont été considérés, à savoir le contrôle mono-membre ou séquentiel multi-membre asynchrone et précis. Un décodeur, le Markov Switching Linear Model (MSLM), a été développé pour limiter les activations erronées de l'ICM, empêcher des mouvements parallèles des effecteurs et décoder avec précision des mouvements complexes. Le MSLM associe des modèles linéaires à différents états possibles, e.g. le contrôle d'un membre spécifique ou une phase de mouvement particulière. Le MSLM réalise une détection d'état dynamique, et les probabilités des états sont utilisées pour pondérer les modèles linéaires.La performance du décodeur MSLM a été évaluée pour la reconstruction asynchrone de trajectoires de poignet et de doigts à partir de signaux electrocorticographiques. Il a permis de limiter les activations erronées du système et d'améliorer la précision du décodage du signal cérébral. / Brain-Computer Interfaces (BCI) are systems that allow severely motor-impaired patients to use their brain activity to control external devices, for example upper-limb prostheses in the case of motor BCIs. The user's intentions are estimated by applying a decoder on neural features extracted from the user's brain activity. Signal processing challenges specific to the clinical deployment of motor BCI systems are addressed in the present doctoral thesis, namely asynchronous mono-limb or sequential multi-limb decoding and accurate decoding during active control states. A switching decoder, namely a Markov Switching Linear Model (MSLM), has been developed to limit spurious system activations, to prevent parallel limb movements and to accurately decode complex movements.The MSLM associates linear models with different possible control states, e.g. activation of a specific limb, specific movement phases. Dynamic state detection is performed by the MSLM, and the probability of each state is used to weight the linear models. The performance of the MSLM decoder was assessed for asynchronous wrist and multi-finger trajectory reconstruction from electrocorticographic signals. It was found to outperform previously reported decoders for the limitation of spurious activations during no-control periods and permitted to improve decoding accuracy during active periods.

abstract: This research is focused on two separate but related topics. The first uses an electroencephalographic (EEG) brain-computer interface (BCI) to explore the phenomenon of motor learning transfer. The second takes a closer look at the EEG-BCI itself and tests an alternate way of mapping EEG signals into machine commands. We test whether motor learning transfer is more related to use of shared neural structures between imagery and motor execution or to more generalized cognitive factors. Using an EEG-BCI, we train one group of participants to control the movements of a cursor using embodied motor imagery. A second group is trained to control the cursor using abstract motor imagery. A third control group practices moving the cursor using an arm and finger on a touch screen. We hypothesized that if motor learning transfer is related to the use of shared neural structures then the embodied motor imagery group would show more learning transfer than the abstract imaging group. If, on the other hand, motor learning transfer results from more general cognitive processes, then the abstract motor imagery group should also demonstrate motor learning transfer to the manual performance of the same task. Our findings support that motor learning transfer is due to the use of shared neural structures between imaging and motor execution of a task. The abstract group showed no motor learning transfer despite being better at EEG-BCI control than the embodied group. The fact that more participants were able to learn EEG-BCI control using abstract imagery suggests that abstract imagery may be more suitable for EEG-BCIs for some disabilities, while embodied imagery may be more suitable for others. In Part 2, EEG data collected in the above experiment was used to train an artificial neural network (ANN) to map EEG signals to machine commands. We found that our open-source ANN using spectrograms generated from SFFTs is fundamentally different and in some ways superior to Emotiv's proprietary method. Our use of novel combinations of existing technologies along with abstract and embodied imagery facilitates adaptive customization of EEG-BCI control to meet needs of individual users. / Dissertation/Thesis / Ph.D. Psychology 2013

This thesis describes recording, processing and classifying brain activity which is being captured by a brain-computer interface (BCI) device manufactured by OpenBCI company. Possibility of use of such a device for controlling an application with brain activity, specifically with thinking of left or right hand movement, is discussed. To solve this task methods of signal processing and machine learning are used. As a result a program that is capable of recording, processing and classifying brain activity using an artificial neural network is created. An average accuracy of classification of synthetic data is 99.156%. An average accuracy of classification of real data is 73.71%.

abstract: The human brain controls a person's actions and reactions. In this study, the main objective is to quantify reaction time towards a change of visual event and figuring out the inherent relationship between response time and corresponding brain activities. Furthermore, which parts of the human brain are responsible for the reaction time is also of interest. As electroencephalogram (EEG) signals are proportional to the change of brain functionalities with time, EEG signals from different locations of the brain are used as indicators of brain activities. As the different channels are from different parts of our brain, identifying most relevant channels can provide the idea of responsible brain locations. In this study, response time is estimated using EEG signal features from time, frequency and time-frequency domain. Regression-based estimation using the full data-set results in RMSE (Root Mean Square Error) of 99.5 milliseconds and a correlation value of 0.57. However, the addition of non-EEG features with the existing features gives RMSE of 101.7 ms and a correlation value of 0.58. Using the same analysis with a custom data-set provides RMSE of 135.7 milliseconds and a correlation value of 0.69. Classification-based estimation provides 79% & 72% of accuracy for binary and 3-class classication respectively. Classification of extremes (high-low) results in 95% of accuracy. Combining recursive feature elimination, tree-based feature importance, and mutual feature information method, important channels, and features are isolated based on the best result. As human response time is not solely dependent on brain activities, it requires additional information about the subject to improve the reaction time estimation. / Dissertation/Thesis / Masters Thesis Electrical Engineering 2019