Intracortical brain-computer interfaces (BCIs) provide the means to do something extraordinary: restore movement to patients with paralysis or amputated limbs. Realizing this potential requires the development of decode algorithms capable of accurately translating measurements of neural activity, in real time, into appropriate time-varying commands for an external device (e.g. prosthetic limb). This problem is fundamentally interdisciplinary, drawing on tools and insights from engineering, neuroscience, statistics, and computer science, among others. Decode algorithms that have been favored historically tend to be computationally efficient, but perform suboptimally, likely because their assumptions fail to fully and accurately capture the complexity in neural population responses. Recent work harnessing the power of contemporary machine learning methods has raised the performance bar, yet these methods can be computationally demanding and it is unclear what properties of neural and/or behavioral data they exploit. In this dissertation, we characterize properties of motor-cortex population geometry and let these properties dictate decoder design, resulting in methods that perform very well, yet retain the benefits of simpler methods. We use this approach to develop a closed-loop navigation BCI, and to design a highly accurate, general, and interpretable decoder. The properties described in this dissertation have implications for any BCI. By designing decoders to explicitly respect (and leverage) these properties, we can construct powerful yet practical BCIs that better meet the needs of patients.

Neuronal activities in the brain encode every decision, desire, or intention. Multiple brain regions are involved in translating intention into action. Detecting and decoding intentions directly from the brain could allow impaired individuals to communicate and interact with their environment despite central nervous system dysfunction. Brain-computer interface (BCI) systems access neuronal activities and translate them into actions using a computer. BCIs are used in research studies to replace, restore, or replace neuromuscular functions. In addition, BCIs provide new insights into how the brain works, aiding in new treatments for neurological conditions. BCI studies commonly target the primary motor cortex, the region of the brain most closely associated with volitional muscle control, with the expectation that signals from its neurons will be best suited for control of external effectors. Consequently, other brain regions are underrepresented in BCI studies. This thesis focuses on two brain regions in primates with access to higher-order control over intention and movement: The prefrontal cortex and the basal ganglia system. These areas are vital for naturalistic movement and must be more widely explored for decoding intentions. We aim to find the movement information while the intentions have yet to transfer into planning. One study in macaque monkeys explored eye movement intention, learning, and memory-related circuitry in the lateral prefrontal cortex (LPFC). In an eight-target saccade task, we could decode the target to which the monkeys would saccade before the eye movement began. Moreover, we decoded the abstract rule information acquired by the monkeys to find the correct target from the neuronal activities recorded from LPFC. In addition, the memory-related activities in LPFC were linked to monkeys' behaviour as evidence of the presence of working- and long-term memory circuitry in the prefrontal cortex. In another study on Parkinson's disease (PD) patients, we explored the possibility of volitional control of brain activities, which can lead to a self-induced procedure to reduce the symptoms of PD. We recorded the local field potentials (LFP) of the subthalamic nucleus (STN) of nine PD patients performing a cognitive task during deep brain stimulation surgery. The patients could modulate their brain activities to change the colour of a central sphere to match the colour of a peripheral cue in a virtual reality task. They modulated the signal power in beta frequencies (13-30 Hz) and the rate of beta bursts (the fast episodes of changing amplitude in a short period in LFP's beta frequencies) based on the task conditions. Both beta power and beta bursts are associated with the pathological state in PD patients. A decodable volitional modulation of both presents the STN as a valuable region for BCI studies which could lead to self-regulation of PD symptoms. The findings of this thesis contribute to the advancement of therapeutic systems used for various brain disorders like PD and Amyotrophic lateral sclerosis (ALS), as well as patients with disabilities that can benefit from assistive communicative technologies. The study on the LPFC increased the decoding accuracy of saccade intentions compared to previous studies. Additionally, decoding associative rules is beyond the complexity of previous studies. We also showed the effects of previously learned associations on the learning rate of new rules and how this memory-retrieved information modulates neuronal activities. Moreover, the study on the STN showed the volitional control of beta power and beta burst rates by PD patients, which can be used as therapeutic methods to improve the severity of the symptoms of PD.

No description available.

This research consists of probabilistic modeling of speech audio signals and deep-brain neurological signals in brain-computer interfaces. A significant portion of this research consists of a collaborative effort with Neural Signals Inc., Duluth, GA, and Boston University to develop an intracortical neural prosthetic system for speech restoration in a human subject living with Locked-In Syndrome, i.e., he is paralyzed and unable to speak. The work is carried out in three major phases. We first use kernel-based classifiers to detect evidence of articulation gestures and phonological attributes speech audio signals. We demonstrate that articulatory information can be used to decode speech content in speech audio signals. In the second phase of the research, we use neurological signals collected from a human subject with Locked-In Syndrome to predict intended speech content. The neural data were collected with a microwire electrode surgically implanted in speech motor cortex of the subject's brain, with the implant location chosen to capture extracellular electric potentials related to speech motor activity. The data include extracellular traces, and firing occurrence times for neural clusters in the vicinity of the electrode identified by an expert. We compute continuous firing rate estimates for the ensemble of neural clusters using several rate estimation methods and apply statistical classifiers to the rate estimates to predict intended speech content. We use Gaussian mixture models to classify short frames of data into 5 vowel classes and to discriminate intended speech activity in the data from non-speech. We then perform a series of data collection experiments with the subject designed to test explicitly for several speech articulation gestures, and decode the data offline. Finally, in the third phase of the research we develop an original probabilistic method for the task of spike-sorting in intracortical brain-computer interfaces, i.e., identifying and distinguishing action potential waveforms in extracellular traces. Our method uses both action potential waveforms and their occurrence times to cluster the data. We apply the method to semi-artificial data and partially labeled real data. We then classify neural spike waveforms, modeled with single multivariate Gaussians, using the method of minimum classification error for parameter estimation. Finally, we apply our joint waveforms and occurrence times spike-sorting method to neurological data in the context of a neural prosthesis for speech.

Une interface cerveau-machine (ICM) est un système qui permet d'établir une communication directe entre le cerveau et un dispositif externe, en contournant les voies de sortie normales du système nerveux périphérique. Différents types d'ICMs existent dans la littérature. Parmi eux, les ICMs basées sur l'imagerie motrice sont les plus prometteuses. Elles sont basées sur l'autorégulation des rythmes sensorimoteurs par l'imagination de mouvement des membres différents (par exemple, imagination du mouvement de la main gauche et la main droite). Les ICMs basées sur l'imagerie motrice sont les meilleurs candidats pour les applications dédiées à des patients sévèrement paralysés mais elles sont difficiles à mettre en place parce que l'autorégulation des rythmes du cerveau n'est pas une tâche simple.Dans les premiers stades de la recherche en ICMs basées sur l'imagerie motrice, l'utilisateur devait effectuer des semaines, voire des mois, d'entrainement afin de générer des motifs d'activité cérébrale stables qui peuvent être décodés de manière fiable par le système. Le développement des techniques d'apprentissage automatique supervisé spécifiques à chaque utilisateur a permis de réduire considérablement la durée d'entrainement en ICMs. Cependant, ces techniques sont toujours confrontées aux problèmes de longue durée de calibrage et non-stationnarité des signaux cérébraux qui limitent l'utilisation de cette technologie dans la vie quotidienne. Bien que beaucoup de techniques d'apprentissage automatique avancées ont été essayées, ça reste toujours pas un problème non résolu.Dans cette thèse, j'étudie de manière approfondie les techniques d'apprentissage automatique supervisé qui ont été tentées afin de surmonter les problèmes de longue durée de calibrage et la non-stationnarité des signaux cérébraux en ICMs basées sur l'imagerie motrice. Ces techniques peuvent être classées en deux catégories: les techniques qui sont invariantes à la non-stationnarité et les techniques qui s'adaptent au changement. Dans la première catégorie, les techniques d'apprentissage par transfert entre différentes sessions et/ou différents individus ont attiré beaucoup d'attention au cours des dernières années. Dans la deuxième catégorie, différentes techniques d'adaptation en ligne des modèles d'apprentissage ont été tentées. Parmi elles, les techniques basées sur les potentiels d'erreurs sont les plus prometteuses. Les deux principales contributions de cette thèse sont basés sur des combinaisons linéaires des classificateurs. Ainsi, ces méthodes sont accordées un intérêt particulier tout au long de ce manuscrit. Dans la première contribution, je étudie l'utilisation de combinaisons linéaires des classificateurs dans les ICMs basées sur l'apprentissage par transfert et je propose une méthode de classification inter-sujets basée sur les combinaisons linéaires de classifieurs afin de réduire le temps de calibrage en ICMs. Je teste l'efficacité de la méthode de combinaison de classifieurs utilisée et j'étudie les cas ou l'apprentissage par transfert a un effet négatif sur les performances des ICMs. Dans la deuxième contribution, je propose une méthode de classification inter-sujets qui permet de combiner l'apprentissage par transfert l'adaptation en ligne. Dans cette méthode, l'apprentissage par transfert est effectué en combinant linéairement des classifieurs appris à partir de signaux EEG de différents sujets. L'adaptation en ligne est effectué en mettant à jours les poids de ces classifieurs d'une manière semi-supervisée. / A brain-computer interface (BCI) is a system that allows establishing direct communication between the brain and an external device, bypassing normal output pathways of peripheral neuromuscular system. Different types of BCIs exist in literature. Among them, BCIs based on motor imagery (MI) are the most promising ones. They rely on self-regulation of sensorimotor rhythms by imagination of movement of different limbs (e.g., left hand and right hand). MI-based BCIs are best candidates for applications dedicated to severely paralyzed patients but they are hard to set-up because self-regulation of brain rhythms is not a straightforward task.In early stages of BCI research, weeks and even months of user training was required in order to generate stable brain activity patterns that can be reliably decoded by the system. The development of user-specific supervised machine learning techniques allowed reducing considerably training periods in BCIs. However, these techniques are still faced with the problems of long calibration time and brain signals non-stationarity that limit the use of this technology in out-of-the-lab applications. Although many out-of-the-box machine learning techniques have been attempted, it is still not a solved problem.In this thesis, I thoroughly investigate supervised machine learning techniques that have been attempted in order to overcome the problems of long calibration time and brain signals non-stationarity in MI-based BCIs. These techniques can be mainly classified into two categories: techniques that are invariant to non-stationarity and techniques that adapt to the change. In the first category, techniques based on knowledge transfer between different sessions and/or subjects have attracted much attention during the last years. In the second category, different online adaptation techniques of classification models were attempted. Among them, techniques based on error-related potentials are the most promising ones. The aim of this thesis is to highlight some important points that have not been taken into consideration in previous work on supervised machine learning in BCIs and that have to be considered in future BCI systems in order to bring this technology out of the lab. The two main contributions of this thesis are based on linear combinations of classifiers. Thus, these methods are given a particular interest throughout this manuscript. In the first contribution, I study the use of linear combinations of classifiers in knowledge transfer-based BCIs and I propose a novel ensemble-based knowledge transfer framework for reducing calibration time in BCIs. I investigate the effectiveness of the classifiers combination scheme used in this framework when performing inter-subjects classification in MI-based BCIs. Then, I investigate to which extent knowledge transfer is useful in BCI applications by studying situations in which knowledge transfer has a negative impact on classification performance of target learning task. In the second contribution, I propose an online inter-subjects classification framework that allows taking advantage from both knowledge transfer and online adaptation techniques. In this framework, called “adaptive accuracy-weighted ensemble” (AAWE), inter-subjects classification is performed using a weighted average ensemble in which base classifiers are learned using EEG signals recorded from different subjects and weighted according to their accuracies in classifying brain signals of the new BCI user. Online adaptation is performed by updating base classifiers' weights in a semi-supervised way based on ensemble predictions reinforced by interaction error-related potentials.

Locked-in syndrome (LIS) is a condition characterized by total or near-total paralysis with preserved cognitive and somatosensory function. For the locked-in, brain-machine interfaces (BMI) provide a level of restored communication and interaction with the world, though this technology has not reached its fullest potential. Several streams of research explore improving BMI performance but very little attention has been given to the paradigms implemented and the resulting constraints imposed on the users. Learning new mental tasks, constant use of external stimuli, and high attentional and cognitive processing loads are common demands imposed by BMI. These paradigm constraints negatively affect BMI performance by locked-in patients. In an effort to develop simpler and more reliable BMI for those suffering from LIS, this dissertation explores using error-related potentials, the neural correlates of error awareness, as an access pathway for adaptive decoding and direct volitional control. In the first part of this thesis we characterize error-related local field potentials (eLFP) and implement a real-time decoder error detection (DED) system using eLFP while non-human primates controlled a saccade BMI. Our results show specific traits in the eLFP that bridge current knowledge of non-BMI evoked error-related potentials with error-potentials evoked during BMI control. Moreover, we successfully perform real-time DED via, to our knowledge, the first real-time LFP-based DED system integrated into an invasive BMI, demonstrating that error-based adaptive decoding can become a standard feature in BMI design. In the second part of this thesis, we focus on employing electroencephalography error-related potentials (ErrP) for direct volitional control. These signals were employed as an indicator of the user’s intentions under a closed-loop binary-choice robot reaching task. Although this approach is technically challenging, our results demonstrate that ErrP can be used for direct control via binary selection and, given the appropriate levels of task engagement and agency, single-trial closed-loop ErrP decoding is possible. Taken together, this work contributes to a deeper understanding of error-related potentials evoked during BMI control and opens new avenues of research for employing ErrP as a direct control signal for BMI. For the locked-in community, these advancements could foster the development of real-time intuitive brain-machine control.

Functional magnetic resonance imaging (fMRI) is a prime example of multi-disciplinary research. Without the beautiful physics of MRI, there wouldnot be any images to look at in the first place. To obtain images of goodquality, it is necessary to fully understand the concepts of the frequencydomain. The analysis of fMRI data requires understanding of signal pro-cessing, statistics and knowledge about the anatomy and function of thehuman brain. The resulting brain activity maps are used by physicians,neurologists, psychologists and behaviourists, in order to plan surgery andto increase their understanding of how the brain works. This thesis presents methods for real-time fMRI and non-parametric fMRIanalysis. Real-time fMRI places high demands on the signal processing,as all the calculations have to be made in real-time in complex situations.Real-time fMRI can, for example, be used for interactive brain mapping.Another possibility is to change the stimulus that is given to the subject, inreal-time, such that the brain and the computer can work together to solvea given task, yielding a brain computer interface (BCI). Non-parametricfMRI analysis, for example, concerns the problem of calculating signifi-cance thresholds and p-values for test statistics without a parametric nulldistribution. Two BCIs are presented in this thesis. In the first BCI, the subject wasable to balance a virtual inverted pendulum by thinking of activating theleft or right hand or resting. In the second BCI, the subject in the MRscanner was able to communicate with a person outside the MR scanner,through a virtual keyboard. A graphics processing unit (GPU) implementation of a random permuta-tion test for single subject fMRI analysis is also presented. The randompermutation test is used to calculate significance thresholds and p-values forfMRI analysis by canonical correlation analysis (CCA), and to investigatethe correctness of standard parametric approaches. The random permuta-tion test was verified by using 10 000 noise datasets and 1484 resting statefMRI datasets. The random permutation test is also used for a non-localCCA approach to fMRI analysis.

No description available.

Thesis (Ph. D.)--University of Washington, 2008. / Vita. Includes bibliographical references (p. 168-177).

abstract: A robotic swarm can be defined as a large group of inexpensive, interchangeable robots with limited sensing and/or actuating capabilities that cooperate (explicitly or implicitly) based on local communications and sensing in order to complete a mission. Its inherent redundancy provides flexibility and robustness to failures and environmental disturbances which guarantee the proper completion of the required task. At the same time, human intuition and cognition can prove very useful in extreme situations where a fast and reliable solution is needed. This idea led to the creation of the field of Human-Swarm Interfaces (HSI) which attempts to incorporate the human element into the control of robotic swarms for increased robustness and reliability. The aim of the present work is to extend the current state-of-the-art in HSI by applying ideas and principles from the field of Brain-Computer Interfaces (BCI), which has proven to be very useful for people with motor disabilities. At first, a preliminary investigation about the connection of brain activity and the observation of swarm collective behaviors is conducted. After showing that such a connection may exist, a hybrid BCI system is presented for the control of a swarm of quadrotors. The system is based on the combination of motor imagery and the input from a game controller, while its feasibility is proven through an extensive experimental process. Finally, speech imagery is proposed as an alternative mental task for BCI applications. This is done through a series of rigorous experiments and appropriate data analysis. This work suggests that the integration of BCI principles in HSI applications can be successful and it can potentially lead to systems that are more intuitive for the users than the current state-of-the-art. At the same time, it motivates further research in the area and sets the stepping stones for the potential development of the field of Brain-Swarm Interfaces (BSI). / Dissertation/Thesis / Masters Thesis Mechanical Engineering 2017