This study focuses on realization of a Brain Computer Interface (BCI)for the paralyzed to control assistive environmental devices. For this purpose, different motor imagery tasks are classified using different signal processing methods. Specifically, band-pass filtering, Laplacian filtering, and common average reference (CAR) filtering areused to enhance the EEG signal. For feature extraction / Common Spatial Pattern (CSP), Power Spectral Density (PSD), and Principal Component Analysis (PCA) are tested. Linear Feature Normalization (LFN), Gaussian Feature Normalization (GFN), and Unit-norm Feature Vector Normalization (UFVN) are studied in Support Vector Machine (SVM) and Artificial Neural Network (ANN) classification. In order to evaluate and compare the performance of the methodologies, classification accuracy, Cohen&rsquo / s kappa coefficient, and Nykopp&rsquo / s information transfer are utilized. The first experiments on classifying motor imagery tasks are realized on the 3-class dataset (V) provided for BCI Competition III. Also, a 4-class problem is studied using the dataset (IIa) provided for BCI Competition IV. Then, 5 different tasks are studied in the METU Brain Research Laboratory to find the optimum number and type of tasks to control a motor imagery based BCI. Thereafter, an interface is designed for the paralyzed to control assistive environmental devices. Finally, a test application is implemented and online performance of the design is evaluated.

The goal of a Brain-Computer Interface (BCI) consists of the development of a unidirectional interface between a human and a computer to allow control of a device only via brain signals. While the BCI systems of almost all other groups require the user to be trained over several weeks or even months, the group of Prof. Dr. Klaus-Robert Müller in Berlin and Potsdam, which I belong to, was one of the first research groups in this field which used machine learning techniques on a large scale. The adaptivity of the processing system to the individual brain patterns of the subject confers huge advantages for the user. Thus BCI research is considered a hot topic in machine learning and computer science. It requires interdisciplinary cooperation between disparate fields such as neuroscience, since only by combining machine learning and signal processing techniques based on neurophysiological knowledge will the largest progress be made.<br><br> In this work I particularly deal with my part of this project, which lies mainly in the area of computer science. I have considered the following three main points:<br><br> <b>Establishing a performance measure based on information theory:</b> I have critically illuminated the assumptions of Shannon's information transfer rate for application in a BCI context. By establishing suitable coding strategies I was able to show that this theoretical measure approximates quite well to what is practically achieveable.<br> <b>Transfer and development of suitable signal processing and machine learning techniques:</b> One substantial component of my work was to develop several machine learning and signal processing algorithms to improve the efficiency of a BCI. Based on the neurophysiological knowledge that several independent EEG features can be observed for some mental states, I have developed a method for combining different and maybe independent features which improved performance. In some cases the performance of the combination algorithm outperforms the best single performance by more than 50 %. Furthermore, I have theoretically and practically addressed via the development of suitable algorithms the question of the optimal number of classes which should be used for a BCI. It transpired that with BCI performances reported so far, three or four different mental states are optimal. For another extension I have combined ideas from signal processing with those of machine learning since a high gain can be achieved if the temporal filtering, i.e., the choice of frequency bands, is automatically adapted to each subject individually.<br> <b>Implementation of the Berlin brain computer interface and realization of suitable experiments:</b> Finally a further substantial component of my work was to realize an online BCI system which includes the developed methods, but is also flexible enough to allow the simple realization of new algorithms and ideas. So far, bitrates of up to 40 bits per minute have been achieved with this system by absolutely untrained users which, compared to results of other groups, is highly successful. / Ein Brain-Computer Interface (BCI) ist eine unidirektionale Schnittstelle zwischen Mensch und Computer, bei der ein Mensch in der Lage ist, ein Gerät einzig und allein Kraft seiner Gehirnsignale zu steuern. In den BCI Systemen fast aller Forschergruppen wird der Mensch in Experimenten über Wochen oder sogar Monaten trainiert, geeignete Signale zu produzieren, die vordefinierten allgemeinen Gehirnmustern entsprechen. Die BCI Gruppe in Berlin und Potsdam, der ich angehöre, war in diesem Feld eine der ersten, die erkannt hat, dass eine Anpassung des Verarbeitungssystems an den Menschen mit Hilfe der Techniken des Maschinellen Lernens große Vorteile mit sich bringt. In unserer Gruppe und mittlerweile auch in vielen anderen Gruppen wird BCI somit als aktuelles Forschungsthema im Maschinellen Lernen und folglich in der Informatik mit interdisziplinärer Natur in Neurowissenschaften und anderen Feldern verstanden, da durch die geeignete Kombination von Techniken des Maschinellen Lernens und der Signalverarbeitung basierend auf neurophysiologischem Wissen der größte Erfolg erzielt werden konnte.<br><br> In dieser Arbeit gehe ich auf meinem Anteil an diesem Projekt ein, der vor allem im Informatikbereich der BCI Forschung liegt. Im Detail beschäftige ich mich mit den folgenden drei Punkten:<br><br> <b>Diskussion eines informationstheoretischen Maßes für die Güte eines BCI's:</b> Ich habe kritisch die Annahmen von Shannon's Informationsübertragungsrate für die Anwendung im BCI Kontext beleuchtet. Durch Ermittlung von geeigneten Kodierungsstrategien konnte ich zeigen, dass dieses theoretische Maß den praktisch erreichbaren Wert ziemlich gut annähert.<br> <b>Transfer und Entwicklung von geeigneten Techniken aus dem Bereich der Signalverarbeitung und des Maschinellen Lernens:</b> Eine substantielle Komponente meiner Arbeit war die Entwicklung von Techniken des Machinellen Lernens und der Signalverarbeitung, um die Effizienz eines BCI's zu erhöhen. Basierend auf dem neurophysiologischem Wissen, dass verschiedene unabhängige Merkmale in Gehirnsignalen für verschiedene mentale Zustände beobachtbar sind, habe ich eine Methode zur Kombination von verschiedenen und unter Umständen unabhängigen Merkmalen entwickelt, die sehr erfolgreich die Fähigkeiten eines BCI's verbessert. Besonders in einigen Fällen übertraf die Leistung des entwickelten Kombinationsalgorithmus die beste Leistung auf den einzelnen Merkmalen mit mehr als 50 %. Weiterhin habe ich theoretisch und praktisch durch Einführung geeigneter Algorithmen die Frage untersucht, wie viele Klassen man für ein BCI nutzen kann und sollte. Auch hier wurde ein relevantes Resultat erzielt, nämlich dass für BCI Güten, die bis heute berichtet sind, die Benutzung von 3 oder 4 verschiedenen mentalen Zuständen in der Regel optimal im Sinne von erreichbarer Leistung sind. Für eine andere Erweiterung wurden Ideen aus der Signalverarbeitung mit denen des Maschinellen Lernens kombiniert, da ein hoher Erfolg erzielt werden kann, wenn der temporale Filter, d.h. die Wahl des benutzten Frequenzbandes, automatisch und individuell für jeden Menschen angepasst wird.<br> <b>Implementation des Berlin Brain-Computer Interfaces und Realisierung von geeigneten Experimenten:</b> Eine weitere wichtige Komponente meiner Arbeit war eine Realisierung eines online BCI Systems, welches die entwickelten Methoden umfasst, aber auch so flexibel ist, dass neue Algorithmen und Ideen einfach zu verwirklichen sind. Bis jetzt wurden mit diesem System Bitraten von bis zu 40 Bits pro Minute von absolut untrainierten Personen in ihren ersten BCI Experimenten erzielt. Dieses Resultat übertrifft die bisher berichteten Ergebnisse aller anderer BCI Gruppen deutlich. <br> <hr> Bemerkung:<br> Der Autor wurde mit dem <i>Michelson-Preis</i> 2005/2006 für die beste Promotion des Jahrgangs der Mathematisch-Naturwissenschaftlichen Fakultät der Universität Potsdam ausgezeichnet.

Brain-computer interface (BCI) or Brain Machine Interface (BMI) provides direct communication channel between user’s brain and an external device without any requirement of user’s physical movement. Primarily BCI has been employed in medical sciences to facilitate the patients with severe motor, visual and aural impairments. More recently many BCI are also being used as a part of entertainment. BCI differs from Neuroprosthetics, a study within Neuroscience, in terms of its usage; former connects the brain with a computer or external device while the later connects the nervous system to an implanted device. A BCI receives the modulated input from user either invasively or non-invasively. The modulated input, concealed in the huge amount of noise, contains distinct brain patterns based on the type of activity user is performing at that point in time. Primary task of a typical BCI is to find out those distinct brain patterns and translates them to meaningful communication command set. Cursor controllers, Spellers, Wheel Chair and robot Controllers are classic examples of BCI applications. This study aims to investigate an Electroencephalography (EEG) based non-invasive BCI in general and its interaction with a web interface in particular. Different aspects related to BCI are covered in this work including feedback techniques, BCI frameworks, commercial BCI hardware, and different BCI applications. BCI paradigm Steady State Visually Evoked Potentials (SSVEP) is being focused during this study. A hybrid solution is developed during this study, employing a general purpose BCI framework OpenViBE, which comprised of a low-level stimulus management and control module and a web based Google Street View client application. This study shows that a BCI can not only provide a way of communication for the impaired subjects but it can also be a multipurpose tool for a healthy person. During this study, it is being established that the major hurdles that hamper the performance of a BCI system are training protocols, BCI hardware and signal processing techniques. It is also observed that a controlled environment and expert assistance is required to operate a BCI system.

No description available.

<p dir="ltr">This research explored a solution to a high accident rate in remotely operating Unmanned Aerial Vehicles (UAVs) in a complex environment; it presented a new Brain-Computer Interface (BCI) enabled supervisory control system to fuse human and machine intelligence seamlessly. This study was highly motivated by the critical need to enhance the safety and reliability of UAV operations, where accidents often stemmed from human errors during manual controls. Existing BCIs confronted the challenge of trading off a fully remote control by humans and an automated control by computers. This study met such a challenge with the proposed supervisory control system to optimize human-machine collaboration, prioritizing safety, adaptability, and precision in operation.</p><p dir="ltr">The research work included designing, training, and testing BCI and the BCI-enabled control system. It was customized to control a UAV where the user’s motion intents and cognitive states were monitored to implement hybrid human and machine controls. The DJI Tello drone was used as an intelligent machine to illustrate the application of the proposed control system and evaluate its effectiveness through two case studies. The first case study was designed to train a subject and assess the confidence level for BCI in capturing and classifying the subject’s motion intents. The second case study illustrated the application of BCI in controlling the drone to fulfill its missions.</p><p dir="ltr">The proposed supervisory control system was at the forefront of cognitive state monitoring to leverage the power of an ML model. This model was innovative compared to conventional methods in that it could capture complicated patterns within raw EEG data and make decisions to adopt an ensemble learning strategy with the XGBoost. One of the key innovations was capturing the user’s intents and interpreting these into control commands using the EmotivBCI app. Despite the headset's predefined set of detectable features, the system could train the user’s mind to generate control commands for all six degrees of freedom of adapting to the quadcopter by creatively combining and extending mental commands, particularly in the context of the Yaw rotation. This strategic manipulation of commands showcased the system's flexibility in accommodating the intricate control requirements of an automated machine.</p><p dir="ltr">Another innovation of the proposed system was its real-time adaptability. The supervisory control system continuously monitors the user's cognitive state, allowing instantaneous adjustments in response to changing conditions. This innovation ensured that the control system was responsive to the user’s intent and adept at prioritizing safety through the arbitrating mechanism when necessary.</p>

Current studies done using a learning test for children have problems as they only make evaluations of Physically and Neurologically Impaired (PNI) children who can succeed in the test and can be considered as a PASS/FAIL test. This pilot study takes a holistic view of cognitive testing of PNI children using a user-test-device triad model and provides a framework using non-PNI children and adults as controls. Comparisons using adapted off-the-shelf novel interfaces to the computer, in particular, an Electroencephalograph (EEG) head-set, an eye-tracker and a head-tracker and a common mouse were carried out. In addition, two novel multi-modal technologies were developed based on the use of brain-waves and eye-tracking as well as head-tracking technologies to support the study. The devices were used on three tests with increasing cognitive complexity. A self-developed measure based on success streaks (consecutive outcomes) was introduced to improve evaluations of PNI children. A theoretical model regarding a fit of ability to devices was initially setup and finally modified to fit the view of the empirical model that emerged from the outcomes of the study. Results suggest that while multi-modal technologies can address weaknesses of the individual component modes, a compromise is made between the user’s ability for multi-tasking between the modes and the benefits of a multi-modal device but the sample size is very small. Results also show children failing a test with a mouse but passing it subsequently when direct communication is used suggesting that a device can affect a test for children who are of a developing age. This study provides a framework for a more meaningful conversation between educational psychologists as well as other professionals and PNI parents because it provides more discrimination of outcomes in cognitive tests for PNI children. The framework provides a vehicle that addresses scientifically the concerns of parents and schools.

This thesis aims to implement methods for recording EEG data obtained with the neural activity sensor OpenBCI Ultracortex IV headset. It also describes neurofeedback, methods of obtaining data from the motor cortex for further analysis and takes a look at the machine learning algorithms best suited for the presented problem. Multiple training and testing datasets are created, as well as a tool for recording the brain activity of a headset-wearing test subject, which is being visually presented with cognitive challenges on the screen in front of him. A neurofeedback demo app has been developed, presented and later used for calibration of new test subjects. Next part is data analysis, which aims to discriminate the left and right hand movement intention signatures in the brain motor cortex. Multiple classification methods are used and their utility reviewed.

Brain-computer interfaces (BCI) have been used to control the gait of a virtual self-avatar with the aim of being used in gait rehabilitation. A BCI decodes the brain signals representing a desire to do something and transforms them into a control command for controlling external devices. The feelings described by the participants when they control a self-avatar in an immersive virtual environment (VE) demonstrate that humans can be embodied in the surrogate body of an avatar (ownership illusion). It has recently been shown that inducing the ownership illusion and then manipulating the movements of one’s self-avatar can lead to compensatory motor control strategies. In order to maximize this effect, there is a need for a method that measures and monitors embodiment levels of participants immersed in virtual reality (VR) to induce and maintain a strong ownership illusion. This is particularly true given that reaching a high level of both BCI performance and embodiment are inter-connected. To reach one of them, the second must be reached as well. Some limitations of many existing systems hinder their adoption for neurorehabilitation: 1- some use motor imagery (MI) of movements other than gait; 2- most systems allow the user to take single steps or to walk but do not allow both, which prevents users from progressing from steps to gait; 3- most of them function in a single BCI mode (cue-paced or self-paced), which prevents users from progressing from machine-dependent to machine-independent walking. Overcoming the aforementioned limitations can be done by combining different control modes and options in one single system. However, this would have a negative impact on BCI performance, therefore diminishing its usefulness as a potential rehabilitation tool. In this case, there will be a need to enhance BCI performance. For such purpose, many techniques have been used in the literature, such as providing modified feedback (whereby the presented feedback is not consistent with the user’s MI), sequential training (recalibrating the classifier as more data becomes available). This thesis was developed over 3 studies. The objective in study 1 was to investigate the possibility of measuring the level of embodiment of an immersive self-avatar, during the performing, observing and imagining of gait, using electroencephalogram (EEG) techniques, by presenting visual feedback that conflicts with the desired movement of embodied participants. The objective of study 2 was to develop and validate a BCI to control single steps and forward walking of an immersive virtual reality (VR) self-avatar, using mental imagery of these actions, in cue-paced and self-paced modes. Different performance enhancement strategies were implemented to increase BCI performance. The data of these two studies were then used in study 3 to construct a generic classifier that could eliminate offline calibration for future users and shorten training time. Twenty different healthy participants took part in studies 1 and 2. In study 1, participants wore an EEG cap and motion capture markers, with an avatar displayed in a head-mounted display (HMD) from a first-person perspective (1PP). They were cued to either perform, watch or imagine a single step forward or to initiate walking on a treadmill. For some of the trials, the avatar took a step with the contralateral limb or stopped walking before the participant stopped (modified feedback). In study 2, participants completed a 4-day sequential training to control the gait of an avatar in both BCI modes. In cue-paced mode, they were cued to imagine a single step forward, using their right or left foot, or to walk forward. In the self-paced mode, they were instructed to reach a target using the MI of multiple steps (switch control mode) or maintaining the MI of forward walking (continuous control mode). The avatar moved as a response to two calibrated regularized linear discriminant analysis (RLDA) classifiers that used the μ power spectral density (PSD) over the foot area of the motor cortex as features. The classifiers were retrained after every session. During the training, and for some of the trials, positive modified feedback was presented to half of the participants, where the avatar moved correctly regardless of the participant’s real performance. In both studies, the participants’ subjective experience was analyzed using a questionnaire. Results of study 1 show that subjective levels of embodiment correlate strongly with the power differences of the event-related synchronization (ERS) within the μ frequency band, and over the motor and pre-motor cortices between the modified and regular feedback trials. Results of study 2 show that all participants were able to operate the cued-paced BCI and the selfpaced BCI in both modes. For the cue-paced BCI, the average offline performance (classification rate) on day 1 was 67±6.1% and 86±6.1% on day 3, showing that the recalibration of the classifiers enhanced the offline performance of the BCI (p < 0.01). The average online performance was 85.9±8.4% for the modified feedback group (77-97%) versus 75% for the non-modified feedback group. For self-paced BCI, the average performance was 83% at switch control and 92% at continuous control mode, with a maximum of 12 seconds of control. Modified feedback enhanced BCI performances (p =0.001). Finally, results of study 3 show that the constructed generic models performed as well as models obtained from participant-specific offline data. The results show that there it is possible to design a participant-independent zero-training BCI. / Les interfaces cerveau-ordinateur (ICO) ont été utilisées pour contrôler la marche d'un égo-avatar virtuel dans le but d'être utilisées dans la réadaptation de la marche. Une ICO décode les signaux du cerveau représentant un désir de faire produire un mouvement et les transforme en une commande de contrôle pour contrôler des appareils externes. Les sentiments décrits par les participants lorsqu'ils contrôlent un égo-avatar dans un environnement virtuel immersif démontrent que les humains peuvent être incarnés dans un corps d'un avatar (illusion de propriété). Il a été récemment démontré que provoquer l’illusion de propriété puis manipuler les mouvements de l’égo-avatar peut conduire à des stratégies de contrôle moteur compensatoire. Afin de maximiser cet effet, il existe un besoin d'une méthode qui mesure et surveille les niveaux d’incarnation des participants immergés dans la réalité virtuelle (RV) pour induire et maintenir une forte illusion de propriété. D'autre part, atteindre un niveau élevé de performances (taux de classification) ICO et d’incarnation est interconnecté. Pour atteindre l'un d'eux, le second doit également être atteint. Certaines limitations de plusieurs de ces systèmes entravent leur adoption pour la neuroréhabilitation: 1- certains utilisent l'imagerie motrice (IM) des mouvements autres que la marche; 2- la plupart des systèmes permettent à l'utilisateur de faire des pas simples ou de marcher mais pas les deux, ce qui ne permet pas à un utilisateur de passer des pas à la marche; 3- la plupart fonctionnent en un seul mode d’ICO, rythmé (cue-paced) ou auto-rythmé (self-paced). Surmonter les limitations susmentionnées peut être fait en combinant différents modes et options de commande dans un seul système. Cependant, cela aurait un impact négatif sur les performances de l’ICO, diminuant ainsi son utilité en tant qu'outil potentiel de réhabilitation. Dans ce cas, il sera nécessaire d'améliorer les performances des ICO. À cette fin, de nombreuses techniques ont été utilisées dans la littérature, telles que la rétroaction modifiée, le recalibrage du classificateur et l'utilisation d'un classificateur générique. Le projet de cette thèse a été réalisé en 3 études, avec objectif d'étudier dans l'étude 1, la possibilité de mesurer le niveau d'incarnation d'un égo-avatar immersif, lors de l'exécution, de l'observation et de l'imagination de la marche, à l'aide des techniques encéphalogramme (EEG), en présentant une rétroaction visuelle qui entre en conflit avec la commande du contrôle moteur des sujets incarnés. L'objectif de l'étude 2 était de développer un BCI pour contrôler les pas et la marche vers l’avant d'un égo-avatar dans la réalité virtuelle immersive, en utilisant l'imagerie motrice de ces actions, dans des modes rythmés et auto-rythmés. Différentes stratégies d'amélioration des performances ont été mises en œuvre pour augmenter la performance (taux de classification) de l’ICO. Les données de ces deux études ont ensuite été utilisées dans l'étude 3 pour construire des classificateurs génériques qui pourraient éliminer la calibration hors ligne pour les futurs utilisateurs et raccourcir le temps de formation. Vingt participants sains différents ont participé aux études 1 et 2. Dans l'étude 1, les participants portaient un casque EEG et des marqueurs de capture de mouvement, avec un avatar affiché dans un casque de RV du point de vue de la première personne (1PP). Ils ont été invités à performer, à regarder ou à imaginer un seul pas en avant ou la marche vers l’avant (pour quelques secondes) sur le tapis roulant. Pour certains essais, l'avatar a fait un pas avec le membre controlatéral ou a arrêté de marcher avant que le participant ne s'arrête (rétroaction modifiée). Dans l'étude 2, les participants ont participé à un entrainement séquentiel de 4 jours pour contrôler la marche d'un avatar dans les deux modes de l’ICO. En mode rythmé, ils ont imaginé un seul pas en avant, en utilisant leur pied droit ou gauche, ou la marche vers l’avant . En mode auto-rythmé, il leur a été demandé d'atteindre une cible en utilisant l'imagerie motrice (IM) de plusieurs pas (mode de contrôle intermittent) ou en maintenir l'IM de marche vers l’avant (mode de contrôle continu). L'avatar s'est déplacé en réponse à deux classificateurs ‘Regularized Linear Discriminant Analysis’ (RLDA) calibrés qui utilisaient comme caractéristiques la densité spectrale de puissance (Power Spectral Density; PSD) des bandes de fréquences µ (8-12 Hz) sur la zone du pied du cortex moteur. Les classificateurs ont été recalibrés après chaque session. Au cours de l’entrainement et pour certains des essais, une rétroaction modifiée positive a été présentée à la moitié des participants, où l'avatar s'est déplacé correctement quelle que soit la performance réelle du participant. Dans les deux études, l'expérience subjective des participants a été analysée à l'aide d'un questionnaire. Les résultats de l'étude 1 montrent que les niveaux subjectifs d’incarnation sont fortement corrélés à la différence de la puissance de la synchronisation liée à l’événement (Event-Related Synchronization; ERS) sur la bande de fréquence μ et sur le cortex moteur et prémoteur entre les essais de rétroaction modifiés et réguliers. L'étude 2 a montré que tous les participants étaient capables d’utiliser le BCI rythmé et auto-rythmé dans les deux modes. Pour le BCI rythmé, la performance hors ligne moyenne au jour 1 était de 67±6,1% et 86±6,1% au jour 3, ce qui montre que le recalibrage des classificateurs a amélioré la performance hors ligne du BCI (p <0,01). La performance en ligne moyenne était de 85,9±8,4% pour le groupe de rétroaction modifié (77-97%) contre 75% pour le groupe de rétroaction non modifié. Pour le BCI auto-rythmé, la performance moyenne était de 83% en commande de commutateur et de 92% en mode de commande continue, avec un maximum de 12 secondes de commande. Les performances de l’ICO ont été améliorées par la rétroaction modifiée (p = 0,001). Enfin, les résultats de l'étude 3 montrent que pour la classification des initialisations des pas et de la marche, il a été possible de construire des modèles génériques à partir de données hors ligne spécifiques aux participants. Les résultats montrent la possibilité de concevoir une ICO ne nécessitant aucun entraînement spécifique au participant.

No description available.

Electroencephalographic data recorded on the human scalp can be modeled as a linear mixture of underlying dipolar source generators. The characterization of such generators is the aim of several families of signal processing methods. In this HDR we consider in several details three of such families, namely 1) EEG distributed inverse solutions, 2) diagonalization methods, including spatial filtering and blind source separation and 3) Riemannian geometry. We highlight our contributions in each of this family, we describe algorithms reporting all necessary information to make purposeful use of these methods and we give numerous examples with real data pertaining to our published studies. Traditionally only the single-subject scenario is considered; here we consider in addition the extension of some methods to the simultaneous multi-subject recording scenario. This HDR can be seen as an handbook for EEG source analysis. It will be particularly useful to students and other colleagues approaching the field.