Cette thèse porte sur la conception et l'évaluation de systèmes interactifs utilisant des interfaces cerveau-machine (BCI pour Brain-Computer Interfaces). Ce type d'interfaces s'est développé dans les années récentes tout d'abord dans le domaine du handicap, afin de fournir aux grands handicapés des moyens d'interaction et de communication, et plus récemment dans d'autres domaines comme celui des jeux vidéo. Néanmoins, la plupart des travaux ont porté sur l'identification des signaux du cerveau susceptibles de porter une information utile, et sur les traitements nécessaires à l'extraction de cette information. Peu de travaux ont porté sur les aspects d'utilisabilité et de prise en compte des facteurs humains dans l'ensemble du système interactif. Cette thèse se concentre sur les systèmes basées sur SSVEP (steady-state visually evoked potentials), et se propose d'étudier l'ensemble du système interactif cerveau-machine, selon les critères de l'interaction homme-machine (IHM). Plus précisément, les points étudiés portent sur la demande cognitive, la frustration de l'utilisateur, les conditions de calibration, et les BCI hybrides. / This PhD deals with the conception and evaluation of interactive systems based on Brain-Computer Interfaces (BCI). This type of interfaces has developed in recent years, first in the domain of handicaps, in order to provide disabled people means of interaction and communication, and more recently in other fields as video games. However, most of the research so far focused on the identification of cerebral pattern carrying useful information, a on signal processing for the detection of these patterns. Less attention has been given to usability aspects. This PhD focuses on interactive systems based on Steady-State Visually Evoked Potentials (SSVEP), and aims at considering the interactive system as a whole, using the concepts of Human-Computer Interaction. More precisely, a focus is made on cognitive demand, user frustration, calibration conditions, and hybrid BCIs.

Les interfaces cerveau machine (BCI pour Brain Computer Interfaces) non invasives permettent à leur utilisateur de contrôler une machine par la pensée. Ce dernier doit porter un dispositif d'acquisition de signaux électroencéphalographiques (EEG), qui sont dotés d'un rapport signal sur bruit assez faible ; à ceci s'ajoute l’importante variabilité tant à travers les sessions d'utilisation qu’à travers les utilisateurs. Par conséquent, la calibration du BCI est souvent nécessaire avant son utilisation. Cette thèse étudie les sources de cette variabilité, dans le but d'explorer, concevoir, et implémenter des méthodes d'autocalibration. Nous étudions la variabilité des potentiels évoqués, particulièrement une composante tardive appelée P300. Nous nous penchons sur trois méthodes d’apprentissage par transfert : la Géométrie Riemannienne, le Transport Optimal, et l’apprentissage ensembliste. Nous proposons un modèle de l'EEG qui tient compte de la variabilité. Les paramètres résultants de nos analyses nous servent à calibrer ce modèle et à simuler une base de données, qui nous sert à évaluer la performance des méthodes d’apprentissage par transfert. Puis ces méthodes sont combinées et appliquées à des données expérimentales. Nous proposons une méthode de classification basée sur le Transport Optimal dont nous évaluons la performance. Ensuite, nous introduisons un marqueur de séparabilité qui nous permet de combiner Géométrie Riemannienne, Transport Optimal et apprentissage ensembliste. La combinaison de plusieurs méthodes d’apprentissage par transfert nous permet d’obtenir un classifieur qui s’affranchit des différentes sources de variabilité des signaux EEG. / Non-invasive Brain Computer Interfaces (BCIs) allow a user to control a machine using only their brain activity. The BCI system acquires electroencephalographic (EEG) signals, characterized by a low signal-to-noise ratio and an important variability both across sessions and across users. Typically, the BCI system is calibrated before each use, in a process during which the user has to perform a predefined task. This thesis studies of the sources of this variability, with the aim of exploring, designing, and implementing zero-calibration methods. We review the variability of the event related potentials (ERP), focusing mostly on a late component known as the P300. This allows us to quantify the sources of EEG signal variability. Our solution to tackle this variability is to focus on adaptive machine learning methods. We focus on three transfer learning methods: Riemannian Geometry, Optimal Transport, and Ensemble Learning. We propose a model of the EEG takes variability into account. The parameters resulting from our analyses allow us to calibrate this model in a set of simulations, which we use to evaluate the performance of the aforementioned transfer learning methods. These methods are combined and applied to experimental data. We first propose a classification method based on Optimal Transport. Then, we introduce a separability marker which we use to combine Riemannian Geometry, Optimal Transport and Ensemble Learning. Our results demonstrate that the combination of several transfer learning methods produces a classifier that efficiently handles multiple sources of EEG signal variability.

Bakgrund: Stroke drabbar miljontals människor världen över varje år och medför ofta ensidiga motoriska nedsättningar som allvarligt reducerar förmågan till självständighet i vardagen. Fysioterapin efter stroke sker därför vanligen genom uppgiftsorienterad träning riktad mot att rehabilitera den motoriska förmågan på den affekterade sidan så att patienten kan återgå till ett självständigt liv. Men processen ställer stora krav på patienten som inte alltid kan förväntas uppnå bästa resultat med sin rehabilitering. Därför forskas det alltmer på innovativa teknologiska hjälpmedel med potential att assistera strokepatient såväl som fysioterapeut i rehabiliteringen. Exoskelett och hjärndatorgränssnitt (BCI) är två sådana hjälpmedel som undersöktes i denna studie. Syfte: Studien hade syftet att sammanställa det vetenskapliga stödet för tillämpning av BCI-styrda exoskelett (BCI-Exo) vid rehabilitering av motorisk arm- och handfunktion efter stroke i dess subakuta samt kroniska fas. Metod: Litteratursökningar utfördes i databaserna PEDRO, PUBMED, AMED och CINAHL vilket gav 22 träffar som efter granskning och sållning resulterade i att fyra artiklar inkluderades i studien. Resultat: Samtliga studier redovisade statistiskt signifikanta förbättringar av motorisk handfunktion i interventionsgruppen jämfört med kontrollgruppen utifrån de utfallsmått som tillämpades. Konklusion: Resultatet indikerade att BCI-Exo kan främja återhämtning och neuroplasticitet för strokepatienter oavsett vilken fas de infinner sig i. Dock är teknologin fortfarande relativt ny varvid fler studier behöver utföras för att bättre specificera och förstå för- och nackdelar jämfört med konventionella behandlingsmetoder. / Background: Stroke affects millions of people around the world each year and often results in unilateral motor impairments that severely reduce the ability for independence in everyday life. Physiotherapy after stroke is therefore usually performed through task-oriented training aimed at rehabilitating the motor functional ability of the affected side so that the patient can return to an independent life. But the process places great demands on the patient who cannot always be expected to achieve the best results from their rehabilitation. Therefore, innovative technologies are increasingly being researched with the potential to assist stroke patients as well as physical therapists in the rehabilitation process. Exoskeletons and brain-computer interfaces (BCI) are two such rehabilitative tools that were investigated in this study. Objective: The study aimed to compile the scientific support for the use of BCI-controlled exoskeletons (BCI-Exo) in motor functional arm and hand rehabilitation after stroke in its subacute and chronic phase. Method: Literature searches were conducted in the databases PEDRO, PUBMED, AMED and CINAHL, which resulted in 22 hits which, after review and screening, resulted in four articles being included in the study. Results: All studies reported statistically significant improvements regarding motor function in the hemiplegic hand in the intervention group compared to the control group based on the outcome measures used. Conclusion: The results indicated that BCI-Exo can promote recovery and neuroplasticity after stroke regardless of its phase. However, the technology is still in its early stages and more studies need to be performed to better specify and understand the advantages and disadvantages compared to conventional treatment methods.

L’objectif de cette thèse est d’étudier l’utilisation d’Interfaces Cerveau-Ordinateur (ICOs) au sein de la Réalité Virtuelle (RV) et de la Réalité Augmentée (RA). Notre but est d’évaluer la compatibilité entre les systèmes basés sur une ICO et la RV/RA, de concevoir de nouveaux outils pour la visualisation de l’activité cérébrale basée sur la RV/RA, et finalement de proposer de nouveaux usages pour les ICOs, plus particulièrement en combinaison avec des vêtements intelligents. Afin de réaliser ces objectifs, nous avons tout d’abord réalisé une étude de faisabilité concernant l’association entre une ICO et la RV. Notre objectif était d’étudier l’influence de l’activité motrice sur une ICO. Nous avons conçu un système similaire à un jeu vidéo, servant comme support à une étude utilisateur montrant que l’ICO peut être utilisée avec succès, même lorsque les participants exécutent une activité musculaire exigeante. Dans un second temps, nous avons également proposé des outils de visualisation de l’activité cérébrale basés sur la RV/RA. Notre premier système nommé «Mind-Mirror» superpose un cerveau virtuel représentant l’activité cérébrale d’un utilisateur à l’image de celui-ci dans un miroir. Une étude utilisateur a montré qu’aucune perte significative de performance de l’ICO n’a été constatée, même avec une complexité additionnelle due à l’affichage basé sur la RA. Notre seconde contribution se nomme «Mind-Window» et étend les possibilités du Mind-Mirror en permettant plusieurs points de vue sur un même enregistrement d’activité cérébrale en utilisant des tablettes. Notre dernière contribution se nomme «Mind-Inside» et permet aux utilisateurs de visualiser en RV leur activité cérébrale en temps réel tout en étant immergés dans un cerveau virtuel. Enfin, nous avons étudié comment les ICOs et la RV/RA peuvent être appliquées au domaine des vêtements intelligents. Nous avons mis en place une plateforme d’expérimentation consistant en une cabine d’essayage virtuelle intégrant une ICO et permettant aux utilisateurs de porter des vêtements intelligents virtuels en RA. Poursuivant ces travaux, nous avons également conçu une «cape d’invisibilité» inspirée par l’univers de Harry Potter. Cette cape virtuelle permet aux utilisateurs de se camoufler en RA en utilisant leur état mental. Une étude utilisateur sur le contrôle de l’effet a mis en avant l’amélioration de l’expérience utilisateur et «l’impression d’avoir un superpouvoir». / The objective of this PhD thesis is to study the use of Brain- Computer Interfaces (BCIs) within Virtual Reality (VR) and Augmented Reality (AR). Our goal is to evaluate the compatibility between systems based on a BCI and VR/AR, to design new tools for the visualization of the brain activity based on VR/AR, and finally to propose new uses for the BCIs, and especially in combination with smart clothes. In order to fulfil these objectives, we have first designed and performed a feasibility study concerning the combination of a BCI and VR. Our objective was to study the influence of motor activity on a BCI. We have designed a system similar to a video game, serving as a base for a user study showing that this BCI can be successfully used, even when participants are performing a demanding muscular activity. We have also proposed three visualization tools for the brain activity based on VR/AR. Our first system called the «Mind- Mirror» which enables the visualization of our own brain activity «inside our own head» by superimposition. A user study has shown that no significant drop in BCI performance occurred, even with the additional complexity due to our AR-based display. Our second contribution is called «Mind- Window» and extends the Mind-Mirror’s possibilities by enabling one or multiple users to visualize the brain activity of another person as if her skull was transparent. Our last contribution is called «Mind-Inside» and allows users to visualize their brain activity in real-time while being immersed in a virtual brain. Finally, we have studied how BCIs and the VR/AR can be applied to smart clothes. We have designed an experimental platform comprising a dressing room integrating a BCI. Following this work, we proposed an «invisibility cloak» inspired by the Harry Potter universe. This virtual cloak allows users to camouflage themselves in AR using their mental state. Results from a preliminary study based on a simple videogame inspired by the Harry Potter universe could notably show that, compared to a standard control made with a keyboard, controlling the optical camouflage directly with the BCI could enhance the user experience and the feeling of «having a super-power».

Event-related brain potentials (ERPs) in electroencephalography are manifestations at the scalp of neural activity that is triggered by, and is involved in, the processing of specific events. This chapter focuses on braincomputer interfaces (BCIs) that use P300, an endogenous ERP component. The P300 is a positive potential that occurs over central-parietal scalp 250- 700 msec after a rare event occurs in the context of the oddball paradigm. This paradigm has three essential attributes: a subject is presented with a series of events (i.e., stimuli), each of which falls into one of two classes; the events that fall into one of the classes are less frequent than those that fall into the other class; and the subject performs a task that requires classifying each event into one of the two classes.

Objective: To assess the reliability and usefulness of an EEG-based brain-computer interface (BCI) for patients with advanced amyotrophic lateral sclerosis (ALS) who used it independently at home for up to 18 months. Methods: Of 42 patients consented, 39 (93%) met the study criteria, and 37 (88%) were assessed for use of the Wadsworth BCI. Nine (21%) could not use the BCI. Of the other 28, 27 (men, age 28-79 years) (64%) had the BCI placed in their homes, and they and their caregivers were trained to use it. Use data were collected by Internet. Periodic visits evaluated BCI benefit and burden and quality of life. Results: Over subsequent months, 12 (29% of the original 42) left the study because of death or rapid disease progression and 6 (14%) left because of decreased interest. Fourteen (33%) completed training and used the BCI independently, mainly for communication. Technical problems were rare. Patient and caregiver ratings indicated that BCI benefit exceeded burden. Quality of life remained stable. Of those not lost to the disease, half completed the study; all but 1 patient kept the BCI for further use. Conclusion: The Wadsworth BCI home system can function reliably and usefully when operated by patients in their homes. BCIs that support communication are at present most suitable for people who are severely disabled but are otherwise in stable health. Improvements in BCI convenience and performance, including some now underway, should increase the number of people who find them useful and the extent to which they are used.

Brain-computer interfaces (BCIs) provide direct access to the brain, serving crucial roles in treating neurological disorders and developing neural prostheses. Recent clinical successes include diagnosing and treating epilepsy and advancing prosthesis for visual and limb impairments. Achieving high spatial and temporal resolution is essential for accurately localizing seizures, mapping brain functions, and controlling neuronal activity. However, existing solutions have substantial form factors, necessitating large-size craniotomy, permanent removal of a part of the skull, or wires running through the body, which limits real-world applicability and complicates post-surgery recovery. We present a minimally invasive, high-bandwidth, and fully wireless brain-machine interface platform that addresses these challenges through a combination of an implantable application-specific integrated circuit (ASIC) chip and a wearable relay station. The platform supports an aggregate sampling rate of 8.68 MSPS at 10-bit resolution and a 108.48/54.24 Mbps data rate using impulse radio ultra-wideband (IR-UWB). A high-density microelectrode array (HD-MEA) with configurable electrode options is integrated into the ASIC implant, enabling simultaneous readout of 1024/256 channels at 8.48/33.9 KSPS. By reducing the ASIC implant to a thickness of 25 µm, the total volume of the implant is only 3.6 mm³, making it thinner than a strand of human hair and occupying less than a third of the volume of a grain of rice. We conducted in-vivo experiments in the cortices of pigs and monkeys and successfully achieved ultra-high resolution receptive field mapping. This work sets a new standard for volumetric efficiency in implantable brain-computer interfaces.

Traditionally, electrical brain-computer interfaces (BCIs) have required the assembly of two separate components: electrodes for interfacing with tissue and electronics for signal acquisition and stimulation. Furthermore, these electronics required cabled connections to workstations for data processing and control. Efforts to overcome these limitations have made significant progress in the last decade. Now, there are in vivo validated monolithic electrophysiological BCI devices, exemplified by the Neuropixels, that integrate the two components onto a single platform. At the same time, a new generation of fully wireless BCI devices that reside entirely under the skin has been developed. Despite these advancements, the current state-of-the-art BCIs have yet to overcome both challenges simultaneously. Multi-channel, high-bandwidth monolithic BCIs still require percutaneous wired connections, whereas wireless BCIs rely on the assembly of discrete components that result in bulky form factors. The next generation of BCIs calls for a new paradigm that integrates electrodes and electronics into a miniaturized form factor while supporting a fully wireless operation. This thesis contributes to the collaborative effort that presents such a paradigm through the development of a wireless, battery-free micro-electrocorticography (μECoG) device that monolithically integrates electrodes, signal processing, data telemetry, and powering onto a single complementary metal-oxide-semiconductor (CMOS) substrate. The device contains 65,536 recording and 16,384 stimulation channels, from which a programmable subset of up to 1024 channels can berecorded at a given time. Implemented in a mechanically flexible, 50-μm-thick form factor with a total volume of only 7.2 mm³, the device is implanted entirely in the subdural space and conforms to the contour of the cortical tissue surface. A custom "relay station" provides wireless powering and bi-directional communication to the implant from outside the body. The system was validated through a series of proof-of-concept in vivo recordings from different cortical regions of a pig and non-human primates, reliably decoding brain signals at high spatiotemporal resolution. By using a unique, fully integrated architecture, the BCI developed in this work achieves orders-of-magnitude improvements in volumetric efficiency and channel count over existing approaches, setting a milestone for the next generation of BCI devices.

Developments in materials science and engineering have significantly enhanced the recording and stimulation capabilities of electrophysiological neural interface devices. Enhanced biocompatibility has increased the viability and longevity of such systems, with particularly interesting advances resulting from the utilization of organic and mixed-conductive conjugated polymers. These materials tend to improve biocompatibility and signal quality by overcoming material limitations of conventional metallic electrodes. Simultaneously, advanced microfabrication methods have increased the spatiotemporal resolution and signal quality of recorded data without added invasiveness. The research work presented in this dissertation touches upon three aspects of relevance for improved neural interface use: i) improving the accuracy of histological verification procedures in research using naturally abundant organic materials; ii) introducing unconventional electrodes for neural recordings and localized drug delivery by utilizing conductive organic polymers and clinical supply items; iii) applying a high-density electrocorticography (ECoG) array to study differential neural oscillation patterns underlying memory processing in vivo. First, we showcase a chitosan (CS) based, solution-processable film for localizing neural implants by leveraging CSs intrinsic fluorescence, without impeding data quality or cell viability. Second, we develop a mixed-conductive suture by using standard silk sutures and the mixed-conductive polymer PEDOT:PSS. The resulting device (E-Suture) is shown to safely interface with live tissue and possess high-fidelity recording and stimulation capabilities as well as applicability for localized drug delivery thanks to the mixed-conductivity of PEDOT:PSS. Finally, we leverage high spatiotemporal resolution ECoG arrays to show that distinctive oscillatory memory biomarkers in the neocortex and hippocampus show significant but differential temporal coupling patterns in response to consolidation of new information and reconsolidation of that information at a later time in rats. This dissertation demonstrates the utility of different organic materials for the enhancement of neural interface functions at multiple phases of the device life cycle as well as a concrete demonstration of improved electrophysiological recording devices in answering key questions of foundational neuroscience.

Thesis (MEng)--Stellenbosch University, 2014. / ENGLISH ABSTRACT: Muscular dystrophy, spinal cord injury, or amyotrophic lateral sclerosis (ALS) are injuries and disorders that disrupts the neuromuscular channels of the human body thus prohibiting the brain from controlling the body. Brain computer interface (BCI) allows individuals to bypass the neuromuscular channels and interact with the environment using the brain. The system relies on the user manipulating his neural activity in order to control an external device. Electroencephalography (EEG) is a cheap, non-invasive, real time acquisition device used in BCI applications to record neural activity. However, noise, known as artifacts, can contaminate the recording, thus distorting the true neural activity. Eye blinks are a common source of artifacts present in EEG recordings. Due to its large amplitude it greatly distorts the EEG data making it difficult to interpret data for BCI applications. This study proposes a new combination of techniques to detect and correct eye blink artifacts to improve the quality of EEG for BCI applications. Independent component analysis (ICA) is used to separate the EEG signals into independent source components. The source component containing eye blink artifacts are corrected by detecting each eye blink within the source component and using a trained wavelet neural network (WNN) to correct only a segment of the source component containing the eye blink artifact. Afterwards, the EEG is reconstructed without distorting or removing the source component. The results show a 91.1% detection rate and a 97.9% correction rate for all detected eye blinks. Furthermore for channels located over the frontal lobe, eye blink artifacts are corrected preserving the neural activity. The novel combination overall reduces EEG information lost, when compared to existing literature, and is a step towards improving EEG pre-processing in order to provide cleaner EEG data for BCI applications. / AFRIKAANSE OPSOMMING: Spierdistrofie, ’n rugmurgbesering, of amiotrofiese laterale sklerose (ALS) is beserings en steurnisse wat die neuromuskulêre kanale van die menslike liggaam ontwrig en dus verhoed dat die brein die liggaam beheer. ’n Breinrekenaarkoppelvlak laat toe dat die neuromuskulêre kanale omlei word en op die omgewing reageer deur die brein. Die BCI-stelsel vertrou op die gebruiker wat sy eie senuwee-aktiwiteit manipuleer om sodoende ’n eksterne toestel te beheer. Elektro-enkefalografie (EEG) is ’n goedkoop, nie-indringende, intydse dataverkrygingstoestel wat gebruik word in BCI toepassings. Nie net senuwee aktiwiteit nie, maar ook geraas , bekend as artefakte word opgeneem, wat dus die ware senuwee aktiwiteit versteur. Oogknip artefakte is een van die algemene artefakte wat teenwoordig is in EEG opnames. Die groot omvang van hierdie artefakte verwring die EEG data wat dit moeilik maak om die data te ontleed vir BCI toepassings. Die studie stel ’n nuwe kombinasie tegnieke voor wat oogknip artefakte waarneem en regstel om sodoende die kwaliteit van ’n EEG vir BCI toepassings te verbeter. Onafhanklike onderdeel analise (Independent component analysis (ICA)) word gebruik om die EEG seine te skei na onafhanklike bron-komponente. Die bronkomponent wat oogknip artefakte bevat word reggestel binne die komponent en gebruik ’n ervare/geoefende golfsenuwee-netwerk om slegs ’n deel van die komponent wat die oogknip artefak bevat reg te stel. Daarna word die EEG hervorm sonder verwringing of om die bron-komponent te verwyder. Die resultate toon ’n 91.1% opsporingskoers en ’n 97.9% regstellingskoers vir alle waarneembare oogknippe. Oogknip artefakte in kanale op die voorste lob word reggestel en behou die senuwee aktiwiteit wat die oorhoofse EEG kwaliteit vir BCI toepassings verhoog.