The recent advent of consumer grade Brain-Computer Interfaces (BCI) provides a new revolutionary and accessible way to control computers. BCI translate cognitive electroencephalography (EEG) signals into computer or robotic commands using specially built headsets. Capable of enhancing traditional interfaces that require interaction with a keyboard, mouse or touchscreen, BCI systems present tremendous opportunities to benefit various fields. Movement restricted users can especially benefit from these interfaces. In this thesis, we present a new way to interface a consumer-grade BCI solution to a mobile robot. A Red-Green-Blue-Depth (RGBD) camera is used to enhance the navigation of the robot with cognitive thoughts as commands. We introduce an interface presenting 3 different methods of robot-control: 1) a fully manual mode, where a cognitive signal is interpreted as a command, 2) a control-flow manual mode, reducing the likelihood of false-positive commands and 3) an automatic mode assisted by a remote RGBD camera. We study the application of this work by navigating the mobile robot on a planar surface using the different control methods while measuring the accuracy and usability of the system. Finally, we assess the newly designed interface’s role in the design of future generation of BCI solutions.

Brain-computer interface (BCI) technologies allow users to control external devices through brain activity alone, circumventing the somatic nervous system and the need for overt physical movement. BCIs may potentially benefit individuals with severe neuromuscular disorders who experience significant, and often total, loss of voluntary muscle control (e.g. amyotrophic lateral sclerosis, multiple sclerosis, brainstem stroke). Though a majority of BCI research to date has focused on electroencephalography (EEG) for brain signal acquisition, recently researchers have noted the potential of an optical imaging technology called near-infrared spectroscopy (NIRS) for BCI applications. This thesis investigates the feasibility of a practical, online optical BCI based on conscious modulation of prefrontal cortex activity through the performance of different cognitive tasks, specifically mental arithmetic (MA) and mental singing (MS). The thesis comprises five studies, each representing a step toward the realization of a practical optical BCI. The first study demonstrates the feasibility of a two-choice synchronized optical BCI based on intentional control states corresponding to MA and MS. The second study explores a more user-friendly alternative - a two-choice system-paced BCI supporting a single intentional control state (either MA or MS) and a natural baseline, or "no-control (NC)", state. The third study investigates the feasibility of a three-choice system-paced BCI supporting both MA and MS, as well as the NC state. The fourth study examines the consistency with which the relevant mental states can be differentiated over multiple sessions. The first four studies involve healthy adult participants; in the final study, the feasibility of optical BCI use by a user with Duchenne muscular dystrophy is explored. In the first study, MA and MS were classified with an average accuracy of 77.2% (n=10), while in the second, MA and MS were differentiated individually from the NC state with average accuracies of 71.2% and 62.7%, respectively (n=7). In the third study, an average accuracy of 62.5% was obtained for the MA vs. MS vs. NC problem (n=4). The fourth study demonstrated that the ability to classify mental states (specifically MA vs. NC) remains consistent across multiple sessions (p=0.67), but that there is intersession variability in the spatiotemporal characteristics that best discriminate the states. In the final study, a two-session average accuracy of 71.1% was achieved in the MA vs. NC classification problem for the participant with Duchenne muscular dystrophy.

Individuals with limited motor control are often unable to interact with their environment. Recently, near-infrared spectroscopy (NIRS) systems have been investigated as potential brain-computer interfaces (BCI). Previous studies examined data offline, preventing users from understanding how their thoughts triggered the NIRS system. This thesis focused on understanding the short-term effects of feedback on user’s ability to learn how to control BCIs. Data were collected from control and experimental groups over seven sessions, as they performed fast singing imagery or mental arithmetic. Significant differences were observed between the control group’s results in non-feedback sessions and the experimental group’s results in feedback sessions. Qualitative results from 3 of the 10 participants suggested they had control of the feedback system. They performed the task with online accuracies of 61% - 88% in the final 2 sessions with feedback. These results suggest that continued investigation of NIRS feedback systems is warranted.

Near-infrared spectroscopy (NIRS) is an emerging non-invasive brain-computer interface (BCI) modality that measures changes in hemoglobin concentrations in neurocortical tissue. Previous NIRS studies have not employed real-time feedback with online classification, a combination which would allow users to alter their mental strategy on the fly. This thesis reports the results of two online studies. The first study contrasted online classification of prefrontal hemodynamics using an artificial neural network (ANN) and a hidden Markov model-based (HMM) classifier. The second study measured the accuracy of an online linear discriminant classifier. In study 1, only the ANN classifier facilitated online classification rates greater than chance (p=0.0289). In study 2, a new feedback system and experimental protocol led to improved classification rates over those of the first study (p=5.1*10^(-5)). While control over instantaneously generated feedback in online NIRS-BCIs has been demonstrated, factors such as user frustration, mental fatigue, and restrictions on ambient lighting may compromise performance.

Severe motor impairments can severely restrict interaction with one's surroundings. Brain computer interfaces combined with text-based communication systems, such as the P300 Speller, have allowed individuals with motor disabilities to spell messages with their EEG signals. Although providing full composition flexibility, they enable communication rates of only a few characters per minute. Utterance-based communication systems have been developed for individuals with disability and have greatly increased communication speeds, but have yet to be applied to BCIs. This paper proposes an utterance-based communication system using the P300-BCI in which words are organized in a network structure that facilitates rapid retrieval. In trials with able-bodied participants, the proposed system achieved greater message speeds, but rated lower in effectiveness than the P300 Speller. Nonetheless, subject preferences and reports of self-perceived effectiveness suggested an inclination towards the proposed word system and thus further investigation of word-based networks is warranted in brain-controlled AAC systems.

Near-infrared spectroscopy (NIRS) is an emerging non-invasive brain-computer interface (BCI) modality that measures changes in hemoglobin concentrations in neurocortical tissue. Previous NIRS studies have not employed real-time feedback with online classification, a combination which would allow users to alter their mental strategy on the fly. This thesis reports the results of two online studies. The first study contrasted online classification of prefrontal hemodynamics using an artificial neural network (ANN) and a hidden Markov model-based (HMM) classifier. The second study measured the accuracy of an online linear discriminant classifier. In study 1, only the ANN classifier facilitated online classification rates greater than chance (p=0.0289). In study 2, a new feedback system and experimental protocol led to improved classification rates over those of the first study (p=5.1*10^(-5)). While control over instantaneously generated feedback in online NIRS-BCIs has been demonstrated, factors such as user frustration, mental fatigue, and restrictions on ambient lighting may compromise performance.

Individuals with limited motor control are often unable to interact with their environment. Recently, near-infrared spectroscopy (NIRS) systems have been investigated as potential brain-computer interfaces (BCI). Previous studies examined data offline, preventing users from understanding how their thoughts triggered the NIRS system. This thesis focused on understanding the short-term effects of feedback on user’s ability to learn how to control BCIs. Data were collected from control and experimental groups over seven sessions, as they performed fast singing imagery or mental arithmetic. Significant differences were observed between the control group’s results in non-feedback sessions and the experimental group’s results in feedback sessions. Qualitative results from 3 of the 10 participants suggested they had control of the feedback system. They performed the task with online accuracies of 61% - 88% in the final 2 sessions with feedback. These results suggest that continued investigation of NIRS feedback systems is warranted.

A Human-Machine Interface is a device that allows humans to inter- act with and use machines. One such device is a Brain-Computer Interface which allows the user to communicate to a computer system through thought patterns. A commonly used technique, electroencephalography, uses multiple sensors positioned on the subject’s cranium to extract electrical changes as a representation of thought patterns. This report investigates the use of a single EEG sensor as a user-friendly BCI implementation. The primary goal of this report is to determine if specific mental tasks can be reliably detected with such a system. / text

Large data sets are difficult to analyze. Visualization has been proposed to assist exploratory data analysis (EDA) as our visual systems can process signals in parallel to quickly detect patterns. Nonetheless, designing an effective visual analytic tool remains a challenge. This challenge is partly due to our incomplete understanding of how common visualization techniques are used by human operators during analyses, either in laboratory settings or in the workplace. This thesis aims to further understand how visualizations can be used to support EDA. More specifically, we studied techniques that display multiple levels of visual information resolutions (VIRs) for analyses using a range of methods. The first study is a summary synthesis conducted to obtain a snapshot of knowledge in multiple-VIR use and to identify research questions for the thesis: (1) low-VIR use and creation; (2) spatial arrangements of VIRs. The next two studies are laboratory studies to investigate the visual memory cost of image transformations frequently used to create low-VIR displays and overview use with single-level data displayed in multiple-VIR interfaces. For a more well-rounded evaluation, we needed to study these techniques in ecologically-valid settings. We therefore selected the application domain of web session log analysis and applied our knowledge from our first three evaluations to build a tool called Session Viewer. Taking the multiple coordinated view and overview + detail approaches, Session Viewer displays multiple levels of web session log data and multiple views of session populations to facilitate data analysis from the high-level statistical to the low-level detailed session analysis approaches. Our fourth and last study for this thesis is a field evaluation conducted at Google Inc. with seven session analysts using Session Viewer to analyze their own data with their own tasks. Study observations suggested that displaying web session logs at multiple levels using the overview + detail technique helped bridge between high-level statistical and low-level detailed session analyses, and the simultaneous display of multiple session populations at all data levels using multiple views allowed quick comparisons between session populations. We also identified design and deployment considerations to meet the needs of diverse data sources and analysis styles.

Session ID: SA-B1-2 / SCIS & ISIS 2010, Joint 5th International Conference on Soft Computing and Intelligent Systems and 11th International Symposium on Advanced Intelligent Systems. December 8-12, 2010, Okayama Convention Center, Okayama, Japan