Severe neuromuscular disorders can produce locked-in syndrome (LIS), a loss of nearly all voluntary muscle control. A brain-computer interface (BCI) using the P300 event-related potential provides communication that does not depend on neuromuscular activity and can be useful for those with LIS. Currently, there is no way of determining the effectiveness of P300-based BCIs without testing a person's performance multiple times. Additionally, P300 responses in BCI tasks may not resemble the typical P300 response. I sought to clarify the relationship between the P300 response and BCI task parameters and examine the possibility of a predictive relationship between traditional oddball tasks and BCI performance. Both waveform and component analysis have revealed several task-dependent aspects of brain activity that show significant correlation with the user's performance. These components may provide a fast and reliable metric to indicate whether the BCI system will work for a given individual.

This thesis presents a novel profile shape matching stereo vision algorithm. This algorithm is able to obtain 3D information in real time from a pair of stereo images. This algorithm produces the 3D information by matching the profile intensity shapes on the same row of the two images from a stereo image pair. The advantage of this profile shape matching algorithm is that the detection of correspondences relies on intensity profile shape not on intensity values, which subject to lighting variations. The user can choose an interval of disparity, and then an object in a desired distance range can be segmented out from the background. In other words, the algorithm detects the object according to its distance to the cameras. Based on the resulting 3D information, the movement and gesture of the control agents, in our test cases the human body and fingers, in space in a desired distance range can be determined. The body movement and gestures can then be analyzed for human-computer interface purposes. In this thesis, the algorithm was applied for human pose and hand gesture estimation. To demonstrate its performance the estimation results were interpreted as inputs and sent to a smart phone to control its functions. While this algorithm does have a trade-off between accuracy and processing speed, we found a balance that can produce the result in real time, and the result has sufficient accuracy for practical use of recognizing human poses and hand gesture. The experimental result shows that the proposed algorithm has higher accuracy and is 1.14× faster than the original version on tested stereo image pairs.

Objective: An electroencephalographic brain-computer interface (BCI) can provide a non-muscular means of communication for people with amyotrophic lateral sclerosis (ALS) or other neuromuscular disorders. We present a novel P300-based BCI stimulus presentation - the checkerboard paradigm (CBP). CBP performance is compared to that of the standard row/column paradigm (RCP) introduced by Farwell and Donchin (1988). Methods: Using an 8 × 9 matrix of alphanumeric characters and keyboard commands, 18 participants used the CBP and RCP in counter-balanced fashion. With approximately 9-12 min of calibration data, we used a stepwise linear discriminant analysis for online classification of subsequent data. Results: Mean online accuracy was significantly higher for the CBP, 92%, than for the RCP, 77%. Correcting for extra selections due to errors, mean bit rate was also significantly higher for the CBP, 23 bits/min, than for the RCP, 17 bits/min. Moreover, the two paradigms produced significantly different waveforms. Initial tests with three advanced ALS participants produced similar results. Furthermore, these individuals preferred the CBP to the RCP. Conclusions: These results suggest that the CBP is markedly superior to the RCP in performance and user acceptability. Significance: The CBP has the potential to provide a substantially more effective BCI than the RCP. This is especially important for people with severe neuromuscular disabilities.

Objective: Brain-computer interface (BCI) technology can provide severely disabled people with non-muscular communication. For those most severely disabled, limitations in eye mobility or visual acuity may necessitate auditory BCI systems. The present study investigates the efficacy of the use of six environmental sounds to operate a 6 × 6 P300 Speller. Methods: A two-group design was used to ascertain whether participants benefited from visual cues early in training. Group A (N = 5) received only auditory stimuli during all 11 sessions, whereas Group AV (N = 5) received simultaneous auditory and visual stimuli in initial sessions after which the visual stimuli were systematically removed. Stepwise linear discriminant analysis determined the matrix item that elicited the largest P300 response and thereby identified the desired choice. Results: Online results and offline analyses showed that the two groups achieved equivalent accuracy. In the last session, eight of 10 participants achieved 50% or more, and four of these achieved 75% or more, online accuracy (2.8% accuracy expected by chance). Mean bit rates averaged about 2 bits/min, and maximum bit rates reached 5.6 bits/min. Conclusions: This study indicates that an auditory P300 BCI is feasible, that reasonable classification accuracy and rate of communication are achievable, and that the paradigm should be further evaluated with a group of severely disabled participants who have limited visual mobility. Significance: With further development, this auditory P300 BCI could be of substantial value to severely disabled people who cannot use a visual BCI.

When analyzing complex scenes, humans often focus their attention on an object at a particular spatial location in the presence of background noises and irrelevant visual objects. The ability to decode the attended spatial location would facilitate brain computer interfaces (BCI) for complex scene analysis. Here, we tested two different neuroimaging technologies and investigated their capability to decode audio-visual spatial attention in the presence of competing stimuli from multiple locations. For functional near-infrared spectroscopy (fNIRS), we targeted dorsal frontoparietal network including frontal eye field (FEF) and intra-parietal sulcus (IPS) as well as superior temporal gyrus/planum temporal (STG/PT). They all were shown in previous functional magnetic resonance imaging (fMRI) studies to be activated by auditory, visual, or audio-visual spatial tasks. We found that fNIRS provides robust decoding of attended spatial locations for most participants and correlates with behavioral performance. Moreover, we found that FEF makes a large contribution to decoding performance. Surprisingly, the performance was significantly above chance level 1s after cue onset, which is well before the peak of the fNIRS response. For electroencephalography (EEG), while there are several successful EEG-based algorithms, to date, all of them focused exclusively on auditory modality where eye-related artifacts are minimized or controlled. Successful integration into a more ecological typical usage requires careful consideration for eye-related artifacts which are inevitable. We showed that fast and reliable decoding can be done with or without ocular-removal algorithm. Our results show that EEG and fNIRS are promising platforms for compact, wearable technologies that could be applied to decode attended spatial location and reveal contributions of specific brain regions during complex scene analysis.

No description available.

No description available.

No description available.

No description available.

No description available.