<p>A series of fusion techniques are developed and applied to EEG and pupillary recording analysis in a rapid serial visual presentation (RSVP) based image triage task, in order to improve the accuracy of capturing single-trial neural/pupillary signatures (patterns) associated with visual target detection.</p><p>The brain response to visual stimuli is not a localized pulse, instead it reflects time-evolving neurophysiological activities distributed selectively in the brain. To capture the evolving spatio-temporal pattern, we divide an extended (``global") EEG data epoch, time-locked to each image stimulus onset, into multiple non-overlapping smaller (``local") temporal windows. While classifiers can be applied on EEG data located in multiple local temporal windows, outputs from local classifiers can be fused to enhance the overall detection performance.</p><p>According to the concept of induced/evoked brain rhythms, the EEG response can be decomposed into different oscillatory components and the frequency characteristics for these oscillatory components can be evaluated separately from the temporal characteristics. While the temporal-based analysis achieves fairly accurate detection performance, the frequency-based analysis can improve the overall detection accuracy and robustness further if frequency-based and temporal-based results are fused at the decision level.</p><p>Pupillary response provides another modality for a single-trial image triage task. We developed a pupillary response feature construction and selection procedure to extract/select the useful features that help to achieve the best classification performance. The classification results based on both modalities (pupillary and EEG) are further fused at the decision level. Here, the goal is to support increased classification confidence through inherent modality complementarities. The fusion results show significant improvement over classification results using any single modality.</p><p>For crucial image triage tasks, multiple image analysts could be asked to evaluate the same set of images to improve the probability of detection and reduce the probability of false positive. We observe significant performance gain by fusing the decisions drawn by multiple analysts.</p><p>To develop a practical real-time EEG-based application system, sometimes we have to work with an EEG system that has a limited number of electrodes. We present methods of ranking the channels, identifying a reduced set of EEG channels that can deliver robust classification performance.</p> / Dissertation