Objective: The current study evaluates the efficacy of a P300-based brain-computer interface (BCI) communication device for individuals with advanced ALS. Methods: Participants attended to one cell of a N x N matrix while the N rows and N columns flashed randomly. Each cell of the matrix contained one character. Every flash of an attended character served as a rare event in an oddball sequence and elicited a P300 response. Classification coefficients derived using a stepwise linear discriminant function were applied to the data after each set of flashes. The character receiving the highest discriminant score was presented as feedback. Results: In Phase I, six participants used a 6 x 6 matrix on 12 separate days with a mean rate of 1.2 selections/min and mean online and offline accuracies of 62% and 82%, respectively. In Phase II, four participants used either a 6 x 6 or a 7 x 7 matrix to produce novel and spontaneous statements with a mean online rate of 2.1 selections/min and online accuracy of 79%. The amplitude and latency of the P300 remained stable over 40 weeks. Conclusions: Participants could communicate with the P300-based BCI and performance was stable over many months. Significance: BCIs could provide an alternative communication and control technology in the daily lives of people severely disabled by ALS.

The flow state is a subjective experience that most people can relate to. It represents an optimal balance between skills and difficulty and is the state that people often refer to when performing their best, with phrases like: “I was in the zone” or “I was in the bubble”. The flow state has mainly been studied through its psychological and behavioral components; it is not until lately the neuroscientific aspects have been investigated. This review attempts to go through the existing literature and find potential neural signatures of the flow state. The studies indicate that flow is related to activity in the dorsolateral prefrontal cortex and putamen, but the findings are too divided to reach a conclusion.

Augmented Reality (AR) head-mounted display (HMD) provides users with an immersive virtual experience in the real world. The portability of this technology affords various information display options for construction workers that are not possible otherwise. The information delivered via an interactive user interface provides an innovative method to display complex building instructions, which is more intuitive and accessible compared with traditional paper documentations. However, there are still challenges hindering the practical usage of this technology at the construction jobsite. As a technical restriction, current AR HMD products have a limited field of view (FOV) compared to the human vision range. It leads to an uncertainty of how the obstructed view of display will affect construction workers' perception of hazards in their surrounding area. Similarly, the information displayed to workers requires rigorous testing and evaluation to make sure that it does not lead to information overload. Therefore, it is essential to comprehensively evaluate the impacts of using AR HMD from both perspectives of task performance and cognitive performance. This dissertation aims to bridge the gap in understanding the cognitive impacts of using AR HMD in construction assembly tasks. Specifically, it focuses on answering the following two questions: (1) How are task performance and cognitive skills affected by AR displays under complex working conditions? (2) How are moment-to-moment changes of mental workload captured and evaluated during construction assembly tasks? To answer these questions, this dissertation proposed two experiments. The first study tests two AR displays (conformal and tag-along) and paper instruction under complex working conditions, involving different framing scales and interference settings. Subjective responses are collected and analyzed to evaluate overall mental workload and situation awareness. The second study focuses on exploring an electroencephalogram (EEG) based approach for moment-to-moment capture and evaluation of mental workload. It uncovers the cognitive change on the time domain and provides room for further quantitative analyzing on mental workload. Especially, two frameworks of mental workload prediction are proposed by using (1) Long Short-Term Memory (LSTM) and (2) one-dimensional Convolutional Neural Network (1D CNN)-LSTM for forecasting EEG signal and, classifying task conditions and mental workload levels respectively. The approaches are tested to be effective and reliable for predicting and recognizing subjects' mental workload during assembly. In brief, this research contributes to the existing knowledge with an assessment of AR HMD use in construction assembly, including task performance evaluation and both subjective and physiological measurements for cognitive skills. / Doctor of Philosophy / Augmented Reality (AR) is an emerging technology that bridges the gap between virtual creatures and physical world with an immersive display experience. Today, head-mounted display (HMD) is well developed to meet the demands for portable AR devices. It provides interactive and intuitive display of 2D graphical information to make it easier to understand for users. Therefore, AR display has been studied in the past few years for a more simplified and productive construction assembly process. However, given the premise that construction is a high-risk industry, introducing such display technology to the jobsite needs to be carefully tested. One obstacle in current AR HMD products is the restriction of field of view (FOV), which may block users' view in presenting large-scale 3D objects. In construction assembly, workers need to deal with tasks in different scopes, such as wood framing for a residential house. Consequently, it is necessary to study how such technical challenge will impact workers' performance under different task conditions. Another concern comes from the mental perspective. Although AR display may bring convenience in acquiring effective information, it is difficult to measure if this generates excessive mental burden to users. Especially for construction workers, whether the overlaid display will cause distraction and information overload is crucial for protecting workers from hazards. To address the problems, this dissertation explores the gap in previous literature, where mental workload is not well studied for using AR HMD in construction assembly. Two experiments are conducted to comprehensively evaluate the impacts of AR displays on both assembly performance and users' mental status. The outcomes bring implications to theoretical and practical aspects. First, it compares two AR displays (2D tag-along image and 3D conformal model) with traditional paper documentation for assembly performance (efficiency and accuracy) and users' cognitive skills (mental workload and situation awareness). The findings revealed the impact of FOV restriction and provided a strategic solution to selecting display method for different task conditions. Second, it proposes a physiological approach to calculate mental workload from analyzing the features from brain waves. It uncovered the latent mental changes during the assembly. Furthermore, two deep learning approaches are applied to predict and classify mental workload. The prediction model depicted the trend of mental workload in eighteen seconds based on an eighty-four-second training set, while the classifier recognized two task conditions with different mental workload levels with an accuracy of 93.6%. The results have promising potential for future research in detecting and preventing abnormality in workers' mental status. In addition, it is generalizable to apply in other construction tasks and AR applications.

Inner speech, or self-talk, is a process by which we talk to ourselves to think through problems or plan actions. Although inner speech is ubiquitous, its neural basis remains largely unknown. This thesis investigates the feasibility of using brain signal analysis to classify the recorded electroencephalography (EEG) data from participants engaged in tasks involving Inner Speech and made publicly available by Nieto et al. (2021). We present the implementation of four machine learning models, demonstrate the results, and compare using different protocols. The results are compared to the ones obtained by Berg et al. (2021), who used the same dataset. Two of the classical models we tried (SVC and LinearSVC) prove superior even against results obtained with deep learning models. We also compare the results from Inner Speech with Pronounced Speech to validate the reusability of the proposed method. We found an apparent regularity in the data on the results, validating the method’s quality and reusability.

No description available.

No description available.

Interpretation of electroencephalograms (EEGs) is a process that is still dependent on the subjective analysis of the examiner. The interrater agreement, even for relevant clinical events such as seizures, can be low. For instance, the differences between interictal, ictal, and post-ictal EEGs can be quite subtle. Before making such low-level interpretations of the signals, neurologists often classify EEG signals as either normal or abnormal. Even though the characteristics of a normal EEG are well defined, there are some factors, such as benign variants, that complicate this decision. However, neurologists can make this classification accurately by only examining the initial portion of the signal. Therefore, in this thesis, we explore the hypothesis that high performance machine classification of an EEG signal as abnormal can approach human performance using only the first few minutes of an EEG recording. The goal of this thesis is to establish a baseline for automated classification of abnormal adult EEGs using state of the art machine learning algorithms and a big data resource – The TUH EEG Corpus. A demographically balanced subset of the corpus was used to evaluate performance of the systems. The data was partitioned into a training set (1,387 normal and 1,398 abnormal files), and an evaluation set (150 normal and 130 abnormal files). A system based on hidden Markov Models (HMMs) achieved an error rate of 26.1%. The addition of a Stacked Denoising Autoencoder (SdA) post-processing step (HMM-SdA) further decreased the error rate to 24.6%. The overall best result (21.2% error rate) was achieved by a deep learning system that combined a Convolutional Neural Network and a Multilayer Perceptron (CNN-MLP). Even though the performance of our algorithm still lags human performance, which approaches a 1% error rate for this task, we have established an experimental paradigm that can be used to explore this application and have demonstrated a promising baseline using state of the art deep learning technology. / Electrical and Computer Engineering

Oscillatory states in the Electroencephalogram (EEG) reflect the rhythmic synchronous activation in large networks of neurons. Time-frequency methods quantify the spectral content of the EEG as a function of time. As such, they are well suited as tools for the study of spontaneous and induced changes in oscillatory states. We have used time-frequency techniques to analyze the flow of activity patterns between two strongly connected brain structures: the entorhinal cortex and the hippocampus, which are believed to be involved in information storage. EEG was recorded simultaneously from the entorhinal cortex and the hippocampus of behaving rats. During the recording, low-intensity trains of electrical pulses at frequencies between 1 and 40 Hz were applied to the olfactory (piriform) cortex. The piriform cortex projects to the entorhinal cortex, which then passes the signal on to the hippocampus. Several time-frequency methods, including the short-time Fourier transform (STFT), Wigner-Ville distribution (WVD) and multiple window (MW) time-frequency analysis (TFA), were used to analyse EEG signals. To monitor the signal transmission between the entorhinal cortex and hippocampus, the time-frequency coherence functions were used. The analysed results showed that stimulation-related power in both sites peaked near 15 Hz, but the coherence between the EEG signals recorded from these two sites increased monotonically with stimulation frequency. Among the time-frequency methods used, the STFT provided time-frequency distributions not only without cross-terms which were present in the WVD, but also with higher resolutions in both time and frequency than the MW-TFA. The STFT seems to be the most suitable time-frequency method to study the stimulation-induced signals presented in this thesis. The MW-TFA, which gives low bias and low variance estimations of the time-frequency distribution when only one realization of data is given, is suitable for stochastic and nonstationary signals such as spontaneous EEG. We also compared the performance of the MW-TFA using two different window functions: Slepian sequences and Hermite functions. By carefully matching the two window functions, we found no noticeable difference in time-frequency plane between them. / Thesis / Master of Engineering (ME)

Mobile phones are one of the essential parts of modern life. Making a phone call is not the main purpose of a smart phone anymore, but merely one of many other features. Online social networking, chatting, short messaging, web browsing, navigating, and photography are some of the other features users enjoy in modern smartphones, most of which are provided by mobile apps. However, with this advancement, many security vulnerabilities have opened up in these devices. Malicious apps are a major threat for modern smartphones. According to Symantec Corp., by the middle of 2013, about 273,000 Android malware apps were identified. It is a complex issue to protect everyday users of mobile devices from the attacks of technologically competent hackers, illegitimate users, trolls, and eavesdroppers. This dissertation emphasizes the concept of intention identification. Then it looks into ways to utilize this intention identification concept to enforce security in a mobile phone platform. For instance, a battery monitoring app requiring SMS permissions indicates suspicious intention as battery monitoring usually does not need SMS permissions. Intention could be either the user's intention or the intention of an app. These intentions can be identified using their behavior or by using their source code. Regardless of the intention type, identifying it, evaluating it, and taking actions by using it to prevent any malicious intentions are the main goals of this research. The following four different security vulnerabilities are identified in this research: Malicious apps, spammers and lurkers in social networks, eavesdroppers in phone conversations, and compromised authentication. These four vulnerabilities are solved by detecting malware applications, identifying malicious users in a social network, enhancing the encryption system of a phone communication, and identifying user activities using electroencephalogram (EEG) for authentication. Each of these solutions are constructed using the idea of intention identification. Furthermore, many of these approaches have utilized different machine learning models. The malware detection approach performed with an 89% accuracy in detecting the given malware dataset. In addition, the social network user identification model's accuracy was above 90%. The encryption enhancement reduced the mobile CPU usage time by 40%. Finally, the EEG based user activities were identified with an 85% accuracy. Identifying intention and using it to improve mobile phone security are the main contributions of this dissertation.

Exposure to interparental conflict has been implicated in children's development. Research suggests that underlying mechanisms, such as neuropsychological indicators of cognitive processes, may shed light on how exposure to interparental conflict differentially influences children's outcomes over time. Event-related potentials (ERP), extracted from electroencephalogram data, allow for examination of neuropsychological markers of cognition based on precise timing and scalp topography of electrical activity in the brain. For example, the late positive potential (LPP) ERP component has been implicated in the timing and magnitude of sustained attention and emotion regulation processes elicited in response to emotionally salient stimuli. LPP amplitudes and peak latencies were compared for a community sample of 23 children (9-11 years of age, 12 females) during an oddball task, which used images of couples looking angry, happy, and neutral toward each other. Linear mixed models were used to analyze whether children's perceptions of interparental conflict, and whether they were from high- compared to low-conflict homes, influenced their level of neuropsychological resources directed toward angry compared to happy emotionally-charged interpersonal images. Significant results were found for when children were directed to respond to angry images. Differences emerged in LPP amplitudes for all children in the sample, with the greatest amplitudes produced for happy images compared to neutral and angry images. Regarding conflict exposure and perceptions of conflict, children from homes with greater levels of conflict and children who blamed themselves for conflicts they witnessed between parents produced greater LPP amplitudes when happy trials were presented compared to neutral trials. Finally, females reached their maximum LPP amplitude faster than males for neutral trials compared to angry trials. Results are discussed in terms of the implications for children's processing of interpersonal emotions as it is related to underlying neuropsychological mechanisms for sustained attention and emotion regulation.