Certain electrophysiological markers hold promise in distinguishing individuals with major depressive disorder (MDD) and in predicting antidepressant response, thereby assisting with assessment and optimizing treatment, respectively. This thesis examined resting brain activity via electroencephalographic (EEG) recordings, as well as EEG-derived event-related potentials (ERPs) to auditory stimuli and facial expression presentations in individuals with MDD and controls. Additionally, the utility of resting EEG as well as auditory ERPs (AEPs), and the associated loudness-dependence of AEPs (LDAEP) slope, were assessed in predicating outcome to chronic treatment with one of three antidepressant regimens [escitalopram (ESC); bupropion (BUP); ESC+BUP]. Relative to controls, depressed adults had lower pretreatment cortical activity in regions implicated in approach motives/positive processing. Increased anterior cingulate cortex (ACC)-localized theta was observed, possibly reflecting emotion/cognitive regulation disturbances in the disorder. AEPs and LDAEPs, putative indices of serotonin activity (implicated in MDD etiology), were largely unaltered in MDD. Assessment of ERPs to facial expression processing indicated slightly blunted late preconscious perceptual processing of expressions, and prolonged processing of intensely sad faces in MDD. Faces were rated as sadder overall in MDD, indicating a negative processing bias. Treatment responders (vs. non-responders) exhibited baseline cortical hypoactivity; after a week of treatment, cortical arousal emerged in responders. Increased baseline left fronto-cortical activity and early shifts towards this profile were noted in responders (vs. non-responders). Responders exhibited a steep, and non-responders shallow, baseline N1 LDAEP derived from primary auditory cortex activity. P2 LDAEP slopes (primary auditory cortex-derived) increased after a week of treatment in responders and decreased in non-responders. Consistent with overall findings, ESC responders displayed baseline cortical hypoactivity and steep LDAEP-sLORETA slopes (vs. non-responders). BUP responders also exhibited steep baseline slopes and high ACC theta. These results indicate that specific resting brain activity profiles appear to distinguish depressed from non-depressed individuals. Subtle ERP modulations to simple auditory and emotive processing also existed in MDD. Resting alpha power, ACC theta activity and LDAEP slopes predicted antidepressant response in general, but were limited in predicting outcome to a particular treatment, which may be associated with limited sample sizes.

An EEG-based brain-computer system can be used to control external devices such as computers, wheelchairs or Virtual Environments. One of the most important applications is a spelling device to aid severely disabled individuals with communication, for example people disabled by amyotrophic lateral sclerosis (ALS). P300-based BCI systems are optimal for spelling characters with high speed and accuracy, as compared to other BCI paradigms such as motor imagery. In this study, 100 subjects tested a P300-based BCI system to spell a 5-character word with only 5 min of training. EEG data were acquired while the subject looked at a 36-character matrix to spell the word WATER. Two different versions of the P300 speller were used: (i) the row/column speller (RC) that flashes an entire column or row of characters and (ii) a single character speller (SC) that flashes each character individually. The subjects were free to decide which version to test. Nineteen subjects opted to test both versions. The BCI system classifier was trained on the data collected for the word WATER. During the real-time phase of the experiment, the subject spelled the word LUCAS, and was provided with the classifier selection accuracy after each of the five letters. Additionally, subjects filled out a questionnaire about age, sex, education, sleep duration, working duration, cigarette consumption, coffee consumption, and level of disturbance that the flashing characters produced. 72.8% (N = 81) of the subjects were able to spell with 100% accuracy in the RC paradigm and 55.3% (N = 38) of the subjects spelled with 100% accuracy in the SC paradigm. Less than 3% of the subjects did not spell any character correctly. People who slept less than 8 h performed significantly better than other subjects. Sex, education, working duration, and cigarette and coffee consumption were not statistically related to differences in accuracy. The disturbance of the flashing characters was rated with a median score of 1 on a scale from 1 to 5 (1, not disturbing; 5, highly disturbing). This study shows that high spelling accuracy can be achieved with the P300 BCI system using approximately 5 min of training data for a large number of non-disabled subjects, and that the RC paradigm is superior to the SC paradigm. 89% of the 81 RC subjects were able to spell with accuracy 80-100%. A similar study using a motor imagery BCI with 99 subjects showed that only 19% of the subjects were able to achieve accuracy of 80-100%. These large differences in accuracy suggest that with limited amounts of training data the P300-based BCI is superior to the motor imagery BCI. Overall, these results are very encouraging and a similar study should be conducted with subjects who have ALS to determine if their accuracy levels are similar.

The use of electroencephalogram (EEG), an electrophysiological monitoring method for recording the brain activity, for authentication has attracted the interest of researchers for over a decade. In addition to exhibiting qualities of biometric-based authentication, they are revocable, impossible to mimic, and resistant to coercion attacks. However, EEG signals carry a wealth of information about an individual and can reveal private information about the user. This brings significant privacy issues to EEG-based authentication systems as they have access to raw EEG signals. This thesis proposes a privacy-preserving EEG-based authentication system that preserves the privacy of the user by not revealing the raw EEG signals while allowing the system to authenticate the user accurately. In that, perceptual hashing is utilized and instead of raw EEG signals, their perceptually hashed values are used in the authentication process. In addition to describing the authentication process, algorithms to compute the perceptual hash are developed based on two feature extraction techniques. Experimental results show that an authentication system using perceptual hashing can achieve performance comparable to a system that has access to raw EEG signals if enough EEG channels are used in the process. This thesis also presents a security analysis to show that perceptual hashing can prevent information leakage.

京都大学 / 0048 / 新制・課程博士 / 博士(工学) / 甲第18280号 / 工博第3872号 / 新制||工||1594(附属図書館) / 31138 / 京都大学大学院工学研究科電気工学専攻 / (主査)教授 小林 哲生, 教授 中村 裕一, 准教授 古谷 栄光 / 学位規則第4条第1項該当 / Doctor of Philosophy (Engineering) / Kyoto University / DFAM

Augmented Reality (AR) head-mounted display (HMD) provides users with an immersive virtual experience in the real world. The portability of this technology affords various information display options for construction workers that are not possible otherwise. The information delivered via an interactive user interface provides an innovative method to display complex building instructions, which is more intuitive and accessible compared with traditional paper documentations. However, there are still challenges hindering the practical usage of this technology at the construction jobsite. As a technical restriction, current AR HMD products have a limited field of view (FOV) compared to the human vision range. It leads to an uncertainty of how the obstructed view of display will affect construction workers' perception of hazards in their surrounding area. Similarly, the information displayed to workers requires rigorous testing and evaluation to make sure that it does not lead to information overload. Therefore, it is essential to comprehensively evaluate the impacts of using AR HMD from both perspectives of task performance and cognitive performance. This dissertation aims to bridge the gap in understanding the cognitive impacts of using AR HMD in construction assembly tasks. Specifically, it focuses on answering the following two questions: (1) How are task performance and cognitive skills affected by AR displays under complex working conditions? (2) How are moment-to-moment changes of mental workload captured and evaluated during construction assembly tasks? To answer these questions, this dissertation proposed two experiments. The first study tests two AR displays (conformal and tag-along) and paper instruction under complex working conditions, involving different framing scales and interference settings. Subjective responses are collected and analyzed to evaluate overall mental workload and situation awareness. The second study focuses on exploring an electroencephalogram (EEG) based approach for moment-to-moment capture and evaluation of mental workload. It uncovers the cognitive change on the time domain and provides room for further quantitative analyzing on mental workload. Especially, two frameworks of mental workload prediction are proposed by using (1) Long Short-Term Memory (LSTM) and (2) one-dimensional Convolutional Neural Network (1D CNN)-LSTM for forecasting EEG signal and, classifying task conditions and mental workload levels respectively. The approaches are tested to be effective and reliable for predicting and recognizing subjects' mental workload during assembly. In brief, this research contributes to the existing knowledge with an assessment of AR HMD use in construction assembly, including task performance evaluation and both subjective and physiological measurements for cognitive skills. / Doctor of Philosophy / Augmented Reality (AR) is an emerging technology that bridges the gap between virtual creatures and physical world with an immersive display experience. Today, head-mounted display (HMD) is well developed to meet the demands for portable AR devices. It provides interactive and intuitive display of 2D graphical information to make it easier to understand for users. Therefore, AR display has been studied in the past few years for a more simplified and productive construction assembly process. However, given the premise that construction is a high-risk industry, introducing such display technology to the jobsite needs to be carefully tested. One obstacle in current AR HMD products is the restriction of field of view (FOV), which may block users' view in presenting large-scale 3D objects. In construction assembly, workers need to deal with tasks in different scopes, such as wood framing for a residential house. Consequently, it is necessary to study how such technical challenge will impact workers' performance under different task conditions. Another concern comes from the mental perspective. Although AR display may bring convenience in acquiring effective information, it is difficult to measure if this generates excessive mental burden to users. Especially for construction workers, whether the overlaid display will cause distraction and information overload is crucial for protecting workers from hazards. To address the problems, this dissertation explores the gap in previous literature, where mental workload is not well studied for using AR HMD in construction assembly. Two experiments are conducted to comprehensively evaluate the impacts of AR displays on both assembly performance and users' mental status. The outcomes bring implications to theoretical and practical aspects. First, it compares two AR displays (2D tag-along image and 3D conformal model) with traditional paper documentation for assembly performance (efficiency and accuracy) and users' cognitive skills (mental workload and situation awareness). The findings revealed the impact of FOV restriction and provided a strategic solution to selecting display method for different task conditions. Second, it proposes a physiological approach to calculate mental workload from analyzing the features from brain waves. It uncovered the latent mental changes during the assembly. Furthermore, two deep learning approaches are applied to predict and classify mental workload. The prediction model depicted the trend of mental workload in eighteen seconds based on an eighty-four-second training set, while the classifier recognized two task conditions with different mental workload levels with an accuracy of 93.6%. The results have promising potential for future research in detecting and preventing abnormality in workers' mental status. In addition, it is generalizable to apply in other construction tasks and AR applications.

No description available.

abstract: In this research work, a novel control system strategy for the robust control of an unmanned ground vehicle is proposed. This strategy is motivated by efforts to mitigate the problem for scenarios in which the human operator is unable to properly communicate with the vehicle. This novel control system strategy consisted of three major components: I.) Two independent intelligent controllers, II.) An intelligent navigation system, and III.) An intelligent controller tuning unit. The inner workings of the first two components are based off the Brain Emotional Learning (BEL), which is a mathematical model of the Amygdala-Orbitofrontal, a region in mammalians brain known to be responsible for emotional learning. Simulation results demonstrated the implementation of the BEL model to be very robust, efficient, and adaptable to dynamical changes in its application as controller and as a sensor fusion filter for an unmanned ground vehicle. These results were obtained with significantly less computational cost when compared to traditional methods for control and sensor fusion. For the intelligent controller tuning unit, the implementation of a human emotion recognition system was investigated. This system was utilized for the classification of driving behavior. Results from experiments showed that the affective states of the driver are accurately captured. However, the driver's affective state is not a good indicator of the driver's driving behavior. As a result, an alternative method for classifying driving behavior from the driver's brain activity was explored. This method proved to be successful at classifying the driver's behavior. It obtained results comparable to the common approach through vehicle parameters. This alternative approach has the advantage of directly classifying driving behavior from the driver, which is of particular use in UGV domain because the operator's information is readily available. The classified driving mode was used tune the controllers' performance to a desired mode of operation. Such qualities are required for a contingency control system that would allow the vehicle to operate with no operator inputs. / Dissertation/Thesis / Doctoral Dissertation Engineering 2015

abstract: The human brain controls a person's actions and reactions. In this study, the main objective is to quantify reaction time towards a change of visual event and figuring out the inherent relationship between response time and corresponding brain activities. Furthermore, which parts of the human brain are responsible for the reaction time is also of interest. As electroencephalogram (EEG) signals are proportional to the change of brain functionalities with time, EEG signals from different locations of the brain are used as indicators of brain activities. As the different channels are from different parts of our brain, identifying most relevant channels can provide the idea of responsible brain locations. In this study, response time is estimated using EEG signal features from time, frequency and time-frequency domain. Regression-based estimation using the full data-set results in RMSE (Root Mean Square Error) of 99.5 milliseconds and a correlation value of 0.57. However, the addition of non-EEG features with the existing features gives RMSE of 101.7 ms and a correlation value of 0.58. Using the same analysis with a custom data-set provides RMSE of 135.7 milliseconds and a correlation value of 0.69. Classification-based estimation provides 79% & 72% of accuracy for binary and 3-class classication respectively. Classification of extremes (high-low) results in 95% of accuracy. Combining recursive feature elimination, tree-based feature importance, and mutual feature information method, important channels, and features are isolated based on the best result. As human response time is not solely dependent on brain activities, it requires additional information about the subject to improve the reaction time estimation. / Dissertation/Thesis / Masters Thesis Electrical Engineering 2019

No description available.

<p>  </p> <p>Hazard identification is one of the most significant components in safety management at construction jobsites to prevent undesired fatalities and injuries of construction workers. The current practice, which relies on a limited number of safety managers’ manual and subjective inspections, and existing research efforts analyzing workers’ physical and physiological signals have achieved limited success, leaving many hazards unidentified at the jobsites. Motivated by this critical need, this research aims to develop a human sensing network that allows for ubiquitous hazard identification in the construction workplace.</p> <p>To attain this overarching goal, this research analyzes construction workers’ collective EEG signals collected from wearable EEG sensors based on machine learning, virtual reality (VR), and advanced signal processing techniques. Three specific research objectives are: (1) establishing a relationship between EEG signals and the existence of construction hazards, (2) identifying correlations between EEG signals/physiological states (e.g., emotion) and different hazard types, and (3) developing an integrated platform for real-time construction hazard mapping and comparing the results developed based on VR and real-world experimental settings.</p> <p>Specifically, the first objective establishes the relationship by investigating the feasibility of identifying construction hazards using a binary EEG classifier developed in VR, which can capture EEG signals associated with perceived hazards. In the second objective, correlations are discovered by testing the feasibility of differentiating construction hazard types based on a multi-class classifier constructed in VR. In the first and second objectives, the complex relationships are also analyzed in terms of brain dynamics and EEG signal components. In the third objective, the platform is developed by fusing EEG signals with heterogeneous data (e.g., location), and the discrepancies in VR and real-world environments are quantitatively assessed in terms of hazard identification performance and human behavioral responses.</p> <p>The primary outcome of this research is that the proposed approach can be applied to actual construction jobsites and used to detect all potential hazards, which was challenging to be achieved based on the current practice and existing research efforts. Also, the human cognitive mechanisms revealed in this research discover new neurocognitive knowledge in construction workers’ hazard perception. As a result, this research contributes to enhancing current hazard identification capability and improving construction workers’ safety and health.</p>