This study was performed with the goal of delineating how drivers' interactions with an Automated Driving System were affected by a Driver Monitoring System (DMS), which provided alerts to the driver when he or she became inattentive to the driving environment. There were two specific research questions. The first was centered on addressing how drivers' trust and satisfaction with an Automated Driving System was affected by a DMS. The second was centered on addressing how drivers' abilities to detect changes in the driving environment that required intervention were affected by the presence of a DMS. Data were collected from fifty-six drivers during a test-track experiment with an Automated Driving System prototype that was equipped with a DMS. DMS attention prompt conditions were treated as the independent variable and trust, satisfaction, and driver performance during the experimenter triggered lane drifts were treated as dependent variables. The findings of this investigation suggested that drivers who receive attention prompts from a DMS have lower levels of trust and satisfaction with the Automated Driving System compared to drivers who do not receive attention prompts from a DMS. While the DMS may result in lower levels of trust and satisfaction, the DMS may help drivers detect changes in the driving environment that require attention. Specifically, drivers who received attention prompts after 7 consecutive seconds of inattention were 5 times more likely to react to a lane drift with no alert compared to drivers who did not receive attention prompts at all. / Master of Science

Driving a vehicle involves a series of events, which are related to and evolve with the mental state (such as sleepiness, mental load, and stress) of the driv- er. These states are also identified as causal factors of critical situations that can lead to road accidents and vehicle crashes. These driver impairments need to be detected and predicted in order to reduce critical situations and road accidents. In the past years, physiological signals have become conven- tional measures in driver impairment research. Physiological signals have been applied in various studies to identify different levels of mental load, sleepiness, and stress during driving. This licentiate thesis work has investigated several artificial intelligence algorithms for developing an intelligent system to monitor driver mental state using physiological signals. The research aims to measure sleepiness and mental load using Electroencephalography (EEG). EEG signals, if pro- cessed correctly and efficiently, have potential to facilitate advanced moni- toring of sleepiness, mental load, fatigue, stress etc. However, EEG signals can be contaminated with unwanted signals, i.e., artifacts. These artifacts can lead to serious misinterpretation. Therefore, this work investigates EEG arti- fact handling methods and propose an automated approach for EEG artifact handling. Furthermore, this research has also investigated how several other physiological parameters (Heart Rate (HR) and Heart Rate Variability (HRV) from the Electrocardiogram (ECG), Respiration Rate, Finger Tem- perature (FT), and Skin Conductance (SC)) to quantify drivers’ stress. Dif- ferent signal processing methods have been investigated to extract features from these physiological signals. These features have been extracted in the time domain, in the frequency domain as well as in the joint time-frequency domain using wavelet analysis. Furthermore, data level signal fusion has been proposed using Multivariate Multiscale Entropy (MMSE) analysis by combining five physiological sensor signals. Primarily Case-Based Reason- ing (CBR) has been applied for drivers’ mental state classification, but other Artificial intelligence (AI) techniques such as Fuzzy Logic, Support Vector Machine (SVM) and Artificial Neural Network (ANN) have been investigat- ed as well. For drivers’ stress classification, using the CBR and MMSE approach, the system has achieved 83.33% classification accuracy compared to a human expert. Moreover, three classification algorithms i.e., CBR, an ANN, and a SVM were compared to classify drivers’ stress. The results show that CBR has achieved 80% and 86% accuracy to classify stress using finger tempera- ture and heart rate variability respectively, while ANN and SVM reached an accuracy of less than 80%. / Vehicle Driver Monitoring

Since forward collision is one of the most common and dangerous types of traffic accidents, many studies and researches have been conducted to develop forward collision avoidance system. To facilitate the tradeoff between comfort and safety for forward collision avoidance, the driver's state needs to be monitored and estimated. Such support is necessary for Forward Collision Warning (FCW) system given human-involved control. Due to the advances of Driver Monitoring System (DMS), the demand for camera-based driver's state estimation has increased. This master thesis project, conducted at Zenuity AB, investigates a method to estimate driver's awareness based on DMS. The estimation of a driver's awareness is expected to help adapt FCW system based on visual attention when facing the unpredictable braking of the leading vehicle. The project consists of three tasks: gaze estimation, Gaze-to-Object Mapping (GTOM), and awareness estimation. A combined Kalman Filter was developed in gaze estimation for compensation of missing data and outliers and reducing the difference to “ground truth” data. The uncertainty matrix from gaze estimation was utilized to extract a gaze-to-object probability signal in GTOM, while the corresponding fixation duration was also obtained in GTOM. The two extracted new features were used in awareness estimation with two methods: Logistic Regression and two-Hidden Markov Model. The comparison between the two methods reveals whether a complex method is preferred or not. Based on the results of this project, Logistic Regression seems to perform better in driver's state estimation, with 92.0% accuracy and 76.3% True Negative rate. However, further research and improvements on the two-Hidden Markov Model are needed to reach a more comprehensive conclusion. The main contribution of this project is an investigation of an end-to-end method for driver's awareness estimation and thereby an identification of challenges for further studies. / Frontkollision (forward collision) är en av de vanligaste och farligaste typerna av trafikolyckor. Många studier och undersökningar har genomförts för att utveckla system för att undvika kollisioner. För att underlätta avvägningar mellan komfort och säkerhet för att undvika Frontkollision måste förarens tillstånd övervakas och skattas. Ett sådant stöd är nödvändigt för Forward Collision Warning (FCW) systemet, som involverar interaktion med människor. Efterfrågan på kamerabaserad uppskattning för föraren har ökat på grund av framsteg Driver Monitoring System (DMS). Det här examensarbete genomfördes på Zenuity AB och undersökte en metod för att skatta förarens medvetenhet baserad på Driver Monitoring System. Uppskattningen av förarens medvetenhet förväntas bidra till att anpassa FCW-systemet. Detta FCW-system är baserat på visuell uppmärksamhet om när oförutsägbar bromsning av det framförvarande fordonet sker. Examensarbetet består av tre uppgifter: blickuppskattning, Gaze-to-Object Mapping (GTOM), och medventenhetsuppskattning. Ett kombinerat Kalman-filter har utvecklats i gaze uppskattning för att kompensera saknade data och outliers samt reducera skillnaden till “ground truth” data. Osäkerhetesmatrisen från gaze uppskattningen användes för att extrahera en gaze-to-object sannolikhetssignal i GTOM. Den motsvarande fixeringsvaraktigheten erhålls också i GTOM. De två extraherade nya egenskaperna användes i medvetenhetsanalys med hjälp av två metoder: logistic regression och two-Hidden Markov Model. Jämförelsen mellan de två metoderna avslöjar om en komplex metod är att föredra eller inte. Resultatet av detta examensarbet visar att logistic regression fungerar bättre i förarens statusuppskattning med 92% noggrannhet och 76.3% True Negative rate. Vidare forskning och förbättringar av den two-hidden Markov modell behövs för att dra en mer omfattande slutsats. Det huvudsakliga bidraget av examensarbetet är en utforskning av en end-to-end metod för att uppskatta förarens medvetenhet och därmed kunna identifiera utmaningar för framtid studie.

Thesis (M.S.)--Linköping University, 2004. / Includes bibliographical references (p. 63-64). Also available online via the VTI web site (www.vti.se).

The dissertation investigates the promises and challenges for application of cardiac signals in biometrics and affective computing, and noninvasive measurement of cardiac signals. We have mainly discussed two major cardiac signals: electrocardiogram (ECG), and photoplethysmogram (PPG). ECG and PPG signals hold strong potential for biometric authentications and identifications. We have shown that by mapping each cardiac beat from time domain to an angular domain using a limit cycle, intra-class variability can be significantly minimized. This is in contrary to conventional time domain analysis. Our experiments with both ECG and PPG signal shows that the proposed method eliminates the effect of instantaneous heart rate on the shape morphology and improves authentication accuracy. For noninvasive measurement of PPG beats, we have developed a systematic algorithm to extract pulse rate from face video in diverse situations using video magnification. We have extracted signals from skin patches and then used frequency domain correlation to filter out non-cardiac signals. We have developed a novel entropy based method to automatically select skin patches from face. We report beat-to-beat accuracy of remote PPG (rPPG) in comparison to conventional average heart rate. The beat-to-beat accuracy is required for applications related to heart rate variability (HRV) and affective computing. The algorithm has been tested on two datasets, one with static illumination condition and the other with unrestricted ambient illumination condition. Automatic skin detection is an intermediate step for rPPG. Existing methods always depend on color information to detect human skin. We have developed a novel standalone skin detection method to show that it is not necessary to have color cues for skin detection. We have used LBP lacunarity based micro-textures features and a region growing algorithm to find skin pixels in an image. Our experiment shows that the proposed method is applicable universally to any image including near infra-red images. This finding helps to extend the domain of many application including rPPG. To the best of our knowledge, this is first such method that is independent of color cues. / Ph. D.