This study was undertaken to simulator test and evaluate a complete drowsy driver detection system. The goal of the study was to recommend optimal specifications for a system to be further studied in an actual vehicle. The system used a set of algorithms developed from previously collected data and a set of previously optimized advisory tones, advisory messages, alarm stimuli, and drowsiness countermeasures. Detection occurred if eye closure or lane excursion exceeded predetermined thresholds. Data were obtained from six sleep-deprived subjects who drove a motion base automobile simulator late at night. Each subject was trained in carefully observing lane boundaries, using a device which sounded an alarm if lane boundaries were exceeded. The performance aspect of the system dominated the detection process. None of the algorithms tracked well with the measures they were designed to estimate; correlations were much lower than expected. The algorithms relied heavily on the positioning of the vehicle relative to the lane. / Master of Science

Conselho Nacional de Desenvolvimento Científico e Tecnológico / Many fatal traffic accidents are caused by fatigued and drowsy drivers. In this context, automatic drowsiness detection devices are an alternative to minimize this issue. In this work, two new methodologies to drowsiness detection are presented, considering a signal obtained from a single electroencephalography channel: (i) drowsiness detection through best m-term approximation, applied to the wavelet expansion of the analysed signal; (ii) drowsiness detection through Mahalanobis distance with wavelet coefficients. The results of both methodologies are compared with a method which uses Mahalanobis distance and Fourier coefficients to drowsiness detection. All methodologies consider the medical evaluation of the brain signal, given by the hypnogram, as a reference. / A sonolência diurna em motoristas, principal consequência da privação de sono, tem sido a causa de diversos acidentes graves de trânsito. Neste contexto, a utilização de dispositivos que alertem o condutor ao detectar automaticamente o estado de sonolência é uma alternativa para a minimização deste problema. Neste trabalho, duas novas metodologias para a detecção automática da sonolência são apresentadas, utilizando um único canal de eletroencefalografia para a obtenção do sinal: (i) detecção da sonolência via melhor aproximação por m-termos, aplicada aos coeficientes wavelets da expansão em série do sinal; e (ii) detecção da sonolência via distância de Mahalanobis e coeficientes wavelets. Os resultados de ambas as metodologias são comparados a uma implementação utilizando distância de Mahalanobis e coeficientes de Fourier. Para todas as metodologias, utiliza-se como referência a avaliação médica do sinal cerebral, dada pelo hipnograma.

Driving while drowsy is a major cause behind road accidents, and exposes the driver to a much higher crash risk compared to driving while alert. Therefore, the use of assistive systems that monitor a driver’s level of vigilance and alert the fatigue driver can be significant in the prevention of accidents. This thesis introduces three different methods towards the detection of drivers’ drowsiness based on yawning measurement. All three approaches involve several steps, including the real time detection of the driver’s face, mouth and yawning. The last approach, which is the most accurate, is based on the Viola-Jones theory for face and mouth detection and the back projection theory for measuring both the rate and the amount of changes in the mouth for yawning detection. Test results demonstrate that the proposed system can efficiently measure the aforementioned parameters and detect the yawning state as a sign of a driver’s drowsiness.

In the human control of motor vehicles, there are situations regularly encountered wherein the vehicle operator becomes drowsy and fatigued due to the influence of long work days, long driving hours, or low amounts of sleep. Although various methods are currently proposed to detect drowsiness in the operator, they are either obtrusive, expensive, or otherwise impractical. The method of drowsy driving detection through the collection of Steering Wheel Movement (SWM) signals has become an important measure as it lends itself to accurate, effective, and cost-effective drowsiness detection. In this dissertation, novel technologies for drowsiness detection using Inertial Measurement Units (IMUs) are investigated and described. IMUs are an umbrella group of kinetic sensors (including accelerometers and gyroscopes) which transduce physical motions into data. Driving performances were recorded using IMUs as the primary sensors, and the resulting data were used by artificial intelligence algorithms, specifically Support Vector Machines (SVMs) to determine whether or not the individual was still fit to operate a motor vehicle. Results demonstrated high accuracy of the method in classifying drowsiness. It was also shown that the use of a smartphone-based approach to IMU monitoring of drowsiness will result in the initiation of feedback mechanisms upon a positive detection of drowsiness. These feedback mechanisms are intended to notify the driver of their drowsy state, and to dissuade further driving which could lead to crashes and/or fatalities. The novel methods not only demonstrated the ability to qualitatively determine a drivers drowsy state, but they were also low-cost, easy to implement, and unobtrusive to drivers. The efficacy, ease of use, and ease of access to these methods could potentially eliminate many barriers to the implementation of the technologies. Ultimately, it is hoped that these findings will help enhance traveler safety and prevent deaths and injuries to users.

Studies have reported that around 20 % of all traffic accidents are caused by a sleepy driver. Sleepy driving has been compared to drunk driving. A sleepy driver is also an issue in the case of automated vehicles in the future. Handing back the control of the vehicle to a sleepy driver is a serious risk. This has increased the need for a sleepiness estimation system that can be used in the car to warn the driver when driving is not recommended. One commonly used method to estimate sleepiness is to study the heart rate variability, HRV, which is said to reflect the activity of the autonomous nervous system, the ANS. The HRV can be expressed through different measures obtained from a signal of RR-intervals. The aim with the thesis is to investigate how well the HRV translates into sleepiness estimation and how the experimental setup might affect the results. In this study, HRV data from 85 sleep deprived drivers was collected together with the drivers’ own ratings of their sleepiness according to the nine graded Karolinska sleepiness scale, KSS. An ANOVA test showed statistical significance for almost all of the used HRV measures when the Driver ID was set as random variable. In order to reduce the number of HRV measures, a feature selection step was performed before training a Support Vector Machine (SVM) used for classification of the data. SVM classifiers are trained to use the input features describing the data to optimize hyperplanes separating the discrete set of classes. Previous research has shown good results in using HRV for sleepiness detection, but common issues are the small data sets used and that most experiments are performed in a simulator instead of at real roads. In some cases, no sleep deprivation is used. The result from the classification in this study is a mean accuracy of around 58-59 %, mean sensitivity of 50-51 %, mean specificity of 75-76 % and mean F1 score of 50-51 % over the three classes ’Alert’, ’Getting sleepy’ and ’Sleepy’. This together with the results of the ANOVA test shows that the HRV measures performed relatively poor when used for classification of the data and that there are large inter-individual differences. This suggests the use of personalized algorithms when developing a sleepiness estimation system and an investigation regarding how other confounding factors could affect the estimation is also motivated.

Driving while drowsy is a major cause behind road accidents, and exposes the driver to a much higher crash risk compared to driving while alert. Therefore, the use of assistive systems that monitor a driver’s level of vigilance and alert the fatigue driver can be significant in the prevention of accidents. This thesis introduces three different methods towards the detection of drivers’ drowsiness based on yawning measurement. All three approaches involve several steps, including the real time detection of the driver’s face, mouth and yawning. The last approach, which is the most accurate, is based on the Viola-Jones theory for face and mouth detection and the back projection theory for measuring both the rate and the amount of changes in the mouth for yawning detection. Test results demonstrate that the proposed system can efficiently measure the aforementioned parameters and detect the yawning state as a sign of a driver’s drowsiness.

Electro-oculography signals are corneo-retinal potentials that carry informationpertaining to eye movements. This information can be used to estimate drowsinesslevel of the subject which could provide interesting insights into research of acci-dent prevention. Of all features present, blink duration has been proved to be aneffective measure of drowsiness. The aim of this thesis work is to build a portablesystem to acquire and analyze electro-oculographic (EOG) signals in real-time.The system contains two sub-systems; a hardware sub-system that consists of thefilters, amplifiers, data acquisition card and isolation and the software sub-systemthat contains the program to acquire and analyze the signal and present the resultsto the observer. The filters were designed starting with simulation, implementa-tion on the prototype board, culminating in the design of a printed circuit board(PCB) and packaging. The complete software was written in PythonTMusing sev-eral relevant libraries for data processing. A text-based user interface was createdto enable easy user interaction. The results are graphically displayed in real-time. Ex-situ tests were done with two volunteers while in-situ test was done onone subject. The data from the in-situ tests showed "good signal quality" in a"noisy" environment concurring with the design specifications. To motivate theimportance of calibration, two calibration paradigms were used during ex-situtests, where one paradigm records only normal blinks while the other records longblinks and the results showed differences in detection and error rates. The obser-vations made from performance tests at various levels gave "satisfactory results"and proved the usefulness of the system for experimental purposes in-situ.

Thesis (M.S.)--Linköping University, 2004. / Includes bibliographical references (p. 63-64). Also available online via the VTI web site (www.vti.se).

Les exigences opérationnelles du métier de pilote sont susceptibles d'engendrer de la somnolence et des niveaux de charge mentale inadéquats (i.e., trop faible ou trop élevé) au cours des vols. Les dettes de sommeil et les perturbations circadiennes liées à divers facteurs (e.g., longues périodes de services, horaires de travail irrégulier, etc.) demandent aux pilotes de repousser sans cesse leurs limites biologiques. Par ailleurs, la charge de travail mental des pilotes présente de fortes variations au cours d'un vol : élevée au cours des phases critiques (i.e., décollage et atterrissage), elle devient très réduite pendant les phases de croisière. Lorsque la charge mentale devient trop élevée ou, à l'inverse, trop faible, les performances se dégradent et des erreurs de pilotage peuvent apparaître. La mise en oeuvre de méthodes de détection de l'état de somnolence et du niveau de charge mentale en temps quasi réel est un défi majeur pour le suivi et le contrôle de l'activité de pilotage. L'objectif de la thèse est de déterminer si la voix humaine peut permettre de détecter d'une part, l'état de somnolence et d'autre part, le niveau de charge mentale d'un individu. Dans une première étude, la voix de participants a été enregistrée lors d'une tâche de lecture avant et après une nuit de privation totale de sommeil (PTS). Les variations de l'état de somnolence consécutives à la PTS ont été évaluées au moyen de mesures auto-évaluatives et électrophysiologiques (ÉlectroEncéphaloGraphie [EEG] et Potentiels Évoqués [PEs]). Les résultats ont montré une variation significative après la PTS de plusieurs paramètres acoustiques liés : (a) à l'amplitude des impulsions glottiques (fréquence de modulation d'amplitude), (b) à la forme du signal acoustique (longueur euclidienne du signal et ses caractéristiques associées) et (c) au spectre du signal des voyelles (rapport harmonique sur bruit, fréquence du second formant, coefficient d'asymétrie, centre de gravité spectral, différences d'énergie, pente spectrale et coefficients cepstraux à échelle Mel). La plupart des caractéristiques spectrales ont montré une sensibilité différente à la privation de sommeil en fonction du type de voyelles. Des corrélations significatives ont été mises en évidence entre plusieurs paramètres acoustiques et plusieurs indicateurs objectifs (EEG et PEs) de l'état de somnolence. Dans une seconde étude, le signal vocal a été enregistré durant une tâche de rappel de listes de mots. La difficulté de la tâche était manipulée en faisant varier le nombre de mots dans chaque liste (i.e., entre un et sept, correspondant à sept conditions de charge mentale). Le diamètre pupillaire - qui est un indicateur objectif pertinent du niveau de charge mentale - a été mesuré simultanément avec l'enregistrement de la voix afin d'attester de la variation du niveau de charge mentale durant la tâche expérimentale. Les résultats ont montré que des paramètres acoustiques classiques (fréquence fondamentale et son écart type, shimmer, nombre de périodes et rapport harmonique sur bruit) et originaux (fréquence de modulation d'amplitude et variations à court-terme de la longueur euclidienne du signal) ont été particulièrement sensibles aux variations de la charge mentale. Les variations de ces paramètres acoustiques étaient corrélées à celles du diamètre pupillaire. L'ensemble des résultats suggère que les paramètres acoustiques de la voix humaine identifiés lors des expérimentations pourraient représenter des indicateurs pertinents pour la détection de l'état de somnolence et du niveau de charge mentale d'un individu. Les résultats ouvrent de nombreuses perspectives de recherche et d'applications dans le domaine de la sécurité des transports, notamment dans le secteur aéronautique. / Operational requirements of aircraft pilots may cause drowsiness and inadequate mental load levels (i.e., too low or too high) during flights. Sleep debts and circadian disruptions linked to various factors (e.g., long working periods, irregular work schedules, etc.) require pilots to challenge their biological limits. Moreover, pilots' mental workload exhibits strong fluctuations during flights: higher during critical phases (i.e., takeoff and landing), it becomes very low during cruising phases. When the mental load becomes too high or, conversely, too low, performance decreases and flight errors may manifest. Implementation of detection methods of drowsiness and mental load levels in near real time is a major challenge for monitoring and controlling flight activity. The aim of this thesis is therefore to determine if the human voice can serve to detect on one hand the drowsiness and on the other hand the mental load level of an individual. In a first study, the voice of participants was recorded during a reading task before and after a night of total sleep deprivation (TSD). Drowsiness variations linked to TSD were assessed using self-evaluative and electrophysiological measures (ElectroEncephaloGraphy [EEG] and Evoked Potentials [EPs]). Results showed significant variations after the TSD in many acoustic features related to: (a) the amplitude of the glottal pulses (amplitude modulation frequency), (b) the shape of the acoustic wave (Euclidean length of the signal and its associated features) and (3) the spectrum of the vowel signal (harmonic-to-noise ratio, second formant frequency, skewness, spectral center of gravity, energy differences, spectral tilt and Mel-frequency cepstral coefficients). Most spectral features showed different sensitivity to sleep deprivation depending on the vowel type. Significant correlations were found between several acoustic features and several objective indicators (EEG and PEs) of drowsiness. In a second study, voices were recorded during a task featuring word-list recall. The difficulty of the task was manipulated by varying the number of words in each list (i.e., between one and seven, corresponding to seven mental load conditions). Evoked pupillary response - known to be a useful proxy of mental load - was recorded simultaneously with speech to attest variations in mental load level during the experimental task. Results showed that classical features (fundamental frequency and its standard deviation, shimmer, number of periods and harmonic-to-noise ratio) and original features (amplitude modulation frequency and short-term variation in digital amplitude length) were particularly sensitive to variations in mental load. Variations in these acoustic features were correlated to those of the pupil size. Results suggest that the acoustic features of the human voice identified during these experiments could represent relevant indicators for the detection of drowsiness and mental load levels of an individual. Findings open up many research and applications perspectives in the field of transport safety, particularly in the aeronautical sector.

Road traffic accidents are a serious burden to the health systems of many countries especially in South Africa. Research aimed at reducing traffic related accidents is of importance as traffic crashes are rated as the second leading cause of fatalities in South Africa and ninth in the world. Despite the extensive efforts into research and development of new technology, driver fatigue still remains a cause of vehicle accidents worldwide. Fatigue plays a role in up to 20% of vehicle accidents with many being serious or fatal. Numerous coping behaviours are employed by drivers to counteract the negative effects of fatigue. The most common coping behaviours include taking short naps, talking to passengers, listening to the radio, opening windows and drinking stimulants. Driving breaks have long been identified as an effective countermeasure against fatigue. Most research done in driving breaks has investigated the duration of the breaks, activity undertaken during the break and the frequency of the breaks taken outside the vehicle. However limited literature is available on the effectiveness of breaks in counteracting the effects of fatigue. The objective of the current study was aimed at assessing whether short duration breaks are an effective countermeasure against fatigue. Physiological, neurophysiological, subjective and performance measures were used as indicators for fatigue. Additional focus of the research was determining whether breaks were more or less effective at counteracting the effects of fatigue at different times of day. Twelve participants were recruited for the study, six males and six females. The participants were required to perform a driving task on a simulator for 90 minutes. The study consisted of four independent conditions, namely driving during the day with breaks, driving during the day without breaks, driving during the night with breaks and driving during the night without breaks. The without breaks conditions were similar except that they occurred at different times of the day, one session at night and the other session during day time, as was the case for the conditions with breaks. The driving task used in the current study was a low fidelity simulator tracking task. The participants were required to follow a centre line displayed on a tracking path as accurately as possible. The measurements that were recorded in this study included physiological, performance, subjective and neurophysiological. Physiological measures included heart rate and heart rate variability (frequency domain) and core body temperature. The ascending threshold of the critical flicker fusion frequency was the only neurophysiological measurement included in the current investigation. Performance was quantified by mean deviation from a centre line participants were meant to track. Two rating scales were used: Karolinska sleepiness scale and the Wits sleepiness scale were used for the measurement of subjective sleepiness. Heart rate, heart rate variability and mean deviation were measured continuously throughout the 90 minute driving task. Critical flicker fusion frequency, temperature and the subjective scales were measured before and after the 90 minute driving task. The results indicated that the short duration breaks during day time had a positive effect on driving performance; however the breaks at night had a negative effect on driving performance. Heart rate was higher during the day compared to night time and the heart rate variability high frequency spectrum values were lower during the day condition, to show the activation of the sympathetic nervous system which is characteristic of day time. The night conditions had lower heart rate values and higher heart rate variability high frequency values, which show the activation of the parasympathetic nervous system which is dominant during periods of fatigue and night time. Subjective sleepiness levels were also higher at night compared to day time.