<p>As more and more people conduct vigilance-based activities at times other than the traditional daytime work hours, the time utilization will continue to escalate in the next century and will further increase the risks of sleepiness-related accidents.</p><p>This project, which is commissioned by Scania CV AB, is to nvestigate the potential of a method for sleepiness detection belonging to esium AB. Our objective is to examine whether Scania CV AB should continue with the investigation of the patent method, and in that case, which patent parameters, that indicate sleepiness, should be more closely inquired. The purpose with the method of patent is to discover a sleepy driving behaviour. This method is based on the interaction that appears between the driver and the vehicle. The interaction consists of small spontaneous corrections with the steering wheel that in this report is called micro communication. How well the interaction is functioning can be measured in degree of interaction, which shows how well the driver and the truck interact with each other. The interaction between the driver and the vehicle is in this report looked upon as answers and questions with a certain reaction time, which appears with a certain answered question frequency. The differences in the signal’s amplitudes are measured in variation in amplitudes.</p><p>Experiments to collect relevant signals have to be conducted in order to investigate the potential with the method of the patent. It is eligible to collect data from a person falling asleep, which implies experiments conducted in a simulator. The experiments are executed in</p><p>a simulator, one test when they are alert and one when they are sleep deprived. Tests are also executed in a Scania truck. The purpose with these experiments is to collect data of the subject’s normal driving pattern in a truck and to investigate if it is possible to obtain</p><p>acceptable data in a truck.</p><p>The sleepiness experiments have indicated that the micro communication takes place in a frequency range of 0.25 to 6.0 Hz. The variables that have been found to detect sleepiness with high reliability are the reaction time and the degree of interaction presented in spectra.</p><p>The validation experiments have shown it is possible to collect exact and accurate data from the lateral acceleration and the steering wheel torque. But, there is more noise in the signals from truck then there is in the signals from the simulator.</p><p>This method for sleepiness detection has, according to the authors, a great potential. However, more experiments have to be conducted. The authors suggest further sleepiness experiments only conducted during night time. The subjects are sufficiently alert in the beginning of the test to receive data from normal driving behaviour. Physiological measurement could be interesting to have by the side of the subjective assessments as an additional base for comparison.</p>

As more and more people conduct vigilance-based activities at times other than the traditional daytime work hours, the time utilization will continue to escalate in the next century and will further increase the risks of sleepiness-related accidents. This project, which is commissioned by Scania CV AB, is to nvestigate the potential of a method for sleepiness detection belonging to esium AB. Our objective is to examine whether Scania CV AB should continue with the investigation of the patent method, and in that case, which patent parameters, that indicate sleepiness, should be more closely inquired. The purpose with the method of patent is to discover a sleepy driving behaviour. This method is based on the interaction that appears between the driver and the vehicle. The interaction consists of small spontaneous corrections with the steering wheel that in this report is called micro communication. How well the interaction is functioning can be measured in degree of interaction, which shows how well the driver and the truck interact with each other. The interaction between the driver and the vehicle is in this report looked upon as answers and questions with a certain reaction time, which appears with a certain answered question frequency. The differences in the signal’s amplitudes are measured in variation in amplitudes. Experiments to collect relevant signals have to be conducted in order to investigate the potential with the method of the patent. It is eligible to collect data from a person falling asleep, which implies experiments conducted in a simulator. The experiments are executed in a simulator, one test when they are alert and one when they are sleep deprived. Tests are also executed in a Scania truck. The purpose with these experiments is to collect data of the subject’s normal driving pattern in a truck and to investigate if it is possible to obtain acceptable data in a truck. The sleepiness experiments have indicated that the micro communication takes place in a frequency range of 0.25 to 6.0 Hz. The variables that have been found to detect sleepiness with high reliability are the reaction time and the degree of interaction presented in spectra. The validation experiments have shown it is possible to collect exact and accurate data from the lateral acceleration and the steering wheel torque. But, there is more noise in the signals from truck then there is in the signals from the simulator. This method for sleepiness detection has, according to the authors, a great potential. However, more experiments have to be conducted. The authors suggest further sleepiness experiments only conducted during night time. The subjects are sufficiently alert in the beginning of the test to receive data from normal driving behaviour. Physiological measurement could be interesting to have by the side of the subjective assessments as an additional base for comparison.

Driver sleepiness contributes to a large amount of all road traffic crashes. Developing an objective measurement of driver sleepiness in order to prevent eventual traffic accidents is desirable. The aim of this master thesis was to investigate if deep learning can be used to provide a driver sleepiness classification from brain activity signals obtained by electroencephalography (EEG). The intention was to study the classification performance when using different representations of the input data and to examine how various deep neural network architectures and class weighting during training affect the classification.  The data was collected from 12 experiments, where 269 participants (1187 driving sessions) were driving either on real roads or in a moving-base driving simulator, while electrophysiological data was recorded. Several deep neural network architectures were developed, depending on the representation of the input data.  Regardless of which data representation that was used as input to the network, the datawas divided into three datasets: Training 60%, validation 20% and test 20%. The data from each participant, with associated driving sessions, were randomly assigned to the different datasets according to the given percentage, which resulted in a subject-independent sleepiness detection. The output was in the form of continuous regression further rounded to the closest integer and divided into five classes according to Karolinska Sleepiness Scale (KSS = 1-5, 6, 7, 8, 9). The best performance was obtained with a convolutional neural network (CNN) combined with Long Short-Term Memory (LSTM) architecture, with time series data as input. This gave an accuracy of 41.44%, a mean absolute error of 0.94 and a macro F1-score of 0.37. Overall, the models with time series data showed better classification results compared to those with time-frequency data. Class weighting, giving all classes inverse proportional weight to their appearance, compensated slightly for class imbalance, but all networks had in general difficulties with generalizing to new data.

Driver sleepiness is a cause for crashes and it is estimated that 3.9 to 33 % of all crashes might be related to sleepiness at the wheel. It is desirable to get an objective measurement of driver sleepiness for reduced sensitivity to subjective variations. Using deep learning for classification of driver sleepiness could be a step toward this objective. In this master thesis, deep learning was used for investigating classification of electrophysiological data, electroencephalogram (EEG) and electrooculogram (EOG), from drivers into levels of sleepiness. The EOG reflects eye position and EEG reflects brain activity. Initially, the intention was to include electrocardiogram (ECG), which reflects heart activity, in the research but this data were later excluded. Both raw time series data and data transformed into time-frequency domain representations were fed into the developed neural networks and for comparison manually extracted features were used in a shallow neural network architecture. Investigation of using EOG and EEG data separately as input was performed as well as a combination as input. The data were labeled using the Karolinska Sleepiness Scale, and the scale was divided into two labels "fatigue" and "alert" for binary classification or in five labels for comparison of classification and regression. The effect of example length was investigated using 150 seconds, 60 seconds and 30 seconds data. Different variations of the main network architecture were used depending on the data representation and the best result was given when using a combination of a convolutional neural network (CNN) and a long short-term memory (LSTM) network with time distributed 150 seconds EOG data as input. The accuracy was in this case 80.4 % and the majority of both alert and fatigue epochs were classified correctly with 85.7 % and 66.7 % respectively. Using the optimal threshold from the created receiver operating characteristics (ROC) curve resulted in a more balanced classifier with 76.3 % correctly classified alert examples and 79.2 % correctly classified fatigue examples. The results from the EEG data, both in terms of accuracy and distribution of correctly classified examples, were shown to be less promising compared to EOG data. Combining EOG and EEG signals was shown to slightly increase the proportion of correctly classified fatigue examples. However, more promising results were obtained when balancing the classifier for solely EOG signals. The overall result from this project shows that there are patterns in the data connected to sleepiness that the neural network can find which makes further work on applying deep learning to the area of driver sleepiness interesting.

Falling asleep while operating a moving vehicle is a contributing factor to the statistics of road related accidents. It has been estimated that 20% of all accidents where a vehicle has been involved are due to sleepiness behind the wheel. To prevent accidents and to save lives are of uttermost importance. In this thesis, given the world’s largest dataset of driver participants, two methods of evaluating driver sleepiness have been evaluated. The first method was based on the creation of epochs from lane departures and KSS, whilst the second method was based solely on the creation of epochs based on KSS. From the epochs, a number of features were extracted from both physiological signals and the car’s controller area network. The most important features were selected via a feature selection step, using sequential forward floating selection. The selected features were trained and evaluated on linear SVM, Gaussian SVM, KNN, random forest and adaboost. The random forest classifier was chosen in all cases when classifying previously unseen data.The results shows that method 1 was prone to overfit. Method 2 proved to be considerably better, and did not suffer from overfitting. The test results regarding method 2 were as follows; sensitivity = 80.3%, specificity = 96.3% and accuracy = 93.5%.The most prominent features overall were found in the EEG and EOG domain together with the sleep/wake predictor feature. However indications have been made that complexities might contribute to the detection of sleepiness as well, especially the Higuchi’s fractal dimension.