Driver fatigue is responsible for up to 30% of fatal car accidents. This issue has been addressed by many scholars in order to save thousands of lives and reduce many costs. The goal of this work is to reduce the number of car accidents caused by mental fatigue or drowsiness. In order to achieve this goal, we propose a personalized Bayesian Network (BN) to detect driver’s fatigue. The detection of driver fatigue is enhanced by combining data that reflects the driver’s performance with context-aware information. The parameters of the system are the angular velocity of the steering wheel, the pressure applied to the gas and brake pedals, the grip force on the steering wheel, weather conditions, current traffic, and time of day. The aforementioned parameters of the network are updated on a regular basis, which makes fatigue detection more reliable. Besides, these parameters allow the system to detect a driver’s fatigue through driving performance which is both individualized and context aware. In our experiment, subjects drove a driving simulator game during six sessions, for a total of one hour. After each session, every subject used the Karolinska Sleepiness Scale (KSS) to rate her fatigue’s level. The system was trained on the data collected separately from each user, allowing BN to be personalized for each subject. The proposed system showed an average accuracy of 96%, and ability to overcome the issue of individual differences and uncertainties which are involved in fatigue detection process.

Driver fatigue is responsible for up to 30% of fatal car accidents. This issue has been addressed by many scholars in order to save thousands of lives and reduce many costs. The goal of this work is to reduce the number of car accidents caused by mental fatigue or drowsiness. In order to achieve this goal, we propose a personalized Bayesian Network (BN) to detect driver’s fatigue. The detection of driver fatigue is enhanced by combining data that reflects the driver’s performance with context-aware information. The parameters of the system are the angular velocity of the steering wheel, the pressure applied to the gas and brake pedals, the grip force on the steering wheel, weather conditions, current traffic, and time of day. The aforementioned parameters of the network are updated on a regular basis, which makes fatigue detection more reliable. Besides, these parameters allow the system to detect a driver’s fatigue through driving performance which is both individualized and context aware. In our experiment, subjects drove a driving simulator game during six sessions, for a total of one hour. After each session, every subject used the Karolinska Sleepiness Scale (KSS) to rate her fatigue’s level. The system was trained on the data collected separately from each user, allowing BN to be personalized for each subject. The proposed system showed an average accuracy of 96%, and ability to overcome the issue of individual differences and uncertainties which are involved in fatigue detection process.

Applications of speech recognition have evolved in recent years from simple transcription tasks to metadata analysis. This thesis explores the use of speech recognition for automated fatigue detection. The fatigue detection system relies on accurate phonetic alignments from a speech recognition system. The main challenge addressed in this thesis was to make the process of phonetic alignment using speech recognition robust to out of vocabulary words. This requirement was achieved by incorporating confidence measures, which significantly reduce false positives in speech recognition output. This allowed the performance of the fatigue detection system to match the results of other cognitive tests based on the Sleep Onset Latency (SOL) and Sleep, Activity, Fatigue, and Task Effectiveness (SAFTE). Confidence measures reduced the squared error between voice-based fatigue prediction and SAFTE by 20% when 67.1% of the words in the test set were out of vocabulary words.

This work provides an innovative solution for monitoring fatigue for users behind workstations. A web camera was adjusted to work in near infrared range and a system of 880 nm IR diodes was implemented to create an IR vision system to localize and track the eye pupils. The software developed monitors and tracks eyes for signs of fatigue by measuring PERCLOS. The software developed runs on the workstation and is designed to draw limited computational power, so as to not interfere with the user task. To overcome low-frame rate imposed by the hardware limitations and to improve real time monitoring, two-phases detection and tacking algorithm is implemented. The proposed system successfully monitors fatigue at a rate of 8 fps. The system is well suited to monitor users in command centers, flight control centers, airport traffic dispatches, military operation and command centers, etc., but the work can be extended to wearable devices and other environments.

The automotive industry is growing bigger each year. The central concern for any automotive company is driver and passenger safety. Many automotive companies have developed driver assistance systems, to help the driver and to ensure driver safety. These systems include adaptive cruise control, lane departure warning, lane change assistance, collision avoidance, night vision, automatic parking, traffic sign recognition, and driver fatigue detection. In this thesis, we aim to build a driver fatigue detection system that advances the research in this area. Using vision in detecting driver fatigue is commonly the key part for driver fatigue detection systems. We have decided to investigate different direction. We examine the driver's voice, heart rate, and driving performance to assess fatigue level. The system consists of three main modules: the audio module, the heart rate and other signals module, and the Bayesian network module. The audio module analyzes an audio recording of a driver and tries to estimate the level of fatigue for the driver. A Voice Activity Detection (VAD) module is used to extract driver speech from the audio recording. Mel-Frequency Cepstrum Coefficients, (MFCC) features are extracted from the speech signal, and then Support Vector Machines (SVM) and Hidden Markov Models (HMM) classifiers are used to detect driver fatigue. Both classifiers are tuned for best performance, and the performance of both classifiers is reported and compared. The heart rate and other signals module uses heart rate, steering wheel position, and the positions of the accelerator, brake, and clutch pedals to detect the level of fatigue. These signals' sample rates are then adjusted to match, allowing simple features to be extracted from the signals, and SVM and HMM classifiers are used to detect fatigue level. The performance of both classifiers is reported and compared. Bayesian networks' abilities to capture dependencies and uncertainty make them a sound choice to perform the data fusion. Prior information (Day/Night driving and previous decision) is also incorporated into the network to improve the final decision. The accuracies of the audio and heart rate and other signals modules are used to calculate certain CPTs for the Bayesian network, while the rest of the CPTs are calculated subjectively. The inference queries are calculated using the variable elimination algorithm. For those time steps where the audio module decision is absent, a window is defined and the last decision within this window is used as a current decision. The performance of the system is assessed based on the average accuracy per second. A dataset was built to train and test the system. The experimental results show that the system is very promising. The performance of the system was assessed based on the average accuracy per second; the total accuracy of the system is 90.5%. The system design can be easily improved by easily integrating more modules into the Bayesian network.

CNPq / Introdução: As pessoas com lesão medular (LM) podem ter seus músculos paralisados ativados por meio da estimulação elétrica funcional (FES) sobre vias neurais presentes próximas à pele. Estas estimulações elétricas são importantes para a recuperação do trofismo neuromuscular ou durante o controle de movimento por próteses neurais. No entanto, ao longo da aplicação da FES, a fadiga ocorre, diminuindo a eficiência da contração, principalmente devido à hipotrofia neuromuscular presente nessa população. A aquisição da vibração das fibras musculares como indicador de fadiga é registrada por meio da técnica de mecanomiografia (MMG), que não sofre interferências elétricas decorrentes da aplicação da FES. Objetivo: Caracterizar a vibração do músculo reto femoral durante protocolo de fadiga neuromuscular eletricamente evocada em pessoas com lesão medular completa. Método: 24 membros (direito e esquerdo) de 15 participantes (idade: 27±5 anos) do sexo masculino (A e B na American Spinal Injury Impairment Scale) foram selecionados. Um estimulador elétrico operando como fonte de tensão, desenvolvido especialmente para pesquisa, foi configurado com: freqüência de pulso em 1 kHz (20% de ciclo de trabalho) e trem de pulsos (modulação) em 70 Hz (20% período ativo). O sinal triaxial [X (transversal), Y (longitudinal) e Z (perpendicular)] da MMG foi processado com filtro Butterworth de terceira ordem e banda passante entre 5 e 50 Hz. Previamente ao protocolo, a tensão de saída do estimulador foi incrementada (~3 V/s evitando-se a adaptação/habituação dos motoneurônios) até alcançar a extensão máxima eletricamente estimulada (EMEE) da articulação do joelho. Uma célula de carga foi usada para registrar a resposta de força, onde após a sua colocação, a intensidade da FES necessária para alcançar a EMEE foi aplicada e registrada pela célula de carga como 100% da força (F100%). Durante o protocolo de fadiga neuromuscular, a intensidade do estímulo foi incrementada durante o controle para manter a força em F100%. Quatro instantes (I - IV) foram selecionados entre F100% e a incapacidade da FES manter a resposta de força acima de 30% (F30%). O sinal foi processado nos domínios temporal (energia), espectral (frequência mediana) e wavelet (temporal-espectral com doze bandas de frequência entre 5 e 53 Hz). Os dados extraídos foram normalizados pelo instante inicial (I) gerando unidades arbitrárias (u.a.), e testados com estatística não paramétrica. Resultados: A frequência mediana não apresentou significância estatística. Em relação aos eixos de deslocamento da MMG, o eixo transversal mostrou o maior número de resultados estatisticamente significantivos. A energia da vibração das fibras musculares (domínio temporal) indicou diminuição entre os instantes I (músculo fresco) e II (pré-fadiga), como também entre os instantes I e IV (fadigado) com redução significativa. O domínio wavelet teve como foco o eixo transversal, especialmente as bandas de frequência de 13, 16, 20, 25 e 35 Hz, por terem indicado redução significativa durante a fadiga neuromuscular; principalmente, a banda de 25 Hz, que indicou redução significativa entre o instante I (valor da mediana dos dados de 0,53 u.a.) e os demais instantes [II (0,30 u.a), III (0,28 u.a.) e IV (0,24 u.a.)]. Conclusão: A fadiga neuromuscular é caracterizada pela redução da energia do sinal no eixo de deslocamento transversal (X) da vibração do músculo reto femoral, em pessoas com lesão medular completa, tanto no domínio temporal quanto principalmente no domínio wavelet, sendo a banda de frequência de 25 Hz a mais relevante, porque sua energia diminui com a ocorrência da fadiga neuromuscular. Estes achados abrem a possibilidade de aplicação em sistemas de malha fechada durante procedimentos de reabilitação física utilizando FES ou no controle de próteses neurais. / Introduction: People with spinal cord injury (SCI) may have the paralyzed muscles activated through functional electrical stimulation (FES) on neural pathways present below the skin. These electrical stimulations are important to restore the neuromuscular trophism or during the movement control using neural prostheses. However, prolonged FES application causes fatigue, which decreases the contraction strength, mainly due the neuromuscular hypotrophy in this population. The acquisition of myofibers’ vibration is recognized by mechanomyography (MMG) system and does not suffer electrical interference from the FES system. Objective: To characterize the rectus femoris muscle vibration during electrically evoked neuromuscular fatigue protocol in complete spinal cord injury subjects. Methods: As sample, 24 limbs (right and left) from 15 male participants (age: 27±5 y.o.) and ranked as A and B according to American Spinal Injury Impairment Scale) were selected. An electrical stimulator operating as voltage source, specially developed for research, was configured as: pulse frequency set to 1 kHz (20% duty cycle) and burst (modulating) frequency set to 70 Hz (20% active period). The triaxial [X (transverse), Y (longitudinal) and Z (perpendicular)] MMG signal of rectus femoris muscle was processed with a third-order 5-50 Hz bandpass Butterworth filter. A load cell was used to register the force. The stimulator output voltage was increased (~3 V/s to avoid motoneuron adaptation/habituation) until the maximal electrically-evoked extension (MEEE) of the knee joint. After the load cell placement, the stimuli magnitude required to reach MEEE was applied and registered by the load cell as muscular F100% response. Stimuli intensity was increased during the control to keep the force in F100%. Four instants (I - IV) were selected from F100% up to the inability to keep the FES response force above 30% (F30%). The signal was processed in temporal (energy), spectral (median frequency) and wavelet (temporal-spectral with twelve band frequencies between 5 and 53 Hz) domains. All data were normalized by initial instant, creating arbitrary units (a.u.), and non-parametric tests were applied. Results: The median frequency did not show statistical significance. Regarding the MMG axes, the transverse axis showed most statistical differences. The MMG energy (temporal domain) indicates the decrease between the instants I (unfatigued) and II (pre-fatigue), as well as instants I and IV (fatigued). The wavelet domain focused on the transverse axis, especially on 13, 16, 20, 25 and 35 Hz frequency bands, for having shown significant reduction proven during neuromuscular fatigue. In focus on 25 Hz band frequency that showed a constant decrease between instants I (median value from data de 0.53 a.u.) with subsequent instants [II (0.30 a.u.), III (0.28 a.u.) and IV (0.24 a.u.). Conclusion: Neuromuscular fatigue is characterized by energy decrease in MMG X-axis (transverse) signal of vibration on the rectus femoris muscle for complete spinal cord injured subjects, in the temporal domain but mainly in the wavelet domain. The 25 Hz is the most important band frequency because its energy decreases with neuromuscular fatigue. These findings open the possibility of application in closed-loop systems during physical rehabilitation procedures using FES or in the control of neural prostheses.

CNPq / Introdução: As pessoas com lesão medular (LM) podem ter seus músculos paralisados ativados por meio da estimulação elétrica funcional (FES) sobre vias neurais presentes próximas à pele. Estas estimulações elétricas são importantes para a recuperação do trofismo neuromuscular ou durante o controle de movimento por próteses neurais. No entanto, ao longo da aplicação da FES, a fadiga ocorre, diminuindo a eficiência da contração, principalmente devido à hipotrofia neuromuscular presente nessa população. A aquisição da vibração das fibras musculares como indicador de fadiga é registrada por meio da técnica de mecanomiografia (MMG), que não sofre interferências elétricas decorrentes da aplicação da FES. Objetivo: Caracterizar a vibração do músculo reto femoral durante protocolo de fadiga neuromuscular eletricamente evocada em pessoas com lesão medular completa. Método: 24 membros (direito e esquerdo) de 15 participantes (idade: 27±5 anos) do sexo masculino (A e B na American Spinal Injury Impairment Scale) foram selecionados. Um estimulador elétrico operando como fonte de tensão, desenvolvido especialmente para pesquisa, foi configurado com: freqüência de pulso em 1 kHz (20% de ciclo de trabalho) e trem de pulsos (modulação) em 70 Hz (20% período ativo). O sinal triaxial [X (transversal), Y (longitudinal) e Z (perpendicular)] da MMG foi processado com filtro Butterworth de terceira ordem e banda passante entre 5 e 50 Hz. Previamente ao protocolo, a tensão de saída do estimulador foi incrementada (~3 V/s evitando-se a adaptação/habituação dos motoneurônios) até alcançar a extensão máxima eletricamente estimulada (EMEE) da articulação do joelho. Uma célula de carga foi usada para registrar a resposta de força, onde após a sua colocação, a intensidade da FES necessária para alcançar a EMEE foi aplicada e registrada pela célula de carga como 100% da força (F100%). Durante o protocolo de fadiga neuromuscular, a intensidade do estímulo foi incrementada durante o controle para manter a força em F100%. Quatro instantes (I - IV) foram selecionados entre F100% e a incapacidade da FES manter a resposta de força acima de 30% (F30%). O sinal foi processado nos domínios temporal (energia), espectral (frequência mediana) e wavelet (temporal-espectral com doze bandas de frequência entre 5 e 53 Hz). Os dados extraídos foram normalizados pelo instante inicial (I) gerando unidades arbitrárias (u.a.), e testados com estatística não paramétrica. Resultados: A frequência mediana não apresentou significância estatística. Em relação aos eixos de deslocamento da MMG, o eixo transversal mostrou o maior número de resultados estatisticamente significantivos. A energia da vibração das fibras musculares (domínio temporal) indicou diminuição entre os instantes I (músculo fresco) e II (pré-fadiga), como também entre os instantes I e IV (fadigado) com redução significativa. O domínio wavelet teve como foco o eixo transversal, especialmente as bandas de frequência de 13, 16, 20, 25 e 35 Hz, por terem indicado redução significativa durante a fadiga neuromuscular; principalmente, a banda de 25 Hz, que indicou redução significativa entre o instante I (valor da mediana dos dados de 0,53 u.a.) e os demais instantes [II (0,30 u.a), III (0,28 u.a.) e IV (0,24 u.a.)]. Conclusão: A fadiga neuromuscular é caracterizada pela redução da energia do sinal no eixo de deslocamento transversal (X) da vibração do músculo reto femoral, em pessoas com lesão medular completa, tanto no domínio temporal quanto principalmente no domínio wavelet, sendo a banda de frequência de 25 Hz a mais relevante, porque sua energia diminui com a ocorrência da fadiga neuromuscular. Estes achados abrem a possibilidade de aplicação em sistemas de malha fechada durante procedimentos de reabilitação física utilizando FES ou no controle de próteses neurais. / Introduction: People with spinal cord injury (SCI) may have the paralyzed muscles activated through functional electrical stimulation (FES) on neural pathways present below the skin. These electrical stimulations are important to restore the neuromuscular trophism or during the movement control using neural prostheses. However, prolonged FES application causes fatigue, which decreases the contraction strength, mainly due the neuromuscular hypotrophy in this population. The acquisition of myofibers’ vibration is recognized by mechanomyography (MMG) system and does not suffer electrical interference from the FES system. Objective: To characterize the rectus femoris muscle vibration during electrically evoked neuromuscular fatigue protocol in complete spinal cord injury subjects. Methods: As sample, 24 limbs (right and left) from 15 male participants (age: 27±5 y.o.) and ranked as A and B according to American Spinal Injury Impairment Scale) were selected. An electrical stimulator operating as voltage source, specially developed for research, was configured as: pulse frequency set to 1 kHz (20% duty cycle) and burst (modulating) frequency set to 70 Hz (20% active period). The triaxial [X (transverse), Y (longitudinal) and Z (perpendicular)] MMG signal of rectus femoris muscle was processed with a third-order 5-50 Hz bandpass Butterworth filter. A load cell was used to register the force. The stimulator output voltage was increased (~3 V/s to avoid motoneuron adaptation/habituation) until the maximal electrically-evoked extension (MEEE) of the knee joint. After the load cell placement, the stimuli magnitude required to reach MEEE was applied and registered by the load cell as muscular F100% response. Stimuli intensity was increased during the control to keep the force in F100%. Four instants (I - IV) were selected from F100% up to the inability to keep the FES response force above 30% (F30%). The signal was processed in temporal (energy), spectral (median frequency) and wavelet (temporal-spectral with twelve band frequencies between 5 and 53 Hz) domains. All data were normalized by initial instant, creating arbitrary units (a.u.), and non-parametric tests were applied. Results: The median frequency did not show statistical significance. Regarding the MMG axes, the transverse axis showed most statistical differences. The MMG energy (temporal domain) indicates the decrease between the instants I (unfatigued) and II (pre-fatigue), as well as instants I and IV (fatigued). The wavelet domain focused on the transverse axis, especially on 13, 16, 20, 25 and 35 Hz frequency bands, for having shown significant reduction proven during neuromuscular fatigue. In focus on 25 Hz band frequency that showed a constant decrease between instants I (median value from data de 0.53 a.u.) with subsequent instants [II (0.30 a.u.), III (0.28 a.u.) and IV (0.24 a.u.). Conclusion: Neuromuscular fatigue is characterized by energy decrease in MMG X-axis (transverse) signal of vibration on the rectus femoris muscle for complete spinal cord injured subjects, in the temporal domain but mainly in the wavelet domain. The 25 Hz is the most important band frequency because its energy decreases with neuromuscular fatigue. These findings open the possibility of application in closed-loop systems during physical rehabilitation procedures using FES or in the control of neural prostheses.