In today’s high paced, hi-tech and high stress environment, with extended work hours, long to-do lists and neglected personal health, sleep deprivation has become common in modern culture. Coupled with these factors is the inherent repetitious and tedious nature of certain occupations and daily routines, which all add up to an undesirable fluctuation in individuals’ cognitive attention and capacity. Given certain critical professions, a momentary or prolonged lapse in attention level can be catastrophic and sometimes deadly. This research proposes to develop a real-time monitoring system which uses fundamental physiological signals such as the Electrocardiograph (ECG), to analyze and predict the presence or lack of cognitive attention in individuals during task execution. The primary focus of this study is to identify the correlation between fluctuating level of attention and its implications on the physiological parameters of the body. The system is designed using only those physiological signals that can be collected easily with small, wearable, portable and non-invasive monitors and thereby being able to predict well in advance, an individual’s potential loss of attention and ingression of sleepiness. Several advanced signal processing techniques have been implemented and investigated to derive multiple clandestine and informative features. These features are then applied to machine learning algorithms to produce classification models that are capable of differentiating between the cases of a person being attentive and the person not being attentive. Furthermore, Electroencephalograph (EEG) signals are also analyzed and classified for use as a benchmark for comparison with ECG analysis. For the study, ECG signals and EEG signals of volunteer subjects are acquired in a controlled experiment. The experiment is designed to inculcate and sustain cognitive attention for a period of time following which an attempt is made to reduce cognitive attention of volunteer subjects. The data acquired during the experiment is decomposed and analyzed for feature extraction and classification. The presented results show that to a fairly reasonable accuracy it is possible to detect the presence or lack of attention in individuals with just their ECG signal, especially in comparison with analysis done on EEG signals. The continual work of this research includes other physiological signals such as Galvanic Skin Response, Heat Flux, Skin Temperature and video based facial feature analysis.

La surveillance des nouveau-nés prématurés placés en unité de soins intensifs a conduit à la notion de monitoring et à l'acquisition de nombreux signaux physiologiques. Si ces informations sont bien utilisées pour le diagnostic et la prévention des situations d'urgence, force est de constater qu'à ce jour, elles le sont beaucoup moins dans un objectif prédictif. La difficulté d'extraction d'informations fiables en temps réel, sans aucun contrôle visuel, à partir de signaux non stationnaires, en est vraisemblablement la cause. Ce mémoire vise donc à proposer des méthodes robustes, adaptées au contexte des unités de soins intensifs néonatals et du temps réel. Pour cela, un ensemble de méthodes génériques appliquées à la variabilité cardiaque, mais capable d'être adaptées à d'autres constantes physiologiques telles que la respiration, ont été développées et testées en contexte clinique. Quatre grandes parties illustrent notre propos : - La proposition d'une méthode originale de détection temps réel probabiliste multicaractéristique permettant de répondre à une problématique d'extraction robuste d'événements d'intérêt à partir de signaux physiologiques bruités. Générique, cette solution est appliquée à la détection robuste du QRS d'un signal ECG. Elle est basée sur le calcul temps réel de plusieurs probabilités a posteriori, concernant les propriétés du signal, qui sont ensuite fusionnées au sein d'un nœud de décision reposant sur l'utilisation pondérée de la divergence de Kullback-Leibler. Comparée à deux méthodes classiques de la littérature sur deux bases de données bruitées, elle obtient un taux d'erreur de détection inférieur (20.91% vs 29.02% (ondelettes) et 33.08% (Pan-Tompkins) sur la base de test). - La proposition d'une méthode impliquant plusieurs modèles semi-markoviens cachés, visant la segmentation de périodes au sein desquelles le détecteur temps réel probabiliste multicaractéristique fournit les détections d'évènements les plus fiables. En comparaison à deux méthodes de la littérature, la solution proposée obtient de meilleures performances, le critère d‘erreur obtenu est significativement plus faible (entre -21.37% et -74.98% selon la base et l'approche évaluée). - La sélection d'un détecteur optimal pour le monitoring d'événements d'apnée-bradycardie, en termes de fiabilité et précocité, à partir de données ECG obtenues chez le nouveau-né. Les performances du détecteur retenu seront comparées aux alarmes générées par un dispositif industriel de suivi continu classiquement utilisé en service de néonatalogie (moniteur Philips IntelliVue). La méthode basée sur le changement abrupt de la moyenne des RR obtient les meilleurs résultats au regard du délai (3.99 s vs 11.53 s pour le moniteur IntelliVue) et de la fiabilité (critère d'erreur de 43.60% vs 80.40%). - La conception et le développement d'une plateforme logicielle SYNaPSE (SYstem for Noninvasive Physiological Signal Explorations) permettant l'acquisition de divers signaux physiologiques en très grande quantité, et de façon non invasive, au sein des unités de soins. La conception modulaire de cette plateforme, ainsi que ses propriétés temps réel, permettent l'intégration simple et rapide de méthodes de traitement du signal complexes. Son intérêt translationnel est montré dans le dépouillement d'une base de données cherchant à étudier l'impact de la bilirubine sur la variabilité cardiaque. / The surveillance of premature newborns placed in intensive care units led to the notion of monitoring and the acquisition of many physiological signals. While this information is well used for the diagnosis and prevention of emergency situations, it must be acknowledged that, to date, it is less the case for predictive purposes. This is mainly due to the difficulty of extracting reliable information in real time, without any visual control, from non-stationary signals. This thesis aims to propose robust methods, adapted to the context of neonatal intensive care units and real time. For this purpose, a set of generic methods applied to cardiac variability, but capable of being adapted to other physiological constants such as respiration, have been developed and tested in clinical context. Four main parts illustrate these points : - The proposal of an original multicharacteristic probabilistic real time detection method for robust detection of interest events of noisy physiological signals. Generic, this solution is applied to the robust QRS complex detection of the ECG signals. It is based on the real time calculation of several posterior probabilities of the signal properties before merging them into a decision node using the weighted Kullback-Leibler divergence. Compared to two classic methods from the literature on two noisy databases, it has a lower detection error rate (20.91% vs. 29.02% (wavelets) and 33.08% (Pan-Tompkins) on the test database). - The proposal of using hidden semi-markovian models for the segmentation of temporal periods with most reliable event detections. Compared to two methods from the literature, the proposed solution achieves better performance, the error criterion obtained is significantly lower (between -21.37% and -74.98% depending on the basis and approach evaluated). - The selection of an optimal detector for the monitoring of apnea-bradycardia events, in terms of reliability and precocity, based on ECG data obtained from newborns. The performance of the selected detector will be compared to the alarms generated by an industrial continuous monitoring device traditionally used in neonatology service (Philips IntelliVue monitor). The method based on the abrupt change of the RR average achieves the best results in terms of time (3.99 s vs. 11.53 s for the IntelliVue monitor) and reliability (error criterion of 43.60% vs. 80.40%). - The design and development of SYNaPSE (SYstem for Noninvasive Physiological Signal Explorations) software platform for the acquisition of various physiological signals in large quantities, and in a non-invasive way, within the care units. The modular design of this platform, as well as its real time properties, allows simple and fast integration of complex signal processing methods. Its translational interest is shown in the analysis of a database in order to study the impact of bilirubin on cardiac variability.

Mémoire numérisé par la Division de la gestion de documents et des archives de l'Université de Montréal

Falling asleep while operating a moving vehicle is a contributing factor to the statistics of road related accidents. It has been estimated that 20% of all accidents where a vehicle has been involved are due to sleepiness behind the wheel. To prevent accidents and to save lives are of uttermost importance. In this thesis, given the world’s largest dataset of driver participants, two methods of evaluating driver sleepiness have been evaluated. The first method was based on the creation of epochs from lane departures and KSS, whilst the second method was based solely on the creation of epochs based on KSS. From the epochs, a number of features were extracted from both physiological signals and the car’s controller area network. The most important features were selected via a feature selection step, using sequential forward floating selection. The selected features were trained and evaluated on linear SVM, Gaussian SVM, KNN, random forest and adaboost. The random forest classifier was chosen in all cases when classifying previously unseen data.The results shows that method 1 was prone to overfit. Method 2 proved to be considerably better, and did not suffer from overfitting. The test results regarding method 2 were as follows; sensitivity = 80.3%, specificity = 96.3% and accuracy = 93.5%.The most prominent features overall were found in the EEG and EOG domain together with the sleep/wake predictor feature. However indications have been made that complexities might contribute to the detection of sleepiness as well, especially the Higuchi’s fractal dimension.

Blood glucose (BG) prediction and classification plays big role in diabetic patients' daily lives. Based on International Diabetes Federation (IDF) in 2019, 463 million people are diabetic globally and the projection by 2045 is that the number will rise to 700 million people. Continuous glucose monitor (CGM) systems assist diabetic patients daily, by alerting them about their BG levels fluctuations continuously. The history of CGM systems started in 1999, when the Food and Drug Administration (FDA) approved the first CGM system, until nowadays where the developments of the system's accurate reading and delay on reporting are continuously improving. CGM systems are key elements in closed-loop systems, that are using BG monitoring in order to calculate and deliver with the patient's supervision the needed insulin to the patient automatically. Data quality and the feature variation are essential for CGM systems, therefore many studies are being conducted in order to support the developments and improvements of CGM systems and diabetics daily lives. This thesis aims to show that physiological signals retrieved from various sensors, can assist the classification and prediction of BG levels and more specifically that breathing rate can enhance the accuracy of CGM systems for diabetic patients and also healthy individuals. The results showed that physiological data can improve the accuracy of prediction and classification of BG levels and improve the performance of CGM systems during classification and prediction tasks. Finally, future improvements could include the use of predictive horizon (PH) regarding the data and also the selection and use of different models.

User-centered computer based learning is an emerging field of interdisciplinary research. Research in diverse areas such as psychology, computer science, neuroscience and signal processing is making contributions the promise to take this field to the next level. Learning systems built using contributions from these fields could be used in actual training and education instead of just laboratory proof-of-concept. One of the important advances in this research is the detection and assessment of the cognitive and emotional state of the learner using such systems. This capability moves development beyond the use of traditional user performance metrics to include system intelligence measures that are based on current neuroscience theories. These advances are of paramount importance in the success and wide spread use of learning systems that are automated and intelligent. Emotion is considered an important aspect of how learning occurs, and yet estimating it and making adaptive adjustments are not part of most learning systems. In this research we focus on one specific aspect of constructing an adaptive and intelligent learning system, that is, estimation of the emotion of the learner as he/she is using the automated training system. The challenge starts with the definition of the emotion and the utility of it in human life. The next challenge is to measure the co-varying factors of the emotions in a non-invasive way, and find consistent features from these measures that are valid across wide population. In this research we use four physiological sensors that are non-invasive, and establish a methodology of utilizing the data from these sensors using different signal processing tools. A validated set of visual stimuli used worldwide in the research of emotion and attention, called International Affective Picture System (IAPS), is used. A dataset is collected from the sensors in an experiment designed to elicit emotions from these validated visual stimuli. We describe a novel wavelet method to calculate hemispheric asymmetry metric using electroencephalography data. This method is tested against typically used power spectral density method. We show overall improvement in accuracy in classifying specific emotions using the novel method. We also show distinctions between different discrete emotions from the autonomic nervous system activity using electrocardiography, electrodermal activity and pupil diameter changes. Findings from different features from these sensors are used to give guidelines to use each of the individual sensors in the adaptive learning environment.

The last advancements in technology leads to an easy acquisition and spreading of multi-dimensional multimedia content, e.g. videos, which in many cases depict human faces. From such videos, valuable information describing the intrinsic characteristic of the recorded user can be retrieved: the features extracted from the facial patch are relevant descriptors that allow for the measurement of subject's emotional status or the identification of synthetic characters. One of the emerging challenges is the development of contactless approaches based on face analysis aiming at measuring the emotional status of the subject without placing sensors that limit or bias his experience. This raises even more interest in the context of Quality of Experience (QoE) measurement, or the measurement of user emotional status when subjected to a multimedia content, since it allows for retrieving the overall acceptability of the content as perceived by the end user. Measuring the impact of a given content to the user can have many implications from both the content producer and the end-user perspectives. For this reason, we pursue the QoE assessment of a user watching multimedia stimuli, i.e. 3D-movies, through the analysis of his facial features acquired by means of contactless approaches. More specifically, the user's Heart Rate (HR) was retrieved by using computer vision techniques applied to the facial recording of the subject and then analysed in order to compute the level of engagement. We show that the proposed framework is effective for long video sequences, being robust to facial movements and illumination changes. We validate it on a dataset of 64 sequences where users observe 3D movies selected to induce variations in users' emotional status. From one hand understanding the interaction between the user's perception of the content and his cognitive-emotional aspects leads to many opportunities to content producers, which may influence people's emotional statuses according to needs that can be driven by political, social, or business interests. On the other hand, the end-user must be aware of the authenticity of the content being watched: advancements in computer renderings allowed for the spreading of fake subjects in videos. Because of this, as a second challenge we target the identification of CG characters in videos by applying two different approaches. We firstly exploit the idea that fake characters do not present any pulse rate signal, while humans' pulse rate is expressed by a sinusoidal signal. The application of computer vision techniques on a facial video allows for the contactless estimation of the subject's HR, thus leading to the identification of signals that lack of a strong sinusoidality, which represent virtual humans. The proposed pipeline allows for a fully automated discrimination, validated on a dataset consisting of 104 videos. Secondly, we make use of facial spatio-temporal texture dynamics that reveal the artefacts introduced by computer renderings techniques when creating a manipulation, e.g. face swapping, on videos depicting human faces. To do so, we consider multiple temporal video segments on which we estimated multi-dimensional (spatial and temporal) texture features. A binary decision of the joint analysis of such features is applied to strengthen the classification accuracy. This is achieved through the use of Local Derivative Patterns on Three Orthogonal Planes (LDP-TOP). Experimental analyses on state-of-the-art datasets of manipulated videos show the discriminative power of such descriptors in separating real and manipulated sequences and identifying the creation method used. The main finding of this thesis is the relevance of facial features in describing intrinsic characteristics of humans. These can be used to retrieve significant information like the physiological response to multimedia stimuli or the authenticity of the human being itself. The application of the proposed approaches also on benchmark dataset returned good results, thus demonstrating real advancements in this research field. In addition to that, these methods can be extended to different practical application, from the autonomous driving safety checks to the identification of spoofing attacks, from the medical check-ups when doing sports to the users' engagement measurement when watching advertising. Because of this, we encourage further investigations in such direction, in order to improve the robustness of the methods, thus allowing for the application to increasingly challenging scenarios.

Recognition of physical and mental responses to stress is important for stress assessment and management as its negative effects in health can be prevented or reduced. Wearable technology, mainly using electroencephalogram (EEG), provides information such as tracking fitness activity, disease detection, and monitoring neurologicalstates of individuals. However, the recording of EEG signals from a wearable device is inconvenient, expensive, and uncomfortable during normal daily activities. This study introduces the application of tensor decomposition of non-EEG data for visualizing and classifying neurological statuses with application to human stress recognition. The multimodal dataset of non-EEG physiological signals publicly available from the PhysioNet database was used for testing the proposed method. To visualize the biosignals in a low dimensional feature space, the multi-way factorization technique known as the PARAFAC was applied for feature extraction. Results show visualizations that well separate the four groups of neurological statuses obtained from twenty healthy subjects. The extracted features were then used for pattern classification. Two statistical classifiers, which are the multinomial logit regression(MLR) and linear discriminant analysis (LDA), were implemented. The results show that the MLR and LDA can identify the four neurological statuses with accuracies of 95% and 98.8%, respectively. This study suggests the potential application of tensor decomposition for the analysis of physiological measurements collected from multiple sensors. Moreover, the proposed study contributes to the advancement of wearable technology for human stress monitoring. With tensor decomposition of complex multi-sensor or multi-channel data, simple classification techniques can be employed to achieve similar results obtained using sophisticated machine-learning techniques.

Les représentations convolutives extraient des motifs récurrents qui aident à comprendre la structure locale dans un jeu de signaux. Elles sont adaptées pour l’analyse des signaux physiologiques, qui nécessite des visualisations mettant en avant les informations pertinentes. Ces représentations sont aussi liées aux modèles d’apprentissage profond. Dans ce manuscrit, nous décrivons des avancées algorithmiques et théoriques autour de ces modèles. Nous montrons d’abord que l’Analyse du Spectre Singulier permet de calculer efficacement une représentation convolutive. Cette représentation est dense et nous décrivons une procédure automatisée pour la rendre plus interprétable. Nous proposons ensuite un algorithme asynchrone, pour accélérer le codage parcimonieux convolutif. Notre algorithme présente une accélération super-linéaire. Dans une seconde partie, nous analysons les liens entre représentations et réseaux de neurones. Nous proposons une étape d’apprentissage supplémentaire, appelée post-entraînement, qui permet d’améliorer les performances du réseau entraîné, en s’assurant que la dernière couche soit optimale. Puis nous étudions les mécanismes qui rendent possible l’accélération du codage parcimonieux avec des réseaux de neurones. Nous montrons que cela est lié à une factorisation de la matrice de Gram du dictionnaire. Finalement, nous illustrons l’intérêt de l’utilisation des représentations convolutives pour les signaux physiologiques. L’apprentissage de dictionnaire convolutif est utilisé pour résumer des signaux de marche et le mouvement du regard est soustrait de signaux oculométriques avec l’Analyse du Spectre Singulier. / Convolutional representations extract recurrent patterns which lead to the discovery of local structures in a set of signals. They are well suited to analyze physiological signals which requires interpretable representations in order to understand the relevant information. Moreover, these representations can be linked to deep learning models, as a way to bring interpretability intheir internal representations. In this disserta tion, we describe recent advances on both computational and theoretical aspects of these models.First, we show that the Singular Spectrum Analysis can be used to compute convolutional representations. This representation is dense and we describe an automatized procedure to improve its interpretability. Also, we propose an asynchronous algorithm, called DICOD, based on greedy coordinate descent, to solve convolutional sparse coding for long signals. Our algorithm has super-linear acceleration.In a second part, we focus on the link between representations and neural networks. An extra training step for deep learning, called post-training, is introduced to boost the performances of the trained network by making sure the last layer is optimal. Then, we study the mechanisms which allow to accelerate sparse coding algorithms with neural networks. We show that it is linked to afactorization of the Gram matrix of the dictionary.Finally, we illustrate the relevance of convolutional representations for physiological signals. Convolutional dictionary learning is used to summarize human walk signals and Singular Spectrum Analysis is used to remove the gaze movement in young infant’s oculometric recordings.

No description available.