Neural Networks are at the core of computer vision solutions for various applications. With the advent of deep neural networks Facial Expression Recognition (FER) has been a very ineluctable and challenging task in the field of computer vision. Micro-expressions (ME) have been quite prominently used in security, psychotherapy, neuroscience and have a wide role in several related disciplines. However, due to the subtle movements of facial muscles, the micro-expressions are difficult to detect and identify. Due to the above, emotion detection and classification have always been hot research topics. The recently adopted networks to train FERs are yet to focus on issues caused due to overfitting, effectuated by insufficient data for training and expression unrelated variations like gender bias, face occlusions and others. Association of FER with the Speech Emotion Recognition (SER) triggered the development of multimodal neural networks for emotion classification in which the application of sensors played a significant role as they substantially increased the accuracy by providing high quality inputs, further elevating the efficiency of the system. This thesis relates to the exploration of different principles behind application of deep neural networks with a strong focus towards Convolutional Neural Networks (CNN) and Generative Adversarial Networks (GAN) in regards to their applications to emotion recognition. A Motion Magnification algorithm for ME's detection and classification was implemented for applications requiring near real-time computations. A new and improved architecture using a Multimodal Network was implemented. In addition to the motion magnification technique for emotion classification and extraction, the Multimodal algorithm takes the audio-visual cues as inputs and reads the MEs on the real face of the participant. This feature of the above architecture can be deployed while administering interviews, or supervising ICU patients in hospitals, in the auto industry, and many others. The real-time emotion classifier based on state-of-the-art Image-Avatar Animation model was tested on simulated subjects. The salient features of the real-face are mapped on avatars that are build with a 3D scene generation platform. In pursuit of the goal of emotion classification, the Image Animation model outperforms all baselines and prior works. Extensive tests and results obtained demonstrate the validity of the approach.

One of the basic problems in computer vision is motion detection and analysis in a given scene. This work focuses on detecting small changes in the image by using the moiré phenomenon. The main goal of this thesis is to detect different types of dislocations with algorithms used in computer vision. In this work synthetically created pictures are analysed, which were created with the mathematical model of a pinhole camera.

Vibration analysis is a method that studies patterns in vibration data and measures vibration levels. It is usually performed on time waveforms of the vibration signal directly and on thefrequency spectrum derived by applying the Fourier Transform on the time waveform. Conventional vibration analysis methods are either expensive, need a complicated setup, or both. Non-contact measurement systems, such as high-speed cameras coupled with computer vision and motion magnification methods, are suitable options for monitoring vibrations of any system. In this work, many classic and state-of-the-art computer vision tracking algorithms were compared. Low and high frame rate videos are used to evaluate their ability to track the oscillatory movement that characterizes vibrations. The trackers are benchmarked with literature and experimental study. Two sets of experiments were carried out in this work, one using a cantilever and another using a robot. The resonance frequencies obtained from the vision camera method are compared to the Laser vibrometer method, which is industry standard. The results show that the resonance frequencies of both methods are closer to each other. The limitations of the tracking algorithm-based approach used for vibration analysis were discussed at the end. Since the methods provided are generic, they may be easily modified for other relevant applications. / Vibrationsanalys är en metod som studerar mönster i vibrationsdata och mäter vibrationsnivåer. Det utförs vanligtvis på tidvågformer av vibrationssignalen direkt och på frekvensen, spektrum som härleds genom att applicera Fourier Transform på tidvågform. Konventionella vibrationsanalysmetoder är antingen dyra, kräver en komplicerad installation eller båda. Beröringsfria mätsystem, till exempel höghastighetskameror i kombination med datorsyn och rörelseförstoringsmetoder, är lämpliga alternativ för att övervaka vibrationer i alla system. I detta arbete jämfördes många klassiska och toppmoderna datorsynsspårningsalgoritmer. Videor med låg och hög bildhastighet används för att utvärdera deras förmåga att spåra den oscillerande rörelsen som kännetecknar vibrationer. Spårarna jämförs med litteratur och experimentell studie. I detta arbete utfördes två uppsättningar experiment, ett med en fribärare och ett annat med en robot. De resonans frekvenser som erhålls från visionkamerametoden jämförs med Laservibrometer metoden, som är branschstandard. Resultaten visar att resonansfrekvenserna för båda metoderna ligger närmare varandra. Begränsningarna av det spårningsalgoritmbaserade tillvägagångssättet som används för vibrationsanalys diskuterades i slutet. Eftersom de angivna metoderna är generiska kan de enkelt modifieras för andra relevanta applikationer.

Dans le monde industriel comme dans le monde scientifique, le développement de capteurs a toujours répondu à la volonté d’observer l’inobservable. La caméra rapide fait partie de ceux-là puisqu’elle permet de dévoiler des dynamiques invisibles, de la formation de fissure au vol du moustique. Dans un environnement extrêmement concurrentiel, ces caméras sont principalement limitées par le nombre d’images acquises par seconde. Le but de cette thèse est d’augmenter la capacité de dévoiler la dynamique invisible en enrichissant l’acquisition initiale par des modèles dynamiques. La problématique consiste alors à élaborer des méthodes permettant de relier en temps réel un modèle et la perception d’un système réel. Les bénéfices de cette utilisation offrent ainsi la possibilité de faire de l’interpolation, de la prédiction et de l’identification. Cette thèse est composée de trois parties. La première est axée sur la philosophie du traitement vidéo et propose d’utiliser des modèles élémentaires et génériques. Un algorithme d’estimation de grands mouvements est proposé mais l’approche actuellement proposée n’est pas assez générique pour être exploitée dans un contexte industriel. La deuxième partie propose d’utiliser des méthodes d’assimilation de données séquentielle basées sur la famille des filtres de Kalman afin d’associer un modèle avec des observations par caméras rapides pour des systèmes mécaniques. La troisième partie est une application à l’analyse modale expérimentale non linéaire. Deux schémas d’assimilation temps réel multicapteurs sont présentés et leur mise en œuvre est illustrée pour de la reconstruction 3D et de la magnification. / The development of sensors has always followed the ambition of industrial and scientific people to observe the unobservable. High speed cameras are part of this adventure, revealing invisible dynamics such as cracks formation or subtle mosquito flight. Industrial high speed vision is a very competitive domain in which cameras stand out through their acquisition speed. This thesis aims to broaden their capacity by augmenting the initial acquisition with dynamic models. This work proposes to develop methods linking in real time a model with a real system. Aimed benefits are interpolation, prediction and identification. Three parts are developed. The first one is based on video processing and submits to use kinematic elementary and generic models. An algorithm of motion estimation for large movements is proposed but the generic nature does not allow a sufficient knowledge to be conclusive. The second part proposes using sequential data assimilation methods known as Kalman filters. A scheme to assimilate video data with a mechanical model is successfully implemented. An application of data assimilation in modal analysis is developed. Two multi sensors real time assimilation schemes for nonlinear modal identification are proposed. These schemes are integrated in two applications on 3D reconstruction and motion magnification.