Cette thèse s’intéresse à définir de nouvelles méthodes cliniques d’investigation permettant de juger de l’impact de l’avance en âge sur la motricité. En particulier, cette thèse se focalise sur deux principales perturbations possibles lors de l’avance en âge : la chute et l’altération de la marche.Ces deux perturbations motrices restent encore mal connues et leur analyse en clinique pose de véritables défis technologiques et scientifiques. Dans cette thèse, nous proposons des méthodes originales de détection qui peuvent être utilisées dans la vie courante ou en clinique, avec un minimum de contraintes techniques.Dans une première partie, nous abordons le problème de la détection de la chute à domicile, qui a été largement traité dans les années précédentes. En particulier, nous proposons une approche permettant d’exploiter le volume du sujet, reconstruit à partir de plusieurs caméras calibrées. Ces méthodes sont généralement très sensibles aux occultationsqui interviennent inévitablement dans le domicile et nous proposons donc une approche originale beaucoup plus robuste à ces occultations. L’efficacité et le fonctionnement en temps réel ont été validés sur plus d’une vingtaine de vidéos de chutes et de leurres, avec des résultats approchant les 100% de sensibilité et de spécificité en utilisant 4 caméras ou plus.Dans une deuxième partie, nous allons un peu plus loin dans l’exploitation des volumes reconstruits d’une personne, lors d’une tâche motrice particulière : la marche sur tapis roulant, dans un cadre de diagnostic clinique. Dans cette partie, nous analysons plus particulièrement la qualité de la marche. Pour cela nous développons le concept d’utilisation de caméras de profondeur pour la quantification de l’asymétrie spatiale au cours du mouvement des membres inférieurs pendant la marche. Après avoir détecté chaque pas dans le temps, cette méthode réalise une comparaison de surfaces de chaque jambe avec sa correspondante symétrique du pas opposé. La validation effectuée sur une cohorte de 20 sujets montre la viabilité de la démarche / This thesis is concerned with defining new clinical investigation method to assess the impact of ageing on motricity. In particular, this thesis focuses on two main possible disturbance during ageing : the fall and walk impairment. This two motricity disturbances still remain unclear and their clinical analysis presents real scientist and technological challenges. In this thesis, we propose novel measuring methods usable in everyday life or in the walking clinic, with a minimum of technical constraints.In the first part, we address the problem of fall detection at home, which was widely discussed in previous years. In particular, we propose an approach to exploit the subject’s volume, reconstructed from multiple calibrated cameras. These methods are generally very sensitive to occlusions that inevitably occur in the home and we therefore propose an original approach much more robust to these occultations. The efficiency and realtime operation has been validated on more than two dozen videos of falls and lures, with results approaching 100 % sensitivity and specificity with at least four or more cameras.In the second part, we go a little further in the exploitation of reconstructed volumes of a person at a particular motor task : the treadmill, in a clinical diagnostic. In this section we analyze more specifically the quality of walking. For this we develop the concept of using depth camera for the quantification of the spatial and temporal asymmetry of lower limb movement during walking. After detecting each step in time, this method makes a comparison of surfaces of each leg with its corresponding symmetric leg in the opposite step. The validation performed on a cohort of 20 subjects showed the viability of the approach.

Recent advancements in 3D data gathering have made it possible to measure the distance to an object at different time stamps through the use of time-of-flight cameras. Therefore, the purpose of this study was to investigate the validity and reliability of a time-of-flight camera on different mechanical sprint properties of the muscle. Fifteen male football players performed four 30m maximal sprint bouts which was simultaneously recorded with a time-of-flight camera and 1080 sprint device. By using an exponential function on the collected positional- and velocity-time data from both the devices, following variables were derived and analyzed: Maximal velocity (nmax), time constant (t), theoretical maximal force (F0), theoretical maximal velocity (V0), peak power output (Pmax), F-V mechanical profile (Sfv) and decrease in ratio of force (Drf). The results showed strong correlation in vmax along with a fairly small standard error of estimate (SEE) (r = 0,817, SEE = 0,27 m/s), while t displayed moderate correlation and relatively high SEE (r = 0,620, SEE = 0,12 s). Furthermore, moderate mean bias (>5%) were revealed for most of the variables, except for vmax and V0. The within-sessions reliability using Intraclass correlation coefficient (ICC) and standard error of measurement (SEM) ranged from excellent to poor with Pmax displaying excellent reliability (ICC = 0,91, SEM = 72W), while vmax demonstrated moderate reliability (ICC = 0,61, SEM = 0,26 m/s) and t poor(ICC = 0,44, SEM = 0,11 s). In conclusion, these findings showed that in its current state, the time-of-flight camera is not a reliable or valid device in estimating different mechanical properties of the muscle during sprint running using Samozino et al’s computations. Further development is needed.

Human behavior analysis is an important challenge in many domains, such as surveillance systems, video content retrieval, human interactive systems, medical diagnosis, etc. With the increasing needs of public safety, intelligent surveillance system becomes an activating issue in computer vision and related research fields. In this thesis we present a method to analyze human behavior in a video sequence with depth information obtained from the depth camera. When interested actions are detected in the scene, the system will trigger alarm information. Contour line and Delaunay triangulation are used to establish human posture model. By traversing the triangulation meshes with the depth first search, we obtain the spanning tree with the depth information, and then construct human posture model with this spanning tree. Posture sequence from video sequence with corresponding posture models can be obtained, and then the posture sequences is clustered into key posture sequence. By querying the key posture sequence, the system can recognize human behavior in real-time and inform users immediately when interested actions detected. Experimental results show that the system is accurate and robust for human behavior recognition.

This thesis develops new algorithms to produce 3D content from a single camera. Today, amateurs can use hand-held camcorders to capture and display the 3D world in 2D, using mature technologies. However, there is always a strong desire to record and re-explore the 3D world in 3D. To achieve this goal, current approaches usually make use of a camera array, which suffers from tedious setup and calibration processes, as well as lack of portability, limiting its application to lab experiments. In this thesis, I try to produce the 3D contents using a single camera, making it as simple as shooting pictures. It requires a new front end capturing device rather than a regular camcorder, as well as more sophisticated algorithms. First, in order to capture the highly detailed object surfaces, I designed and developed a depth camera based on a novel technique called light fall-off stereo (LFS). The LFS depth camera outputs color+depth image sequences and achieves 30 fps, which is necessary for capturing dynamic scenes. Based on the output color+depth images, I developed a new approach that builds 3D models of dynamic and deformable objects. While the camera can only capture part of a whole object at any instance, partial surfaces are assembled together to form a complete 3D model by a novel warping algorithm. Inspired by the success of single view 3D modeling, I extended my exploration into 2D-3D video conversion that does not utilize a depth camera. I developed a semi-automatic system that converts monocular videos into stereoscopic videos, via view synthesis. It combines motion analysis with user interaction, aiming to transfer as much depth inferring work from the user to the computer. I developed two new methods that analyze the optical flow in order to provide additional qualitative depth constraints. The automatically extracted depth information is presented in the user interface to assist with user labeling work. In this thesis, I developed new algorithms to produce 3D contents from a single camera. Depending on the input data, my algorithm can build high fidelity 3D models for dynamic and deformable objects if depth maps are provided. Otherwise, it can turn the video clips into stereoscopic video.

Multiterminal (MT) video coding is a practical application of the MT source coding theory. For MT source coding theory, two problems associated with achievable rate regions are well investigated into in this thesis: a new sufficient condition for BT sum-rate tightness, and the sum-rate loss for quadratic Gaussian MT source coding. Practical code design for ideal Gaussian sources with quadratic distortion measure is also achieved for cases more than two sources with minor rate loss compared to theoretical limits. However, when the theory is applied to practical applications, the performance of MT video coding has been unsatisfactory due to the difficulty to explore the correlation between different camera views. In this dissertation, we present an MT video coding scheme under the H.264/AVC framework. In this scheme, depth camera information can be optionally sent to the decoder separately as another source sequence. With the help of depth information at the decoder end, inter-view correlation can be largely improved and thus so is the compression performance. With the depth information, joint estimation from decoded frames and side information at the decoder also becomes available to improve the quality of reconstructed video frames. Experimental result shows that compared to separate encoding, up to 9.53% of the bit rate can be saved by the proposed MT scheme using decoder depth information, while up to 5.65% can be saved by the scheme without depth camera information. Comparisons to joint video coding schemes are also provided.

Master of Science / Department of Industrial & Manufacturing Systems Engineering / Shing I. Chang / This research aims to quantify human walking patterns through depth cameras to (1) detect walking pattern changes of a person with and without a motion-restricting device or a walking aid, and to (2) identify distinct walking patterns from different persons of similar physical attributes. Microsoft Kinect™ devices, often used for video games, were used to provide and track coordinates of 25 different joints of people over time to form a human skeleton. Two main studies were conducted. The first study aims at deciding whether motion-restricted devices such as a knee brace, an ankle brace, or walking aids – walkers or canes affect a person’s walking pattern or not. This study collects gait data from ten healthy subjects consisting of five females and five males walking a 10-foot path multiple times with and without motion-restricting devices. Their walking patterns were recorded in a form of time series via two Microsoft Kinect™ devices through frontal and sagittal planes. Two types of statistics were generated for analytic purposes. The first type is gait parameters converted from Microsoft Kinect™ coordinates of six selected joints. Then Sample Entropy (SE) measures were computed from the gait parameter values over time. The second method, on the other hand, applies the SE computations directly on the raw data derived from Microsoft Kinect™ devices in terms of (X, Y, Z) coordinates of 15 selected joints over time. The SE values were then used to compare the changes in each joint with and without motion-restricting devices. The experimental results show that both types of statistics are capable of detecting differences in walking patterns with and without motion-restricting devices for all ten subjects. The second study focuses on distinguishing two healthy persons with similar physical conditions. SE values from three gait parameters were used to distinguish one person from another via their walking patterns. The experimental results show that the proposed method using a star glyph summarizing the shape produced by the gait parameters is capable of distinguishing these two persons. Then multiple machine learning (ML) models were applied to the SE datasets from ten college-age subjects - five males and five females. In particular, ML models were applied to classify subjects into two categories: normal walking and abnormal walking (i.e. with motion-restricting devices). The best ML model (K-nearest neighborhood) was able to predict 97.3% accuracy using 10-fold cross-validation. Finally, ML models were applied to classify five gait conditions: walking normally, walking while wearing the ankle brace, walking while wearing the ACL brace, walking while using a cane, and walking while using a walker. The best ML model was again the K-nearest neighborhood performing at 98.7% accuracy rate.

Multiple object tracking (MOT) can be an efficient tool for finding patterns in video monitoring data. In this thesis, we investigate which type of video data works best for MOT in an indoor combat training scenario. The three types of camera data evaluated are color data, near-infrared (NIR) data, and depth data. In order to evaluate which of these lend themselves best for MOT, we develop object tracking models based on YOLOv5 and DeepSORT, and train the models on the respective types of data. In addition to the individual models, ensembles of the three models are also developed, to see if any increase in performance can be gained. The models are evaluated using the well-established MOT evaluation metrics, as well as studying the frame rate performance of each model. The results are rigorously analyzed using statistical significance tests, to ensure only well-supported conclusions are drawn. These evaluations and analyses show mixed results. Regarding the MOT metrics, the performance of most models were not shown to be significantly different from most other models, so while a difference in performance was observed, it cannot be assumed to hold over larger sample sizes. Regarding frame rate, we find that the ensemble models are significantly slower than the individual models on their own.

Le vieillissement de la population est un enjeu majeur pour les prochaines années en raison, notamment, de l'augmentation du nombre de personnes dépendantes. La question du maintien à domicile de ces personnes se pose alors, du fait de l'impossibilité pour les instituts spécialisés de les accueillir toutes et, surtout, de la volonté des personnes âgées de rester chez elles le plus longtemps possible. Or, le développement de systèmes technologiques peut aider à résoudre certains problèmes comme celui de la sécurisation en détectant les chutes, et de l'évaluation du degré d'autonomie pour prévenir les accidents. Plus particulièrement, nous nous intéressons au développement des systèmes ambiants, peu coûteux, pour l'équipement du domicile. Les caméras de profondeur permettent d'analyser en temps réel les déplacements de la personne. Nous montrons dans cette thèse qu'il est possible de reconnaître l'activité de la personne et de mesurer des paramètres de sa marche à partir de l'analyse de caractéristiques simples extraites des images de profondeur. La reconnaissance d'activité est réalisée à partir des modèles de Markov cachés, et permet en particulier de détecter les chutes et des activités à risque. Lorsque la personne marche, l'analyse de la trajectoire du centre de masse nous permet de mesurer les paramètres spatio-temporels pertinents pour l'évaluation de la fragilité de la personne. Ce travail a été réalisé sur la base d'expérimentations menées en laboratoire, d'une part, pour la construction des modèles par apprentissage automatique et, d'autre part, pour évaluer la validité des résultats. Les expérimentations ont montré que certains modèles de Markov cachés, développés pour ce travail, sont assez robustes pour classifier les différentes activités. Nous donnons, également dans cette thèse, la précision, obtenue avec notre système, des paramètres de la marche en comparaison avec un tapis actimètrique. Nous pensons qu'un tel système pourrait facilement être installé au domicile de personnes âgées, car il repose sur un traitement local des images. Il fournit, au quotidien, des informations sur l'analyse de l'activité et sur l'évolution des paramètres de la marche qui sont utiles pour sécuriser et évaluer le degré de fragilité de la personne. / Population ageing is a major issue for society in the next years, especially because of the increase of dependent people. The limits in specialized institutes capacity and the wish of the elderly to stay at home as long as possible explain a growing need for new specific at home services. Technologies can help securing the person at home by detecting falls. They can also help in the evaluation of the frailty for preventing future accidents. This work concerns the development of low cost ambient systems for helping the stay at home of elderly. Depth cameras allow analysing in real time the displacement of the person. We show that it is possible to recognize the activity of the person and to measure gait parameters from the analysis of simple feature extracted from depth images. Activity recognition is based on Hidden Markov Models and allows detecting at risk behaviours and falls. When the person is walking, the analysis of the trajectory of her centre of mass allows measuring gait parameters that can be used for frailty evaluation. This work is based on laboratory experimentations for the acquisition of data used for models training and for the evaluation of the results. We show that some of the developed Hidden Markov Models are robust enough for classifying the activities. We also evaluate de precision of the gait parameters measurement in comparison to the measures provided by an actimetric carpet. We believe that such a system could be installed in the home of the elderly because it relies on a local processing of the depth images. It would be able to provide daily information on the person activity and on the evolution of her gait parameters that are useful for securing her and evaluating her frailty

Natural human motion is highly demanded and widely used in a variety of applications such as video games and virtual realities. However, acquisition of full-body motion remains challenging because the system must be capable of accurately capturing a wide variety of human actions and does not require a considerable amount of time and skill to assemble. For instance, commercial optical motion capture systems such as Vicon can capture human motion with high accuracy and resolution while they often require post-processing by experts, which is time-consuming and costly. Microsoft Kinect, despite its high popularity and wide applications, does not provide accurate reconstruction of complex movements when significant occlusions occur. This dissertation explores two different approaches that accurately reconstruct full-body human motion from noisy and ambiguous input data captured by commercial motion capture devices. The first approach automatically generates high-quality human motion from noisy data obtained from commercial optical motion capture systems, eliminating the need for post-processing. The second approach accurately captures a wide variety of human motion even under significant occlusions by using color/depth data captured by a single Kinect camera. The common theme that underlies two approaches is the use of prior knowledge embedded in pre-recorded motion capture database to reduce the reconstruction ambiguity caused by noisy and ambiguous input and constrain the solution to lie in the natural motion space. More specifically, the first approach constructs a series of spatial-temporal filter bases from pre-captured human motion data and employs them along with robust statistics techniques to filter noisy motion data corrupted by noise/outliers. The second approach formulates the problem in a Maximum a Posterior (MAP) framework and generates the most likely pose which explains the observations as well as consistent with the patterns embedded in the pre-recorded motion capture database. We demonstrate the effectiveness of our approaches through extensive numerical evaluations on synthetic data and comparisons against results created by commercial motion capture systems. The first approach can effectively denoise a wide variety of noisy motion data, including walking, running, jumping and swimming while the second approach is shown to be capable of accurately reconstructing a wider range of motions compared with Microsoft Kinect.

La previz on-set est une étape de prévisualisation qui a lieu directement pendant la phase de tournage d’un film à effets spéciaux. Cette proposition de prévisualisation consiste à montrer au réalisateur une vue assemblée du plan final en temps réel. Le travail présenté dans cette thèse s’intéresse à une étape spécifique de la prévisualisation : le compositing. Cette étape consiste à mélanger plusieurs sources d’images pour composer un plan unique et cohérent. Dans notre cas, il s’agit de mélanger une image de synthèse avec une image issue de la caméra présente sur le plateau de tournage. Les effets spéciaux numériques sont ainsi ajoutés à la prise de vue réelle. L’objectif de cette thèse consiste donc à proposer un système permettant l’ajustement automatique du mélange entre les deux images. La méthode proposée nécessite la mesure de la géométrie de la scène filmée. Pour cette raison, un capteur de profondeur est ajouté à la caméra de tournage. Les données sont relayées à l’ordinateur qui exécute un algorithme permettant de fusionner les données du capteur de profondeur et de la caméra de tournage. Par le biais d’un démonstrateur matériel, nous avons formalisé une solution intégrée dans un moteur de jeux vidéo. Les expérimentations menées montrent dans un premier temps des résultats encourageants pour le compositing en temps réel. Nous avons observé une amélioration des résultats suite à l’introduction de la méthode de segmentation conjointe. La principale force de ce travail réside dans la mise en place du démonstrateur qui nous a permis d’obtenir des algorithmes efficaces dans le domaine de la previz on-set. / Previz on-set is a preview step that takes place directly during the shootingphase of a film with special effects. The aim of previz on-set is to show to the film director anassembled view of the final plan in realtime. The work presented in this thesis focuses on aspecific step of the previz : the compositing. This step consists in mixing multiple images tocompose a single and coherent one. In our case, it is to mix computer graphics with an imagefrom the main camera. The objective of this thesis is to propose a system for automaticadjustment of the compositing. The method requires the measurement of the geometry ofthe scene filmed. For this reason, a depth sensor is added to the main camera. The data issent to the computer that executes an algorithm to merge data from depth sensor and themain camera. Through a hardware demonstrator, we formalized an integrated solution in avideo game engine. The experiments gives encouraging results for compositing in real time.Improved results were observed with the introduction of a joint segmentation method usingdepth and color information. The main strength of this work lies in the development of ademonstrator that allowed us to obtain effective algorithms in the field of previz on-set.