Although upper limb motor impairments are common, the primary tools for assessing and tracking these impairments in a clinical setting are subjective, qualitative rating scales that lack resolution and repeatability. Markerless motion capture technology has the potential to greatly improve clinical assessment by providing quick, low-cost, and accurate tools to objectively quantify motor deficits. Here we lay some of the groundwork necessary to enable markerless motion capture systems to be used in clinical settings. First, we adapted five motor tests common in clinical assessments so they can be administered via markerless motion capture. We implemented these modified tests using a particular motion capture sensor (Leap MotionTM Controller, hereafter referred to as the Leap Motion sensor) and administered the tests to 100 healthy subjects to evaluate the feasibility of administrating these tests via markerless motion capture. Second, to determine the ability of the Leap Motion sensor to accurately measure tremor, we characterized the frequency response of the Leap Motion sensor. During the administration of the five modified motor tests on 100 healthy subjects, the subjects had little trouble interfacing with the Leap Motion sensor and graphical user interface, performing the tasks with ease. The Leap Motion sensor maintained an average sampling rate above 106 Hz across all subjects during each of the five tests. The rate of adverse events caused by the Leap Motion sensor (mainly jumps in time or space) was generally below 1%. In characterizing the frequency response of the Leap Motion sensor, we found its bandwidth to vary between 1.7 and 5.5 Hz for actual tremor amplitudes above 1.5 mm, with larger bandwidth for larger amplitudes. To improve the accuracy of tremor measurements, we provide the magnitude ratios that can be used to estimate the actual amplitude of the oscillations from the measurements by the Leap Motion sensor. These results suggest that markerless motion capture systems are on the verge of becoming suitable for routine clinical use, but more work is necessary to further improve the motor tests before they can be administered via markerless motion capture with sufficient robustness for clinical settings.

Dopo l’avvento del sonoro e quello del colore, la settima arte sta attraversando una terza, forse più radicale, fase di rivoluzione: un vero e proprio tsunami che corrisponde all’avvento del digitale, dove il video insidia il primato della pellicola e la Computer Grafica è sempre più dilagante nel campo degli effetti speciali. Muovendosi all’interno di questo quadro di sostanziale cambiamento la presente ricerca, intitolata "Il corpo nell’era digitale: dal simulacro alla Performance Capture", si sofferma in particolare sull’analisi delle nuove prospettive che il corpo (attoriale) deve affrontare con la smaterializzazione dell’elemento organico. A partire da una panoramica storica sull’avvento del digitale nel cinema contemporaneo, si arriva a tracciare una linea evolutiva del corpo e del volto per meglio contestualizzare i campi di azione della Performance Capture. Il digitale e la Performance Capture sono fenomeni il cui sviluppo è ancora pienamente in atto, dagli esiti futuri ancora imprevedibili. Il proposito di questa ricerca è stabilire se il cinema stia diventando come paventa Lev Manovich, “a slave to the computer” o se, al contrario, le nuove tecnologie possano rafforzare le capacità artistiche, e perciò profondamente umane, del mezzo. / After the revolutionary phases brought by the introduction of sound and color, the seventh art is going through another, even more radical, renewal: the digital tsunami, where video is taking over the film and Computer Graphic is ruling the field of special effects. This project , situated inside a perspective of substantial change, is focused on the analysis of the role of the new actor and of his body, reconstructed after the disappearance of the organic elements in favor of the digital ones. Starting from an historical overview on the advent of digital cinema, this research aims to follow the development of Performance Capture, a technique capable to rewrite the rules of acting and directing. Digital moviemaking, CGI and Performance Capture, in particular, are phenomena in constant development, whose future effects are still not predictable. The purpose of this research is to establish whether cinema is becoming, like Manovich suggests, like Lev Manovich says, a “slave to the computer” , or if, on the contrary, new technologies will be able to strengthen its artistic, therefore human, sides.

In this thesis we develop a class of nonlinear generative models for high-dimensional time series. The first key property of these models is their distributed, or "componential" latent state, which is characterized by binary stochastic variables which interact to explain the data. The second key property is the use of an undirected graphical model to represent the relationship between latent state (features) and observations. The final key property is composability: the proposed class of models can form the building blocks of deep networks by successively training each model on the features extracted by the previous one. We first propose a model based on the Restricted Boltzmann Machine (RBM) that uses an undirected model with binary latent variables and real-valued "visible" variables. The latent and visible variables at each time step receive directed connections from the visible variables at the last few time-steps. This "conditional" RBM (CRBM) makes on-line inference efficient and allows us to use a simple approximate learning procedure. We demonstrate the power of our approach by synthesizing various motion sequences and by performing on-line filling in of data lost during motion capture. We also explore CRBMs as priors in the context of Bayesian filtering applied to multi-view and monocular 3D person tracking. We extend the CRBM in a way that preserves its most important computational properties and introduces multiplicative three-way interactions that allow the effective interaction weight between two variables to be modulated by the dynamic state of a third variable. We introduce a factoring of the implied three-way weight tensor to permit a more compact parameterization. The resulting model can capture diverse styles of motion with a single set of parameters, and the three-way interactions greatly improve its ability to blend motion styles or to transition smoothly among them. In separate but related work, we revisit Products of Hidden Markov Models (PoHMMs). We show how the partition function can be estimated reliably via Annealed Importance Sampling. This enables us to demonstrate that PoHMMs outperform various flavours of HMMs on a variety of tasks and metrics, including log likelihood.

Potentially transformational ideas in several applications of human and computer interaction form the motivation for this work. It is targeted towards a systematic approach to the design of systems with complex, intelligent and interactive exchange of information between a system and the environment it is meant to monitor, and gather knowledge about. The Pahl and Beitz systematic design method is modified with the inclusion of generic sub-systems from Living Systems Theory, modeling and simulation tools and other adaptations within the context of the validation square to synthesize a design method for the design of systems with intelligent and interactive information flow. The validation of the proposed design method is carried out with the aid of an example wherein a motion capture system is designed based on the Nintendo Wii Remote(TM). Results include an evaluation of the performance of a prototype as well as the design method itself in the context of the requirements that the method must fulfill.

With the rise autonomous and robotic systems in field applications, the need for dexterous, highly adaptable end effectors has become a major research topic. Control mechanisms of robotics hands with a high number independent actuators is recognized as a complex, high dimensional problem, with exponentially complex algorithms. However, recent studies have shown that human hand motion possesses very high joint correlation which translates into a set of predefined postures, or synergies. The hand produces a motion using a complementing contribution of multiple joints, called synergies. The similarities place variables onto a common dimensional space, effectively reducing the number of independent variables. In this thesis, we analyze the motion of the hand during a set of objects grasps using mul- tivariate Principal Component Analysis (mPCA) to extract both the principal variables and their correlation during grasping. We introduce the use of Functional PCA (fPCA) primarily on princi- pal components to study the dynamic requirements of the motion. The goal is to defined a set of synergies common and specific to all motions. We expand the analysis by classifying the objects grasps, or tasks, using their functional components, or harmonics over the entire motion. A set of groups are described based on these classification that confirms empirical findings. Lastly, we evaluate the motions generated from the analysis by applying them onto robotic hands. The results from the mPCA and fPCA procedures are used to map the principal components from each motion onto underactuated robotic designs. We produce a viable routine that indicates how the mapping is performed, and finally, we implement the motion generated onto a real hand. The resultant robotic motion was evaluated on how it mimics the human motion.

Αντικείμενο της παρούσας διπλωματικής εργασίας είναι αρχικά η καταγραφή της ανθρώπινης κίνησης με κάποια συσκευή παρακολούθησης και κατόπιν η δημιουργία ενός αντιπροσωπευτικού μοντέλου, ώστε να μπορεί να μελετηθεί η δυναμική του συμπεριφορά. Ως συσκευή καταγραφής χρησιμοποιήθηκε ο αισθητήρας Kinect της Microsoft. Το μοντέλο που αναπτύχθηκε αφορά κυρίως τα κάτω άκρα του ανθρώπου και επιπλέον διαθέτει μυοσκελετική δομή με 86 μύες. Στα πλαίσια των αναλύσεων χρησιμοποιήθηκαν διάφορες τεχνικές για την εξαγωγή των αποτελεσμάτων, όπως είναι η αντίστροφη κινηματική, αντίστροφη δυναμική, υπολογισμός μυϊκών διεγέρσεων και ορθή δυναμική και προτείνουμε μια στρατηγική για την ανάλυση και την εξαγωγή αποτελεσμάτων. / The research developed in this thesis first deal with the problem of capturing the human body motion and then concentrates on the creation of musculoskeletal models, which can capture and accurately study its dynamical behavior. The Microsoft's Kinect sensor was utilized to capture the human motion. The model used for the simulations is the human lower extremity with 86 attached muscles. For the analysis phase we used some common methods such as inverse kinematics, inverse dynamics, computed muscle control and forward dynamics and we showed a general pipeline strategy for generating correct results.

In this thesis we develop a class of nonlinear generative models for high-dimensional time series. The first key property of these models is their distributed, or "componential" latent state, which is characterized by binary stochastic variables which interact to explain the data. The second key property is the use of an undirected graphical model to represent the relationship between latent state (features) and observations. The final key property is composability: the proposed class of models can form the building blocks of deep networks by successively training each model on the features extracted by the previous one. We first propose a model based on the Restricted Boltzmann Machine (RBM) that uses an undirected model with binary latent variables and real-valued "visible" variables. The latent and visible variables at each time step receive directed connections from the visible variables at the last few time-steps. This "conditional" RBM (CRBM) makes on-line inference efficient and allows us to use a simple approximate learning procedure. We demonstrate the power of our approach by synthesizing various motion sequences and by performing on-line filling in of data lost during motion capture. We also explore CRBMs as priors in the context of Bayesian filtering applied to multi-view and monocular 3D person tracking. We extend the CRBM in a way that preserves its most important computational properties and introduces multiplicative three-way interactions that allow the effective interaction weight between two variables to be modulated by the dynamic state of a third variable. We introduce a factoring of the implied three-way weight tensor to permit a more compact parameterization. The resulting model can capture diverse styles of motion with a single set of parameters, and the three-way interactions greatly improve its ability to blend motion styles or to transition smoothly among them. In separate but related work, we revisit Products of Hidden Markov Models (PoHMMs). We show how the partition function can be estimated reliably via Annealed Importance Sampling. This enables us to demonstrate that PoHMMs outperform various flavours of HMMs on a variety of tasks and metrics, including log likelihood.

L'analyse de la marche et la mesure du déplacements des articulations humaines ont été largement étudiées. Les artefacts de tissus "mous" sont une source fréquente d'erreur pour la plupart des méthodes de mesure utilisées. La procédure standard en analyse de la marche consiste à utiliser une combinaison de mesures pour l'estimation efficace des angles articulaires et de la position des segments du corps humain. Ce travail propose le développement d'un système d'analyse de la marche reposant sur un robot parallèle à câbles équipé de plusieurs capteurs mesurant spécifiquement les déplacements du genou. Nous considérons le cas général pour lequel les articulations humaines se comportent comme des joints à 6 degrés de liberté reliant deux segments du corps. Afin de déterminer la position et l'orientation de ces segments, 14 câbles y sont attachés, ce qui permet de considérer ces segments comme les organes effecteurs de robots parallèles. Leur position peut alors être calculée à partir de la mesure de la longueur des câbles. Cependant, ces mesures sont entachées de bruit à cause des artefacts de tissus "mous". Afin d'améliorer la précision des résultats, le système propose aussi l'utilisation d'autres capteurs de nature différente : plusieurs capteurs inertiels (avec accéléromètres et gyroscopes), un système de motion capture, des capteurs de pression plantaire, des capteurs de distance (IR et résistance variable) et des capteurs de force pour mesurer la contraction musculaire. Plusieurs approches globales sont disponibles pour l'analyse du genou lors de la marche. Les choix technologiques effectués impactent directement sur la conception de notre système et imposent le développement de matériel spécifique pour mener à bien les mesures, tel que le collier flexible utilisé d'une part pour permettre l'attache des câbles sur les segments du patient et d'autres part pour supporter les capteurs supplémentaires. Nous traitons le collier comme une chaîne cinématique sérielle et nous proposons une méthode d'étalonnage qui ne nécessite pas d'utiliser les mesures angulaires des articulations contrairement aux méthodes existantes. Nous décrivons le protocole expérimental ainsi que les méthodes utilisées pour synchroniser les données issues de plusieurs ordinateurs. Les données sont ensuite fusionnées pour obtenir la pose du collier et donc celle des segments du patient. Enfin, ce travail permet d'identifier les modifications à apporter au système pour une meilleure analyse de la marche, ce qui pourra servir de base à un système de rééducation complet.

Time series data arise in many applications, from motion capture, environmental monitoring, temperatures in data centers, to physiological signals in health care. In the thesis, I will focus on the theme of learning and mining large collections of co-evolving sequences, with the goal of developing fast algorithms for finding patterns, summarization, and anomalies. In particular, this thesis will answer the following recurring challenges for time series: 1. Forecasting and imputation: How to do forecasting and to recover missing values in time series data? 2. Pattern discovery and summarization: How to identify the patterns in the time sequences that would facilitate further mining tasks such as compression, segmentation and anomaly detection? 3. Similarity and feature extraction: How to extract compact and meaningful features from multiple co-evolving sequences that will enable better clustering and similarity queries of time series? 4. Scale up: How to handle large data sets on modern computing hardware? We develop models to mine time series with missing values, to extract compact representation from time sequences, to segment the sequences, and to do forecasting. For large scale data, we propose algorithms for learning time series models, in particular, including Linear Dynamical Systems (LDS) and Hidden Markov Models (HMM). We also develop a distributed algorithm for finding patterns in large web-click streams. Our thesis will present special models and algorithms that incorporate domain knowledge. For motion capture, we will describe the natural motion stitching and occlusion filling for human motion. In particular, we provide a metric for evaluating the naturalness of motion stitching, based which we choose the best stitching. Thanks to domain knowledge (body structure and bone lengths), our algorithm is capable of recovering occlusions in mocap sequences, better in accuracy and longer in missing period. We also develop an algorithm for forecasting thermal conditions in a warehouse-sized data center. The forecast will help us control and manage the data center in a energy-efficient way, which can save a significant percentage of electric power consumption in data centers.

動作擷取法是現今相當受到歡迎的角色動作產生方法，而一般多是使用已擷取好的動作，以人工的方式將數個不同的動作混合以產生出所需的動作。但想要大量產生符合需求的混合動作仍相當不容易，因此有人提出了「動作圖」這個方法。動作圖是一種根據使用者所給定的動作擷取資料集合，經過自動化的計算找出各個動作資料之間可以連接的動作片段。藉由這個自動化的程序，各個動作擷取資料可以相互連接起來，達到在不同的動作間平順轉換，且同時保有原動作擷取資料擬真特性的目的。但縱使有上述的好處，目前動作圖的技術僅能就所擷取的全身動作進行串接，品質與彈性往往決定於一開始動作擷取資料的準備，因此如何讓既有的全身動作資料得以分解再利用，以發揮最大的價值，是一個重要的問題。在本研究中，我們提出了一個階層式的動作圖結構名為多層式動作圖，在這個多層式動作圖的結構中，我們將身體的動作區分成數個部位，分別計算各自的動作圖後再合併成一個多層式的架構，而合併的過程中我們提出「整體動作相似度」的計算方式，以做為兩個動作是否容易轉接的比較依據。我們也提出了在不同階層間動作圖運作的規則，以使計算的複雜度及系統的可用性取得合理的平衡。此外，我們更進一步提出名為Motion Script的簡易語意描述語言，來輔助控制這個具有高複雜度的動作圖結構。實驗的結果顯示，我們的方法可以即時根據使用者的指令，搜尋並產生出原動作資料所沒有的動作組合。與傳統的動作圖相比，我們的方法能更進一步的發揮原動作擷取資料的價值，以有系統的方式讓動作組合自動產生更具豐富性及彈性。 / Motion capture is a popular method for generating realistic character animation. In most applications, a motion usually is prepared by manually blending existing captured motion clips to generate a desired motion clip. However, finding a good transition points manually for two motion clips is a time-consuming task and cannot be scaled up easily. Motion Graph is a technique that has been proposed to automate this process by finding suitable connection points and the corresponding transition motions between motion data. With this automatic procedure, motions captured separately can be smoothly connected while keeping the realism of the captured motions. However, most motion graph techniques only consider the transition of full-body motions in two motion clips, and therefore, the resulting motion .depends on the variety of motions available in the motion database. It is an important issue to be able to compose new motion clips as much as possible with given motion capture database. In this research, we propose a hierarchical motion graph structure called Multi-Layered Motion Graph. In this structure, we divide motion data into layers of parts depending on the articulated structure of human body, and then compute a motion graph for each part of the motion. We then combine these motion graphs into an interconnected hierarchical structure. In order to facilitate the composition of motions for different parts from different motion clips, we propose a new metric called Overall Motion Similarity to find reasonable composition of motions in run time. We also propose several rules about how to traverse the motion graphs in different layers to generate feasible motions. Furthermore, we have designed a scripting language called Motion Script to facilitate the specification and search of desirable animation to be generated. Our experimental results reveal that our method is able to compose animations that the original motion graph cannot generate in real time. Compared to the traditional motion graph method, our method is able to make good use of existing motion capture library to compose new motions in a systematic way.