With the recent advances in computer vision and pattern recognition, methods from these fields are successfully applied to solve problems in various domains, including health care and social sciences. In this thesis, two such problems, from different domains, are discussed. First, an application of computer vision and broader pattern recognition in physical therapy is presented. Home-based physical therapy is an essential part of the recovery process in which the patient is prescribed specific exercises in order to improve symptoms and daily functioning of the body. However, poor adherence to the prescribed exercises is a common problem. In our work, we explore methods for improving home-based physical therapy experience. We begin by proposing DyAd, a dynamically difficulty adjustment system which captures the trajectory of the hand movement, evaluates the user's performance quantitatively and adjusts the difficulty level for the next trial of the exercise based on the performance measurements. Next, we introduce ExerciseCheck, a remote monitoring and evaluation platform for home-based physical therapy. ExerciseCheck is capable of capturing exercise information, evaluating the performance, providing therapeutic feedback to the patient and the therapist, checking the progress of the user over the course of the physical therapy, and supporting the patient throughout this period. In our experiments, Parkinson patients have tested our system at a clinic and in their homes during their physical therapy period. Our results suggests that ExerciseCheck is a user-friendly application and can assist patients by providing motivation, and guidance to ensure correct execution of the required exercises. As the second application, and within computer vision paradigm, we focus on visual complexity, an image attribute that humans can subjectively evaluate based on the level of details in the image. Visual complexity has been studied in psychophysics, cognitive science, and, more recently, computer vision, for the purposes of product design, web design, advertising, etc. We first introduce a diverse visual complexity dataset which compromises of seven image categories. We collect the ground-truth scores by comparing the pairwise relationship of images and then convert the pairwise scores to absolute scores using mathematical methods. Furthermore, we propose a method to measure the visual complexity that uses unsupervised information extraction from intermediate convolutional layers of deep neural networks. We derive an activation energy metric that combines convolutional layer activations to quantify visual complexity. The high correlations between ground-truth labels and computed energy scores in our experiments show superiority of our method compared to the previous works. Finally, as an example of the relationship between visual complexity and other image attributes, we demonstrate that, within the context of a category, visually more complex images are more memorable to human observers.

Nowadays Virtual Reality (VR) and Human Robot Collaboration (HRC) are becoming more and more important in Industry as well as science. This investigation studies the applications of these two technologies in the ergonomic field by developing a system able to visualise and present ergonomics evaluation results in real time assembly tasks in a VR Environment, and also, evaluating the advantages of Human Robot Collaboration by studying in Virtual Reality a specific operation carried at Volvo Global Trucks Operation´s factory in Skövde. Regarding the first part of this investigation an innovative system was developed able to show ergonomic feedbacks in real time, as well as make ergonomic evaluations of the whole workload inside of a VR environment. This system can be useful for future research in the Virtual Ergonomics field regarding matters related to ergonomic learning rate of the workers when performing assembly tasks, design of ergonomic workstations, effect of different types assembly instructions in VR and a wide variety of different applications. The assembly operation with and without robot was created in IPS to use its VR functionality in order to test the assembly task in real users with natural movements of the body. The posture data of the users performing the tasks in Virtual Reality was collected. The users performed the task without the collaborative robot and then, with the collaborative robot. Their posture data was collected by using a Motion Capture equipment called Smart Textiles (developed at the University of Skövde) and the two different ergonomic evaluations (Using Smart Textiles’ criteria) of the two different task compared. The results show that when the robot implemented in this specific assembly task, the posture of the workers (specially the posture of the arms) has a great improvement if it is compared to the same task without the robot.

Ageing is linked to a decrease in mobility, which affects the quality of life of many elderly individuals. This is a growing challenge in industrialised societies since the proportion of elderly individuals is becoming larger. One potential solution that would keep these individuals active and independent is the use of mobility assist devices. These devices are designed to reduce the energy demand of the user with the use of electric motors providing torques at joints of the lower limb. Although promising, these devices have a problem: they become uncomfortable after prolonged usage. This is especially true for devices designed to produce substantial assistance. The research goal consisted of quantifying the performance of the physical interfaces, or points of attachments, of an experimental device with multiple interface adjustments. The device was fabricated with design criteria similar to active assist devices to simulate the mechanical behaviour of these particular devices. This analysis provided design recommendations that could ultimately enhance the performance of assist devices available on the market and thus the quality of life of many individuals. This research used force mapping and motion capture to quantify the kinetic and the kinematic compatibility of the device. Experimental results have shown that the position, shape and other parameters of the interfaces had an effect on the relative movement of the brace, or the brace performance. The device interface migration was greater when the interfaces were positioned furthest away form the joint. An increasing level of assistance showed more relative movement between the brace and the user. Interface geometry had a noticeable effect on force distribution over the interface. The results and methodology of this research offers an in depth understanding of the mechanical behavior of the physical interfaces of the developed assist device. Nevertheless, further research and development in the field of human machine interactions are needed in order to develop a physical human-machine interface that will ensure the success of powered assist devices in the future.

Le sens du toucher joue un rôle fondamental dans la façon dont nous percevons notre environnement, nous déplaçons, et interagissons délibérément avec d'autres objets ou êtres vivants. Ainsi, les forces de contact informent à la fois sur l'action réalisée et sa motivation. Néanmoins, l'utilisation de capteurs de force traditionnels est coûteuse, lourde, et intrusive. Dans cette thèse, nous examinons la perception haptique par la capture de mouvement. Ce problème est difficile du fait qu'un mouvement donné peut généralement être causé par une infinité de distributions de forces possibles, en multi-contact. Dans ce type de situations, l'optimisation sous contraintes physiques seule ne permet que de calculer des distributions de forces plausibles, plutôt que fidèles à celles appliquées en réalité. D'un autre côté, les méthodes d'apprentissage de type `boîte noire' pour la modélisation de structures cinématiquement et dynamiquement complexes sont sujettes à des limitations en termes de capacité de généralisation. Nous proposons une formulation du problème de la distribution de forces exploitant ces deux approches ensemble plutôt que séparément. Nous capturons ainsi la variabilité dans la façon dont on contrôle instinctivement les forces de contact tout en nous assurant de leur compatibilité avec le mouvement observé. Nous présentons notre approche à la fois pour la manipulation et les interactions corps complet avec l'environnement. Nous validons systématiquement nos résultats avec des mesures de référence et fournissons des données exhausives pour encourager et évaluer les travaux futurs sur ce nouveau sujet. / The human sense of touch is of fundamental importance in the way we perceive our environment, move ourselves, and purposefully interact with other objects or beings. Thus, contact forces are informative on both the realized task and the underlying intent. However, monitoring them with force transducers is a costly, cumbersome and intrusive process. In this thesis, we investigate the capture of haptic information from motion tracking. This is a challenging problem, as a given motion can generally be caused by an infinity of possible force distributions in multi-contact. In such scenarios, physics-based optimization alone may only capture force distributions that are physically compatible with a given motion, rather than those really applied. In contrast, machine learning techniques for the black-box modelling of kinematically and dynamically complex structures are often prone to generalization issues. We propose a formulation of the force distribution problem utilizing both approaches jointly rather than separately. We thus capture the variability in the way humans instinctively regulate contact forces while also ensuring their compatibility with the observed motion. We present our approach on both manipulation and whole-body interaction with the environment. We consistently back our findings with ground-truth measurements and provide extensive datasets to encourage and serve as benchmarks for future research on this new topic.

Conselho Nacional de Desenvolvimento Científico e Tecnológico / O objetivo do presente trabalho foi avaliar comparativamente parâmetros biomecânicos (tanto antropométricos, quanto cinemáticos) de dados obtidos a partir do console Microsoft Kinect (2010). A avaliação destes parâmetros foi realizada para validar seu uso para obter informações complementares à Análise Ergonomica do Trabalho (AET) e em outras pesquisas, cujos objetivos envolvem o diagnóstico de uso de produtos ou ambientes de trabalho a partir da análises posturais e interações da população que o utiliza. A pesquisa com este console em particular é justificada uma vez que seu lançamento modificou o cenário da biomecânica, já que se trata de um equipamento acessível e portátil. Porém, sua precisão em relação à outros equipamentos ainda está em aberto, sendo inclusive, objeto de estudo de muitas pesquisas em andamento. Os dados obtidos por meio de sistemas de captura de movimentos tridimensionais permitem a avaliação de produtos, atividades e análises de interações homem-objeto. No campo do Design, é uma importante realização, uma vez que permite que profissionais tenham acesso à ferramenta que, anteriormente, era limitada à nichos especializados. O console foi comparado com o sistema de captura de movimentos inercial MVN Biomech (XSENS TECHNOLOGIES) e com o tradicional registro por meio de vídeo. Para obter dados do console Kinect, um software disponível no mercado foi selecionado a partir de critérios predefinidos para obter dados cinemáticos do console. Dois experimentos laboratoriais foram realizados: o primeiro, teve como objetivo obter dados operacionais dos equipamentos e suas limitações de uso; e o segundo foi realizado de forma a obter dados biomecânicos e compará-los a partir de três parâmetros estáticos e um dinâmico. Os parâmetros estáticos envolveram ângulos articulares e segmentares em posturas selecionadas e dimensões segmentares, onde a proposta foi avaliar dados antropométricos e as características do modelo biomecânico referente à manter os corpos rígidos durante a movimentação. O parâmetro dinâmico foi realizado de forma a obter dados de deslocamento global das articulações em movimentações selecionadas. Para possibilitar esta análise, uma plataforma digital foi desenvolvida, constituindo um campo neutro para o tratamento dos dados. A plataforma mantém os dados originais dos sistemas, permitindo a distinção entre os modelos biomecânicos e a retirada de dados que possam ser comparados. Os experimentos realizados permitiram avaliar a usabilidade do console, fornecendo diretrizes para seu uso. Para avaliar a utilização do console em ambientes reais de trabalho, foram realizados registros preliminares em laboratórios químicos, os quais se mostraram viáveis se as limitações, semelhantes às de sistemas baseados em tecnologia ótica, sejam consideradas. Futuras análises devem ser conduzidas para validar estatisticamente os resultados obtidos. Porém, considerando o objetivo do trabalho, pode-se concluir que o sistema avaliado é uma alternativa confiável no contexto proposto. / The aim of the present study was to evaluate biomechanical parameters (both, anthropometric and kinematic) from data acquired through Microsoft Kinect (2010). The validation of its biomechanical characteristics will allow its use in gathering information to complement Ergonomic Work Analysis (EWA) and other user researches that provide product and work environment diagnosis through postural and dynamic assessment. The use of MS Kinect in this research is justified by the fact that is it has changed biomechanical scenario due to the fact that it is an inexpensive and portable alternative to the available motion capture systems available in the market. However, its accuracy has been the subject of many ongoing researches Data acquired from 3D motion capture systems provide means for evaluating products, activities and human-object interactions. In Design, its an important achievement, since it allows professionals to use the tool, considering that most motion capture systems available in the market are not accessible. The Microsoft console was compared to MVN Biomech inertial system (XSENS TECHNOLOGIES) and traditional 2D video data. In order to acquire kinematic data form MS Kinect, a software was chosen, following a few criteria. Two laboratorial experiments were held, where the first aimed to acquire operational details and limitations from the equipments, as for the second aimed to compare biomechanical data through three static parameters and one dynamic. The static parameters involved joint and segment angles, as well as a segments dimensional comparison, whereas the proposals were to assess anthropometric features and to understand if the biomechanical model considers the segments as rigid bodies. The dynamic comparison was based on joints global displacement in selected movements. In order to analyze data from different systems, a digital platform based on a game engine was developed, constituting an even ground for evaluation. The platform maintains original data from the biomechanical models and extracts kinematic parameters, which can be compared. The experiments have provided understanding on the consoles operations, providing guidelines for its usage. In order to also evaluate its usage in real work environments, trial recordings were held in chemical laboratories. These trials have shown the alternative to be viable if the limitations, similar to optical systems, are considered. Future analyses should be conducted in order to statistically validate the results. However, given the goals and context of this research, it can be concluded that the evaluated system is a reliable motion capture alternative.

Les environnements virtuels collaboratifs permettent à plusieurs utilisateurs d’interagir à distance par Internet. Ils peuvent partager une impression de téléprésence en animant à distance un avatar qui les représente. Toutefois, le contrôle de cet avatar peut être difficile et mal restituer les gestes de l’utilisateur. Ce travail vise à animer l’avatar à partir d’une acquisition 3D des gestes de l’utilisateur par vision monoculaire en temps réel, et à rendre la téléprésence virtuelle possible au moyen d’un PC grand public équipé d’une webcam. L’approche suivie consiste à recaler un modèle 3D articulé de la partie supérieure du corps humain sur une séquence vidéo. Ceci est réalisé en cherchant itérativement la meilleure correspondance entre des primitives extraites du modèle 3D d’une part et de l’image d’autre part. Le recalage en deux étapes peut procéder sur les régions, puis sur les contours. La première contribution de cette thèse est une méthode de répartition des itérations de calcul qui optimise la robustesse et la précision sous la contrainte du temps-réel. La difficulté majeure pour le suivi 3D à partir d’images monoculaires provient des ambiguïtés 3D/2D et de l’absence d’information de profondeur. Le filtrage particulaire est désormais une approche classique pour la propagation d’hypothèses multiples entre les images. La deuxième contribution de cette thèse est une amélioration du filtrage particulaire pour le recalage 3D/2D en un temps de calcul limité par des heuristiques, dont la contribution est démontée expérimentalement. Un paramétrage de l’attitude des bras par l’extrémité de leur chaîne cinématique est proposé qui permet de mieux modéliser l’incertitude sur la profondeur. Enfin, l’évaluation est accélérée par calcul sur GPU. En conclusion, l’algorithme proposé permet un suivi 3D robuste en temps-réel à partir d’une webcam pour une grande variété des gestes impliquant des occlusions partielles et des mouvements dans la direction de la profondeur. / Networked 3D virtual environments allow multiple users to interact with each other over the Internet. Users can share some sense of telepresence by remotely animating an avatar that represents them. However, avatar control may be tedious and still render user gestures poorly. This work aims at animating a user’s avatar from real time 3D motion capture by monoscopic computer vision, thus allowing virtual telepresence to anyone using a personal computer with a webcam. The approach followed consists of registering a 3D articulated upper-body model to a video sequence. This involves searching iteratively for the best match between features extracted from the 3D model and from the image. A two-step registration process matches regions and then edges. The first contribution of this thesis is a method of allocating computing iterations under real-time constrain that achieves optimal robustness and accuracy. The major issue for robust 3D tracking from monocular images is the 3D/2D ambiguities that result from the lack of depth information. Particle filtering has become a popular framework for propagating multiple hypotheses between frames. As a second contribution, this thesis enhances particle filtering for 3D/2D registration under limited computation constrains with a number of heuristics, the contribution of which is demonstrated experimentally. A parameterization of the arm pose based on their end-effector is proposed to better model uncertainty in the depth direction. Finally, evaluation is accelerated by computation on GPU. In conclusion, the proposed algorithm is demonstrated to provide robust real-time 3D body tracking from a single webcam for a large variety of gestures including partial occlusions and motion in the depth direction.

Articular fracture reduction is a complex surgical task that requires surgeons to be competent at multiple surgical skills to successfully complete. The list of skills needed includes the ability to use fluoroscopic images to build a 3D mental model of the fracture during reconstruction, the proper handling and use of surgical instruments, how to manipulate the fracture fragment into a reduced configuration with minimal hand motion, proper k-wire placement, and the preservation of surrounding soft tissues. Current training methodology is based on an apprenticeship model. The resident learns by watching a senior surgeon, and then preforms the procedure on live patients under the guidance of the senior surgeon to gain competence. This endangers the patient and does not provide the best outcome for either patient or resident. The work presented in this thesis is the early development of an articular fracture reduction simulator, the subsequent use of the simulator in the training of orthopaedic residents, and assessment of the improvement of residents after practice on the simulator. To date, the simulator has been tested on four different groups of residents,3 different groups from the University of Iowa and one group from the University of Minnesota. Considerable effort has been made to validate the improvement seen in resident performance through objective means. The Objective Structured Assessment of Technical Skills (OSATS) is a global rating score and procedural checklist that has been previously validated to objectively measure surgical skill. Other assessment metrics include hand motion capture to count the number of discrete actions and measure distance traveled during the surgical procedure, fluoroscopic usage and radiation exposure, articular `step-off', the surface deviation from an intact or ideal reconstruction, and contact stress exposure. The results indicate that the goals for the simulator have been met, that the simulator provides a means of training orthopaedic residents, assessing improvement, decreased the cost of training, and improved patient safety. The simulator is not without limitations including sample size, and radiation exposure. The task being trained is complex and can be broken down into basic subtasks that could be trained individually. Even with flaws, the simulator is an improvement over current training methods and is an excellent first step toward creating a surgical skills curriculum to comply with new mandates from orthopaedic surgery's governing bodies.

<p>Hasomed GmbH, a German company in the Field of medicine technology, intends to introduce a gait analysis system on the market. The system includes an inertial sensor which collects data used for generating movement patterns of the feet. This thesis describes the development and evaluation of a new version of the sensor, aimed at minimizing costs, maximizing performance and facilitating production.</p><p>Algorithms used in the gait analysis system are sensitive to noise. Noise sources and precautions taken in order to minimize noise levels are described and discussed. By minimizing the physical size of analogue electronics blocks, static noise and occasional high frequency components were substantially reduced.</p><p>New features including internal temperature sensors, firmware update via serial interface, self-test functions and a wireless link were implemented. Additional improvements are e.g. lower power consumption and an extension of the interface from 2 to 256 (theoretical limit) attached devices. By reducing the number of included components and PCB (Printed Circuit Board) layers, together with use of components that do not require advanced soldering techniques, easier and cheaper production was obtained.</p><p>Research and development presented in this thesis resulted in a sensor with overall good performance and new features.</p>

Hasomed GmbH, a German company in the Field of medicine technology, intends to introduce a gait analysis system on the market. The system includes an inertial sensor which collects data used for generating movement patterns of the feet. This thesis describes the development and evaluation of a new version of the sensor, aimed at minimizing costs, maximizing performance and facilitating production. Algorithms used in the gait analysis system are sensitive to noise. Noise sources and precautions taken in order to minimize noise levels are described and discussed. By minimizing the physical size of analogue electronics blocks, static noise and occasional high frequency components were substantially reduced. New features including internal temperature sensors, firmware update via serial interface, self-test functions and a wireless link were implemented. Additional improvements are e.g. lower power consumption and an extension of the interface from 2 to 256 (theoretical limit) attached devices. By reducing the number of included components and PCB (Printed Circuit Board) layers, together with use of components that do not require advanced soldering techniques, easier and cheaper production was obtained. Research and development presented in this thesis resulted in a sensor with overall good performance and new features.

The Niagara Foot (NF) is a relatively new prosthetic design, primarily intended for use in developing countries. It combines low cost and durability with high performance energy return features. The design has been successfully tested mechanically and in field trials, but to date there has been little quantitative gait data describing the performance of the foot. Biomechanical gait analysis techniques will be used to extract quantitative gait measures. The current study is designed to characterize the effect of heel section stiffness parameter differences between a NF normal heel and a NF with a reduced material heel section., on gait characteristics in persons with unilateral trans-tibial amputations (TTA). Standardized biomechanical gait analysis techniques, adapted for this population, were used to extract quantitative gait measures. Five persons with TTA performed walking tasks while 3D ground reaction forces were recorded via an embedded force platform. A motion capture system also recorded the 3D segmental motion of the lower limbs and torso of each subject. These were combined to calculate net joint moments and mechanical power at the hip and knee of both limbs. These data were compared between a normal NF and a NF with a modified heel. Each participant had a period of two-week adaptation prior to any testing. An EMG system and a prosthesis evaluation questionnaire were used to help analyze the condition. The overall hypothesis of this study was that modification of the heel section stiffness would change several aspects of gait. Although the gait pattern differences between participants and the low participant number produced no significant differences between the conditions for all variables, trends were observed in multiple outcomes. These results report preliminary evidence that for some participants the heel material reduction does impact their gait by showing a different loading phase during the transition between the heel strike and the full contact with the ground. The NF2 may move the gait toward a more flexed knee position. Furthermore, despite a reduction in the material of the heel section results showed that the overall foot stiffness increased. This may be the result of the one-piece design and mechanics of the NF. Further investigations with a bigger cohort of people with TTA are required to look at the importance of the impact of the prosthetic foot heel stiffness.