In this paper, a Kinect-based distributed and real-time motion capture system is developed. A trigonometric method is applied to calculate the relative positions of Kinect v2 sensors with a calibration wand and register the sensors’ positions automatically. By combining results from multiple sensors with a nonlinear least square method, the accuracy of motion capture is optimized. Moreover, to exclude inaccurate results from sensors, a computational geometry is applied in the occlusion approach to discover occluded joint data. The synchronization approach is based on the NTP protocol, which synchronizes the time between the clocks of a server and of clients dynamically, leading to the proposed system being real time. Experiments to validate the proposed system are conducted from the perspective of calibration, occlusion, and accuracy. More specifically, the mean absolute error of the calibration results is 0.73 cm, the proposed occlusion method is tested on upper and lower limbs, and the synchronization component guarantees the clock synchronization and real-time performance for more than 99% of the measurement process. Furthermore, to demonstrate the practical performance of our system, a comparison with previously developed motion capture systems (the linear trilateration approach [52] and the geometric trilateration approach [51]) with the benchmark Opti Track system is performed for the tracked joints of the head, shoulder, elbow, and wrist, therein showing that the accuracy of our proposed system is 38.3% and 24.1% better than the aforementioned two trilateration systems. Quantitative analysis is also conducted on our proposed system with the commercial inertial motion capture system Delsys smart sensor system by comparing the measurements of lower limbs (i.e., hips, knees, and ankles), and the standard deviation of our proposed system’s measurement results is 4.92 cm.

Une des innovations technologiques majeures de ces dernières années a été le lancement des caméras de profondeur qui peuvent être utilisées dans un large spectre d’applications, notamment pour la robotique, la vision par ordinateur, l’automatisation, etc. Ces dispositifs ont ouvert de nouvelles opportunités pour la recherche scientifique appliquée au domaine médical. Dans le cadre de cette thèse, nous évaluerons l’apport potentiel de l’utilisation du capteur de profondeur grand public « Kinect V2 » dans l’optique de répondre à des problématiques cliniques actuelles en radiothérapie ainsi qu’en réanimation. Le traitement par radiothérapie étant administré sur plusieurs séances, l'un des objectifs clés de ce traitement est le positionnement quotidien du patient dont la précision est impactée par les mouvements respiratoires. D’autre part, les mouvements de la machine ainsi que les éventuels mouvements du patient peuvent entraîner des collisions machine/machine ou machine/patient. Nous proposons un système de détection surfacique pour la gestion des mouvements inter- et intrafractions en radiothérapie externe. Celui-ci est basé sur un algorithme rigide de recalage surfacique pour estimer la position de traitement et un système de détection de collisions en temps réel pour satisfaire les conditions de sécurité durant le traitement. Les résultats obtenus sont encourageants et montrent un bon accord avec les systèmes cliniques. Coté réanimation médicale, la recherche de nouveaux dispositifs non invasifs et sans contact tend à optimiser la prise en charge des patients. La surveillance non invasive de la respiration des patients sous ventilation spontanée est capitale pour les patients instables mais aucun système de suivi à distance n’existe à ce jour. Dans ce contexte, nous proposons un système de mesure sans contact capable de calculer les paramètres ventilatoires en observant les changements morphologiques de la zone thoracique des patients. La méthode développée donne une précision de mesures cliniquement acceptable. / In recent years, one of the major technological innovations has been the introduction of depth cameras that can be used in a wide range of applications, including robotics, computer vision, automation, etc. These devices have opened up new opportunities for scientific research applied to the medical field. In this thesis, we will evaluate the potential use of the "Kinect V2" depth camera in order to respond to current clinical issues in radiotherapy and resuscitation in intensive care unit.Given that radiotherapy treatment is administered over several sessions, one of the key task is to daily reposition the patient in the same way as during the planning session.The precision of such repositioning is impacted by the respiratory motion. On the other hand, the movements of the machine as well as the possible movements of the patient can lead to machine / machine or machine /patient collisions. We propose a surface detection system for the management of inter and intra-fraction motion in external radiotherapy. This system is based on a rigid surface registration algorithm to estimate the treatment position and a real-time collision detection system to ensure patient safety during the treatment.Obtained results are encouraging and show a good agreement with available clinical systems.Concerning medical resuscitation, there is a need for new non-invasive and non-contact devices in order to optimize patient care. Non-invasive monitoring of spontaneous breathing for unstable patients is crucial in the intensive care unit. In this context, we propose a non-contact measurement system capable of calculating the parameters of patient's ventilation by observing thoracic morphological movements. The developed method gives a clinically acceptable precision. Such system is the first to solve previously described issue.

Motion capture technology has been applied in many fields such as animation, medicine, military, etc. since it was first proposed in the 1970s. Based on the principles applied, motion capture technology is generally classified into six categories: 1) Optical; 2) Inertial; 3) Magnetic; 4) Mechanical; 5) Acoustic and 6) Markerless. Different from the other five kinds of motion capture technologies which try to track path of specific points with different equipment, markerless systems recognize human or non-human body's motion with vision-based technology which focuses on analyzing and processing the captured images for motion capture. The user doed not need to wear any equipment and is free to do any action in an extensible measurement area while a markerless motion capture system is working. Though this kind of system is considered as the preferred solution for motion capture, the difficulty for realizing an effective and high accuracy markerless system is much higher than the other technologies mentioned, which makes markerless motion capture development a popular research direction. Microsoft Kinect sensor has attracted lots of attention since the launch of its first version with its depth sensing feature which gives the sensor the ability to do motion capture without any extra devices. Recently, Microsoft released a new version of Kinect sensor with improved hardware and and targeted at the consumer market. However, to the best of our knowlege, the accuracy assessment of the sensor remains to be answered since it was released. In this thesis, we measure the depth accuracy of the newly released Kinect v2 depth sensor from different aspects and propose a trilateration method to improve the depth accuracy with multiple Kinects simultaneously. Based on the trilateration method, a low-cost, no wearable equipment requirement and easy setup human gait tracking system is realized.

The thesis deals with the design and implementation of user interface for robotic manipulator control by means of gesture detection of the operator. The used manipulator is the Katana 6M180 with five degrees of freedom. Gestures are detected using an innovated sensor system which includes Microsoft Kinect version 2 and Leap Motion sensors. The first part provides an overview of existing solutions and used technologies. The second part describes the design of the user interface, its implementation and testing.

Sensor Fusion for 3D data is a popular topic. Multisensor data combination enhance the qualities of each other while single sensor lacks accuracy. In this thesis, an evaluation platform for Multimodal 3D data from Kinect v2 and Microphone Array is designed and implemented by using ReactJS. In automotive industry and computer vision area, 3D detection and localization are widely used. Solutions of 3D detection and localization using different measurement systems are discussed in a large number of papers. Data Fusion systems are normally using ultrasound based, radio waves based, Time-of-Flight, structured light, stereo cameras and sound based sensors. All of these measurement systems might provide different 3D data models. And each system works fine separately. However, in some cases, multiple measurement systems need to work together. Their 3D data sets are different and could not be compared and combined directly. In order to simplify the design process of multiple measurement systems, this web based evaluation platform is focused on comparison and combination of 3D data sets from different coordinate systems. It provides a quick and easy development method between multiple measurement systems. In this thesis, an evaluation platform which based on Kinect v2 body detection and microphone array sound detection systems will be discussed. First an introduction about project overview is given. The second section of this paper deals with several project related technologies. The third section provides the concept of this project. The forth section describes development and implement detail. The next section is about data visualization and statistical analysis. Further the final results, evaluation and discussion are given.

This master thesis project is carried out by one student at the Royal Institute of Technology in collaboration with Cybercom Group. The aim was to evaluate and compare system design strategies for fruit recognition in harvesting robots and the performance of supervised machine learning classification methods when applied to this specific task. The thesis covers the basics of these systems; to which parameters, constraints, requirements, and design decisions have been investigated. The framework is used as a foundation for the implementation of both sensing system, and processing and classification algorithms. A plastic tomato plant with fruit of varying maturity was used as a basis for training and testing, and a Kinect v2 for Windows including sensors for high resolution color-, depth, and IR data was used for image acquisition. The obtained data were processed and features of objects of interest extracted using MATLAB and a SDK for Kinect provided by Microsoft. Multiple views of the plant were acquired by having the plant rotate on a platform controlled by a stepper motor and an Ardunio Uno. The algorithms tested were binary classifiers, including Support Vector Machine, Decision Tree, and k-Nearest Neighbor. The models were trained and validated using a five fold cross validation in MATLABs Classification Learner application. Peformance metrics such as precision, recall, and the F1-score, used for accuracy comparison, were calculated. The statistical models k-NN and SVM achieved the best scores. The method considered most promising for fruit recognition purposes was the SVM. / Det här masterexamensarbetet har utförts av en student från Kungliga Tekniska Högskolan i samarbete med Cybercom Group. Målet var att utvärdera och jämföra designstrategier för igenkänning av frukt i en skörderobot och prestandan av klassificerande maskininlärningsalgoritmer när de appliceras på det specifika problemet. Arbetet omfattar grunderna av dessa system; till vilket parametrar, begränsningar, krav och designbeslut har undersökts. Ramverket användes sedan som grund för implementationen av sensorsystemet, processerings- och klassifikationsalgoritmerna. En tomatplanta i pplast med frukter av varierande mognasgrad användes som bas för träning och validering av systemet, och en Kinect för Windows v2 utrustad med sensorer för högupplöst färg, djup, och infraröd data anvöndes för att erhålla bilder. Datan processerades i MATLAB med hjälp av mjukvaruutvecklingskit för Kinect tillhandahållandet av Windows, i syfte att extrahera egenskaper ifrån objekt på bilderna. Multipla vyer erhölls genom att låta tomatplantan rotera på en plattform, driven av en stegmotor Arduino Uno. De binära klassifikationsalgoritmer som testades var Support Vector MAchine, Decision Tree och k-Nearest Neighbor. Modellerna tränades och valideras med hjälp av en five fold cross validation i MATLABs Classification Learner applikation. Prestationsindikatorer som precision, återkallelse och F1- poäng beräknades för de olika modellerna. Resultatet visade bland annat att statiska modeller som k-NN och SVM presterade bättre för det givna problemet, och att den sistnömnda är mest lovande för framtida applikationer.

[ES] Los desórdenes del equilibrio y la marcha se encuentran entre los déficits motores más frecuentes entre aquellos individuos que han sufrido un ictus. En la clínica, estas habilidades son comúnmente evaluadas mediante herramientas clínicas que, pese a ser generalmente fáciles y rápidas de administrar, adolecen de poca precisión y estar sesgadas. Los sistemas instrumentados de laboratorio existentes para valorar la postura y la marcha resuelven potencialmente estas limitaciones a costa de requerir una preparación previa por parte de los evaluadores, un amplio espacio reservado en la clínica, un elevado tiempo de realización, y tener un coste muy elevado. El desarrollo tecnológico del sector del entretenimiento ha dado lugar en la última década a periféricos, como plataformas de presión y sensores de profundidad, que permiten la interacción mediante movimientos corporales manteniendo un bajo coste y una gran portabilidad y accesibilidad. Estudios iniciales han mostrado un rendimiento de estos dispositivos muy prometedor, y a veces comparable al de sistemas de laboratorio. Sin embargo, la falta de acceso a los sistemas desarrollados, la escasa investigación en personas con ictus y el desconocimiento de las propiedades psicométricas de las pruebas basadas en estos dispositivos en esta población comprometen la relevancia clínica que podrían tener estos sistemas. La hipótesis principal de este trabajo es que plataformas de fuerzas y sensores de profundidad de bajo coste, como la Nintendo Wii Balance Board y la Microsoft Kinect v2, respectivamente, pueden proporcionar información válida para cuantificar y evaluar la postura y la marcha de sujetos que han sufrido un ictus. Durante la presente tesis doctoral, por tanto, se ha llevado a cabo el desarrollo de herramientas de valoración de la postura y la marcha mediante los dispositivos nombrados, se ha posibilitado su acceso libre, se han determinado los valores normativos de las pruebas incluidas en las herramientas desarrolladas, y, finalmente, se ha investigado su sensibilidad, su validez convergente con herramientas clínicas estandarizadas y su fiabilidad inter e intraevaluador. Los resultados obtenidos de la participación de un total de 544 sujetos sanos y 173 sujetos con ictus en los cinco estudios que comprenden este trabajo evidencian que las herramientas desarrolladas permiten caracterizar satisfactoriamente la postura y la marcha de sujetos con ictus con respecto a la de sujetos sanos, poseen una validez convergente con instrumentos variables y coherente, y tienen una fiabilidad inter e intraevaluador excelente para casi todas las pruebas incluidas. Estos hallazgos sugieren que, pese a las limitaciones existentes, las herramientas desarrolladas podrían ser potencialmente usadas como una alternativa de bajo coste a los sistemas de laboratorio existentes para complementar la valoración de la postura y la marcha de sujetos con ictus. / [CA] Els desordres de l'equilibri i la marxa es troben entre els dèficits motors més freqüents entre aquells individus que han patit un ictus. A la clínica, aquestes habilitats són comunament avaluades mitjançant instruments clínics que, tot i ser generalment fàcils i ràpids d'administrar, poden tindre poca precisió i solen estar esbiaixades. Els sistemes instrumentats de laboratori existents per avaluar la postura i la marxa permeten resoldre aquestes limitacions, però requereixen una preparació prèvia per part dels avaluadors, un ampli espai reservat a la clínica, un elevat temps per realitzar cada prova i tenen un cost molt elevat. El desenvolupament tecnològic del sector de l'entreteniment ha donat lloc a plataformes de pressió i sensors de profunditat de baix cost, gran portabilitat i accessibilitat, que permeten la interacció amb entorns virtuals mitjançant moviments corporals. Estudis preliminars han mostrat un rendiment d'aquests dispositius molt prometedor i, de vegades, comparable al de sistemes de laboratori. No obstant això, la falta d'accés a les aplicacions desenvolupades, l'escassa investigació en persones amb ictus i el desconeixement de les propietats psicomètriques de les proves basades en aquests dispositius en aquesta població comprometen la rellevància clínica dels resultats obtinguts. La hipòtesi principal d'aquest treball és que plataformes de forces i sensors de profunditat de baix cost, com la Nintendo Wii Balance Board i la Microsoft Kinect v2, respectivament, poden proporcionar informació vàlida per quantificar i avaluar la postura i la marxa de subjectes amb ictus. Durant la present tesi doctoral, per tant, s'han desenvolupat eines de valoració de la postura i la marxa mitjançant els dispositius anomenats, s'ha possibilitat el seu accés lliure, s'han determinat els valors normatius de les proves incloses en les eines desenvolupades, i, finalment, s'ha investigat la seva sensibilitat, la seva validesa convergent amb instruments clínics estandarditzats i la seua fiabilitat inter i intraavaluador. Els resultats obtinguts de la participació d'un total de 544 subjectes sans i 173 subjectes amb ictus en els cinc estudis que comprenen aquest treball evidencien que les eines desenvolupades permeten caracteritzar satisfactòriament la postura i la marxa de subjectes amb ictus respecte a la de subjectes sans, tenen una validesa convergent amb instruments variable i coherent, i tenen una fiabilitat inter i intraavaluador excel·lent en gairebé totes les proves incloses. Aquestes troballes suggereixen que, tot i les limitacions existents, les aplicacions desenvolupades podrien ser potencialment usades com una alternativa de baix cost als sistemes de laboratori existents per complementar la valoració de la postura i la marxa de subjectes amb ictus. / [EN] Balance and gait disorders are common after stroke. In the clinical setting, these skills are usually assessed using clinical instruments that, despite being generally quick and easy to administer, may have limited accuracy and be biased. Instrumented laboratory-grade systems aimed at assessing posture and gait can potentially overcome these limitations. However, they require specific training to be operated and a long time to perform the assessments, and are usually bulky and expensive. In the last decade, the technological advances in the gaming industry have given rise to low-cost, portable and off-the-shelf devices, such as pressure platforms and depth sensors, which enable interaction with videogames through body movements. Previous research on the performance of these devices has shown promising results, and suggests that some measures could have comparable accuracy to those estimated by laboratory-grade systems. However, the lack of access to the software used in the experiments, the limited research in stroke patients, and the absence of knowledge about the psychometric properties of the assemment tests based on these devices, could limit the clinical relevance of the preliminary findings. The main hypothesis of this thesis is that low-cost force platforms and depth sensors, such as the Nintendo Wii Balance Board and the Microsoft Kinect v2, can provide sensitive, valid and reliable information to quantify and asses the postural control and gait of individuals with stroke, respectively. This work describes the development of two customized applications to assess posture and gait using the devices mentioned above, their publication on a dedicated website, the exploration of the normative values of tests included in the assessment, and, finally, the investigation of the sensitivity, convergent validity with standardized clinical instruments, and their inter- and intra-rater reliability. A total of 544 healthy subjects and 173 individuals with stroke have participated in the five studies that encompass this thesis. The results of these studies showed good sensitivity to motor impairment, variable and consistent convergent validity with clinical instruments, and excellent inter- and intra-rater reliability for almost all the tests examined. All these findings suggest that, despite their limitations, the developed applications interfaced with low-cost force platforms and depth sensors, could be potentially used as a low-cost alternative to instrumented laboratory-grade systems to complement the clinical assessment of the posture and gait of individuals with stroke. / Latorre Grau, J. (2022). Valoración del equilibrio y la marcha mediante sistemas de bajo coste en sujetos con ictus [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/181339