No description available.

Sensor technology is rapidly developing and, consequently, the generation of point cloud data is constantly increasing. Since the recent release of PointNet, it is possible to process this unordered 3-dimensional data directly in a neural network. The company TLT Screen AB, which develops cutting-edge tracking technology, seeks to optimize the localization of the fingertips of a hand in a point cloud. To do so, the identification of relevant 3D neural network models for modeling hands and detection of fingertips in various hand orientations is essential. The Hand PointNet processes point clouds of hands directly and generate estimations of fixed points (joints), including fingertips, of the hands. Therefore, this model was selected to optimize the localization of fingertips for TLT Screen AB and forms the subject of this research. The model has advantages over conventional convolutional neural networks (CNN). First of all, in contrast to the 2D CNN, the Hand PointNet can use the full 3-dimensional spatial information. Compared to the 3D CNN, moreover, it avoids unnecessarily voluminous data and enables more efficient learning. The model was trained and evaluated on the public dataset MRSA Hand. In contrast to previously published work, the main object of this investigation is the estimation of only 5 joints, for the fingertips. The behavior of the model with a reduction from the usual 21 to 11 and only 5 joints are examined. It is found that the reduction of joints contributed to an increase in the mean error of the estimated joints. Furthermore, the examination of the distribution of the residuals of the estimate for fingertips is found to be less dense. MC dropout to study the prediction uncertainty for the fingertips has shown that the uncertainty increases when the joints are decreased. Finally, the results show that the uncertainty is greatest for the prediction of the thumb tip. Starting from the tip of the thumb, it is observed that the uncertainty of the estimates decreases with each additional fingertip.

Serial articulated industrial manipulators are increasingly used in machining applications due to their flexibility in application and their cost-effectiveness compared to conventional machinery. However, the use of industrial manipulators in machining processes that subject the robot to high loads such as in drilling is limited. The relatively low mechanical stiffness leads to position offsets from the anticipated position. Efforts have been made in the past to create manipulator calibration methods to compensate for their low stiffness and to increase their pose accuracy. The Department of Production Engineering at KTH Royal Institute of Technology defined a static and quasi-static compliance calibration procedure for industrial manipulators. Contrary to the hypothesis, the two methods produce different results in terms of the measured magnitude of Cartesian deflections. This study compares static and quasi-static compliance measurement procedures on an ABB IRB 6700-300/2.70 and aims at finding causes for the difference in the measured deflection of the manipulator between the two methods. Therefore, a literature review is performed and based on the review a novel quasi-static measurement procedure is presented. Deflections during the application of static and quasi-static loads with a frequency of less than 0.5 Hz on the manipulator are measured and compared. Differences in deflection are seen and potential causes are analyzed in several experiments. Namely, by changing parameters the resulting effects on the manipulator due to kinematic errors and dynamic effects are investigated. The results stress that unlike the expectation based on the theory of mechanics the system shows a dynamic behavior if a periodic loading with a frequency of less than 0.5Hz is applied during the quasi-static experiments. The difference in deflection is thus explained through load dissipation by damping and inertial forces during the quasi-static measurements of the novel method. This does not apply to the quasi-static measurement procedure defined by the Production Engineering department. Moreover, differences in deflection were identified due to friction and backlash acting in the transmissions system of the motors when static loads are applied in certain regions of the task space. Future work in the analysis of differences in compliance measurement procedures is encouraged to find causes for the quasi-static measurement results of the department. / Serieartikulerade industriella manipulatorer används allt mer i bearbetande operationer tack vare dess flexibilitet i användande och dess kostnadseffektivitet jämfört med konventionella maskiner. Dock är användandet av industriella manipulatorer i bearbetningsprocesser som utsätter roboten för höga laster så som borrande begränsat. Den relativt höga mekaniska stelheten leder till positionsförskjutningar från den förväntade positionen. Ansträngningar har tidigare gjorts för att skapa kalibreringsmetoder för manipulatorer som ska kompensera för dess låga stelhet och öka dess position och orienterings-exakthet. Institutionen för industriell produktion vid Kungliga Tekniska Högskolan har definerat en statisk och kvasistatisk efterlevnads-kalibrerings-procedur för industriella manipulatorer. I motsats till hypotesen producerade de två metoderna olika resultat i avseende till den uppmätta magnituden för Kartesiska böjningar. Denna studie jämför statiska och kvasistatiska efterlevnads-mätnings-procedurer hos en ABB IRB 6700-300/2.70 och siktar på att hitta orsaker för skillnaden i den uppmätta böjningen av manipulatorn mellan de två metoderna. Därmed genomförs en litteraturstudie och baserat på en översikt presenteras en ny kvasistatisk mätningsprocedur. Böjningar under påverkan av statiska och kvasistatiska laster på under 0.5 Hz på manipulatorn uppmäts och jämförs. Skillnader i böjningar kan ses och potentiella orsaker analyseras i flera experiment. Genom byte av parametrar kan effekter på den industriella manipulatorn som orsakas av kinematiska fel och dynamiska effekter undersökas. Resultatet understryker att olikt förväntningarna som baserats på teorier från mekaniken uppvisar systemet ett dymaniskt beteende om en periodisk last på mindre än 0.5 Hz appliceras under de kvasistatiska experimenten. Skillnaden i böjningen förklaras därmed genom ett lastminskande som beror på dämpande och tröga krafter under de kvasistatiska mätningarna av den nya metoden. Detta gäller inte den kvasistatiska mätningsproceduren som definerats av Institutionen för industriell produktion. Utöver detta identifieras skillnader i böjningar med avseende på friktion och glapp i överföringssystemet i motorerna när statiska laster appliceras på specifika regioner i arbetsområdet. Framtida arbete i analys av skillnader i efterlevnadsmätnings-procedurerna uppmuntras för att hitta orsaker till institutionens kvasistatiska mätningsresultat.

Eyes allow us to see and gather information about the environment. Eyes mainly act as an input organ as they collect light, but they also can be considered an output organ as they indicate the subject's gaze direction. Using the orientation of the head and the position of the eyes, it is possible to estimate the gaze path of an individual. Gaze estimation is a fast growing technology that track a person's eyes and head movements to "pin point" where the subject is looking at on a computer screen. The gaze direction is described as a person's line of sight. The gaze point, also known as the focus point, is defined as the intersection of the line of sight with the screen. Gaze tracking has an infinite number of applications such as monitoring driver alertness or helping track a person's eyes with a psychological disorder that cannot communicate his/her issues. Gaze tracking is also used as a human-machine interface for disabled people that have lost total control of their limbs. Another application of gaze estimation is marketing. Companies use the information given by the gaze estimation system from their customers to design their advertisements and products. / Electrical and Computer Engineering

This paper presents 3-D face modeling with head pose and depth information estimated from a 2-D query face image. Many recent approaches to 3-D face modeling are based on a 3-D morphable model that separately encodes the shape and texture in a parameterized model. The model parameters are often obtained by applying statistical analysis to a set of scanned 3-D faces. Such approaches tend to depend on the number and quality of scanned 3-D faces, which are difficult to obtain and computationally intensive. To overcome the limitations of 3-D morphable models, several modeling techniques from 2-D images have been proposed. We propose a novel framework for depth estimation from a single 2-D image with an arbitrary pose. The proposed scheme uses a set of facial features in a query face image and a reference 3-D face model to estimate the head pose angles of the face. The depth information of the subject at each feature point is represented by the depth information of the reference 3-D face model multiplied by a vector of scale factors. We use the positions of a set of facial feature points on the query 2-D image to deform the reference face dense model into a person specific 3-D face by minimizing an objective function. The objective function is defined as the feature disparity between the facial features in the face image and the corresponding 3-D facial features on the rotated reference model projected onto 2-D space. The pose and depth parameters are iteratively refined until stopping criteria are reached. The proposed method requires only a face image of arbitrary pose for the reconstruction of the corresponding 3-D face dense model with texture. Experiment results with USF Human-ID and Pointing'04 databases show that the proposed approach is effective to estimate depth and head pose information with a single 2-D image. / Electrical and Computer Engineering

Textual data entry is an increasingly-important part of Human-Computer Interaction (HCI), but there is room for improvement in this domain. First, the keyboard -- a foundational text-entry device -- presents ergonomic challenges in terms of comfort and accuracy for even well-trained typists. Second, touch-screen smartphones -- some of the most ubiquitous mobile devices -- lack the physical space required to implement a full-size physical keyboard, and settle for a reduced input that can be slow and inaccurate. This thesis proposes and examines "DeepType" to begin addressing both of these problems in the form of a fully-virtual keyboard, realized through a deep recurrent neural network (DRNN) trained to recognize skeletal movement during typing. This network enables typing data to be extracted without a physical keyboard: a user can type on a flat surface as though on a keyboard, and the movement of their fingers (as recorded via monocular camera and estimated using a pre-trained model) is input into the DeepType network to provide output compatible with that output by a physical keyboard with 91.2% accuracy without any autocorrection. We show that this architecture is computationally feasible and sufficiently accurate for use when tailored to a specific subject, and suggest optimizations that may enable generalization. We also present a novel data capture system used to generate the training dataset for DeepType, including effective hand pose data normalization techniques.

Human pose estimation (HPE) is an ever-growing research field, with an increasing number of publications in the computer vision and deep learning fields and it covers a multitude of practical scenarios, from sports to entertainment and from surveillance to medical applications. Despite the impressive results that can be obtained with HPE, there are still many problems that need to be tackled when dealing with real-world applications. Most of the issues are linked to a poor or completely wrong detection of the pose that emerges from the inability of the network to model the viewpoint. This thesis shows how designing viewpoint-equivariant neural networks can lead to substantial improvements in the field of human pose estimation, both in terms of state-of-the-art results and better real-world applications. By jointly learning how to build hierarchical human body poses together with the observer viewpoint, a network can learn to generalise its predictions when dealing with previously unseen viewpoints. As a result, the amount of training data needed can be drastically reduced, simultaneously leading to faster and more efficient training and more robust and interpretable real-world applications.

This research presents improvements to the velocity field reconstruction process achieved through computer vision. The first improvement of the velocity reconstruction process is the automation of the scanning laser Doppler vibrometer (SLDV) pose procedure. This automated process results in superior estimates of the position and orientation of the SLDV. The second improvement is the refinement of the formulation for reconstruction of the velocity field. The refined formulation permits faster computation, evaluation, and interpretation of the reconstructed structural velocity field. Taken together, these new procedures significantly improve the overall velocity reconstruction process which results in better, unbiased out-of-plane velocity estimates in the presence of noise. The automation of the SLDV pose procedure is achieved through a computer vision model of the SLDV. The SLDV is modeled as a projective camera, i.e. an imager which preserves projectivities. This projective camera model permits the precise association of object features with image features. Specifically, circular features in the object space are seen by the SLDV as ellipses in the image space. In order to extract object points, the bitangents among the circular features are constructed and the bitangent points selected. The accuracy and precision of the object points are improved through the use of a calibrated object whose circular features are measured with a coordinate measuring machine. The corresponding image points are determined by constructing the bitangents among the ellipses and selecting the tangent points. Taken together, these object/image bitangent point sets are a significantly improved data set for previously developed SLDV pose algorithms. Experimental verification of this automated pose procedure includes demonstrated repeatability, independent validation of the estimated pose parameters, and comparison of the estimated poses with previous methods. The refinement of the velocity reconstruction formulation is a direct result of the computer vision viewpoint adapted for this research. By viewing the velocity data as images of the harmonically excited structure's velocity field, analytical techniques developed for holographic interferometry are extended and applied to SLDV velocity images. Specifically, the "absolute" and "relative" fringe-order methods are used to reconstruct the velocity field with the "best" set of bases. Full and partial least squares solutions with experimental velocity data are calculated. Statistical confidence bounds of the regressed velocity coefficients are analyzed and interpreted to reveal accurate out-of-plane, but poor in-plane velocity estimates. Additionally, the reconstruction process is extended to recover the velocity field of a family of surfaces in the neighborhood of the "real" surface. This refinement relaxes the need for the exact experimental geometry. Finally, the velocity reconstruction procedure is reformulated so that independent least squares solutions are obtained for the two in-plane directions and the out-of plane direction. This formulation divides the original least squares problem into three smaller problems which can be analyzed and interpreted separately. These refinements to the velocity reconstruction process significantly improve the out-of-plane velocity solution and interpretation of the regressed velocity parameters. / Ph. D.

Orientation estimation of objects plays a pivotal role in robotics, self-driving cars, and augmented reality. Beyond mere position, accurately determining the orientation of objects is essential for constructing precise models of the physical world. While 2D object detection has made significant strides, the field of orientation estimation still faces several challenges. Our research addresses these hurdles by proposing an efficient pipeline which facilitates rapid creation of labeled training data and enables direct regression of object orientation from a single image. We start by creating a digital twin of a physical object using an iPhone, followed by generating synthetic images using the Unity game engine and domain randomization. Our deep learning model, trained exclusively on these synthetic images, demonstrates promising results in estimating the orientations of common objects. Notably, our model achieves a median geodesic distance error of 3.9 degrees and operates at a brisk 15 frames per second.

With an increase in age comes an increase in the risk of frailty and mobility decline, which can lead to dangerous falls and can even be a cause of mortality. Despite these serious consequences, healthcare systems remain reactive, highlighting the need for technologies to predict functional mobility decline. In this thesis, we present an end-to-end autonomous functional mobility assessment system that seeks to bridge the gap between robotics research and clinical rehabilitation practices. Unlike many fully integrated black-box models, our approach emphasizes the need for a system that is both reliable as well as transparent to facilitate its endorsement and adoption by healthcare professionals and patients. Our proposed system is characterized by the sensor fusion of multimodal data using an optimization framework known as factor graphs. This method, widely used in robotics, enables us to obtain visually interpretable 3-D estimations of the human body in recorded footage. These representations are then used to implement autonomous versions of standardized assessments employed by physical therapists for measuring lower-limb mobility, using a combination of custom neural networks and explainable models. To improve the accuracy of the estimations, we investigate the application of the Koopman operator framework to learn linear representations of human dynamics: We leverage these outputs as prior information to enhance the temporal consistency across entire movement sequences. Furthermore, inspired by the inherent stability of natural human movement, we propose ways to impose stability constraints in the dynamics during the training of linear Koopman models. In this light, we propose a sufficient condition for the stability of discrete-time linear systems that can be represented as a set of convex constraints. Additionally, we demonstrate how it can be seamlessly integrated into larger-scale gradient descent optimization methods. Lastly, we report the performance of our human pose detection and autonomous mobility assessment systems by evaluating them on outcome mobility datasets collected from controlled laboratory settings and unconstrained real-life home environments. While we acknowledge that further research is still needed, the study results indicate that the system can demonstrate promising performance in assessing mobility in home environments. These findings underscore the significant potential of this and similar technologies to revolutionize physical therapy practices.