Serial articulated industrial manipulators are increasingly used in machining applications due to their flexibility in application and their cost-effectiveness compared to conventional machinery. However, the use of industrial manipulators in machining processes that subject the robot to high loads such as in drilling is limited. The relatively low mechanical stiffness leads to position offsets from the anticipated position. Efforts have been made in the past to create manipulator calibration methods to compensate for their low stiffness and to increase their pose accuracy. The Department of Production Engineering at KTH Royal Institute of Technology defined a static and quasi-static compliance calibration procedure for industrial manipulators. Contrary to the hypothesis, the two methods produce different results in terms of the measured magnitude of Cartesian deflections. This study compares static and quasi-static compliance measurement procedures on an ABB IRB 6700-300/2.70 and aims at finding causes for the difference in the measured deflection of the manipulator between the two methods. Therefore, a literature review is performed and based on the review a novel quasi-static measurement procedure is presented. Deflections during the application of static and quasi-static loads with a frequency of less than 0.5 Hz on the manipulator are measured and compared. Differences in deflection are seen and potential causes are analyzed in several experiments. Namely, by changing parameters the resulting effects on the manipulator due to kinematic errors and dynamic effects are investigated. The results stress that unlike the expectation based on the theory of mechanics the system shows a dynamic behavior if a periodic loading with a frequency of less than 0.5Hz is applied during the quasi-static experiments. The difference in deflection is thus explained through load dissipation by damping and inertial forces during the quasi-static measurements of the novel method. This does not apply to the quasi-static measurement procedure defined by the Production Engineering department. Moreover, differences in deflection were identified due to friction and backlash acting in the transmissions system of the motors when static loads are applied in certain regions of the task space. Future work in the analysis of differences in compliance measurement procedures is encouraged to find causes for the quasi-static measurement results of the department. / Serieartikulerade industriella manipulatorer används allt mer i bearbetande operationer tack vare dess flexibilitet i användande och dess kostnadseffektivitet jämfört med konventionella maskiner. Dock är användandet av industriella manipulatorer i bearbetningsprocesser som utsätter roboten för höga laster så som borrande begränsat. Den relativt höga mekaniska stelheten leder till positionsförskjutningar från den förväntade positionen. Ansträngningar har tidigare gjorts för att skapa kalibreringsmetoder för manipulatorer som ska kompensera för dess låga stelhet och öka dess position och orienterings-exakthet. Institutionen för industriell produktion vid Kungliga Tekniska Högskolan har definerat en statisk och kvasistatisk efterlevnads-kalibrerings-procedur för industriella manipulatorer. I motsats till hypotesen producerade de två metoderna olika resultat i avseende till den uppmätta magnituden för Kartesiska böjningar. Denna studie jämför statiska och kvasistatiska efterlevnads-mätnings-procedurer hos en ABB IRB 6700-300/2.70 och siktar på att hitta orsaker för skillnaden i den uppmätta böjningen av manipulatorn mellan de två metoderna. Därmed genomförs en litteraturstudie och baserat på en översikt presenteras en ny kvasistatisk mätningsprocedur. Böjningar under påverkan av statiska och kvasistatiska laster på under 0.5 Hz på manipulatorn uppmäts och jämförs. Skillnader i böjningar kan ses och potentiella orsaker analyseras i flera experiment. Genom byte av parametrar kan effekter på den industriella manipulatorn som orsakas av kinematiska fel och dynamiska effekter undersökas. Resultatet understryker att olikt förväntningarna som baserats på teorier från mekaniken uppvisar systemet ett dymaniskt beteende om en periodisk last på mindre än 0.5 Hz appliceras under de kvasistatiska experimenten. Skillnaden i böjningen förklaras därmed genom ett lastminskande som beror på dämpande och tröga krafter under de kvasistatiska mätningarna av den nya metoden. Detta gäller inte den kvasistatiska mätningsproceduren som definerats av Institutionen för industriell produktion. Utöver detta identifieras skillnader i böjningar med avseende på friktion och glapp i överföringssystemet i motorerna när statiska laster appliceras på specifika regioner i arbetsområdet. Framtida arbete i analys av skillnader i efterlevnadsmätnings-procedurerna uppmuntras för att hitta orsaker till institutionens kvasistatiska mätningsresultat.

Eyes allow us to see and gather information about the environment. Eyes mainly act as an input organ as they collect light, but they also can be considered an output organ as they indicate the subject's gaze direction. Using the orientation of the head and the position of the eyes, it is possible to estimate the gaze path of an individual. Gaze estimation is a fast growing technology that track a person's eyes and head movements to "pin point" where the subject is looking at on a computer screen. The gaze direction is described as a person's line of sight. The gaze point, also known as the focus point, is defined as the intersection of the line of sight with the screen. Gaze tracking has an infinite number of applications such as monitoring driver alertness or helping track a person's eyes with a psychological disorder that cannot communicate his/her issues. Gaze tracking is also used as a human-machine interface for disabled people that have lost total control of their limbs. Another application of gaze estimation is marketing. Companies use the information given by the gaze estimation system from their customers to design their advertisements and products. / Electrical and Computer Engineering

This paper presents 3-D face modeling with head pose and depth information estimated from a 2-D query face image. Many recent approaches to 3-D face modeling are based on a 3-D morphable model that separately encodes the shape and texture in a parameterized model. The model parameters are often obtained by applying statistical analysis to a set of scanned 3-D faces. Such approaches tend to depend on the number and quality of scanned 3-D faces, which are difficult to obtain and computationally intensive. To overcome the limitations of 3-D morphable models, several modeling techniques from 2-D images have been proposed. We propose a novel framework for depth estimation from a single 2-D image with an arbitrary pose. The proposed scheme uses a set of facial features in a query face image and a reference 3-D face model to estimate the head pose angles of the face. The depth information of the subject at each feature point is represented by the depth information of the reference 3-D face model multiplied by a vector of scale factors. We use the positions of a set of facial feature points on the query 2-D image to deform the reference face dense model into a person specific 3-D face by minimizing an objective function. The objective function is defined as the feature disparity between the facial features in the face image and the corresponding 3-D facial features on the rotated reference model projected onto 2-D space. The pose and depth parameters are iteratively refined until stopping criteria are reached. The proposed method requires only a face image of arbitrary pose for the reconstruction of the corresponding 3-D face dense model with texture. Experiment results with USF Human-ID and Pointing'04 databases show that the proposed approach is effective to estimate depth and head pose information with a single 2-D image. / Electrical and Computer Engineering

Textual data entry is an increasingly-important part of Human-Computer Interaction (HCI), but there is room for improvement in this domain. First, the keyboard -- a foundational text-entry device -- presents ergonomic challenges in terms of comfort and accuracy for even well-trained typists. Second, touch-screen smartphones -- some of the most ubiquitous mobile devices -- lack the physical space required to implement a full-size physical keyboard, and settle for a reduced input that can be slow and inaccurate. This thesis proposes and examines "DeepType" to begin addressing both of these problems in the form of a fully-virtual keyboard, realized through a deep recurrent neural network (DRNN) trained to recognize skeletal movement during typing. This network enables typing data to be extracted without a physical keyboard: a user can type on a flat surface as though on a keyboard, and the movement of their fingers (as recorded via monocular camera and estimated using a pre-trained model) is input into the DeepType network to provide output compatible with that output by a physical keyboard with 91.2% accuracy without any autocorrection. We show that this architecture is computationally feasible and sufficiently accurate for use when tailored to a specific subject, and suggest optimizations that may enable generalization. We also present a novel data capture system used to generate the training dataset for DeepType, including effective hand pose data normalization techniques.

Human pose estimation (HPE) is an ever-growing research field, with an increasing number of publications in the computer vision and deep learning fields and it covers a multitude of practical scenarios, from sports to entertainment and from surveillance to medical applications. Despite the impressive results that can be obtained with HPE, there are still many problems that need to be tackled when dealing with real-world applications. Most of the issues are linked to a poor or completely wrong detection of the pose that emerges from the inability of the network to model the viewpoint. This thesis shows how designing viewpoint-equivariant neural networks can lead to substantial improvements in the field of human pose estimation, both in terms of state-of-the-art results and better real-world applications. By jointly learning how to build hierarchical human body poses together with the observer viewpoint, a network can learn to generalise its predictions when dealing with previously unseen viewpoints. As a result, the amount of training data needed can be drastically reduced, simultaneously leading to faster and more efficient training and more robust and interpretable real-world applications.

This research presents improvements to the velocity field reconstruction process achieved through computer vision. The first improvement of the velocity reconstruction process is the automation of the scanning laser Doppler vibrometer (SLDV) pose procedure. This automated process results in superior estimates of the position and orientation of the SLDV. The second improvement is the refinement of the formulation for reconstruction of the velocity field. The refined formulation permits faster computation, evaluation, and interpretation of the reconstructed structural velocity field. Taken together, these new procedures significantly improve the overall velocity reconstruction process which results in better, unbiased out-of-plane velocity estimates in the presence of noise. The automation of the SLDV pose procedure is achieved through a computer vision model of the SLDV. The SLDV is modeled as a projective camera, i.e. an imager which preserves projectivities. This projective camera model permits the precise association of object features with image features. Specifically, circular features in the object space are seen by the SLDV as ellipses in the image space. In order to extract object points, the bitangents among the circular features are constructed and the bitangent points selected. The accuracy and precision of the object points are improved through the use of a calibrated object whose circular features are measured with a coordinate measuring machine. The corresponding image points are determined by constructing the bitangents among the ellipses and selecting the tangent points. Taken together, these object/image bitangent point sets are a significantly improved data set for previously developed SLDV pose algorithms. Experimental verification of this automated pose procedure includes demonstrated repeatability, independent validation of the estimated pose parameters, and comparison of the estimated poses with previous methods. The refinement of the velocity reconstruction formulation is a direct result of the computer vision viewpoint adapted for this research. By viewing the velocity data as images of the harmonically excited structure's velocity field, analytical techniques developed for holographic interferometry are extended and applied to SLDV velocity images. Specifically, the "absolute" and "relative" fringe-order methods are used to reconstruct the velocity field with the "best" set of bases. Full and partial least squares solutions with experimental velocity data are calculated. Statistical confidence bounds of the regressed velocity coefficients are analyzed and interpreted to reveal accurate out-of-plane, but poor in-plane velocity estimates. Additionally, the reconstruction process is extended to recover the velocity field of a family of surfaces in the neighborhood of the "real" surface. This refinement relaxes the need for the exact experimental geometry. Finally, the velocity reconstruction procedure is reformulated so that independent least squares solutions are obtained for the two in-plane directions and the out-of plane direction. This formulation divides the original least squares problem into three smaller problems which can be analyzed and interpreted separately. These refinements to the velocity reconstruction process significantly improve the out-of-plane velocity solution and interpretation of the regressed velocity parameters. / Ph. D.

With the maturity of conventional industrial robots, there has been increasing interest in designing robots that emulate realistic animal motions. This discipline requires careful and systematic investigation of a wide range of animal motions from biped, to quadruped, and even to serpentine motion of centipedes, millipedes, and snakes. Collecting optical motion capture data of such complex animal motions can be complicated for several reasons. Often there is the need to use many high-quality cameras for detailed subject tracking, and self-occlusion, loss of focus, and contrast variations challenge any imaging experiment. The problem of self-occlusion is especially pronounced for animals. In this thesis, we walk through the process of collecting motion capture data of a running lizard. In our collected raw video footage, it is difficult to make temporal correspondences using interpolation methods because of prolonged blurriness, occlusion, or the limited field of vision of our cameras. To work around this, we first make a model data set by making our best guess of the points' locations through these corruptions. Then, we randomly eclipse the data, use Gappy POD to repair the data and then see how closely it resembles the initial set, culminating in a test case where we simulate the actual corruptions we see in the raw video footage. / Master of Science

In this thesis we explore an infrared light-based approach to the problem of pose estimation during aircraft landing in urban environments where GPS is unreliable or unavailable. We introduce a novel fiducial constellation composed of sparse infrared lights that incorporates projective invariant properties in its design to allow for robust recognition and association from arbitrary camera perspectives. We propose a pose estimation pipeline capable of producing high accuracy pose measurements at real-time rates from monocular infrared camera views of the fiducial constellation, and present as part of that pipeline a data association method that is able to robustly identify and associate individual constellation points in the presence of clutter and occlusions. We demonstrate the accuracy and efficiency of this pose estimation approach on real-world data obtained from multiple flight tests, and show that we can obtain decimeter level accuracy from distances of over 100 m from the constellation. To achieve greater robustness to the potentially large number of outlier infrared detections that can arise in urban environments, we also explore learning-based approaches to the outlier rejection and data association problems. By formulating the problem of camera image data association as a 2D point cloud analysis, we can apply deep learning methods designed for 3D point cloud segmentation to achieve robust, high-accuracy associations at constant real-time speeds on infrared images with high outlier-to-inlier ratios. We again demonstrate the efficiency of our learning-based approach on both synthetic and real-world data, and compare the results and limitations of this method to our first-principles-based data association approach.

<p dir="ltr">There is growing research in analyzing human gait in the context of various applications. This has been aided by the improvement in sensing technologies and computation power. A complex motor skill that it is, gait has found its use in medicine for diagnosing different neurological ailments and injuries. In sports, gait can be used to provide feedback to the player/athlete to improve his/her skill and to prevent injuries. In biometrics, gait can be used to identify and authenticate individuals. This can be easier to scale to perform biometrics of individuals in large crowds compared to conventional biometric methods. In the field of Human Computer Interaction (HCI), gait can be an additional input that could be provided to be used in applications such as video games. Gait analysis has also been used for Human Activity Recognition (HAR) for purposes such as personal fitness, elderly care and rehabilitation. </p><p dir="ltr">The current state-of-the-art methods for gait analysis involves non-wearable technology due to its superior performance. The sophistication afforded in non-wearable technologies, such as cameras, is better able to capture gait information as compared to wearables. However, non-wearable systems are expensive, not scalable and typically, inaccessible to the general public. These systems sometimes need to be set up in specialized clinical facilities by experts. On the other hand, wearables offer scalability and convenience but are not able to match the performance of non-wearables. So the current work is a step in the direction to bridge the gap between the performance of non-wearable systems and the convenience of wearables. </p><p dir="ltr">A magnetic tracking system is developed to be applied for gait analysis. The system performs position and orientation tracking, i.e. 6 degrees of freedom or 6DoF tracking. One or more tracker modules, called Rx modules, is tracked with respect to a module called the Tx module. The Tx module mainly consists of a magnetic field generating coil, Inertial Measurement Unit (IMU) and magnetometer. The Rx module mainly consists of a tri-axis sensing coil, IMU and magnetometer. The system is minimally intrusive, works with Non-Line-of-Sight (NLoS) condition, low power consuming, compact and light weight. </p><p dir="ltr">The magnetic tracking system has been applied to the task of Human Activity Recognition (HAR) in this work as a proof-of-concept. The tracking system was worn by participants, and 4 activities - walking, walking with weight, marching and jogging - were performed. The Tx module was worn on the waist and the Rx modules were placed on the feet. To compare magnetic tracking with the most commonly used wearable sensors - IMUs + magnetometer - the same system was used to provide IMU and magnetometer data for the same 4 activities. The gait data was processed by 2 commonly used deep learning models - Convolutional Neural Network (CNN) and Long Short Term Memory (LSTM). The magnetic tracking system shows an overall accuracy of 92\% compared to 86.69\% of the IMU + magnetometer system. Moreover, an accuracy improvement of 8\% is seen with the magnetic tracking system in differentiating between the walking and walking with weight activities, which are very similar in nature. This goes to show the improvement in gait information that 6DoF tracking brings, that manifests as increased classification accuracy. This increase in gait information will have a profound impact in other applications of gait analysis as well.</p>

Orientation estimation of objects plays a pivotal role in robotics, self-driving cars, and augmented reality. Beyond mere position, accurately determining the orientation of objects is essential for constructing precise models of the physical world. While 2D object detection has made significant strides, the field of orientation estimation still faces several challenges. Our research addresses these hurdles by proposing an efficient pipeline which facilitates rapid creation of labeled training data and enables direct regression of object orientation from a single image. We start by creating a digital twin of a physical object using an iPhone, followed by generating synthetic images using the Unity game engine and domain randomization. Our deep learning model, trained exclusively on these synthetic images, demonstrates promising results in estimating the orientations of common objects. Notably, our model achieves a median geodesic distance error of 3.9 degrees and operates at a brisk 15 frames per second.