Golf is a sport where the correct technical execution is important for performance and injury prevention. The existing feedback systems are often cumbersome and not readily available to recreational players. To address this issue, this thesis explores the potential of using 2D Human Pose Estimation as a mobile phone-based swing analysis tool. The developed system allows to identify three events in the swing movement (toe-up, top and impact) and to measure specific angles during these events by using an algorithmic approach. The system focuses on quantifying the knee flexion and primary spine angle during the address, and lateral bending at the top of the swing. By using only the wrist coordinates in the vertical direction, the developed system identified 37% of investigated events, independently of whether the swing was filmed in the frontal of sagittal frame. Within five frames, 95% of the events were correctly identified. Using additional joint coordinates and the event data obtained by the above-mentioned event identification algorithm, the knee flexion at address was correctly assessed in 66% of the cases, with a mean absolute error of 3.7°. The mean absolute error of the primary spine angle measurement at address was of 10.5°. The lateral bending angle was correctly identified in 87% ofthe videos. This system highlights the potential of using 2D Human Pose Estimation for swing analysis. This thesis primarily focused on exploring the feasibility of the approach and further research is needed to expand the system and improve its accuracy. This work serves as a foundation, providing valuable insights for future advancements in the field of 2D Human Pose Estimation-based swing analysis. / Golf är en sport där korrekt tekniskt utförande är avgörande för prestation och skadeförebyggelse. Feedbacksystem som finns är ofta besvärliga och inte lättillgängliga för fritidsspelare. För att åtgärda detta problem undersöker detta examensarbete potentialen att använda 2D mänsklig poseuppskattning som mobiltelefonsbaserat svinganalysverktyg. Det utvecklade systemet gör det möjligt att identifiera tre händelser i svingen (toe-up, top och impact) och att mäta specifika vinklar under dessa händelser genom en algoritmisk metod. Systemet fokuserar på att kvantifiera knäböjningen och primära ryggradsvinkeln under uppställningen, och laterala böjningen vid svingtoppen. Genom att endast använda handledskoordinater i vertikalriktning identifierade det utvecklade systemet 37% av de undersökta händelserna oavsett om svingen filmades från frontal- eller medianplanet. Inom fem bildrutor identifierades 95% av händelserna korrekt. Genom att använda ytterligare ledkoordinater och händelsedata som erhållits genom den tidigare nämnda algoritmen för händelseidentifiering, bedömdes knäböjningen vid uppställningen vara korrekt i 66% av fallen med en medelabsolutfel på 3.7°. Medelabsolutfelet för mätningen av primär ryggradsvinkel vid uppställningen var 10.5°. Laterala böjningen identifierades korrekt i 87% av tillfällena. Detta system belyser potentialen i 2D mänsklig poseuppskattning för svinganalys. Detta examensarbete fokuserade främst på att utforska tillvägagångssättets genomförbarhet och ytterligare forskning behövs för att utveckla systemet och förbättra dess noggrannhet. Detta arbete är grundläggande och ger värdefulla insikter för framtida forskning inom området för svinganalys baserad på 2D mänsklig poseuppskattning.

<p>This thesis aims to design and develop the spatial adaptation approach through spatial transformers to improve the accuracy of human keypoint recognition models. We have studied different model types and design choices to gain an accuracy increase over models without spatial transformers and analyzed how spatial transformers increase the accuracy of predictions. A neural network called Widenet has been leveraged as a specialized network for providing the parameters for the spatial transformer. Further, we have evaluated methods to reduce the model parameters, as well as the strategy to enhance the learning supervision for further improving the performance of the model. Our experiments and results have shown that the proposed deep learning framework can effectively detect the human key points, compared with the baseline methods. Also, we have reduced the model size without significantly impacting the performance, and the enhanced supervision has improved the performance. This study is expected to greatly advance the deep learning of human key points and movement dynamics. </p>

This thesis presents methods useful in a bin picking application, such as detection and representation of local features, pose estimation and multi-cue integration. The scene tensor is a representation of multiple line or edge segments and was first introduced by Nordberg in [30]. A method for estimating scene tensors from gray-scale images is presented. The method is based on orientation tensors, where the scene tensor can be estimated by correlations of the elements in the orientation tensor with a number of 1D filters. Mechanisms for analyzing the scene tensor are described and an algorithm for detecting interest points and estimating feature parameters is presented. It is shown that the algorithm works on a wide spectrum of images with good result. Representations that are invariant with respect to a set of transformations are useful in many applications, such as pose estimation, tracking and wide baseline stereo. The scene tensor itself is not invariant and three different methods for implementing an invariant representation based on the scene tensor is presented. One is based on a non-linear transformation of the scene tensor and is invariant to perspective transformations. Two versions of a tensor doublet is presented, which is based on a geometry of two interest points and is invariant to translation, rotation and scaling. The tensor doublet is used in a framework for view centered pose estimation of 3D objects. It is shown that the pose estimation algorithm has good performance even though the object is occluded and has a different scale compared to the training situation. An industrial implementation of a bin picking application have to cope with several different types of objects. All pose estimation algorithms use some kind of model and there is yet no model that can cope with all kinds of situations and objects. This thesis presents a method for integrating cues from several pose estimation algorithms for increasing the system stability. It is also shown that the same framework can also be used for increasing the accuracy of the system by using cues from several views of the object. An extensive test with several different objects, lighting conditions and backgrounds shows that multi-cue integration makes the system more robust and increases the accuracy. Finally, a system for bin picking is presented, built from the previous parts of this thesis. An eye in hand setup is used with a standard industrial robot arm. It is shown that the system works for real bin-picking situations with a positioning error below 1 mm and an orientation error below 1o degree for most of the different situations. / <p>Report code: LiU-TEK-LIC-2005:15.</p>

The focus of this work is the task of 3D human pose estimation, more specifically by making use of key points located in single monocular images in order to estimate the location of human body joints in a 3D space. It was done in association with Tracab, a company based in Stockholm, who specialises in advanced sports tracking and analytics solutions. Tracab’s core product is their optical tracking system for football, which involves installing multiple highspeed cameras around the sports venue. One of the main benefits of this work will be to reduce the number of cameras required to create the 3D skeletons of the players, hence reducing production costs as well as making the whole process of creating the 3D skeletons much simpler in the future. The main problem we are tackling consists in going from a set of 2D joint locations and lifting them to a 3D space, which would add an information of depth to the joint locations. One problem with this task is the limited availability of in-thewild datasets with corresponding 3D ground truth labels. We hope to tackle this issue by making use of the restricted Human3.6m dataset along with the Tracab dataset in order to achieve adequate results. Since the Tracab dataset is very large, i.e millions of unique poses and skeletons, we have focused our experiments on a single football game. Although extensive research has been done in the field by using architectures such as convolutional neural networks, transformers, spatial-temporal architectures and more, we are tackling this issue by making use of a simple feedforward neural network developed by Martinez et al, this is mainly possible due to the abundance of data available at Tracab. / Fokus för detta arbete är att estimera 3D kroppspositioner, genom att använda detekterade punkter på människokroppen i enskilda monokulära bilder för att uppskatta 3D positionen av dessa ledpunkter. Detta arbete genomfördes i samarbete med Tracab, ett företag baserat i Stockholm, som specialiserar sig på avancerade lösningar för följning och analys inom idrott. Tracabs huvudprodukt är deras optiska följningssystem, som innebär att flera synkroniserade höghastighetskameror installeras runt arenan. En av de främsta fördelarna med detta arbete kommer att vara att minska antalet kameror som krävs för att skapa 3D-skelett av spelarna, vilket minskar produktionskostnaderna och förenklar hela processen för att skapa 3D-skelett i framtiden. Huvudproblemet vi angriper är att gå från en uppsättning 2D-ledpunkter och lyfta dem till 3D-utrymme. Ett problem är den begränsade tillgången till datamängder med 3D ground truth från realistiska miljöer. Vi angriper detta problem genom att använda den begränsade Human3.6m-datasetet tillsammans med Tracab-datasetet för att uppnå tillräckliga resultat. Eftersom Tracab-datamängden är mycket stor, med miljontals unika poser och skelett, .har vi begränsat våra experiment till en fotbollsmatch. Omfattande forskning har gjorts inom området med användning av arkitekturer som konvolutionella neurala nätverk, transformerare, rumsligttemporala arkitekturer med mera. Här använder vi ett enkelt framåtriktat neuralt nätverk utvecklat av Martinez et al, vilket är möjligt tack vare den stora mängden data som är tillgänglig hos Tracab.

В области компьютерного зрения оценка позы человека приобретает все большее значение. Это одна из самых привлекательных областей исследований, и она вызывает большой интерес благодаря своей полезности и гибкости в самых разных областях, включая здравоохранение, игры, дополненную реальность, виртуальные тренировки и спорт. На ряду с этим люди все чаще начинают заниматься спортом. А в спорте травмы неизбежны. В данной статье предлагается приложение для оценки выполнения фитнес-упражнений, которое контролирует правильность техники и дает обратную связь по ее исправлению, что помогает уменьшить травматизм при занятиях. Предварительно обученная модель MediaPipe использовалась для оценки поз, по результатам которой вычисляются углы между конкретными суставами. / In the field of computer vision, human pose estimation is becoming increasingly important. This is one of the most attractive areas of research, and it is of great interest due to its usefulness and flexibility in a wide variety of fields, including healthcare, games, augmented reality, virtual training, and sports. Along with this, people are increasingly starting to do sports. And in sports, injuries are inevitable. This article offers an application for evaluating the performance of fitness exercises, which monitors the correctness of the technique and gives feedback on its correction, which helps to reduce injuries during classes. A pre-trained MediaPipe model was used to evaluate poses, based on the results of which the angles between specific joints are calculated.

The deep learning technique Human Pose Estimation (or Human Keypoint Detection) is a promising field in tracking a person and identifying its posture. As posture and balance are two closely related concepts, the use of human pose estimation could be applied to fall prevention. By deriving the location of a persons Center of Mass and thereafter its Center of Pressure, one can evaluate the balance of a person without the use of force plates or sensors and solely using cameras. In this study, a human pose estimation model together with a predefined human weight distribution model were used to extract the location of a persons Center of Pressure in real time. The proposed method utilized two different methods of acquiring depth information from the frames - stereoscopy through two RGB-cameras and with the use of one RGB-depth camera. The estimated location of the Center of Pressure were compared to the location of the same parameter extracted while using the force plate Wii Balance Board. As the proposed method were to operate in real-time and without the use of computational processor enhancement, the choice of human pose estimation model were aimed to maximize software input/output speed. Thus, three models were used - one smaller and faster model called Lightweight Pose Network, one larger and accurate model called High-Resolution Network and one model placing itself somewhere in between the two other models, namely Pose Residual Network. The proposed method showed promising results for a real-time method of acquiring balance parameters. Although the largest source of error were the acquisition of depth information from the cameras. The results also showed that using a smaller and faster human pose estimation model proved to be sufficient in relation to the larger more accurate models in real-time usage and without the use of computational processor enhancement. / Djupinlärningstekniken Kroppshållningsestimation är ett lovande medel gällande att följa en person och identifiera dess kroppshållning. Eftersom kroppshållning och balans är två närliggande koncept, kan användning av kroppshållningsestimation appliceras till fallprevention. Genom att härleda läget för en persons tyngdpunkt och därefter läget för dess tryckcentrum, kan utvärdering en persons balans genomföras utan att använda kraftplattor eller sensorer och att enbart använda kameror. I denna studie har en kroppshållningsestimationmodell tillsammans med en fördefinierad kroppsviktfördelning använts för att extrahera läget för en persons tryckcentrum i realtid. Den föreslagna metoden använder två olika metoder för att utvinna djupseende av bilderna från kameror - stereoskopi genom användning av två RGB-kameror eller genom användning av en RGB-djupseende kamera. Det estimerade läget av tryckcentrat jämfördes med läget av samma parameter utvunnet genom användning av tryckplattan Wii Balance Board. Eftersom den föreslagna metoden var ämnad att fungera i realtid och utan hjälp av en GPU, blev valet av kroppshållningsestimationsmodellen inriktat på att maximera mjukvaruhastighet. Därför användes tre olika modeller - en mindre och snabbare modell vid namn Lightweight Pose Network, en större och mer träffsäker modell vid namn High-Resolution Network och en model som placerar sig någonstans mitt emellan de två andra modellerna gällande snabbhet och träffsäkerhet vid namn Pose Resolution Network. Den föreslagna metoden visade lovande resultat för utvinning av balansparametrar i realtid, fastän den största felfaktorn visade sig vara djupseendetekniken. Resultaten visade att användning av en mindre och snabbare kroppshållningsestimationsmodellen påvisar att hålla måttet i jämförelse med större och mer träffsäkra modeller vid användning i realtid och utan användning av externa dataprocessorer.

Autonomous navigation for ground vehicles has many challenges. Autonomous systems must be able to self-localise, avoid obstacles and determine navigable surfaces. This thesis studies several aspects of autonomous navigation with a particular emphasis on vision, motivated by it being a primary component for navigation in many high-level biological organisms.  The key problem of self-localisation or pose estimation can be solved through analysis of the changes in appearance of rigid objects observed from different view points. We therefore describe a system for structure and motion estimation for real-time navigation and obstacle avoidance. With the explicit assumption of a calibrated camera, we have studied several schemes for increasing accuracy and speed of the estimation.The basis of most structure and motion pose estimation algorithms is a good point tracker. However point tracking is computationally expensive and can occupy a large portion of the CPU resources. In thisthesis we show how a point tracker can be implemented efficiently on the graphics processor, which results in faster tracking of points and the CPU being available to carry out additional processing tasks.In addition we propose a novel view interpolation approach, that can be used effectively for pose estimation given previously seen views. In this way, a vehicle will be able to estimate its location by interpolating previously seen data.Navigation and obstacle avoidance may be carried out efficiently using structure and motion, but only whitin a limited range from the camera. In order to increase this effective range, additional information needs to be incorporated, more specifically the location of objects in the image. For this, we propose a real-time object recognition method, which uses P-channel matching, which may be used for improving navigation accuracy at distances where structure estimation is unreliable. / Diplecs

This thesis aimed to assess the feasibility of using an off-the-shelf virtual reality tracking system as a low cost precision pose estimation solution for robotic operations in both indoor and outdoor environments. Such a tracking solution has the potential of assisting critical operations related to planetary exploration missions, parcel handling/delivery, and wildfire detection/early warning systems. The boom of virtual reality experiences has accelerated the development of various low-cost, precision indoor tracking technologies. For the purpose of this thesis we choose to adapt the SteamVR Lighthouse system developed by Valve, which uses photo-diodes on the trackers to detect the rotating IR laser sheets emitted from the anchored base stations, also known as lighthouses. Some previous researches had been completed using the first generation of lighthouses, which has a few limitations on communication from lighthouses to the tracker. A NASA research has cited poor tracking performance under sunlight. We choose to use the second generation lighthouses which has improved the method of communication from lighthouses to the tracker, and we performed various experiments to assess their performance outdoors, including under sunlight. The studies of this thesis have two stages, the first stage focused on a controlled, indoor environment, having an Unmanned Aerial Vehicle (UAS) perform repeatable flight patterns and simultaneously tracked by the Lighthouse and a reference indoor tracking system, which showed that the tracking precision of the lighthouse is comparable to the industrial standard indoor tracking solution. The second stage of the study focused on outdoor experiments with the tracking system, comparing UAS flights between day and night conditions as well as positioning accuracy assessments with a CNC machine under indoor and outdoor conditions. The results showed matching performance between day and night while still comparable to industrial standard indoor tracking solution down to centimeter precision, and matching simulated CNC trajectory down to millimeter precision. There is also some room for improvement in regards to the experimental method and equipment used, as well as improvements on the tracking system itself needed prior to adaptation in real-world applications. / Master of Science / This thesis aimed to assess the feasibility of using an off-the-shelf virtual reality tracking system as a low cost precision pose estimation solution for robotic operations in both indoor and outdoor environments. Such a tracking solution has the potential of assisting critical operations related to planetary exploration missions, parcel handling/delivery, and wildfire detection/early warning systems. The boom of virtual reality experiences has accelerated the development of various low-cost, precision indoor tracking technologies. For the purpose of this thesis we choose to adapt the SteamVR Lighthouse system developed by Valve, which uses photo-diodes on the trackers to detect the rotating IR laser sheets emitted from the anchored base stations, also known as lighthouses. Some previous researches had been completed using the first generation of lighthouses, which has a few limitations on communication from lighthouses to the tracker. A NASA research has cited poor tracking performance under sunlight. We choose to use the second generation lighthouses which has improved the method of communication from lighthouses to the tracker, and we performed various experiments to assess their performance outdoors, including under sunlight. The studies of this thesis have two stages, the first stage focused on a controlled, indoor environment, having an Unmanned Aerial Vehicle (UAS) perform repeatable flight patterns and simultaneously tracked by the Lighthouse and a reference indoor tracking system, which showed that the tracking precision of the lighthouse is comparable to the industrial standard indoor tracking solution. The second stage of the study focused on outdoor experiments with the tracking system, comparing UAS flights between day and night conditions as well as positioning accuracy assessments with a CNC machine under indoor and outdoor conditions. The results showed matching performance between day and night while still comparable to industrial standard indoor tracking solution down to centimeter precision, and matching simulated CNC trajectory down to millimeter precision. There is also some room for improvement in regards to the experimental method and equipment used, as well as improvements on the tracking system itself needed prior to adaptation in real-world applications.

This thesis aims to address the concurrent challenges of multi-person 2D pose estimation and object detection within a unified bottom-up framework. Our foundational solutions encompass a recently proposed pose estimation framework named OpenPifPaf, grounded in composite fields. OpenPifPaf employs the Composite Intensity Field (CIF) for precise joint localization and the Composite Association Field (CAF) for seamless joint connectivity. To assess the model’s robustness against scale variances, a Feature Pyramid Network (FPN) is incorporated into the baseline. Additionally, we present a variant of OpenPifPaf known as CifDet. CifDet utilizes the Composite Intensity Field to classify and detect object centers, subsequently regressing bounding boxes from these identified centers. Furthermore, we introduce an extended version of CifDet specifically tailored for enhanced object detection capabilities—CifCafDet. This augmented framework is designed to more effectively tackle the challenges inherent in object detection tasks. The baseline OpenPifPaf model outperforms most existing bottom-up pose estimation methods and achieves comparable results with some state-of-the-art top-down methods on the COCO keypoint dataset. Its variant, CifDet, adapts the OpenPifPaf’s composite field-based architecture for object detection tasks. Further modifications result in CifCafDet, which demonstrates enhanced performance on the MS COCO detection dataset over CifDet, suggesting its viability as a multi-task framework. / Denna avhandling syftar till att ta itu med de samtidiga utmaningarna med flerpersons 2D-posestimering och objektdetektion inom en enhetlig bottom-up-ram. Våra grundläggande lösningar omfattar ett nyligen föreslaget ramverk för posestimering med namnet OpenPifPaf, som grundar sig i kompositfält. OpenPifPaf använder Composite Intensity Field (CIF) för exakt leddlokalisering och Composite Association Field (CAF) för sömlös ledanslutning. För att bedöma modellens robusthet mot skalvariationer införlivas ett Feature Pyramid Network (FPN) i baslinjen. Dessutom presenterar vi en variant av OpenPifPaf känd som CifDet. CifDet använder Composite Intensity Field för att klassificera och detektera objektcentrum, för att sedan regrediera inramningslådor från dessa identifierade centrum. Vidare introducerar vi en utökad version av CifDet som är speciellt anpassad för förbättrade objektdetekteringsförmågor—CifCafDet. Detta förstärkta ramverk är utformat för att mer effektivt ta itu med de utmaningar som är inneboende i objektdetekteringsuppgifter. Basmodellen OpenPifPaf överträffar de flesta befintliga bottom-up-metoder för posestimering och uppnår jämförbara resultat med vissa toppmoderna top-down-metoder på COCO-keypoint-datasetet. Dess variant, CifDet, anpassar OpenPifPafs kompositfältbaserade arkitektur för objekt-detekteringsuppgifter. Ytterligare modifieringar resulterar i CifCafDet, som visar förbättrad prestanda på MS COCO-detektionsdatasetet över CifDet, vilket antyder dess livskraft som ett ramverk för flera uppgifter.

Ice hockey is a sport whose athletes are at high risk for traumatic head injuries due to the violence of potential impacts with other athletes, ice, or glass during games. In order to develop the best protective strategies for the players, it is necessary to have a deep understanding of accident mechanisms during ice hockey games. Accident reconstructions using the finite element (FE) method are a way to perform a systematic analysis of impact cases, but require input data on the circumstances of the accidents. Thus, this project focused on finding a way to extract the position and velocity of the players involved from readily available videos of ice hockey accidents using motion tracking methods. This project included two parts: pose estimation and velocity estimation. The pose estimation aimed to align a human body model (HBM) with the players' poses and the key steps included estimating 2D joints from impact images, estimating the players' 3D poses, skeleton inferencing, and aligning the results with the baseline HBM via pelvic registration. The velocity estimation defined the initial conditions for simulating the collision and key steps included identifying the players' 2D joints across impact video frames, tracking of the players using a simplified pelvis projection on the rink plane, and estimating the players’ velocity using homography to identify their position on the ice hockey rink. Then, both parts were applied to accident cases from a video database of collisions that occurred during a hockey league season. The cases in which the pipeline was fully applied ultimately resulted in LS-DYNA positioning files for the Total Human Model for Safety (THUMS) model, and problematic cases were used to get an overview of the limits of the chosen methodology. Said limitations were mostly linked to the quality of the source videos, which is highly dependent on the source of the videos and possibly not controllable. Due to this, selection criteria are required, such as checking the blurriness and quality of the videos and the viewing angles to ensure as few occlusions as possible. Overall, this project resulted in a working semi-automatic pipeline for pose and velocity estimation in contact sports collisions, as well as a first set of personalized input information that should allow the reconstruction of ice hockey accidents using FE simulations. / Ishockey är en sport vars utövare löper stor risk att drabbas av traumatiska huvudskador på grund av de våldsamma potentiella kollisionerna med andra utövare, is eller glas under matcherna. För att kunna utveckla de bästa skyddsstrategierna för spelarna är det nödvändigt att ha en djup förståelse för olycksmekanismerna under ishockeymatcher. Olycksrekonstruktioner med hjälp av finita elementmetoden är ett sätt att utföra en systematisk analys av kollisionsfall, men kräver indata om omständigheterna kring olyckorna. Detta projekt fokuserade därför på att hitta ett sätt att extrahera de inblandade spelarnas position och hastighet från lättillgängliga videor av ishockeyolyckor med hjälp av rörelsespårningsmetoder. Projektet bestod av två delar: poseuppskattning och hastighetsuppskattning. Poseuppskattningen syftade till att anpassa en humanmodell till spelarnas poser och de viktigaste stegen omfattade uppskattning av 2D-leder från kollisionsbilder, uppskattning av spelarnas 3D-poser, skelettinferens och anpassning av resultaten till baslinjen HBM via bäckenregistrering. Hastighets-uppskattningen definierade de initiala villkoren för simulering av kollisionen och viktiga steg inkluderade identifiering av spelarnas 2D-led i videobilder av kollisionen, spårning av spelarna med hjälp av en förenklad bäckenprojektion på rinkplanet och uppskattning av spelarnas hastighet med hjälp av homografi för att identifiera deras position på ishockeyrinken. Därefter tillämpades båda delarna på olycksfall från en videodatabas med kollisioner som inträffade under en säsong i en hockeyliga. De fall där pipelinen tillämpades fullt ut resulterade slutligen i LS-DYNA-positioneringsfiler, och problematiska fall användes för att få en överblick över gränserna för den valda metoden. Begränsningarna var främst kopplade till kvaliteten på källvideorna, som är starkt beroende av källan till videorna och eventuellt inte kan kontrolleras. På grund av detta krävs urvalskriterier, t.ex. kontroll av videornas oskärpa och kvalitet samt betraktningsvinklar för att säkerställa så få ocklusioner som möjligt. Sammantaget resulterade detta projekt i en fungerande halvautomatisk pipeline för pose- och hastighetsuppskattning vid kollisioner i kontaktsporter, samt en första uppsättning personlig indatainformation som bör möjliggöra rekonstruktion av ishockeyolyckor med hjälp av simulering med finita element.