In the field of robotics and self-navigation, Simultaneous Localization and Mapping (SLAM) is a technique crucial for estimating poses while concurrently creating a map of the environment. Robotics applications often rely on various sensors for pose estimation, including cameras, inertial measurement units (IMUs), and more. Traditional discrete SLAM, utilizing stereo camera pairs and inertial measurement units, faces challenges such as time offsets between sensors. A solution to this issue is the utilization of continuous-time models for pose estimation. This thesis delves into the exploration and implementation of a continuous-time SLAM system, investigating the advantages of multi-modal sensor fusion over discrete stereo vision models. The findings indicate that incorporating an IMU into the system enhances pose estimation, providing greater robustness and accuracy compared to relying solely on visual SLAM. Furthermore, leveraging the continuous model's derivative and smoothness allows for decent pose estimation with fewer measurements, reducing the required quantity of measurements and computational resources.

Today’s most widely used tracking solutions for consumers involve the Global Positioning System (GPS) which meets most needs when it comes to rough estimation of location. GPS however is limited in accuracy with a horizontal error of around 5 meters and cannot be used in the areas where the satellites cannot provide a strong enough signal e.g. indoor areas or near mountains and other sources of blockage. Ultra-wideband (UWB) abolishes these two issues, providing an accuracy at centimeter level and works great in indoor areas. This thesis dives into the theory behind tracking devices with UWB and includes an implementation of the tracking as well as covers noteworthy issues, shortcomings, and future work. The app that is developed within this thesis runs on Android mobile devices and can locate and track another Android mobile device running the same app. Results were clear that the concept works, but more filtering needs to be done in order to remove the remaining noise.

Hydrofoiling in general has the potential to revolutionize watercraft in the future since it allows smoother and faster transport on water with less energy consumption than traditional planning hulls. Even if the concept of hydrofoiling has been around since the last century, development in control theory and material science together with increased computing power has led to a growing interest for the technology. Especially in water sports such as speed sailing and surfing due to its superiority in speed and comfort. Researchers and students at the Engineering Mechanics Department at KTH, Royal Institute of Technology, Stockholm are working on a new type of watercraft, utilizing only one single hydrofoil with the intention to minimize drag for faster and smoother rides in various wave and weather conditions. The difficulties lie in understanding the relationship between actuators and the mechanics. This thesis is a continuation work from a previous thesis which designed a control strategy based on a model with 4 degrees of freedom (DOF). Due to simplifications and linearizations, the 4 DOF model was not rich enough to meet the performance requirements. This thesis presents a 6 DOF model by deriving the mechanical equations for the spherical inverted pendulum and actuation from the hydrofoiling module. The inverted pendulum model is a well-known control problem that can be solved with different strategies. By showing that the hydrofoiling concept can be modelled as an inverted pendulum, it is also shown that it can be controlled as an inverted pendulum. The derived model is used together with an Extended Kalman Filter to create an observer. The observer is validated with a spherical inverted pendulum model in Matlab and the block diagram environment, Simulink. Simulation results show that the 6 DOF model is able to produce accurate state estimation of the watercraft even in the presence of stochastic measurement noise. It is also concluded that viscous forces, that arise from the watercraft being partly surrounded by water and partly by air, need further investigation. / Principen för bärplan är att generera lyftkraft från vattnet på samma sätt som flygplansvingar genererar lyftkraft från luften för att lyfta farkostens skrov ur vattnet. Detta minskar motståndet från friktionen mellan skrov och vatten vilket möjliggör snabbare och jämnare transport på vatten med en lägre energiförbrukning än traditionella planande skrov. På senare år har tekniken fått ett uppsving i och med framsteg inom strömningsmekanik, reglerteknik och materiallära. Detta i takt med datorers ökande beräkningskraft har lett till att bärplanskonstruktioner har kunnat uppvisa en överlägsenhet i vattensporter som kappsegling och surfing när det kommer till fart och komfort. Forskare och studenter på avdelningen för farkostteknik och solidmekanik vid Kungliga Tekniska Högskolan, Stockholm arbetar med att ta fram en ny typ av farkost med en minimal bärplansdesign, FoilCart. Dess utformning gör att det mekaniska beteendet kan liknas vid en inverterad pendel, vilket är ett välkänt, olinjärt reglerproblem som kan lösas på flera sätt. Denna avhandling är ett vidarearbete som bygger på en modell med fyra frihetsgrader från en tidigare avhandling kring FoilCart-projektet. Modellen med fyra frihetsgrader var, på grund av förenklingar och linjärisering av systemdynamiken, bristfällig och kunde inte garantera en robust balansering av farkosten förutom i linjäriseringspunkten. Modellen som presenteras i denna avhandling har sex frihetsgrader. Mekaniken och systemdynamiken härleds från den sfäriska inverterade pendeln tillsammans med styrningen från bärplansmodulen, utan förenklingar och linjärisering. Modellen används i ett Kalmanfilter för att konstruera en observatör för tillståndsrekonstruktion. Den framtagna modellen valideras med en FoilCart-modell i Simulink. Resultaten visar att observatören kan ge en noggrann tillståndsrekonstruktion även vid simulerat mätbrus i mätsignalen. Avhandlingen syftar till att visa hur den inverterade pendelmodellen kan användas vid framtida implementation av rekonstruerad tillståndsåterkoppling. I och med avgränsningar i avhandlingen finns det också en del strömningsmekaniska aspekter som inte tagits med vid framtagningen av denna modell. Eftersom farkosten delvis är omgiven av vatten och delvis av luft skulle det vara intressant att undersöka om noggrannheten i tillståndsrekonstruktionen kan förbättras genom att använda avancerad strömningsmekanik.

This bachelor's thesis explores the integration of machine learning (ML) with sensor fusion techniques to enhance spatial data accuracy in Extended Reality (XR) environments. With XR's revolutionary impact across various sectors, accurate localization in virtual environments becomes imperative. The thesis conducts a comprehensive literature review, highlighting advancements in indoor positioning technologies and the pivotal role of machine learning in refining sensor fusion for precise localization. It underscores the challenges in the XR field, such as signal interference, device heterogeneity, and data processing complexities. Through critical analysis, this study aims to bridge the gap in practical application of ML, offering insights into developing scalable solutions for immersive virtual productions. It offers insights into the practical integration of advanced machine learning techniques in XR applications, thereby providing valuable implications for technology development and user experience in XR. This contribution is not merely theoretical; it showcases practical applications and advancements in real-time processing and adaptability in complex environments, aligning well with existing research and extending it by addressing scalability and practical implementation challenges in XR environments. This study identifies key themes in the integration of ML with sensor fusion for XR, such as the enhancement of spatial data accuracy, challenges in real-time processing, and the need for scalable solutions. It concludes that the fusion of ML and sensor technologies not only enhances the accuracy of XR environments but also paves the way for more immersive and realistic virtual experiences.

No description available.

Autonomous navigation has the opportunity to make roads safer and help perform search and rescue missions by reducing human error. Odometry methods are essential to allow for autonomous navigation because they estimate how the robot will move based on the available sensors. This thesis aims to compare and evaluate the Cubature Kalman filter (CKF) based approach for visual-inertial odometry (VIO) to traditional Extended Kalman Filter (EKF) based methods on criteria such as the accuracy of the results. VIO methods use camera and IMU sensor for the predictions. The Multi-State-Constraint Kalman filter (MSCKF) was utilized as the foundation VIO approach to evaluate the underlying filter between EKF and CKF while maintaining the background conditions like visual tracking pipeline, IMU model, and measurement model constant. Evaluation metrics of absolute trajectory error (ATE) and relative error (RE) was used after tuning the filters on EuRoC and KAIST datasets. It is shown that, based on the existing implementation, the filters have no statistically significant difference in performance when predicting motion estimates, despite the fact that the absolute trajectory error of position for EKF estimation is lower. It is further shown that as the length of the trajectory increases, the estimation error for both filters rises unboundedly. Under the visual inertial framework of MSCKF, the CKF filter, which does not linearize the system, works equally as well as the well-established EKF filter and has the potential to perform better with more accurate nonlinear system and measurement models. / Autonom navigering har möjlighet att göra vägar säkrare och hjälpa till att utföra räddningsuppdrag genom att minska mänskliga fel. Odometrimetoder är viktiga för att möjliggöra autonom navigering eftersom de skattar hur roboten rör sig baserat på tillgängliga sensorer. Detta examensarbete syftar till att utvärdera Cubature Kalman filter (CKF) för visuell tröghetsodometri (VIO) och jämföra med traditionella Extended Kalman Filter (EKF) gällande bland annat noggrannhet. VIO-metoder använder kamera och IMU-sensor för skattningarna. MultiState Constraint Kalmanfiltret (MSCKF) användes som grund VIO-metoden för att utvärdera filteralgoritmerna EKF och CKF, samtidigt som de VIO-specifika delarna så som IMU-modell och mätmodell kunde förbli desamma. Utvärderingen gjordes baserat på absolut banfel (ATE) och relativa fel (RE) på EuRoC- och KAIST-datauppsättningar. Det visas att, baserat på den befintliga implementeringen, har filtren ingen statistiskt signifikant skillnad i prestanda när de förutsäger rörelsen, trots att det absoluta banafelet för positionen för EKF-uppskattning är lägre. Det visas vidare att när längden på banan ökar, ökar uppskattningsfelet för båda filtren obegränsat. Under MSCKFs visuella tröghetsramverk fungerar CKF-filtret, som inte linjäriserar systemet, lika bra som det väletablerade EKF-filtret och har potential att prestera bättre med mer exakta olinjära system och mätmodeller.

<p dir="ltr">This dissertation delves into a significant challenge for Autonomous Vehicles (AVs): achieving efficient and robust perception under adverse weather and lighting conditions. Systems that rely solely on cameras face difficulties with visibility over long distances, while radar-only systems struggle to recognize features like stop signs, which are crucial for safe navigation in such scenarios.</p><p dir="ltr">To overcome this limitation, this research introduces a novel deep camera-radar fusion approach using neural networks. This method ensures reliable AV perception regardless of weather or lighting conditions. Cameras, similar to human vision, are adept at capturing rich semantic information, whereas radars can penetrate obstacles like fog and darkness, similar to X-ray vision.</p><p dir="ltr">The thesis presents NeXtFusion, an innovative and efficient camera-radar fusion network designed specifically for robust AV perception. Building on the efficient single-sensor NeXtDet neural network, NeXtFusion significantly enhances object detection accuracy and tracking. A notable feature of NeXtFusion is its attention module, which refines critical feature representation for object detection, minimizing information loss when processing data from both cameras and radars.</p><p dir="ltr">Extensive experiments conducted on large-scale datasets such as Argoverse, Microsoft COCO, and nuScenes thoroughly evaluate the capabilities of NeXtDet and NeXtFusion. The results show that NeXtFusion excels in detecting small and distant objects compared to existing methods. Notably, NeXtFusion achieves a state-of-the-art mAP score of 0.473 on the nuScenes validation set, outperforming competitors like OFT by 35.1% and MonoDIS by 9.5%.</p><p dir="ltr">NeXtFusion’s excellence extends beyond mAP scores. It also performs well in other crucial metrics, including mATE (0.449) and mAOE (0.534), highlighting its overall effectiveness in 3D object detection. Visualizations of real-world scenarios from the nuScenes dataset processed by NeXtFusion provide compelling evidence of its capability to handle diverse and challenging environments.</p>

<p dir="ltr">In the realm of autonomous vehicle technology, the Multimodal Vehicle Detection Network (MVDNet) represents a significant leap forward, particularly in the challenging context of weather conditions. This paper focuses on the enhancement of MVDNet through the integration of a multi-head attention layer, aimed at refining its performance. The integrated multi-head attention layer in the MVDNet model is a pivotal modification, advancing the network's ability to process and fuse multimodal sensor information more efficiently. The paper validates the improved performance of MVDNet with multi-head attention through comprehensive testing, which includes a training dataset derived from the Oxford Radar Robotcar. The results clearly demonstrate that the Multi-Head MVDNet outperforms the other related conventional models, particularly in the Average Precision (AP) estimation, under challenging environmental conditions. The proposed Multi-Head MVDNet not only contributes significantly to the field of autonomous vehicle detection but also underscores the potential of sophisticated sensor fusion techniques in overcoming environmental limitations.</p>

La combinaison de données multi-capteurs joue un rôle croissant dans les systèmes de percep- tion artificielle. Les données de profondeur et les capteurs LiDAR en particulier sont devenus la norme pour les systèmes de vision dans les applications de robotique et de conduite auto- nome. La fusion de capteurs peut améliorer la précision des tâches et a été largement étudiée dans des environnements à ressources élevées, mais elle est moins bien comprise dans les ap- plications où les systèmes peuvent être limités en termes de puissance de calcul et de stockage d’énérgie. Dans l’analyse de la démarche chez l’homme, la compréhension du contexte local de la marche joue un rôle important, et l’analyse en laboratoire à elle même peut limiter la capacité des chercheurs à évaluer correctement la marche réelle des patients. La capacité de classifier automatiquement les terrains de marche dans divers environnements pourrait donc constituer un élément important des systèmes d’analyse de l’activité de marche. Le ter- rain de marche peut être mieux identifié à partir de données visuelles. Plusieurs contraintes (notamment les problèmes de confidentialité liés à l’envoi de données visuelles en temps réel hors appareil) limitent cette tâche de classification au dispositif Edge Computing lui- même, un environnement aux ressources limitées. Ainsi, dans ce travail, nous présentons une architecture de réseau neuronal convolutif parallèle, à fusion tardive et optimisée par calcul de bord pour l’identification des terrains de marche. L’analyse est effectuée sur un nouvel ensemble de données intitulé L-AVATeD: l’ensemble de données Lidar et visibles de terrain de marche, composé d’environ 8000 paires de données de scène visuelles (RVB) et de profondeur (LiDAR). Alors que les modèles formés sur des données visuelles uniquement produisent un modèle de calcul de bord capable d’une précision de 82%, une architecture composée d’instances parallèles de MobileNetV2 utilisant à la fois RVB et LiDAR améliore de manière mesurable la précision de la classification (92%) / Terrain classification is a critical sub-task of many autonomous robotic control processes and important to the study of human gait in ecological contexts. Real-time terrain iden- tification is traditionally performed using computer vision systems with input from visual (camera) data. With the increasing availability of affordable multi-sensor arrays, multi- modal data inputs are becoming ubiquitous in mobile, edge and Internet of Things (IoT) devices. Combinations of multi-sensor data therefore play an increasingly important role in artificial perception systems. Depth data in general and LiDAR sensors in particular are becoming standard for vision systems in applications in robotics and autonomous driving. Sensor fusion using depth data can enhance perception task accuracy and has been widely studied in high resource environments (e.g. autonomous automobiles), but is less well understood in applications where resources may be limited in compute, memory and battery power. An understanding of local walking context also plays an important role in the analysis of gait in humans, and laboratory analysis of on its own can constrain the ability of researchers to properly assess real-world gait in patients. The ability to automatically classify walking terrain in diverse environments is therefore an important part of gait analysis systems for use outside the laboratory. Several important constraints (notably privacy concerns associated with sending real-time image data off-device) restrict this classification task to the edge- computing device, itself a resource-constrained environment. In this study, we therefore present an edge-computation optimized, late-fusion, parallel Convolutional Neural Network (CNN) architecture for the real-time identification of walking terrain. Our analysis is performed on a novel dataset entitled L-AVATeD: the Lidar And Visible wAlking Terrain Dataset, consisting of approximately 8,000 pairs of visual (RGB) and depth (LiDAR) scene data. While simple models trained on visual only data produce an edge-computation model capable of 82% accuracy, an architecture composed of parallel instances of MobileNetV2 using both RGB and LiDAR data, measurably improved classifi- cation accuracy (92%).

This document details the design, implementation, testing, and analysis of an inverted short baseline acoustic positioning system. The system presented here is an above-water, air-based prototype for an underwater acoustic positioning system; it is designed to determine the position of remotely-operated underwater vehicles (ROVs) and autonomous underwater vehicles (AUVs) in the global frame using a method that does not drift over time. A ground-truth positioning system is constructed using a stacked hexapod platform actuator, which mimics the motion of an AUV and provides the true position of an ultrasonic microphone array. An ultrasonic transmitter sends a pulse of sound towards the array; microphones on the array record the pulse of sound and use the time shift between the microphone signals to determine the position of the transmitter relative to the receiver array. The orientation of the array, which is necessary to transform the position estimate to the global frame, is calculated using a Madgwick filter and data from a MEMS IMU. Additionally, a dead reckoning change-in-position estimate is formed using the IMU data. The acoustic position estimate is combined with the dead reckoning estimate using a Kalman filter. The accuracy of this filtered position estimate was verified to 22.1mm within a range of 3.88m in this air-based implementation. The ground-truth positioning system runs on an ESP32 microcontroller using code written in C++, and the acoustic positioning system runs on two STM32 microcontrollers using code written in C. Extrapolation of these results to the underwater regime, as well as recommendations for improving upon this work, are included at the end of the document. All code written for this thesis is available on GitHub and is open-source and well-documented.