<p>In traffic scene perception for autonomous vehicles, driving videos are available from in-car sensors such as camera and LiDAR for road detection and collision avoidance. There are some existing challenges in computer vision tasks for video processing, including object detection and tracking, semantic segmentation, etc. First, due to that consecutive video frames have a large data redundancy, traditional spatial-to-temporal approach inherently demands huge computational resource. Second, in many real-time scenarios, targets move continuously in the view as data streamed in. To achieve prompt response with minimum latency, an online model to process the streaming data in shift-mode is necessary. Third, in addition to shape-based recognition in spatial space, motion detection also replies on the inherent temporal continuity in videos. While current works either lack long-term memory for reference or consume a huge amount of computation. </p> <p><br></p> <p>The purpose of this work is to achieve strongly temporal-associated sensing results in real-time with minimum memory, which is continually embedded to a pragmatic framework for speed and path planning. It takes a temporal-to-spatial approach to cope with fast moving vehicles in autonomous navigation. It utilizes compact road profiles (RP) and motion profiles (MP) to identify path regions and dynamic objects, which drastically reduces video data to a lower dimension and increases sensing rate. Specifically, we sample one-pixel line at each video frame, the temporal congregation of lines from consecutive frames forms a road profile image; while motion profile consists of the average lines by sampling one-belt pixels at each frame. By applying the dense temporal resolution to compensate the sparse spatial resolution, this method reduces 3D streaming data into 2D image layout. Based on RP and MP under various weather conditions, there have three main tasks being conducted to contribute the knowledge domain in perception and planning for autonomous driving. </p> <p><br></p> <p>The first application is semantic segmentation of temporal-to-spatial streaming scenes, including recognition of road and roadside, driving events, objects in static or motion. Since the main vision sensing tasks for autonomous driving are identifying road area to follow and locating traffic to avoid collision, this work tackles this problem by using semantic segmentation upon road and motion profiles. Though one-pixel line may not contain sufficient spatial information of road and objects, the consecutive collection of lines as a temporal-spatial image provides intrinsic spatial layout because of the continuous observation and smooth vehicle motion. Moreover, by capturing the trajectory of pedestrians upon their moving legs in motion profile, we can robustly distinguish pedestrian in motion against smooth background. The experimental results of streaming data collected from various sensors including camera and LiDAR demonstrate that, in the reduced temporal-to-spatial space, an effective recognition of driving scene can be learned through Semantic Segmentation.</p> <p><br></p> <p>The second contribution of this work is that it accommodates standard semantic segmentation to sequential semantic segmentation network (SE3), which is implemented as a new benchmark for image and video segmentation. As most state-of-the-art methods are greedy for accuracy by designing complex structures at expense of memory use, which makes trained models heavily depend on GPUs and thus not applicable to real-time inference. Without accuracy loss, this work enables image segmentation at the minimum memory. Specifically, instead of predicting for image patch, SE3 generates output along with line scanning. By pinpointing the memory associated with the input line at each neural layer in the network, it preserves the same receptive field as patch size but saved the computation in the overlapped regions during network shifting. Generally, SE3 applies to most of the current backbone models in image segmentation, and furthers the inference by fusing temporal information without increasing computation complexity for video semantic segmentation. Thus, it achieves 3D association over long-range while under the computation of 2D setting. This will facilitate inference of semantic segmentation on light-weighted devices.</p> <p><br></p> <p>The third application is speed and path planning based on the sensing results from naturalistic driving videos. To avoid collision in a close range and navigate a vehicle in middle and far ranges, several RP/MPs are scanned continuously from different depths for vehicle path planning. The semantic segmentation of RP/MP is further extended to multi-depths for path and speed planning according to the sensed headway and lane position. We conduct experiments on profiles of different sensing depths and build up a smoothly planning framework according to their them. We also build an initial dataset of road and motion profiles with semantic labels from long HD driving videos. The dataset is published as additional contribution to the future work in computer vision and autonomous driving. </p>

Many autonomous robots operating in unknown and unstructured environments rely on building a dense 3D map of it during exploration. What tasks the robot can perform depends on the information stored in this map. Most 3D maps currently in use store information required for robot control and environment reconstruction – is this point in space occupied, or safe to navigate to? To enable more complex tasks additional information is required. We introduce Semantic UFOMap, an open-source octree based mapping framework designed for online use on limited hardware. Capable of real-time fusion and querying of semantic instances into the map – enabling high-level robot tasks and human-robot interaction. The online capabilities are evaluated using ground-truth data, where we show competitive results compared to voxel hashing, with optimizations still available. Additionally, we demonstrate a potential application with a simulated autonomous exploration and object navigation experiment. The evaluation shows that Semantic UFOMap is capable of real-time online performance. Storing semantic information in the map has the potential to open up new autonomous robot applications and yield improvements in existing tasks. / Autonoma robotar som opererar i okända och ostrukturerade mijöer är ofta beroende av att skapa en 3D-karta under utforskning av området. Villka uppgifter roboten kan utföra beror på informationen som finns tillgänglig i kartan. De flesta nuvarande kartor som används sparar information som behövs för säker navigation och miljörekonstruktion – är den här positionen ett hinder, eller är den säker att navigera till? För att möjligjöra mer komplexa uppgifter behöver roboten ha tillgång till ytterligare information. Vi presenterar Semantic UFOMap, ett öppen källkods kartläggnings ramverk för realtids användning på begränsad hårdvara. Genom att klara av realtids integrering och sökning av semantiska instanser i kartan möjliggör ramverket mer komplexa uppgifter och öppnar upp fler användningsområden i människa-robot interaktion. Utvärdering görs med hjälp av inspelad data, vi visar konkurrenskraftiga resultat jämfört med voxel hashning, med optimering fortfarande tillgänglig. Ett användningsområde demonstreras med ett simulerat autonomt utforsknings och objektnavigerings experiment. Utvärderingen visar att Semantic UFOMap klarar av realtids applikationer. Att spara semantisk information i kartan har potential att öppna upp för nya användningsområden inom robotik och leda till förbättringar i befintliga uppgifter.

No description available.

<p dir="ltr">Path-tracking control is an integral part of motion planning in autonomous vehicles, in which the vehicle's lateral and longitudinal positions are controlled by a control system that will provide acceleration and steering angle commands to ensure accurate tracking of longitudinal and lateral movements in reference to a pre-defined trajectory. Extensive research has been conducted to address the growing need for efficient algorithms in this area. In this dissertation, a scenario and machine learning-based data-driven control approach is proposed for a path-tracking controller. Firstly, a Deep Reinforcement Learning model is developed to facilitate the control of longitudinal speed. A Deep Deterministic Policy Gradient algorithm is employed as the primary algorithm in training the reinforcement learning model. The main objective of this model is to maintain a safe distance from a lead vehicle (if present) or track a velocity set by the driver. Secondly, a lateral steering controller is developed using Neural Networks to control the steering angle of the vehicle with the main goal of following a reference trajectory. Then, a path-planning algorithm is developed using a hybrid A* planner. Finally, the longitudinal and lateral control models are coupled together to obtain a complete path-tracking controller that follows a path generated by the hybrid A* algorithm at a wide range of vehicle speeds. The state-of-the-art path-tracking controller is also built using Model Predictive Control and Stanley control to evaluate the performance of the proposed model. The results showed the effectiveness of both proposed models in the same scenario, in terms of velocity error, lateral yaw angle error, and lateral distance error. The results from the simulation show that the developed hybrid A* algorithm has good performance in comparison to the state-of-the-art path planning algorithms.</p>

The research on autonomous vehicles, and more specifically cooperation between autonomous vehicles, has become a prominent research field during the last cou- ple of decades. One example is the combination of an unmanned aerial vehicle (UAV) together with an unmanned ground vehicle (UGV). The benefits of this are that the two vehicles complement each other, where the UAV provides an aerial view and can reach areas where a ground vehicle can not. Furthermore, since the UAV has a limited range, the UGV can then serve as transport and recharge sta- tion for the UAV. This master thesis studies how model predictive control (MPC) can be used to land a UAV on a moving UGV.  A linear MPC is chosen, since previous work using this has shown promising results. The UAV is chosen to be controlled using commands in pitch, roll and climbing rate. The MPC is designed as a decoupled controller, with a separate horizontal and vertical controller. This allows for a spatial constraint to be im- plemented, which constrains the UAV from entering ground level before arriving above the UGV. It also constrains the UAV from potentially hitting protruding ob- jects on the UGV. The horizontal controller uses a simple planner, which guides the UAV to land on the UGV from behind.  The MPC is evaluated using a additive white Gaussian noise (AWGN) sen- sor error model with zero mean. The scenario used is that the UAV starts 50 m from the UGV, and the UGV starts driving in a given direction turning randomly. The MPC lands successfully in 100 % of the simulations for a wide range of tun- ings. The MPC maintains the same landing statistics with a delay in the sensor information of up to 500 ms. The AWGN could be increased while maintaining successful landings, however with significantly more retakes and longer landing times. Lower AWGN variance only slightly improves performance, suggesting that the MPC is quite robust towards high variance in the state estimation.  The MPC is also compared to a PID controller. The MPC has significantly shorter landing times. The PID has a more oscillatory control signal, however, the PID has a lower variance in landing positions, but a slightly less centered mean on the UGV. The overall results show that an MPC can be used to achieve a flexible controller that can be tuned and reformulated to fit the situation, and performs as good or better compared to a PID controller.  The hardware tests show promising results for the implementation of the MPC. The controller is not tuned and no system identification is done specifi- cally for the physical UAV, suggesting that the controller is robust for varying settings. Even though the UAV never lands on the UGV, the visual behavior and control signal plots suggest that it would be able to land. However, these tests are performed using global navigation satellite system state estimation on a sta- tionary UGV, therefore further tests need to be performed in more challenging scenarios.

<p>This thesis presents a computational model of the surgical stitching tasks and a path planning algorithm for robotic assisted stitching. The overall goal of the research is to enable surgical robots to perform automatic suturing. Suturing comprises several distinct steps, one of them is the stitching. During stitching, reaching the desired exit point is difficult because it must be accomplished without direct visual feedback. Moreover, the stitching is a time consuming procedure repeated multiple times during suturing. Therefore, it would be desirable to enhance the surgical robots with the ability of performing automatic suturing. The focus of this work is on the automation of the stitching task. The thesis presents a model based path planning algorithm for the autonomous stitching. The method uses a nonlinear model for the curved needle - soft tissue interaction. The tissue is modeled as a deformable object using continuum mechanics tools. This thesis uses a mesh free deformable tissue model namely, Reproducing Kernel Particle Method (RKPM). RKPM was chosen as it has been proven to accurately handle large deformation and requires no re-meshing algorithms. This method has the potential to be more realistic in modeling various material characteristics by using appropriate strain energy functions. The stitching task is simulated using a constrained deformable model; the deformable tissue is constrained by the interaction with the curved needle. The stitching model was used for needle trajectory path planning during stitching. This new path planning algorithm for the robotic stitching was developed, implemented, and evaluated. Several simulations and experiments were conducted. The first group of simulations comprised random insertions from different insertion points without planning to assess the modeling method and the trajectory of the needle inside the tissue. Then the parameters of the simulations were set according to the measured experimental parameters. The proposed path planning method was tested using a surgical ETHICON needle of type SH 1=2 Circle with the radius of 8:88mm attached to a robotic manipulator. The needle was held by a grasper which is attached to the robotic arm. The experimental results illustrate that the path planned curved needle insertions are fifty percent more accurate than the unplanned ones. The results also show that this open loop approach is sensitive to model parameters.</p> / Master of Applied Science (MASc)

<p dir="ltr">The use of autonomous vehicles, such as Unmanned Aerial Systems (UASs), Unmanned Ground Vehicles (UGVs), and Unmanned Surface Vessels (USVs), has globally increased in various applications. Their rising popularity and high accessibility have also increased the use of UASs in criminal or hazardous actions.</p><p dir="ltr">It is beneficial to rapidly compute possible surveillance system evasive paths to evaluate the effectiveness of a given sensor deployment scheme. To find these evasive trajectories, we assume full knowledge of the current and future state of the surveillance system. This assumption allows the defender to identify worst-case trajectories to counteract. The surveillance system path planning presented in this work can be leveraged for game theoretic sensor deployment.</p><p dir="ltr">A sensor deployment scheme determines the overall surveillance efficiency. Through redeployment after each assessment, it aims to approach an equilibrium that maximizes defense capabilities. Therefore, a method of evaluation that models mobile, directional sensors is demanded.</p><p dir="ltr">In response to this demand, this thesis explores the design of a computationally efficient path-planning algorithm for the space-time domain. The Space-Time Parallel RRT* (STP-RRT*) algorithm obtains multiple goal candidates, drawn from a uniform distribution over the time horizon. A set of parallel RRT* trees is simultaneously populated by each goal candidate. By leveraging a connect heuristic from RRT-Connect, parallel goal trees converge to an RRT* tree populated from a start point. This simultaneous tree growth structure returns a computation complexity of O(N log(N)), where N is the number of random samples.</p><p dir="ltr">Due to its low complexity, the STP-RRT* algorithm is suitable to be used as an evaluation metric that computes the cost of the infiltration path of a malicious autonomous system to assess the performance of the deployment layout. The feedback assessment can be used for the surveillance system redeployment to strengthen the vulnerability.</p><p dir="ltr">To identify potential and existing bottlenecks in the algorithm, a computation complexity analysis is conducted, and complexity reduction techniques are employed. Given that surveillance system characteristics are known, 1-dimensional and 2-dimensional environments are generated where positions and surveillance patterns of stationary and dynamic obstacles are randomly selected. In each randomized environment, the STP-RRT*, RRT*, and ST-RRT* are evaluated by comparing success rate, computation time, tree size, and normalized cost through 100-trial Monte Carlo simulations. Under the provided conditions, the proposed STP-RRT* algorithm outperforms two other algorithms with an improved mean success rate and reduced mean computation time by 10.02% and 12.88%, respectively, while maintaining a similar cost level, showing its potential application in surveillance-evasive path-planning problems for surveillance deployment evaluation. Finally, we integrate our algorithm with Nav2, an open-source navigation stack for various robotics applications, including UAV, UGV, and USV. We demonstrate its effectiveness via software-in-the-loop (SiTL) experiments utilizing open-source autopilot software.</p>

<p dir="ltr">The emergence of Autonomous Mobile Robots (AMR) signifies a pivotal shift in vehicle-based material handling systems, demonstrating their effectiveness across a broad spectrum of applications. Advancing beyond the traditional Automated Guided Vehicles (AGV), AMRs offer unprecedented flexibility in movement, liberated from electromagnetic guidance constraints. Their decentralized control architecture not only enables remarkable scalability but also fortifies system resilience through advanced conflict resolution mechanisms. Nevertheless, transitioning from AGV to AMR presents intricate challenges, chiefly due to the expanded complexity in path planning and task selection, compounded by the heightened potential for conflicts from their dynamic interaction capabilities. This dissertation confronts these challenges by fully leveraging the technological advancements of AMRs. A kinematic-enabled agent-based simulator was developed to replicate AMR system behavior, enabling detailed analysis of fleet dynamics and interactions within AMR intralogistics systems and their environments. Additionally, a comprehensive fleet management protocol was formulated to enhance the throughput of AMR-based intralogistics systems from an integrated perspective. A pivotal discovery of this research is the inadequacy of existing path planning protocols to provide reliable plans throughout their execution, leading to task allocation decisions based on inaccurate plan information and resulting in false optimality. In response, a novel machine learning enhanced probabilistic Multi-Robot Path Planning (MRPP) protocol was introduced to ensure the generation of dependable path plans, laying a solid foundation for task allocation decisions. The contributions of this dissertation, including the kinematic-enabled simulator, the fleet management protocol, and the MRPP protocol, are intended to pave the way for practical enhancements in autonomous vehicle-based material handling systems, fostering the development of solutions that are both innovative and applicable in industrial practices.<br></p>

Formation planning and control for multi-agent robotic systems enables tasks to be completed more efficiently and robustly compared to using a single agent. Applications are found in fields such as agriculture, mining, autonomousvehicle platooning, surveillance, space exploration, etc. In this paper, a complete framework for formation path planning for holonomic ground robots in an obstacle-rich environment is proposed. The method utilizes the Fast Marching Square (FM2) path planning algorithm, and a formation keeping approach which falls within the Leader-Follower category. Contrary to most related works, the role of leader is dynamically assigned to avoid unnecessary rotation of the formation. Furthermore, the roles of the followers are also dynamically assigned to fit the current geometry of the formation. A flexible spring-damper system prevents inter-robot collisions and helps maintain the formation shape. An obstacle avoidance step at the end of the pipeline keeps the spring forces from driving robots into obstacles. The framework is tested on a formation consisting of three Unitree Go1 quadruped robots, both in the Gazebo simulation environment and in lab experiments. The results are successful and indicate that the method is feasible, although further work is needed to adjust the role assignment for larger formations, combine the framework with Simultaneous Localization and Mapping (SLAM) and provide a more robust handling of dynamic obstacles.

This thesis presents the initial components of an integrated guidance, navigation, and control system for vertical take-off and landing (VTOL) autonomous unmanned aerial vehicles (UAVs) such that they may map complex environments that may be hostile. The first part of this thesis presents an autonomous guidance system. For goal selection, the map is partitioned around the presence of obstacles and whether that area has been explored. To perform this partitioning, the Octree algorithm is implemented. In this thesis, we test this algorithm to find a parameter set that optimizes this algorithm. Having selected goal points, we perform a comparison of the LPA* and A* path planning algorithms with a custom heuristic that enables reckless or tactical maneuvers as the UAV maps the environment. For trajectory planning, the fMPC algorithm is applied to the feedback-linearized equations of motion of a quadcopter. For collision avoidance, standalone versions of 4 different constraint generation algorithms are evaluated to compare their resulting computation times, accuracy, and computed volume on a voxel map that simulates a 2-story house along with fixed paths that vary in length at fixed intervals as basis of tests. The second part of this thesis presents the theory of Model Reference Adaptive Control(MRAC) along with augmentation for output signal tracking and switched-dynamic systems. We then detail the development of longitudinal and lateral controllers a Quad-Rotor Tailsitter(QRBP) style UAV. In order to successfully implement the proposed controller on the QRBP, significant effort was placed upon physical design and testing apparatus. / Master of Science / For an autonomously operated, Unmanned Aerial Vehicle (UAV), to operate, it requires a guidance system to determine where and how to go, and a control system to effectively actuate the guidance system's commands. In this thesis, we detail the characterization and optimization of the algorithms comprising the guidance system. We then delve into the theory of MRAC and apply it toward a control system for a QRBP. We then detail additional tools developed to support the testing of the QRBP.