In recent years, the rapid development of autonomous vehicles (AVs) has necessitated the need for high-quality perception systems. Perception is a fundamental requirement for AVs, with cameras and LiDARs being commonly used sensors for environmental understanding and localization. However, there is a research gap in assessing the quality of AVs perception systems. To address this gap, this dissertation proposes a novel paradigm for evaluating AVs perception quality by studying the perception quality of cameras and LiDARs sensors. Our proposed paradigm aims to provide a comprehensive assessment of the quality of perception systems used in AVs.To achieve our research goals, we first validate the concept of surrounding environmental complexity through subjective experiments that rate complexity scores. In this study, we propose a neural network to classify complexity. Subsequently, we study image-based perception quality assessment by using image saliency and 2D object detection algorithms to create an image-based quality index. We then develop a neural network model to regress the proposed quality index score. Furthermore, we extend our research to LiDAR-based point cloud quality assessment by using the image-based saliency map as guidance to generate a point cloud quality index score. We then develop a neural network model to regress the score. Finally, we validate the proposed perception quality index with a novel designed AVs perception algorithm. In conclusion, this dissertation makes a significant contribution to the field of AVs perception by proposing a new paradigm for assessing perception quality. Our research findings can be used to improve the overall performance and safety of AVs, which has significant implications for the transportation industry and society as a whole. / Doctor of Philosophy / This dissertation delves into the fundamentals of autonomous vehicles (AVs), which is perception, with the aim of developing a new paradigm for evaluating the quality of perception algorithms. AVs are the dream of humanity, and perception is the fundamental requirement for achieving their full potential. Our research proposes a new approach to assessing the quality of perception algorithms, which can have significant implications for the performance and safety of AVs. By studying the perception algorithm quality, we aim to identify areas for improvement, leading to better AV performance and enhancing user trust. Our findings highlight the importance of perception in the development of AVs and demonstrate the need for continuous evaluation and improvement of the perception algorithms used in AVs.

No description available.

No description available.

Sensor technology is rapidly developing and, consequently, the generation of point cloud data is constantly increasing. Since the recent release of PointNet, it is possible to process this unordered 3-dimensional data directly in a neural network. The company TLT Screen AB, which develops cutting-edge tracking technology, seeks to optimize the localization of the fingertips of a hand in a point cloud. To do so, the identification of relevant 3D neural network models for modeling hands and detection of fingertips in various hand orientations is essential. The Hand PointNet processes point clouds of hands directly and generate estimations of fixed points (joints), including fingertips, of the hands. Therefore, this model was selected to optimize the localization of fingertips for TLT Screen AB and forms the subject of this research. The model has advantages over conventional convolutional neural networks (CNN). First of all, in contrast to the 2D CNN, the Hand PointNet can use the full 3-dimensional spatial information. Compared to the 3D CNN, moreover, it avoids unnecessarily voluminous data and enables more efficient learning. The model was trained and evaluated on the public dataset MRSA Hand. In contrast to previously published work, the main object of this investigation is the estimation of only 5 joints, for the fingertips. The behavior of the model with a reduction from the usual 21 to 11 and only 5 joints are examined. It is found that the reduction of joints contributed to an increase in the mean error of the estimated joints. Furthermore, the examination of the distribution of the residuals of the estimate for fingertips is found to be less dense. MC dropout to study the prediction uncertainty for the fingertips has shown that the uncertainty increases when the joints are decreased. Finally, the results show that the uncertainty is greatest for the prediction of the thumb tip. Starting from the tip of the thumb, it is observed that the uncertainty of the estimates decreases with each additional fingertip.

The title of this thesis is 'Workflow from point cloud to BIM'. Thus, an attempt is made to present and analyse all the steps followed in such a process. The building used as a case study is U-Building which is located at KTH campus. Briefly, a report is made on the various methods of mapping existing buildings. Also, the principles of operation of 3D laser scanners are presented as well as an analysis of BIM technology. Furthermore, it is analysed the process of creating a 3D representation of the building in the form of a point cloud as well as the process of creating the 3D model with the help of two software packages. / Titeln på denna uppsats är 'Arbetsflöde från punktmoln till BIM'. Således görs ett försök att presentera och analysera alla steg som följs i en sådan process. Byggnaden som används som fallstudie är U-Building som ligger på KTH campus. Kortfattat görs en redovisning av de olika metoderna för att kartlägga befintlig bebyggelse. Dessutom presenteras principerna för driften av 3D-laserskannrar samt en analys av BIM-teknik. Vidare analyseras processen att skapa en 3D-representation av byggnaden i form av ett punktmoln samt processen att skapa 3D-modellen med hjälp av två mjukvarupaket.

Yes / Optimization plays a pivotal role in computer graphics and vision. Learning-based optimization algorithms have emerged as a powerful optimization technique for solving problems with robustness and accuracy because it learns gradients from data without calculating the Jacobian and Hessian matrices. The key aspect of the algorithms is the least-squares method, which formulates a general parametrized model of unconstrained optimizations and makes a residual vector approach to zeros to approximate a solution. The method may suffer from undesirable local optima for many applications, especially for point cloud registration, where each element of transformation vectors has a different impact on registration. In this paper, Reweighted Discriminative Optimization (RDO) method is proposed. By assigning different weights to components of the parameter vector, RDO explores the impact of each component and the asymmetrical contributions of the components on fitting results. The weights of parameter vectors are adjusted according to the characteristics of the mean square error of fitting results over the parameter vector space at per iteration. Theoretical analysis for the convergence of RDO is provided, and the benefits of RDO are demonstrated with tasks of 3D point cloud registrations and multi-views stitching. The experimental results show that RDO outperforms state-of-the-art registration methods in terms of accuracy and robustness to perturbations and achieves further improvement than non-weighting learning-based optimization.

Modern robotics and automation systems require high-level reasoning capability in representing, identifying, and interpreting the three-dimensional data of the real world. Understanding the world's geometric structure by visual data is known as 3D perception. The necessity of analyzing irregular and complex 3D data has led to the development of high-dimensional frameworks for data learning. Here, we design several sparse learning-based approaches for high-dimensional data that effectively tackle multiple perception problems, including data filtering, data recovery, and data retrieval. The frameworks offer generative solutions for analyzing complex and irregular data structures without prior knowledge of data. The first part of the dissertation proposes a novel method that simultaneously filters point cloud noise and outliers as well as completing missing data by utilizing a unified framework consisting of a novel tensor data representation, an adaptive feature encoder, and a generative Bayesian network. In the next section, a novel multi-level generative chaotic Recurrent Neural Network (RNN) has been proposed using a sparse tensor structure for image restoration. In the last part of the dissertation, we discuss the detection followed by localization, where we discuss extracting features from sparse tensors for data retrieval. / Doctor of Philosophy / The development of automation systems and robotics brought the modern world unrivaled affluence and convenience. However, the current automated tasks are mainly simple repetitive motions. Tasks that require more artificial capability with advanced visual cognition are still an unsolved problem for automation. Many of the high-level cognition-based tasks require the accurate visual perception of the environment and dynamic objects from the data received from the optical sensor. The capability to represent, identify and interpret complex visual data for understanding the geometric structure of the world is 3D perception. To better tackle the existing 3D perception challenges, this dissertation proposed a set of generative learning-based frameworks on sparse tensor data for various high-dimensional robotics perception applications: underwater point cloud filtering, image restoration, deformation detection, and localization. Underwater point cloud data is relevant for many applications such as environmental monitoring or geological exploration. The data collected with sonar sensors are however subjected to different types of noise, including holes, noise measurements, and outliers. In the first chapter, we propose a generative model for point cloud data recovery using Variational Bayesian (VB) based sparse tensor factorization methods to tackle these three defects simultaneously. In the second part of the dissertation, we propose an image restoration technique to tackle missing data, which is essential for many perception applications. An efficient generative chaotic RNN framework has been introduced for recovering the sparse tensor from a single corrupted image for various types of missing data. In the last chapter, a multi-level CNN for high-dimension tensor feature extraction for underwater vehicle localization has been proposed.

Digital models are essential to quantifying the behavior of structural systems. In many cases, the creation of these models involves manual measurements taken in the field, followed by a manual creation of this model using these measurements. Both of these steps are time consuming and prohibitively expensive, leading to a lack of utilization of accurate models. We propose a framework built on the processing of 3D laser scanning data to partially automate the creation of these models. We focus on steel structures, as they represent a gap in current research into this field. Previous research has focused on segmentation of the point cloud data in order to extract relevant geometries. These approaches cannot easily be extended to steel structures, so we propose a novel method of processing this data with the goal of creating a full finite element model from the information extracted. Our approach sidesteps the need for segmentation by directly extracting the centerlines of structural elements. We begin by taking "slices" of the point cloud in the three principal directions. Each of these slices is flattened into an image, which allows us to take advantage of powerful image processing techniques. Within these images we use 2d convolution as a template match to isolate structural cross sections. This gives us the centroids of cross sections in the image space, which we can map back to the point cloud space as points along the centerline of the element. By fitting lines in 3d space to these points, we can determine the equations for the centerline of each element. This information could be easily passed into a finite element modeling software where the cross sections are manually defined for each line element. / Modern buildings require a digital counterpart to the physical structure for accurate analysis. Historically, these digital counterparts would be created by hand using the measurements that the building was intended to be built to. Often this is not accurate enough and the as-built system must be measured on site to capture deviations from the original plans. In these cases, a large amount of time must be invested to send personnel out into the field and take large amounts of measurements of the structure. Additionally, these "hand measurements" are prone to user error. We propose a novel method of gathering these field measurements quickly and accurately by using a technique called "laser scanning". These laser scans essentially take a 3D snapshot of the site, which contains all the geometric information of visible elements. While it is difficult to locate items such as steel beams in the 3D data, the cross sections of these structural elements are easily defined in 2D. Our method involves taking 2D slices of this 3D scan which allows us to locate the cross sections of the structural members by searching for template cross-sectional shapes. Once the cross sections have been isolated, their centers can be mapped back from the 2D slice to the 3D space as points along the centerlines of the structural elements. These centerlines represent one of the most time consuming requirements to building digital models of modern buildings, so this method could drastically reduce the total modeling time required by automating this particular step.

Autonomous driving requires lightweight and robust perception systems that can rapidly and accurately interpret the complex driving environment. This dissertation investigates the transformative capacity of discrete wavelet transform (DWT), inverse DWT, CNNs, and transformers as foundational elements to develop lightweight perception architectures for autonomous vehicles. The inherent properties of DWT, including its invertibility, sparsity, time-frequency localization, and ability to capture multi-scale information, present an inductive bias. Similarly, transformers capture long-range dependency between features. By harnessing these attributes, novel wavelet-enhanced deep learning architectures are introduced. The first contribution is introducing a lightweight backbone network that can be employed for real-time processing. This network balances processing speed and accuracy, outperforming established models like ResNet-50 and VGG16 in terms of accuracy while remaining computationally efficient. Moreover, a multiresolution attention mechanism is introduced for CNNs to enhance feature extraction. This mechanism directs the network's focus toward crucial features while suppressing less significant ones. Likewise, a transformer model is proposed by leveraging the properties of DWT with vision transformers. The proposed wavelet-based transformer utilizes the convolution theorem in the frequency domain to mitigate the computational burden on vision transformers caused by multi-head self-attention. Furthermore, a proposed wavelet-multiresolution-analysis-based 3D object detection model exploits DWT's invertibility, ensuring comprehensive environmental information capture. Lastly, a multimodal fusion model is presented to use information from multiple sensors. Sensors have limitations, and there is no one-fits-all sensor for specific applications. Therefore, multimodal fusion is proposed to use the best out of different sensors. Using a transformer to capture long-range feature dependencies, this model effectively fuses the depth cues from LiDAR with the rich texture derived from cameras. The multimodal fusion model is a promising approach that integrates backbone networks and transformers to achieve lightweight and competitive results for 3D object detection. Moreover, the proposed model utilizes various network optimization methods, including pruning, quantization, and quantization-aware training, to minimize the computational load while maintaining optimal performance. The experimental results across various datasets for classification networks, attention mechanisms, 3D object detection, and multimodal fusion indicate a promising direction in developing a lightweight and robust perception system for robotics, particularly in autonomous driving.

Reindeer are important drivers of ecosystem processes in arctic and subarctic ecosystems, changing nutrient conditions and influencing vegetation by grazing and trampling. Mountain birch forests are one of the ecosystems impacted by reindeer grazing, but the effect on the 3D structure of these forests is not well understood. Drones are revolutionising ecological studies, allowing to create high-resolution 3D point clouds at low costs. I investigated the effect of year-round and autumn reindeer grazing on mountain birch forest vegetation in historically separated grazing areas at the Finnish-Norwegian border, using a combination of field data and drone data. The two sampling techniques were further compared to evaluate the possibility to use photogrammetric point clouds to characterise mountain birch forests. I found lower productivity in the year-round grazing regime, coinciding with generally higher vegetation density. Vertically, higher densities were found above browsing height, while the understory showed lower densities compared to autumn grazed areas. These results suggest that mountain birches allocate more biomass to the canopy area, which can be direct or indirect grazing effects. Nevertheless, overall productivity is lowered by grazing indicating changes in vegetation biomass and composition. The point clouds generally matched field data, but the understory vegetation tended to be underrepresented, arising the question if found effects are ecological or technical. As this could not be disentangled, cautious interpretation of my results is required. I conclude that using photogrammetric point clouds is a promising technique for ecological studies, but needs further development to improve accuracy and reliability of results.