In recent years, the rapid development of autonomous vehicles (AVs) has necessitated the need for high-quality perception systems. Perception is a fundamental requirement for AVs, with cameras and LiDARs being commonly used sensors for environmental understanding and localization. However, there is a research gap in assessing the quality of AVs perception systems. To address this gap, this dissertation proposes a novel paradigm for evaluating AVs perception quality by studying the perception quality of cameras and LiDARs sensors. Our proposed paradigm aims to provide a comprehensive assessment of the quality of perception systems used in AVs.To achieve our research goals, we first validate the concept of surrounding environmental complexity through subjective experiments that rate complexity scores. In this study, we propose a neural network to classify complexity. Subsequently, we study image-based perception quality assessment by using image saliency and 2D object detection algorithms to create an image-based quality index. We then develop a neural network model to regress the proposed quality index score. Furthermore, we extend our research to LiDAR-based point cloud quality assessment by using the image-based saliency map as guidance to generate a point cloud quality index score. We then develop a neural network model to regress the score. Finally, we validate the proposed perception quality index with a novel designed AVs perception algorithm. In conclusion, this dissertation makes a significant contribution to the field of AVs perception by proposing a new paradigm for assessing perception quality. Our research findings can be used to improve the overall performance and safety of AVs, which has significant implications for the transportation industry and society as a whole. / Doctor of Philosophy / This dissertation delves into the fundamentals of autonomous vehicles (AVs), which is perception, with the aim of developing a new paradigm for evaluating the quality of perception algorithms. AVs are the dream of humanity, and perception is the fundamental requirement for achieving their full potential. Our research proposes a new approach to assessing the quality of perception algorithms, which can have significant implications for the performance and safety of AVs. By studying the perception algorithm quality, we aim to identify areas for improvement, leading to better AV performance and enhancing user trust. Our findings highlight the importance of perception in the development of AVs and demonstrate the need for continuous evaluation and improvement of the perception algorithms used in AVs.

No description available.

No description available.

Sensor technology is rapidly developing and, consequently, the generation of point cloud data is constantly increasing. Since the recent release of PointNet, it is possible to process this unordered 3-dimensional data directly in a neural network. The company TLT Screen AB, which develops cutting-edge tracking technology, seeks to optimize the localization of the fingertips of a hand in a point cloud. To do so, the identification of relevant 3D neural network models for modeling hands and detection of fingertips in various hand orientations is essential. The Hand PointNet processes point clouds of hands directly and generate estimations of fixed points (joints), including fingertips, of the hands. Therefore, this model was selected to optimize the localization of fingertips for TLT Screen AB and forms the subject of this research. The model has advantages over conventional convolutional neural networks (CNN). First of all, in contrast to the 2D CNN, the Hand PointNet can use the full 3-dimensional spatial information. Compared to the 3D CNN, moreover, it avoids unnecessarily voluminous data and enables more efficient learning. The model was trained and evaluated on the public dataset MRSA Hand. In contrast to previously published work, the main object of this investigation is the estimation of only 5 joints, for the fingertips. The behavior of the model with a reduction from the usual 21 to 11 and only 5 joints are examined. It is found that the reduction of joints contributed to an increase in the mean error of the estimated joints. Furthermore, the examination of the distribution of the residuals of the estimate for fingertips is found to be less dense. MC dropout to study the prediction uncertainty for the fingertips has shown that the uncertainty increases when the joints are decreased. Finally, the results show that the uncertainty is greatest for the prediction of the thumb tip. Starting from the tip of the thumb, it is observed that the uncertainty of the estimates decreases with each additional fingertip.

The title of this thesis is 'Workflow from point cloud to BIM'. Thus, an attempt is made to present and analyse all the steps followed in such a process. The building used as a case study is U-Building which is located at KTH campus. Briefly, a report is made on the various methods of mapping existing buildings. Also, the principles of operation of 3D laser scanners are presented as well as an analysis of BIM technology. Furthermore, it is analysed the process of creating a 3D representation of the building in the form of a point cloud as well as the process of creating the 3D model with the help of two software packages. / Titeln på denna uppsats är 'Arbetsflöde från punktmoln till BIM'. Således görs ett försök att presentera och analysera alla steg som följs i en sådan process. Byggnaden som används som fallstudie är U-Building som ligger på KTH campus. Kortfattat görs en redovisning av de olika metoderna för att kartlägga befintlig bebyggelse. Dessutom presenteras principerna för driften av 3D-laserskannrar samt en analys av BIM-teknik. Vidare analyseras processen att skapa en 3D-representation av byggnaden i form av ett punktmoln samt processen att skapa 3D-modellen med hjälp av två mjukvarupaket.

Yes / Optimization plays a pivotal role in computer graphics and vision. Learning-based optimization algorithms have emerged as a powerful optimization technique for solving problems with robustness and accuracy because it learns gradients from data without calculating the Jacobian and Hessian matrices. The key aspect of the algorithms is the least-squares method, which formulates a general parametrized model of unconstrained optimizations and makes a residual vector approach to zeros to approximate a solution. The method may suffer from undesirable local optima for many applications, especially for point cloud registration, where each element of transformation vectors has a different impact on registration. In this paper, Reweighted Discriminative Optimization (RDO) method is proposed. By assigning different weights to components of the parameter vector, RDO explores the impact of each component and the asymmetrical contributions of the components on fitting results. The weights of parameter vectors are adjusted according to the characteristics of the mean square error of fitting results over the parameter vector space at per iteration. Theoretical analysis for the convergence of RDO is provided, and the benefits of RDO are demonstrated with tasks of 3D point cloud registrations and multi-views stitching. The experimental results show that RDO outperforms state-of-the-art registration methods in terms of accuracy and robustness to perturbations and achieves further improvement than non-weighting learning-based optimization.

Modern robotics and automation systems require high-level reasoning capability in representing, identifying, and interpreting the three-dimensional data of the real world. Understanding the world's geometric structure by visual data is known as 3D perception. The necessity of analyzing irregular and complex 3D data has led to the development of high-dimensional frameworks for data learning. Here, we design several sparse learning-based approaches for high-dimensional data that effectively tackle multiple perception problems, including data filtering, data recovery, and data retrieval. The frameworks offer generative solutions for analyzing complex and irregular data structures without prior knowledge of data. The first part of the dissertation proposes a novel method that simultaneously filters point cloud noise and outliers as well as completing missing data by utilizing a unified framework consisting of a novel tensor data representation, an adaptive feature encoder, and a generative Bayesian network. In the next section, a novel multi-level generative chaotic Recurrent Neural Network (RNN) has been proposed using a sparse tensor structure for image restoration. In the last part of the dissertation, we discuss the detection followed by localization, where we discuss extracting features from sparse tensors for data retrieval. / Doctor of Philosophy / The development of automation systems and robotics brought the modern world unrivaled affluence and convenience. However, the current automated tasks are mainly simple repetitive motions. Tasks that require more artificial capability with advanced visual cognition are still an unsolved problem for automation. Many of the high-level cognition-based tasks require the accurate visual perception of the environment and dynamic objects from the data received from the optical sensor. The capability to represent, identify and interpret complex visual data for understanding the geometric structure of the world is 3D perception. To better tackle the existing 3D perception challenges, this dissertation proposed a set of generative learning-based frameworks on sparse tensor data for various high-dimensional robotics perception applications: underwater point cloud filtering, image restoration, deformation detection, and localization. Underwater point cloud data is relevant for many applications such as environmental monitoring or geological exploration. The data collected with sonar sensors are however subjected to different types of noise, including holes, noise measurements, and outliers. In the first chapter, we propose a generative model for point cloud data recovery using Variational Bayesian (VB) based sparse tensor factorization methods to tackle these three defects simultaneously. In the second part of the dissertation, we propose an image restoration technique to tackle missing data, which is essential for many perception applications. An efficient generative chaotic RNN framework has been introduced for recovering the sparse tensor from a single corrupted image for various types of missing data. In the last chapter, a multi-level CNN for high-dimension tensor feature extraction for underwater vehicle localization has been proposed.

Digital models are essential to quantifying the behavior of structural systems. In many cases, the creation of these models involves manual measurements taken in the field, followed by a manual creation of this model using these measurements. Both of these steps are time consuming and prohibitively expensive, leading to a lack of utilization of accurate models. We propose a framework built on the processing of 3D laser scanning data to partially automate the creation of these models. We focus on steel structures, as they represent a gap in current research into this field. Previous research has focused on segmentation of the point cloud data in order to extract relevant geometries. These approaches cannot easily be extended to steel structures, so we propose a novel method of processing this data with the goal of creating a full finite element model from the information extracted. Our approach sidesteps the need for segmentation by directly extracting the centerlines of structural elements. We begin by taking "slices" of the point cloud in the three principal directions. Each of these slices is flattened into an image, which allows us to take advantage of powerful image processing techniques. Within these images we use 2d convolution as a template match to isolate structural cross sections. This gives us the centroids of cross sections in the image space, which we can map back to the point cloud space as points along the centerline of the element. By fitting lines in 3d space to these points, we can determine the equations for the centerline of each element. This information could be easily passed into a finite element modeling software where the cross sections are manually defined for each line element. / Modern buildings require a digital counterpart to the physical structure for accurate analysis. Historically, these digital counterparts would be created by hand using the measurements that the building was intended to be built to. Often this is not accurate enough and the as-built system must be measured on site to capture deviations from the original plans. In these cases, a large amount of time must be invested to send personnel out into the field and take large amounts of measurements of the structure. Additionally, these "hand measurements" are prone to user error. We propose a novel method of gathering these field measurements quickly and accurately by using a technique called "laser scanning". These laser scans essentially take a 3D snapshot of the site, which contains all the geometric information of visible elements. While it is difficult to locate items such as steel beams in the 3D data, the cross sections of these structural elements are easily defined in 2D. Our method involves taking 2D slices of this 3D scan which allows us to locate the cross sections of the structural members by searching for template cross-sectional shapes. Once the cross sections have been isolated, their centers can be mapped back from the 2D slice to the 3D space as points along the centerlines of the structural elements. These centerlines represent one of the most time consuming requirements to building digital models of modern buildings, so this method could drastically reduce the total modeling time required by automating this particular step.

Reindeer are important drivers of ecosystem processes in arctic and subarctic ecosystems, changing nutrient conditions and influencing vegetation by grazing and trampling. Mountain birch forests are one of the ecosystems impacted by reindeer grazing, but the effect on the 3D structure of these forests is not well understood. Drones are revolutionising ecological studies, allowing to create high-resolution 3D point clouds at low costs. I investigated the effect of year-round and autumn reindeer grazing on mountain birch forest vegetation in historically separated grazing areas at the Finnish-Norwegian border, using a combination of field data and drone data. The two sampling techniques were further compared to evaluate the possibility to use photogrammetric point clouds to characterise mountain birch forests. I found lower productivity in the year-round grazing regime, coinciding with generally higher vegetation density. Vertically, higher densities were found above browsing height, while the understory showed lower densities compared to autumn grazed areas. These results suggest that mountain birches allocate more biomass to the canopy area, which can be direct or indirect grazing effects. Nevertheless, overall productivity is lowered by grazing indicating changes in vegetation biomass and composition. The point clouds generally matched field data, but the understory vegetation tended to be underrepresented, arising the question if found effects are ecological or technical. As this could not be disentangled, cautious interpretation of my results is required. I conclude that using photogrammetric point clouds is a promising technique for ecological studies, but needs further development to improve accuracy and reliability of results.

<p dir="ltr">Consistent intake and feeding behavior records collected on a per-cow basis are useful measures for optimization of feed efficiency, production, and ultimately, resource and economic sustainability of dairy operations. However, current methods for collection are often labor-intensive and impractical to maintain for both individual- and group-housed cows. Across the dairy industry, total mixed rations (TMR) are fed to promote balanced nutrient intake and satisfy evolving energy requirements. TMR intake is an extensively investigated phenotype of dairy cattle and is known to be highly variable due to both intrinsic and extrinsic determinants, which can include composition and palatability of offered TMR, intensity of environmental stressors, and biological aspects of the individual animal. Reductions in TMR intake negatively impact health and production; thus, industry demand has heightened for precise intake monitoring systems. Cyber-physical systems that employ cameras as a sensing device are proposed solutions to ambiguity in existing feeding strategies. Prior studies have demonstrated the efficacy of camera systems to monitor other phenotypes of dairy cattle including body condition, locomotion and gait, social interaction, and early detection of negative health events. In this study, an OAK-D PoE stereo vision camera system was employed to estimate volume of TMR and monitor feeding behavior in a dynamic barn environment. The system leveraged open-source Python software to measure relative depth in near real time and autonomously estimate the amount of TMR present in a feed bunk. Image data were processed to generate a point cloud for which volume of TMR was estimated at a rate of approximately 50 estimates/min. Two experiments were conducted in which mass, volume, and density of TMR, as well as feeding behavior (exclusive to Exp. 2) were manually recorded to be compared to volume estimates of TMR output by the camera system. In Exp. 1, diet type (high-density vs. low-density; HD and LD, respectively), lighting (10,000 Lm vs. existing barn lighting; on vs. off, respectively), and shape of offered TMR (undisturbed vs. simulated post-meal bout; no divot vs. divot, respectively) were assessed for impact on system accuracy across five intervals of known TMR volume. In Exp. 2, system volume estimates were evaluated over time when a cow was present and exhibiting normal feeding behavior. The system accurately estimated volume of TMR across evaluated conditions in Exp. 1, despite significance of the divot condition. As TMR disappeared over time in Exp. 2, system volume estimates decreased with a similar pattern. When the cow was removed and measured TMR volume was unchanged at 2 h collection timepoints in Exp. 2, system volume estimates also remained unchanged and consistent. Post-collection of replicates in Exp. 2, frequency and duration of meal bout events were estimated based on differences in volume when cows were eating. Estimated frequency and durations were similar to manually recorded data and indicated feasibility of behavioral monitoring as an opportunity for further system development. Prior studies have integrated machine learning approaches for refinement of camera monitoring systems and mitigation of reported environmental impact on accurate quantification of TMR volume. Further development of the current system through integration of machine learning applications will improve accuracy and industry applicability as an automated feed bunk management tool for collection of TMR intake and behavioral data on a per-cow basis.</p>