As-Is Building Information Models aid in the management, maintenance and renovation of existing buildings. However, most existing buildings do not have an accurate geometric depiction of their As-Is conditions. The process of generating As-Is models of existing structures involves practitioners, who manually convert Point Cloud Data (PCD) into semantically meaningful 3D models. This process requires a significant amount of manual effort and time. Previous research has been able to model objects by segmenting the point clouds into planes and classifying each one separately into classes, such as walls, floors and ceilings; this is insufficient for modelling, as BIM objects are comprised of multiple planes that form volumetric objects. This thesis introduces a novel method that focuses on the geometric creation of As-Is BIMs with enriched information. It tackles the problem by detecting objects, modelling them and enriching the model with spaces and object adjacencies from PCD. The first step of the proposed method detects objects by exploiting the relationships the segments should satisfy to be grouped into one object. It further proposes a method for detecting slabs with variations in height by finding local maxima in the point density. The second step models the geometry of walls and finally enriches the model with closed spaces encoded in the Industry Foundation Classes (IFC) standard. The method uses the point cloud density of detected walls to determine their width by projecting the wall into two directions and finding the edges with the highest density. It identifies adjacent walls by finding gaps or intersections between walls and exploits walls adjacency for correcting their boundaries, creating an accurate 3D geometry of the model. Finally, the method detects closed spaces by using a shortest-path algorithm. The method was tested on three original PCD which represent office floors. The method detects objects of class walls, floors and ceilings in PCD with an accuracy of approximately 96%. The precision and recall for the room detection were found to be 100%.

The design of a texture display for three-dimensional Light Detection and Ranging (LiDAR) point clouds is investigated. The objective is to present a low fidelity display that is simple to compute in real-time, which utilizes the pattern processing capabilities of a human operator to afford an understanding of the environment. The efficacy of the display is experimentally evaluated by in comparison with a baseline point cloud rendering. Subjects were shown data based on virtual hills, and were asked to plan the least-steep traversal, and identify the hill from a set of distractors. The major conclusions are: comprehension of LiDAR point clouds from the sensor origin is difficult without further processing of the data, a separated vantage point improves understanding of the data, and a simple computation to present local point cloud derivative data significantly improves the understanding of the environment, even when observed from the sensor origin.

The design of a texture display for three-dimensional Light Detection and Ranging (LiDAR) point clouds is investigated. The objective is to present a low fidelity display that is simple to compute in real-time, which utilizes the pattern processing capabilities of a human operator to afford an understanding of the environment. The efficacy of the display is experimentally evaluated by in comparison with a baseline point cloud rendering. Subjects were shown data based on virtual hills, and were asked to plan the least-steep traversal, and identify the hill from a set of distractors. The major conclusions are: comprehension of LiDAR point clouds from the sensor origin is difficult without further processing of the data, a separated vantage point improves understanding of the data, and a simple computation to present local point cloud derivative data significantly improves the understanding of the environment, even when observed from the sensor origin.

The PointNet architecture is a foundational deep learning model for 3D point clouds, solving classification and segmentation tasks. We hypothesize that the full potential of PointNet has not been reached and is greatly restrained by a single Max pooling layer. First, this thesis introduces new and more complex learnable aggregation functions. Secondly, a novel normalization technique, Context Normalization, is proposed for further feature extraction. Context Normalization is similar to Batch Normalization but independently normalizes each point cloud within a mini-batch and always uses dynamic statistics. The experiments show that replacing Max pooling with Principal Neighborhood Aggregation (PNA) increased classification accuracy from 73.3% to 78.7% on an SO(3) augmented version of the ModelNet40 dataset. Combining PNA with Context Normalization further increased accuracy to 84.6%.

<p>Buildings, maps and objects et cetera, can be modeled using a computer or reconstructed in 3D by data from different kinds of cameras or laser scanners. This thesis concerns the latter. The recent improvements of Time-of-Flight cameras have brought a number of new interesting research areas to the surface. Registration of several ToF camera point clouds is such an area.</p><p>A literature study has been made to summarize the research done in the area over the last two decades. The most popular method for registering point clouds, namely the Iterative Closest Point (ICP), has been studied. In addition to this, an error relaxation algorithm was implemented to minimize the accumulated error of the sequential pairwise ICP.</p><p>A few different real-world test scenarios and one scenario with synthetic data were constructed. These data sets were registered with varying outcome. The obtained camera poses from the sequential ICP were improved by loop closing and error relaxation.</p><p>The results illustrate the importance of having good initial guesses on the relative transformations to obtain a correct model. Furthermore the strengths and weaknesses of the sequential ICP and the utilized error relaxation method are shown.</p>

Buildings, maps and objects et cetera, can be modeled using a computer or reconstructed in 3D by data from different kinds of cameras or laser scanners. This thesis concerns the latter. The recent improvements of Time-of-Flight cameras have brought a number of new interesting research areas to the surface. Registration of several ToF camera point clouds is such an area. A literature study has been made to summarize the research done in the area over the last two decades. The most popular method for registering point clouds, namely the Iterative Closest Point (ICP), has been studied. In addition to this, an error relaxation algorithm was implemented to minimize the accumulated error of the sequential pairwise ICP. A few different real-world test scenarios and one scenario with synthetic data were constructed. These data sets were registered with varying outcome. The obtained camera poses from the sequential ICP were improved by loop closing and error relaxation. The results illustrate the importance of having good initial guesses on the relative transformations to obtain a correct model. Furthermore the strengths and weaknesses of the sequential ICP and the utilized error relaxation method are shown.

Au cours des vingt dernières années, de nombreux algorithmes de reconstruction de surface ont été développés. Néanmoins, des données additionnelles telles que les normales orientées sont souvent requises et la robustesse aux données imparfaites est encore un vrai défi. Dans cette thèse, nous traitons de nuages de points non-orientés et imparfaits, et proposons deux nouvelles méthodes gérant deux différents types de surfaces. La première méthode, adaptée au bruit, s'applique aux surfaces lisses et fermées. Elle prend en entrée un nuage de points avec du bruit variable et des données aberrantes, et comporte trois grandes étapes. Premièrement, en supposant que la surface est lisse et de dimension connue, nous calculons une fonction distance adaptée au bruit. Puis nous estimons le signe et l'incertitude de la fonction sur un ensemble de points-sources, en minimisant une énergie quadratique exprimée sur les arêtes d'un graphe uniforme aléatoire. Enfin, nous calculons une fonction implicite signée par une approche dite « random walker » avec des contraintes molles choisies aux points-sources de faible incertitude. La seconde méthode génère des surfaces planaires par morceaux, potentiellement non-variétés, représentées par des maillages triangulaires simples. En faisant croitre des primitives planaires convexes sous une erreur de Hausdorff bornée, nous déduisons à la fois la surface et sa connectivité et générons un complexe simplicial qui représente efficacement les grandes régions planaires, les petits éléments et les bords. La convexité des primitives est essentielle pour la robustesse et l'efficacité de notre approche. / Over the last two decades, a high number of reliable algorithms for surface reconstruction from point clouds has been developed. However, they often require additional attributes such as normals or visibility, and robustness to defect-laden data is often achieved through strong assumptions and remains a scientific challenge. In this thesis we focus on defect-laden, unoriented point clouds and contribute two new reconstruction methods designed for two specific classes of output surfaces. The first method is noise-adaptive and specialized to smooth, closed shapes. It takes as input a point cloud with variable noise and outliers, and comprises three main steps. First, we compute a novel noise-adaptive distance function to the inferred shape, which relies on the assumption that this shape is a smooth submanifold of known dimension. Second, we estimate the sign and confidence of the function at a set of seed points, through minimizing a quadratic energy expressed on the edges of a uniform random graph. Third, we compute a signed implicit function through a random walker approach with soft constraints chosen as the most confident seed points. The second method generates piecewise-planar surfaces, possibly non-manifold, represented by low complexity triangle surface meshes. Through multiscale region growing of Hausdorff-error-bounded convex planar primitives, we infer both shape and connectivity of the input and generate a simplicial complex that efficiently captures large flat regions as well as small features and boundaries. Imposing convexity of primitives is shown to be crucial to both the robustness and efficacy of our approach.

Large scale semantic segmentation is considered as one of the fundamental tasks in 3D scene understanding. Point clouds provide a basic and rich geometric representation of scenes and tangible objects. Convolutional Neural Networks (CNNs) have demonstrated an impressive success in processing regular discrete data such as 2D images and 1D audio. However, CNNs do not directly generalize to point cloud processing due to their irregular and un-ordered nature. One way to extend CNNs to point cloud understanding is to derive an intermediate euclidean representation of a point cloud by projecting onto image domain, voxelizing, or treating points as vertices of an un-directed graph. Graph-CNNs (GCNs) have demonstrated to be a very promising solution for deep learning on irregular data such as social networks, biological systems, and recently point clouds. Early works in literature for graph based point networks relied on constructing dynamic graphs in the node feature space to define a convolution kernel. Later works constructed hierarchical static graphs in 3D space for an encoder-decoder framework inspired from image segmentation. This thesis takes a closer look at both dynamic and static graph neighborhoods of graph- based point networks for the task of semantic segmentation in order to: 1) discuss a potential cause for why going deep in dynamic GCNs does not necessarily lead to an improved performance, and 2) propose a new approach in treating points in a static graph neighborhood for an improved information aggregation. The proposed method leads to an efficient graph based 3D semantic segmentation network that is on par with current state-of-the-art methods on both indoor and outdoor scene semantic segmentation benchmarks such as S3DIS and Semantic3D.

Autonomous vehicles are complex robotic and artificial intelligence systems working together to achieve safe operation in unstructured environments. The objective of this work is to provide a foundation to develop more advanced algorithms for off-road autonomy. The project explores the sensors used for off-road autonomy and the data capture process. Additionally, the point cloud data captured from lidar sensors is processed to restore some of the geometric information lost during sensor sampling. Because ground truth values are needed for quantitative comparison, the MAVS was leveraged to generate a large off-road dataset in a variety of ecosystems. The results demonstrate data capture from the sensor suite and successful reconstruction of the selected geometric information. Using this geometric information, the point cloud data is more accurately segmented using the SqueezeSeg network.

Object detection and seafloor segmentation for conventional 2D side-scan sonar imagery is a well-investigated problem. However, due to recent advances in sensing technology, the side-scan sonar now produces a true 3D point cloud representation of the seafloor embedded with echo intensity. This creates a need to develop algorithms to process the incoming 3D data for applications such as object detection and segmentation, and an opportunity to leverage advances in 3D point cloud processing developed for terrestrial applications using optical sensors (e.g. LiDAR). A bottleneck in deploying 3D side-scan sonar sensors for online applications is attributed to the complexity in handling large amounts of data which requires higher memory for storing and processing data on embedded computers. The present research aims to improve data processing capabilities on-board autonomous underwater vehicles (AUVs). A supervoxel-based framework for over-segmentation and object detection is proposed which reduces a dense point cloud into clusters of similar points in a neighborhood. Supervoxels extracted from the point cloud are then described using feature vectors which are computed using geometry, echo intensity and depth attributes of the constituent points. Unsupervised density based clustering is applied on the feature space to detect objects which appear as outliers. / Master of Science / Acoustic imaging using side-scan sonar sensors has proven to be useful for tasks like seafloor mapping, mine countermeasures and habitat mapping. Due to advancements in sensing technology, a novel type of side-scan sonar sensor is developed which provides true 3D representation of the seafloor along with the echo intensity image. To improve the usability of the novel sensors on-board the carrying vehicles, efficient algorithms needs to be developed. In underwater robotics, limited computational and data storage capabilities are available which poses additional challenges in online perception applications like object detection and segmentation. In this project, I investigate a clustering based approach followed by an unsupervised machine learning method to perform detection of objects on the seafloor using the novel side scan sonar. I also show the usability of the approach for performing segmentation of the seafloor.