Achieving robot autonomy is an extremely challenging task and it starts with developing algorithms that help the robot understand how humans perceive the environment around them. Once the robot understands how to make sense of its environment, it is easy to make efficient decisions about safe movement. It is hard for robots to perform tasks that come naturally to humans like understanding signboards, classifying traffic lights, planning path around dynamic obstacles, etc. In this work, we take up one such challenge of motion segmentation using Light Detection and Ranging (LiDAR) point clouds. Motion segmentation is the task of classifying a point as either moving or static. As the ego-vehicle moves along the road, it needs to detect moving cars with very high certainty as they are the areas of interest which provide cues to the ego-vehicle to plan it's motion. Motion segmentation algorithms segregate moving cars from static cars to give more importance to dynamic obstacles. In contrast to the usual LiDAR scan representations like range images and regular grid, this work uses a modern representation of LiDAR scans using permutohedral lattices. This representation gives ease of representing unstructured LiDAR points in an efficient lattice structure. We propose a machine learning approach to perform motion segmentation. The network architecture takes in two sequential point clouds and performs convolutions on them to estimate if 3D points from the first point cloud are moving or static. Using two temporal point clouds help the network in learning what features constitute motion. We have trained and tested our learning algorithm on the FlyingThings3D dataset and a modified KITTI dataset with simulated motion.

The creation of 3D models of bone from CT images has become popular for surgical planning, the design of implants, and educational purposes. Software is available to convert CT images into 3D models of bone, however, these can be expensive and technically taxing. The goal of this project was to create an open-source and easy-to-use methodology to create 3D models of bone and allow the user to interact with the model to extract desired regions. The method was first created in MATLAB and ported to Python. The CT images were imported into Python and the images were then binarized using a desired threshold determined by the user and based on Hounsfield Units (HU). A Canny edge detector was applied to the binarized images, this extracted the inner and outer surfaces of the bone. Edge points were assigned x, y, and z coordinates based on their pixel location, and the location of the slice in the stack of CT images to create a 3D point cloud. The application of a Delaunay tetrahedralization created a mesh object, the surface was extracted and saved as an STL file. An add-on in Blender was created to allow the user to select the CT images to import, set a threshold, create a 3D mesh model, draw an ROI on the model, and extract that region based on the desired thickness and create a new 3D object. The method was fully open-sourced so was inexpensive and was able to create models of a skull and allow the segmentation of portions of that mesh to create new objects. Future work needs to be conducted to improve the quality of the mesh, implement sampling to reduce the time to create the mesh, and add features that would benefit the end-user. / Thesis / Master of Applied Science (MASc) / The creation of 3D models of bone from CT images has become popular for education, surgical planning, and the design of implants. Software is available to convert CT images into 3D models but can be expensive and technically taxing. The purpose of this project was to develop a process to allow surgeons to create and interact with models from imaging data. This project applied a threshold to binarize a set of CT images, extracted the edges using a Canny Edge detector, and used the edge pixels to create a 3D point cloud. The 3D point cloud was then converted to a mesh object. A user interface was implemented that allowed the selection of portions of the model and a new 3D model to be created from the selection. The process can be improved by improving the quality of the mesh output and adding features to the user interface.

This diploma thesis is dealing with creation of thematical map of castle courtyard of the Veveří castle using the method of laser scanning. The thesis describes the construction of a survey net which is important for measuring control points and then scanning of the area using the scanner FARO Focus 3D. The thesis contains a description of the method of laser scanning and point cloud processing in Trimble RealWorks software. Furthermore, this thesis deals with the accuracy of the point cloud and compares the laser scanning method with the typical tachymetric method according to time duration.

Digital models are essential to quantifying the behavior of structural systems. In many cases, the creation of these models involves manual measurements taken in the field, followed by a manual creation of this model using these measurements. Both of these steps are time consuming and prohibitively expensive, leading to a lack of utilization of accurate models. We propose a framework built on the processing of 3D laser scanning data to partially automate the creation of these models. We focus on steel structures, as they represent a gap in current research into this field. Previous research has focused on segmentation of the point cloud data in order to extract relevant geometries. These approaches cannot easily be extended to steel structures, so we propose a novel method of processing this data with the goal of creating a full finite element model from the information extracted. Our approach sidesteps the need for segmentation by directly extracting the centerlines of structural elements. We begin by taking "slices" of the point cloud in the three principal directions. Each of these slices is flattened into an image, which allows us to take advantage of powerful image processing techniques. Within these images we use 2d convolution as a template match to isolate structural cross sections. This gives us the centroids of cross sections in the image space, which we can map back to the point cloud space as points along the centerline of the element. By fitting lines in 3d space to these points, we can determine the equations for the centerline of each element. This information could be easily passed into a finite element modeling software where the cross sections are manually defined for each line element. / Modern buildings require a digital counterpart to the physical structure for accurate analysis. Historically, these digital counterparts would be created by hand using the measurements that the building was intended to be built to. Often this is not accurate enough and the as-built system must be measured on site to capture deviations from the original plans. In these cases, a large amount of time must be invested to send personnel out into the field and take large amounts of measurements of the structure. Additionally, these "hand measurements" are prone to user error. We propose a novel method of gathering these field measurements quickly and accurately by using a technique called "laser scanning". These laser scans essentially take a 3D snapshot of the site, which contains all the geometric information of visible elements. While it is difficult to locate items such as steel beams in the 3D data, the cross sections of these structural elements are easily defined in 2D. Our method involves taking 2D slices of this 3D scan which allows us to locate the cross sections of the structural members by searching for template cross-sectional shapes. Once the cross sections have been isolated, their centers can be mapped back from the 2D slice to the 3D space as points along the centerlines of the structural elements. These centerlines represent one of the most time consuming requirements to building digital models of modern buildings, so this method could drastically reduce the total modeling time required by automating this particular step.

In this thesis a methodology for representing 3D subjects and their deformations in adverse situations is studied. The study is focused in providing methods based on registration techniques to improve the data in situations where the sensor is working in the limit of its sensitivity. In order to do this, it is proposed two methods to overcome the problems which can difficult the process in these conditions. First a rigid registration based on model registration is presented, where the model of 3D planar markers is used. This model is estimated using a proposed method which improves its quality by taking into account prior knowledge of the marker. To study the deformations, it is proposed a framework to combine multiple spaces in a non-rigid registration technique. This proposal improves the quality of the alignment with a more robust matching process that makes use of all available input data. Moreover, this framework allows the registration of multiple spaces simultaneously providing a more general technique. Concretely, it is instantiated using colour and location in the matching process for 3D location registration.

Orchards demand large areas of land, thus they are often situated far from major population centres. As a result it is often difficult to obtain the necessary personnel, limiting both growth and productivity. However, if autonomous robots could be integrated into the operation of the orchard, the manpower demand could be reduced. A key problem for any autonomous robot is localisation; how does the robot know where it is? In agriculture robots, the most common approach is to use GPS positioning. However, in an orchard environment, the dense and tall vegetation restricts the usage to large robots that reach above the surroundings. In order to enable the use of smaller robots, it is instead necessary to use a GPS independent system. However, due to the similarity of the environment and the lack of strong recognisable features, it appears unlikely that typical non-GPS solutions will prove successful. Therefore we present a GPS independent localisation system, specifically aimed for orchards, that utilises the inherent structure of the surroundings. Furthermore, we examine and individually evaluate three related sub-problems. The proposed system utilises a 3D point cloud created from a 2D LIDAR and the robot’s movement. First, we show how the data can be segmented into individual trees using a Hidden Semi-Markov Model. Second, we introduce a set of descriptors for describing the geometric characteristics of the individual trees. Third, we present a robust localisation method based on Hidden Markov Models. Finally, we propose a method for detecting segmentation errors when associating new tree measurements with previously measured trees. Evaluation shows that the proposed segmentation method is accurate and yields very few segmentation errors. Furthermore, the introduced descriptors are determined to be consistent and informative enough to allow localisation. Third, we show that the presented localisation method is robust both to noise and segmentation errors. Finally it is shown that a significant majority of all segmentation errors can be detected without falsely labeling correct segmentations as incorrect. / Eftersom fruktodlingar kräver stora markområden är de ofta belägna långt från större befolkningscentra. Detta gör det svårt att finna tillräckligt med arbetskraft och begränsar expansionsmöjligheterna. Genom att integrera autonoma robotar i drivandet av odlingarna skulle arbetet kunna effektiviseras och behovet av arbetskraft minska. Ett nyckelproblem för alla autonoma robotar är lokalisering; hur vet roboten var den är? I jordbruksrobotar är standardlösningen att använda GPS-positionering. Detta är dock problematiskt i fruktodlingar, då den höga och täta vegetationen begränsar användandet till större robotar som når ovanför omgivningen. För att möjliggöra användandet av mindre robotar är det istället nödvändigt att använda ett GPS-oberoende lokaliseringssystem. Detta problematiseras dock av den likartade omgivningen och bristen på distinkta riktpunkter, varför det framstår som osannolikt att existerande standardlösningar kommer fungera i denna omgivning. Därför presenterar vi ett GPS-oberoende lokaliseringssystem, speciellt riktat mot fruktodlingar, som utnyttjar den naturliga strukturen hos omgivningen.Därutöver undersöker vi och utvärderar tre relaterade delproblem. Det föreslagna systemet använder ett 3D-punktmoln skapat av en 2D-LIDAR och robotens rörelse. Först visas hur en dold semi-markovmodell kan användas för att segmentera datasetet i enskilda träd. Därefter introducerar vi ett antal deskriptorer för att beskriva trädens geometriska form. Vi visar därefter hur detta kan kombineras med en dold markovmodell för att skapa ett robust lokaliseringssystem.Slutligen föreslår vi en metod för att detektera segmenteringsfel när nya mätningar av träd associeras med tidigare uppmätta träd. De föreslagna metoderna utvärderas individuellt och visar på goda resultat. Den föreslagna segmenteringsmetoden visas vara noggrann och ge upphov till få segmenteringsfel. Därutöver visas att de introducerade deskriptorerna är tillräckligt konsistenta och informativa för att möjliggöra lokalisering. Ytterligare visas att den presenterade lokaliseringsmetoden är robust både mot brus och segmenteringsfel. Slutligen visas att en signifikant majoritet av alla segmenteringsfel kan detekteras utan att felaktigt beteckna korrekta segmenteringar som inkorrekta.

Face recognition can be used in various biometric applications ranging from identifying criminals entering an airport to identifying an unconscious patient in the hospital With the introduction of 3-dimensional scanners in the last decade, researchers have begun to develop new methods for 3-D face recognition. This thesis focuses on 3-D face recognition using the one- and two-dimensional Discrete Cosine Transform (DCT) . A feature ranking based dimensionality reduction strategy is introduced to select the DCT coefficients that yield the best classification accuracies. Two forms of 3-D representation are used: point cloud and depth map images. These representations are extracted from the original VRML files in a face database and are normalized during the extraction process. Classification accuracies exceeding 97% are obtained using the point cloud images in conjunction with the 2-D DCT.

Cette recherche vise à la modélisation 3D depuis un système de vision fisheye embarqué, utilisée pour une application GNSS dans le cadre du projet Predit CAPLOC. La propagation des signaux satellitaires en milieu urbain est soumise à des réflexions sur les structures, altérant la précision et la disponibilité de la localisation. L’ambition du projet est (1) de définir un système de vision omnidirectionnelle capable de fournir des informations sur la structure 3D urbaine et (2) de montrer qu’elles permettent d’améliorer la localisation.Le mémoire expose les choix en (1) calibrage automatique, (2) mise en correspondance entre images, (3) reconstruction 3D ; chaque algorithme est évalué sur images de synthèse et réelles. De plus, il décrit une manière de corriger les réflexions des signaux GNSS depuis un nuage de points 3D pour améliorer le positionnement. En adaptant le meilleur de l’état de l’art du domaine, deux systèmes sont proposés et expérimentés. Le premier est un système stéréoscopique à deux caméras fisheye orientées vers le ciel. Le second en est l’adaptation à une unique caméra.Le calibrage est assuré à travers deux étapes : l’algorithme des 9 points adapté au modèle « équisolide » couplé à un RANSAC, suivi d’un affinement par optimisation Levenberg-Marquardt. L’effort a été porté sur la manière d’appliquer la méthode pour des performances optimales et reproductibles. C’est un point crucial pour un système à une seule caméra car la pose doit être estimée à chaque nouvelle image.Les correspondances stéréo sont obtenues pour tout pixel par programmation dynamique utilisant un graphe 3D. Elles sont assurées le long des courbes épipolaires conjuguées projetées de manière adaptée sur chaque image. Une particularité est que les distorsions ne sont pas rectifiées afin de ne pas altérer le contenu visuel ni diminuer la précision. Dans le cas binoculaire il est possible d’estimer les coordonnées à l’échelle. En monoculaire, l’ajout d’un odomètre permet d’y arriver. Les nuages successifs peuvent être calés pour former un nuage global en SfM.L’application finale consiste dans l’utilisation du nuage 3D pour améliorer la localisation GNSS. Il est possible d’estimer l’erreur de pseudodistance d’un signal après multiples réflexions et d’en tenir compte pour une position plus précise. Les surfaces réfléchissantes sont modélisées grâce à une extraction de plans et de l’empreinte des bâtiments. La méthode est évaluée sur des paires d’images fixes géo-référencées par un récepteur bas-coût et un récepteur GPS RTK (vérité terrain). Les résultats montrent une amélioration de la localisation en milieu urbain. / This research deals with 3D modelling from an embedded fisheye vision system, used for a GNSS application as part of CAPLOC project. Satellite signal propagation in urban area implies reflections on structures, impairing localisation’s accuracy and availability. The project purpose is (1) to define an omnidirectional vision system able to provide information on urban 3D structure and (2) to demonstrate that it allows to improve localisation.This thesis addresses problems of (1) self-calibration, (2) matching between images, (3) 3D reconstruction ; each algorithm is assessed on computer-generated and real images. Moreover, it describes a way to correct GNSS signals reflections from a 3D point cloud to improve positioning. We propose and evaluate two systems based on state-of-the-art methods. First one is a stereoscopic system made of two sky facing fisheye cameras. Second one is the adaptation of the former to a single camera.Calibration is handled by a two-steps process: the 9-point algorithm fitted to “equisolid” model coupled with a RANSAC, followed by a Levenberg-Marquardt optimisation refinement. We focused on the way to apply the method for optimal and repeatable performances. It is a crucial point for a system composed of only one camera because the pose must be estimated for every new image.Stereo matches are obtained for every pixel by dynamic programming using a 3D graph. Matching is done along conjugated epipolar curves projected in a suitable manner on each image. A distinctive feature is that distortions are not rectified in order to neither degrade visual content nor to decrease accuracy. In the binocular case it is possible to estimate full-scale coordinates.In the monocular case, we do it by adding odometer information. Local clouds can be wedged in SfM to form a global cloud.The end application is the usage of the 3D cloud to improve GNSS localisation. It is possible to estimate and consider a signal pseudodistance error after multiple reflections in order to increase positioning accuracy. Reflecting surfaces are modelled thanks to plane and buildings trace fitting. The method is evaluated on fixed image pairs, georeferenced by a low-cost receiver and a GPS RTK receiver (ground truth). Study results show the localisation improvement ability in urban environment.

<p>Plant root phenotyping technologies play an important role in breeding, plant protection, and other plant science research projects. The root phenotyping customers urgently need technologies that are low-cost, in situ, non-destructive to the roots, and suitable for the natural soil environment. Many recently developed root phenotyping methods such as minirhizotron, CT, and MRI scanners have their unique advantages in observing plant roots, but they also have disadvantages and cannot meet all the critical requirements simultaneously. The study in this paper focuses on the development of a new plant root phenotyping robot that is minimally invasive to plants and working in situ inside natural soil, called “MISIRoot”. The MISIRoot system (patent pending) mainly consists of an industrial-level robotic arm, a mini-size camera with lighting set, a plant pot holding platform, and the image processing software for root recognition and feature extraction. MISIRoot can take high-resolution color images of the roots in soil with minimal disturbance to the root and reconstruct the plant roots’ three-dimensional (3D) structure at an accuracy of 0.1 mm. In a test assay, well-watered and drought-stressed groups of corn plants were measured by MISIRoot at V3, V4, and V5 stages. The system successfully acquired the RGB color images of the roots and extracted the 3D points cloud data which showed the locations of the detected roots in the soil. The plants measured by MISIRoot and plants not measured (controls) were carefully compared with Purdue’s Lilly 13-4 Hyperspectral Imaging Facility (reference). No significant differences were found between the two groups of plants at different growth stages. Therefore, it was concluded that MISIRoot measurements had no significant disturbance to the corn plant’s growth.</p>

Autonomous driving is an emerging area that has been receiving an increasing amount of interest from different companies and researchers. 3D point cloud map is a significant foundation of autonomous driving as it provides essential information for localization and environment perception. However, when trying to gather road information for map creation, the presence of dynamic objects like vehicles, pedestrians, and cyclists will add noise and unnecessary information to the final map. In order to solve the problem, this thesis presents a novel two-stage model that contains a scan-to-scan removal stage and a scan-to-map generation stage. By designing the new three-branch neural network and new attention-based fusion block, the scan-to-scan part achieves a higher mean Intersection-over-Union (mIoU) score. By improving the ground plane estimation, the scan-to-map part can preserve more static points while removing a large number of dynamic points. The test on SemanticKITTI dataset and Scania dataset shows our two-stage model outperforms other baselines. / Autonom körning är ett nytt område som har fått ett allt större intresse från olika företag och forskare. Kartor med 3D-punktmoln är en viktig grund för autonom körning eftersom de ger viktig information för lokalisering och miljöuppfattning. När man försöker samla in väginformation för kartframställning kommer dock närvaron av dynamiska objekt som fordon, fotgängare och cyklister att lägga till brus och onödig information till den slutliga kartan. För att lösa problemet presenteras i den här avhandlingen en ny tvåstegsmodell som innehåller ett steg för borttagning av skanningar och ett steg för generering av skanningar och kartor. Genom att utforma det nya neurala nätverket med tre grenar och det nya uppmärksamhetsbaserade fusionsblocket uppnår scan-to-scan-delen högre mean Intersection-over-Union (mIoU)-poäng. Genom att förbättra uppskattningen av markplanet kan skanning-till-kartor-delen bevara fler statiska punkter samtidigt som ett stort antal dynamiska punkter avlägsnas. Testet av SemanticKITTI-dataset och Scania-dataset visar att vår tvåstegsmodell överträffar andra baslinjer.