This thesis addresses the problem of domain shift in 3D point cloud perception. In the last decades, there has been tremendous progress in within-domain training and testing. However, the performance of perception models is affected when training on a source domain and testing on a target domain sampled from different data distributions. As a result, a change in sensor or geo-location can lead to a harmful drop in model performance. While solutions exist for image perception, addressing this problem in point clouds remains unresolved. The focus of this thesis is the study and design of solutions for mitigating domain shift in 3D point cloud perception. We identify several settings differing in the level of target supervision and the availability of source data. We conduct a thorough study of each setting and introduce a new method to solve domain shift in each configuration. In particular, we study three novel settings in domain adaptation and domain generalization and propose five new methods for mitigating domain shift in 3D point cloud perception. Our methods are used by the research community, and at the time of writing, some of the proposed approaches hold the state-of-the-art. In conclusion, this thesis provides a valuable contribution to the computer vision community, setting the groundwork for the development of future works in cross-domain conditions.

Pour agir de manière autonome et intelligente dans un environnement, un robot mobile doit disposer de cartes. Une carte contient les informations spatiales sur l’environnement. La géométrie 3D ainsi connue par le robot est utilisée non seulement pour éviter la collision avec des obstacles, mais aussi pour se localiser et pour planifier des déplacements. Les robots de prochaine génération ont besoin de davantage de capacités que de simples cartographies et d’une localisation pour coexister avec nous. La quintessence du robot humanoïde de service devra disposer de la capacité de voir comme les humains, de reconnaître, classer, interpréter la scène et exécuter les tâches de manière quasi-anthropomorphique. Par conséquent, augmenter les caractéristiques des cartes du robot à l’aide d’attributs sémiologiques à la façon des humains, afin de préciser les types de pièces, d’objets et leur aménagement spatial, est considéré comme un plus pour la robotique d’industrie et de services à venir. Une carte sémantique enrichit une carte générale avec les informations sur les entités, les fonctionnalités ou les événements qui sont situés dans l’espace. Quelques approches ont été proposées pour résoudre le problème de la cartographie sémantique en exploitant des scanners lasers ou des capteurs de temps de vol RGB-D, mais ce sujet est encore dans sa phase naissante. Dans cette thèse, une tentative de reconstruction sémantisée d’environnement d’intérieur en utilisant une caméra temps de vol qui ne délivre que des informations de profondeur est proposée. Les caméras temps de vol ont modifié le domaine de l’imagerie tridimensionnelle discrète. Elles ont dépassé les scanners traditionnels en termes de rapidité d’acquisition des données, de simplicité fonctionnement et de prix. Ces capteurs de profondeur sont destinés à occuper plus d’importance dans les futures applications robotiques. Après un bref aperçu des approches les plus récentes pour résoudre le sujet de la cartographie sémantique, en particulier en environnement intérieur. Ensuite, la calibration de la caméra a été étudiée ainsi que la nature de ses bruits. La suppression du bruit dans les données issues du capteur est menée. L’acquisition d’une collection d’images de points 3D en environnement intérieur a été réalisée. La séquence d’images ainsi acquise a alimenté un algorithme de SLAM pour reconstruire l’environnement visité. La performance du système SLAM est évaluée à partir des poses estimées en utilisant une nouvelle métrique qui est basée sur la prise en compte du contexte. L’extraction des surfaces planes est réalisée sur la carte reconstruite à partir des nuages de points en utilisant la transformation de Hough. Une interprétation sémantique de l’environnement reconstruit est réalisée. L’annotation de la scène avec informations sémantiques se déroule sur deux niveaux : l’un effectue la détection de grandes surfaces planes et procède ensuite en les classant en tant que porte, mur ou plafond; l’autre niveau de sémantisation opère au niveau des objets et traite de la reconnaissance des objets dans une scène donnée. A partir de l’élaboration d’une signature de forme invariante à la pose et en passant par une phase d’apprentissage exploitant cette signature, une interprétation de la scène contenant des objets connus et inconnus, en présence ou non d’occultations, est obtenue. Les jeux de données ont été mis à la disposition du public de la recherche universitaire. / Intelligent autonomous actions in an ordinary environment by a mobile robot require maps. A map holds the spatial information about the environment and gives the 3D geometry of the surrounding of the robot to not only avoid collision with complex obstacles, but also selflocalization and for task planning. However, in the future, service and personal robots will prevail and need arises for the robot to interact with the environment in addition to localize and navigate. This interaction demands the next generation robots to understand, interpret its environment and perform tasks in human-centric form. A simple map of the environment is far from being sufficient for the robots to co-exist and assist humans in the future. Human beings effortlessly make map and interact with environment, and it is trivial task for them. However, for robots these frivolous tasks are complex conundrums. Layering the semantic information on regular geometric maps is the leap that helps an ordinary mobile robot to be a more intelligent autonomous system. A semantic map augments a general map with the information about entities, i.e., objects, functionalities, or events, that are located in the space. The inclusion of semantics in the map enhances the robot’s spatial knowledge representation and improves its performance in managing complex tasks and human interaction. Many approaches have been proposed to address the semantic SLAM problem with laser scanners and RGB-D time-of-flight sensors, but it is still in its nascent phase. In this thesis, an endeavour to solve semantic SLAM using one of the time-of-flight sensors which gives only depth information is proposed. Time-of-flight cameras have dramatically changed the field of range imaging, and surpassed the traditional scanners in terms of rapid acquisition of data, simplicity and price. And it is believed that these depth sensors will be ubiquitous in future robotic applications. In this thesis, an endeavour to solve semantic SLAM using one of the time-of-flight sensors which gives only depth information is proposed. Starting with a brief motivation in the first chapter for semantic stance in normal maps, the state-of-the-art methods are discussed in the second chapter. Before using the camera for data acquisition, the noise characteristics of it has been studied meticulously, and properly calibrated. The novel noise filtering algorithm developed in the process, helps to get clean data for better scan matching and SLAM. The quality of the SLAM process is evaluated using a context-based similarity score metric, which has been specifically designed for the type of acquisition parameters and the data which have been used. Abstracting semantic layer on the reconstructed point cloud from SLAM has been done in two stages. In large-scale higher-level semantic interpretation, the prominent surfaces in the indoor environment are extracted and recognized, they include surfaces like walls, door, ceiling, clutter. However, in indoor single scene object-level semantic interpretation, a single 2.5D scene from the camera is parsed and the objects, surfaces are recognized. The object recognition is achieved using a novel shape signature based on probability distribution of 3D keypoints that are most stable and repeatable. The classification of prominent surfaces and single scene semantic interpretation is done using supervised machine learning and deep learning systems. To this end, the object dataset and SLAM data are also made publicly available for academic research.

Corrosion damage reduces the load-carrying capacity of bridges which poses a threat to passenger safety. The objective of this research was to reduce the resources involved in conventional bridge inspections which are an important tool in the condition assessment of bridges and to help in determining if live load testing is necessary. This research proposes a framework to link semi-automated damage detection on prestressed concrete bridge girders with the estimation of their residual flexural capacity. The framework was implemented on four full-scale corrosion damaged girders from decommissioned bridges in Virginia. 3D point clouds of the girders reconstructed from images using Structure from Motion (SfM) approach were textured with images containing cracks detected at pixel level using a U-Net (Fully Convolutional Network). Spalls were detected by identifying the locations where normals associated with the points in the 3D point cloud deviated from being perpendicular to the reference directions chosen, by an amount greater than a threshold angle. 3D textured mesh models, overlaid with the detected cracks and spalls were used as 3D damage maps to determine reduced cross-sectional areas of prestressing strands to account for the corrosion damage as per the recommendations of Naito, Jones, and Hodgson (2011). Scaling them to real-world dimensions enabled the measurement of any required dimension, eliminating the need for physical contact. The flexural capacities of a box beam and an I-beam estimated using strain compatibility analysis were validated with the actual capacities at failure sections determined from four destructive tests conducted by Al Rufaydah (2020). Along with the reduction in the cross-sectional areas of strands, limiting the ultimate strain that heavily corroded strands can develop was explored as a possible way to improve the results of the analysis. Strain compatibility analysis was used to estimate the ultimate rupture strain, in the heavily corroded bottommost layer prestressing strands exposed before the box beam was tested. More research is required to associate each level of strand corrosion with an average ultimate strain at which the corroded strands rupture. This framework was found to give satisfactory estimates of the residual strength. Reduction in resources involved in current visual inspection practices and eliminating the need for physical access, make this approach worthwhile to be explored further to improve the output of each step in the proposed framework. / Master of Science / Corrosion damage is a major concern for bridges as it reduces their load carrying capacity. Bridge failures in the past have been attributed to corrosion damage. The risk associated with corrosion damage caused failures increases as the infrastructure ages. Many bridges across the world built forty to fifty years ago are now in a deteriorated condition and need to be repaired and retrofitted. Visual inspections to identify damage or deterioration on a bridge are very important to assess the condition of the bridge and determine the need for repairing or for posting weight restrictions for the vehicles that use the bridge. These inspections require close physical access to the hard-to-reach areas of the bridge for physically measuring the damage which involves many resources in the form of experienced engineers, skilled labor, equipment, time, and money. The safety of the personnel involved in the inspections is also a major concern. Nowadays, a lot of research is being done in using Unmanned Aerial Vehicles (UAVs) like drones for bridge inspections and in using artificial intelligence for the detection of cracks on the images of concrete and steel members. Girders or beams in a bridge are the primary longitudinal load carrying members. Concrete inherently is weak in tension. To address this problem, High Strength steel reinforcement (called prestressing steel or prestressing strands) in prestressed concrete beams is pre-loaded with a tensile force before the application of any loads so that the regions which will experience tension under the service loads would be subjected to a pre-compression to improve the performance of the beam and delay cracking. Spalls are a type of corrosion damage on concrete members where portions of concrete fall off (section loss) due to corrosion in the steel reinforcement, exposing the reinforcement to the environment which leads to accelerated corrosion causing a loss of cross-sectional area and ultimately, a rupture in the steel. If the process of detecting the damage (cracks, spalls, exposed or severed reinforcement, etc.) is automated, the next logical step that would add great value would be, to quantify the effect of the damage detected on the load carrying capacity of the bridges. Using a quantified estimate of the remaining capacity of a bridge, determined after accounting for the corrosion damage, informed decisions can be made about the measures to be taken. This research proposes a stepwise framework to forge a link between a semi-automated visual inspection and residual capacity evaluation of actual prestressed concrete bridge girders obtained from two bridges that have been removed from service in Virginia due to extensive deterioration. 3D point clouds represent an object as a set of points on its surface in three dimensional space. These point clouds can be constructed either using laser scanning or using Photogrammetry from images of the girders captured with a digital camera. In this research, 3D point clouds are reconstructed from sequences of overlapping images of the girders using an approach called Structure from Motion (SfM) which locates matched pixels present between consecutive images in the 3D space. Crack-like features were automatically detected and highlighted on the images of the girders that were used to build the 3D point clouds using artificial intelligence (Neural Network). The images with cracks highlighted were applied as texture to the surface mesh on the point cloud to transfer the detail, color, and realism present in the images to the 3D model. Spalls were detected on 3D point clouds based on the orientation of the normals associated with the points with respect to the reference directions. Point clouds and textured meshes of the girders were scaled to real-world dimensions facilitating the measurement of any required dimension on the point clouds, eliminating the need for physical contact in condition assessment. Any cracks or spalls that went unidentified in the damage detection were visible on the textured meshes of the girders improving the performance of the approach. 3D textured mesh models of the girders overlaid with the detected cracks and spalls were used as 3D damage maps in residual strength estimation. Cross-sectional slices were extracted from the dense point clouds at various sections along the length of each girder. The slices were overlaid on the cross-section drawings of the girders, and the prestressing strands affected due to the corrosion damage were identified. They were reduced in cross-sectional area to account for the corrosion damage as per the recommendations of Naito, Jones, and Hodgson (2011) and were used in the calculation of the ultimate moment capacity of the girders using an approach called strain compatibility analysis. Estimated residual capacities were compared to the actual capacities of the girders found from destructive tests conducted by Al Rufaydah (2020). Comparisons are presented for the failure sections in these tests and the results were analyzed to evaluate the effectiveness of this framework. More research is to be done to determine the factors causing rupture in prestressing strands with different degrees of corrosion. This framework was found to give satisfactory estimates of the residual strength. Reduction in resources involved in current visual inspection practices and eliminating the need for physical access, make this approach worthwhile to be explored further to improve the output of each step in the proposed framework.

L'imagerie numérique, de la synthèse d'images à la vision par ordinateur est en train de connaître une forte évolution, due entre autres facteurs à la démocratisation et au succès commercial des caméras 3D. Dans le même contexte, l'impression 3D grand public, qui est en train de vivre un essor fulgurant, contribue à la forte demande sur ce type de caméra pour les besoins de la numérisation 3D. L'objectif de cette thèse est d'acquérir et de maîtriser un savoir-faire dans le domaine de la capture/acquisition de modèles 3D en particulier sur l'aspect rendu réaliste. La réalisation d'un scanner 3D à partir d'une caméra RGB-D fait partie de l'objectif. Lors de la phase d'acquisition, en particulier pour un dispositif portable, on est confronté à deux problèmes principaux, le problème lié au référentiel de chaque capture et le rendu final de l'objet reconstruit. / Digital imaging, from the synthesis of images to computer vision isexperiencing a strong evolution, due among other factors to the democratization and commercial success of 3D cameras. In the same context, the consumer 3D printing, which is experiencing a rapid rise, contributes to the strong demand for this type of camera for the needs of 3D scanning. The objective of this thesis is to acquire and master a know-how in the field of the capture / acquisition of 3D models in particular on the rendered aspect. The realization of a 3D scanner from a RGB-D camera is part of the goal. During the acquisition phase, especially for a portable device, there are two main problems, the problem related to the repository of each capture and the final rendering of the reconstructed object.