For a long time stereo-cameras have been deployed in visual Simultaneous Localization And Mapping (SLAM) systems to gain 3D information. Even though stereo-cameras show good performance, the main disadvantage is the complex and expensive hardware setup it requires, which limits the use of the system. A simpler and cheaper alternative are monocular cameras, however monocular images lack the important depth information. Recent works have shown that having access to depth maps in monocular SLAM system is beneficial since they can be used to improve the 3D reconstruction. This work proposes a deep neural network that predicts dense high-resolution depth maps from monocular RGB images by casting the problem as a supervised regression task. The network architecture follows an encoder-decoder structure in which multi-scale information is captured and skip-connections are used to recover details. The network is trained and evaluated on the KITTI dataset achieving results comparable to state-of-the-art methods. With further development, this network shows good potential to be incorporated in a monocular SLAM system to improve the 3D reconstruction.

La localisation d’un véhicule via les techniques de SLAM (Simultaneous Localization And Mapping pour cartographie et localisation simultanées) a connu un essor important durant les 20 dernières années. Pourtant, peu d’approches ont tenté d’étendre ces algorithmes à une flotte de véhicules malgré les nombreuses applications potentielles. C’est ici l’objectif de cette thèse. Pour ce faire, une approche de SLAM monoculaire pour un seul véhicule a d’abord été développée. Celle-ci propose de coupler un filtre de Kalman étendu avec une représentation cartésienne des amers afin de produire des cartes de faible densité mais de qualité. En effet, l’extension à plusieurs véhicules nécessite des échanges permanents par l’intermédiaire de communications sans fil. Avec peu d’amers dans les cartes, notre approche s’accommode bien du nombre de véhicules de la flotte. Des capteurs peu onéreux ont aussi été privilégiés (une unique caméra et un odomètre) afin de réduire le coût d’une extension multivéhicule. Des correctifs ont été proposés afin d’éviter les problèmes de divergence induits par les choix précédents. Des expérimentations ont montré que la solution de SLAM produite était légère et rapide tout en fournissant une localisation de qualité. La dérive, inhérente à tout algorithme de SLAM, a également fait l’objet d’une analyse. Celle-ci a été intégrée au SLAM par l’intermédiaire d’une architecture dédiée et d’un modèle dynamique. Le but est de pouvoir rendre consistante la localisation fournie par le SLAM, même en l’absence d’estimation de la dérive. Cela permet d’effectuer des fermetures de boucle ou encore d’intégrer des informations géo-référencées de manière naturelle tout en conservant l’intégrité de la solution. En multivéhicule, cet aspect est un point clef puisque chaque véhicule dérive différemment des autres. Il est donc important de le prendre en compte. Enfin, le SLAM a été étendu à plusieurs véhicules. Une structure générique a été prévue afin que notre approche monoculaire puisse être remplacée par n’importe quel algorithme de SLAM. Notre architecture décentralisée évite la consanguinité des données (le fait de compter deux fois une même information) et gère les défaillances réseau, que cela soit des ruptures de communication ou encore des latences dans la réception des données. La partie statique du modèle de dérive permet également de prendre en compte le fait que les positions initiales des véhicules d’une flotte puissent être inconnues. L’intégrité est ainsi maintenue en permanence. Enfin, notre approche étant entièrement décentralisée, elle a pu être testée et validée en simulation et avec des expérimentations réelles dans diverses configurations (convoi en colonne ou en ligne, avec 2 ou 3 véhicules). / The localization of a vehicle with the use of SLAM techniques (Simultaneous Localization And Mapping) has been extensively studied during the last 20 years. However, only a few approaches have tried to extend these algorithms to a fleet of vehicles despite the many potential applications. It is the objective of this thesis. First of all, a monocular SLAM for a single vehicle has been developed. This one proposes to pair an Extended Kalman Filter with a Cartesian representation for landmarks so as to produce accurate low density maps. Indeed, the extension of SLAM to several vehicles requires permanent communications inside the fleet. With only a few landmarks mapped, our approach scales nicely with the number of vehicles. Cheap sensors have been favored (a single camera and an odometer) in order to spread more easily the use of multi-vehicle applications. Correctives have been proposed in order to avoid the divergence problems induced by such a scheme. The experiments showed that our SLAM is able to furnish good localization results while being light and fast.The drift affecting every SLAM algorithm has also been studied. Its integration inside the SLAM process, thanks to a dedicated architecture and a dynamic model, allows to ensure consistency even without an estimation of it. Loop closures or the integration of geo-referenced information becomes straightforward. They naturally correct all the past positions while still maintaining consistency. In a multi-vehicle scenario, it is a key aspect as each vehicle drifts differently from one another. It is consequently important to take it into account. Our SLAM algorithm has then been extended to several vehicles. A generic structure has been used so as to allow any SLAM algorithm to replace our monocular SLAM. The multi-vehicle architecture avoids data incest (double-counting information) and handles network failures, be they communication breakdowns or latencies when receiving data. The static part of the drift model allows to take into account the fact that the initial positions of the different vehicles composing the fleet might be unknown. Consistency is thus permanently preserved. Our approach has been successfully tested using simulations and real experiments with various settings (row or column convoy with 2 or 3 vehicles) in a fully decentralized way.

Modern automobiles and particularly those with off-road lineage possess subsystems that can be configured to better negotiate certain terrain types. Different terrain classes amount to different adherence (or surface grip) and compressibility properties that impact vehicle ma-noeuvrability and should therefore incur a tailored throttle response, suspension stiffness and so on. This thesis explores prospective terrain recognition for an anticipating terrain response driver assistance system. Recognition of terrain and road terrain is cast as a semantic segmen-tation task whereby forward driving images or point clouds are pre-segmented into atomic units and subsequently classified. Terrain classes are typically of amorphous spatial extent con-taining homogenous or granularly repetitive patterns. For this reason, colour and texture ap-pearance is the saliency of choice for monocular vision. In this work, colour, texture and sur-face saliency of atomic units are obtained with a bag-of-features approach. Five terrain classes are considered, namely grass, dirt, gravel, shrubs and tarmac. Since colour can be ambiguous among terrain classes such as dirt and gravel, several texture flavours are explored with scalar and structured output learning in a bid to devise an appropriate visual terrain saliency and predictor combination. Texture variants are obtained using local binary patters (LBP), filter responses (or textons) and dense key-point descriptors with daisy. Learning algorithms tested include support vector machine (SVM), random forest (RF) and logistic regression (LR) as scalar predictors while a conditional random field (CRF) is used for structured output learning. The latter encourages smooth labelling by incorporating the prior knowledge that neighbouring segments with similar saliency are likely segments of the same class. Once a suitable texture representation is devised the attention is shifted from monocular vision to stereo vision. Sur-face saliency from reconstructed point clouds can be used to enhance terrain recognition. Pre-vious superpixels span corresponding supervoxels in real world coordinates and two surface saliency variants are proposed and tested with all predictors: one using the height coordinates of point clouds and the other using fast point feature histograms (FPFH). Upon realisation that road recognition and terrain recognition can be assumed as equivalent problems in urban en-vironments, the top most accurate models consisting of CRFs are augmented with composi-tional high order pattern potentials (CHOPP). This leads to models that are able to strike a good balance between smooth local labelling and global road shape. For urban environments the label set is restricted to road and non-road (or equivalently tarmac and non-tarmac). Ex-periments are conducted using a proprietary terrain dataset and a public road evaluation da-taset.

Master’s thesis deals with object pose estimation using monocular camera. As an object is considered every rigid, shape fixed entity with strong edges, ideally textureless. Object position in this work is represented by transformation matrix, which describes object translation and rotation towards world coordinate system. First chapter is dedicated to explanation of theory of geometric transformations and intrinsic and extrinsic parameters of camera. This chapter also describes detection algorithm Chamfer Matching, which is used in this work. Second chapter describes all development tools used in this work. Third, fourth and fifth chapter are dedicated to practical realization of this works goal and achieved results. Last chapter describes created application, that realizes known object pose estimation in scene.

QC 20110412

No description available.

Yes / Recovering high-quality 3D point clouds from monocular endoscopic images is a challenging task. This paper proposes a novel deep learning-based computational framework for 3D point cloud reconstruction from single monocular endoscopic images. An unsupervised mono-depth learning network is used to generate depth information from monocular images. Given a single mono endoscopic image, the network is capable of depicting a depth map. The depth map is then used to recover a dense 3D point cloud. A generative Endo-AE network based on an auto-encoder is trained to repair defects of the dense point cloud by generating the best representation from the incomplete data. The performance of the proposed framework is evaluated against state-of-the-art learning-based methods. The results are also compared with non-learning based stereo 3D reconstruction algorithms. Our proposed methods outperform both the state-of-the-art learning-based and non-learning based methods for 3D point cloud reconstruction. The Endo-AE model for point cloud completion can generate high-quality, dense 3D endoscopic point clouds from incomplete point clouds with holes. Our framework is able to recover complete 3D point clouds with the missing rate of information up to 60%. Five large medical in-vivo databases of 3D point clouds of real endoscopic scenes have been generated and two synthetic 3D medical datasets are created. We have made these datasets publicly available for researchers free of charge. The proposed computational framework can produce high-quality and dense 3D point clouds from single mono-endoscopy images for augmented reality, virtual reality and other computer-mediated medical applications.

Depth sensing is pivotal in robotics; however, monocular depth estimation encounters significant challenges. Existing algorithms relying on large-scale labeled data and large Deep Convolutional Neural Networks (DCNNs) hinder real-world applications. We propose two lightweight architectures that achieve commendable accuracy rates of 91.2% and 90.1%, simultaneously reducing the Root Mean Square Error (RMSE) of depth to 4.815 and 5.036. Our lightweight depth model operates at 29-44 FPS on the Jetson Nano GPU, showcasing efficient performance with minimal power consumption. Moreover, we introduce a mask network designed to visualize and analyze the compact depth network, aiding in discerning informative samples for the active learning approach. This contributes to increased model accuracy and enhanced generalization capabilities. Furthermore, our methodology encompasses the introduction of an active learning framework strategically designed to enhance model performance and accuracy by efficiently utilizing limited labeled training data. This novel framework outperforms previous studies by achieving commendable results with only 18.3% utilization of the KITTI Odometry dataset. This performance reflects a skillful balance between computational efficiency and accuracy, tailored for low-cost devices while reducing data training requirements.:1. Introduction 2. Literature Review 3. AI Technologies for Edge Computing 4. Monocular Depth Estimation Methodology 5. Implementation 6. Result and Evaluation 7. Conclusion and Future Scope Appendix

Rail track asset sighting distance must be checked regularly to ensure the continued and safe operation of rolling stock. Methods currently used to check asset line-of-sight involve manual labour or laser systems. Video cameras and computer vision techniques provide one possible route for cheaper, automated systems. Three categories of computer vision method are identified for possible application: two-dimensional object recognition, two-dimensional object tracking and three-dimensional scene recovery. However, presented experimentation shows recognition and tracking methods produce less accurate asset line-of-sight results for increasing asset-camera distance. Regarding three-dimensional scene recovery, evidence is presented suggesting a relationship between image feature and recovered scene information. A novel framework which learns these relationships is proposed. Learnt relationships from recovered image features probabilistically limit the search space of future features, improving efficiency. This framework is applied to several scene recovery methods and is shown (on average) to decrease computation by two-thirds for a possible, small decrease in accuracy of recovered scenes. Asset line-of-sight results computed from recovered three-dimensional terrain data are shown to be more accurate than two-dimensional methods, not effected by increasing asset-camera distance. Finally, the analysis of terrain in terms of effect on asset line-of-sight is considered. Terrain elements, segmented using semantic information, are ranked with a metric combining a minimum line-of-sight blocking distance and the growth required to achieve this minimum distance. Since this ranking measure is relative, it is shown how an approximation of the terrain data can be applied, decreasing computation time. Further efficiency increases are found by decomposing the problem into a set of two-dimensional problems and applying binary search techniques. The combination of the research elements presented in this thesis provide efficient methods for automatically analysing asset line-of-sight and the impact of the surrounding terrain, from captured monocular video.

Les travaux présentés dans ce manuscrit concernent le domaine de la localisation et la reconstruction 3D par vision artificielle. Dans ce contexte, la trajectoire d’une caméra et la structure3D de la scène filmée sont initialement estimées par des algorithmes linéaires puis optimisées par un algorithme non-linéaire, l’ajustement de faisceaux. Cette thèse présente tout d’abord une technique de recherche de l’amplitude de déplacement (recherche linéaire), ou line search pour les algorithmes de minimisation itérative. La technique proposée est non itérative et peut être rapidement implantée dans un ajustement de faisceaux traditionnel. Cette technique appelée recherche linéaire algébrique globale (G-ALS), ainsi que sa variante à deux dimensions (Two way-ALS), accélèrent la convergence de l’algorithme d’ajustement de faisceaux. L’approximation de l’erreur de reprojection par une distance algébrique rend possible le calcul analytique d’une amplitude de déplacement efficace (ou de deux pour la variante Two way-ALS), par la résolution d’un polynôme de degré 3 (G-ALS) ou 5 (Two way-ALS). Nos expérimentations sur des données simulées et réelles montrent que cette amplitude, optimale en distance algébrique, est performante en distance euclidienne, et permet de réduire le temps de convergence des minimisations. Une difficulté des algorithmes de localisation en temps réel par la vision (SLAM monoculaire) est que la trajectoire estimée est souvent affectée par des dérives : dérives d’orientation, de position et d’échelle. Puisque ces algorithmes sont incrémentaux, les erreurs et approximations sont cumulées tout au long de la trajectoire, et une dérive se forme sur la localisation globale. De plus, un système de localisation par vision peut toujours être ébloui ou utilisé dans des conditions qui ne permettent plus temporairement de calculer la localisation du système. Pour résoudre ces problèmes, nous proposons d’utiliser un capteur supplémentaire mesurant les déplacements de la caméra. Le type de capteur utilisé varie suivant l’application ciblée (un odomètre pour la localisation d’un véhicule, une centrale inertielle légère ou un système de navigation à guidage inertiel pour localiser une personne). Notre approche consiste à intégrer ces informations complémentaires directement dans l’ajustement de faisceaux, en ajoutant un terme de contrainte pondéré dans la fonction de coût. Nous évaluons trois méthodes permettant de sélectionner dynamiquement le coefficient de pondération et montrons que ces méthodes peuvent être employées dans un SLAM multi-capteur temps réel, avec différents types de contrainte, sur l’orientation ou sur la norme du déplacement de la caméra. La méthode est applicable pour tout autre terme de moindres carrés. Les expérimentations menées sur des séquences vidéo réelles montrent que cette technique d’ajustement de faisceaux contraint réduit les dérives observées avec les algorithmes de vision classiques. Ils améliorent ainsi la précision de la localisation globale du système. / The works presented in this manuscript are in the field of computer vision, and tackle the problem of real-time vision based localization and 3D reconstruction. In this context, the trajectory of a camera and the 3D structure of the filmed scene are initially estimated by linear algorithms and then optimized by a nonlinear algorithm, bundle adjustment. The thesis first presents a new technique of line search, dedicated to the nonlinear minimization algorithms used in Structure-from-Motion. The proposed technique is not iterative and can be quickly installed in traditional bundle adjustment frameworks. This technique, called Global Algebraic Line Search (G-ALS), and its two-dimensional variant (Two way-ALS), accelerate the convergence of the bundle adjustment algorithm. The approximation of the reprojection error by an algebraic distance enables the analytical calculation of an effective displacement amplitude (or two amplitudes for the Two way-ALS variant) by solving a degree 3 (G-ALS) or 5 (Two way-ALS) polynomial. Our experiments, conducted on simulated and real data, show that this amplitude, which is optimal for the algebraic distance, is also efficient for the Euclidean distance and reduces the convergence time of minimizations. One difficulty of real-time tracking algorithms (monocular SLAM) is that the estimated trajectory is often affected by drifts : on the absolute orientation, position and scale. Since these algorithms are incremental, errors and approximations are accumulated throughout the trajectory and cause global drifts. In addition, a tracking vision system can always be dazzled or used under conditions which prevented temporarily to calculate the location of the system. To solve these problems, we propose to use an additional sensor measuring the displacement of the camera. The type of sensor used will vary depending on the targeted application (an odometer for a vehicle, a lightweight inertial navigation system for a person). We propose to integrate this additional information directly into an extended bundle adjustment, by adding a constraint term in the weighted cost function. We evaluate three methods (based on machine learning or regularization) that dynamically select the weight associated to the constraint and show that these methods can be used in a real time multi-sensor SLAM, and validate them with different types of constraint on the orientation or on the scale. Experiments conducted on real video sequences show that this technique of constrained bundle adjustment reduces the drifts observed with the classical vision algorithms and improves the global accuracy of the positioning system.