Les tâches réalisées en traitement d'image tendent à devenir de plus en plus complexes. Par exemple, dans le contexte routier, les systèmes d'aide à la conduite, (Advanced driver-assistance systems), visent à une automatisation complète de la tâche de conduite. L’évaluation de la fiabilité représente un enjeu important pour ce type d’application. Face à la difficulté des tâches à réaliser, les chaînes de traitements sont souvent divisées en de nombreuses étapes de calculs de sorte qu'il est difficile de caractériser les sorties de la chaîne en fonction des perturbations des entrées. Les étapes du traitement consistent le plus souvent en des problèmes formulés comme la minimisation d'une énergie. Cette énergie est généralement difficile à optimiser, ce qui nécessite la mise en œuvre de méthodes d’optimisation adaptées. Dans cette thèse, nous cherchons à caractériser la solution d’un traitement à partir des calculs réalisés au cours de l’étape d'optimisation. Cette approche nous a permis de proposer des indices de stabilité de la solution dans le cadre de deux méthodes d’optimisation discrètes : la coupure de graphe et la programmation dynamique. Tout d’abord, nous nous sommes intéressés au problème de la reconstruction stéréoscopique en contexte routier et au dé-bruitage, dans le cadre de l’optimisation par coupure de graphe. Les modèles issus de l’interprétation bayésienne amènent à optimiser des énergies qui ne peuvent pas être traitées avec les schémas d’optimisation classiques par fusion binaire. Nous avons proposé un schéma adapté qui met en jeu des fusions binaires par expansion et par saut. L’application de ce schéma aux problèmes de la reconstruction stéréoscopique et au dé-bruitage, nous a permis d’obtenir des solutions possédant les caractéristiques que nous recherchions : des contours d’objets nets et des dégradés progressifs dans les zones homogènes. Ensuite, dans le contexte de la programmation dynamique, nous avons réinterprété l’a priori mis en jeu dans la méthode de reconstruction Semi-Global Matching ainsi que certaines de ses variantes. Nous avons proposé d’ajouter un paramètre à ces méthodes afin de modifier les directions privilégiées par l’a priori. Enfin, nous avons proposé des indices de stabilité de la solution dans le cadre de la coupure de graphe et de la programmation dynamique. La prise en compte de ces indices, dans une étape de raffinement des solutions, permet une amélioration des résultats / Problems solved by image processing tend to be more and more complex. For instance, in road context, ADAS (Advanced driver-assistance systems) aim to a completely automatic diving tack. Evaluating system reliability is an important challenge in that case. These tasks being hard to perform, processing chains are often divide in numerous processing steps. As a consequence, characterizing the output using the input of the chain is not obvious. Most of the time, image processing steps are formulate as an energy minimization. These energies are often hard to minimize and need to apply suitable optimization methods. In this thesis, we aim to characterize the solution during the optimization step. Using this approach, we proposed stability index with two discrete optimization methods : graph-cut and dynamic optimization. First, we focused on stereoscopic reconstruction problem in road context and on denoising problem using graph-cut. Models obtained by Bayesian interpretation lead to optimize energies witch cannot be handled by classical binary fusion optimization scheme. We proposed a suitable scheme composed of fusion by expansion and fusions by step. When this scheme is apply to stereoscopic reconstruction and denoising, obtained solution have the wanted characteristics : sharp edges and shading in homogeneous areas. Next, in dynamic programming context, we reinterpreted the prior used in Semi-Global Matching (SGM) stereoscopic reconstruction method and in some of its variants. We proposed an additional parameter in order to modify the favored direction in the prior. At last, we proposed stability index of the solution in graph-cut and dynamic programing context. Using this index in a solution refinement step shows improvements

As escavações subterrâneas têm estado sempre afetadas pelas condições dos maciços, os quais apresentam diferentes características e comportamentos frente a deformações ocasionadas após a escavação de uma galeria ou túnel. Em virtude disto, torna-se necessário uma análise dos parâmetros que permitam estabelecer as condições de segurança do túnel e propõe-se o uso de um sistema estéreo de rastreamento ótico para analisar o deslocamento de artefatos (conjunto de marcadores) instalados sobre o contorno das faces do túnel, a fim de detectarem-se deslocamentos do maciço em pontos estabelecidos com precisão menor a ±1 mm. O sistema proposto opera na região espectral do infravermelho, permitindo assim uma operação em ambientes com baixa luminosidade e com inserção de ruído ótico. Adicionalmente o trabalho inclui resultados de validação e desenvolvimento do sistema de visão estéreo baixo condições de dois cenários de diferente volume espacial, assim como o procedimento realizado para a calibração dos parâmetros intrínsecos e extrínsecos dos sensores do sistema (câmeras monocromáticas). Os resultados obtidos apresentam uma precisão de ±0,1881 mm para artefatos com três marcadores e uma precisão de ±0,4952 para artefatos com quatro marcadores. Finalmente, o sistema opera com uma sensibilidade de 0,99 mm na determinação da posição de um artefato associado a um ponto do perfil do túnel, e desta forma, podem-se definir as condições de estabilidade do maciço rochoso e os requisitos para auxilio as decisões de métodos de tratamento e aumento da segurança. / Underground excavations have always been affected by the conditions of mass, which have different characteristics and behaviors against deformation caused after excavation of a gallery or tunnel. Because of this, it is necessary an analysis of the parameters needed to establish the safety conditions of the tunnel and it is proposed the use of a stereo optical tracking to analyze the displacement of artifacts (set of markers) installed on the contour of tunnels’ faces in order to detect displacements of the mass points established with a precision lower than ±1 mm. This system operates in the infrared spectral region to take advantage of the low light and reduce the optical noise in environments with grace. Further work includes validation results and development of low stereo vision conditions of two different scenarios spatial volume, as well as the procedure carried out to calibrate the intrinsic and extrinsic parameters of the system sensors (monochrome camera) system. The results show an accuracy of ± 0,1881 mm for artifacts with three markers and an accuracy of ± 0,4952 for artifacts with four markers. Finally, the system operates with a sensitivity of 0,99 mm in the determination of an artifact associated with a point in the tunnel profile position, and thus, one can define the conditions for stability of the rock mass and assistance requirements for the decisions of treatment methods and increased.

In this thesis we have studied detection of 3D structures having a forward camera movement which has strong influence of translation along the optical axis of the camera. During the forward movement the camera might undergoes rotation and translation .We have used “Plane plus Parallax” algorithm to cancel out this unwanted rotation. The input to the algorithm is a sequence of frames aligned with respect to a certain planar surface. The algorithm gives three types of outputs. (i) Dense correspondence across all frames. (ii) Dense 3D structure relative to the planar surface. (ii) Focus of Expansion (FOE) in all frames with respect to reference frame. Camera calibration is not needed for this algorithm. We have applied this algorithm to real world images and synthetic images. In both cases the 3D structure information could be obtained clearly even for objects far from the reference plane. Our result shows the potential of the method in 3D reconstruction implementing ego-motion of a single camera. / dm_aiub@yahoo.com 008801681006314

This thesis describes how a pipeline was obtained to reconstruct an urban environment from terrestrial laser scanning and photogrammetric 3D maps of Norrköping, visualized in first prison and real-time. Together with LIU University and the city planning office of Norrköping the project was carried out as a preliminary study to get an idea of how much work is needed and in what accuracy we can recreate a few buildings. The visualization is intended to demonstrate a new way of exploring the city in virtual reality as well as visualize the geometrical and textural details in a higher quality comparing to the 3D map that Municipality of Norrköping uses today. Before, the map has only been intended to be displayed from a bird’s eye view and has poor resolution from closer ranges. In order to improve the resolution, HDR photos were used to texture the laser scanned model and cover a particular area of the low res 3D map. This thesis will explain which method was used to process a point based environment for texturing and setting up an environment in Unreal using both the 3d map and the laser scanned model.

Le pilotage de véhicule blindé est rendu difficile par la faible visibilité offerte aux pilotes face aux environnements et aux situations complexes qu’ils doivent traverser.La protection des opérateurs de véhicules militaires et l’intégrité de ces véhicules sont des besoins primordiaux pour l’armée de terre.Afin de répondre à la problématique : sécuriser le pilotage des véhicules militaires avec comme périmètre la définition d’un système de perception d’environnement, nous avons procédé à l’étude au sens large de l’aide au pilotage dans le contexte militaire en environnement naturel et semi-structuré afin de mettre en exergue les moyens et les capteurs utilisables pour réaliser un système d’aide au pilotage.Ainsi, nous offrons une réponse technique pour la réalisation d’un tel système au travers premièrement d’une étude des méthodes et algorithmes existants applicables à notre cas d’application. Ensuite nous définissons les capteurs utilisables avec de telles méthodes. De cet état de l’art, nous définissonsune système répondant à notre problématique et nous expliquons sa mise en pratique au travers de la création d’une plateforme d’expérimentation.Cette plateforme se compose des solutions présentées et permet de valider le concept par l’évaluation des solutions d’acquisition de l’environnement afin d’offrir les données nécessaires à une aide au pilotage.Puis, l’étude des moyens d’analyse de cet environnement offre des pistes de réflexion sur le futur système d’aide au pilotage.Enfin, une l’étude d’un moyen alternatif de restitution de l’information à l’opérateur complète la solution présentée en offrant une piste de réflexion sur l’impact de la restitution dans les performances des opérateurs. / Armored vehicule driving is difficult because of low visibility given to pilots in tough environnements conditions and complex situations they have to manage.Soldiers safety and vehicle integrity are part of main topics for French “Armée de Terre”. To answer the problem Make the driving of military vehicles safer by improving environnement perception through driver asssistance systems, we study driving assistance in unstructured environnemnt by looking for sensors and methods which are suitable to realize such a system.First, we study existing methods and algorithms which fit our application case. Conclusion of this study is the definition of our system.Second, thanks to the previous study we explain the creation of an experimentation platform allowing evaluation of our concept. Data obtained from reconstruction are then exploited through environment analysis to bring obstacle extraction methods.Third, study of an alternative display solution is exposed and complete this work in explaining impact of restitution in operating cycle.

Several automatic methods exist for creating 3D point clouds extracted from 2D photos. In manycases, the result is a sparse point cloud, unevenly distributed over the scene.After determining the coordinates of the same point in two images of an object, the 3D positionof that point can be calculated using knowledge of camera data and relative orientation. A model created from a unevenly distributed point clouds may loss detail and precision in thesparse areas. The aim of this thesis is to study methods for densification of point clouds. This thesis contains a literature study over different methods for extracting matched point pairs,and an implementation of Least Square Template Matching (LSTM) with a set of improvementtechniques. The implementation is evaluated on a set of different scenes of various difficulty. LSTM is implemented by working on a dense grid of points in an image and Wallis filtering isused to enhance contrast. The matched point correspondences are evaluated with parameters fromthe optimization in order to keep good matches and discard bad ones. The purpose is to find detailsclose to a plane in the images, or on plane-like surfaces. A set of extensions to LSTM is implemented in the aim of improving the quality of the matchedpoints. The seed points are improved by Transformed Normalized Cross Correlation (TNCC) andMultiple Seed Points (MSP) for the same template, and then tested to see if they converge to thesame result. Wallis filtering is used to increase the contrast in the image. The quality of the extractedpoints are evaluated with respect to correlation with other optimization parameters and comparisonof standard deviation in x- and y- direction. If a point is rejected, the option to try again with a largertemplate size exists, called Adaptive Template Size (ATS).

Depth information of tissue surfaces and laparoscope poses are crucial for accurate surgical guidance and navigation in Computer Assisted Surgeries (CAS). Intra-operative Three Dimensional (3D) reconstruction and laparoscope localization are therefore two fundamental tasks in CAS. This dissertation focuses on the abdominal Minimally Invasive Surgeries (MIS) and presents laparoscopic-video-based methods for these two tasks. Different kinds of methods have been presented to recover 3D surface structures of surgical scenes in MIS. Those methods are mainly based on laser, structured light, time-of-flight cameras, and video cameras. Among them, laparoscopic-video-based surface reconstruction techniques have many significant advantages. Specifically, they are non-invasive, provide intra-operative information, and do not introduce extra-hardware to the current surgical platform. On the other side, laparoscopic-video-based 3D reconstruction and laparoscope localization are challenging tasks due to the specialties of the abdominal imaging environment. The well-known difficulties include: low texture, homogeneous areas, tissue deformations, and so on. The goal of this dissertation is to design novel 3D reconstruction and laparoscope localization methods and overcome those challenges from the abdominal imaging environment. Two novel methods are proposed to achieve accurate 3D reconstruction for MIS. The first method is based on the detection of distinctive image features, which is difficult in MIS images due to the low-texture and homogeneous tissue surfaces. To overcome this problem, this dissertation first introduces new types of image features for MIS images based on blood vessels on tissue surfaces and designs novel methods to efficiently detect them. After vessel features have been detected, novel methods are presented to match them in stereo images and 3D vessels can be recovered for each frame. Those 3D vessels from different views are integrated together to obtain a global 3D vessel network and Poisson reconstruction is applied to achieve large-area dense surface reconstruction. The second method is texture-independent and does not rely on the detection of image features. Instead, it proposes to mount a single-point light source on the abdominal wall. Shadows are cast on tissue surfaces when surgical instruments are waving in front of the light. Shadow boundaries are detected and matched in stereo images to recover the depth information. The recovered 3D shadow curves are interpolated to achieve dense reconstruction of tissue surfaces. One novel stereoscope localization method is designed specifically for the abdominal environment. The method relies on RANdom SAmple Consensus (RANSAC) to differentiate rigid points and deforming points. Since no assumption is made on the tissue deformations, the proposed methods is able to handle general tissue deformations and achieve accurate laparoscope localization results in the abdominal MIS environment. With the stereoscope localization results and the large-area dense surface reconstruction, a new scene visualization system, periphery augmented system, is designed to augment the peripheral areas of the original video so that surgeons can have a larger field of view. A user-evaluation system is designed to compare the periphery augmented system with the original MIS video. 30 subjects including 4 surgeons specialized in abdominal MIS participate the evaluation and a numerical measure is defined to represent their understanding of surgical scenes. T-test is performed on the numerical errors and the null hypothesis that the periphery augmented system and the original video have the same mean of errors is rejected. In other words, the results validate that the periphery augmented system improves users' understanding and awareness of surgical scenes.

Cette thèse contribue au problème de la reconstruction 3D pour les surfaces déformables avec une seule caméra. Afin de modéliser la déformation de la surface, nous considérons l’isométrie puisque de nombreuses déformations d’objets réels sont quasi-isométriques. L’isométrie implique que, lors de sa déformation, la surface ne peut pas être étirée ou compressée. Nous étudions deux problèmes. Le premier est le problème basé sur une modèle 3D de référence et une seule image. L’état de l’art propose une méthode locale et analytique de calcul direct de profondeur sous l’hypothèse d’isométrie. Dans cette méthode, la solution pour le gradient de la profondeur n’est pas utilisée. Nous prouvons que cette méthode s’avère instable lorsque la géométrie de la caméra tend à être affine. Nous fournissons des méthodes alternatives basées sur les solutions analytiques locales des quantités de premier ordre, telles que les gradients de profondeur ou les normales de la surface. Nos méthodes sont stables dans toutes les géométries de projection. Dans le deuxième type de problème de reconstruction sans modèle 3D de référence, on obtient les formes de l’objet à partir d’un ensemble d’images où il apparaît déformé. Nous fournissons des solutions locales et globales basées sur le modéle de la caméra perspective. Dans la méthode locale ou par point, nous résolvons pour la normale de la surface en chaque point en supposant que la surface est infinitésimalement plane. Nous calculons ensuite la surface par intégration. Dans la méthode globale, nous trouvons une relaxation convexe du problème. Celle-ci est basée sur la relaxation de l’isométrie en contrainte d’inextensibilité et sur la maximisation de la profondeur en chaque point de la surface. Cette solution combine toutes les contraintes en un seul programme d’optimisation convexe qui calcule la profondeur et utilise une représentation éparse de la surface. Nous détaillons les expériences approfondies qui ont été réalisées pour démontrer l’efficacité de chacune des méthodes. Les expériences montrent que notre solution libre de modèle de référence local fonctionne mieux que la plupart des méthodes précédentes. Notre méthode local avec un modèle 3D de référence et notre méthode globale sans modèle 3D apportent de meilleurs résultats que les méthodes de l’état de l’art en étant robuste au bruit de la correspondance. En particulier, nous sommes en mesure de reconstruire des déformations complexes, non-lisses et d’articulations avec la seconde méthode; alors qu’avec la première, nous pouvons reconstruire avec précision de déformations larges à partir d’images prises avec des très longues focales. / This thesis contributes to the problem of 3D reconstruction for deformable surfaces using a single camera. In order to model surface deformation, we use the isometric prior because many real object deformations are near-isometric. Isometry implies that the surface cannot stretch or compress. We tackle two different problems. The first is called Shape-from-Template where the object’s deformed shape is computed from a single image and a texture-mapped 3D template of the object surface. Previous methods propose a differential model of the problem and compute the local analytic solutions. In the methods the solution related to the depth-gradient is discarded and only the depth solution is used. We demonstrate that the depth solution lacks stability as the projection geometry tends to affine. We provide alternative methods based on the local analytic solutions of first-order quantities, such as the depth-gradient or surface normals. Our methods are stable in all projection geometries. The second type of problem, called Non-Rigid Shape-from-Motion is the more general templatefree reconstruction scenario. In this case one obtains the object’s shapes from a set of images where it appears deformed. We contribute to this problem for both local and global solutions using the perspective camera. In the local or point-wise method, we solve for the surface normal at each point assuming infinitesimal planarity of the surface. We then compute the surface by integration. In the global method we find a convex relaxation of the problem. This is based on relaxing isometry to inextensibility and maximizing the surface’s average depth. This solution combines all constraints into a single convex optimization program to compute depth and works for a sparse point representation of the surface. We detail the extensive experiments that were used to demonstrate the effectiveness of each of the proposed methods. The experiments show that our local template-free solution performs better than most of the previous methods. Our local template-based method and our global template-free method performs better than the state-of-the-art methods with robustness to correspondence noise. In particular, we are able to reconstruct difficult, non-smooth and articulating deformations with the latter; while with the former we can accurately reconstruct large deformations with images taken at very long focal lengths.

Three-Dimensional (3-D) imaging is vital in computer-assisted surgical planning including minimal invasive surgery, targeted drug delivery, and tumor resection. Selective Internal Radiation Therapy (SIRT) is a liver directed radiation therapy for the treatment of liver cancer. Accurate calculation of anatomical liver and tumor volumes are essential for the determination of the tumor to normal liver ratio and for the calculation of the dose of Y-90 microspheres that will result in high concentration of the radiation in the tumor region as compared to nearby healthy tissue. Present manual techniques for segmentation of the liver from Computed Tomography (CT) tend to be tedious and greatly dependent on the skill of the technician/doctor performing the task. This dissertation presents the development and implementation of a fully integrated algorithm for 3-D liver and tumor segmentation from tri-phase CT that yield highly accurate estimations of the respective volumes of the liver and tumor(s). The algorithm as designed requires minimal human intervention without compromising the accuracy of the segmentation results. Embedded within this algorithm is an effective method for extracting blood vessels that feed the tumor(s) in order to plan effectively the appropriate treatment. Segmentation of the liver led to an accuracy in excess of 95% in estimating liver volumes in 20 datasets in comparison to the manual gold standard volumes. In a similar comparison, tumor segmentation exhibited an accuracy of 86% in estimating tumor(s) volume(s). Qualitative results of the blood vessel segmentation algorithm demonstrated the effectiveness of the algorithm in extracting and rendering the vasculature structure of the liver. Results of the parallel computing process, using a single workstation, showed a 78% gain. Also, statistical analysis carried out to determine if the manual initialization has any impact on the accuracy showed user initialization independence in the results. The dissertation thus provides a complete 3-D solution towards liver cancer treatment planning with the opportunity to extract, visualize and quantify the needed statistics for liver cancer treatment. Since SIRT requires highly accurate calculation of the liver and tumor volumes, this new method provides an effective and computationally efficient process required of such challenging clinical requirements.

This thesis presents a 3D reconstruction approach to the detection of static obstacles from a single rear view parking camera. Corner features are tracked to estimate the vehicle’s motion and to perform multiview triangulation in order to reconstruct the scene. We model the camera motion as planar motion and use the knowledge of the camera pose to efficiently solve motion parameters. Based on the observed motion, we selected snapshots from which the scene is reconstructed. These snapshots guarantee a sufficient baseline between the images and result in more robust scene modeling. Multiview triangulation of a feature is performed only if the feature obeys the epipolar constraint. Triangulated features are semantically labelled according to their 3D location. Obstacle features are spatially clustered to reduce false detections. Finally, the distance to the nearest obstacle cluster is reported to the driver.