Ce travail concerne la reconstruction 3D de vaisseaux sanguins à partir de coupes transversales en nombre éventuellement réduit. Si des données sont manquantes, une reconstruction cohérente avec un réseau de vaisseaux est obtenue. Cette approche permet en outre de limiter les interventions humaines lors du traitement des images des coupes transversales 2D. Sachant que les images utilisées sont obtenues par scanner,la difficulté est de connecter les vaisseaux sanguins entre deux coupes espacées pour obtenir un graphe qui correspond au cœur des vaisseaux. En associant les vaisseaux sanguins sur les coupes à des masses à transporter, on construit un graphe solution d’un problème de transport ramifié. La reconstruction 3D de la géométrie résulte des données 2D d’imagerie issues des différentes coupes transversales et du graphe. La géométrie 3D des vaisseaux sanguins est représentée par la donnée d’une fonction Level Set définie en tout point de l’espace dont l’iso-valeur zéro correspond aux parois des vaisseaux. On s’intéresse ensuite à résoudre numériquement le modèle de Navier-Stokes en écoulement incompressible sur un maillage cartésien inclus dans la géométrie reconstruite. Ce choix est motivé par la rapidité d’assemblage du maillage et des opérateurs discrets de dérivation, en vue d’éventuelles déformation des vaisseaux. L’inadaptation du maillage avec l’interface de la géométrie amène à considérer une condition limite modifiée permettant un calcul consistant des contraintes aux parois. / This work concerns the 3D reconstruction of blood vessels from a limited number of 2D transversal cuts obtained from scanners. If data are missing, a coherentreconstruction with a vessel network is obtained. This approach allows to limit human interventions in processing images of 2D transversal cuts. Knowing that the images used are obtained by scanner, the difficulty is to connect the blood vessels between some widely spaced cuts in order to produce the graph corresponding to the network of vessels. We identify the vessels on each trnasversal cut as a mass to be transported, we construct a graph solution of a branched transport problem. At this stage, we are able to reconstruct the 3D geometry by using the 2D Level Set Functions given by the transversal cuts and the graph information. The 3D geometry of blood vessels is represented by the data of the Level Set function defined at any point of the space whose 0-level corresponds to the vessel walls. The resulting geometry is usually integrated in a fluid mechanic code solving the incompressible Navier-Stokes equations on a Cartesian grid strictly included in a reconstructed geometry. The inadequacy of the mesh with the interface of the geometry is overcomed thanks to a modified boundary condition leading to an accurate computation of the constraints to the walls.

This thesis outlines the development of a mathematical model which can be used to perform 3D reconstruction of a target object from surveillance images. 3D reconstruction is a common procedure in photogrammetry, but performing 3D reconstruction from surveillance images can be more difficult than typical photogrammetry applications. Surveillance images are generally captured in an unsystematic manner because there is no control over the target that is being photographed. Surveillance images can have a wide variety of fields of view, are often captured with uncalibrated cameras, and typically the targets are objects for which there is no other a priori information. For these reasons, performing 3D reconstruction from surveillance images may not be possible using standard photogrammetric methods, especially when the angular fields of view of the images are rather narrow.

Ce document constitue une synth`ese de travaux de recherche en vue de l'obten- tion du diplˆome d'habilitation `a diriger les recherches. A la suite ce cette in- troduction r ́edig ́ee en franc ̧ais, le reste de ce document sera en anglais. Je suis actuellement charg ́e de recherches INRIA au centre de Rennes Bretagne Atlantique. J'ai rejoint en Septembre 2001 l' ́equipe Vista dirig ́ee par Patrick Bouthemy, puis l' ́equipe Visages dirig ́ee par Christian Barillot en Janvier 2004. Depuis Janvier 2010, je travaille dans l' ́equipe-projet Serpico dirig ́ee par Charles Kervrann dont l'objet est l'imagerie et la mod ́elisation de la dynamique intra- cellulaire. Parmi mes activit ́es pass ́ees, ce document va se concentrer uniquement sur les activit ́es portant sur la neurochirurgie guid ́ee par l'image. En parti- culier, les travaux effectu ́es sur le recalage non-rigide ne seront pas pr ́esent ́es ici. Concernant le recalage, ces travaux ont commenc ́e pendant ma th`ese avec le d ́eveloppement d'une m ́ethode de recalage 3D bas ́e sur le flot optique [72], l'incorporation de contraintes locales dans ce processus de recalage [74] et la validation de m ́ethodes de recalage inter-sujets [71]. J'ai poursuivi ces travaux apr`es mon recrutement avec Anne Cuzol et Etienne M ́emin sur la mod ́elisation fluide du recalage [44], avec Nicolas Courty sur l'acc ́el ́eration temps-r ́eel de m ́ethode de recalage [42], et sur l' ́evaluation des m ́ethodes de recalage dans deux contextes : celui de l'implantation d' ́electrodes profondes [29] et le re- calage inter-sujets [92]. L'utilisation de syst`emes dits de neuronavigation est maintenant courante dans les services de neurochirurgie. Les b ́en ́efices, attendus ou report ́es dans la litt ́erature, sont une r ́eduction de la mortalit ́e et de la morbidit ́e, une am ́elio- ration de la pr ́ecision, une r ́eduction de la dur ́ee d'intervention, des couˆts d'hospitalisation. Tous ces b ́en ́efices ne sont pas `a l'heure actuelle d ́emontr ́es `a ma connaissance, mais cette question d ́epasse largement le cadre de ce doc- ument. Ces syst`emes de neuronavigation permettent l'utilisation du planning chirurgical pendant l'intervention, dans la mesure ou` le patient est mis en cor- respondance g ́eom ́etrique avec les images pr ́eop ́eratoires `a partir desquelles est pr ́epar ́ee l'intervention. Ces informations multimodales sont maintenant couramment utilis ́ees, com- prenant des informations anatomiques, vasculaires, fonctionnelles. La fusion de ces informations permet de pr ́eparer le geste chirurgical : ou` est la cible, quelle est la voie d'abord, quelles zones ́eviter. Ces informations peuvent main- tenant ˆetre utilis ́ees en salle d'op ́eration et visualis ́ees dans les oculaires du mi- croscope chirurgical grˆace au syst`eme de neuronavigation. Malheureusement, cela suppose qu'il existe une transformation rigide entre le patient et les im- ages pr ́eop ́eratoires. Alors que cela peut ˆetre consid ́er ́e comme exact avant l'intervention, cette hypoth`ese tombe rapidement sous l'effet de la d ́eformation des tissus mous. Ces d ́eformations, qui doivent ˆetre consid ́er ́ees comme un ph ́enom`ene spatio-temporel, interviennent sous l'effet de plusieurs facteurs, dont la gravit ́e, la perte de liquide c ́ephalo-rachidien, l'administration de pro- duits anesth ́esiants ou diur ́etiques, etc. Ces d ́eformations sont tr`es difficiles `a mod ́eliser et pr ́edire. De plus, il s'agit d'un ph ́enom`ene spatio-temporel, dont l'amplitude peut varier consid ́era- blement en fonction de plusieurs facteurs. Pour corriger ces d ́eformations, l'imagerie intra-op ́eratoire apparait comme la seule piste possible.

Three-dimensional models which contain both geometry and texture have numerous applications such as urban planning, physical simulation, and virtual environments. A major focus of computer vision (and recently graphics) research is the automatic recovery of three-dimensional models from two-dimensional images. After many years of research this goal is yet to be achieved. Most practical modeling systems require substantial human input and unlike automatic systems are not scalable. This thesis presents a novel method for automatically recovering dense surface patches using large sets (1000's) of calibrated images taken from arbitrary positions within the scene. Physical instruments, such as Global Positioning System (GPS), inertial sensors, and inclinometers, are used to estimate the position and orientation of each image. Essentially, the problem is to find corresponding points in each of the images. Once a correspondence has been established, calculating its three-dimensional position is simply a matter of geometry. Long baseline images improve the accuracy. Short baseline images and the large number of images greatly simplifies the correspondence problem. The initial stage of the algorithm is completely local and scales linearly with the number of images. Subsequent stages are global in nature, exploit geometric constraints, and scale quadratically with the complexity of the underlying scene. We describe techniques for: 1) detecting and localizing surface patches; 2) refining camera calibration estimates and rejecting false positive surfels; and 3) grouping surface patches into surfaces and growing the surface along a two-dimensional manifold. We also discuss a method for producing high quality, textured three-dimensional models from these surfaces. Some of the most important characteristics of this approach are that it: 1) uses and refines noisy calibration estimates; 2) compensates for large variations in illumination; 3) tolerates significant soft occlusion (e.g. tree branches); and 4) associates, at a fundamental level, an estimated normal (i.e. no frontal-planar assumption) and texture with each surface patch.

Reconstructing a 3-dimensional scene from a set of2-dimensional images is a fundamental problem in computervision. A system capable of performing this task can be used inmany applications in robotics, architecture, archaeology,biometrics, human computer interaction and the movie andentertainment industry. Most existing reconstruction approaches exploit one sourceof information to tackle the problem. This is the motion of thecamera, the 2D images are taken from different viewpoints. Weexploit an additional information source, the reference plane,which makes it possible to reconstruct difficult scenes whereother methods fail. A real scene plane may serve as thereference plane. Furthermore, there are many alternativetechniques to obtain virtual reference planes. For instance,orthogonal directions in the scene provide a virtual referenceplane, the plane at infinity, or images taken with a parallelprojection camera. A collection of known and novel referenceplane scenarios is presented in this thesis. The main contribution of the thesis is a novel multi-viewreconstruction approach using a reference plane. The techniqueis applicable to three different feature types, points, linesand planes. The novelty of our approach is that all cameras andall features (off the reference plane) are reconstructedsimultaneously from a single linear system of imagemeasurements. It is based on the novel observation that camerasand features have a linear relationship if a reference plane isknown. In the absence of a reference plane, this relationshipis non-linear. Thus many previousmethods must reconstructfeatures and cameras sequentially. Another class of methods,popular in the literature, is factorization, but, in contrastto our approach, this has the serious practical drawback thatall features are required to be visible in all views. Extensiveexperiments show that our approach is superior to allpreviously suggested reference plane and non-reference planemethods for difficult reference plane scenarios. Furthermore, the thesis studies scenes which do not have aunique reconstruction, so-called critical configurations. It isproven that in the presence of a reference plane the set ofcritical configurations is small. Finally, the thesis introduces a complete, automaticmulti-view reconstruction system based on the reference planeapproach. The input data is a set of images and the output a 3Dpoint reconstruction together with the correspondingcameras.

This thesis presents a 3D reconstruction approach to the detection of static obstacles from a single rear view parking camera. Corner features are tracked to estimate the vehicle’s motion and to perform multiview triangulation in order to reconstruct the scene. We model the camera motion as planar motion and use the knowledge of the camera pose to efficiently solve motion parameters. Based on the observed motion, we selected snapshots from which the scene is reconstructed. These snapshots guarantee a sufficient baseline between the images and result in more robust scene modeling. Multiview triangulation of a feature is performed only if the feature obeys the epipolar constraint. Triangulated features are semantically labelled according to their 3D location. Obstacle features are spatially clustered to reduce false detections. Finally, the distance to the nearest obstacle cluster is reported to the driver.

This thesis presents a 3D reconstruction approach to the detection of static obstacles from a single rear view parking camera. Corner features are tracked to estimate the vehicle’s motion and to perform multiview triangulation in order to reconstruct the scene. We model the camera motion as planar motion and use the knowledge of the camera pose to efficiently solve motion parameters. Based on the observed motion, we selected snapshots from which the scene is reconstructed. These snapshots guarantee a sufficient baseline between the images and result in more robust scene modeling. Multiview triangulation of a feature is performed only if the feature obeys the epipolar constraint. Triangulated features are semantically labelled according to their 3D location. Obstacle features are spatially clustered to reduce false detections. Finally, the distance to the nearest obstacle cluster is reported to the driver.

Today, 3D models of cities are created from aerial images using a camera rig. Images, together with sensor data from the flights, are stored for further processing when building 3D models. However, there is a market demand for a more mobile solution of satisfactory quality. If the camera position can be calculated for each image, there is an existing algorithm available for the creation of 3D models. This master thesis project aims to investigate whether the iPhone 4 offers good enough image and sensor data quality from which 3D models can be created. Calculations on movements and rotations from sensor data forms the foundation of the image processing, and should refine the camera position estimations. The 3D models are built only from image processing since sensor data cannot be used due to poor data accuracy. Because of that, the scaling of the 3D models are unknown and a measurement is needed on the real objects to make scaling possible. Compared to a test algorithm that calculates 3D models from only images, already available at the SBD’s system, the quality of the 3D model in this master thesis project is almost the same or, in some respects, even better when compared with the human eye.

Nowadays, 3D modelling of objects from multiple images is a topic that has gained great recognition and is widely used in various fields. Recently, lots of progress has been made in identification of people using 3D face models, which are usually reconstructed from multiple face images. In this thesis, a system including stereo cameras and structured light is built for the purpose of 3D modelling. The system outputs are 3D shapes of the face and also the texture information registered to this shape. Although the system in this thesis is developed for face reconstruction, it is not specific to faces. Using the same methodology proposed in this study 3D reconstruction of any object can be achieved.

<p>Reconstructing a 3-dimensional scene from a set of2-dimensional images is a fundamental problem in computervision. A system capable of performing this task can be used inmany applications in robotics, architecture, archaeology,biometrics, human computer interaction and the movie andentertainment industry.</p><p>Most existing reconstruction approaches exploit one sourceof information to tackle the problem. This is the motion of thecamera, the 2D images are taken from different viewpoints. Weexploit an additional information source, the reference plane,which makes it possible to reconstruct difficult scenes whereother methods fail. A real scene plane may serve as thereference plane. Furthermore, there are many alternativetechniques to obtain virtual reference planes. For instance,orthogonal directions in the scene provide a virtual referenceplane, the plane at infinity, or images taken with a parallelprojection camera. A collection of known and novel referenceplane scenarios is presented in this thesis.</p><p>The main contribution of the thesis is a novel multi-viewreconstruction approach using a reference plane. The techniqueis applicable to three different feature types, points, linesand planes. The novelty of our approach is that all cameras andall features (off the reference plane) are reconstructedsimultaneously from a single linear system of imagemeasurements. It is based on the novel observation that camerasand features have a linear relationship if a reference plane isknown. In the absence of a reference plane, this relationshipis non-linear. Thus many previousmethods must reconstructfeatures and cameras sequentially. Another class of methods,popular in the literature, is factorization, but, in contrastto our approach, this has the serious practical drawback thatall features are required to be visible in all views. Extensiveexperiments show that our approach is superior to allpreviously suggested reference plane and non-reference planemethods for difficult reference plane scenarios.</p><p>Furthermore, the thesis studies scenes which do not have aunique reconstruction, so-called critical configurations. It isproven that in the presence of a reference plane the set ofcritical configurations is small.</p><p>Finally, the thesis introduces a complete, automaticmulti-view reconstruction system based on the reference planeapproach. The input data is a set of images and the output a 3Dpoint reconstruction together with the correspondingcameras.</p>