Accurate motion capture of deformable objects from monocular video sequences is a challenging Computer Vision problem with immense applicability to domains ranging from virtual reality, animation to image guided surgery. Existing dense motion capture methods rely on expensive setups with multiple calibrated cameras,structured light, active markers or prior scene knowledge learned from a large 3D dataset. In this thesis, we propose an end-to-end pipeline for 3D reconstruction of deformable scenes from a monocular video sequence. Our method relies on a two step pipeline in which temporally consistent video registration is followed by a dense non-rigid structure from motion approach. We present a data-driven method to reconstruct non-rigid smooth surfaces densely, using only a single video as input, without the need for any prior models or shape templates. We focus on the well explored low-rank prior for deformable shape reconstruction and propose its convex relaxation to introduce the first variational energy minimisation approach to non-rigid structure from motion. To achieve realistic dense reconstruction of sparsely textured surfaces, we incorporate an edge preserving spatial smoothness prior into the low-rank factorisation framework and design a single variational energy to address the non-rigid structure from motion problem. We also discuss the importance of long-term 2D trajectories for several vision problems and explain how subspace constraints can be used to exploit the redundancy present in the motion of real scenes for dense video registration. To that end, we adopt a variational optimisation approach to design a robust multi-frame video registration algorithm that combines a robust subspace prior with a total variation spatial regulariser. Throughout this thesis, we advocate the use of GPU-portable and scalable energy minimisation algorithms to progress towards practical dense non-rigid 3D motion capture from a single video in the presence of occlusions and illumination changes.

Componentes metálicos em operação podem estar sujeitos a diversas condições de operação deletérias. Visando avaliar os riscos de operação para evitar falhas, a análise de integridade estrutural é uma ferramenta amplamente aplicada e requer constante evolução. Por utilizar dados de ensaios não destrutivos, essa ferramenta requer cada vez mais precisão, para aprimorar seus resultados e reduzir ao máximo as falhas de componentes em operação. Por apresentar menor tempo de inspeção e maior probabilidade de detecção, o Phased Array surge como alternativa às técnicas convencionais de ultrassom. Dentre as técnicas de Phased Array, o Total Focusing Method (TFM) apresenta um dos resultados mais promissores, apresentando grande vantagem sobre técnicas convencionais de ultrassom e de Phased Array devido a sua melhor precisão e fácil interpretação dos resultados. Neste trabalho, o TFM foi aplicado em diversos blocos contendo descontinuidades usinadas, as quais simulam diferentes tipos de defeitos. O foco do TFM foi gerar blocos em três dimensões (3D) dos defeitos, facilitando a interpretação dos resultados, além da possibilidade da inserção desses blocos em softwares de análise de integridade estrutural, melhorando a precisão dos resultados. O TFM foi aplicado através do tratamento de dados de um sensor linear de Phased Array, com frequência de 5 MHz e com 64 elementos ativos. Nesse trabalho, foram desenvolvidas e aplicadas novas etapas no algoritmo do TFM para melhorar a precisão dos resultados, como a compensação da perda de energia e o half-skip. As etapas adicionais aplicadas ao algoritmo do TFM geraram bons resultados para entalhe e furos usinados nos blocos de aço carbono. Além dos blocos com defeitos usinados, foi realizada a inspeção de um bloco de aço inoxidável austenítico soldado. Para o bloco soldado, os resultados não foram satisfatórios, não sendo possível detectar defeitos existentes. Após a aplicação do TFM, foi simulado o deslocamento do sensor ao longo da peça, gerando diversas imagens 2D, as quais foram conectadas via isosuperfícies, gerando sólidos 3D dos defeitos presentes em cada bloco. Por fim, esses blocos foram exportados para um software CAD, apresentando excelente correspondência. / Metallic components in operation can be submitted to several harming operation conditions. Aiming to evaluate the operational risks to avoid failure, the structural integrity analysis is a widely applied tool and requires constant evolution. As it uses non-destructive testing data, this tool requires increasingly precision, to enhance its results and reduce the majority of the operating components failures. Because of the shorter inspection time and greater probability of detection, Phased Array is an alternative to conventional ultrasonic techniques. Among the techniques of Phased Array, the Total Focusing Method (TFM) presents one of the most promising results, offering great advantage over conventional ultrasonic and Phased Array techniques due to its better precision and easy interpretation of results. In this work, the TFM was applied to many blocks, which contain machined discontinuities simulating several defects. The objective of applying the TFM was to generate three-dimensional (3D) blocks, turning the interpretation of the results easier and giving the possibility to insert these blocks into a structural integrity analysis software, enhancing the results precision. The TFM algorithm was applied through treatment of data from a 5 MHz linear Phased Array sensor with 64 active elements. In this work, it were developed and applied new steps in the TFM algorithm, as energy loss compensation and half-skip interaction, to improve its precision. The additional steps applied to the TFM algorithm achieved great results for notch and holes machined in carbon steel blocks. In addition to the blocks with machined defects, a welded austenitic stainless steel block was inspected. For the welded block, the results were not satisfactory and it was not possible to detect the present defects. After the application of the TFM, the displacement of the transducer along the block was simulated, generating several 2D images, which were connected via isosurfaces, generating 3D solids corresponding to the defects present in each block. To conclude, these blocks were exported to a CAD software, presenting excellent matching.

Ce travail concerne la reconstruction 3D de vaisseaux sanguins à partir de coupes transversales en nombre éventuellement réduit. Si des données sont manquantes, une reconstruction cohérente avec un réseau de vaisseaux est obtenue. Cette approche permet en outre de limiter les interventions humaines lors du traitement des images des coupes transversales 2D. Sachant que les images utilisées sont obtenues par scanner,la difficulté est de connecter les vaisseaux sanguins entre deux coupes espacées pour obtenir un graphe qui correspond au cœur des vaisseaux. En associant les vaisseaux sanguins sur les coupes à des masses à transporter, on construit un graphe solution d’un problème de transport ramifié. La reconstruction 3D de la géométrie résulte des données 2D d’imagerie issues des différentes coupes transversales et du graphe. La géométrie 3D des vaisseaux sanguins est représentée par la donnée d’une fonction Level Set définie en tout point de l’espace dont l’iso-valeur zéro correspond aux parois des vaisseaux. On s’intéresse ensuite à résoudre numériquement le modèle de Navier-Stokes en écoulement incompressible sur un maillage cartésien inclus dans la géométrie reconstruite. Ce choix est motivé par la rapidité d’assemblage du maillage et des opérateurs discrets de dérivation, en vue d’éventuelles déformation des vaisseaux. L’inadaptation du maillage avec l’interface de la géométrie amène à considérer une condition limite modifiée permettant un calcul consistant des contraintes aux parois. / This work concerns the 3D reconstruction of blood vessels from a limited number of 2D transversal cuts obtained from scanners. If data are missing, a coherentreconstruction with a vessel network is obtained. This approach allows to limit human interventions in processing images of 2D transversal cuts. Knowing that the images used are obtained by scanner, the difficulty is to connect the blood vessels between some widely spaced cuts in order to produce the graph corresponding to the network of vessels. We identify the vessels on each trnasversal cut as a mass to be transported, we construct a graph solution of a branched transport problem. At this stage, we are able to reconstruct the 3D geometry by using the 2D Level Set Functions given by the transversal cuts and the graph information. The 3D geometry of blood vessels is represented by the data of the Level Set function defined at any point of the space whose 0-level corresponds to the vessel walls. The resulting geometry is usually integrated in a fluid mechanic code solving the incompressible Navier-Stokes equations on a Cartesian grid strictly included in a reconstructed geometry. The inadequacy of the mesh with the interface of the geometry is overcomed thanks to a modified boundary condition leading to an accurate computation of the constraints to the walls.

The structure and dynamics of the flow field created by a plunging flat plate airfoil are investigated at a chord Reynolds number of 10,000 while varying plunge amplitude and Strouhal number. Digital particle image velocimetry measurements are used to characterize the shedding patterns and the interactions between the leading and trailing edge vortex structures (LEV and TEV), resulting in the development of a wake classification system based on the nature and timing of interactions between the leading- and trailing-edge vortices. The convection speed of the LEV and its resulting interaction with the TEV is primarily dependent on reduced frequency; however, at Strouhal numbers above approximately 0.4, a significant influence of Strouhal number (or plunge amplitude) is observed in which LEV convection is retarded, and the contribution of the LEV to the wake is diminished. It is shown that this effect is caused by an enhanced interaction between the LEV and the airfoil surface, due to a significant increase in the strength of the vortices in this Strouhal number range, for all plunge amplitudes investigated. Comparison with low-Reynolds-number studies of plunging airfoil aerodynamics reveals a high degree of consistency and suggests applicability of the classification system beyond the range examined in the present work. Some important differences are also observed. The three-dimensional flow field was characterized for a plunging two-dimensional flat-plate airfoil using three-dimensional reconstructions of planar PIV data. Whereas the phase-averaged description of the flow field shows the secondary vortex penetrating the leading-edge shear layer to terminate LEV formation on the airfoil, time-resolved, instantaneous PIV measurements show a continuous and growing entrainment of secondary vorticity into the shear layer and LEV. A planar control volume analysis on the airfoil indicated that the generation of secondary vorticity produced approximately one half the circulation, in magnitude, as the leading-edge shear layer flux. A small but non-negligible vorticity source was also attributed to spanwise flow toward the end of the downstroke. Preliminary measurements of the structure and dynamics of the leading-edge vortex (LEV) are also investigated for plunging finite-aspect-ratio wings at a chord Reynolds number of 10,000 while varying aspect ratio and root boundary condition. Stereoscopic particle image velocimetry (SPIV) measurements are used to characterize LEV dynamics and interactions with the plate in multiple chordwise planes. The relationship between the vorticity field and the spanwise flow field over the wing, and the influence of root boundary conditions on these quantities has been investigated. The viscous symmetry plane is found to influence this flow field, in comparison to other studies \cite{YiRo:2010,Vi:2011b,CaWaGuVi:2012}, by influencing tilting of the LEV near the symmetry wall, and introducing a corewise root-to-tip flow near the symmetry plane. Modifications in the root boundary conditions are found to significantly affect this. LEV circulations for the different aspect ratio plates are also compared. At the bottom of the downstroke, the maximum circulation is found at the middle of the semi-span in each case. The circulation of the $sAR=2$ wing is found to significantly exceed that of the $sAR=1$ wing and, surprisingly, the maximum circulation value is found to be independent of root boundary conditions for the $sAR=2$ case and also closely matched that of the quasi-2D case. Furthermore, the 3-D flow field of a finite wing of $sAR=2$ was characterized using three-dimensional reconstructions of planar PIV data after minimizing the gap between the plunging plate and the top stationary wall. The LEV on the finite wing rapidly evolved into an arch structure centered at approximately the 50\% spanwise position, similar to previous observations by Calderon et al. \cite{CaWaGu:2010}, and Yilmaz and Rockwell \cite{YiRo:2010}. At that location, the circulation contribution due to spanwise flow was approximately half that of the shear layer flux because of the significantly greater three-dimensionality in the flow. Increased tilting at the 25\% and 75\% spanwise locations suggests increasing three-dimensionality at those locations compared to the symmetry plane of the arch (50\% spanwise location). The deviation between the LEV circulation and integrated convective vorticity fluxes at the 50\% spanwise location suggests that entrainment of secondary vorticity plays a similar role in regulating LEV circulation as in the 2D case. While the wing surface flux of vorticity could not be measured in that case, the significant difference between LEV circulation and the known integrated fluxes is comparable to that for the 2D plate, suggesting that a significant boundary flux of secondary vorticity may exist.

This thesis outlines the development of a mathematical model which can be used to perform 3D reconstruction of a target object from surveillance images. 3D reconstruction is a common procedure in photogrammetry, but performing 3D reconstruction from surveillance images can be more difficult than typical photogrammetry applications. Surveillance images are generally captured in an unsystematic manner because there is no control over the target that is being photographed. Surveillance images can have a wide variety of fields of view, are often captured with uncalibrated cameras, and typically the targets are objects for which there is no other a priori information. For these reasons, performing 3D reconstruction from surveillance images may not be possible using standard photogrammetric methods, especially when the angular fields of view of the images are rather narrow.

Ce document constitue une synth`ese de travaux de recherche en vue de l'obten- tion du diplˆome d'habilitation `a diriger les recherches. A la suite ce cette in- troduction r ́edig ́ee en franc ̧ais, le reste de ce document sera en anglais. Je suis actuellement charg ́e de recherches INRIA au centre de Rennes Bretagne Atlantique. J'ai rejoint en Septembre 2001 l' ́equipe Vista dirig ́ee par Patrick Bouthemy, puis l' ́equipe Visages dirig ́ee par Christian Barillot en Janvier 2004. Depuis Janvier 2010, je travaille dans l' ́equipe-projet Serpico dirig ́ee par Charles Kervrann dont l'objet est l'imagerie et la mod ́elisation de la dynamique intra- cellulaire. Parmi mes activit ́es pass ́ees, ce document va se concentrer uniquement sur les activit ́es portant sur la neurochirurgie guid ́ee par l'image. En parti- culier, les travaux effectu ́es sur le recalage non-rigide ne seront pas pr ́esent ́es ici. Concernant le recalage, ces travaux ont commenc ́e pendant ma th`ese avec le d ́eveloppement d'une m ́ethode de recalage 3D bas ́e sur le flot optique [72], l'incorporation de contraintes locales dans ce processus de recalage [74] et la validation de m ́ethodes de recalage inter-sujets [71]. J'ai poursuivi ces travaux apr`es mon recrutement avec Anne Cuzol et Etienne M ́emin sur la mod ́elisation fluide du recalage [44], avec Nicolas Courty sur l'acc ́el ́eration temps-r ́eel de m ́ethode de recalage [42], et sur l' ́evaluation des m ́ethodes de recalage dans deux contextes : celui de l'implantation d' ́electrodes profondes [29] et le re- calage inter-sujets [92]. L'utilisation de syst`emes dits de neuronavigation est maintenant courante dans les services de neurochirurgie. Les b ́en ́efices, attendus ou report ́es dans la litt ́erature, sont une r ́eduction de la mortalit ́e et de la morbidit ́e, une am ́elio- ration de la pr ́ecision, une r ́eduction de la dur ́ee d'intervention, des couˆts d'hospitalisation. Tous ces b ́en ́efices ne sont pas `a l'heure actuelle d ́emontr ́es `a ma connaissance, mais cette question d ́epasse largement le cadre de ce doc- ument. Ces syst`emes de neuronavigation permettent l'utilisation du planning chirurgical pendant l'intervention, dans la mesure ou` le patient est mis en cor- respondance g ́eom ́etrique avec les images pr ́eop ́eratoires `a partir desquelles est pr ́epar ́ee l'intervention. Ces informations multimodales sont maintenant couramment utilis ́ees, com- prenant des informations anatomiques, vasculaires, fonctionnelles. La fusion de ces informations permet de pr ́eparer le geste chirurgical : ou` est la cible, quelle est la voie d'abord, quelles zones ́eviter. Ces informations peuvent main- tenant ˆetre utilis ́ees en salle d'op ́eration et visualis ́ees dans les oculaires du mi- croscope chirurgical grˆace au syst`eme de neuronavigation. Malheureusement, cela suppose qu'il existe une transformation rigide entre le patient et les im- ages pr ́eop ́eratoires. Alors que cela peut ˆetre consid ́er ́e comme exact avant l'intervention, cette hypoth`ese tombe rapidement sous l'effet de la d ́eformation des tissus mous. Ces d ́eformations, qui doivent ˆetre consid ́er ́ees comme un ph ́enom`ene spatio-temporel, interviennent sous l'effet de plusieurs facteurs, dont la gravit ́e, la perte de liquide c ́ephalo-rachidien, l'administration de pro- duits anesth ́esiants ou diur ́etiques, etc. Ces d ́eformations sont tr`es difficiles `a mod ́eliser et pr ́edire. De plus, il s'agit d'un ph ́enom`ene spatio-temporel, dont l'amplitude peut varier consid ́era- blement en fonction de plusieurs facteurs. Pour corriger ces d ́eformations, l'imagerie intra-op ́eratoire apparait comme la seule piste possible.

Three-dimensional models which contain both geometry and texture have numerous applications such as urban planning, physical simulation, and virtual environments. A major focus of computer vision (and recently graphics) research is the automatic recovery of three-dimensional models from two-dimensional images. After many years of research this goal is yet to be achieved. Most practical modeling systems require substantial human input and unlike automatic systems are not scalable. This thesis presents a novel method for automatically recovering dense surface patches using large sets (1000's) of calibrated images taken from arbitrary positions within the scene. Physical instruments, such as Global Positioning System (GPS), inertial sensors, and inclinometers, are used to estimate the position and orientation of each image. Essentially, the problem is to find corresponding points in each of the images. Once a correspondence has been established, calculating its three-dimensional position is simply a matter of geometry. Long baseline images improve the accuracy. Short baseline images and the large number of images greatly simplifies the correspondence problem. The initial stage of the algorithm is completely local and scales linearly with the number of images. Subsequent stages are global in nature, exploit geometric constraints, and scale quadratically with the complexity of the underlying scene. We describe techniques for: 1) detecting and localizing surface patches; 2) refining camera calibration estimates and rejecting false positive surfels; and 3) grouping surface patches into surfaces and growing the surface along a two-dimensional manifold. We also discuss a method for producing high quality, textured three-dimensional models from these surfaces. Some of the most important characteristics of this approach are that it: 1) uses and refines noisy calibration estimates; 2) compensates for large variations in illumination; 3) tolerates significant soft occlusion (e.g. tree branches); and 4) associates, at a fundamental level, an estimated normal (i.e. no frontal-planar assumption) and texture with each surface patch.

Reconstructing a 3-dimensional scene from a set of2-dimensional images is a fundamental problem in computervision. A system capable of performing this task can be used inmany applications in robotics, architecture, archaeology,biometrics, human computer interaction and the movie andentertainment industry. Most existing reconstruction approaches exploit one sourceof information to tackle the problem. This is the motion of thecamera, the 2D images are taken from different viewpoints. Weexploit an additional information source, the reference plane,which makes it possible to reconstruct difficult scenes whereother methods fail. A real scene plane may serve as thereference plane. Furthermore, there are many alternativetechniques to obtain virtual reference planes. For instance,orthogonal directions in the scene provide a virtual referenceplane, the plane at infinity, or images taken with a parallelprojection camera. A collection of known and novel referenceplane scenarios is presented in this thesis. The main contribution of the thesis is a novel multi-viewreconstruction approach using a reference plane. The techniqueis applicable to three different feature types, points, linesand planes. The novelty of our approach is that all cameras andall features (off the reference plane) are reconstructedsimultaneously from a single linear system of imagemeasurements. It is based on the novel observation that camerasand features have a linear relationship if a reference plane isknown. In the absence of a reference plane, this relationshipis non-linear. Thus many previousmethods must reconstructfeatures and cameras sequentially. Another class of methods,popular in the literature, is factorization, but, in contrastto our approach, this has the serious practical drawback thatall features are required to be visible in all views. Extensiveexperiments show that our approach is superior to allpreviously suggested reference plane and non-reference planemethods for difficult reference plane scenarios. Furthermore, the thesis studies scenes which do not have aunique reconstruction, so-called critical configurations. It isproven that in the presence of a reference plane the set ofcritical configurations is small. Finally, the thesis introduces a complete, automaticmulti-view reconstruction system based on the reference planeapproach. The input data is a set of images and the output a 3Dpoint reconstruction together with the correspondingcameras.

This thesis presents a 3D reconstruction approach to the detection of static obstacles from a single rear view parking camera. Corner features are tracked to estimate the vehicle’s motion and to perform multiview triangulation in order to reconstruct the scene. We model the camera motion as planar motion and use the knowledge of the camera pose to efficiently solve motion parameters. Based on the observed motion, we selected snapshots from which the scene is reconstructed. These snapshots guarantee a sufficient baseline between the images and result in more robust scene modeling. Multiview triangulation of a feature is performed only if the feature obeys the epipolar constraint. Triangulated features are semantically labelled according to their 3D location. Obstacle features are spatially clustered to reduce false detections. Finally, the distance to the nearest obstacle cluster is reported to the driver.

This thesis presents a 3D reconstruction approach to the detection of static obstacles from a single rear view parking camera. Corner features are tracked to estimate the vehicle’s motion and to perform multiview triangulation in order to reconstruct the scene. We model the camera motion as planar motion and use the knowledge of the camera pose to efficiently solve motion parameters. Based on the observed motion, we selected snapshots from which the scene is reconstructed. These snapshots guarantee a sufficient baseline between the images and result in more robust scene modeling. Multiview triangulation of a feature is performed only if the feature obeys the epipolar constraint. Triangulated features are semantically labelled according to their 3D location. Obstacle features are spatially clustered to reduce false detections. Finally, the distance to the nearest obstacle cluster is reported to the driver.