No description available.

[Truncated abstract] The aim of visual recognition is to identify objects in a scene and estimate their pose. Object recognition from 2D images is sensitive to illumination, pose, clutter and occlusions. Object recognition from range data on the other hand does not suffer from these limitations. An important paradigm of recognition is model-based whereby 3D models of objects are constructed offline and saved in a database, using a suitable representation. During online recognition, a similar representation of a scene is matched with the database for recognizing objects present in the scene . . . The tensor representation is extended to automatic and pose invariant 3D face recognition. As the face is a non-rigid object, expressions can significantly change its 3D shape. Therefore, the last part of this thesis investigates representations and matching techniques for automatic 3D face recognition which are robust to facial expressions. A number of novelties are proposed in this area along with their extensive experimental validation using the largest available 3D face database. These novelties include a region-based matching algorithm for 3D face recognition, a 2D and 3D multimodal hybrid face recognition algorithm, fully automatic 3D nose ridge detection, fully automatic normalization of 3D and 2D faces, a low cost rejection classifier based on a novel Spherical Face Representation, and finally, automatic segmentation of the expression insensitive regions of a face.

Dans cette thèse nous proposons des algorithmes pour le suivi 3D du mouvement des objects déformables à partir de plusieurs caméras vidéo. Bien qu’une suite de reconstructions tridimensionnelles peut être obtenue par des méthodes de reconstruction statique, celle-ci ne représente pas le mouvement. Nous voulons produire une représentation temporellement cohérente de la suite de formes prises par l’object. Précisément, nous souhaitons représenter l’objet par une surface maillée 3D dont les sommets se déplacent au cours du temps mais dont la topologie reste identique.Contrairement à beaucoup d’approches existantes, nous proposons de représenter le mouvement du volume intérieur des formes, dans le but de mieux représenter la nature volumétrique des objets. Nous traitons de manière volumétrique les problèmes fondamentaux du suivi déformable que sont l’association d’éléments semblables entre deux formes et la modélisation de la déformation. En particulier, nous adaptons au formes volumétriques les modèles d’association EM-ICP non-rigide ansi que l’association par détection par apprentissage automatique.D’autre part, nous abordons la question de la modélisation de l’évolution temporelle de la déformation au cours d’une séquence dans le but de mieux contraindre le problème du suivi temporel. Pour cela, nous modélisons un espace de forme construit autour de propriétés de déformations locales que nous apprenons automatiqument lors du suivi.Nous validons nos algorithmes de suivi sur des séquences vidéo multi-caméras avec vérité terrain (silhouettes et suivi par marqueurs). Nos résultats se révèlent meilleurs ou équivalents à ceux obtenus avec les méthodes de l’état de l’art.Enfin, nous démontrons que le suivi volumétrique et la représentation que nous avons choisie permettent de produire des animations 3D qui combinent l’acquisition et la simulation de mouvement. / In this thesis we propose algorithms for tracking 3D deformable shapes in motion from multiview video. Although series of reconstructed 3D shapes can be obtained by applying a static reconstruction algorithm to each temporal frame independently, such series do not represent motion. Instead, we want to provide a temporally coherent representation of the sequence of shapes resulting from temporal evolutions of a shape. Precisely, we want to represent the observed shape sequence as a 3D surface mesh whose vertices move in time but whose topology is constant.In contrast with most existing approaches, we propose to represent the motion of inner shape volumes, with the aim of better accounting for the volumetric nature of the observed object. We provide a fully volumetric approach to the fundamental problems of deformable shape tracking, which are the association between corresponding shape elements and the deformation model. In particular, we extend to a volumetric shape representation the EM-ICP tracking framework and the association-by-detection strategy.Furthermore, in order to better constrain the shape tracking problem, we propose a model for the temporal evolution of deformation. Our deformation model defines a shape space parametrized by variables that capture local deformation properties of the shape and whose values are automatically learned during the tracking process.We validate our tracking algorithms on several multiview video sequences with ground truth (silhouette and marker-based tracking). Our results are better or comparable to state of the art approaches.Finally, we show that volumetric tracking and the shape representation we choose can be leveraged for producing shape animations which combine captured and simulatated motion.

<p>Digital fringe projection (DFP) methods are commonly used to obtain high-accuracy shape measurements of opaque, diffusely-reflective objects. While some objects may have constant texture across its surface, this is not true for all; many measured objects may have high-contrast texture caused by edges of dark- and light-colored sections of the object. In these high-contrast areas, a phase artifact has been consistently observed, which in turn creates a specific measurement error that is sometimes referred to as ``discontinuity-induced measurement artifacts" (DMA). Our study indicated that this error is most likely caused by camera defocusing, which produces a Gaussian point spread function (PSF) that is convoluted across every captured image, thus creating an phase artifact shaped like a Gaussian function. Based on this finding, this thesis outlines a method for removing this error via Gaussian curve fitting on the affected regions. These regions can be found by locating large spikes in the image intensity gradient, which directly correspond to the edge of the phase artifact, and then using a weighted least squared method to fit a Gaussian function to the affected area. We propose to use this error removal method in two ways: first, to remove errors on a checkerboard calibration target in order to increase calibration accuracy; and second, to directly remove errors in high-contrast areas in order to decrease shape measurement error. Experimental results demonstrate that the proposed method succeeds in decreasing calibration error for a checkerboard calibration target by as much as 12\%. Shape measurement experiments were not only conducted across simple, flat boards, but also more complex surfaces, such as that of a coffee mug. This thesis will show that this measurement error can be significantly decreased for both simple and complex surfaces.</p>

A multiple-sensor optical shape measurement system (SMS) based on the principle of white-light fringe projection has been developed and commercialised by Loughborough University and Phase Vision Ltd for over 10 years. The use of the temporal phase unwrapping technique allows precise and dense shape measurements of complex surfaces; and the photogrammetry-based calibration technique offers the ability to calibrate multiple sensors simultaneously in order to achieve 360° measurement coverage. Nevertheless, to enhance the applicability of the SMS in industrial environments, further developments are needed (i) to improve the calibration speed for quicker deployment, (ii) to broaden the application range from shape measurement to deformation field measurement, and (iii) to tackle practically-challenging surfaces of which specular components may disrupt the acquired data and result in spurious measurements. The calibration process typically requires manual positioning of an artefact (i.e., reference object) at many locations within the view of the sensors. This is not only timeconsuming but also complicated for an operator with average knowledge of metrology. This thesis introduces an automated artefact positioning system which enables automatic and optimised distribution of the artefacts, automatic prediction of their whereabouts to increase the artefact detection speed and robustness, and thereby greater overall calibration performance. This thesis also describes a novel technique that integrates the digital image correlation (DIC) technique into the present fringe projection SMS for the purpose of simultaneous shape and deformation field measurement. This combined technique offers three key advantages: (a) the ability to deal with geometrical discontinuities which are commonly present on mechanical surfaces and currently challenging to most deformation measurement methods, (b) the ability to measure 3D displacement fields with a basic single-camera single-projector SMS with no additional hardware components, and (c) the simple implementation on a multiple-sensor hardware platform to achieve complete coverage of large-scale and complex samples, with the resulting displacement fields automatically lying in a single global coordinate system. A displacement measurement accuracy of ≃ 1/12,000 of the measurement volume, which is comparable to that of an industry-standard DIC system, has been achieved. The applications of this novel technique to several structural tests of aircraft wing panels on-site at the research centre of Airbus UK in Filton are also presented. Mechanical components with shiny surface finish and complex geometry may introduce another challenge to present fringe projection techniques. In certain circumstances, multiple reflections of the projected fringes on an object surface may cause ambiguity in the phase estimation process and result in incorrect coordinate measurements. This thesis presents a new technique which adopts a Fourier domain ranging (FDR) method to correctly identifying multiple phase signals and enables unambiguous triangulation for a measured coordinate. Experiments of the new FDR technique on various types of surfaces have shown promising results as compared to the traditional phase unwrapping techniques.

<div>With robots expanding their applications in more and more scenarios, practical problems from different scenarios are challenging current 3D measurement techniques. For instance, infrastructure inspection robots need large-scale and high-spatial-resolution 3D data for crack and defect detection, medical robots need 3D data well registered with temperature information, and warehouse robots need multi-resolution 3D shape measurement to adapt to different tasks. In the past decades, a lot of progress has been made in improving the performance of 3D shape measurement methods. Yet, measurement scale and speed and the fusion of multiple modalities of 3D shape measurement techniques remain vital aspects to be improved for robots to have a more complete perception of the real scene. In this dissertation, we will focus on the digital fringe projection technique, which usually can achieve high-accuracy 3D data, and expand the capability of that technique to complicated robot applications by 1) extending the measurement scale, 2) registering with multi-modal information, and 3) improving the measurement speed of the digital fringe projection technique.</div><div><br></div><div>The measurement scale of the digital fringe projection technique mainly focused on a small scale, from several centimeters to tens of centimeters, due to the lack of a flexible and convenient calibration method for a large-scale digital fringe projection system. In this study, we first developed a flexible and convenient large-scale calibration method and then extended the measurement scale of the digital fringe projection technique to several meters. The meter scale is needed in many large-scale robot applications, including large infrastructure inspection. Our proposed method includes two steps: 1) accurately calibrate intrinsics (i.e., focal lengths and principal points) with a small calibration board at close range where both the camera and projector are out of focus, and 2) calibrate the extrinsic parameters (translation and rotation) from camera to projector with the assistance of a low-accuracy large-scale 3D sensor (e.g., Microsoft Kinect). The two-step strategy avoids fabricating a large and accurate calibration target, which is usually expensive and inconvenient for doing pose adjustments. With a small calibration board and a low-cost 3D sensor, we calibrated a large-scale 3D shape measurement system with a FOV of (1120 x 1900 x 1000) mm^3 and verified the correctness of our method.</div><div><br></div><div> Multi-modal information is required in applications such as medical robots, which may need both to capture the 3D geometry of objects and to monitor their temperature. To allow robots to have a more complete perception of the scene, we further developed a hardware system that can achieve real-time 3D geometry and temperature measurement. Specifically, we proposed a holistic approach to calibrate both a structured light system and a thermal camera under exactly the same world coordinate system, even though these two sensors do not share the same wavelength; and a computational framework to determine the sub-pixel corresponding temperature for each 3D point, as well as to discard those occluded points. Since the thermal 2D imaging and 3D visible imaging systems do not share the same spectrum of light, they can perform sensing simultaneously in real time. We developed a hardware system that achieved real-time 3D geometry and temperature measurement at 26Hz with 768 x 960 points per frame.</div><div><br></div><div> In dynamic applications, where the measured object or the 3D sensor could be in motion, the measurement speed will become an important factor to be considered. Previously, people projected additional fringe patterns for absolute phase unwrapping, which slowed down the measurement speed. To achieve higher measurement speed, we developed a method to unwrap a phase pixel by pixel by solely using geometric constraints of the structured light system without requiring additional image acquisition. Specifically, an artificial absolute phase map $\Phi_{min}$, at a given virtual depth plane $z = z_{min}$, is created from geometric constraints of the calibrated structured light system, such that the wrapped phase can be pixel-by-pixel unwrapped by referring to $\Phi_{min}$. Since $\Phi_{min}$ is defined in the projector space, the unwrapped phase obtained from this method is an absolute phase for each pixel. Experimental results demonstrate the success of this proposed novel absolute-phase unwrapping method. However, the geometric constraint-based phase unwrapping method using a virtual plane is constrained in a certain depth range. The depth range limitations cause difficulties in two measurement scenarios: measuring an object with larger depth variation, and measuring a dynamic object that could move beyond the depth range. To address the problem of depth limitation, we further propose to take advantage of an additional 3D scanner and use additional external information to extend the maximum measurement range of the pixel-wise phase unwrapping method. The additional 3D scanner can provide a more detailed reference phase map $\Phi_{ref}$ to assist us to do absolute phase unwrapping without the depth constraint. Experiments demonstrate that our method, assisted by an additional 3D scanner, can work for a large depth range, and the maximum speed of the low-cost 3D scanner is not necessarily an upper bound of the speed of the structured light system. Assisted by Kinect V2, our structured light system achieved 53Hz with a resolution 1600 x 1000 pixels when we measured dynamic objects that were moving in a large depth range.</div><div><br></div><div> In summary, we significantly advanced the 3D shape measurement technology for robots to have a more complete perception of the scene by enhancing the digital fringe projection technique in measurement scale (space domain), speed (time domain), and fusion with other modality information. This research can potentially enable robots to have a better understanding of the scene for more complicated tasks, and broadly impact many other academic studies and industrial practices.</div>

No description available.

Atrial fibrillation (AF) is the most common arrhythmia that leads to thrombus formation, mostly in the left atrial appendage (LAA). The current standard of stratifying stroke risk, based on the CHA2DS2-VASc score, does not consider LAA morphology, and the clinically accepted LAA morphology-based classification is highly subjective. The aim of this study was to determine whether LAA blood-borne particle residence time distribution and the proposed quantitative index of LAA 3D geometry can add independent information to the CHA2DS2-VASc score. Data were collected from 16 AF subjects. Subject-specific measurements included left atrial (LA) and LAA 3D geometry obtained by cardiac computed tomography, cardiac output, and heart rate.We quantified 3D LAA appearance in terms of a novel LAA appearance complexity index (LAA-ACI). We employed computational fluid dynamics analysis and a systems-based approach to quantify residence time distribution and associated calculated variable (LAA mean residence time, tm) in each subject. The LAA-ACI captured the subject-specific LAA 3D geometry in terms of a single number. LAA tm varied significantly within a given LAA morphology as defined by the current subjectivemethod and it was not simply a reflection of LAA geometry/appearance. In addition, LAA-ACI and LAA tm varied significantly for a given CHA2DS2-VASc score, indicating that these two indices of stasis are not simply a reflection of the subjects’ clinical status. We conclude that LAA-ACI and LAA tm add independent information to the CHA2DS2-VASc score about stasis risk and thereby can potentially enhance its ability to stratify stroke risk in AF patients.

The field of computer vision has long pursued the challenge of understanding the three-dimensional world. This endeavour is further fuelled by the increasing demand for technologies that rely on accurate perception of the 3D environment such as autonomous driving and augmented reality. However, the labelled data scarcity in the 3D domain continues to be a hindrance to extensive research and development. Semi-Supervised Learning is a valuable tool to overcome data scarcity yet most of the state-of-art methods are primarily developed and tested for two-dimensional vision problems. To address this challenge, there is a need to explore innovative approaches that can bridge the gap between 2D and 3D domains. In this work, we propose a technique that both leverages the existing abundance of two-dimensional data and makes the state-of-art semi-supervised learning methods directly applicable to 3D tasks. Multi-View Meta Pseudo Labelling (MV-MPL) combines one of the best-performing architectures in 3D shape recognition, Multi-View Convolutional Neural Networks, together with the state-of-art semi-supervised method, Meta Pseudo Labelling. To evaluate the performance of MV-MPL, comprehensive experiments are conducted on widely used shape recognition benchmarks ModelNet40, ShapeNetCore-v1, and ShapeNetCore-v2, as well as, Objaverse-LVIS. The results demonstrate that MV-MPL achieves competitive accuracy compared to fully supervised models, even when only \(10%\) of the labels are available. Furthermore, the study reveals that the object descriptors extracted from the MV-MPL model exhibit strong performance on shape retrieval tasks, indicating the effectiveness of the approach beyond classification objectives. Further analysis includes the evaluation of MV-MPL under more restrained scenarios, the enhancements to the view aggregation and pseudo-labelling processes; and the exploration of the potential of employing multi-views as augmentations for semi-supervised learning. / Forskningsområdet för datorseende har länge strävat efter utmaningen att förstå den tredimensionella världen. Denna strävan drivs ytterligare av den ökande efterfrågan på teknologier som är beroende av en korrekt uppfattning av den tredimensionella miljön, såsom autonom körning och förstärkt verklighet. Dock fortsätter bristen på märkt data inom det tredimensionella området att vara ett hinder för omfattande forskning och utveckling. Halv-vägledd lärning (semi-supervised learning) framträder som ett värdefullt verktyg för att övervinna bristen på data, ändå är de flesta av de mest avancerade semisupervised-metoderna primärt utvecklade och testade för tvådimensionella problem inom datorseende. För att möta denna utmaning krävs det att utforska innovativa tillvägagångssätt som kan överbrygga klyftan mellan 2D- och 3D-domänerna. I detta arbete föreslår vi en teknik som både utnyttjar den befintliga överflöd av tvådimensionella data och gör det möjligt att direkt tillämpa de mest avancerade semisupervised-lärandemetoderna på 3D-uppgifter. Multi-View Meta Pseudo Labelling (MV-MPL) kombinerar en av de bästa arkitekturerna för 3D-formigenkänning, Multi-View Convolutional Neural Networks, tillsammans med den mest avancerade semisupervised-metoden, Meta Pseudo Labelling. För att utvärdera prestandan hos MV-MPL genomförs omfattande experiment på väl använda uvärderingar för formigenkänning., ModelNet40, ShapeNetCore-v1 och ShapeNetCore-v2. Resultaten visar att MV-MPL uppnår konkurrenskraftig noggrannhet jämfört med helt vägledda modeller, även när endast \(10%\) av etiketterna är tillgängliga. Dessutom visar studien att objektbeskrivningarna som extraherats från MV-MPL-modellen uppvisar en stark prestanda i formåterhämtningsuppgifter, vilket indikerar effektiviteten hos tillvägagångssättet bortom klassificeringsmål. Vidare analys inkluderar utvärderingen av MV-MPL under mer begränsade scenarier, förbättringar av vyaggregerings- och pseudomärkningsprocesserna samt utforskning av potentialen att använda bilder från flera vinklar som en metod att få mer data för halv-vägledd lärande.

Analyse de la forme 3D est un sujet de recherche extrêmement actif dans les deux l'infographie et vision par ordinateur. Dans la vision par ordinateur, l'acquisition de formes et de modélisation 3D sont généralement le résultat du traitement des données complexes et des méthodes d'analyse de données. Il existe de nombreuses situations concrètes où une forme visuelle est modélisé par un nuage de points observés avec une variété de capteurs 2D et 3D. Contrairement aux données graphiques, les données sensorielles ne sont pas, dans le cas général, uniformément répartie sur toute la surface des objets observés et ils sont souvent corrompus par le bruit du capteur, les valeurs aberrantes, les propriétés de surface (diffusion, spécularités, couleur, etc), l'auto occlusions, les conditions d'éclairage variables. Par ailleurs, le même objet que l'on observe par différents capteurs, à partir de points de vue légèrement différents, ou à des moments différents cas peuvent donner la répartition des points tout à fait différentes, des niveaux de bruit et, plus particulièrement, les différences topologiques, par exemple, la fusion des mains. Dans cette thèse, nous présentons une représentation de multi-échelle des formes articulés et concevoir de nouvelles méthodes d'analyse de forme, en gardant à l'esprit les défis posés par les données de forme visuelle. En particulier, nous analysons en détail le cadre de diffusion de chaleur pour représentation multi-échelle de formes 3D et proposer des solutions pour la segmentation et d'enregistrement en utilisant les méthodes spectrales graphique et divers algorithmes d'apprentissage automatique, à savoir, le modèle de mélange gaussien (GMM) et le Espérance-Maximisation (EM). Nous présentons d'abord l'arrière-plan mathématique sur la géométrie différentielle et l'isomorphisme graphique suivie par l'introduction de la représentation spectrale de formes 3D articulés. Ensuite, nous présentons une nouvelle méthode non supervisée pour la segmentation de la forme 3D par l'analyse des vecteurs propres Laplacien de graphe. Nous décrivons ensuite une solution semi-supervisé pour la segmentation de forme basée sur un nouveau paradigme d'apprendre, d'aligner et de transférer. Ensuite, nous étendre la représentation de forme 3D à une configuration multi-échelle en décrivant le noyau de la chaleur cadre. Enfin, nous présentons une méthode d'appariement dense grâce à la représentation multi-échelle de la chaleur du noyau qui peut gérer les changements topologiques dans des formes visuelles et de conclure par une discussion détaillée et l'orientation future des travaux. / 3D shape analysis is an extremely active research topic in both computer graphics and computer vision. In computer vision, 3D shape acquisition and modeling are generally the result of complex data processing and data analysis methods. There are many practical situations where a visual shape is modeled by a point cloud observed with a variety of 2D and 3D sensors. Unlike the graphical data, the sensory data are not, in the general case, uniformly distributed across the surfaces of the observed objects and they are often corrupted by sensor noise, outliers, surface properties (scattering, specularities, color, etc.), self occlusions, varying lighting conditions. Moreover, the same object that is observed by different sensors, from slightly different viewpoints, or at different time instances may yield completely different point distributions, noise levels and, most notably, topological differences, e.g., merging of hands. In this thesis we outline single and multi-scale representation of articulated 3D shapes and devise new shape analysis methods, keeping in mind the challenges posed by visual shape data. In particular, we discuss in detail the heat diffusion framework for multi-scale shape representation and propose solutions for shape segmentation and dense shape registration using the spectral graph methods and various other machine learning algorithms, namely, the Gaussian Mixture Model (GMM) and the Expectation Maximization (EM). We first introduce the mathematical background on differential geometry and graph isomorphism followed by the introduction of pose-invariant spectral embedding representation of 3D articulated shapes. Next we present a novel unsupervised method for visual shape segmentation by analyzing the Laplacian eigenvectors. We then outline a semi-supervised solution for shape segmentation based upon a new learn, align and transfer paradigm. Next we extend the shape representation to a multi-scale setup by outlining the heat-kernel framework. Finally, we present a topologically-robust dense shape matching method using the multi-scale heat kernel representation and conclude with a detailed discussion and future direction of work.