Object-centric geometric perception aims at extracting the geometric attributes of 3D objects. These attributes include shape, pose, and motion of the target objects, which enable fine-grained object-level understanding for various tasks in graphics, computer vision, and robotics. With the growth of 3D geometry data and 3D deep learning methods, it becomes more and more likely to achieve such tasks directly using 3D input data. Among different 3D representations, a 3D point cloud is a simple, common, and memory-efficient representation that could be directly retrieved from multi-view images, depth scans, or LiDAR range images. Different challenges exist in achieving object-centric geometric perception, such as achieving a fine-grained geometric understanding of common articulated objects with multiple rigid parts, learning disentangled shape and pose representations with fewer labels, or tackling dynamic and sequential geometric input in an end-to-end fashion. Here we identify and solve these challenges from a 3D deep learning perspective by designing effective and generalizable 3D representations, architectures, and pipelines. We propose the first deep pose estimation for common articulated objects by designing a novel hierarchical invariant representation. To push the boundary of 6D pose estimation for common rigid objects, a simple yet effective self-supervised framework is designed to handle unlabeled partial segmented scans. We further contribute a novel 4D convolutional neural network called PointMotionNet to learn spatio-temporal features for 3D point cloud sequences. All these works advance the domain of object-centric geometric perception from a unique 3D deep learning perspective. / Doctor of Philosophy / 3D sensors these days are widely equipped on various mobile devices like a depth camera on iPhone, or laser LiDAR sensors on an autonomous driving vehicle. These 3D sensing techniques could help us get accurate measurements of the 3D world. For the field of machine intel- ligence, we also want to build intelligent system and algorithm to learn useful information and understand the 3D world better. We human beings have the incredible ability to sense and understand this 3D world through our visual or tactile system. For example, humans could infer the geometry structure and arrangement of furniture in a room without seeing the full room, we are able to track an 3D object no matter how its appearance, shape and scale changes, we could also predict the future motion of multiple objects based on sequential observation and complex reasoning. Here my work designs various frameworks to learn such 3D information from geometric data represented by a lot of 3D points, which achieves fine-grained geometric understanding of individual objects, and we can help machine tell the target objects' geometry, states, and dynamics. The work in this dissertation serves as building blocks towards a better understanding of this dynamic world.

This thesis addresses the problem of investigating the properties and abilities of a variety of computer vision representations with respect to spatial geometric transformations. Our approach is to employ machine learning methods for finding the behaviour of existing image representations empirically and to apply deep learning to new computer vision tasks where the underlying spatial information is of importance. The results help to further the understanding of modern computer vision representations, such as convolutional neural networks (CNNs) in image classification and object detection and to enable their application to new domains such as local feature detection. Because our theoretical understanding of CNNs remains limited, we investigate two key mathematical properties of representations: equivariance (how transformations of the input image are encoded) and equivalence (how two representations, for example two different parameterizations, layers or architectures share the same visual information). A number of methods to establish these properties empirically are proposed. These methods reveal interesting aspects of their structure, including clarifying at which layers in a CNN geometric invariances are achieved and how various CNN architectures differ. We identify several predictors of geometric and architectural compatibility. Direct applications to structured-output regression are demonstrated as well. Local covariant feature detection has been difficult to approach with machine learning techniques. We propose the first fully general formulation for learning local covariant feature detectors which casts detection as a regression problem, enabling the use of powerful regressors such as deep neural networks. The derived covariance constraint can be used to automatically learn which visual structures provide stable anchors for local feature detection. We support these ideas theoretically, and show that existing detectors can be derived in this framework. Additionally, in cooperation with Imperial College London, we introduce a novel large-scale dataset for evaluation of local detectors and descriptors. It is suitable for training and testing modern local features, together with strictly defined evaluation protocols for descriptors in several tasks such as matching, retrieval and verification. The importance of pixel-wise image geometry for object detection is unknown as the best results used to be obtained with combination of CNNs with cues from image segmentation. We propose a detector which uses constant region proposals and, while it approximates objects poorly, we show that a bounding box regressor using intermediate convolutional features can recover sufficiently accurate bounding boxes, demonstrating that the required geometric information is contained in the CNN itself. Combined with other improvements, we obtain an excellent and fast detector that processes an image only with the CNN.

Capsule Networks are an alternative to the conventional CNN structure for object recognition. They replace max pooling with a dynamic routing of capsule activation. The goal is to better exploit the spatial relationships of the learned features, not only to increase recognition performance, but also improve generalization capability and sample-efficiency. Recently, two algorithms for dynamic routing of capsules have been proposed. Although they received a lot of interest and they are from the same group, an experimental comparison of both is still missing. In this work we compare these two routing algorithms and provide experimental results on data efficiency and generalization to increased input images. Although the experiments are limited to variants of the MNIST dataset, they indicate that the approach of Sabour et al. (2017) is better at learning from few training samples and the EM routing of Hinton et al. (2018) is better at generalizing to changed image sizes.

Human pose estimation (HPE) is an ever-growing research field, with an increasing number of publications in the computer vision and deep learning fields and it covers a multitude of practical scenarios, from sports to entertainment and from surveillance to medical applications. Despite the impressive results that can be obtained with HPE, there are still many problems that need to be tackled when dealing with real-world applications. Most of the issues are linked to a poor or completely wrong detection of the pose that emerges from the inability of the network to model the viewpoint. This thesis shows how designing viewpoint-equivariant neural networks can lead to substantial improvements in the field of human pose estimation, both in terms of state-of-the-art results and better real-world applications. By jointly learning how to build hierarchical human body poses together with the observer viewpoint, a network can learn to generalise its predictions when dealing with previously unseen viewpoints. As a result, the amount of training data needed can be drastically reduced, simultaneously leading to faster and more efficient training and more robust and interpretable real-world applications.

Numerical methods, the collective name for numerical analysis and optimization techniques, have been widely used in the field of computer vision and deep learning. In this thesis, we investigate the algorithms of some numerical methods and their relevant applications in deep learning. These studied numerical techniques mainly include differentiable matrix power functions, differentiable eigendecomposition (ED), feasible orthogonal matrix constraints in optimization and latent semantics discovery, and physics-informed techniques for solving partial differential equations in disentangled and equivariant representation learning. We first propose two numerical solvers for the faster computation of matrix square root and its inverse. The proposed algorithms are demonstrated to have considerable speedup in practical computer vision tasks. Then we turn to resolve the main issues when integrating differentiable ED into deep learning -- backpropagation instability, slow decomposition for batched matrices, and ill-conditioned input throughout the training. Some approximation techniques are first leveraged to closely approximate the backward gradients while avoiding gradient explosion, which resolves the issue of backpropagation instability. To improve the computational efficiency of ED, we propose an efficient ED solver dedicated to small and medium batched matrices that are frequently encountered as input in deep learning. Some orthogonality techniques are also proposed to improve input conditioning. All of these techniques combine to mitigate the difficulty of applying differentiable ED in deep learning. In the last part of the thesis, we rethink some key concepts in disentangled representation learning. We first investigate the relation between disentanglement and orthogonality -- the generative models are enforced with different proposed orthogonality to show that the disentanglement performance is indeed improved. We also challenge the linear assumption of the latent traversal paths and propose to model the traversal process as dynamic spatiotemporal flows on the potential landscapes. Finally, we build probabilistic generative models of sequences that allow for novel understandings of equivariance and disentanglement. We expect our investigation could pave the way for more in-depth and impactful research at the intersection of numerical methods and deep learning.

Convolutional neural networks are equivariant to translations; equivariance to other symmetries, however, is not defined and the class output may vary depending on the input's orientation. To mitigate this, the training data can be augmented at the cost of increased redundancy in the model. Another solution is to build an equivariant neural network and thereby increasing the equivariance to a larger symmetry group. In this study, two convolutional neural networks and their respective equivariant counterparts are constructed and applied to the symmetry groups D4 and C8 to explore the impact on performance when removing and adding batch normalisation and data augmentation. The results suggest that data augmentation is irrelevant to an equivariant model and equivariance to more symmetries can slightly improve accuracy. The convolutional neural networks rely heavily on batch normalisation, whereas the equivariant models achieve high accuracy, although lower than with batch normalisation present.

A proposta deste trabalho é apresentar resultados para o estudo sistemático de sistemas dinâmicos reversíveis-equivariantes, ou seja, em presença simultânea de simetrias e antisimetrias. Este é o caso em que o domínio e as equações que regem o sistema são invariantes pela ação de um grupo de Lie compacto Γ formado pelas simetrias e anti-simetrias do problema. Apresentamos métodos de teoria de Singularidades e teoria de invariantes para classificar bifurcações a um parâmetro de pontos de equilíbrio destes sistemas. Para isso, separamos o estudo de aplicações Γ-reversíveis-equivariantes em dois casos: auto-dual e não auto-dual. No primeiro caso, a existência de um isomorfismo linear Γ-reversível-equivariante estabelece uma correspondência entre a classificação de problemas Γ-reversíveis-equivariantes e a classificação de problemas Γ-equivariantes associados, para os quais todos os elementos de Γ agem como simetria. Os resultados obtidos para o caso não auto-dual se baseiam em teoria de invariantes e envolvem técnicas algébricas que reduzem a análise ao caso polinomial invariante. Dois algoritmos simbólicos são estabelecidos para o cálculo de geradores para o módulo das funções anti-invariantes e para o módulo das aplicações reversíveis-equivariantes. / The purpose of this work is to present results for the sistematic study of reversible-equivariant dynamical systems, namely in simultaneous presence of symmetries and reversing simmetries. This is the case when the domain and the equations modeling the system are invariant under the action of a compact Lie group Γ formed by the symmetries and reversing symmetries of the problem. We present methods in Singularities and Invariant theory to classify oneparameter steady-state bifurcations of these systems. For that, we split the study of the ¡¡reversible-equivariant mapping into two cases: self-dual and non self-dual. In the first case, the existence of a Γ-reversible-equivariant linear isomorphism establishes a one-toone correspondence between the classification of Γ-reversible-equivariant problems and the classification of the associated Γ-equivariant problems, for which all elements in Γ act as symmetries. The results obtained for the non self-dual case are based on Invariant theory and involve algebraic techniques that reduce the analysis to the invariant polynomial case. Two symbolic algorithms are established for the computation of generators for the module of anti-invariant functions and for the module of reversible-equivariant mappings.

When statistical assumptions do not hold and coherent structures are present in spatially extended systems such as fluid flows, flame fronts and field theories, a dynamical description of turbulent phenomena becomes necessary. In the dynamical systems approach, theory of turbulence for a given system, with given boundary conditions, is given by (a) the geometry of its infinite-dimensional state space and (b) the associated measure, that is, the likelihood that asymptotic dynamics visits a given state space region. In this thesis this vision is pursued in the context of Kuramoto-Sivashinsky system, one of the simplest physically interesting spatially extended nonlinear systems. With periodic boundary conditions, continuous translational symmetry endows state space with additional structure that often dictates the type of observed solutions. At the same time, the notion of recurrence becomes relative: asymptotic dynamics visits the neighborhood of any equivalent, translated point, infinitely often. Identification of points related by the symmetry group action, termed symmetry reduction, although conceptually simple as the group action is linear, is hard to implement in practice, yet it leads to dramatic simplification of dynamics. Here we propose a scheme, based on the method of moving frames of Cartan, to efficiently project solutions of high-dimensional truncations of partial differential equations computed in the original space to a reduced state space. The procedure simplifies the visualization of high-dimensional flows and provides new insight into the role the unstable manifolds of equilibria and traveling waves play in organizing Kuramoto-Sivashinsky flow. This in turn elucidates the mechanism that creates unstable modulated traveling waves (periodic orbits in reduced space) that provide a skeleton of the dynamics. The compact description of dynamics thus achieved sets the stage for reduction of the dynamics to mappings between a set of Poincare sections.

A proposta deste trabalho é apresentar resultados para o estudo sistemático de sistemas dinâmicos reversíveis-equivariantes, ou seja, em presença simultânea de simetrias e antisimetrias. Este é o caso em que o domínio e as equações que regem o sistema são invariantes pela ação de um grupo de Lie compacto Γ formado pelas simetrias e anti-simetrias do problema. Apresentamos métodos de teoria de Singularidades e teoria de invariantes para classificar bifurcações a um parâmetro de pontos de equilíbrio destes sistemas. Para isso, separamos o estudo de aplicações Γ-reversíveis-equivariantes em dois casos: auto-dual e não auto-dual. No primeiro caso, a existência de um isomorfismo linear Γ-reversível-equivariante estabelece uma correspondência entre a classificação de problemas Γ-reversíveis-equivariantes e a classificação de problemas Γ-equivariantes associados, para os quais todos os elementos de Γ agem como simetria. Os resultados obtidos para o caso não auto-dual se baseiam em teoria de invariantes e envolvem técnicas algébricas que reduzem a análise ao caso polinomial invariante. Dois algoritmos simbólicos são estabelecidos para o cálculo de geradores para o módulo das funções anti-invariantes e para o módulo das aplicações reversíveis-equivariantes. / The purpose of this work is to present results for the sistematic study of reversible-equivariant dynamical systems, namely in simultaneous presence of symmetries and reversing simmetries. This is the case when the domain and the equations modeling the system are invariant under the action of a compact Lie group Γ formed by the symmetries and reversing symmetries of the problem. We present methods in Singularities and Invariant theory to classify oneparameter steady-state bifurcations of these systems. For that, we split the study of the ¡¡reversible-equivariant mapping into two cases: self-dual and non self-dual. In the first case, the existence of a Γ-reversible-equivariant linear isomorphism establishes a one-toone correspondence between the classification of Γ-reversible-equivariant problems and the classification of the associated Γ-equivariant problems, for which all elements in Γ act as symmetries. The results obtained for the non self-dual case are based on Invariant theory and involve algebraic techniques that reduce the analysis to the invariant polynomial case. Two symbolic algorithms are established for the computation of generators for the module of anti-invariant functions and for the module of reversible-equivariant mappings.

No description available.