In this dissertation, we focus on solving Nonlinear Least Squares problems using a supervised approach. In particular, we developed a Supervised Descent Method (SDM), performed thorough theoretical analysis, and demonstrated its effectiveness on optimizing analytic functions, and four other real-world applications: Inverse Kinematics, Rigid Tracking, Face Alignment (frontal and multi-view), and 3D Object Pose Estimation. In Rigid Tracking, SDM was able to take advantage of more robust features, such as, HoG and SIFT. Those non-differentiable image features were out of consideration of previous work because they relied on gradient-based methods for optimization. In Inverse Kinematics where we minimize a non-convex function, SDM achieved significantly better convergence than gradient-based approaches. In Face Alignment, SDM achieved state-of-the-arts results. Moreover, it was extremely computationally efficient, which makes it applicable for many mobile applications. In addition, we provided a unified view of several popular methods including SDM on sequential prediction, and reformulated them as a sequence of function compositions. Finally, we suggested some future research directions on SDM and sequential prediction.

Machine face recognition has traditionally been studied under the assumption of a carefully controlled image acquisition process. By controlling image acquisition, variation due to factors such as pose, lighting, and background can be either largely eliminated or specifically limited to a study over a discrete number of possibilities. Applications of face recognition have had mixed success when deployed in conditions where the assumption of controlled image acquisition no longer holds. This dissertation focuses on this unconstrained face recognition problem, where face images exhibit the same amount of variability that one would encounter in everyday life. We formalize unconstrained face recognition as a binary pair matching problem (verification), and present a data set for benchmarking performance on the unconstrained face verification task. We observe that it is comparatively much easier to obtain many examples of unlabeled face images than face images that have been labeled with identity or other higher level information, such as the position of the eyes and other facial features. We thus focus on improving unconstrained face verification by leveraging the information present in this source of weakly supervised data. We first show how unlabeled face images can be used to perform unsupervised face alignment, thereby reducing variability in pose and improving verification accuracy. Next, we demonstrate how deep learning can be used to perform unsupervised feature discovery, providing additional image representations that can be combined with representations from standard hand-crafted image descriptors, to further improve recognition performance. Finally, we combine unsupervised feature learning with joint face alignment, leading to an unsupervised alignment system that achieves gains in recognition performance matching that achieved by supervised alignment.

Etude bibliographique sur le recalage d'images de visage et sur le recalage d'images et travail en collaboration avec Son VuS, pour définir la précision nécessaire du recalage en fonction des exigences des méthodes de reconnaissance de visages. / Face alignment is an important step in a typical automatic face recognition system.This thesis addresses the alignment of faces for face recognition applicationin video surveillance context. The main challenging factors of this research includethe low quality of images (e.g., low resolution, motion blur, and noise), uncontrolledillumination conditions, pose variations, expression changes, and occlusions. In orderto deal with these problems, we propose several face alignment methods using differentstrategies. The _rst part of our work is a three-stage method for facial pointlocalization which can be used for correcting mis-alignment errors. While existingalgorithms mostly rely on a priori knowledge of facial structure and on a trainingphase, our approach works in an online mode without requirements of pre-de_nedconstraints on feature distributions. The proposed method works well on images underexpression and lighting variations. The key contributions of this thesis are aboutjoint image alignment algorithms where a set of images is simultaneously alignedwithout a biased template selection. We respectively propose two unsupervised jointalignment algorithms : \Lucas-Kanade entropy congealing" (LKC) and \gradient correlationcongealing" (GCC). In LKC, an image ensemble is aligned by minimizing asum-of-entropy function de_ned over all images. GCC uses gradient correlation coef-_cient as similarity measure. The proposed algorithms perform well on images underdi_erent conditions. To further improve the robustness to mis-alignments and thecomputational speed, we apply a multi-resolution framework to joint face alignmentalgorithms. Moreover, our work is not limited in the face alignment stage. Since facealignment and face acquisition are interrelated, we develop an adaptive appearanceface tracking method with alignment feedbacks. This closed-loop framework showsits robustness to large variations in target's state, and it signi_cantly decreases themis-alignment errors in tracked faces.

<div>The research includes machine Learning and Deep Learning Approaches to Print Defect Detection, Face Set Recognition and Face Alignment, and Visual-Enhancement in Space and Time. This thesis consists of six parts which are related to 6 projects:</div><div><br></div><div>In Chapter 1, the first project focuses on detection of local printing defects including gray spots and solid spots. We propose a coarse-to-fine method to detect local defects in a block-wise manner and aggregate the blockwise attributes to generate the feature vector of the whole test page for a further ranking task. In the detection part, we first select candidate regions by thresholding a single feature. Then more detailed features of candidate blocks are calculated and sent to a decision tree that is previously trained on our training dataset. The final result is given by the decision tree model to control the false alarm rate while maintaining the required miss rate.</div><div><br></div><div>Chapter 2 introduces face set recognition and Chapter 3 is about face alignment. In order to reduce the computational complexity of comparing face sets, we propose a deep neural network that can compute and aggregate the face feature vectors with different weights. As for face alignment, our goal is to solve the jittering of landmark locations when applied on video. We propose metrics and corresponding methods around this goal.</div><div><br></div><div>In recent years, mobile photography has become increasingly prevalent in our lives with social media due to its high portability and convenience. However, many challenges still exist in distributing high-quality mobile images and videos under the limit of data capacity, hardware storage, and network bandwidth. Therefore, we have been exploring enhancement techniques to improve the image and video qualities, considering both effectiveness and efficiency for a wide variety of applications, including WhatsApp, Portal, TikTok, even the printing industry. Chapter 4 introduces single image super-resolution to handle real-world images with various degradations, and its influence on several downstream high-level computer vision tasks. Next, Chapter 5 studies on headshot image restoration with multiple references, which is an application of visual enhancement under more specific scenarios. Finally, as a step towards the temporal domain enhancement, the Zooming SlowMo framework for fast and accurate space-time video super-resolution will be introduced in Chapter 6.</div>