We propose the use of depth-information to protect privacy in person-aware visual systems while preserving important foreground subjects and scene structures. We aim to preserve the identity of foreground subjects while hiding superfluous details in the background that may contain sensitive information. We achieve this goal by using depth information and relevant human detection mechanisms provided by the Kinect sensor. In particular, for an input color and depth image pair, we first create a sensitivity map which favors background regions (where privacy should be preserved) and low depth-gradient pixels (which often relates a lot to scene structure but little to identity). We then combine this per-pixel sensitivity map with an inhomogeneous image obscuration process for privacy protection. We tested the proposed method using data involving different scenarios including various illumination conditions, various number of subjects, different context, etc. The experiments demonstrate the quality of preserving the identity of humans and edges obtained from the depth information while obscuring privacy intrusive information in the background. / Computer and Information Science

Leveraging task relatedness has been proven to be beneficial in many machine learning tasks. Extensive researches has been done to exploit task relatedness in various forms. A common assumption for the tasks is that they are intrinsically similar to each other. Based on this assumption, joint learning algorithms are usually implemented via some forms of information sharing. Various forms of information sharing have been proposed, such as shared hidden units of neural networks, common prior distribution in hierarchical Bayesian model, shared weak learners of a boosting classifier, distance metrics and a shared low rank structure for multiple tasks. However, another very common and important task relationship, i.e., task competition, has been largely overlooked. Task competition means that tasks are competing with each other if there are competitions or conflicts between their goals. Considering that tasks with competition relationship are universal, this dissertation is to accommodate this intuition from an algorithmic perspectives and apply the algorithms to various visual recognition problems. Focus on exploiting the task competition relationships in visual recognition, the dissertation presents three types of algorithms and applied them to different visual recognition tasks. First, hypothesis competition has been exploited in a boosting framework. The proposed algorithm CompBoost jointly model the target and auxiliary tasks with a generalized additive regression model regularized by competition constraints. This model treats the feature selection as the weak learner (\ie, base functions) selection problem, and thus provides a mechanism to improve feature filtering guided by task competition. More specifically, following a stepwise optimization scheme, we iteratively add a new weak learner that balances between the gain for the target task and the inhibition on the auxiliary ones. We call the proposed algorithm CompBoost, since it shares similar structures with the popular AdaBoost algorithm. In this dissertation, we use two test beds for evaluation of CompBoost: (1) content-independent writer identification by exploiting competing tasks of handwriting recognition, and (2) actor-independent facial expression recognition by exploiting competing tasks of face recognition. In the experiments for both applications, the approach demonstrates promising performance gains by exploiting the between-task competition relationship. Second, feature competition has been instantiated through an alternating coordinate gradient algorithm. Sharing the same feature pool, two tasks are modeled together in a joint loss framework, with feature interaction encouraged via an orthogonal regularization over feature importance vectors. Then, an alternating greedy coordinate descent learning algorithm (AGCD) is derived to estimate the model. The algorithm effectively excludes distracting features in a fine-grained level for improving face verification. In other words, the proposed algorithm does not forbid feature sharing between competing tasks in a macro level; it instead selectively inhibits distracting features while preserving discriminative ones. For evaluation, the proposed algorithm is applied to two widely tested face-aging benchmark datasets: FG-Net and MORPH. On both datasets, our algorithm achieves very promising performances and outperforms all previously reported results. These experiments, together with detailed experimental analysis, show clearly the benefit of coordinating conflicting tasks for improving visual recognition. Third, two ad-hoc feature competition algorithms have been proposed to apply to visual privacy protection problems. Visual privacy protection problem is a practical case of competition factors in real world application. Algorithms are specially designed to achieve best balance between competing factors in visual privacy protection based on different modeling frameworks. Two algorithms are developed to apply to two applications, license plate de-identification and face de-identification. / Computer and Information Science

The prevalence of wireless networks and the convenience of mobile cameras enable many new video applications other than security and entertainment. From behavioral diagnosis to wellness monitoring, cameras are increasing used for observations in various educational and medical settings. Videos collected for such applications are considered protected health information under privacy laws in many countries. Visual privacy protection techniques, such as blurring or object removal, can be used to mitigate privacy concern, but they also obliterate important visual cues of affect and social behaviors that are crucial for the target applications. In this dissertation, we propose to balance the privacy protection and the utility of the data by preserving the privacy-insensitive information, such as pose and expression, which is useful in many applications involving visual understanding. The Intellectual Merits of the dissertation include a novel framework for visual privacy protection by manipulating facial image and body shape of individuals, which: (1) is able to conceal the identity of individuals; (2) provide a way to preserve the utility of the data, such as expression and pose information; (3) balance the utility of the data and capacity of the privacy protection. The Broader Impacts of the dissertation focus on the significance of privacy protection on visual data, and the inadequacy of current privacy enhancing technologies in preserving affect and behavioral attributes of the visual content, which are highly useful for behavior observation in educational and medical settings. This work in this dissertation represents one of the first attempts in achieving both goals simultaneously.