Back to search

University of Denver

Facial Action Unit Detection With Deep Convolutional Neural Networks

Abstract

dc:description.abstract

The facial features are the most important tool to understand an individual's state of mind. Automated recognition of facial expressions and particularly Facial Action Units defined by Facial Action Coding System (FACS) is challenging research problem in the field of computer vision and machine learning. Researchers are working on deep learning algorithms to improve state of the art in the area. Automated recognition of facial action units has man applications ranging from developmental psychology to human robot interface design where companies are using this technology to improve their consumer devices (like unlocking phone) and for entertainment like FaceApp. Recent studies suggest that detecting these facial features, which is a multi-label classification problem, can be solved using a problem transformation approach in which multi-label problems converted into single-label problem with BinaryRelevance classifier. In this thesis, convolutional neural network is used as it can go substantially deeper, more accurate, though requires lots of data to train the algorithm. It usually results in a significant feature map obtained from each layer of the network. We introduce Modified DenseNet considering DenseNet as a baseline model. Averaging all the features obtained from each block of DenseNet gives importance to each level of features which can get lost during concatenating the layers in DenseNet and other state of the art classification models. Detection of Facial Action Units (AUs) can be determined by selecting threshold for the probabilities obtained by training the Modified DenseNet model. Threshold selection can be done with the help of Matthew Correlation Coefficient. Using Matthew Correlation Coefficient, AU correlation can take into account which was missing for previous studies using BinaryRelevance classifier as it does not consider label’s correlation because it treats every target variable independently. Modifying DenseNet model helped to improve results by reusing features and alleviating the vanishing-gradient problem. We evaluated our proposed architecture on a competitive Facial Action Unit Detection task (EmotioNet) database which includes 950,000 images with annotated AUs. Modified DenseNet obtain significant improvements over the state-of-the-art methods on most of them by comparing with the accuracy and other metrics of evaluation and requiring less computation time as compared to problem transformation methods.

Degree

thesis:*
Name thesis:degree_name
M. S.
Level thesis:degree_level
Masters Thesis

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Padwal, Siddhesh
Contributors dc:contributor
  • Mohammad Mahoor
  • Goncalo Martins
  • Yun-Bo Yi
  • Mohammad A. Matin

Subjects

dc:subject × 7

Rights

dc:rights
Statement dc:rights
  • <p>Copyright is held by the author. User is responsible for all copyright compliance.</p>
  • Siddhesh Padwal
Language dc:language
en

Identifiers

dc:identifier.*
Repository record dc:identifier
https://digitalcommons.du.edu/etd/1819

Chain of custody

source
Harvested from
University of Denver
Base URL
digitalcommons.du.edu/do/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Padwal, Siddhesh. Facial Action Unit Detection With Deep Convolutional Neural Networks. Masters Thesis thesis, https://digitalcommons.du.edu/etd/1819