As diferenças entre os sexos e o impacto dos hormônios sexuais no processamento emocional normal e patológico destacam-se na investigação do dimorfismo sexual na frequência, diagnóstico e terapêutica de patologias psiquiátricas. Transtornos depressivos e ansiosos não apenas são mais comuns em mulheres, quando comparadas aos homens, como parecem ser influenciados pelas concentrações hormonais séricas das mulheres em diferentes fases do ciclo reprodutivo. Ao mesmo tempo, o sexo e as concentrações dos hormônios sexuais, mostram influência na função do cérebro em uma diversidade de tarefas cognitivas e emocionais. O reconhecimento de expressões faciais de emoções básicas tem sido visto como função de extrema importância na adaptação social do indivíduo e existem evidências de que esteja relacionado com o desenvolvimento de transtornos psiquiátricos. Já foi demonstrado que esta tarefa é influenciada pelo sexo do indivíduo e seu ambiente hormonal, no entanto, a literatura carece de resposta sobre os mecanismos pelos quais estas diferenças acontecem. Em dois experimentos buscamos maior entendimento de como se dão as diferenças entre os sexos no reconhecimento de expressões faciais de emoções básicas (raiva, asco, medo, tristeza, surpresa e alegria). No experimento 1, 33 voluntários saudáveis do sexo masculino e 30 do sexo feminino foram testados quanto à acurácia no reconhecimento de expressões faciais, ao tipo de erro ao realizar esta tarefa e à emoção despertada durante este reconhecimento. No experimento 2, 24 voluntárias saudáveis foram testadas quanto à acurácia no reconhecimento de expressões faciais em três diferentes fases do ciclo menstrual: fase folicular precoce (primeiro ao quinto dia do ciclo), periovulatória (décimo segundo ao décimo quarto dia do ciclo), e lútea (vigésimo primeiro ao vigésimo terceiro dia do ciclo), em delineamento cruzado. Foi realizada dosagem sanguínea de estradiol, progesterona e testosterona ao final de cada sessão experimental, com o intuito de confirmar a fase do ciclo das voluntárias e buscar possíveis correlações entre esses hormônios e o processamento de expressões faciais. Utilizou-se análise de contraste na avaliação do desempenho no reconhecimento de todas as emoções básicas com o desempenho no reconhecimento da emoção alegria. No experimento 1, raiva e medo em faces femininas foram reconhecidos com maior acurácia por mulheres, quando comparadas aos homens. Não foram encontradas diferenças significativas entre os sexos quanto à emoção despertada durante a visualização de expressões faciais. O experimento 2 mostrou que o reconhecimento das emoções asco e tristeza em faces masculinas variou de maneira significativa durante as fases do ciclo menstrual. As mulheres na fase lútea obtiveram maior acurácia no reconhecimento de expressões de asco em comparação com a fase folicular precoce, enquanto o desempenho no reconhecimento de tristeza foi maior na fase periovulatória do que na fase lútea. Os resultados sugerem que as diferenças entre homens e mulheres na capacidade de reconhecer emoções não estejam relacionadas à valência da emoção despertada nos indivíduos durante o processamento emocional. A modulação do reconhecimento de expressões faciais pelas fases do ciclo menstrual aponta que este seja um dos fatores implicados nas diferenças entre os sexos nesta tarefa / The impact of sex and sexual hormones in the normal and pathological emotional processing has reached unique importance in the investigation of sexual dimorphism in prevalence, diagnostic features and therapeutics of psychiatric disorders. Depressive and anxiety disorders are not only more common in women compared to men, but they also seem to be influenced by the hormonal status of women at different stages of the reproductive cycle. At the same time, the sex of the subject and the level of sex hormones have been suggested to play a role in brain function in a variety of emotional and cognitive tasks. The recognition of facial expressions of basic emotions has been recognized not only as of extreme importance in social adjustment as there is also evidence of its relation to the development of psychiatric disorders. It has been shown that this task is influenced by the sex and hormonal status of subjects, however, the literature shows a gap in explanations about how these differences occur. In two experiments we sought a better understanding of how sex differences in facial expressions recognition of basic emotion (anger, disgust, fear, sadness, surprise, happiness and neutral) happens. In experiment 1, 33 male and 30 female healthy volunteers were tested for accuracy in the recognition of facial expressions, the type of error when performing this task as well as the emotion aroused during this recognition. In experiment 2, 24 healthy female volunteers were tested for accuracy in the recognition of facial expressions in 3 different phases of menstrual cycle, early follicular (days 1 to 5), periovulatory phase (days 12 to 14) and luteal phase (days 21 to 23), in a crossover study design. Volunteers were tested for blood levels of estrogen, progesterone and testosterone at the end of each experimental session in order to confirm cycle phase and look for possible correlations between hormones and processing of facial expressions. We used contrast analysis in the recognition of each basic emotion against the recognition of happiness. In experiment 1, anger and fear, in feminine faces, were more accurately recognized by women in comparison to men. No significant differences among sexes were found on the emotion aroused while viewing facial expressions. Experiment 2 showed that the recognition of the emotions disgust and sadness, in male faces, varied significantly during the menstrual cycle phases. Women in luteal phase showed greater accuracy in recognizing expressions of disgust than when in early follicular phase whereas the recognition of sadness were more accurate during periovulatory phase than during luteal phase. These results suggest that differences between men and women in the ability to recognize emotions are not related to the valence of the emotions aroused in the subjects during emotional processing. This study also showed that the role played by the menstrual cycle in the ability to recognize facial expressions points to this feature as an important factor implicated in sex differences in this task.

Puisque les traits de biométrie douce peuvent fournir des preuves supplémentaires pour aider à déterminer précisément l’identité de l’homme, il y a eu une attention croissante sur la reconnaissance faciale basée sur les biométrie douce ces dernières années. Parmi tous les biométries douces, le sexe et l’ethnicité sont les deux caractéristiques démographiques importantes pour les êtres humains et ils jouent un rôle très fondamental dans l’analyse de visage automatique. En attendant, la reconnaissance des expressions faciales est un autre challenge dans le domaine de l’analyse de visage en raison de la diversité et de l’hybridité des expressions humaines dans différentes cultures, genres et contextes. Ce thèse est dédié à combiner la texture du visage 2D et la morphologie du visage 3D pour estimer les biométries douces: le sexe, l’ethnicité, etc., et reconnaître les expressions faciales. Pour la reconnaissance du sexe et de l’ethnicité, nous présentons une approche efficace en combinant à la fois des textures locales et des caractéristiques de forme extraites à partir des modèles de visage 3D, contrairement aux méthodes existantes qui ne dépendent que des textures ou des caractéristiques de forme. Afin de souligne exhaustivement la différence entre les groupes sexuels et ethniques, nous proposons un nouveau descripteur, à savoir local circular patterns (LCP). Ce descripteur améliore Les motifs binaires locaux (LBP) et ses variantes en remplaçant la quantification binaire par une quantification basée sur le regroupement, entraînant d’une puissance plus discriminative et une meilleure résistance au bruit. En même temps, l’algorithme Adaboost est engagé à sélectionner les caractéristiques discriminatives fortement liés au sexe et à l’ethnicité. Les résultats expérimentaux obtenus sur les bases de données FRGC v2.0 et BU-3DFE démontrent clairement les avantages de la méthode proposée. Pour la reconnaissance des expressions faciales, nous présentons une méthode automatique basée sur les multi-modalité 2D + 3D et démontrons sa performance sur la base des données BU-3DFE. Notre méthode combine des textures locales et des descripteurs de formes pour atteindre l’efficacité et la robustesse. Tout d’abord, un grand ensemble des points des caractéristiques d’images 2D et de modèles 3D sont localisés à l’aide d’un nouvel algorithme, à savoir la cascade parallèle incrémentielle de régression linéaire (iPar-CLR). Ensuite, on utilise un nouveau descripteur basé sur les histogrammes des gradients d’ordre secondaire (HSOG) en conjonction avec le descripteur SIFT pour décrire la texture locale autour de chaque point de caractéristique 2D. De même, la géométrie locale autour de chaque point de caractéristique 3D est décrite par deux nouveaux descripteurs de forme construits à l’aide des quantités différentielle de géométries de la surface au premier ordre et au second ordre, à savoir meshHOG et meshHOS. Enfin, les résultats de reconnaissance des descripteurs 2D et 3D fournis par le classifier SVM sont fusionnés à la fois au niveau de fonctionnalité et de score pour améliorer la précision. Les expérimentaux résultats démontrent clairement qu’il existe des caractéristiques complémentaires entre les descripteurs 2D et 3D. Notre approche basée sur les multi-modalités surpasse les autres méthodes de l’état de l’art en obtenant une précision de reconnaissance 86,32%. De plus, une bonne capacité de généralisation est aussi présentée sur la base de données Bosphorus. / Since soft biometrics traits can provide sufficient evidence to precisely determine the identity of human, there has been increasing attention for face based soft biometrics identification in recent years. Among those face based soft biometrics, gender and ethnicity are both key demographic attributes of human beings and they play a very fundamental and important role in automatic machine based face analysis. Meanwhile, facial expression recognition is another challenge problem in face analysis because of the diversity and hybridity of human expressions among different subjects in different cultures, genders and contexts. This Ph.D thesis work is dedicated to combine 2D facial Texture and 3D face morphology for estimating people’s soft biometrics: gender, ethnicity, etc., and recognizing facial expression. For the gender and ethnicity recognition, we present an effective and efficient approach on this issue by combining both boosted local texture and shape features extracted from 3D face models, in contrast to the existing ones that only depend on either 2D texture or 3D shape of faces. In order to comprehensively represent the difference between different genders or ethnics groups, we propose a novel local descriptor, namely local circular patterns (LCP). LCP improves the widely utilized local binary patterns (LBP) and its variants by replacing the binary quantization with a clustering based one, resulting in higher discriminative power as well as better robustness to noise. Meanwhile, the following Adaboost based feature selection finds the most discriminative gender- and ethnic-related features and assigns them with different weights to highlight their importance in classification, which not only further raises the performance but reduces the time and memory cost as well. Experimental results achieved on the FRGC v2.0 and BU-3DFE data sets clearly demonstrate the advantages of the proposed method. For facial expression recognition, we present a fully automatic multi-modal 2D + 3D feature-based facial expression recognition approach and demonstrate its performance on the BU–3DFE database. Our approach combines multi-order gradientbased local texture and shape descriptors in order to achieve efficiency a nd robustness. First, a large set of fiducial facial landmarks of 2D face images along with their 3D face scans are localized using a novel algorithm namely incremental Parallel Cascade of Linear Regression (iPar–CLR). Then, a novel Histogram of Second Order Gradients (HSOG) based local image descriptor in conjunction with the widely used first-order gradient based SIFT descriptor are employed to describe the local texture around each 2D landmark. Similarly, the local geometry around each 3D landmark is described by two novel local shape descriptors constructed using the first-order and the second-order surface differential geometry quantities, i.e., Histogram of mesh Gradients (meshHOG) and Histogram of mesh Shape index (curvature quantization, meshHOS). Finally, the Support Vector Machine (SVM) based recognition results of all 2D and 3D descriptors are fused at both featurelevel and score-level to further improve the accuracy. Comprehensive experimental results demonstrate that there exist impressive complementary characteristics between the 2D and 3D descriptors. We use the BU–3DFE benchmark to compare our approach to the state-of-the-art ones. Our multi-modal feature-based approach outperforms the others by achieving an average recognition accuracy of 86,32%. Moreover, a good generalization ability is shown on the Bosphorus database.

As diferenças entre os sexos e o impacto dos hormônios sexuais no processamento emocional normal e patológico destacam-se na investigação do dimorfismo sexual na frequência, diagnóstico e terapêutica de patologias psiquiátricas. Transtornos depressivos e ansiosos não apenas são mais comuns em mulheres, quando comparadas aos homens, como parecem ser influenciados pelas concentrações hormonais séricas das mulheres em diferentes fases do ciclo reprodutivo. Ao mesmo tempo, o sexo e as concentrações dos hormônios sexuais, mostram influência na função do cérebro em uma diversidade de tarefas cognitivas e emocionais. O reconhecimento de expressões faciais de emoções básicas tem sido visto como função de extrema importância na adaptação social do indivíduo e existem evidências de que esteja relacionado com o desenvolvimento de transtornos psiquiátricos. Já foi demonstrado que esta tarefa é influenciada pelo sexo do indivíduo e seu ambiente hormonal, no entanto, a literatura carece de resposta sobre os mecanismos pelos quais estas diferenças acontecem. Em dois experimentos buscamos maior entendimento de como se dão as diferenças entre os sexos no reconhecimento de expressões faciais de emoções básicas (raiva, asco, medo, tristeza, surpresa e alegria). No experimento 1, 33 voluntários saudáveis do sexo masculino e 30 do sexo feminino foram testados quanto à acurácia no reconhecimento de expressões faciais, ao tipo de erro ao realizar esta tarefa e à emoção despertada durante este reconhecimento. No experimento 2, 24 voluntárias saudáveis foram testadas quanto à acurácia no reconhecimento de expressões faciais em três diferentes fases do ciclo menstrual: fase folicular precoce (primeiro ao quinto dia do ciclo), periovulatória (décimo segundo ao décimo quarto dia do ciclo), e lútea (vigésimo primeiro ao vigésimo terceiro dia do ciclo), em delineamento cruzado. Foi realizada dosagem sanguínea de estradiol, progesterona e testosterona ao final de cada sessão experimental, com o intuito de confirmar a fase do ciclo das voluntárias e buscar possíveis correlações entre esses hormônios e o processamento de expressões faciais. Utilizou-se análise de contraste na avaliação do desempenho no reconhecimento de todas as emoções básicas com o desempenho no reconhecimento da emoção alegria. No experimento 1, raiva e medo em faces femininas foram reconhecidos com maior acurácia por mulheres, quando comparadas aos homens. Não foram encontradas diferenças significativas entre os sexos quanto à emoção despertada durante a visualização de expressões faciais. O experimento 2 mostrou que o reconhecimento das emoções asco e tristeza em faces masculinas variou de maneira significativa durante as fases do ciclo menstrual. As mulheres na fase lútea obtiveram maior acurácia no reconhecimento de expressões de asco em comparação com a fase folicular precoce, enquanto o desempenho no reconhecimento de tristeza foi maior na fase periovulatória do que na fase lútea. Os resultados sugerem que as diferenças entre homens e mulheres na capacidade de reconhecer emoções não estejam relacionadas à valência da emoção despertada nos indivíduos durante o processamento emocional. A modulação do reconhecimento de expressões faciais pelas fases do ciclo menstrual aponta que este seja um dos fatores implicados nas diferenças entre os sexos nesta tarefa / The impact of sex and sexual hormones in the normal and pathological emotional processing has reached unique importance in the investigation of sexual dimorphism in prevalence, diagnostic features and therapeutics of psychiatric disorders. Depressive and anxiety disorders are not only more common in women compared to men, but they also seem to be influenced by the hormonal status of women at different stages of the reproductive cycle. At the same time, the sex of the subject and the level of sex hormones have been suggested to play a role in brain function in a variety of emotional and cognitive tasks. The recognition of facial expressions of basic emotions has been recognized not only as of extreme importance in social adjustment as there is also evidence of its relation to the development of psychiatric disorders. It has been shown that this task is influenced by the sex and hormonal status of subjects, however, the literature shows a gap in explanations about how these differences occur. In two experiments we sought a better understanding of how sex differences in facial expressions recognition of basic emotion (anger, disgust, fear, sadness, surprise, happiness and neutral) happens. In experiment 1, 33 male and 30 female healthy volunteers were tested for accuracy in the recognition of facial expressions, the type of error when performing this task as well as the emotion aroused during this recognition. In experiment 2, 24 healthy female volunteers were tested for accuracy in the recognition of facial expressions in 3 different phases of menstrual cycle, early follicular (days 1 to 5), periovulatory phase (days 12 to 14) and luteal phase (days 21 to 23), in a crossover study design. Volunteers were tested for blood levels of estrogen, progesterone and testosterone at the end of each experimental session in order to confirm cycle phase and look for possible correlations between hormones and processing of facial expressions. We used contrast analysis in the recognition of each basic emotion against the recognition of happiness. In experiment 1, anger and fear, in feminine faces, were more accurately recognized by women in comparison to men. No significant differences among sexes were found on the emotion aroused while viewing facial expressions. Experiment 2 showed that the recognition of the emotions disgust and sadness, in male faces, varied significantly during the menstrual cycle phases. Women in luteal phase showed greater accuracy in recognizing expressions of disgust than when in early follicular phase whereas the recognition of sadness were more accurate during periovulatory phase than during luteal phase. These results suggest that differences between men and women in the ability to recognize emotions are not related to the valence of the emotions aroused in the subjects during emotional processing. This study also showed that the role played by the menstrual cycle in the ability to recognize facial expressions points to this feature as an important factor implicated in sex differences in this task.