Different ways to measure human affective and deceptive reactions to stimulus have been developed. One method is a multimodal approach using web camera, thermal imaging camera and physiological sensors data to extract different features in the human face (verbal and non-verbal behavior) such as breathing rate, heart rate, face temperature, skin conductance, eye tracking, language analysis and facial expressions among others. Human subjects from different ages and ethnicity were exposed to two different experiments were they watched videos (affection recognition) and others answered an interview session (deception recognition). With the data collected from videos (thermal and visual), different regions of interest (ROI) of the face were selected as well as the whole picture. The ROI were determined based on the most sensitive parts of the face where larger changes of temperature or other physiological features are recorded. It was also analyzed the language (written and spoken) in order to obtain the verbal modalities. The data has been compared among the subjects to determine whether the deceptive and affective reactions of a person can be predicted using multimodal approach. From the multiple data obtained, a characterization of reactions is proposed when subjects are exposed to different stimulus, positive or negative, as well as deceptive behavior and later on recognize if the person is happy, sad, nervous, anxious, telling the truth, lying etc. Using the multimodal approach we were able to predict automatically, with higher accuracy than the baseline, affective and deceptive states of a person. In the affective state recognition, the classifier software differentiated affective state versus neutral state with 92.85% accuracy. Then it differentiated Positive State, Negative State and Neutral State with 57.14% accuracy. Additionally, it differentiated Positive State versus Negative State with 73.21% accuracy. Finally, the classifier was able to predict Deceptive State (people lying) and Non Deceptive State (people telling the truth) with 72.72% accuracy.
| Identifer | oai:union.ndltd.org:unt.edu/info:ark/67531/metadc271869 |
| Date | 05 1900 |
| Creators | Narvaez-Valle, Alexis |
| Contributors | Burzo, Mihai, Mihalcea, Rada, 1974-, Choi, Tae-Youl |
| Publisher | University of North Texas |
| Source Sets | University of North Texas |
| Language | English |
| Detected Language | English |
| Type | Thesis or Dissertation |
| Format | Text |
| Rights | Public, Narvaez-Valle, Alexis, Copyright, Copyright is held by the author, unless otherwise noted. All rights Reserved. |
Page generated in 0.003 seconds