The accurate integration of audio-visual emotion cues is critical for social interactions and requires efficient processing of facial cues. Gaze behavior of typically developing young adults was measured via eye-tracking during the perception of dynamic audio-visual emotion (DAVE) stimuli. Participants were able to identify basic emotions (angry, fearful, happy, neutral) and determine the congruence of facial expression and prosody. Perception of incongruent videos resulted in increased reaction times and emotion identification consistent with the facial expression. Participants consistently demonstrated a featural processing approach across all tasks, with a significant preference for the eyes. Evidence of hemispheric lateralization was indicated by preferential fixation to the left (happy, angry) or right eye (fearful). Fixation patterns differed according to the facially expressed emotion, with the pattern that emerged during fearful movies supporting the significance of automatic threat processing. Finally, fixation pattern during the perception of incongruent movies varied according to task instructions.

The accurate integration of audio-visual emotion cues is critical for social interactions and requires efficient processing of facial cues. Gaze behavior of typically developing (TD) individuals and individuals with autism spectrum disorders (ASD) was measured via eye-tracking during the perception of dynamic audio-visual emotion (DAVE) stimuli. This study provides information about the regions of the face sampled during an emotion perception task that is relatively more complex than those used in previous studies, providing both bimodal (auditory and visual) and dynamic (biological motion) cues. Results indicated that the ASD group was less accurate at emotion detection and demonstrated less of a visual-affective bias than TD individuals. Both groups displayed similar fixation patterns across regions during the perception of congruent audio-visual stimuli. However, between-group analyses revealed that fixation patterns differed significantly by facial regions during the perception of both congruent and incongruent movies together. In addition, fixation duration to critical regions (i.e., face, core, eyes) was negatively correlated with measures of ASD symptomatology and social impairment. Findings suggest weaknesses in the early integration of audio-visual information, automatic perception of emotion, and efficient detection of affective conflict in individuals with ASD. Implications for future research and social skills intervention programs are discussed.

京都大学 / 0048 / 新制・課程博士 / 博士(情報学) / 甲第17925号 / 情博第507号 / 新制||情||90(附属図書館) / 30745 / 京都大学大学院情報学研究科知能情報学専攻 / (主査)教授 西田 豊明, 教授 河原 達也, 教授 黒橋 禎夫 / 学位規則第4条第1項該当 / Doctor of Informatics / Kyoto University / DGAM

京都大学 / 0048 / 新制・課程博士 / 博士(情報学) / 甲第20735号 / 情博第649号 / 新制||情||112(附属図書館) / 京都大学大学院情報学研究科知能情報学専攻 / (主査)准教授 川嶋 宏彰, 教授 河原 達也, 教授 熊田 孝恒 / 学位規則第4条第1項該当 / Doctor of Informatics / Kyoto University / DFAM

京都大学 / 新制・課程博士 / 博士(情報学) / 甲第24821号 / 情博第837号 / 新制||情||140(附属図書館) / 京都大学大学院情報学研究科知能情報学専攻 / (主査)教授 熊田 孝恒, 教授 西田 眞也, 教授 内田 由紀子, 准教授 中島 亮一 / 学位規則第4条第1項該当 / Doctor of Informatics / Kyoto University / DFAM

Our study aims to investigate if people who are hungry are affected by that when shopping groceries online. Nine people participated in two user tests, one in the morning, just before lunch and one in the afternoon, just after lunch. They were asked not to eat any breakfast if they were to conduct a test in the morning. This meant that the participants would be hungry in one of the cases and full in the other. The study was based on the participants being able to add groceries to the cart from a list that was read for them. They did this in two different interfaces, coop.se and ica.se, while an eye tracker logged their eye movements. The study was supplemented with surveys that determined whether the participants were hungry or full after each test and a final survey with the participants' preferences, BMI and habits in digital interfaces. After the study was completed we looked for deviations to see if the hunger affected the number of deviations and what the participants were looking for. Deviations were calculated partly as the number of products viewed as not belonging to the task, and partly as products with a high number of calories. Specially viewed products include sauces, breakfast products and ready meals and recipes. A distinction between the two interfaces could be found in that one interface received more deviations than the other. A significant difference in the number of data deviations between morning and afternoon could not be determined, on the other hand, at a 10% level of significance, we could determine that there was a difference between the different interfaces and that the subjects deviated more the first time the test was performed. We were able to contradict our hypothesis, that people would deviate with their eyes more when they are hungry than when they are not. Our results show, rather that other factors are crucial to what affects the deviations, such as which interface you’re in and whether it's the first time you do grocery shopping online. / Vår undersökning ämnar ta reda på om personer som är hungriga påverkas av den aspekten när de handlar mat på internet. Nio deltagare gjorde två användartester, ett på förmiddagen, precis innan lunch och ett på eftermiddagen, precis efter lunch. Detta för att personerna skulle känna sig hungriga i ett fall och känna sig mätta i det andra. Personerna blev ombedda att inte äta någon frukost om de skulle utföra ett test på förmiddagen. Undersökningen gick ut på att deltagarna fick handla från en uppläst inhandlingslista i två olika gränssnitt, coop.se samt ica.se, medan en eye-tracker loggade deras ögonrörelser. Undersökningen kompletterades med enkäter som fastställde om deltagarna var hungriga eller mätta efter varje undersökning och en avslutande med deltagarnas matpreferenser, BMI och vana i digitala gränssnitt. Därefter letade vi avvikelser för att se huruvida hungern påverkade antalet avvikelser och vad deltagarna tittade på för varor. Avvikelser räknades dels som antalet varor man titta på som inte tillhörde uppgiften, och dels som varor med högt antal kalorier. Särskilt förekommande varor var bland annat såser, frukostprodukter och färdiga maträtter och recept. En skillnad mellan de två gränssnitten kunde hittas i form av att det ena gränssnittet fick fler antal avvikelser än det andra. En signifikant skillnad i antal uppgiftsavvikelser mellan förmiddag och eftermiddag kunde inte fastslås, däremot kunde vi på en 10% signifikansnivå fastslå att det var en skillnad mellan de olika gränssnitten och att försökspersonerna avvek mer den första gången man fick utföra testet. Vår hypotes, att personer skulle avvika med blicken mer när de är hungriga än när de inte är det, kan vi i efterhand dementera. Våra resultat visar snarare på att andra faktorer är avgörande för vad som påverkar blickavvikelserna, såsom gränssnitt och om det är första gången man handlar mat online.