The introduction of input devices with natural user interfaces in gaming hardware has changed the way we interact with games. Hardware with motion-sensing and gesture recognizing capabilities remove the constraint of interacting with games through typical traditional devices like mouse-keyboard and gamepads. This changes the way we approach games and how the game communicates back to us as the player opening new levels of interactivity. This thesis covers how eye tracker technology can be used to affect rendering effects in games.

Clinicians are becoming increasingly aware of the effect of various pathologies on the characteristics of saccadic eye movements. As such, an efficient and non-invasive means of measuring eye-movement in a clinical environment is of interest to many. The aim of this thesis is to investigate the clinical application of a non-invasive eye movement recording technique as a part of a clinical examination. Eye movements were measured using an IRIS 6500 infrared limbal eye tracker, which we customized for the direct recording of oblique eye movements as well as horizontal and vertical. Firstly, the eye-tracker itself was assessed. Visually normal observers made saccadic eye movements to a 10' stimulus in eight directions of gaze. Primary (ANOVA) and secondary analyses (mean error less than 5%) resulted in acceptance that averaging four measurements would give a representative measurement of saccadic latency, peak velocity, amplitude and duration. Test-retest results indicated that this technique gives statistically (± 1.96*STDEVDifference) repeatable responses. Several factors that could potentially influence clinically based measures of eye-movements were examined. These included, the effect of ageing, viewing distances, dioptric blur and cataract. The results showed that saccadic latency and duration are significantly (p < 0.05) longer in older (60-89 years) observers compared to younger (20-39 years). Peak velocity and amplitude were not significantly affected by the age of the observer. All saccadic parameters (SP) were significantly affected by direction (Chapter 5). The compact nature of this eye movement methodology is obtainable since there is no significant effect on viewing distance (300 cm vs. 49 cm) (Chapter 6). There is also no significant effect of dioptric blur (up to +LOODS) on any of the four SP. In contrast, a higher level of defocus (+3.O ODS) has a larger probability of interfering with the measurements of peak velocity and duration (Chapter 7). Saccadic eye-movements were also recorded whilst normally sighted subjects wore cataract simulation goggles. The results suggested that the presence of dense cataract introduces significant increases in saccadic latencies and durations. No effect was found on the peak velocities and amplitudes. The effect of amblyopia on SP was also investigated in order to examine if this methodology is able to detect normal from abnormal responses (i.e. increased saccadic latencies). This set of data (Chapter9 ) showed that using IRIS 6500, longer than normal latencies may be recorded from the amblyopic eye but no consistent effect was found for the other SP (peak velocity, amplitude, duration). Overall, the results of this thesis demonstrate that the IRIS 6500 eye-tracker has many desirable elements (it is non-invasive; comfortable for the observers and gives repeatable and precise results in an acceptable time) that would potentially make it a useful clinical tool as a part of a routine examination.

Avec ce travail, nous avons voulu mettre l'accent sur la validité externe des expérimentations et nous focaliser sur des situations dynamiques naturelles. Or dans ce type de situation, les traitements top-down, découlant des connaissances préalables et de l'objectif, guident préférentiellement l'allocation de l'attention, tandis que les traitements bottom-up (caractéristiques des entrées perceptives) ont une influence mineure. Ce travail est donc centré sur l'attention top-down. La situation choisie est une approche de carrefour en conduite automobile, les sous-tâches étudiées sont le contrôle du véhicule (la trajectoire) et la décision de s'arrêter ou non au carrefour. Deux objectifs opérationnels sont investigués à travers l'analyse des mouvements oculaires : 1/ distinguer les sous-tâches de contrôle du véhicule et de prise de décision, 2/ distinguer les processus de la sous-tâche de prise de décision. Pour chaque objectif, une approche qualitative puis quantitative sont explorées. Des techniques de classification supervisées ont notamment été utilisées pour distinguer les différents processus. Deux études ont été conduites afin de répondre à ces objectifs, une en simulateur de conduite et une en situation réelle de conduite, mettant toutes deux en scène des approches de carrefours. / In this work, we focus on the external validity of experiments and on natural dynamic situations. In this type of situation, the top-down treatments resulting from prior knowledge and goal preferably guide the allocation of attention, while the bottom-up treatments (characteristics of perceptual input) play a minor role. This work is then focused on top-down attention. The chosen situation is an approach of driving crossroads, the studied subtasks are the control of vehicle (trajectory) and the decision to stop or not at the crossroads. Two operational objectives are investigated by analyzing eye movements: 1/ discriminating the vehicle's control and decision making subtasks, 2/ discriminating the process of the decision making subtask. For each objective, we explore a qualitative and quantitative approach. In particular, supervised techniques of classification have been used to distinguish the different processes. Two studies were conducted in order to meet these objectives, one with a driving simulator and one in a real situation of driving, both involving approaches to crossroads.

閱讀是吸收知識的途徑，不同的閱讀模式所帶來的閱讀成效也會不同。如何透過機器學習的方式，從凝視點找出閱讀行為的關聯性，將是本研究的目標。實驗選擇低成本眼動儀紀錄讀者閱讀過程中的眼動資料，採用dispersion-based演算法找出凝視點，以計算凝視點特徵，包含凝視時間、凝視距離、凝視位置以及凝視方向。 本研究將閱讀模式分成五種類別，包含快讀、慢讀、精讀、跳讀與關鍵字識別，透過不同文章的呈現，引導30位測試者遵循其內容進行閱讀，藉此收集不同行為模式的眼動資料。實驗流程中所有的眼動資料會隨機被分成為兩份，依序建立不同維度的訓練資料，由交叉驗證的分類結果找出理想之特徵與維度。以每次挑選6位測試者的眼動數據為測試資料進行5次分類驗證，其平均正確率為78.24%、74.19%、93.75%、87.96%以及96.20%，均達到不錯的分類結果。 / Reading is one of the paths to acquire knowledge. The efficiency is different when different reading patterns are involved. It is the objective of this research to classify reading patterns from fixation data using machine learning techniques. In our experiment, a low-cost eye tracker is employed to record the eye movements during the reading process. A dispersion-based algorithm is implemented to identify fixation from the recorded data. Features pertaining to fixation including duration, path length, landing position and fixation direction are extracted for classification purposes. Five categories of reading pattern are defined and investigated in this study, namely, speed reading, slow reading, in-depth reading, skim-and-skip, and keyword spotting. We have recruited thirty subjects to participate in our experiment. The participants are instructed to read different articles using specific styles designated by the experimenter in order to assign label to the collected data. Feature selection is achieved by analyzing the predictive results of cross-validation from the training data obtained from all subjects. The average classification accuracies in five-fold cross-validation are 78.24%, 74.19%, 93.75%, 87.96% and 96.20% using the eye movements of the six randomly selected subjects as test data.

An open question in autism research is how to assess language abilities in this population. We investigated language development in monolingual and bilingual children with varying degrees of autism, ages 3 to 9, with the aim of better understanding vocabulary comprehension. Two different methodologies were used: the Receptive One-Word Picture Vocabulary Test (ROWPVT) and eye-tracker technique. We examined whether the eye-tracker could help in the assessment of these children because it does not require the child to point during the test. Four typically developing control children, 14 monolingual English children with moderate/mild autism, and 4 children (2 monolingual English, 2 bilingual Spanish/English) with severe autism were tested and the results of the ROWPVT test were compared to the eye-tracker results. Interestingly, bilingual children with severe autism had better results using eye-tracker than the traditional ROWPVT test. These results suggest that these children know more vocabulary than traditional test measures indicate.

This diploma thesis focuses on drivers of motor vehicles and the usual time they spend by specific objects. The theoretical part of the thesis brings an overview of literature on human perception. The eye-tracker is defined here as the instrument used for scanning the movements of the drivers eyes, here are also named its types and its applications in various fields of life. Finally, this part deals with the advertisements placed alongside roads acting as distractive elements. The experimental part builds upon the collected video recordings and offers an analysis of drivers’ observations of outdoor advertisements that attract their attention, mainly billboards.

No description available.

根據恐懼管理理論（Terror Management Theory, 1986）人們會適時地管理死亡想法的威脅，故Pyszczynski、Greenberg及Solomon（1999）依據此理論提出了一套雙元歷程模式來解釋人們的防衛方式：近端防衛與遠端防衛。在遠端防衛中文化世界觀尤為重要，在過去的西方研究之中，發現人們常以對自身文化世界觀持正向或偏好的態度，來防衛死亡想法的威脅；然而這樣的現象卻在亞洲研究中難以得到驗證，因此本研究的目的即在於重新檢驗文化世界觀的防衛方式。本研究操弄參與者的死亡想法，並藉由分心作業的方式，使其死亡想法掉入意識邊陲之中，以探究遠端防衛的本質。本研究除了以偏好的評估作為依變項外，且從認知的注意力面向出發，探討文化世界觀與防衛死亡想法之間的關係。本研究設計兩個實驗：實驗一以視覺搜尋作業的派典進行研究，以對自身文化的偏好、正確率與反應時間當作依變項。結果發現當參與者進行遠端防衛時，會對於與自身文化有關的線索，表現出較高的正確率，但在對自身文化的偏好，與反應時間的變項上，未能發現有顯著差異。實驗二以眼動追蹤儀器當作研究工具，並以眼動指標中首次凝視位置的數量，以及凝視時間作為依變項。結果發現當參與者進行遠端防衛時，會對於與自身文化有關的線索，表現出較長的凝視時間，但在首次凝視位置的數量上，亦未能發現有顯著差異。因此，綜合以上兩個實驗，本研究並未能完全支持文化世界觀的遠端防衛效果。 / According to terror management theory (1986), people can manage the threat from death thought. Thus, Pyszczynski, Greenberg, & Solomon (1999) proposed a dual-process model to explain how people defend the death. There are two defense routes: the proximal defense and the distal defense. The culture-worldview is an important function in the distal defense. In the western research, it was found that people were more positive or preferred to self-culture-worldview. However, there are mixed findings in the eastern research. In order to investigate the distal defense mechanism, this study manipulated participants’ death thought, and let the thought drop into the unconscious level via a distracting task. Furthermore, this study examined the relationship between the culture-worldview and the death defense from attentional paradigms. We conducted two experiments. In the first experiment, we adopted the visual search paradigm, and the dependent variables were self-cultural preference, reaction time and the rate of accuracy. We found it was more accurate when participants were asked to search the self-related cultural cue after reminding one’s death comparing to those are not reminding about death. But they were no significant difference in the self-cultural preference and reaction time. In the second experiment, we adopted the eye-tracker as the instrument, and the dependent variables were the index of number of first fixation and the duration of fixations. We found the fixation duration was longer for the self-related cultural cues than for the non-self-related cultural cues when participants being reminded about one’s death. But it was no significant difference in the number of first fixation. So, through these two experiments we found an inconsistent result in the distal defense.

Dans notre communication quotidienne avec les autres, nous avons autant de considération pour l’interlocuteur lui-même que pour l’information transmise. En permanence coexistent en effet deux modes de transmission : le verbal et le non-verbal. Sur ce dernier thème intervient principalement l’expression faciale avec laquelle l’interlocuteur peut révéler d’autres émotions et intentions. Habituellement, un processus de reconnaissance d’émotions faciales repose sur 3 étapes : le suivi du visage, l’extraction de caractéristiques puis la classification de l’expression faciale. Pour obtenir un processus robuste apte à fournir des résultats fiables et exploitables, il est primordial d’extraire des caractéristiques avec de forts pouvoirs discriminants (selon les zones du visage concernées). Les avancées récentes de l’état de l’art ont conduit aujourd’hui à diverses approches souvent bridées par des temps de traitement trop couteux compte-tenu de l’extraction de descripteurs sur le visage complet ou sur des heuristiques mathématiques et/ou géométriques.En fait, aucune réponse bio-inspirée n’exploite la perception humaine dans cette tâche qu’elle opère pourtant régulièrement. Au cours de ces travaux de thèse, la base de notre approche fut ainsi de singer le modèle visuel pour focaliser le calcul de nos descripteurs sur les seules régions du visage essentielles pour la reconnaissance d’émotions. Cette approche nous a permis de concevoir un processus plus naturel basé sur ces seules régions émergentes au regard de la perception humaine. Ce manuscrit présente les différentes méthodologies bio-inspirées mises en place pour aboutir à des résultats qui améliorent généralement l’état de l’art sur les bases de référence. Ensuite, compte-tenu du fait qu’elles se focalisent sur les seules parties émergentes du visage, elles améliorent les temps de calcul et la complexité des algorithmes mis en jeu conduisant à une utilisation possible pour des applications temps réel. / Communication in any form i.e. verbal or non-verbal is vital to complete various daily routine tasks and plays a significant role inlife. Facial expression is the most effective form of non-verbal communication and it provides a clue about emotional state, mindset and intention. Generally automatic facial expression recognition framework consists of three step: face tracking, feature extraction and expression classification. In order to built robust facial expression recognition framework that is capable of producing reliable results, it is necessary to extract features (from the appropriate facial regions) that have strong discriminative abilities. Recently different methods for automatic facial expression recognition have been proposed, but invariably they all are computationally expensive and spend computational time on whole face image or divides the facial image based on some mathematical or geometrical heuristic for features extraction. None of them take inspiration from the human visual system in completing the same task. In this research thesis we took inspiration from the human visual system in order to find from where (facial region) to extract features. We argue that the task of expression analysis and recognition could be done in more conducive manner, if only some regions are selected for further processing (i.e.salient regions) as it happens in human visual system. In this research thesis we have proposed different frameworks for automatic recognition of expressions, all getting inspiration from the human vision. Every subsequently proposed addresses the shortcomings of the previously proposed framework. Our proposed frameworks in general, achieve results that exceeds state-of-the-artmethods for expression recognition. Secondly, they are computationally efficient and simple as they process only perceptually salient region(s) of face for feature extraction. By processing only perceptually salient region(s) of the face, reduction in feature vector dimensionality and reduction in computational time for feature extraction is achieved. Thus making them suitable for real-time applications.

Les émotions des êtres humains changent régulièrement et parfois de manière brusque entrainant un changement de l’état mental c’est-à-dire de l’aptitude cérébrale à fonctionner normalement. Il en résulte une capacité cognitive (ou état cognitif) de l’individu à pouvoir raisonner, accéder à la mémoire, ou effectuer des déductions, variable selon l’état mental. Ceci affecte, en conséquence, les performances des utilisateurs qui varient en fonction de leurs état cognitifs. Cette thèse vise à optimiser l’état cognitif d’un utilisateur lors de ses interactions avec un environnement virtuel. Comme cet état dépend des émotions, l’optimisation de l’état cognitif peut être réalisée à travers l’optimisation des émotions et en particulier la réduction des émotions négatives. Une première partie concerne les moyens de mesurer en temps réel (par un Module de mesures) l’état émotionnel et mental d’un utilisateur lors de ses interactions avec un environnement virtuel. Nous avons réalisé pour cela quatre études expérimentales avec quatre environnements différents. Nous avons montré que ces mesures peuvent être réalisées en utilisant différents capteurs physiologiques. Nous avons aussi montré qu’il est possible de prédire la tendance de l’excitation (un état mental) à partir d’un traceur de regard. Dans une deuxième partie, nous présentons l’Agent Neural qui modifie les environnements virtuels afin de provoquer une modification de l’état émotionnel d’un utilisateur pour améliorer son état cognitif. Nous avons réalisé quatre études expérimentales avec quatre environnements virtuels, où l’Agent Neural intervient dans ces environnements afin de changer l’état émotionnel de l’utilisateur. Nous avons montré que l’agent est capable d’intervenir dans plusieurs types d’environnements et de modifier les émotions de l’utilisateur. Dans une troisième partie, présentons l’Agent Limbique, qui personnalise et améliore les adaptations faites par l’Agent Neural à travers l’observation et l’apprentissage des impacts des changements des environnements virtuels et des réactions émotionnelles des utilisateurs. Nous avons montré que cet agent est capable d’analyser les interventions de l’Agent Neural et de les modifier. Nous avons montré aussi que l’Agent Limbique est capable de générer une nouvelle règle d’intervention et de prédire son impact sur l’utilisateur. La combinaison du Module de mesures, de l’Agent Neural, et de l’Agent Limbique, nous a permis de créer un système de contrôle cognitif intelligent que nous avons appelé Système Limbique Digital. / The human’s emotions change regularly and sometimes suddenly leading to changes in their mental state which is the brain’s ability to function normally. This mental state’s changes affect the users’ cognitive ability (or cognitive state) to reason, access memory, or make inferences, which varies depending on the mental state. Consequently, this affects the users’ performances which varies according to their cognitive states. This thesis aims to optimize the users’ cognitive state during their interactions with a virtual environment. Since this state depends on emotions, optimization of cognitive state can be achieved through the optimization of emotions and in particular the reduction of negative emotions. In a first part, we present the means of measuring in real time (using a Measuring module) the users’ emotional and mental state during their interactions with a virtual environment. We performed four experimental studies with four different environments. We have shown that these measurements can be performed using different physiological sensors. We have also shown that it is possible to predict the tendency of excitement (a mental state) using an eye tracker. In a second part, we present the Neural Agent which modifies virtual environments to provoke a modification on the users’ emotional state in order to improve their cognitive state. We performed four experimental studies with four virtual environments, in which the Neural Agent intervenes in these environments to change the users’ emotional state. We have shown that the agent is able to intervene in several types of environments and able to modify the users’ emotions. In a third part, we present the Limbic Agent, which personalizes and improves the adaptations performed by the Neural Agent through the observation and the learning from the virtual environments changes’ impacts and the users’ emotional reactions. We have shown that this agent is able to analyze the Neural Agent’s interventions and able to modify them. We have also shown that the Limbic Agent is able to generate a new intervention rule and predict its impact on the user. The combination of the Measuring Module, the Neural Agent, and the Limbic Agent, allowed us to create an intelligent cognitive control system that we called the Digital Limbic System.