Cette recherche a pour objectif d’examiner l’influence de la présence, dans des annonces presse, d’un personnage, d’un visage ou de la direction d’un regard sur l’attention portée à cette annonce ainsi que sur la mémorisation et les évaluations de cette annonce. Afin d’éprouver nos hypothèses quatre expérimentations ont été menées. Dans l’une d’entre elles un système eye-tracking a été utilisé permettant de mieux cerner les processus attentionnels. Pour nous rapprocher des conditions d’exposition habituelles à la publicité, les annonces presse ont été dissimulées dans un magazine fictif. Les résultats obtenus montrent que la présence du personnage ou du visage (versus leur absence), ainsi que le regard dirigé vers le produit (versus le regard dirigé vers l’observateur), exercent une influence sur l’attention portée à l’annonce, la mémorisation de son contenu et son évaluation. / This research aims to examine the influence of the presence in a print advertisement of a character, a face or a gaze direction (gaze directed toward the product versus gaze directed toward the observer) on attention toward the advertisement, the memorization and the evaluation of the advertisement content. To test our hypotheses, four experiments were conducted. In one of these experiments an eye-tracker has been used to better measure attentional processes. In order to approach the ordinary conditions of exposure to advertising, print advertisements were inserted in a fictive magazine (folder test procedure). The results show that the presence of the character or the face (versus their absence) and the gaze directed toward the product (versus gaze directed toward the observer) improve the attention toward the advertisement, the memorization and the assessment of advertisement content.

Introduktion – Millennials, är den första generation som har växt med teknologi nära tillhands. För dem har mobilen fått bli en integrerad del av deras liv, vilket har påverkat hur dem som konsumenter interagerar med en butik och dess miljö. På många sätt har mobilen blivit en distraktion för dem när de är i butiken, men på vissa sätt så kan den också vara en hjälp för dem, speciellt i komplexa köp. Detaljhandlare har däremot fram tills nu sett mobilen som ett problem då den leder till att konsumenten ignorerar det som finns i butiken, men eftersom mobilen har betydande roll i konsumenternas liv bör de istället byta fokus. De måste istället hitta sätt att dra nytta av mobilanvändningen, både för deras egen fördel och förkonsumenternas. Syfte och metod – Studiens syfte var därmed att få en djupare förståelse för hur detaljhandlare kan hitta värde i mobilanvändandet för millennials i komplexa köp. Vidare, syftade den till att kunna ge teoretiska bidrag kring konsumentens värdeskapande, genom att sammanfläta mobilanvändandet med butiksmiljön. Studien gjordes som en kvalitativ fallstudie på IKEA genom användningen av två metoder. 47 respondenter tillfrågades att göra tre uppgifter på köksavdelning medans de bar eye trackingglasögon som filmade vad de gjorde. Efter det, fick de göra en komplimenterande semistrukturerad intervju, och alla data analyserades sedan med hjälp av en tematisk analys. Resultat och slutsatser – Resultatet visar att millennials använder sin mobil för att hjälpa dem navigera sig i butiken, hitta information, och viktigast, för att ta foton på information och inspiration. Däremot, även ifall konsumenten ser mobilen som ett värdefullt verktyg, visar också resultatet att trots det fortfarande är viktigt med personlig service i komplexa köp. Till sist, finns det också indikationer på att serviceerbjudande i butiken måste anpassas för att återge mer tydlig och relevant information, samt att skyltarna behöver skapa interaktioner mellan konsumenterna och mobilen, till exempel genom användningen av QR-koder. Teoretiskt, bidrar denna studie både genom att ge ytterligare stöd till tidigare studier och genom att bidra med nya insikter kring värdeskapande i serviceerbjudandet. / Introduction – The digital native generation, millennials, is the first generation to grow up with technology close at hand. For them the mobile phone is an integrated part of their lives, which has an impact on how they as consumers interact with the retail environment. In many ways the mobile phone can be a distraction for them in store, but in some ways, it can also be a help for them, especially in high involvement purchases. Retailers has up until now found the mobile phone to be a problem since it can make the consumers ignore the retail landscape. However, since the prominent role the mobile phone has in the consumers life, there need to be a switch in focus, and retailers needs to utilise it, both for their own and the consumers benefit. Purpose and methodology - The aim of this study was therefore to get a deeper understanding of how retailers can find value in mobile phone use for the millennial consumer in high involvement purchases. In addition, it aimed to make theoretical contributions to consumer value creation, by incorporating the mobile phone usage into the retail environment. The study was done using a qualitative multiple research method in a single case study on IKEA. 47 millennials respondents were asked to perform three tasks in the kitchen department while wearing eye tracking glasses to record their actions. Thereafter, a complementary semistructured interview was performed, and the data was later analysed using thematic analysis. Result and conclusion – The results show that the millennial consumers use their mobile phone to help them navigate the store, find information, but most importantly to take photos for information and inspirational purposes. However, even though many consumers found the mobile phone as a valuable tool, in high involvement purchases personalised service is still highly valued. Finally, the result indicated the services cape needs to be adapted with clearer and more relevant information as well as signs to create interactions with the consumer and the mobile phones such as QR codes. Theoretically, this study both supports the results from previous studies as well as contributes with new insights to the theoretical field of value in service.

This thesis investigates the relationship between eye movements, memory and thinking in five studies based on eye tracking experiments. The studies draw on the human ability to spatially index multimodal events as demonstrated by people’s gaze reverting back to emptied spatial locations when retrieving information that was associated with this location during a preceding encoding phase – the so called “looking-at-nothing” phenomenon. The first part of this thesis aimed at gaining a better understanding of the relationship between eye movements and memory in relation to verbal information. The second part of this thesis investigated what could be learned about the memory processes involved in reasoning and decision-making by studying eye movements to blank spaces. The first study presented in this thesis clarified the role of eye movements for the retrieval of verbal information from memory. More precisely, it questioned if eye movements to nothing are functionally related to memory retrieval for verbal information, i.e. auditorily presented linguistic information. Eye movements were analyzed following correct and incorrect retrievals of previously presented auditory statements concerning artificial places that were probed during a subsequent retrieval phase. Additionally, eye movements were manipulated as the independent variable with the aid of a spatial cue that either guided the eyes towards or away from associated spatial locations. Using verbal materials elicited eye movements to associated but emptied spatial locations, thereby replicating previous findings on eye movements to nothing. This behaviour was more pronounced for correct in comparison to incorrect retrievals. Retrieval performance was higher when the eyes were guided towards in comparison to being guided away from associated spatial locations. In sum, eye movements play a functional role for the retrieval of verbal materials. The second study tested if the looking-at-nothing behaviour can also diminish; for example, does its effect diminish if people gain enough practice in a retrieval task? The same paradigm was employed as in the first study. Participants listened to four different sentences. Each sentence was associated with one of four areas on the screen and was presented 12 times. After every presentation, participants heard a statement probing one sentence, while the computer screen remained blank. More fixations were found to be located in areas associated with the probed sentence than in other locations. Moreover, the more trials participants completed, the less frequently they exhibited the looking-at-nothing behaviour. Looking-at-nothing behaviour can in this way be seen to indeed diminish when knowledge becomes strongly represented in memory. In the third and fourth study eye movements were utilized as a tool to investigate memory search during rule- versus similarity-based decision-making. In both studies participants first memorized multiple pieces of information relating to job candidates (exemplars). In subsequent test trials they judged the suitability of new candidates that varied in their similarity to the previously learned exemplars. Results showed that when using similarity, but not when using a rule, participants fixated longer on the previous location of exemplars that were similar to the new candidates than on the location of dissimilar exemplars. This suggests that people using similarity retrieve previously learned exemplars, whereas people using a rule do not. Eye movements were used yet again as a tool in the fifth study. On this occasion, eye movements were investigated during memory-based diagnostic reasoning. The study tested the effects of symptom order and diversity with symptom sequences that supported two or three contending hypotheses, and which were ambiguous throughout the symptom sequence. Participants first learned information about causes and symptoms presented in spatial frames. Gaze allocation on emptied spatial frames during symptom processing and during the diagnostic response reflected the subjective status of hypotheses held in memory and the preferred interpretation of ambiguous symptoms. Gaze data showed how the diagnostic decision develops and revealed instances of hypothesis change and biases in symptom processing. The results of this thesis demonstrate in very different scenarios the tight interplay between eye movements, memory and thinking. They show that eye movements are not automatically directed to spatial locations. Instead, they reflect the dynamic updating of internal, multimodal memory representations. Eye movements can be used as a direct behavioural correlate of memory processes involved in similarity- versus rule-based decision-making, and they reveal rich time-course information about the process of diagnostic reasoning. The results of this thesis are discussed in light of the current theoretical debates on cognitive processes that guide eye movements, memory and thinking. This thesis concludes by outlining a list of recommendations for using eye movements to investigate thinking processes, an outlook for future research and possible applications for the research findings. / Diese Dissertation beschäftigt sich mit der Interaktion von Blickbewegungen, Gedächtnis- und Denkprozessen. In fünf experimentellen Untersuchungen, die auf der Messung von Blickbewegungen beruhen, wurde die menschliche Fähigkeit zum räumlichen Indizieren multimodaler Ereignisse untersucht. Diese Fähigkeit manifestiert sich u.a. im sogenannten „Looking-at-nothing“ Phänomen, das beschreibt, dass Menschen beim Abruf von Informationen aus dem Gedächtnis an Orte zurückblicken, die in einer vorhergehenden Enkodierphase mit den abzurufenden Informationen assoziiert wurden, selbst wenn diese räumlichen Positionen keinerlei erinnerungsrelevante Informationen mehr enthalten. In der ersten Untersuchung wurde der Frage nachgegangen, ob Blickbewegungen an geleerte räumliche Positionen den Abruf von Informationen aus dem Gedächtnis erleichtern. Während ein solches Verhalten für den Abruf zuvor visuell dargebotener Informationen bereits gezeigt werden konnte, ist die Befundlage für die Erinnerungsleistung bei auditiv dargebotenen, linguistischen Informationen unklar. Um diesen Zusammenhang zu untersuchen, wurde das Blickverhalten zunächst als Folge von richtigen und falschen Antworten untersucht. In einem weiteren Schritt wurde das Blickverhalten experimentell manipuliert. Dies geschah mit Hilfe eines räumlichen Hinweisreizes, der die Blicke entweder hin zu der Position leitete, die mit dem abzurufenden Stimulus assoziiert war, oder weg von dieser Position. Die Ergebnisse dieser Untersuchung konnten bisherige Befunde zum Looking-at-nothing Verhalten replizieren. Zudem zeigte sich, dass beim korrekten Abruf von Informationen aus dem Gedächtnis vermehrt Looking-at-nothing gezeigt wurde, während das bei fehlerhaften Abrufen nicht der Fall war. Die Blickmanipulation ergab, dass die Gedächtnisleistung besser war, wenn der Hinweisreiz den Blick hin zur assoziierten räumlichen Position leitete. Im Gegensatz dazu war die Erinnerungsleistung schlechter, wenn der Blick von der assoziierten räumlichen Position weggeleitet wurde. Blickbewegungen an geleerte räumliche Positionen scheinen demnach auch den Abruf verbaler Stimuli zu erleichtern. In der zweiten Untersuchung wurde erforscht, ob das Looking-at-nothing Verhalten nachlässt, wenn das experimentelle Material stark gelernt, d.h. stark im Gedächtnis repräsentiert ist. Dazu wurde das gleiche experimentelle Paradigma, wie in der ersten Untersuchung verwendet. Vier verschiedene Sätze wurden während der Enkodierphase mit vier verschiedenen räumlichen Positionen assoziiert. Nach jeder Präsentation aller vier Sätze, wurde einer der Sätze getestet. Diese Prozedur wiederholte sich in zwölf Durchgängen. In den ersten vier Durchgängen sahen die Versuchspersonen beim Abruf häufiger in das Feld, dass mit der getesteten Information assoziiert war, d.h. sie zeigten wie erwartet das Looking-at-nothing Verhalten. Je mehr Durchgänge die Versuchspersonen bearbeiteten, desto seltener blickten sie zu der assoziierten räumlichen Position. Demnach verschwindet das Looking-at-nothing Verhalten, wenn Informationen stark im Gedächtnis repräsentiert sind. In der dritten und vierten Untersuchung wurden Blickbewegungen an geleerte räumliche Positionen als Methode verwendet um Denkprozesse zu untersuchen. In der dritten Untersuchung lernten Versuchsteilnehmer zunächst Informationen über fiktive Bewerber (Exemplare) für eine freie Position in einem Unternehmen. Jedes Exemplar wurde mit seinen Eigenschaften während der Lernphase mit einer distinkten räumlichen Position verknüpft. In einer nachfolgenden Entscheidungsphase beurteilten die Versuchsteilnehmer neue Bewerber. Diese neuen Bewerber variierten in ihrer Ähnlichkeit mit den zuvor gelernten Bewerbern. Versuchsteilnehmer die eine ähnlichkeitsbasierte Entscheidungsstrategie verwendeten, sahen an die geleerten räumlichen Positionen zurück, die in der Lernphase mit den Exemplaren verknüpft wurden. Wendeten sie jedoch eine abstrakte Regel an, um die neuen Bewerber zu beurteilten, so zeigten sie kein Looking-at-nothing Verhalten. Dieses Ergebnis lässt darauf schließen, dass eine ähnlichkeitsbasierte im Gegensatz zu einer regelbasierten Strategie den Abruf zuvor gelernter Exemplare bewirkt. Auch in der fünften Untersuchung wurden Blickbewegungen als Methode eingesetzt, diesmal zur Untersuchung gedächtnisbasierter Schlussfolgerungsprozesse, wie sie beim Finden von Erklärungen für eine Anzahl gegebener Informationen auftreten. Manipuliert wurden die Reihenfolge der präsentierten Informationen und die Diversität der möglichen Erklärungen. Die getesteten Symptomsequenzen unterstützen stets mindestens zwei mögliche Erklärungen. Die Versuchsteilnehmer lernten in einer vorangestellten Lernphase die Symptome und ihre möglichen Erklärungen. Symptome und Erklärungen wurden mit räumlichen Positionen verknüpft. In einer anschließenden Diagnosephase wurden verschiedene Symptomsequenzen getestet. Das Blickverhalten während der Diagnosephase reflektierte die Interpretation der Symptome im Sinne der subjektiv wahrscheinlichsten Erklärung. Die Aufzeichnung und Analyse der Blickbewegungen erlaubte es die Entwicklung dieser Interpretation über die gesamte Sequenz hinweg zu beobachten und Hypothesenwechsel lokalisieren zu können. Insgesamt stützen die Ergebnisse dieser Dissertation die Annahme einer engen funktionalen Verbindung von Blickbewegungen, Gedächtnis- und Denkprozessen. Sie zeigen, dass Blickbewegungen nicht automatisch an alle assoziierten räumlichen Positionen gerichtet werden, sondern dass sie vielmehr den situations- und aufgabenabhängigen Abruf von Informationen aus dem Gedächtnis widerspiegeln. Blickbewegungen können als direktes Verhaltensmaß zur Messung von Gedächtnisprozessen beim ähnlichkeitsbasierten Entscheiden herangezogen werden und liefern wertvolle Prozessdaten über die Integration von Symptominformationen beim diagnostischen Schließen. Die Ergebnisse dieser Dissertation werden im Lichte der aktuellen theoretischen Diskussion über kognitive Prozesse beim Bewegen der Augen, beim Gedächtnisabruf und beim komplexen Denken betrachtet. Abschließend werden Empfehlungen für die Verwendung der Methode der Blickbewegungsmessung als Prozessmaß zur Untersuchung gedächtnisbasierter Denkprozesse gegeben, ein Überblick über zukünftige Forschungsmöglichkeiten präsentiert und Ideen für Anwendungsmöglichkeiten der präsentierten Befunde aufgezeigt.

Abductive reasoning is the process of finding the best explanation for a set of observations. The theory of abductive reasoning (TAR, Johnson & Krems, 2001) allows detailed process assumptions that were only partly tested in detail up until now. This thesis employs an artificial abductive reasoning task, the Black Box task, and eye tracking measures in order to gain insight into the process. The first part of this thesis aims at evaluating process measures based on eye tracking and using them in order to gain a better understanding of the processes postulated in TAR such as the construction of a situation model or retrieval of relevant information. The second part investigates the relationship between working memory and abductive reasoning by manipulating the amount of information stored in memory and examining the relationship between visual abductive reasoning and working memory skills. In a last part a perspective to the transferability of our results to everyday life tasks is given. The first study focuses on differentiating between processes that take place during the encoding and the evaluation of observation information by comparing eye movement measures. In the second study, we tested process assumptions such as the construction of a mental representation from TAR using memory indexing, an eye tracking method that makes it possible to trace the retrieval of explanations currently held in working memory. Gaze analysis revealed that participants encode the presented evidence (i.e., observations) together with possible explanations into memory. When new observations are presented, the previously presented evidence and explanations are retrieved. With the memory indexing method, we were able to assess the process of information retrieval in abductive reasoning, which was previously believed to be unobservable. The theory of abductive reasoning (TAR; Johnson & Krems, 2001) assumes that when information is presented sequentially, new information is integrated into a mental representation called a situation model, the central data structure on which all reasoning processes are based. Since working memory capacity is limited, the question arises how reasoning might change with the amount of information that has to be processed in memory. To answer this question, we conducted a third experimental study, in which we manipulated whether previous observation information and previously found explanations had to be retrieved from memory or were still present in the visual array. We analyzed individual ratings of difficulty as well as behavioral data and reasoning outcomes. Our results provide evidence that people experience differences in task difficulty when more information has to be retrieved from memory. This is also evident in changes in the mental representation as reflected by eye tracking measures. However, these differences are not evident in the reasoning outcome. These findings suggest that individuals construct their situation model from both information in memory as well as external memory stores. The complexity of the model depends on the task at hand: when memory demands are high, only relevant information is included. With this compensation strategy, people are able to achieve similar reasoning outcomes even when faced with more difficult tasks. The precise relationship between reasoning and working memory capacity remains largely opaque. Combining data of both studies from chapter 3 and 4, we firstly investigated if reasoning performance differs due to differences in working memory capacity. Secondly, using eye tracking, we explored the relationship between the facets of working memory and the process of visuospatial reasoning. Therefore both, a test for storage and processing, and content components (verbal-numerical/ spatial) of working memory as well as an intelligence measure, were engaged. Results show a clear relationship between reasoning accuracy, spatial storage and processing components as well as intelligence. Process measures suggest that high spatial working memory ability might lead to the use of strategies optimizing the content and complexity of the mental representation on which abductive reasoning is based. In a fifth study, we aimed to investigate whether there are also indicators for the mechanisms postulated by TAR in a task that is closer to real life reasoning. Therefore, we asked participants to solve 12 jigsaw puzzles whereby the abductive task was the identification of the motive presented on the puzzles. Thereby, the pieces of the puzzles posed as observation and hypotheses to the motive of the puzzle as explanations. As a process tracing measure, we used thinking aloud. Verbal protocols were recorded, transcripted and carefully coded according to the operators and explanation types postulated in TAR. We found evidence that participants use most of the operators with a likeliness that significantly lies above chance level. We also found evidence of the existence of the different explanation types. Eye movements were able to give insight in the interrelations between working memory, attention, and action. Therefore, this work contributes to understanding abductive reasoning, not only by testing the assumptions of TAR, but also by finding relations between memory, action and thought. The results do not only account for abductive reasoning in an artificial task but also in everyday life reasoning.:1 Introduction 1 1.1 Theories on Abductive Reasoning and Beyond 4 1.1.1 Theory of Abductive Reasoning 4 1.1.2 Other Theories 7 1.2 Reasoning, Working Memory, and Mental Representation 9 1.3 Process Tracing 11 1.4 An Artificial Abductive Task: The Black Box 12 1.5 Overview and Research Objectives 15 1.5.1 Differentiating between Encoding and Processing 15 1.5.2 Current Explanations in Memory 16 1.5.3 Information Stored in Memory 16 1.5.4 More than Storage of Information 17 1.5.5 In the Context of Everyday Life 18 1.5.6 Summary, Perspectives, and Conclusion 18 2 The Possibilities of Eye Tracking: Differentiating between Encoding and Processing 21 2.1 Abstract 22 2.2 Introduction 23 2.3 Method 26 2.3.1 Participants 26 2.3.2 Task and Apparatus 27 2.3.3 Procedure 28 2.3.4 Analysis 29 2.4 Results 30 2.5 Discussion 32 3 Tracing Current Explanations in Memory: A Process Analysis Based on Eye Tracking 37 3.1 Abstract 38 3.2 Introduction 39 3.2.1 Current Explanations of Abductive Reasoning 41 3.2.2 Tracing the Reasoning Process 44 3.2.3 Present Study 45 3.3 Method 48 3.3.1 Participants 49 3.3.2 Apparatus 49 3.3.3 Material 50 3.3.4 Procedure 53 3.4 Results 54 3.4.1 Performance 54 3.4.2 Gaze Analyses 55 3.4.3 Hypothesis 1: Information Stored in the Situation Model 57 3.4.4 Hypothesis 2: Different Types of Explanations—Concrete vs. Abstract 61 3.5 Discussion 67 3.5.1 Information Stored in the Situation Model 68 3.5.2 Concretely and Abstractly Explained Observations 68 3.5.3 TAR and Current Theories on Abductive Reasoning 70 3.5.4 Tracing Memory Processes 72 3.5.5 Conclusion 74 Appendix 3.1 75 Appendix 3.2 76 Appendix 3.3 77 Appendix 3.4 78 4 Information Stored in Memory Affects Abductive Reasoning 79 4.1 Abstract 80 4.2 Introduction 81 4.2.1 The Reasoning Process 82 4.2.2 Visual Attention 85 4.2.3 Research Objectives 86 4.2.4 This Study 87 4.2.5 Using Eye Movements as a Method to Assess Memory Retrieval 89 4.2.6 Hypotheses 89 4.3 Method 92 4.3.1 Participants 92 4.3.2 Apparatus 92 4.3.3 The Black Box Task 92 4.3.4 Procedure 95 4.3.5 Pairwise Comparisons 96 4.4 Results 96 4.4.1 Performance 96 4.4.2 Gaze Analysis 99 4.4.3 Hypothesis 1: Differences Experienced in Task Difficulty 101 4.4.4 Hypothesis 2: Elements of the Situation Model 102 4.4.5 Hypothesis 3: Integrative Solutions 105 4.5 Discussion 107 4.5.1 Differences Experienced in Task Difficulty 108 4.5.2 Elements of the Situation Model 108 4.5.3 Integrative Solutions 110 4.5.4 Summary 112 5 More than Storage of Information – What Working Memory Contributes to Visual Abductive Reasoning 113 5.1 Abstract 114 5.2 Introduction 115 5.2.1 Working memory 116 5.2.2 Relations between Abductive Reasoning Working Memory Capacity 118 5.2.3 Eye Movements as a Process Tracing Method 119 5.2.4 Abductive Reasoning Outcomes and Working Memory Ability. 120 5.2.5 Abductive Reasoning Processes and Working Memory Ability 121 5.3 Method 123 5.3.1 Participants 124 5.3.2 Apparatus 124 5.3.3 Material 125 5.3.4 Procedure 127 5.4 Results 128 5.4.1 Analysis 128 5.4.2 Abductive Reasoning Accuracy and Working Memory Ability 131 5.4.3 Abductive Reasoning Processes and Working Memory Ability 132 5.5 Discussion 135 5.5.1 The Interaction of Reasoning Accuracy and Memory Ability 135 5.5.2 The Interaction of the Process of Reasoning and Memory Ability 136 5.5.3 Conclusion 138 6 The Theory of Abductive Reasoning in the Context of Everyday Life 141 6.1 Abstract 142 6.2 Introduction 143 6.2.1 Abduction in “Real Life” 145 6.3 Method 146 6.3.1 Participants 146 6.3.2 Task 147 6.3.3 Material 148 6.3.4 Apparatus 148 6.3.5 Procedure 149 6.3.6 Coding system 150 6.4 Results 153 6.4.1 Analysis 153 6.4.2 Descriptive Data 153 6.3.3. Likeliness of Operator Use 155 6.5 Discussion 156 6.5.1 Operator Use 156 6.5.2 Explanation Types 157 6.5.3 Perspectives 158 7 Summary, Perspectives, and Conclusion 159 7.1 The Process of Abductive Reasoning 159 7.2 Contributions of other Theories 162 7.3 Eye Tracking and its Methodological Implications 164 7.4 Future Research and Applications 167 7.5 Conclusion 169 8 References 171 Curriculum Vitae 191 Publications 196

In this thesis the Tobii eye-tracker 4L was used to investigate how well eye-tracking solutions such as a confirmation-click and dwell-time algorithm compares to the standard mouse input device when performing selection tasks in a map environment. In order to distinguish the different complexity one could face, two user cases are proposed. Scenario one includes non clustered objects. Scenario two include clustered occluded objects. A user study with nine different participants where conducted in order to compare the execution times and find out how error prone the different methods were. Each test participant performed eight different tests, three in the non-clustered scenario and five in the clustered scenario. In two of the tests in the clustered scenario test participants were aided with zooming through a zoom algorithm. The methods was evaluated by calculating the average execution times and errors along with the corresponding standard deviations. In order to grasp the users experience a subjective cognitive load score was calculated with the help of a questionnaire. The eye-tracker methods was found to be competitive in comparison to mouse interaction in the more simple non-clustered case. However, in a more complex scenario such as the clustered case the mouse interaction had the lowest average completion time and cognitive load score. A different type of selection behaviour was discovered among the test participants in the clustered scenario due to the difference in precision between the eye-tracker and mouse interaction. Finally interesting areas to consider in the future is presented and discussed. / I denna avhandling användes en Tobii eye-tracker 4L för att undersöka hur väl eye-tracking metoder så som en bekräftelseklick och dwell-time algoritm jämför sig med standard mus interaktion vid objekt selektion i en kartmiljö. För att urskilja variationen i komplexitet man kan möta föreslås två olika användarfall. Scenario ett inkluderar objekt som är distinktivt separerade och därav ej grupperade. Scenario två inkluderar grupperade samt ockluderade objekt. En användarstudie med nio olika deltagare genomfördes för att jämföra exekveringstiderna och ta reda på hur felbenägna de olika metoderna var. Varje testdeltagare utförde åtta olika tester, tre i det icke-grupperade scenariot och fem i det grupperade scenariot. I två av testerna i det grupperade scenariot fick deltagarna hjälp med att zooma genom en zoomalgoritm. Metoderna utvärderades genom att beräkna de genomsnittliga exekveringstiderna samt antal fel tillsammans med motsvarande standardavvikelser. För att förstå hur användarna upplevde de olika metoderna togs en subjektiv kognitiv belastningspoäng fram genom ett frågeformulär. Eye-tracker metoderna var konkurrenskraftiga i jämförelse med musinteraktion i det enklare fallet där objekt ej var grupperade. I ett mer komplext scenario, såsom i det grupperade fallet, hade dock musinteraktionen den lägsta genomsnittliga exekveringstiden och kognitiva belastningspoängen. En annan typ av selektions beteende upptäcktes bland testdeltagarna i det grupperade scenariot på grund av skillnaden i precision mellan eye-trackern och musinteraktionen. Slutligen presenteras och diskuteras intressanta områden att överväga vid framtida arbeten.

A European project is gradually replacing the current train control systems to a mutual control system called as the European Railway Traffic Management System (ERTMS). I Sweden they have implemented the new train control system on a few train-lines in the northern part of Sweden. There has been reported problems from train drivers that have used the new control system, the problems consist of both negative affects on train performance and railroad capacity. This study is part of a larger project conducted by The Swedish National Road and Transport Research Institute, with an aim to investigate how the attention is affected in the new train control system compared to the Automatic Train Control (ATC) system. Data collection through eye-tracking have shown significant results in the distribution of visual glances inside and outside the cabin of the train. Drivers tend to distribute more and longer glances inside the train with the ERTMS. Drivers also tend to miss information presented on the track, such as signs informing when to make a sound before an unattended level crossing. These results suggest that ERTMS could have a negative impact on the drivers situation awareness and also could have a negative affect on the drivers attention that impacts the ability to notice what is happening outside the cabin. / I dagsläget utförs ett projekt i Europa med syfte att ersätta nuvarande tågskyddssystem till ett gemensamt signalsystem kallat ERTMS (European Railway Traffic Management System). Detta nya system har implementerats på ett fåtal tåg-linjer i norra delen av Sverige. Olika problem har rapporterats från tågförare som använt det nya signalsystemet. Dessa problem innefattar negativ påverkan på prestation och järnvägens kapacitet. Denna studie är en del av ett större projekt hos Statens väg- och transportforskningsinstitut med ett syfte att studera hur uppmärksamhet påverkas i det nya tågskyddssystemet ERTMS jämfört med det nuvarande systemet ATC (Automatic Train Control). Data som samlades in via eye-tracking har visat signifikanta resultat i fördelning av visuella blickar inne i och utanför tågets kabin. Förare tenderar att fördela mer och även längre blickar inuti tågets kabin med ERTMS. Dessutom tenderar förare att missa information som presenteras ute på spåret, detta kan exempelvis vara information som påvisar att tuta innan ett obevakat övergångsställe. Dessa resultat föreslår att ERTMS kan ha en negativ påverkan på förares uppmärksamhet vilket kan påverka förarens förmåga att bli varse om vad som händer utanför kabinen, på spåret.

Multi-User Virtual Environments (MUVEs) and Massively Multi-player On- line Games (MMOGs) are a popular, immersive genre of computer game. For some disabled users, eye-gaze offers the only input modality with the potential for sufficiently high bandwidth to support the range of time-critical interaction tasks required to play. Although, there has been much research into gaze interaction techniques for computer interaction over the past twenty years, much of this has focused on 2D desktop application control. There has been some work that investigates the use of gaze interaction as an additional input device for gaming but very little on using gaze on its own. Further, configuration of these techniques usually requires expert knowledge often beyond the capabilities of a parent, carer or support worker. The work presented in this thesis addresses these issues by the investigation of novel gaze-only interaction techniques. These are to enable at least a beginner level of game play to take place together with a means of adapting the techniques to suit an individual. To achieve this, a collection of novel gaze based interaction techniques have been evaluated through empirical studies. These have been encompassed within an extensible software architecture that has been made available for free download. Further, a metric of reliability is developed that when used as a measure within a specially designed diagnostic test, allows the interaction technique to be adapted to suit an individual. Methods of selecting interaction techniques based upon game task are also explored and a novel methodology based on expert task analysis is developed to aid selection.

From the start of their lives, infants and young children are surrounded by a tremendous amount of multimodal social information. One intriguing question in the study of early social cognition is how vital social information is detected and processed and how and when young infants begin to make sense of what they see and hear and learn to understand other people’s behavior. The overall aim of this thesis was to provide new insights to this exciting field. Investigating behavior and/or neural mechanisms in early life, the three different studies included in this thesis therefore strive to increase our understanding on perception and processing of social information. Study I used eye-tracking to examine infants´ observations of gaze in a third-party context. The results showed that 9-, 16- and 24-month-old infants differentiate between the body orientations of two individuals on the basis of static visual information. More particularly, they shift their gaze more often between them when the social partners face each other than when they are turned away from each other. Using ERP technique, Study II demonstrated that infants at the age of 4 to 5 months show signs of integrating visual and auditory information at a neural level. Further, direct gaze in combination with backwards-spoken words leads to earlier or enhanced neural processing in comparison to other gaze-word combinations. Study III, also an EEG investigation, found that children between 18 and 30 months of age show a desynchronization of the mu rhythm during both the observation and execution of object-directed actions. Also, the results suggest motor system activation when young children observe others’ mimed actions. To summarize, the findings reported in this thesis strengthen the idea that infants are sensitive to others´ gaze and that this may extend to third-party contexts. Also, gaze is processed together with other information, for instance words, even before infants are able to understand others’ vocabulary. Furthermore, the motor system in young children is active during both the observation and imitation of another person’s goal-directed actions. This is in line with findings in infants, children and adults, indicating that these processes are linked at neural level.

在現今的基於內容的圖像檢索的研究中，會將人的主觀認知考慮進去。因為傳統的圖像檢索中採取低階特徵來找出圖片上可能的重要區域的方法和人的感覺還是有著相當大的語意上的鴻溝。然而藉由考慮人對圖片的主觀認知，可以讓人找到對它而言圖片上重要的部分，再去做圖像檢索，找出使用者想要的圖片。這樣的作法是比較自然且直觀的。還能達到個人化的效果，因為每個人對同一張圖片上覺得重要的物體可能不盡相同。在本論文中的圖像檢索系統採用眼動軌跡當作人的主觀認知來輔助檢索。因為在心理學的研究中有提到，人在看圖片的時候會有較多的凝視點落在他覺得重要的區域上。所以藉由這個理論，本論文利用使用者看圖片的眼動軌跡即時的調整圖片上物體的重要性。最後將重要性高的數個物體去做圖像檢索，找出含有這些對這個使用者是重要的物體的圖片。經由實驗證實，眼動軌跡輔助圖像檢索的確可以減少不重要的物體對圖像檢索的干擾，繼而可以提升圖像檢索系統的效能。 / Recently, researches in Content-Based Image Retrieval (CBIR) focuses on incorporation of knowledge about human perception in the systems’ design and implementation process. This enables the design of more natural and intuitive image retrieval techniques in order to overcome some of the challenges faced by modern CBIR system such as the difficulty to extract important regions of an image. By researches of psychology, user’s eye tracking reflects his interest. So, in my CBIR system, user’s eye movements were used online to adjust the importance for objects in query image. Thus in my system, only those images with important objects will be retrieved. One experiment was performed: record the eye movement of participants on query images. Then compare my approach with a classic CBIR system according to performance. The results reveal that higher retrieval performance of my image retrieval system because of decreasing the influence of not importance objects to image retrieval system.

眼動儀目前的主要用途在分析使用者的觀看行為，藉以改善介面的設計，或幫助身體有缺陷但眼睛還能轉動的使用者與外界溝通。隨著相關技術的發展，眼動儀將可能如同滑鼠、鍵盤一般，成為使用者輸入裝置的選項。本論文的目的在於設計並實作低成本之穿戴式與遠距眼動儀，並將其應用於以視線軌跡為基礎的人機介面，希望能夠增進人與電腦之間的互動方式。由於眼動儀會受到雜訊與角膜反射點等的影響，本研究提出利用瞳孔周圍暗色點比例較高的特性，增加定位之準確性，以改善眼動儀之精確度，此外，頭部的移動亦會造成眼動儀在計算投射位置時之誤差，本研究也針對這個問題提出因應之解決方案。利用前述製作的眼動儀，本論文實作數個以視線軌跡為基礎的人機介面，包括視線軌跡網頁瀏覽器、強化眼睛注視照片區域、井字遊戲、互動式媒體等，並利用眼動儀記錄使用者觀看手機介面的行為。 / Eye tracker, a device for measuring eye position and movements, has traditionally been used for research in human visual system, psychology and interface design. It has also served as an input device for people with disability. With recent progresses in hardware and imaging technology, it has the potential to complement, even replace popular devices such as mouse or keyboard for average users to communicate with the computer. It is the objective of this research to design and implement low-cost head-mounted and remote eye trackers and subsequently develop applications that take advantage of gaze-based interactions. Specifically, we improve the precision of the tracking result by designing a new pupil detection algorithm as well as compensating for head movement. We then present several gaze-based user interfaces, including eye-controlled web browser, attention-based photo browser, interactive game (tic-tac-toe) and media design. We also investigate the feasibility of utilizing the eye trackers to analyze and evaluate the design of mobile user interface.