If researchers are able to derive usability simply by analysing gaze data it provides a quick, objective and potentially automatic way of measuring the usability of an interface. In order to do that it is essential to know which traits of the gaze data that have an impact on usability. This paper investigates these traits by analysing different eye tracking metrics in the data. The goal is to see which of these metrics have a general correlation with usability. Previous research provides a clue about which metrics are useful when analysing usability. However, much of the research is based on subjective analysis or lacks in general applicability. This research provides an objective analysis that is independent of characteristics of the interface. A user study is done on 20 participants. They complete tasks on nine different ecommerce websites while their eye movements are recorded. Correlation is measured between usability and eye tracking metrics in order to investigate which metrics that are sensitive to changes in usability. The results show that fixational backtracks and number of fixations have the strongest correlation with usability. Previous research did suggest that both of these eye tracking metrics have an impact on usability. / Om det är möjligt för forskare att härleda användbarhet endast genom att analysera ögondata så tillhandahålls ett snabbt, objektivt och potentiellt automatiserat sätt att mäta användbarheten hos ett gränssnitt. För att göra detta är det avgörande att veta vilka karakteristiska drag i ögondatan som påverkar användbarheten. Denna rapport utforskar dessa drag genom att analysera olika mått i ögondatan. Målet är att se vilka av dessa mått som uppvisar en generell korrelation med användbarhet. Tidigare forskning förser oss med ledtrådar om vilka mått som är lämpliga att analysera när användbarhet ska mätas. Dock så är mycket av denna forskning baserad på subjektiv analys eller saknar generell tillämpbarhet. Denna rapport tillhandahåller en objektiv analys som är oberoende av karakteristiken hos gränssnittet. En användarstudie utförs på 20 deltagare. De utför uppgifter på nio olika webbsidor för e-handel medan deras ögonrörelser spelas in. Korrelation mäts mellan användbarhet och mått i ögondatan för att undersöka vilka mått som är känsliga för förändringar i användbarhet. Resultatet visar att tillbakabildande sackader och antal fixeringar har starkast korrelation med användbarhet. Tidigare forskning visade att båda dessa mått påverkades av användbarheten.

Research has demonstrated that everyday functional abilities are compromised in mild cognitive impairment (MCI), a transitional stage between normal cognitive aging and dementia, as well as in healthy aging. These functional changes have been shown to be strong predictors of future decline, highlighting their importance. However, early changes in everyday functioning remain poorly characterized, largely due to a scarcity of sensitive measures capable of detecting subtle disruption. Recent research suggests that eye-tracking methodology may be effective in addressing this gap. Fifty-two participants (27 younger adults and 25 non-demented older adults) completed a novel eye-tracking task involving passive viewing of a naturalistic scene and verbalization of a task goal (e.g., make coffee, pack a lunch). Participants also completed a performance-based measure of everyday action that required them to enact the same tasks (e.g., coffee, lunch) that were included in the eye-tracking paradigm, self-report measures of functional ability, and neuropsychological measures. Mixed ANOVAs were conducted to examine group (young, old) and condition (passive viewing, verbalization)/task (simple, complex) effects on eye-tracking and everyday action performance. Independent samples t-tests/Mann-Whitney U tests were conducted to examine group differences in eye-tracking and everyday action performance. Correlation analyses across all measures were conducted to evaluate the potential mechanisms of eye-tracking and everyday action results. Results showed no significant group differences in the primary eye-tracking variables, but both groups made a lower proportion of fixations to distractor (i.e., non-target) objects during task verbalization compared to passive scene viewing. Older adults made more inefficient actions during performance-based everyday task completion, particularly when task demands were high. Eye tracking and everyday action variables were related to different measures of self-reported functional ability. Finally, eye-tracking variables were primarily related to neuropsychological measures of executive functions/working memory, whereas everyday action performance was most strongly related to measures of verbal learning and memory. These findings suggest that age-related functional changes at the level of eye movements may occur after changes in behavioral performance of everyday tasks. Importantly, performance-based assessment of everyday action appears sensitive to age-related decline. Additionally, naturalistic eye movements and everyday task performance may reflect distinct components of self-reported functioning and may be driven by distinct cognitive processes. Future research with refined naturalistic eye-tracking tasks and samples with a wider range of impairment is necessary to further explore these findings and improve characterization and detection of risk for dementia. / Psychology

[pt] O rastreamento ocular (eye-tracking) possibilita rastrear a posição e a direção do olhar de uma pessoa sobre algum tipo de estímulo (e.g., imagens ou vídeos). O uso desta tecnologia permite identificar eventos inerentes à visão humana, que contém informações implícitas capazes de revelar aspectos importantes sobre o comportamento de um indivíduo durante uma determinada tarefa. Porém, identificar essas informações requer um conjunto de habilidades para interpretar os dados de rastreamento ocular e relacioná-los com conhecimentos de domínios específicos. Nesse contexto, pode-se fazer grande proveito de sistemas inteligentes para agregar os conhecimentos e experiências de especialistas junto às respostas do dispositivo de rastreamento ocular. Dessa forma, o objetivo principal deste trabalho é propor uma metodologia para criar sistemas baseados em eye-tracking, para enriquecer o processo de avaliação de um indivíduo frente a uma determinada tarefa, resultando em um modelo para representar o conhecimento dos especialistas sobre aspectos subjetivos, visando automatizar esse processo avaliativo. Portanto, o presente trabalho toma como caso de uso a avaliação da relação entre comportamento visual e eficácia de indivíduos na resolução de testes inspirados em Matrizes Progressivas de Raven. Esses testes são comumente usados na psicologia para medir inteligência e a capacidade de raciocínio abstrato a partir da visualização de imagens. Optamos por utilizar uma abordagem baseada em regras fuzzy, por permitir armazenar conhecimento de forma mais transparente e legível aos usuários finais. As regras do modelo foram desenvolvidas e validadas com o auxílio de um especialista da área da psicologia. O sistema foi testado com dados extraídos de um grupo de usuários e apresentou resultados promissores. Os achados e modelos obtidos nessa pesquisa poderão ser utilizados como alicerce para o desenvolvimento de sistemas mais robustos. / [en] Eye-tracking makes it possible to track the position and direction of a person s gaze on some stimulus (e.g., images or videos). This technology allows us to identify events inherent to human vision, containing implicit information capable of revealing essential aspects of one s behavior during a given task. However, identifying these pieces of information is a complex task that requires a set of skills to interpret the eye-tracking data and relate it to domain-specific knowledge. In this context, one can use intelligent systems to couple the knowledge and experience of specialists with the responses from the eye-tracking device. Thus, the main objective of this work is to propose a methodology to create eye-tracking-based systems to improve the assessment of subjects during specific tasks, resulting in a model that can represent the specialist s knowledge over subjective aspects to automate this process. Therefore, the present work s use case is the evaluation of the relationship between visual behavior and efficiency in solving tests inspired by Raven s Progressive Matrices. Those tests are commonly used in psychology to measure intelligence and abstract reasoning through image visualization. We chose an approach based on fuzzy rules, as it allows us to represent knowledge in a more readable way to end-users. The model s rules were developed and validated alongside a specialist in psychology. The system was tested with data extracted from users and showed promising results. The findings and models obtained in this research may be used as a foundation for the development of more robust systems.

Yes / Recent research has shown that adults with autism spectrum disorders (ASD) have difficulty interpreting others' emotional responses, in order to work out what actually happened to them. It is unclear what underlies this difficulty; important cues may be missed from fast paced dynamic stimuli, or spontaneous emotional responses may be too complex for those with ASD to successfully recognise. To explore these possibilities, 17 adolescents and adults with ASD and 17 neurotypical controls viewed 21 videos and pictures of peoples' emotional responses to gifts (chocolate, a handmade novelty or Monopoly money), then inferred what gift the person received and the emotion expressed by the person while eye movements were measured. Participants with ASD were significantly more accurate at distinguishing who received a chocolate or homemade gift from static (compared to dynamic) stimuli, but significantly less accurate when inferring who received Monopoly money from static (compared to dynamic) stimuli. Both groups made similar emotion attributions to each gift in both conditions (positive for chocolate, feigned positive for homemade and confused for Monopoly money). Participants with ASD only made marginally significantly fewer fixations to the eyes of the face, and face of the person than typical controls in both conditions. Results suggest adolescents and adults with ASD can distinguish subtle emotion cues for certain emotions (genuine from feigned positive) when given sufficient processing time, however, dynamic cues are informative for recognising emotion blends (e.g. smiling in confusion). This indicates difficulties processing complex emotion responses in ASD.

In the evolving landscape of building systems and human-building interaction (HBI), the complexity of building interfaces has significantly increased, posing both challenges and opportunities for enhancing energy consumption, indoor environmental quality (IEQ), and building services. This dissertation, titled "User-centered Evaluation of Multi-modal Building Interfaces," delves into the realm of HBI by focusing on the user's experience and perception of multimodal building control interfaces, particularly the various visual modalities of Connected Thermostats (CTs). This body of work aims to support CTs' ongoing adoption, expansion, and performance through a user-centered perspective. The research is motivated by the observation that the design process in the current building industry often overlooks a human-centered approach, leading to a disconnection between occupants' needs and building interface design. This misalignment not only results in user dissatisfaction but also leads to a missed opportunity in leveraging smart building technologies to enhance building performance for achieving climate change mitigation goals. This research attempts to address the main identified gaps in the literature and AEC industry concerning 1) human interaction and perception of multimodal CT interfaces,2) the scarcity of knowledge in the field of human-computer-building interaction (HCBI) regarding the user study methods, 3) the exiting highly non-standard practices in the design of building interfaces. This research highlights 1) the necessity of a multimodal interaction approach, 2) robust mixed-methods User Experience (UX) summative evaluation studies, and 3) the need for standardization in HCBI. This body of work is grounded in the Technology Acceptance Model (TAM) and Human Information Processing (HIP) theories, aiming to foster the adoption of connected building controls with a special focus on usability by suggesting best practices in design and research. The methodology comprised three-step mixed-methods summative evaluation studies designed using a funnel approach to answer the general question: "How do users interact with connected thermostats, and how do these interactions inform our understanding of human-building interaction?": 1) The first and broadest study leveraged texting mining big data of user reviews to identify the general themes and patterns that affect the UX and acceptance of CTs. 2) The second study employed mixed-methods lab experiments to further focus on usability, being recognized as the most determining factor in the adoption of CTs in the first study. This study investigated human interaction with three of the most prevalent modalities of CTs: the Fixed Visual Display (FVD), the phone app, and the web portal. 3) The third study investigated human interaction with a specific visual aspect of UI of FVD and phone app modalities, the interface icons, with the goal of providing some data-driven guidelines for their standardization. Throughout the three studies, the dissertation employed and evaluated some novel and established HCI summative user evaluation methods, including a grounded theory approach for text mining and analyzing user-generated content, eye-tracking think-aloud protocol and contextual inquiry, A/B testing and NASA TLX and SUS surveys to evaluate UX, usability and mental workload. The dissertation outlined three discrete contributions: 1) It bridged some of the well-established UX research methods into HCBI and highlighted the potential of knowledge in the HCI field, 2) Provided guidance for human-centered design of multimodal building interfaces through identifying the main strengths, weaknesses, opportunities, and threats in UX of CTs, 3) Informed the standardization of UI of multimodal building interfaces. / Doctor of Philosophy / Modern building systems are becoming increasingly complex to enhance energy savings and improve occupant convenience. Traditional buildings are typically controlled by wall-mounted devices that manage lighting, temperature, and security. However, in smart buildings, occupants now have new ways to interact with these systems, including remote control options like smartphone apps. This research explores how people interact with and perceive different types of building controls. Specifically, it compares user experiences with two main types of controls: fixed controls, such as wall-mounted thermostats, and mobile controls, like smartphone apps. The goal is to improve the design of these controls to make them easier to use, widely adopted, and more effective in promoting energy efficiency and comfort. Among building controls, smart residential thermostats are particularly important. How they are designed and how people use them can significantly impact energy use in homes. However, advanced thermostats, especially those with programming features, are often seen as difficult to use for building occupants. Designers sometimes overlook the needs of users due to agile design processes or a limited understanding of user-centered design. These complex interfaces can lead to user frustration and missed opportunities to improve building performance. This research addresses these challenges by studying how people actually interact with different smart thermostat controls, including wall-mounted devices, phone apps, and web portals, emphasizing the importance of designing with user needs in mind. This study identifies the common issues users face with the dashboard of these devices, proposes solutions, and offers best practices for creating user-friendly building controls. This study makes three key contributions: First, it introduces well-known methods from other fields for designing and evaluating technology and applies these methods to evaluate the user experience of building controls. Methods include text analysis of user reviews, surveys, eye-tracking, and user performance measurements, some of which are relatively new in the context of human-building interaction. Additionally, the study provides guidance for future researchers in developing a user-centered framework for building interfaces by comparing these methods. Finally, it helps designers by identifying critical usability problems in building interfaces and highlighting opportunities for improvement.

Данная работа посвящена созданию модели, при помощи которой может осуществляться классификация эмоций для дальнейшего анализа психофункционального состояния человека с использованием машинного обучения. Цель исследования заключалась в разработке системы оценки психофункционального состояния человека на основе биосигнала Eye tracking с помощью моделей машинного обучения. Для достижения поставленной цели предполагается провести анализ современных методов машинного обучения для задачи оценки психофункционального состояния человека, собрать и обработать собственный набор данных, применить алгоритмы и модели машинного обучения к выбранному открытому набору данных и собственному набору данных, получить оценки точности и сравнить полученные результаты. Сформировать выводы. Разработка системы позволит производить оценку психофункционального состояния человека на основе данных Eye tracking, путем определения изменений в образце движения глаз. Помочь в диагностике и мониторинге психических заболеваний, и сократить время проведения и обработки тестов на выявление психических отклонений врачами. / This work is devoted to the creation of a model by which the classification of emotions can be carried out for further analysis of the psychofunctional state of a person using machine learning. The aim of the study was to develop a system for assessing the psycho-functional state of humans based on the Eye tracking biosignature using machine learning models. In order to achieve this goal, it is planned to conduct an analysis of modern methods of machine learning for the purpose of assessing the psychofunctional state of a person, to collect and process its own set of data, Apply algorithms and machine learning models to the selected open data set and its own data set, obtain accuracy estimates and compare the results. Draw conclusions. The development of the system will allow to assess the psycho-functional state of a person based on Eye tracking data, by determining changes in the pattern of eye movement. Assist in the diagnosis and monitoring of mental illness, and reduce the time for conducting and processing of tests for the detection of mental disorders by doctors.

Das Verständnis über visuelle Aufmerksamkeitsprozesse ist nicht nur für die Kognitionsforschung von großem Interesse. Auch in alltäglichen Bereichen des Lebens stellt sich die Frage, wie wir unsere Umwelt in unterschiedlichen Situationen visuell wahrnehmen. Entsprechende Untersuchungen können in realen Szenarien und aufgrund neuer innovativer 3D-Verfahren auch in Umgebungen der virtuellen Realität (VR) durchgeführt werden. Zur Erforschung von Aufmerksamkeitsprozessen wird unter anderem die Methode der Blickbewegungsmessung (Eyetracking) angewandt, da das Sehen für uns Menschen die wichtigste Sinnesmodalität darstellt. Herkömmliche Blickbewegungsmessungen beziehen sich allerdings überwiegend auf zweidimensionale Messebenen, wie Bildschirm, Leinwand oder Szenevideo. Die vorliegende Arbeit stellt eine Methode vor, mit der dreidimensionale Blickorte und Blickbewegungen sowohl in einer realen als auch in einer stereoskopisch projizierten VR-Umgebung anhand moderner Eyetracking-Technologien bestimmt werden können. Dafür wurde zunächst in Studie I geprüft, ob die Blickbewegungsmessung durch die für eine stereoskopische Bildtrennung notwendigen 3D-Brillen hindurch möglich ist und inwieweit durch diesen Versuchsaufbau die Qualität der erhobenen Eyetracking-Daten beeinflusst wird. Im nächsten Schritt wurde zur Berechnung dreidimensionaler Blickorte das Anforderungsprofil an einen universellen Algorithmus erstellt und mit einem vektorbasierten Ansatz umgesetzt. Die Besonderheit hierbei besteht in der Berechnung der Blickvektoren anhand der Augen- bzw. Foveaposition und binokularen Eyetracking-Daten. Wie genau dreidimensionale Blickorte anhand dieses Algorithmus berechnet werden können, wurde nachfolgend in realer (Studie II) als auch stereoskopisch projizierter VR-Umgebung (Studie III) untersucht. Anschließend erfolgte die Bestimmung dreidimensionaler Blickbewegungen aus den berechneten 3D-Blickorten. Dazu wurde ein ellipsoider Fixationserkennungsalgorithmus konzipiert und implementiert. Für die dispersionsbasierte Blickbewegungserkennung waren sowohl ein zeitlicher als auch örtlicher Parameter für die Identifikation einer Fixation erforderlich. Da es noch keinerlei Erkenntnisse im dreidimensionalen Bereich gab, wurden die in Studie II und III ermittelten 3D-Blickorte der ellipsoiden Fixationserkennung übergeben und die daraus berechneten Fixationsparameter analysiert. Die entwickelte Methode der räumlichen Blickbewegungsmessung eröffnet die Möglichkeit, bislang in zwei Dimensionen untersuchte Blickmuster nunmehr räumlich zu bestimmen und grundlegende Zusammenhänge zwischen Blickbewegungen und kognitiven Prozessen dreidimensional sowohl in einer realen als auch virtuellen Umgebung zu analysieren.

本研究旨在探討國內三大電子商務網站之商品展示頁面的版面配置與瀏覽動線之間的關係，藉由眼動追蹤儀器實驗(Eye Movement Tracking)及專家訪談的結果，歸納出最後的結論與建議方向，並作為未來實務界在設計與規劃電子商務網站時的參考依據。 在實驗研究中，共分為兩個階段進行測試，由23位受測者進行觀看，有效樣本20人。實驗一主要是測試三家電子商務網站的商品瀏覽效率。透過實驗找出哪一家的商品版面配置與動線規劃，可以讓使用者在短時間內快速地找到所要的商品以及在找尋商品的過程中，受測者的凝視焦點分佈與凝視停留時間為何。 而實驗二的部份，則是測試商品的版面配置情形。主要是操弄三家網站之不同的商品配置特性，測試不同的商品配置特性對使用者的視覺焦點(Fixation Count)與停留時間(Fixation Time)所產生的變化，試圖瞭解何種商品配置最吸引使用者的目光，讓使用者停留更多的瀏覽時間。 並透過專家訪談法，瞭解業界資深的網頁設計師及網站規劃者他們對於電子商務網站之商品展示頁的版面配置與使用者瀏覽動線的看法與建議，以及在規劃及設計自家電子商務網站時的構思想法，最後將訪談資料以內容分析法予以歸納分析，供業界在執行電子商務網站實務設計時之參考依據。 / This study aims to investigate the relationship between the layout and browsing routes for merchandise display pages on Taiwan’s three major e-commerce websites, as ascertained by results from eye tracking experiments and expert interviews. The study offers some overall conclusions and suggestions which may serve as a reference for future practices in designing and implementing e-commence websites. The experiment was divided into two stages, and included 23 participants whose viewing behavior was recorded; the effective sample pool was 20. The objective of the first test was to examine the merchandise viewing efficacy rendered by these three e-commerce websites. Through this experiment, the website with the merchandise page setup and browsing routes which enabled users to locate their desired merchandise in the shortest time was determined, as well as the samples’ visual fixation distributions and durations. In the second experiment, merchandise page layouts were tested. Different arrangement schemes of merchandise displayed by these same three websites were manipulated to test the effects they have on user eye fixation count and fixation time, in an attempt to understand what type of merchandise display arrangement scheme draws the most attention from viewers and encourages them to spend more viewing time. Expert interviews provided insights and suggestions from skilled web page designers and website builders regarding their perspectives on merchandise display page arrangement and user browsing routes, and in particular, the concepts employed when building their own e-commerce websites. Finally, interview data is introduced and analyzed using content analysis methodology, which may serve as a reference for e-commerce website design practices.

L’objectif de cette thèse est d’étudier l’impact de la fatigue du consommateur sur son traitement de l’information ublicitaire. Plus précisément, nous désirons comprendre comment la fatigue influence le processus de traitement visuel de la publicité, ainsi que les attitudes et la mémorisation. En nous appuyant sur la théorie d’adéquation des ressources, nous postulons que le traitement de l’information publicitaire, et donc l’efficacité de la publicité, est optimal quand le niveau de ressources cognitives disponible (niveau de fatigue du consommateur) correspond au niveau de ressources exigé (niveau de complexité de la publicité). Nos résultats mettent en lumière des effets de la fatigue différents selon les deux échantillons de notre recherche (étudiants et salariés). Les étudiants fatigués adoptent une stratégie visuelle de balayage pour compenser les effets de la fatigue et rendre le traitement de l’information plus facile. Les salariés fatigués adoptent une stratégie visuelle d’attention soutenue consistant à mobiliser le peu de ressources disponibles pour traiter l’information de façon efficace. Cette stratégie visuelle d’attention soutenue a un impact positif sur la mémorisation de la publicité. Nous montrons enfin que la théorie d’adéquation des ressources n’est pas le cadre théorique approprié pour étudier notre objet de recherche. En effet, les ressources cognitives s’avèrent être dynamiques : si elles y sont motivées, les personnes peuvent adapter leur niveaude ressources à la tache à accomplir. Les implications théoriques, méthodologiques et managériales de ces résultats sont discutées. / This research aims at studying the impact of consumers’ tiredness on the way they perceive and process advertising information. More specifically, we try to understand how tiredness influences advertisements’ visual processing as well as consumers’ attitudes and memorization. Relying on Resource Matching theory, we hypothesize that advertising informationprocessing, and therefore advertising efficiency, is optimal when the level of available resources (consumers’ tiredness) matches the level of required resources (advertisement complexity). Results show that the impact of tiredness differs according to sample under consideration (students or staff members). Tired students adopt a “screening” visual strategyin order to compensate for tiredness effects and ease their information processing. Tired staff members adopt a “sustained attention” visual strategy consisting in mobilizing their few available resources in order to efficiently process information. In this second case, sustained visual attention leads to better advertising memorization. We show that the Resource Matching theory is not the appropriate theoretical framework to study our research object. Indeed, we find that cognitive resources are dynamic:if motivated to do so, individuals are able to adapt their resource level according to the processing task. Theoretical, methodological and managerial implications are discussed.

Motor and communication disabilities are common conditions that may implicate restrictions in daily life. With development of eye tracking technology, a solution referred to as eye-gaze interaction has been generated to support people with their limiting conditions to solve communication and computer access issues. By using eye tracking technology, which calculates the user’s eye-gaze location on a computer screen, user’s are able to control computers with their eyes as an input. This interaction method is quite unique and complex since the eyes serves both as an input and output source. Usability aspects revolving human information processing are therefore important to consider when designing user interfaces. In collaboration with Tobii AB, the study evaluated two separate eye-gaze interaction systems for controlling computers. 7 participants conducted user tests, one for each application, and answered interview questions during the tests regarding their usability experience. Based on the collected data,17 design guidelines was established with a purpose to enhance usability for eye-gaze interaction systems.