This thesis deals with the eye gaze tracking and determining the direction and the place of the view. The overview of methods for determining the view direction is provided here. The direct 3D gaze estimation technique is proposed and implemented. The eye gaze can be estimated by using a single camera and under nature head movement. This paper describes the algorithms for the image processing and the image description. The center of the pupil is determined by the genetic algorithm for the ellipse detection. The procedures of the instrumentation calibration are proposed here as well. The work also addresses the issue of the determining the eye location. Finally, the comparison of the accuracy with similar projects is mentioned in my thesis.

Embodied Conversational Agents (ECAs)’ gestures can enhance human perception in many dimensions during interactions. In recent years, data-driven gesture generation approaches for ECAs have attracted considerable research attention and effort, and methods have been continuously optimized. Researchers have typically used human-agent interaction for user studies when evaluating systems of ECAs that generate rule-based gestures. However, when evaluating the performance of ECAs that generate gestures based on data-driven methods, participants are often required to watch prerecorded videos, which cannot provide an adequate assessment of human perception during the interaction. To address this limitation, we proposed two main research objectives: First, to explore the workflow of assessing data-driven gesturing ECAs through real-time interaction. Second, to investigate whether gestures could affect ECAs’ human-likeness, animacy, perceived intelligence, and humans’ focused attention in ECAs. Our user study required participants to interact with two ECAs by setting two experimental conditions with and without hand gestures. Both subjective data from the participants’ self-report questionnaire and objective data from the gaze tracker were collected. To our knowledge, the current study represents the first attempt to evaluate data-driven gesturing ECAs through real-time interaction and the first experiment using gaze-tracking to examine the effect of ECA gestures. The eye-gazing data indicated that when an ECA can generate gestures, it would attract more attention to its body. / Förkroppsligade konversationsagenter (Embodied Conversational Agents, ECAs) gester kan förbättra människans uppfattning i många dimensioner under interaktioner. Under de senaste åren har datadrivna metoder för att generera gester för ECA:er fått stor uppmärksamhet och stora ansträngningar inom forskningen, och metoderna har kontinuerligt optimerats. Forskare har vanligtvis använt sig av interaktion mellan människa och agent för användarstudier när de utvärderat system för ECA:er som genererar regelbaserade gester. När man utvärderar prestandan hos ECA:er som genererar gester baserat på datadrivna metoder måste deltagarna ofta titta på förinspelade videor, vilket inte ger en adekvat bedömning av människans uppfattning under interaktionen. För att åtgärda denna begränsning föreslog vi två huvudsakliga forskningsmål: För det första att utforska arbetsflödet för att bedöma datadrivna ECA:er för gester genom interaktion i realtid. För det andra att undersöka om gester kan påverka ECA:s människoliknande, animerade karaktär, upplevd intelligens och människors fokuserade uppmärksamhet i ECA:s. I vår användarstudie fick deltagarna interagera med två ECA:er genom att ställa in två experimentella villkor med och utan handgester. Både subjektiva data från deltagarnas självrapporterande frågeformulär och objektiva data från gaze tracker samlades in. Såvitt vi vet är den aktuella studien det första försöket att utvärdera datadrivna ECA:er med gester genom interaktion i realtid och det första experimentet där man använder blickspårning för att undersöka effekten av ECA:s gester. Uppgifterna om blickspårning visade att när en ECA kan generera gester skulle den locka mer uppmärksamhet till sin kropp.

Den ständigt ökande mängden av smarta enheter i vår vardag har lett till nya problem inom HCI så som hur vi människor ska interagera med dessa enheter på ett effektivt och enkelt sätt. Än så länge har kontextuellt medvetna system visat sig kunna vara ett möjligt sätt att lösa detta problem. Om ett system hade kunnat automatiskt detektera personers aktiviteter och avsikter, kunde det agera utan någon explicit inmatning från användaren. Ögon har tidigare visat sig avslöja mycket information om en persons kognitiva tillstånd och skulle kunna vara en möjlig modalitet för att extrahera aktivitesinformation ifrån.I denna avhandling har vi undersökt möjligheten att detektera aktiviteter genom att använda en billig, hemmabyggd ögonspårningsapparat. Ett experiment utfördes där deltagarna genomförde aktiviteter i ett kök för att samla in data om deras ögonrörelser. Efter experimentet var färdigt, annoterades, förbehandlades och klassificerades datan med hjälp av en multilayer perceptron--och en random forest--klassificerare.Trots att mängden data var relativt liten, visade resultaten att igenkänningsgraden var mellan 30-40% beroende på vilken klassificerare som användes. Detta bekräftar tidigare forskning att aktivitetsigenkänning genom att analysera ögonrörelser är möjligt. Dock visar det även att det fortfarande är svårt att uppnå en hög igenkänningsgrad. / The increasing amount of smart devices in our everyday environment has created new problems within human-computer interaction such as how we humans are supposed to interact with these devices efficiently and with ease. So far, context-aware systems could be a possible candidate to solve this problem. If a system automatically could detect people's activities and intentions, it could act accordingly without any explicit input from the user. Eyes have previously shown to be a rich source of information about a person's cognitive state and current activity. Because of this, eyes could be a viable input modality for extracting information from. In this thesis, we examine the possibility of detecting human activity by using a low cost, home-built monocular eye tracker. An experiment was conducted were participants performed everyday activities in a kitchen to collect eye movement data. After conducting the experiment, the data was annotated, preprocessed and classified using multilayer perceptron and random forest classifiers.Even though the data set collected was small, the results showed a recognition rate of between 30-40% depending on the classifier used. This confirms previous work that activity recognition using eye movement data is possible but that achieving high accuracy is challenging.

No description available.

A presente tese de doutorado insere-se na área de interação pelo olhar. A interação pelo olhar é uma forma de comunicação com o computador utilizando os movimentos oculares do usuário. Pessoas com deficiência física, que não conseguem usar dispositivos convencionais como um teclado e um mouse de computador, podem se beneficiar da interação pelo olhar para se comunicarem e se inserirem na sociedade. Para isso a entrada de texto ou digitação pelo olhar é um recurso importante e assunto principal dessa tese. O instrumento mais comum para entrada de texto pelo olhar consiste de um teclado virtual onde os caracteres são selecionados por tempo de latência. Essa forma de interação, embora simples, sofre de seleções involuntárias (problema conhecido como toque de Midas) se o tempo de latência for curto (menos de 500 ms). Já para tempos de latência mais longos, a interação se torna lenta. Alternativas para entrada de texto pelo olhar são os gestos discretos ou contínuos do olhar. O uso de gestos discretos permite reduzir o toque de Midas, porém o desempenho é inferior ao tempo de latência. Já nos métodos baseados em gestos contínuos, o olhar está sempre preso ao controle da interface. Uma técnica de interação proposta recentemente, chamada de \"alternância entre contextos\", permite reduzir o efeito do toque de Midas, utilizando apenas uma sacada para cada seleção. Além disso, essa técnica permite aos usuários manterem o ritmo de interação sem ajustar nenhum parâmetro na interface. A presente tese de doutorado visa melhorar a usabilidade e experiência dos usuários na interação pelo olhar com teclados virtuais. Os objetivos específicos são: investigar a relação entre a manipulação do contraste dos estímulos visuais e o tempo de reação sacádico para facilitar os movimentos oculares e tornar a interação mais rápida e agradável; propor e investigar novas extensões e aplicações da alternância entre contextos, visando reduzir o toque de Midas e ao mesmo tempo generalizar o método para outras tarefas de navegação e seleção de objetos pelo olhar; e desenvolver novos métodos de entrada de texto pelo olhar para melhorar a velocidade de digitação dos usuários, sem incrementar a carga de trabalho e mantendo a interação simples e fácil de aprender. A avaliação dos novos métodos e modelos propostos foi feita por meio de vários estudos com usuários. Os dados coletados nos estudos, tanto quantitativos quanto qualitativos, foram analisados com métodos estatísticos utilizados na área de interação homem-computador. As contribuições originais apresentadas na presente tese são: a proposta e a avaliação do efeito gap gradiente como feedback visual para facilitar a execução de movimentos sacádicos durante a interação pelo olhar; a proposta e investigação de contextos dinâmicos como extensão da alternância entre contextos, para permitir um melhor aproveitamento da área útil do monitor com uma baixa taxa de erros de seleção, assim como de meta-keys para navegação e execução de comandos de forma geral; e a proposta e a avaliação de AugFix, um novo modelo de feedback visual que melhora a velocidade e a experiência dos usuários na entrada de texto pelo olhar, com aplicação em teclados virtuais baseados nos paradigmas do tempo de latência e a alternância entre contextos. / This PhD thesis lays in the context of gaze-based interaction. Gaze-based interfaces allows the user to control a computer using his/her eye movements. Gaze interaction is specially useful for people with physical disabilities who cannot use conventional devices (such as a keyboard and/or mouse) to communicate. Entering text by gaze (also known as eye typing) is a very important activity and the main focus of this dissertation. The most common eye typing technique uses a virtual keyboard, where the letters are selected by dwell time. Though simple, this interaction technique suffers from involuntary activations (known as the Midas\' touch problem) for short dwell times (shorter than 500 ms). On the other hand, with longer dwell times the interaction becomes slow. Alternatives to dwell time are discrete and continuous gaze gestures. The use of discrete gaze gestures reduces the Midas\' touch problem, but its performance is slower compared to dwell time. When using continuous gaze gestures, the user gaze is always controlling the interface. A recently proposed technique called \"context switching\" avoids the Midas touch problem by mapping selection to a single saccade. Furthermore, with this technique the users can keep their rhythm of interaction without adjusting any parameter of the interface. This PhD thesis aims at improving the usability and user experience in gaze interaction with virtual keyboards. The objectives are: to investigate the relationship between contrast manipulation of visual stimuli and saccadic reaction time, to facilitate eye movements and make the interaction faster and more comfortable; to propose and investigate new extensions and applications of the context switching paradigm, in order to reduce the Midas touch problem and generalize the extensions to other tasks such as browsing and selection; and to develop new methods of eye typing that can improve typing performance without increasing the user workload, while keeping the interaction simple and easy to learn. Several user studies were designed and conducted to evaluate the methods and models proposed in this thesis. The data (both quantitative and qualitative) collected in the experiments is rigorously analysed using statistical methods. The original contributions of this thesis are: the proposal and investigation of the gradient gap effect as a visual feedback to facilitate the execution of saccadic movements during gaze interaction; the proposal and investigation of two extensions to the context switching paradigm: dynamic contexts that improve the utilization of the screen space while keeping a low selection error rate, and meta-keys for browsing and executing general-purpose commands; and the proposal and evaluation of AugFix, a new method that improves eye typing speed and user experience without increasing the cognitive load, that can be used with virtual keyboards controlled by both dwell time and context switching.

隨著互動式資料視覺化工具越來越多，設計者需要一個方法來衡量其作品是否好用、能否被理解、使用效率高低。互動式資料視覺化需要透過使用者的互動才能觀察到資料的不同面向，再進一步產生洞見，然而現有的評估方式多僅聚焦於靜態資料圖表，設計者無法從中得知使用者的操作困難之處，並據此進行加強與改善，因此本研究提出一個整合量化分析與質化記錄的系統性評估方式，應用於互動式資料視覺化的優使性(usability)分析。 本研究的方法為追蹤使用者的眼動和滑鼠操作過程，先將其記錄成量化數據，透過興趣區域的標定與將轉換使用者行為成序列後，進行序列運算和統計分析；同時，從使用者經驗研究方法得到實驗過程的質化記錄，用來輔助解釋量化分析的結果。 本論文藉由兩個互動式資料視覺化工具來驗證以眼動與滑鼠追蹤評估互動式資料視覺化是可行的，我們提出了具體的實驗流程、量化紀錄與分析方式，並建議以下評估指標：吸引力、易發現性、困難度、易識別性、易理解性、精準表達程度、細部困難度、使用效率。 / As more and more interactive data visualization tools emerge, designers need an organized evaluation method to provide timely feedback and understand user behavior. In contrast to traditional graphical presentations, interactive data visualization tools call for user manipulation to gain specific insights. It is therefore imperative to study the intermediate operation process, rather than the final outcome, to provide a critical understanding of the developed tool. Toward this objective, we propose a systematic approach combining quantitative analysis and qualitative assessment to gauge the usability of interactive data visualization tools in this research. Firstly, quantitative data including gaze and mouse movements are collected. By combining the definition of area of interest, these trajectories can be converted into user sequences, which are conveniently accessible for further statistical analysis as well as path comparison. Secondly, qualitative information obtained by observing user operation is gathered to offer additional insight and complement/support conclusions obtained from quantitative analysis. Two interactive data visualization tools are employed to examine the feasibility and universality of our experimental and analytical procedure. To conclude, we come up with several key indicators to evaluate interactive data visualization, including attraction, discoverability, difficulty, identifiability, comprehensibility, precision of expression, difficulty(detailed) and efficiency.

A presente tese de doutorado insere-se na área de interação pelo olhar. A interação pelo olhar é uma forma de comunicação com o computador utilizando os movimentos oculares do usuário. Pessoas com deficiência física, que não conseguem usar dispositivos convencionais como um teclado e um mouse de computador, podem se beneficiar da interação pelo olhar para se comunicarem e se inserirem na sociedade. Para isso a entrada de texto ou digitação pelo olhar é um recurso importante e assunto principal dessa tese. O instrumento mais comum para entrada de texto pelo olhar consiste de um teclado virtual onde os caracteres são selecionados por tempo de latência. Essa forma de interação, embora simples, sofre de seleções involuntárias (problema conhecido como toque de Midas) se o tempo de latência for curto (menos de 500 ms). Já para tempos de latência mais longos, a interação se torna lenta. Alternativas para entrada de texto pelo olhar são os gestos discretos ou contínuos do olhar. O uso de gestos discretos permite reduzir o toque de Midas, porém o desempenho é inferior ao tempo de latência. Já nos métodos baseados em gestos contínuos, o olhar está sempre preso ao controle da interface. Uma técnica de interação proposta recentemente, chamada de \"alternância entre contextos\", permite reduzir o efeito do toque de Midas, utilizando apenas uma sacada para cada seleção. Além disso, essa técnica permite aos usuários manterem o ritmo de interação sem ajustar nenhum parâmetro na interface. A presente tese de doutorado visa melhorar a usabilidade e experiência dos usuários na interação pelo olhar com teclados virtuais. Os objetivos específicos são: investigar a relação entre a manipulação do contraste dos estímulos visuais e o tempo de reação sacádico para facilitar os movimentos oculares e tornar a interação mais rápida e agradável; propor e investigar novas extensões e aplicações da alternância entre contextos, visando reduzir o toque de Midas e ao mesmo tempo generalizar o método para outras tarefas de navegação e seleção de objetos pelo olhar; e desenvolver novos métodos de entrada de texto pelo olhar para melhorar a velocidade de digitação dos usuários, sem incrementar a carga de trabalho e mantendo a interação simples e fácil de aprender. A avaliação dos novos métodos e modelos propostos foi feita por meio de vários estudos com usuários. Os dados coletados nos estudos, tanto quantitativos quanto qualitativos, foram analisados com métodos estatísticos utilizados na área de interação homem-computador. As contribuições originais apresentadas na presente tese são: a proposta e a avaliação do efeito gap gradiente como feedback visual para facilitar a execução de movimentos sacádicos durante a interação pelo olhar; a proposta e investigação de contextos dinâmicos como extensão da alternância entre contextos, para permitir um melhor aproveitamento da área útil do monitor com uma baixa taxa de erros de seleção, assim como de meta-keys para navegação e execução de comandos de forma geral; e a proposta e a avaliação de AugFix, um novo modelo de feedback visual que melhora a velocidade e a experiência dos usuários na entrada de texto pelo olhar, com aplicação em teclados virtuais baseados nos paradigmas do tempo de latência e a alternância entre contextos. / This PhD thesis lays in the context of gaze-based interaction. Gaze-based interfaces allows the user to control a computer using his/her eye movements. Gaze interaction is specially useful for people with physical disabilities who cannot use conventional devices (such as a keyboard and/or mouse) to communicate. Entering text by gaze (also known as eye typing) is a very important activity and the main focus of this dissertation. The most common eye typing technique uses a virtual keyboard, where the letters are selected by dwell time. Though simple, this interaction technique suffers from involuntary activations (known as the Midas\' touch problem) for short dwell times (shorter than 500 ms). On the other hand, with longer dwell times the interaction becomes slow. Alternatives to dwell time are discrete and continuous gaze gestures. The use of discrete gaze gestures reduces the Midas\' touch problem, but its performance is slower compared to dwell time. When using continuous gaze gestures, the user gaze is always controlling the interface. A recently proposed technique called \"context switching\" avoids the Midas touch problem by mapping selection to a single saccade. Furthermore, with this technique the users can keep their rhythm of interaction without adjusting any parameter of the interface. This PhD thesis aims at improving the usability and user experience in gaze interaction with virtual keyboards. The objectives are: to investigate the relationship between contrast manipulation of visual stimuli and saccadic reaction time, to facilitate eye movements and make the interaction faster and more comfortable; to propose and investigate new extensions and applications of the context switching paradigm, in order to reduce the Midas touch problem and generalize the extensions to other tasks such as browsing and selection; and to develop new methods of eye typing that can improve typing performance without increasing the user workload, while keeping the interaction simple and easy to learn. Several user studies were designed and conducted to evaluate the methods and models proposed in this thesis. The data (both quantitative and qualitative) collected in the experiments is rigorously analysed using statistical methods. The original contributions of this thesis are: the proposal and investigation of the gradient gap effect as a visual feedback to facilitate the execution of saccadic movements during gaze interaction; the proposal and investigation of two extensions to the context switching paradigm: dynamic contexts that improve the utilization of the screen space while keeping a low selection error rate, and meta-keys for browsing and executing general-purpose commands; and the proposal and evaluation of AugFix, a new method that improves eye typing speed and user experience without increasing the cognitive load, that can be used with virtual keyboards controlled by both dwell time and context switching.

Vision is an important tool for us humans and significant effort has been put into creating solutions that let us measure how we use it. Most common among the techniques to measure gaze direction is to use specialised hardware such as infrared eye trackers. Recently, several Convolutional Neural Network (CNN) based architectures have been suggested yielding impressive results on single Red Green Blue (RGB) images. However, limited research has been done around whether using several sequential images can lead to improved tracking performance. Expanding this research to include low frequency and low quality RGB images can further open up the possibility to improve tracking performance for models using off-the-shelf hardware such as web cameras or smart phone cameras. GazeCapture is a well known dataset used for training RGB based CNN models but it lacks sequences of images and natural eye movements. In this thesis, a geometric gaze estimation model is introduced and synthetic data is generated using Unity to create sequences of images with both RGB input data as well as ground Point of Gaze (POG). To make these images more natural appearing domain adaptation is done using a CycleGAN. The data is then used to train several different models to evaluate whether temporal information can increase accuracy. Even though the improvement when using a Gated Recurrent Unit (GRU) based temporal model is limited over simple sequence averaging, the network achieves smoother tracking than a single image model while still offering faster updates over a saccade (eye movement) compared to averaging. This indicates that temporal features could improve accuracy. There are several promising future areas of related research that could further improve performance such as using real sequential data or further improving the domain adaptation of synthetic data. / Synen är ett viktigt sinne för oss människor och avsevärd energi har lagts ner på att skapa lösningar som låter oss mäta hur vi använder den. Det vanligaste sättet att göra detta idag är att använda specialiserad hårdvara baserad på infrarött ljus för ögonspårning. På senare tid har maskininlärning och modeller baserade på CNN uppnått imponerande resultat för enskilda RGB-bilder men endast begränsad forskning har gjorts kring huruvida användandet av en sekvens av högupplösta bilder kan öka prestandan för dessa modeller ytterligare. Genom att uttöka denna till bildserier med lägre frekvens och kvalitet kan det finnas möjligheter att förbättra prestandan för sekventiella modeller som kan använda data från standard-hårdvara såsom en webbkamera eller kameran i en vanlig telefon. GazeCapture är ett välkänt dataset som kan användas för att träna RGB-baserade CNN-modeller för enskilda bilder. Dock innehåller det inte bildsekvenser eller bilder som fångar naturliga ögonrörelser. För att hantera detta tränades de sekventiella modellerna i denna uppsats med data som skapats från 3D-modeller i Unity. För att den syntetiska datan skulle vara jämförbar med riktiga bilder anpassades den med hjälp av ett CycleGAN. Även om förbättringen som uppnåddes med sekventiella GRU-baserade modeller var begränsad jämfört med en modell som använde medelvärdet för sekvensen så uppnådde den tränade sekventiella modellen jämnare spårning jämfört med enbildsmodeller samtidigt som den uppdateras snabbare vid en sackad (ögonrörelse) än medelvärdesmodellen. Detta indikerar att den tidsmässiga information kan förbättra ögonspårning även för lågfrekventa bildserier med lägre kvalitet. Det finns ett antal intressanta områden att fortsätta undersöka för att ytterligare öka prestandan i liknande system som till exempel användandet av större mängder riktig sekventiell data eller en förbättrad domänanpassning av syntetisk data.