Individuals who have Attention Deficit/Hyperactivity Disorder (ADHD) experience adverse effects relating to driving; in addition, they experience deficits in scanning ability (Barkely et.al, 1996; Fischer et al., 2007; Munoz et al., 2003; Naja-Raja et al., 2007). The present study examined the effects of ADHD on eye tracking while driving. Ten participants consisting of both ADHD and individuals who do not have ADHD were included in this study. It was hypothesized that individuals who have ADHD will make more saccadic eye movements and thus shorter fixations than individuals who do not have ADHD. Furthermore, it was hypothesized that despite the fact that individuals who have ADHD will make more saccadic eye movements than individuals without ADHD, those individuals with ADHD will commit more traffic violations including collisions compared to individuals who do not have such a diagnosis. Findings indicated that hypothesis one was not supported by the data, whereas hypothesis two was supported in that ADHD individuals had more collisions and committed more traffic violations than the Control group. Additionally, upon conducting a Chi Square test for independence, a significant difference was found in the spatial distributions of the fixations between the ADHD and Control groups. The findings of this study could help better understand the factors involved in ADHD driving and could be used to train individuals with ADHD to become more aware of their surroundings and driving habits and thus become safer drivers.

Las dificultades del aprendizaje se cuentan entre las principales razones de la deserción escolar, siendo éste un problema a nivel global. El diagnóstico temprano de estas dificultades es primordial para abordar un tratamiento de manera temprana y un acompañamiento adecuado. Hoy en día el diagnóstico de estas dificultades se lleva a cabo por psicopedagogos y/o psicólogos evaluando la resolución de una batería de tests. La relación 1 a 1 entre una persona a diagnosticar y el profesional dificulta la tarea de estudios masivos para detectar tempranamente a niños en edad escolar que sufran alguna dificultad específica del aprendizaje. Un síntoma estudiado para detectar procesos cognitivos anómalos son comportamientos atípicos en los movimientos oculares. Con la tecnología de acceso masivo se pueden registrar los movimientos oculares, de manera sencilla y económica, con dispositivos de seguimiento ocular llamados por su nombre en inglés eye trackers. En esta tesis se desarrollaron herramientas para el estudio de las señales de los dispositivos mencionados. Una de estas herramientas es clave para estudiar los movimientos oculares y es la clasificación de estos en dos grandes grupos llama- dos fijaciones y sacadas. Las características estadísticas que poseen estos movimientos están íntimamente relacionados con procesos cognitivos. Otra herramienta desarrolla- da es el cálculo de diferentes cantidades de carácter estadístico que permiten describir en pocos números múltiples propiedades de señales obtenidas con los eye trackers. Una dificultad específica del aprendizaje que padece alrededor del 10 % de la población es la dislexia. Ésta es un trastorno con el que, aún hoy, hay controversia al momento de definirlo, pero que es una dificultad que afecta el desarrollo el desarrollo de la alfabetización y las habilidades relacionadas con el lenguaje. En esta tesis se estudiaron los movimientos oculares de niños que fueron diagnosticados con dislexia y de niños de un grupo control que han tenido un desarrollo típico. Describimos la dinámica de la trayectoria de la mirada de los niños al leer con un modelo estocástico basado en lo que se conoce como Continuous Time Random Walk (CTRW). Con dicho modelo fuimos capaces de generar señales sintéticas con iguales características que las señales reales. La generación de señales sintéticas es una técnica clave en el aprendizaje automático, o aprendizaje de máquina (machine learning) y es importante porque ayuda a mejorar el rendimiento de los modelos, combatir el sobre-ajuste y manejar la escasez de datos. Con datos reales hemos probado una serie de clasificadores de disléxicos y no disléxicos utilizando herramientas de machine learning. Dos de estos métodos tuvieron una alta precisión. Tener herramientas para analizar movimientos oculares podría ayudar en el diagnóstico rápido de dificultades del aprendizaje. A su vez podría ayudar a evaluar de forma masiva a niños en sus colegios y así bajar de forma drástica la deserción escolar. / Learning difficulties are among the main reasons for school dropout, this being a global problem. The early diagnosis of these difficulties is essential to address early treatment and adequate follow-up. Today the diagnosis of these difficulties is carried out by educational psychologists and/or psychologists evaluating the resolution of a battery of tests. The 1 to 1 relationship between a person to be diagnosed and the professional makes the task of massive studies difficult to detect early school-age children who suffer from a specific learning difficulty. A symptom studied to detect abnormal cognitive processes are atypical behaviors in eye movements. With mass access technology, eye movements can be recorded easily and cheaply with eye tracking devices. In this thesis, tools were developed to study the signals of the mentioned devices. One of these tools is key to studying eye movements and it is the classification of these into two large groups called fixations and saccades. The statistical characteristics that these movements have are closely related to cognitive processes. Another tool developed is the calculation of different quantities of a statistical nature that allow multiple properties of signals obtained with eye trackers to be described in a few numbers. A specific learning difficulty that about 10 % of the population suffers from is dyslexia. This is a disorder with which, even today, there is controversy when defining it, but it is a difficulty that affects the development of literacy and language-related skills. In this thesis, the eye movements of children who were diagnosed with dyslexia and of children from a control group who have had a typical development were studied. We describe the dynamics of children’s gaze trajectory when reading with a stochastic model based on what is known as Continuous Time Random Walk (CTRW). With this model we were able to generate synthetic signals with the same characteristics as the real signals. The generation of synthetic signals is a key technique in machine learning, or machine learning, and is important because it helps improve model performance, combat overfitting, and handle data sparseness. With real data we have tested a series of dyslexic and non-dyslexic classifiers using machine learning tools. Two of these methods had high precision. Having tools to analyze eye movements could help in the rapid diagnosis of learning difficulties. In turn, it could help to massively evaluate children in their schools and thus drastically reduce school dropouts, with all the benefits that this entails for society.

Studies examining impaired emotion recognition and perceptual processing in autism spectrum disorders (ASD) show inconsistent results (Harms, Martin, & Wallace, 2010; Jemel, Mottron, & Dawson, 2006), and many of these studies include eye tracking data. The current study utilizes a novel task, emotion recognition of a dynamic talking face with sound, to compare children with ASD (n=8; aged 6-10, 7 male) with mental age (MA) and gender matched controls (n=8; aged 4-10, 7 male) on an emotion identification and eye tracking task. Children were asked to watch several short video clips (2.5-5 seconds) portraying the emotions of happy, sad, excited, scared, and angry and identify the emotion portrayed in the video. A mixed factorial ANOVA analysis was conducted to examine group differences in attention when viewing the stimuli. Differences in emotion identification ability were examined using a t-test and Fisher's exact tests of independence. Findings indicated that children with ASD spent less time looking at faces and the mouth region than controls. Additionally, the amount of time children with ASD spent looking at the mouth region predicted better performance on the emotion identification task. The study was underpowered; however, so these results were preliminary and require replication. Results are discussed in relation to natural processing of emotion and social stimuli. <i>[revised ETD per Dean DePauw 10/25/12 GMc]</i> / Master of Science

Die Kombination von Eyetracking und fMRI in den Neurowissenschaften ist eine relativ neue Methode, die einerseits eine technische Herauforderung darstellt, andererseits neue Möglichkeiten des Zugangs zu neuronalen Prozessen darbietet. In der vorliegenden Arbeit wurden durch Kombination beider Methoden Prozesse der neuronalen und visuellen Verarbeitung von Emotionen untersucht. Zunächst wurde die Rolle von verschiedenen Gehirnregionen innerhalb des emotionalen Netzwerks sowie die Frage nach der Lateralität der emotionalen Verarbeitung untersucht. Die Ergebnisse zeigten, dass die neuronale Antwort in den unterschiedlichen Regionen in erster Linie die Anforderungen an die jeweilige funktionelle Einheit spiegelt. Im Rahmen der Untersuchungen von visueller Verarbeitung wurden die einzelnen spezifischen Blickbewegungsmuster für Emotionen Angst, Ekel und Freude erstmals charakterisiert. Es wurden auch Habituationseffekte auf die beschriebenen Blickbewegungsmuster untersucht. Die gemeinsame Analyse beider Datensätze zeigte, dass zwischen visuellen und neuronalen Prozessen eine enge qualitative Interaktion besteht. Es wurde ein Zusammenhang zwischen der Betrachtungsdauer und der tiefe der Verarbeitung nachgewiesen.

Today’s assembly operations represent about 15-70% of all manufacturing time and about 40% of all manufacturing costs, and manual assembly processes are still a significant portion of today’s assembly operations. Furthermore, today’s manufacturing environment requires a well-trained and flexible workforce that can easily adapt to changing products and processes. Unfortunately, manufacturing training is often performed using the master-apprentice model in the assembly line resulting in unsafe and expensive training conditions as this model is a slow and expensive process. Previous research has considered the use of virtual environments (VEs) for training purposes in different fields such as aviation, driving, construction, medicine, and manufacturing among many others. However, to this date, no assembly studies have been successful in providing a positive transfer of knowledge between virtual environments and real environments. On the other hand, several eye-tracking studies in radiology, air-traffic control, driving, and reading show that participants with higher levels of experience have different eye-scan patterns than participants with lower levels of experience. However, it is unknown how visual scans are affected by practice. Furthermore, several empirical visuomotor studies of task-oriented processes in real environments show that observers fixated their eyes on the areas that are crucial to the required task. However, we do not know the necessary visual elements to observe when performing and when learning how to perform an assembly task, nor the effects of following visual instructions and having visual distractors during this process. Finally, we have yet to establish what observation differences may exist between real and virtual environments with regards to these unknowns. This work presents the results of an assembly task which required participants to follow visual instructions and to select assembly objects among similar distractors. This assembly task was performed for ten cycles in real and virtual environments, and we used an eye-tracking device to register participants’ visual scans. We successfully identified the areas that are needed to observe for an assembly task in both environments and the effect of visual instructions and distractors in a visual scan. We found statistically significant differences for visual scans by assembly cycle and environment, with a p-value of <0.05. We also identified a connection between learning curves and participant eye scan, showing a significant decrease in the incidence of eye tracking metrics (visit count, visit duration, fixation count and fixation duration) between the first and the tenth cycles (ΔΜ), particularly for visual distractors ranging from 37.36% to 48.77%, and for visual instructions ranging from 35.17% to 54.82%. We found that participants’ observations became more efficient with practice, not only in terms of identifying distractors and following visual instructions but also in terms of developing an ability to observe key visual elements. For the RE we found a positive Pearson correlation between the proportion of fixation duration and assembly cycle for the key visual areas with p-values<0.002 and a negative Pearson correlation between the proportion of fixation duration for the non-key visual areas with p-values<0.046. Similar results were obtained for the VE.

Die Kombination von Eyetracking und fMRI in den Neurowissenschaften ist eine relativ neue Methode, die einerseits eine technische Herauforderung darstellt, andererseits neue Möglichkeiten des Zugangs zu neuronalen Prozessen darbietet. In der vorliegenden Arbeit wurden durch Kombination beider Methoden Prozesse der neuronalen und visuellen Verarbeitung von Emotionen untersucht. Zunächst wurde die Rolle von verschiedenen Gehirnregionen innerhalb des emotionalen Netzwerks sowie die Frage nach der Lateralität der emotionalen Verarbeitung untersucht. Die Ergebnisse zeigten, dass die neuronale Antwort in den unterschiedlichen Regionen in erster Linie die Anforderungen an die jeweilige funktionelle Einheit spiegelt. Im Rahmen der Untersuchungen von visueller Verarbeitung wurden die einzelnen spezifischen Blickbewegungsmuster für Emotionen Angst, Ekel und Freude erstmals charakterisiert. Es wurden auch Habituationseffekte auf die beschriebenen Blickbewegungsmuster untersucht. Die gemeinsame Analyse beider Datensätze zeigte, dass zwischen visuellen und neuronalen Prozessen eine enge qualitative Interaktion besteht. Es wurde ein Zusammenhang zwischen der Betrachtungsdauer und der tiefe der Verarbeitung nachgewiesen.

This thesis is a systematic investigation of preparatory reorienting of task-relevant spatial attention. Task switching experiments typically report a performance overhead when the current task is different to that performed on the previous trial relative to when the task repeats. This ‘switch cost’ tends to reduce as participants are given more time to prepare (consistent with an active reconfiguration process) but a ‘residual’ switch cost usually remains even at very long preparation intervals (often interpreted as evidence of carryover of response selection parameters from the previous trial which are immune to preparation). Although attentional selection of perceptual attributes is often considered to be part of task-set and is included in some models of task-set control, little research has investigated the dynamics of this component in detail. Over a series of seven experiments in which tasks were consistently mapped to screen locations, eye-tracking was used to systematically investigate task-relevant spatial selection of perceptual attributes during the preparation interval and early after stimulus onset. Experiment 1 revealed a switch-induced delay in appropriate attention orientation and a measure of ‘attentional inertia’ which could not be explained by task-independent re-orienting to locations or low-level oculomotor phenomena but were markers of task-relevant spatial selection. Experiment 2 provided a sensitive measure of both of these attentional handicaps and demonstrated that they both contribute to the switch cost (including its residual component). Although attentional inertia reduced with preparation, both handicaps were present at the longest preparation intervals. The constancy of the delay in attending to the relevant attribute reflects the effort to re-allocate attention, rather than peculiarities of spatial orienting when the cue and stimulus are presented near-simultaneously on trials with short cue-stimulus intervals. The presence of attentional inertia in blocks with long preparation intervals suggested some component of inertia immune to preparation (though see Experiments 5 and 6 below). Experiments 3 and 4 investigated the extent to which attentional selection can be decoupled from other task-set components. Cues which explicitly provided location information reduced (or eliminated) the attentional effects found in Experiment 2 indicating that attentional selection can be decoupled from other task-set components. However, Experiment 3 found that the ‘natural’ state is for attentional selection to be coupled at least to a degree (and accessed via) task-set. Experiment 5 combined eye-tracking with ERPs to investigate the relative order of attentional selection and reconfiguration of other task-set components. A well-documented ERP marker of task-set preparation always followed onset of the first fixation on the currently relevant stimulus element indicating that (at least some) task-set components are reconfigured in a serial order with spatial selection preceding other components (e.g., loading of S-R rules or other parameters into working memory). Experiments 6 and 7 investigated the nature of attentional inertia. In Experiment 6 participants were given ultimate control over the duration of the preparation interval which eliminated attentional inertia (at least as indexed by preferential fixation of the previously relevant element on switch trials). In Experiment 7 the stimulus comprised three items which were from perceptually distinct classes (digits, letters, objects) to investigate whether the presence of task-specific features would elicit extra attentional inertia and whether early spatial selection was effective enough to block the processing of task-irrelevant features once the stimulus was presented. Although there was some evidence that the previously relevant stimulus element ‘captured’ attention, this tendency was modest in the fixations and absent in performance measures (response congruence effects).

This thesis investigates the mechanisms underlying the formation, maintenance, and sharing of reference in tasks in which language and vision interact. Previous research in psycholinguistics and visual cognition has provided insights into the formation of reference in cross-modal tasks. The conclusions reached are largely independent, with the focus on mechanisms pertaining to either linguistic or visual processing. In this thesis, we present a series of eye-tracking experiments that aim to unify these distinct strands of research by identifying and quantifying factors that underlie the cross-modal interaction between scene understanding and sentence processing. Our results show that both low-level (imagebased) and high-level (object-based) visual information interacts actively with linguistic information during situated language processing tasks. In particular, during language understanding (Chapter 3), image-based information, i.e., saliency, is used to predict the upcoming arguments of the sentence, when the linguistic material alone is not sufficient to make such predictions. During language production (Chapter 4), visual attention has the active role of sourcing referential information for sentence encoding. We show that two important factors influencing this process are the visual density of the scene, i.e., clutter, and the animacy of the objects described. Both factors influence the type of linguistic encoding observed and the associated visual responses. We uncover a close relationship between linguistic descriptions and visual responses, triggered by the cross-modal interaction of scene and object properties, which implies a general mechanism of cross-modal referential coordination. Further investigation (Chapter 5) shows that visual attention and sentence processing are closely coordinated during sentence production: similar sentences are associated with similar scan patterns. This finding holds across different scenes, which suggests that coordination goes beyond the well-known scene-based effects guiding visual attention, again supporting the existence of a general mechanism for the cross-modal coordination of referential information. The extent to which cross-modal mechanisms are activated depends on the nature of the task performed. We compare the three tasks of visual search, object naming, and scene description (Chapter 6) and explore how the modulation of cross-modal reference is reflected in the visual responses of participants. Our results show that the cross-modal coordination required in naming and description triggers longer visual processing and higher scan pattern similarity than in search. This difference is due to the coordination required to integrate and organize visual and linguistic referential processing. Overall, this thesis unifies explanations of distinct cognitive processes (visual and linguistic) based on the principle of cross-modal referentiality, and provides a new framework for unraveling the mechanisms that allow scene understanding and sentence processing to share and integrate information during cross-modal processing.

Ingeniero Civil Industrial / El presente Trabajo de Título tiene como objetivo principal conocer el impacto que significa la incorporación de una nueva fuente de información a la metodología de identificación de Website Keyobjects. Esta nueva fuente de información será la medida de la actividad bioeléctrica cerebral frente a los estímulos presentados en una página web. Específicamente, se busca diseñar e implementar un módulo con algoritmos de Data Mining para clasificar los objetos relevantes presentes en un sitio web según variables de actividad cerebral. El trabajo se enmarca en el proyecto Fondef Plataforma informática basada en web-intelligence y herramientas de análisis de exploración visual para la mejora de la estructura y contenido de sitios web (AKORI: Advanced Kernel for Ocular Research and Web Intelligence) . Este proyecto está siendo llevado a cabo en conjunto por el Laboratorio de Neurosistemas y el Departamento de Ingeniería Industrial de la Universidad de Chile. La hipótesis que se valida en este trabajo es: "Es posible realizar una clasificación de los objetos relevantes de un sitio web según variables que caractericen la actividad bioelétrica cerebral". Para la validación de esta hipótesis se realiza un experimento en que 20 usuarios navegan libremente por un sitio web de estudio, mientras que un dispositivo de Eye Tracking guarda el posicionamiento de los ojos en la pantalla y los cambios en el tamaño de las pupilas y un electroencefalograma graba los potenciales eléctricos en la corteza cerebral en tiempo real. El análisis de los datos obtenidos se realiza mediante el proceso KDD, en un principio se realiza para la dilatación pupilar con el fin de obtener una línea base de estudio. Luego se aplica a los datos del EEG para realizar una comparación a lo obtenido de las señales oculares. Para ambos tipos de data se seleccionan 19 sujetos, cuyos datos son preprocesados y limpiados, luego transformados según distintas características a los que se les aplican diversos algoritmos de clasificación. Con esto se determina como resultado una lista de objetos relevantes dentro del sitio de estudio. Los resultados obtenidos indican que utilizando variables obtenidas de las señales eléctricas producidas en la corteza cerebral es posible clasificar Website Keyobjects con un 90\% de precisión, mediante el algoritmo K-Means. Esta clasificación es en base a la línea base obtenida, donde de un total de 20 objetos, 18 fueron clasificados correctamente, y fue la mejor que se obtuvo dentro de todas las combinaciones de variables y clusterizaciones realizadas. Finalmente se concluye que el trabajo fue exitoso y se proponen diversos trabajos futuros que aportan al proyecto AKORI y la metodología. Además se entrega una reflexión final correspondiente al uso de metodologías, algoritmos y conocimientos similares en otras áreas de estudio que generen valor para la sociedad.

Ingeniero Civil Industrial / El proyecto AKORI está pensado como una plataforma informática basada en técnicas de Web Intelligence, y análisis de la exploración visual para mejorar la estructura y los contenidos de un sitio web. En su etapa de investigación, se han incorporado estudios relacionados con la dilatación pupilar, el posicionamiento ocular y análisis de la actividad cerebral de los usuarios web. Una de las líneas de investigación indagó en la relación entre la dilatación pupilar y la toma de decisión de los usuarios de sitios web. Para profundizar en este estudio, se pretende realizar una investigación exploratoria, relacionando la toma de decisión con un nuevo marcador fisiológico. El objetivo general es determinar si existe una relación entre el potencial eléctrico producido por la actividad cerebral y la toma de decisión de un usuario en un sitio web. La hipótesis de investigación se plantea de la siguiente manera: El potencial bioeléctrico producido por la actividad cerebral puede predecir la toma de decisión de un usuario web . La metodología de esta investigación considera cuatro etapas: Análisis del estado del arte en el área de la neurociencia y técnicas de Data Mining; Diseño e implementación de experimentos; Análisis y procesamiento de los datos para validar la hipótesis de investigación y la construcción del clasificador; y finalmente comparar los resultados de la metodología obtenidas anteriormente. Para obtener los datos, se utilizaron herramientas de Eye-Tracking y EEG, en experimentos que simulan una situación discriminatoria dentro de un sitio web, mostrando imágenes de valencia neutra y donde el usuario debía escoger una de ellas. Para poder realizar extrapolación, se experimentó con 20 sujetos, de los cuales 18 finalizaron con la totalidad de los registros. Para analizar los datos, se separaron las observaciones que coinciden con el objeto elegido y las que no. Aquellas que tenía más de 600ms. de duración, fueron promediadas y graficadas. Al graficar cada una de las curvas, no se observan diferencias significativas. Sin embargo, se observa un patrón de comportamiento común, que consiste en la disminución de la actividad cerebral en los primeros 100ms. y un alza abrupta en la actividad durante los siguientes 200ms. Dado que no hay diferencias significativas, se realizaron dos tipos de caracterizaciones descripción estadística y transformada de Wavelet, que finalmente no permitieron validar la hipótesis de este trabajo. Si bien, la hipótesis de investigación no fue validada con los parámetros descritos, se reconoce que existen variables fisiológicas que constituyen una fuente de información que puede ser aplicadas en diversos ámbitos, ya sea en web-intelligence, web-mining, u otra disciplina ajena a la medicina pura.