Normalerweise stimmen die Ausrichtung der Augen und die der visuellen Aufmerksamkeit überein. Frühere Arbeiten, vor allem im Kontext so genannter ‚Fixieren-Springen’ Paradigmen, haben gezeigt, dass zum Zeitpunkt der Programmierung einer neuen Sakkade die Aufmerksamkeit bereits zum Sakkadenziel verlagert wird. Bei Untersuchungen in natürlicheren Umgebungen konnte dieser Befund allerdings nicht gezeigt werden. Aus diesen teilweise widersprüchlichen Befunden ergibt sich die Fragestellung der vorliegenden Arbeit: Wie wirken sich visuelle Aufgaben am aktuellen Fixationsort innerhalb einer Sequenz von Blickbewegungen auf die Verlagerung der visuellen Aufmerksamkeit aus? Welche Rolle spielt dabei der Zeitpunkt innerhalb der aktuellen Fixation? Diese Fragen werden auf der Basis des Zusammenhangs zwischen objektivem und subjektivem Blickort untersucht. Als objektiver Blickort wird dabei der durch ein Blickbewegungsmesssystem ermittelte aktuelle Fixationsort begriffen. Während der Ausführung verschiedener visueller Aufgaben – Lokalisation, Identifikation und Katego¬risierung – wird der subjektive Blickort als derjenige bezeichnet und untersucht, den Probanden bei einer plötzlichen Unterbrechung ihrer aktuellen Tätigkeit angeben können. In einer Serie von drei Experimenten wurde getestet, welcher Zusammen¬hang zwischen objektivem und subjektivem Maß vor dem Hinter¬grund der verschiedenen Aufgaben und dem Zeitpunkt innerhalb der aktuellen Fixation besteht. Dabei wurden den Probanden in jedem Durchgang sechs kreisförmig angeordnete Piktogramme präsentiert, die im Uhrzeigersinn nacheinander zu betrachten waren. Zu einem nicht vorhersehbaren Zeitpunkt wurde die Betrachtung unterbrochen, wobei der subjektive Blickort durch einen Mausklick auf das aktuell betrachtete Piktogramm angegeben werden sollte. In einem vierten Experiment wurde eine blickgesteuerte Schreibumgebung untersucht, bei der auf der Basis der Ergebnisse der Experimente 1 bis 3 die Rückmeldung der eigenen Blickposition und die Schwellzeit zum Auslösen einer Taste systematisch variiert wurden. Es wurde davon ausgegangen, dass die Rückmeldung weniger an den aktuellen tatsächlichen, als an den subjektiven Blickort gebunden sein sollte. Schreiben mit den Augen setzt voraus, dass Buchstaben auf der Tastatur lokalisiert, und danach identifiziert werden müssen, um eine Entscheidung treffen zu können, ob die Taste ausgelöst werden soll oder nicht. Die vorhergehenden Experimente zum subjektiven und objektiven Blickort bei der Identifikation haben gezeigt, dass der subjektive hinter dem objektiven Blickort eher zurückbleibt. Aus diesem Grund ist anzunehmen, dass eine Verzögerung der Rückmeldung gegenüber der objektiven Blickposition als angenehm empfunden werden müsste. Es zeigte sich, dass die besten Ergebnisse in Bezug auf Fehlerrate und Schreibgeschwindigkeit bei Schwellzeiten um die 500 ms erreicht werden. Die subjektiven Einschätzungen der Probanden ergab eine signifikante Präferenz der verzögerten Rückmeldung. Insgesamt zeigen die Ergebnisse der Untersuchungen, dass der Zusammenhang zwischen subjektiven und objektiven Blickort nicht nur durch den Zeitpunkt innerhalb der aktuellen Fixation, sondern vor allem auch durch die momentan bearbeitete Aufgabe moduliert wird. Besonders letzteres wird in aktuellen Theorien der visuellen Aufmerksamkeit nur wenig beachtet. Für die vorliegenden Ergebnisse wird ein Erklärungsansatz entwickelt, der die Idee aufeinander aufbauender Ebenen der Informationsverarbeitung kognitiver Aufmerksamkeitsmodelle aufgreift. Besonders das von Velichkovsky (2002) vorgeschlagene Modell erweist sich dabei als fruchtbar, da es feiner differenziert, und darüber hinaus zwischen Hintergrundkoordination und Führungsebene unterscheidet. In diesem Sinne lassen sich die untersuchten Aufgaben mit einer zunehmenden Tiefe der Verarbeitung assoziieren. Damit einhergehend steigt die Komplexität der zu verarbeitenden Information und sinkt die Wahr¬scheinlichkeit, dass Informationen jenseits der aktuellen Fixation abgerufen werden können: Im Gegenteil, früh innerhalb einer Fixation werden bei tiefer Verarbeitung sogar noch Informationen aus der vorhergehenden Fixation berichtet. Diese Konzeptionalisierung der Ergebnisse gestattet es, vordergründig unterschiedliche Befunde aus ‚Fixieren und Springen’ Paradigmen und Studien in natürlicheren Umgebungen zu integrieren. Die Ergebnisse von Experiment 4 zeigen, dass dieser Erklärungsansatz auch praktische Ableitungen für die Gestaltung von blick¬gesteuerter Interaktion mit dem Computer geeignet ist. Die führende Aufgabe bei solchen intentionalen Fixationen bestimmt die Art und Weise, wie der subjektive Blickort dem Nutzer zurückgemeldet werden sollte.

Current developments of hardware devices have allowed the computer vision technologies to analyze complex human activities in real time. High quality computer algorithms for human activity interpretation are required by many emerging applications, such as patient behavior analysis, surveillance, gesture control video games, and other human computer interface systems. Despite great efforts that have been made in the past decades, it is still a challenging task to provide a generic gesture recognition solution that can facilitate the developments of different gesture-based applications. Human vision is able to perceive scenes continuously, recognize objects and grasp motion semantics effortlessly. Neuroscientists and psychologists have tried to understand and explain how exactly the visual system works. Some theories/hypotheses on visual perception such as the visual attention and the Gestalt Laws of perceptual organization (PO) have been established and shed some light on understanding fundamental mechanisms of human visual perception. In this dissertation, inspired by those visual attention models, we attempt to model and integrate important visual perception discoveries into a generic gesture recognition framework, which is the fundamental component of full-tier human activity understanding tasks. Our approach handles challenging tasks by: (1) organizing the complex visual information into a hierarchical structure including low-level feature, object (human body), and 4D spatiotemporal layers; 2) extracting bottom-up shape-based visual salience entities at each layer according to PO grouping laws; 3) building shape-based hierarchical salience maps in favor of high-level tasks for visual feature selection by manipulating attention conditions of the top-down knowledge about gestures and body structures; and 4) modeling gesture representations by a set of perceptual gesture salience entities (PGSEs) that provide qualitative gesture descriptions in 4D space for recognition tasks. Unlike other existing approaches, our gesture representation method encodes both extrinsic and intrinsic properties and reflects the way humans perceive the visual world so as to reduce the semantic gaps. Experimental results show our approach outperforms the others and has great potential in real-time applications. / PhD Thesis

The visual system has evolved to deal with the consequences of our own movements onour perception. In particular, evolution has given us the ability to perceive our visual world as stableand continuous despite large shift of the image on our retinas when we move our eyes, head orbody. Animal studies have recently shown that in some cortical and sub-cortical areas involved inattention and saccade control, neurons are able to anticipate the consequences of voluntary eyemovements on their visual input. These neurons predict how the world will look like after a saccadeby remapping the location of each attended object to the place it will occupy following a saccade.In a series of studies, we first showed that remapping could be evaluated in a non-invasive fashion in human with simple apparent motion targets. Using eye movement recordingsand psychophysical methods, we evaluated the distribution of remapping errors across the visualfield and found that saccade compensation was fairly accurate. The pattern of errors observedsupport a model of space constancy based on a remapping of attention pointers and excluded otherknown models. Then using targets that moved continuously while a saccade was made across themotion path, we were able to directly visualize the remapping processes. With this novel method wedemonstrated again the existence of systematic errors of correction for the saccade, best explainedby an inaccurate remapping of expected moving target locations. We then extended our model toother body movements, and studied the contribution of sub-cortical receptors (otoliths and semi-circular canals) in the maintenance of space constancy across head movements. Contrary tostudies reporting almost perfect compensations for head movements, we observed breakdowns ofspace constancy for head tilt as well as for head translation. Then, we tested remapping of targetlocations to correct for saccades at the very edge of the visual field, remapping that would place theexpected target location outside the visual field. Our results suggest that visual areas involved inremapping construct a global representation of space extending out beyond the traditional visualfield. Finally, we conducted experiments to determine the allocation of attention across saccades.We demonstrated that the attention captured by a brief transient was remapped to the correctspatial location after the eye movement and that this shift can be observed even before thesaccade.Taken together these results demonstrate the management of attention pointers to therecovery of target locations in space as well as the ability of behavioral measurements to address atopic pioneered by eletrophysiologists.

L’objectif de ce travail de thèse était de mieux caractériser l’hétérogénéité de la dyslexie développementale et d’étudier les implications de cette hétérogénéité pour la remédiation de ce trouble d’acqusition de la lecture. L’Etude I met en évidence une hétérogénéité cognitive au sein d’une population d’enfants dyslexiques, qui pourtant ont un profil de lecture homogène de dyslexie mixte. Ce profil est classiquement interprété comme relevant d’un double déficit. Or nous montrons que la plupart de ces enfants présente en fait un déficit cognitif unique, soit phonologique, soit de l’empan visuo-attentionnel (VA). De plus, ces deux sous-groupes à déficit unique ont des profils de lecture très similaires lorsqu’ils sont directement comparés. Ces données suggèrent donc que la classification en sous-types basée sur les profils de lecture n’est pas pertinente pour identifier des sous-groupes cognitivement homogènes dans la population dyslexique. Dans l’Etude II, nous décrivons un cas de dyslexie avec trouble sélectif de la lecture des pseudo-mots. Ce profil est classiquement interprété comme reflétant un déficit phonologique. Or, le cas que nous décrivons présente un trouble de l’empan VA en l’absence de toute atteinte verbale ou phonologique. Cette étude confirme une relation entre trouble cognitif et profil de lecture complexe et non univoque. L’Etude III interroge plus spécifiquement les liens entre troubles de l’empan VA, de la conscience phonémique et de la perception catégorielle des phonèmes. Nous montrons que la conscience phonémique joue un rôle de médiation entre perception catégorielle et lecture, et que cette relation est indépendante des capacités d’empan VA. Ces données nous ont ainsi permis de mieux caractériser l’hétérogénéité cognitive de la population dyslexique. Dans les deux dernières études, nous nous sommes interrogés sur la prise en compte de cette hétérogénéité cognitive dans la remédiation de la dyslexie. L’Etude IV est une étude d’entrainement cognitif. Deux types d’entraînements ont été successivement proposés à des enfants dyslexiques : un entraînement à la perception catégorielle et un entraînement de l’empan VA. Nous avons étudié les effets de chacun de ces entraînements sur des sous-groupes qui présentent un trouble cognitif unique, soit phonologique, soit de l’empan VA. Les résultats montrent l’intérêt de proposer une remédiation ciblée sur le déficit cognitif sous-jacent. Cette étude a également des enjeux théoriques majeurs, puisqu’elle nous a permis d’interroger les relations de causalité entre ces déficits cognitifs sous-jacents et la dyslexie. Enfin, dans l’Etude V la méthode d’adaptation visuelle par saillance syllabique étudiée n’améliore pas la lecture des enfants dyslexiques. L’ensemble des ces études confirment l’hétérogénéité cognitive de la population dyslexique et l’importance de prendre en compte cette hétérogénéité dans les méthodes de remédiation qui peuvent être proposées. / The first aim of this doctoral thesis was to better understand the heterogeneity of developmental dyslexia focusing on the relationship between reading profiles and cognitive disorders. Our second purpose was to question the implications of cognitive heterogeneity for the remediation of developmental dyslexia. Study I revealed cognitive heterogeneity despite the homogeneous reading profile of the recruited population. Most children with a mixed reading profile typically interpreted as resulting from a double deficit, actually suffered from a single cognitive disorder, either a phoneme awareness or a visual attention (VA) span disorder. In addition, direct comparisons of these two cognitively distinct subgroups revealed very similar reading profiles. These findings suggest that the classification based on reading profiles is irrelevant to identify cognitively homogeneous subgroups in dyslexia. In Study II, we report the case study of a child with a selective pseudo-word reading disorder, a profile typically attributed to a phonological deficit. Contrary to this prediction however, we show that this dyslexic child shows preserved verbal and phonological skills but a VA span deficit. This finding again suggests the absence of one-to-one relationships between reading profiles and cognitive disorders. Study III more specifically explored the links between three types of cognitive deficits, namely the VA span disorder, the phoneme awareness deficit and the categorical perception disorder. Results show that phoneme awareness mediates the relationship between categorical perception and reading: poor categorical perception causes poor phoneme awareness, which in turn impacts reading performance. Poor VA span also impacts reading performance but independently of phoneme awareness and categorical perception. These findings are further evidence for the independent contribution of phonology and VA span to the reading outcome of dyslexic children. In Study IV, we assessed the longitudinal effects of two cognitive trainings. Dyslexic participants with a single phonological or a single VA span disorder were administered successively either a categorical perception then a VA span intervention, or the two trainings in the reverse order. At the clinical level, results show that intervention is more efficient when targeted on the underlying cognitive deficit. At the theoretical level, they question causal relationship between these cognitive deficit and dyslexia. Last, Study V explored the compensation power of a visual adaptation method using syllabic saliency. Results failed to show any positive effect of this method on reading performance. The overall findings support the cognitive heterogeneity of the dyslexic population and show that taking this issue into account is crucial to improve the effects of intervention programs.

Submitted by Johnny Rodrigues (johnnyrodrigues@ufcg.edu.br) on 2018-07-31T14:31:59Z No. of bitstreams: 1 EANES TORRES PEREIRA - DISSERTAÇÃO PPGCC 2007..pdf: 4422108 bytes, checksum: 9b27a661074ca88e823ffb0472a68502 (MD5) / Made available in DSpace on 2018-07-31T14:31:59Z (GMT). No. of bitstreams: 1 EANES TORRES PEREIRA - DISSERTAÇÃO PPGCC 2007..pdf: 4422108 bytes, checksum: 9b27a661074ca88e823ffb0472a68502 (MD5) Previous issue date: 2007-03 / A atenção visual é um mecanismo biologicamente inspirado, o qual corresponde à habilidade de selecionar e processar somente as regiões mais relevantes de uma cena visual. Para fins didáticos, a atenção visual pode ser dividida em duas categorias principais: bottom-up e top-down. A atenção visual bottom-up guia o foco de atenção a partir de características primitivas (como descontinuidades de intensidade em diferentes escalas e orientações) computadas diretamente na imagem,sem qualquer informação contextual. A atenção visual top-down, por outro lado, realiza uma busca por regiões de interesse a partir de características de alto nível, especificadas na forma de conhecimento prévio na forma ou modelos sobre o que se está buscando na cena. A principal questão de pesquisa que procuramos responder nessa dissertação é a seguinte: como seria possível agregar algum comportamento de alto nível a um mecanismo típico de atenção visual bottom-up (guiando dessa forma o foco de atenção para classes de regiões pré-estabelecidas)? O modelo mais conhecido de atenção visual bottom-up utiliza vários mapas de características primitivas para formar um mapa de saliência, o qual indica a importância do ponto de vista atencional das diferentes regiões de uma cena. Nesse trabalho, atribuímos pesos aos mapas de características e desenvolvemos um processo de otimização baseado em algoritmos genéticos simulados em uma grade computacional. Foram realizados experimentos com quatro classes de objetos (carros, faces de pessoas, objetos genéricos e pistolas). Os resultados utilizando atenção bottom-up com otimização foram comparados com os resultados de um mecanismo sem otimização de pesos e com um sistema existente que implementa o difundido modelo de atenção visual proposto por Itti et al. [Itti et al., 1998]. Os resultados mostraram ganhos de até 30% utilizando-se a abordagem proposta. Desta forma, este trabalho mostra que a atenção visual pode ser guiada para regiões pré-definidas, podendo ser utilizada como parte de sistemas de detecção de objetos. / Visual attention is a biologically inspired mechanism, which corresponds to the ability of selecting and processing only the most relevant regions of a visual scene. For didactic purposes, visual attention can be divided into two main categories: bottom-up and top down. Bottom-up visual attention guides the attention focus by using primitive visual features (such as discontinuities in intensity across different scales and orientations) computed directly from the input image,without the need of any context information. Top-down visual attention, on the other side, performs a search for interest regions from higher-level features, specified in the form of previous knowledge or models about what is being sought in the scene. Themain research question that we intended to answer in this dissertation was the following: how it would be possible to incorporate some higher-level be haviour into a typical bottom-up visual attention mechanism (thus guiding the attention focus to pre-established classes of objects)? The most known bottom-up visual attention model uses several primitive feature maps to form a saliency map, which indicates the importance of the different scene regions. In this work, we assigned weights to the feature maps and developed an optimization process based on genetic algorithms running on a computational grid. Experiments involving four object classes (cars, human faces, generic objects and pistols)have been performed. The results of the optimized bottom-up mechanism have been compared with the results of a mechanism not using optimized weights and with an existing system that implemented the well known visual attention mechanism proposed by Itti et al. [Itti et al., 1998]. The results have shown an improvement of up to 30% when using the optimized mechanism. Thus,this work shows that visual attention can indeed be guided towards pre-defined regions and can be used as part of object detection systems.

Attention is a cognitive mechanism that guides our perception in order to prioritize the limited resources to the most relevant information while ignoring distracting information. Attention can be voluntarily deployed to stimuli during tasks or goals, or the features of the stimulus can capture our attention either by being salient or being emotionally induced. Emotions affect multiple different cognitive processes such as attention because emotional stimuli can be relevant for defending or sustain life. This relationship between attention and emotion indicates that there should be interactive but distinct networks between these cognitive mechanisms as well as a modulative effect on perceptional and attentional systems. Emotions were in general demonstrating a facilitation affect on attentional and saccadic processes as well as broadening or narrowing the scope of attention. The reason behind emotions impact on attention was proposed to be for eliciting a change in the application of resources in order to solve the limited capacity problem and possibly to protect and sustain life. Inconsistent findings as well as limitations for emotional attention studies are discussed.

Nós realizamos três experimentos para estudar a distribuição da atenção pelo campo visual e a possibilidade dos recursos atentivos serem mais concentrados no interior de uma área delimitada por uma figura geométrica de aparecimento abrupto. Os participantes realizaram uma tarefa de julgamento de ordem temporal (JOT), na qual julgaram a ordem de duas letras apresentadas em seqüência. Uma letra foi apresentada dentro e a outra fora de uma moldura, que não foi relacionada com a tarefa. A percepção de ordem temporal foi influenciada pela moldura nos três experimentos. A variação da distância entre as duas letras e o tempo de exposição da moldura foram manipulados, alterando o julgamento dos participantes. Os resultados demonstraram uma variação na concentração dos recursos atentivos em função da distância entre as duas letras e em função do tempo de exposição da moldura. Os resultados sugerem que a forma de uma figura geométrica pode produzir uma reorganização dos recursos atentivos, que parecem estar mais concentrados dentro do que fora das suas bordas. / We conducted three experiments to examine the distribution of visual attention and the possibility of the attentional resources to be concentrated along an area delimited by a frame of abrupt onset. The participants performed a temporal order judgment (TOJ) task, in which judged the order of two letters presented in sequence. One letter was showed inside and other outside a frame that was not related to the task. Perception of temporal order was influenced by the frame in all experimental conditions. The judgement of participants was affected by the distance between the two letters and the cue lead time. The findings suggest that the form of a geometric figure can reallocate the resources of attention that should be more concentrated inside than outside of its borders.

O presente trabalho analisou algumas das possíveis características do efeito da atenção intermodal entre elas: relação sinal/ruído e assincronia entre início dos estímulos (AIE) em relação ao tipo de tarefa realizada. Os Experimentos 1 e 2 mostraram que os efeitos da atenção intermodal se manifestam na presença de ruído visual externo, e que não se manifestam na ausência de ruído em uma AIE de 133 ms. No entanto, o Experimento 3 mostrou que quando a AIE é maior que a utilizada nos experimentos anteriores, o efeito intermodal se manifesta no comportamento na ausência de ruído visual externo. Finalmente, o Experimento 4 mostrou que em uma AIE curta (133 ms), e em uma tarefa de localização, o efeito intermodal se manifesta. Os resultados sugerem que o efeito atencional intermodal se manifesta na presença e na ausência de ruído visual e que o mecanismo de discriminação da frequência do alvo demora mais tempo para se completar do que o mecanismo de localização do alvo. / The current work examined the possible contribution of signal to noise ratio, the asynchrony between the onsets of the cue and the target (SOA) and the kind of task performed by the observer to the manifestation of crossmodal attentional effects. The Experiments 1 and 2 showed that crossmodal attentional effect appears when there is visual noise, but it does not appear when there is no visual noise at 133 ms SOA. The Experiment 3 showed that when the SOA is longer than 133 ms, the crossmodal attentional effect appears when there is no visual noise. The Experiment 4 showed that in a localization task, the crossmodal attentional effect appears even in a short SOA (133 ms). Taken together, the results indicate that crossmodal attentional effects appear when there is visual noise and when there is no visual noise. However, in the later condition and when the target has to be identified, the crossmodal attentional effect takes longer to appear.

Investigamos, no Experimento, se o estímulo precedente, que captura a atenção quando existem demarcações, deixa de capturar a atenção na ausência de demarcações por ter seu processamento filtrado precocemente. Além disso, investigamos, nos experimentos 2, 3 e 4, se alterações no fundo da tela, que gerassem maior competição no processamento dos estímulos presentes na cena visual, poderiam levar à captura da atenção, mesmo sem a presença das demarcações classicamente utilizadas. Observamos que quando a intensidade dos estímulos competidores presentes na tela era alta, o estímulo precedente passou a capturar a atenção. Explicamos esse resultado com a hipótese de que as demarcações são importantes por adicionarem ruído ao processamento dos estímulos apresentados, influenciando a representação desses estímulos e, portanto, a possibilidade de um estímulo capturar a atenção. Por fim, no Experimento 5, investigamos se os resultados da tarefa de escolha de local seriam replicados em uma tarefa de discriminação de forma. / In Experiment 1 we investigated if a peripheral stimulus, which is able to capture attention when there are placeholders on the screen, is no longer able to capture attention in the absence of placeholders because is early filtered. In Experimento 2, 3 and 4 we investigated if changes on screen background, which would generate stronger processing competition between stimuli, could lead to attentional capture even in absence of placeholders. Accorging to our results the peripheral stimuli is able to capture attention when competing stimuli of high intensity are presented togueter on screen. Our explanation is that placeholders are important because they add noise to stimuli processing, affecting stimuli representation and, therefore, the probability that a stimulus will capture attention. Lastly, in Experimento 5 we investigated if similar results would be obtained in a form discrimination task.

Context. Video games usually have visual aids guiding the players in 3D-environments. The designers need to know which visual feature is the most effective in attracting a player's gaze and what features are preferred by players as visual aid. Objectives. This study investigates which feature of the bottom-upvisual attention process attracts the gaze faster. Methods. With the use of the Tobii T60 eye tracking system, a user study with 32 participants was conducted in a controlled environment. An experiment was created where each participant looked at a slideshow consisting of 18 pictures with 8 objects on each picture. One object per picture had a bottom-up visual feature applied that made it stand out as different. Video games often have a goal or a task and to connect the experiment to video games a goal was set. This goal was to find the object with the visual feature applied. The eye tracker measured the gaze while the participant was trying to find the object. A survey to determine which visual feature was preferredby the players was also made. Results. The result showed that colour was the visual feature with the shortest time to attract attention. It was closely followed by intensity,motion and a pulsating highlight. Small size had the longest attraction time. Results also showed that the preferred visual feature for visual aid by the players was intensity and the least preferred was orientation. Conclusions. The results show that visual features with contrast changes in the texture seems to draw attention faster, with colour the fastest, than changes on the object itself. These features were also the most preferred as visual aid by the players with intensity the most preferred. If this study was done on a larger scale within a 3D-environment, this experiment could show promise to help designers in decisions regarding visual aid in video games.