Doctor of Philosophy / Department of Psychological Sciences / Lester C. Loschky / Political videos are created as persuasive media, and at a basic level that persuasion would require that the videos guide viewer attention to the relevant persuasive content. Recent work has shown that filmmakers have techniques that allow them to guide where viewers look, and this guidance occurs even when viewers have very different understandings of the film. The current research tested if these attentional effects carry over to political videos, or if the top-down factors of attitude congruence and social vigilantism, belief superiority and the tendency to impress one’s “superior” beliefs on others (O'Dea, Bueno, & Saucier, 2018; Saucier & Webster, 2010; Saucier, Webster, Hoffman, & Strain, 2014), will break the ability of videos to guide viewers’ attention. Attentional selection was measured through participants’ eye movements, and memory encoding was measured through recall and recognition for both verbal and visual information. Three overarching competing hypotheses predicted different relationships between attitude congruence, social vigilantism, and visual attention and memory. The Tyranny of Film Hypothesis predicted that the videos would guide viewer attention, regardless of attitude congruence. This would result in similar eye-movements and memory for all participants. The Selective Exposure Hypothesis predicted that participants would avoid processing attitude-incongruent information. As a result, viewers’ visual attention would be directed away from attitude-incongruent information, and subsequent memory would be worse. Lastly, the Social Vigilantism Hypothesis predicted that people high in Social Vigilantism would engage more with attitude-incongruent information. Two experiments tested these hypotheses. The first was the Memory experiment (conducted online), and the second was the Eye movement experiment. In each experiment, participants watched a series of political advertisement and debate videos, and attitudes were measured to identify which information in the videos was attitude-congruent and incongruent. The Memory experiment showed some support for the Social Vigilantism Hypothesis, with People high in Social Vigilantism having better memory for attitude-incongruent information on certain memory measures. Conversely, the Eye movement experiment consistently showed strong stimulus driven effects in support of the Tyranny of Film, but also weaker attitude and social vigilantism effects that were independent of attitude congruence. Altogether, these results show dynamic video stimuli features are the best predictors of viewer attention and memory, but viewer attitude and social vigilantism have subtle top-down effects. The support for different hypotheses between the two experiments indicates the strength of top-down effects may depend on the format of the viewing experience, and specifically how much control the viewer has over the experience.

Context. Reaching photo realistic results when rendering 3D graphics in real-time is a hard computational task. Ray-tracing gives results close to this but is too expensive to be run at real-time frame rates. On the other hand rasterized methods such as deferred rendering are able to keep the tight time constraints with the support of modern hardware. Objectives. The basic objective is to merge deferred rendering and ray-tracing into one rasterized pipeline for dynamic scenes. In the thesis the proposed method is explained and compared to the methods it merges. Image quality, execution time and VRAM usage impact are investigated. Methods. The proposed method uses deferred rendering to render the result of the primary rays. Some pixels are marked, based on material properties for further rendering with ray-tracing. Only reflections are presented in the thesis but it has been proven that other global illumination effects can be implemented in the ray-tracing framework used. Results and Conclusions. The hybrid method is proved through experiments to be between 2.49 to 4.19 times faster than pure ray-tracing in the proposed pipeline. For smaller scenes it can be run at frame rates close to real-time, but, for larger scenes such as the Crytek Sponza scene the real-time feeling is lost. However, interactivity is never lost. It is also proved that a simple adjustment to the original framework can save almost 2/3 of the memory spent on A-buffers. Image comparisons prove that the technique can compete with offline ray tracers in terms of image quality.

Nous étudions l'influence de différents attributs audiovisuels sur l'exploration visuelle de scènes naturelles dynamiques. Nous démontrons que si la façon dont nous explorons une scène dépend avant tout de son contenu visuel, dans certaines situations le son influence significativement les mouvements oculaires. La présence de son assure une meilleure cohérence entre les positions oculaires de différents observateurs, attirant leur attention et donc leur regard vers les mêmes régions. L'effet du son se retrouve tout particulièrement dans les scènes de conversation, où la présence du signal de parole associé augmente le nombre de fixations sur le visage des locuteurs, et donc la cohérence entre les scanpaths. Nous proposons un modèle de saillance audiovisuelle repérant automatiquement le visage des locuteurs afin d'en rehausser la saillance. Ces résultats s'appuient sur les mouvements oculaires de 148 participants enregistrés sur un total de plus de 75 400 frames (125 vidéos) dans 5 conditions expérimentales différentes. / We study the influence of different audiovisual features on the visualexploration of dynamic natural scenes. We show that, whilst the way a person explores a scene primarily relies on its visual content, sound sometimes significantly influences eye movements. Sound assures a better coherence between the eye positions of different observers, attracting their attention and thus their gaze toward the same regions. The effect of sound is particularly strong in conversation scenes, where the related speech signal boosts the number of fixations on speakers' faces, and thus increases the consistency between scanpaths. We propose an audiovisual saliency model able to automatically locate speakers' faces so as to enhance their saliency. These results are based on the eye movements of 148 participants recorded on more than 75,400 frames (125 videos) in 5 different experimental conditions.

De nombreuses applications de la vision par ordinateur requièrent la détection, la localisation et le suivi de régions ou d’objets d’intérêt dans une image ou une séquence d’images. De nombreux modèles d’attention visuelle, inspirés de la vision humaine, qui détectent de manière automatique les régions d’intérêt dans une image ou une vidéo, ont récemment été développés et utilisés avec succès dans différentes applications. Néanmoins, la plupart des approches existantes sont limitées à l’analyse de scènes statiques et très peu de méthodes exploitent la nature temporelle des séquences d’images.L'objectif principal de ce travail de thèse est donc l'étude de modèles d'attention visuelle pour l'analyse de scènes dynamiques complexes. Une carte de saliance est habituellement obtenue par la fusion d'une carte statitque (saliance spatiale dans une image) d'une part, et d'une carte dynamique (salience temporelle entre une série d'image) d'autre part. Dans notre travail, nous modélisons les changements dynamiques par un opérateur de texture LBP-TOP (Local Binary Patterns) et nous utilisons l'information couleur pour l'aspect spatial.Les deux cartes de saliances sont calculées en utilisant une formulation discriminante inspirée du système visuel humain, et fuionnées de manière appropriée en une carte de saliance spatio-temporelle.De nombreuses expériences avec des bases de données publiques, montrent que notre approche obteint des résulats meilleurs ou comparables avec les approches de la littérature. / Visual saliency is an important research topic in the field of computer vision due to its numerouspossible applications. It helps to focus on regions of interest instead of processingthe whole image or video data. Detecting visual saliency in still images has been widelyaddressed in literature with several formulations. However, visual saliency detection invideos has attracted little attention, and is a more challenging task due to additional temporalinformation. Indeed, a video contains strong spatio-temporal correlation betweenthe regions of consecutive frames, and, furthermore, motion of foreground objects dramaticallychanges the importance of the objects in a scene. The main objective of thethesis is to develop a spatio-temporal saliency method that works well for complex dynamicscenes.A spatio-temporal saliency map is usually obtained by the fusion of a static saliency mapand a dynamic saliency map. In our work, we model the dynamic textures in a dynamicscene with Local Binary Patterns (LBP-TOP) to compute the dynamic saliency map, andwe use color features to compute the static saliency map. Both saliency maps are computedusing a bio-inspired mechanism of Human Visual System (HVS) with a discriminantformulation known as center surround saliency, and are fused in a proper way.The proposed models have been extensively evaluated with diverse publicly availabledatasets which contain several videos of dynamic scenes. The evaluation is performed intwo parts. First, the method in locating interesting foreground objects in complex scene.Secondly, we evaluate our model on the task of predicting human observers fixations.The proposed method is also compared against state-of-the art methods, and the resultsshow that the proposed approach achieves competitive results.In this thesis we also evaluate the performance of different fusion techniques, because fusionplays a critical role in the accuracy of the spatio-temporal saliency map. We evaluatethe performances of different fusion techniques on a large and diverse complex datasetand the results show that a fusion method must be selected depending on the characteristics,in terms of color and motion contrasts, of a sequence. Overall, fusion techniqueswhich take the best of each saliency map (static and dynamic) in the final spatio-temporalmap achieve best results.

In this thesis, we discuss three image processing topics: High Dynamic Range (HDR) image creation in scenes with motion, Tone Mapping (TM), and Demosaicking. The first part of this thesis focuses on the creation of HDR images using gradient fusion techniques, and proposes a method that deals with motion and avoids bleeding and ghost artifacts. In the second part, we tackle the TM problem, whose goal is to produce a low dynamic range picture from an HDR image that reproduces the sensation of an observer in the scene. We review the perceptual principles that we find important for TM purposes and present a new method that compares well to the state of the art. Finally, we propose a new method to reconstruct the three color channels of a picture taken with a Bayer filter. This problem is called Demosaicking and will be presented in the third part of this thesis. / En esta tesis tratamos tres temas de procesamiento de imagen: creación de imágenes de alto rango dinámico o HDR, Tone Mapping (TM) y Demosaicking. En la primera parte proponemos un método para la creación de imágenes HDR con movimiento que permite generar resultados sin artefactos de tipo bleeding y ghosting. En la segunda parte de la tesis tratamos el problema de TM cuyo objetivo es comprimir el rango dinámico de una imagen HDR para ser mostrada en una pantalla o impresa, simulando lo mejor posible la percepción de un sujeto en la escena. Presentaremos los principios sicofísicos que consideramos relevantes para TM y propondremos un método nuevo que mejora los resultados del estado del arte. Finalmente, en la tercera parte presentamos un método de Demosaicking o reconstrucción de los tres canales de color de una imagen tomada con un filtro de Bayer.

[pt] O objetivo deste trabalho é desenvolver uma solução completa para o traçado de raios de cenas dinâmicas utilizando a GPU. Para que este algoritmo atinja desempenho interativo, é necessário utilizar uma estrutura espacial para reduzir os testes de interseção entre raios e triângulos da cena. Observa-se que, quando há movimento na cena, é necessário atualizar esta estrutura de aceleração, seja alterando-a parcialmente ou reconstruindo-a inteiramente. Adotamos a segunda estratégia por ser capaz de tratar o caso geral de movimento não-estruturado. Como a construção da estrutura deve ser feita da forma mais eficiente possível, escolhemos utilizar uma Grade Uniforme como foco de nossa pesquisa. Suas vantagens incluem um algoritmo de construção simples e um percurso de raios eficiente. Para explorar o poder de processamento em paralelo de uma GPU, é necessário manter os dados da cena e da estrutura de aceleração dentro da placa gráfica, evitando transferências custosas de memória entre a GPU e a CPU. Propomos neste trabalho uma técnica para construir uma grade uniforme inteiramente na GPU. Usando nosso método, é possível reconstruir toda a estrutura em poucos milissegundos, enquanto mantém-se a alta qualidade da grade obtida. Além disso, propomos uma implementaçoes do algoritmo de traçado de raios de forma a aproveitar o processamento em paralelo da GPU. Nosso procedimento é implementado inteiramente dentro da placa gráfica, onde há acesso direto para os dados dos triângulos da cena, bem como as informações da grade uniforme construída. Utilizando a solução proposta, somos capazes de obter taxas de visualização interativas mesmo para cenas com movimentos não-estruturados, incluindo texturas, sombras e até mesmo reflexões. / [en] We present a technique for ray tracing dynamic scenes using the GPU. In order to achieve interactive rendering rates, it is necessary to use a spatial structure to reduce the number of ray-triangle intersections performed. Whenever there is movement in the scene, this structure is entirely rebuilt. This way, it is possible to handle general unstructured motion. For this purpose, we have developed an algorithm for reconstructing Uniform Grids entirely inside the graphics hardware. In addition, we present ray-traversal and shading algorithms fully implemented on the GPU, including textures, shadows and reections. Combining these techniques, we demonstrate interactive ray tracing performance for dynamic scenes, even with unstructured motion and advanced shading effects.