The current thesis explores semantic and action effects on visual perception and specifically how higher-level knowledge can co-affect the process of visual perception, along with the well established effects of low level image characteristics, such as colour, image/object saliency and general gist of the scene. Recent evidence on object recognition supports perceptual grouping effects of familiar pairings of functionally interacting objects. This leads to an advantage for their perception as compared with objects positioned in a non-interacting configuration, in cases where there are attentional limitations in perception. Similar effects were previously reported in clinical cases of people diagnosed with neglect (Humphreys & Riddoch, 2001, 2007; Riddoch et al, 2003, 2006), but the fact that they are also present in normally functioning individuals (Green & Hummel, 2006) makes them a clear example of higher order effects on perception. Given the evidence about the abstract nature of the information stored in visual memory and the fact that orientation is part of the spatial information related to an object representation, our first series of experiments aimed at further exploring the nature of this perceptual grouping and whether objects separation would have an effect on it. By combining this paradigm with a paradigm used to explore linguistic factors of perceiving space (Carlson-Radvansky & Radvansky, 1996; Carlson-Radvansky, Covey & Lattanzi, 1999; Carlson-Radvansky & Tang, 2000), we additionally explored the effect of functional interactions at higher levels of post-perceptual processing. We manipulated the locations of various pairs of objects as well as the semantic and functional relationship between them to explore if spatial configurations affect the way people talk about the relationship of the objects in the same way as they affect the same objects‟ recognition. The results revealed a difference, with the same distance manipulation affecting linguistic descriptions of spatial relationships between pairs of objects but having no effect in their perceptual grouping. One of the aims of this thesis is the interpretation of such effects according to a recently growing body of evidence on the interaction between action and perception systems. These systems which were traditionally considered to be two separate disciplines seem to connect, with information from action systems feeding on perceptual systems. Through such an interaction, for example, information about the functionally related objects could lead to their perceptual grouping. A series of experiments have demonstrated effects of action affordances on object perception and their combined results seem to imply pre-attentive effects on object perception independent of the person‟s intention to act on an object (Riddoch, Humphreys, Edwards, Baker & Wilson, 2002; Tipper, Paul & Hayes, 2006; Symes, Ellis & Tucker, 2007).To further explore the role of functional relationships and action affordances in natural scene viewing, a second series of experiments was designed. These experiments also provided evidence to an old debate about the nature of visual memory and its organisation, adding further evidence for the role of semantic relationship and action affordances in the memory encoding of a scene. This series of experiments took advantage of the phenomenon of object prioritization during unexpected object onsets or feature changes while viewing real world scenes (Brockmole & Henderson, 2005a). Using a variation of classic change detection paradigms, eye-tracking data were recorded to measure at which point action affordance manipulations would have an effect and to reveal whether object functionality changes can still produce attention capture (quantified as fixation probability to the object of interest), similarly to previously tested semantic changes. Functionality manipulation was achieved by orientation changes of a critical object in the scene, but in a way which constitutes it non functional to the specific context. By comparing action affordance interference during object onsets against interference during object orientation changes we differentiated between pre-attentive and post-selection mechanisms. Our results indicate that although there is no evidence of pre-attentive modulation of object prioritization, action affordances do have an effect in post-selection mechanisms, with functionally inconsistent objects attracting attention faster and affecting the encoding of an object in the scene representation during memory guided prioritization but not during oculomotor capture. Our results also support the existence of two separate mechanisms for object prioritization. As a summary, this family of semantic relationships, action affordances and the interplay between action and perception systems has been tested during my PhD research from the very early stages of perception until post perceptual and linguistic accounts of the perceived image. Their role in attention capture and their mediating role to visual memory have also been explored using eye-tracking technology and realistic and rich in information real world scenes. Overall my thesis is oriented towards the aspects that tie all these effects together and further explores the role of action affordances in memory encoding.

Conduire un véhicule dans le brouillard requiert d’intégrer de l’information visuelle bruitée avec des attentes sur la scène routière pour rechercher des indices visuels importants pour la navigation. Les tâches d’identification et de recherche visuelle peuvent être vues comme des processus de prise de décision où l’information est accumulée et où des attentes sur l’objet et son contexte sont intégrées. L’accumulation d’information est souvent modélisée comme un processus passif. Cette thèse vise à mettre en avant des mécanismes actifs, intégrant les attentes sur la cible (sur sa position, sur son identité) et de l’information sensorielle dégradée (e.g., brouillard). Nous avons employé le paradigme de mouse-tracking, permettant d’inférer des aspects dynamiques du processus de prise de décision via les mouvements de la souris d’ordinateur. L’Étude 1 évalue l’effet du contexte dans la catégorisation de cible et suggère un compromis entre rapidité et exactitude de l’accumulation d’évidence pouvant être vu comme influençant activement la décision. Mais elle n’évalue pas directement la collecte active d’évidence. Les Études 2 et 3 incluent la mesure de la détection et vérification de la cible via les mouvement des yeux lors de la recherche visuelle dans des scènes dégradée. Les attentes sur la localisation (Étude 2) et sur l’identité de la cible (Étude 3) sont manipulées. Ces études éclairent les contributions de la détection et de la vérification dans l’accumulation d’évidence pour la réponse cible absente et cible présente. Pour conclure, nous proposons une ébauche de modèle de prise de décision intégrant une dynamique entre accumulation d’évidence et système oculomoteur. / Driving a vehicle in the fog requires the integration of noisy visual information with expectations about the visual road scene, in order to search for visual clues important for navigating. The visual search and identification of relevant objects can be seen as decision-making processes where sensory information is accumulated and where the expectations about the target object and its context are integrated. The accumulation of information is often modelled as a passive process. This thesis focuses on the contribution of active mechanisms integrating expectations about the target (its identity, its location) with degraded sensory information (with fog or artificial noise). We used the mouse-tracking paradigm, allowing to infer dynamic aspects of the decision-making process through a computer mouse movements. Study 1 evaluates the effect of the context for categorizing a target and suggests a trade-off between the speed and accuracy of the evidence accumulation process which can be seen as actively influencing the decision. But this study cannot directly evaluate the active collection of evidence. In Studies 2 and 3, target detection and verification are directly measured through eye movements during visual search tasks in visually degraded scenes. We manipulated the expectations about the location (Study 2) and the target’s identity (Study 3). These studies emphasize the contributions of the detection and verification processes in the accumulation of evidence toward the target present and target absent responses. In conclusion, we propose the draft of a decision-making model which integrates the dynamics between the accumulation of evidence, and the oculomotor system.