The real-world sensorimotor paradigm is based on the premise that sufficient ecological complexity is a prerequisite for inducing naturally relevant sensorimotor relations in the experimental context. The aim of this thesis is to embed visual attention research within the real-world sensorimotor paradigm using an innovative mobile gaze-tracking system (EyeSeeCam, Schneider et al., 2009). Common laboratory set-ups in the field of attention research fail to create natural two-way interaction between observer and situation because they deliver pre-selected stimuli and human observer is essentially neutral or passive. EyeSeeCam, by contrast, permits an experimental design whereby the observer freely and spontaneously engages in real-world situations. By aligning a video camera in real time to the movements of the eyes, the system directly measures the observer’s perspective in a video recording and thus allows us to study vision in the context of authentic human behaviour, namely as resulting from past actions and as originating future actions. The results of this thesis demonstrate that (1) humans, when freely exploring natural environments, prefer directing their attention to local structural features of the world, (2) eyes, head and body perform distinct functions throughout this process, and (3) coordinated eye and head movements do not fully stabilize but rather continuously adjust the retinal image also during periods of quasi-stable “fixation”. These findings validate and extend the common laboratory concept of feature salience within whole-body sensorimotor actions outside the laboratory. Head and body movements roughly orient gaze, potentially driven by early stages of processing. The eyes then fine-tune the direction of gaze, potentially during higher-level stages of visual-spatial behaviour (Studies 1 and 2). Additional head-centred recordings reveal distinctive spatial biases both in the visual stimulation and the spatial allocation of gaze generated in a particular real-world situation. These spatial structures may result both from the environment and form the idiosyncrasies of the natural behaviour afforded by the situation. By contrast, when the head-centred videos are re-played as stimuli in the laboratory, gaze directions reveal a bias towards the centre of the screen. This “central bias” is likely a consequence of the laboratory set-up with its limitation to eye-in-head movements and its restricted screen (Study 3). Temporal analysis of natural visual behaviour reveals frequent synergistic interactions of eye and head that direct rather than stabilize gaze in the quasi-stable eye movement periods following saccades, leading to rich temporal dynamics of real-world retinal input (Study 4) typically not addressed in laboratory studies. Direct comparison to earlier data with respect to the visual system of cats (CatCam), frequently taken as proxy for human vision, shows that stabilizing eye movements play an even less dominant role in the natural behaviour of cats. This highlights the importance of realistic temporal dynamics of vision for models and experiments (Study 5). The approach and findings presented in this thesis demonstrate the need for and feasibility of real- world research on visual attention. Real-world paradigms permit the identification of relevant features triggered in the natural interplay between internal-physiological and external-situational sensorimotor factors. Realistic spatial and temporal characteristics of eye, head and body interactions are essential qualitative properties of reliable sensorimotor models of attention but difficult to obtain under laboratory conditions. Taken together, the data and theory presented in this thesis suggest that visual attention does not represent a pre-processing stage of object recognition but rather is an integral component of embodied action in the real world.

No description available.

There is a difference between the raw sensory input to the brain and our stable perception of entities in the environment. A first approach to investigate perception is to study relationships between properties of currently presented stimuli and biological correlates of perceptual processes. However, it is known that such processes are not only dependent on the current stimulus. Sampling of information and the concurrent neuronal processing of stimulus content rely on contextual relationships in the environment, and between the environment and the body. Perceptual processes dynamically adjust to relevant context, such as the current task of the organism and its immediate history. To understand perception, we have to study how processing of current stimulus content is influenced by such contextual factors. This thesis investigates the influence of such factors on visual processing. In particular, it investigates effects of temporal context in early visual processing and the effect of task context in eye movement control. To investigate effects of contextual factors on early visual processing of current stimulus content, we study neuronal processing of visual information in the primary visual cortex. We use real-time optical imaging with voltage sensitive dyes to capture neuronal population activity in the millisecond range across several millimeters of cortical area. To characterize the cortical layout concerning the mapping of orientation, previous to further investigations, we use smoothly moving grating stimuli. Investigating responses to this stimulus type systematically, we find independent encoding of local contrast and orientation, and a direct mapping of current stimulus content onto cortical activity (Study 1). To investigate the influence of the previous stimulus as context on processing of current stimulus content, we use abrupt visual changes in sequences of modified natural images. In earlier studies, investigating relatively fast timescales, it was found that the primary visual cortex continuously represents current input (ongoing encoding), with little interference from past stimuli. We investigate whether this coding scheme generalizes to cases in which stimuli change more slowly, as frequently encountered in natural visual input. We use sequences of natural scene contours, comprised of vertically and horizontally filtered natural images, their superpositions, and a blank stimulus, presented with 10 or 33 Hz. We show that at the low temporal frequency, cortical activity patterns do not encode the present orientations but instead reflect their relative changes in time. For example, when a stimulus with horizontal orientation is followed by the superposition of both orientations, the pattern of cortical activity represents the newly added vertical orientations instead of the full sum of orientations. Correspondingly, contour removal from the superposition leads to the representation of orientations that have disappeared rather than those that remain. This is in sharp contrast to more rapid sequences for which we find an ongoing representation of present input, consistent with earlier studies. In summary, we find that for slow stimulus sequences, populations of neurons in the primary visual cortex are no longer tuned to orientations within individual stimuli but instead represent the difference between consecutive stimuli. Our results emphasize the influence of the temporal context on early visual processing and consequentially on information transmission to higher cortical areas (Study 2). To study effects of contextual factors on the sampling of visual information, we focus on human eye movement control. The eyes are actively moved to sample visual information from the environment. Some traditional approaches predict eye movements solely on simple stimulus properties, such as local contrasts (stimulus-driven factors). Recent arguments, however, emphasize the influence of tasks (task context) and bodily factors (spatial bias). To investigate how contextual factors affect eye movement control, we quantify the relative influences of the task context, spatial biases and stimulus-driven factors. Participants view and classify natural scenery and faces while their eye movements are recorded. The stimuli are composed of small image patches. For each of these patches we derive a measure that quantifies stimulus-driven factors, based on the image content of a patch, and spatial viewing biases, based on the location of the patch. Utilizing the participants’ classification responses, we additionally derive a measure, which reflects the information content of a patch in the context of a given task. We show that the effect of spatial biases is highest, that task context is a close runner-up, and that stimulus-driven factors have, on average, a smaller influence. Remarkably, all three factors make independent and significant contributions to the selection of viewed locations. Hence, in addition to stimulus-driven factors and spatial biases, the task context contributes to visual sampling behavior and has to be considered in a model of human eye movements. Visual processing of current stimulus content, in particular visual sampling behavior and early processing, is inherently dependent on context. We show that already in the first cortical stage, temporal context strongly affects the processing of new visual information and that visual sampling by eye movements is significantly influenced by the task context, independently of spatial factors and stimulus-driven factors. The empirical results presented provide foundations for an improved theoretical understanding of the role of context in perceptual processes.

Though a major illness in modern society, depression is still not completely understood. A number of empirical observations point to the importance of basic cognitive processes as well as personality variables as antecedents of a depressive disorder. In this work it is argued that “state orientation”, a personality style characterized by the inability to actively influence one’s focus of thought, plays an important role in the development of at least some forms of major depressive disorder. In the present work, it is suggested that (1) state-oriented cognitions are equivalent to sustained information processing, that (2) depressed individuals are characterized in particular by state-oriented cognitions related to prior failure experiences, that (3) sustained processing of affective information will interfere with normal executive cognitive functioning in depressed individuals resulting in impairments of normal behavior, and that (4) both sustained information processing and “affective interference” will be associated with specific dysfunctional patterns of brain activity in depressed individuals. In the first chapter of this thesis, theorizing pertaining to “action control” and the relationship between action control and state orientation are reviewed. After having established the potential functional significance of state-oriented cognitions, their possible link to depression is developed by introducing the “degenerated-intention hypothesis”. Afterwards, the role of state orientation in the advent of the depressive state is discussed against the background of the “functional helplessness” model of depression. Next, recent empirical findings related to executive dysfunction associated with state-oriented cognitions in major depressive disorder and related dysfunctional patterns of brain activity are reviewed. By considering evidence from studies on executive functioning, brain imaging, and neurophysiological studies, support is found for a possible frontocingulate dysfunction associated with a state-oriented cognitive style underlying a major depressive disorder. Consistent with the proposed link between depression and state orientation, in the second chapter of the thesis, Studies 1a – 1c demonstrate that subclinically and clinically depressed individuals are specifically characterized by failure-related state orientation. Moreover, the results of Study 2, described in Chapter 3, reveal that sustained processing of affectively valenced information may indeed interfere with subsequent executive cognitive functioning, especially in individuals demonstrating relatively high levels of depression. Finally, in line with the idea that sustained information processing and affective interference will be related to an individual’s level of state orientation and will be reflected in specific patterns of neural activity, Study 3, presented in Chapter 4, provides considerable evidence for disturbed brain function in clinically depressed individuals during processing of affective information as well as subsequent executive cognitive functioning and its relation to state-oriented thought. The current research supports the idea that state orientation, in particular its failure-focused form, is a crucial process involved in the development and maintenance of a depressive disorder. Specifically, the present findings suggest that certain forms of major depressive disorder are associated with sustained processing of affective information and with the resulting affective interference with executive cognitive functioning. Findings further suggest that sustained information processing is experienced by affected individuals as ruminative, state-oriented thought on past aversive experiences, and that both sustained information processing and affective interference are associated with distinct patterns of brain activity, which are related to early stimulus evaluation, conflict monitoring, and conflict resolution. The processes possibly underlying some forms of depression, as proposed in this thesis, comprise what may be called “the spinning mind”, whose important functional significance is to hinder an individual from adaptive behavior by impairing the ability to direct thought. Although state orientation may therefore appear to be maladaptive per se, it may be argued instead that this mode of action control is also an adaptive process as long as critical limits of certain parameters are met and the spinning mind is prevented. These and similar considerations are addressed in the concluding discussion in Chapter 5.