We experience the world through our senses. But we can only make sense of the incoming information because it is weighted and interpreted against our perceptual experience which we gather throughout our lives. In this thesis I present several approaches we used to investigate the learning of prior-experience and its utilization for prediction-based computations in decision making. Teaching participants new categories is a good example to demonstrate how new information is used to learn about, and to understand the world. In the first study I present, we taught participants new visual categories using a reinforcement learning paradigm. We recorded their brain activity before, during, and after prolonged learning over 24 sessions. This allowed us to show that initial learning of categories occurs relatively late during processing, in prefrontal areas. After extended learning, categorization occurs early during processing and is likely to occur in temporal structures. One possible computational mechanism to express prior information is the prediction of future input. In this thesis, I make use of a prominent theory of brain function, predictive coding. We performed two studies. In the first, we showed that expectations of the brain can surpass the reliability of incoming information: In a perceptual decision making task, a percept based on fill-in from the physiological blind spot is judged as more reliable to an identical percept from veridical input. In the second study, we showed that expectations occur between eye movements. There, we measured brain activity while peripheral predictions were violated over eye movements. We found two sets of prediction errors early and late during processing. By changing the reliability of the stimulus using the blind spots, we in addition confirm an important theoretical idea: The strength of prediction-violation is modified based on the reliability of the prediction. So far, we used eye-movements as they are useful to understand the interaction between the current information state of the brain and expectations of future information. In a series of experiments we modulated the amount of information the visual system is allowed to extract before a new eye movement is made. We developed a new paradigm that allows for experimental control of eye-movement trajectories as well as fixation durations. We show that interrupting the extraction of information influences the planning of new eye movements. In addition, we show that eye movement planning time follow Hick's law, a logarithmic increase of saccadic reaction time with increasing number of possible targets. Most of the studies presented here tried to identify causal effects in human behavior or brain-computations. Often direct interventions in the system, like brain stimulation or lesions, are needed for such causal statements. Unfortunately, not many methods are available to directly control the neurons of the brain and even less the encoded expectations. Recent developments of the new optogenetic agent Melanopsin allow for direct activation and silencing of neuronal cells. In cooperation with researchers from the field of optogenetics, we developed a generative Bayesian model of Melanopsin, that allows to integrate physiological data over multiple experiments, include prior knowledge on bio-physical constraints and identify differences between proteins. After discussing these projects, I will take a meta-perspective on my field and end this dissertation with a discussion and outlook of open science and statistical developments in the field of cognitive science.

Seeing is more than sight: it is the entire action-perception loop involved in taking in the world around us. Unlike a camera, our eyes can only resolve a small part of the environment sharply. Therefore, we must constantly move our eyes to scrutinise the parts of our environment that seem most worthy of our highest visual acuity. Eye movements are thus the observable consequences of a complex and crucial decision-making process that is fundamental to how we interact with the world. This thesis investigates properties and the neural basis of eye-movement behavior in humans and monkeys. In the interdisciplinary tradition of cognitive science, the thesis spans fields and utilizes computational models as explanatory vehicles. A central theme is the so-called saliency map model of attention, the de facto computational model of viewing behavior. The saliency map model assumes that attention is directed at the peaks of a map that encodes the saliency of locations in the visual field. Saliency can roughly be thought of as how worthy a location is of attention. It forms a common currency that allows different processes to influence the distribution of attention. The four different studies in this thesis provide four different perspectives on viewing behavior and the saliency map model. The first study establishes a methodology to evaluate the predictive power of models of viewing behavior, and determines which properties of viewing behavior are important for this evaluation. Applying this methodological foundation to the saliency map model reveals that state-of-the-art models do not provide satisfactory explanations of viewing behavior. The second study investigates spatio-temporal properties of eye-movements, finding that observers often re-fixate locations in pictures and that their eye movements possess a rich spatio-temporal structure. These results speak directly against a causal role of "inhibition of return", which is a popular component of many saliency map models. The third study shifts focus to the neural basis of the oculomotor behaviour. fMRI is used to probe the relationship between the computation of saliency and actual processing in the brain. Our results, in contrast to those of other studies, suggest that early visual areas do not compute saliency, but instead compute visual features upon which the saliency map operates. Much of what we know about the neural basis of oculomotor control comes from invasive studies in animals, but it is unclear to what extent saliency computations are comparable between species. Thus, the fourth study compares the viewing behavior of monkeys and humans, to look for evidence of the same underlying processes. We find a strong similarity between the species in saliency-driven viewing behavior. The many saliency-processing areas that have been identified in monkeys therefore likely have a role in saliency processing in the human brain as well. This thesis contributes to our understanding of oculomotor control on multiple levels. The results in this thesis suggest that models of viewing behavior should treat saccade-target selection as a dynamic process where past decisions influence future decisions and where saliency varies over time. This selection process likely takes place in a distributed network in the brain which receives bottom-up input from early visual areas. Encouraged by these results, we speculate that normative and embodied models of cognition offer an explanation of oculomotor control that takes these results into account. In turn, explaining oculomotor control is an important part of the much deeper question of how our mind interacts with the world.

The real-world sensorimotor paradigm is based on the premise that sufficient ecological complexity is a prerequisite for inducing naturally relevant sensorimotor relations in the experimental context. The aim of this thesis is to embed visual attention research within the real-world sensorimotor paradigm using an innovative mobile gaze-tracking system (EyeSeeCam, Schneider et al., 2009). Common laboratory set-ups in the field of attention research fail to create natural two-way interaction between observer and situation because they deliver pre-selected stimuli and human observer is essentially neutral or passive. EyeSeeCam, by contrast, permits an experimental design whereby the observer freely and spontaneously engages in real-world situations. By aligning a video camera in real time to the movements of the eyes, the system directly measures the observer’s perspective in a video recording and thus allows us to study vision in the context of authentic human behaviour, namely as resulting from past actions and as originating future actions. The results of this thesis demonstrate that (1) humans, when freely exploring natural environments, prefer directing their attention to local structural features of the world, (2) eyes, head and body perform distinct functions throughout this process, and (3) coordinated eye and head movements do not fully stabilize but rather continuously adjust the retinal image also during periods of quasi-stable “fixation”. These findings validate and extend the common laboratory concept of feature salience within whole-body sensorimotor actions outside the laboratory. Head and body movements roughly orient gaze, potentially driven by early stages of processing. The eyes then fine-tune the direction of gaze, potentially during higher-level stages of visual-spatial behaviour (Studies 1 and 2). Additional head-centred recordings reveal distinctive spatial biases both in the visual stimulation and the spatial allocation of gaze generated in a particular real-world situation. These spatial structures may result both from the environment and form the idiosyncrasies of the natural behaviour afforded by the situation. By contrast, when the head-centred videos are re-played as stimuli in the laboratory, gaze directions reveal a bias towards the centre of the screen. This “central bias” is likely a consequence of the laboratory set-up with its limitation to eye-in-head movements and its restricted screen (Study 3). Temporal analysis of natural visual behaviour reveals frequent synergistic interactions of eye and head that direct rather than stabilize gaze in the quasi-stable eye movement periods following saccades, leading to rich temporal dynamics of real-world retinal input (Study 4) typically not addressed in laboratory studies. Direct comparison to earlier data with respect to the visual system of cats (CatCam), frequently taken as proxy for human vision, shows that stabilizing eye movements play an even less dominant role in the natural behaviour of cats. This highlights the importance of realistic temporal dynamics of vision for models and experiments (Study 5). The approach and findings presented in this thesis demonstrate the need for and feasibility of real- world research on visual attention. Real-world paradigms permit the identification of relevant features triggered in the natural interplay between internal-physiological and external-situational sensorimotor factors. Realistic spatial and temporal characteristics of eye, head and body interactions are essential qualitative properties of reliable sensorimotor models of attention but difficult to obtain under laboratory conditions. Taken together, the data and theory presented in this thesis suggest that visual attention does not represent a pre-processing stage of object recognition but rather is an integral component of embodied action in the real world.

The diploma thesis deals with the identification of elements that play a significant role in increasing the conversion rate of e-commerce websites and offers a more detailed look at several selected elements in terms of UX. The thesis also includes an assessment of the efficiency of the layout of selected elements. Qualitative me-thods – eye-tracking (n = 35), user testing (n = 36), in-depth interviews (n = 36) and quantitative method in the form of a questionnaire survey (n = 309) were used to fulfill the objective of the thesis. Based on the results, recommendations were for-mulated not only for the selected websites, but also applicable for various types of e-commerce websites.

No description available.

Indiana University-Purdue University Indianapolis (IUPUI) / A common impairment after a traumatic brain injury (TBI) is a deficit in emotional recognition, such as inferences of others’ intentions. Some researchers have found these impairments in 39\% of the TBI population. Our research information needed to make inferences about emotions and mental states comes from visually presented, nonverbal cues (e.g., facial expressions or gestures). Theory of mind (ToM) deficits after TBI are partially explained by impaired visual attention and the processing of these important cues. This research found that patients with deficits in visual processing differ from healthy controls (HCs). Furthermore, we found visual processing problems can be determined by looking at the eye tracking data developed from industry standard eye tracking hardware and software. We predicted that the eye tracking data of the overall population is correlated to the TASIT test. The visual processing of impaired (who got at least one answer wrong from TASIT questions) and unimpaired (who got all answer correctly from TASIT questions) differs significantly. We have divided the eye-tracking data into 3 second time blocks of time series data to detect the most salient individual blocks to the TASIT score. Our preliminary results suggest that we can predict the whole population's impairment using eye-tracking data with an improved f1 score from 0.54 to 0.73. For this, we developed optimized support vector machine (SVM) and random forest (RF) classifier.

No description available.

No description available.

Task-based language teaching (TBLT), a research-informed pedagogy for fostering second language (L2) learning through functional language use, advocates the use of tasks for organizing instructional content and the sequencing of tasks based on task complexity. While the focus of much research has been on the complexity of speaking and writing tasks, to date, scant research has been directed at the impact of reading task complexity, especially when aimed at the learning of subject matter (i.e., content learning). With increasing numbers of multinational learner classrooms, the effectiveness of such instruction constitutes an ever more indispensable factor in all levels of education, exerting a profound impact on the lives of millions of L2 learners as well as on the cultivation of skilled bilingual and multilingual citizens capable of applying content area knowledge to tackle society’s wider challenges such as pandemics. Adopting a within-subject design, this dissertation zeroed in on a specific type of reading task—read to summarize—examining the degree to which the manipulation of reading task complexity affected L2 learners’ reading processes (i.e., attention allocation and depth of processing) and reading outcomes (i.e., content learning and language use). 30 international students enrolled in graduate programs in the U.S. were recruited to complete three read-to-summarize tasks online while their eye and mouse movements were recorded. Follow-up stimulated recall interviews based on the eye-tracking heatmaps and mouse-tracking recordings were conducted to probe depth of processing. Written summaries served as measures of content learning and language use; additionally, familiarity ratings and short-answer responses were included to gauge learning of main ideas and specific details, respectively. Screening and exit surveys were also administered to collect participants’ demographic information and task perception ratings. Data analyses were performed in Python 3.9 and R Studio 2021.9.1. Findings from the language use measures show that the most complex task, in general, elicited greater phrasal complexity and the least complex task engendered greater amounts of subordination and coordination. As for content learning, the task of medium complexity yielded more correct major and minor idea units. These findings collectively suggest that while the most complex task was more facilitative of advanced language use, the task of medium complexity was more conducive to content learning. Regarding the results of the process measures, more complex tasks generally led to longer dwell time and more fixation counts than less complex ones. However, when disaggregating the results, the high-performing group had shorter dwell time and produced more main ideas in the most complex task than its low-performing counterpart. Results from the interview data further reveal that the high-performing group strategically engaged in efficient higher- and lower-level processing, whereas the low-performing group tended to demonstrate inefficient lower-level processing. Furthermore, focused analyses of four participants uncover a great deal of individual variability both in online processing and in the resulting learning outcomes. These findings are discussed in relation to the comprehension and production processes as encapsulated within one pedagogic task; theoretical, methodological, and pedagogical implications are expounded.

This thesis investigates individual differences and their impact on reading proficiency using different measures of proficiency, a variety of data collection and statistical methods, and different populations. The goal was to examine the impact that individual differences in certain reading-related skills and cognitive abilities have on reading proficiency and how these differences interact. Through three key studies that make up this thesis, several important discoveries and contributions were made to the field. Chapter 2 introduces an easy-to-use application for measuring cloze probability. ‘ClozApp’, was created and made publicly available, along with a user manual and sample code for programming. Chapter 3 contributed through the development of a novel statistical method used to analyze variance between populations with different linguistic backgrounds. This method was used to demonstrate how an individual’s linguistic background (i.e., whether they were first- or second-language speakers of English) impacted how individual differences in reading skills influence their reading fluency, as indicated through their eye-movements. This statistical prediction method is open source and was made widely available for use along with sample data and code. In Chapter 4, a new connection was found between two important cognitive factors that are well-known in the reading literature: statistical learning and motivation. Using mediation analyses, this project discovered an interaction between these factors that further highlights the ways they impact reading proficiency. This thesis demonstrates a comprehensive approach to investigating individual differences in reading proficiency in the following ways: (i) both reading fluency and comprehension were investigated as measures of reading proficiency, (ii) data collection included a variety of reading-related skills, cognitive abilities, and group differences, and (iii) unique statistical analysis methods were utilized to investigate both individual and group differences. This thesis highlights important new discoveries and makes significant lasting contributions to the field of reading research. / Thesis / Doctor of Philosophy (PhD) / This thesis investigates how individual differences influence reading proficiency. Specifically, it asks how the ways in which people differ on certain reading-related skills and cognitive abilities can determine how well they read. Using different measures of proficiency, a variety of data collection and statistical methods, and looking across different populations, the goal of this thesis was to examine the ways in which people differ in these skills and abilities, how these differences interact, and the resulting impact on reading proficiency. This thesis resulted in three significant contributions to the field. First, it made available a new application for collecting data on an important variable in reading research – cloze probability. In addition, it culminated in the development of a novel statistical method that demonstrates how an individual’s linguistic background can influence their reading fluency. Finally, a new connection was found between two important cognitive factors that interact to influence reading comprehension.