During scene viewing, visual working memory (VWM) is used to retain information from recently attended and fixated objects. In the present study, I examined whether and how people can strategically control the content of VWM during scene viewing, prioritizing task-relevant objects for retention even as the eyes are directed to subsequent objects. Participants viewed a set of real-world objects presented serially within a 3-D rendered scene. One object in the sequence was cued by a tone as to-be-remembered. At the end of the sequence, memory for the visual form of one object was tested. Participants exhibited tight control over the content of VWM, implementing prioritization after the encoding of an object into VWM, protecting that item from subsequent interference. Participants also successfully reallocated protection to subsequent objects, regardless of the duration of prioritization of the original item. Such strategic maintenance of objects in VWM is likely to play an important role in real-world visual behavior, especially when object information must be maintained across shifts of attention and the eyes to other objects (such as when comparing two spatially separated objects).

We live in a rich visual world that we experience as a seamless and detailed stream of continuous information. However, we can only attend to and remember a small portion of our visual environment. The visual system is tasked with stitching together snapshots of the world through near constant eye movements, with around three saccades per second. The situation is further complicated with the visual system being contralaterally organized. Each eye movement can bring items in our environment into a different visual hemifield. Despite the many challenges and limitations of attention and the visual system, how does the brain stitch together our experience of our visual environment? One potential mechanism that could contribute to our conscious perception of a continuous visual experience could be visual working memory (VWM) working to maintain representations of items across saccades. Electrophysiological activity using event-related potentials has revealed the contralateral delay activity (CDA), which is a sustained negativity contralateral to the side of the visual field where subjects are attending. However, how does this work if we are constantly moving our eyes? How do we form a stable representation of items across eye movements? Does the representation transfer over to the other side of the brain, constantly shuffling the items between the hemispheres? Or does it stay in the hemisphere contralateral to the visual field where the items were located when we originally created the representation? The consequences of eye movements need to be examined at multiple levels and time points throughout the process. The goal of my doctoral dissertation is to investigate VWM representations throughout the dynamic peri-saccadic window. In Experiment 1, I will first compare VWM representations across shifts of attention and eye position. With the focus on the effect of maintaining attention on items across eye movements, Experiment 2 will also explore eye movements both towards and away from attended visual hemifields. Finally, Experiment 3 is designed to substantiate our use of the CDA as a tool for examining VWM representations across eye movements by confirming that the CDA is indeed established in retinotopic coordinates.

In all past research, iconic memory shows a significant benefit over visual working memory for storage capacity of visual items. However, this effect has only been studied on simple items such as colors and letters. The goal of this thesis was to determine whether an iconic benefit also exists for visual stimuli with higher visual complexity, such as shapes and faces. Five experiments tested iconic and working memory capacity for complex face stimuli, intermediate-complexity shape stimuli, and simple color stimuli, as well as examining feature binding of objects in iconic memory. Results from these five experiments indicated that increased visual complexity of stimuli negatively impacts the iconic capacity benefit. High- and intermediate-complexity items had little to no iconic benefit, unlike all previously tested simple stimuli. Iconic memory may only be able to represent simple features, or may not be able to transfer complex information into visual working memory as quickly as simple information. Additionally, results showed that feature representations in iconic memory were sometimes bound into complex objects. The results of these five experiments challenge the traditional characterization of visual sensory memory as a precise snapshot; this early memory store may be more complex than a simple visual icon.

Many tasks rely on our ability to hold information about a stimulus in mind after it is no longer visible and to compare this information with incoming perceptual information. This ability relies on a short-term form of memory known as visual working memory. Research and theory at the behavioral and neural levels has begun to provide important insights into the basic properties of the neuro-cognitive systems underlying this form of memory. However, to date, no neurally-plausible theory has been proposed that addresses both the storage of information in working memory and the comparison process in a single framework. To address these limitations, I have developed a new model where working memory is realized via peaks of activation in dynamic neural fields, and comparison emerges as a result of interactions among the model's layers. In a series of simulations, I show how the model can be used to capture each of the components underlying performance in simple visual comparison tasks--from the encoding, consolidation, and maintenance of information in working memory, to comparison and updating in response to changed inputs. Importantly, the proposed model demonstrates how these elementary perceptual and cognitive functions emerge from the coordinated activity of an integrated, dynamic neural system. The model also makes novel predictions that were tested in a series of behavioral experiments. Specifically, when similar items are stored, shared lateral inhibition produces a sharpening of the peaks of activation associated with each item in memory. In the context of the model, this leads to the prediction that change detection will be enhanced for similar versus dissimilar features. This prediction was confirmed in a series of change detection experiments exploring memory for both color and orientation. In addition to sharpening, shared lateral inhibition among similar items produces mutual repulsion between nearby peaks. This leads to the prediction that when similar features are held, they will be systematically biased away from each other over delays. This prediction was confirmed in a cued color recall experiment comparing memory for a "far" color with memory for two "close" colors.

Examining the neural correlates associated with the moment a stimulus enters or exits conscious awareness is one way to potentially identify the neural mechanisms that give rise to consciousness. In the present study, we examined neural activity using electroencephalogram (EEG) recordings while participants observed a bilateral shape-from-motion (SFM) display. While the display is in motion, the observer perceives an object that is immediately segregated from a noisy background. After the motion stops, the observer’s experience of the object remains momentarily in awareness, before it eventually fades out of consciousness back into the noisy background. Consistent with subjective reports of perceptual experience, we observed a prominent sustained posterior contralateral negativity (SPCN), but only in conditions associated with sustained awareness. Importantly, the amplitude of the SPCN was correlated with individual differences in visual awareness, suggesting that this activity plays a significant role in the maintenance of objects in consciousness.

Examining the neural correlates associated with the moment a stimulus enters or exits conscious awareness is one way to potentially identify the neural mechanisms that give rise to consciousness. In the present study, we examined neural activity using electroencephalogram (EEG) recordings while participants observed a bilateral shape-from-motion (SFM) display. While the display is in motion, the observer perceives an object that is immediately segregated from a noisy background. After the motion stops, the observer’s experience of the object remains momentarily in awareness, before it eventually fades out of consciousness back into the noisy background. Consistent with subjective reports of perceptual experience, we observed a prominent sustained posterior contralateral negativity (SPCN), but only in conditions associated with sustained awareness. Importantly, the amplitude of the SPCN was correlated with individual differences in visual awareness, suggesting that this activity plays a significant role in the maintenance of objects in consciousness.

The processing of information within the visual system is limited by several cognitive and neural bottlenecks. One critical bottleneck occurs in visual working memory (VWM), as the amount of information that can be maintained on-line is limited to three to four items. While numerous theories have addressed this limited capacity of VWM, it is unclear how processing bottlenecks in the initial selection and perception of visual information affect the number or precision of representations that can be maintained in VWM. The purpose of this dissertation was to examine whether early competition for resources within the visual system limits the number or precision of representation that can be maintained in VWM. To establish whether competitive interactions affect VWM, Chapters 1 – 4 tested whether performance on VWM tasks was related to the distance between memory items. The results of these experiments reveal that when objects are presented close together in space, VWM performance is impaired relative to when those same objects are presented further apart. Using a three-component model of continuous responses in a recall task, Chapters 3 – 4 demonstrated that the distance between objects primarily affects the precision of responses, and increases the number of non-target errors. Chapter 5 extended these findings to distractors, demonstrating that multiple distractors affect the precision and accuracy of VWM responses. Chapters 6 – 7 tested how attentional selection can bias memory representations, revealing that objects that are given high attentional priority were reported with greater precision. Finally, Chapters 8 and 9 examined bias-signals as a potential source of individual differences in VWM performance, revealing that high-performers have more precise representations of sub-capacity representations than low-performers. Together, these results reveal that VWM performance is limited by competition for representation within the visual system, and that attention plays a critical role in resolving competition and consequently, determining the contents of VWM.

The processing of information within the visual system is limited by several cognitive and neural bottlenecks. One critical bottleneck occurs in visual working memory (VWM), as the amount of information that can be maintained on-line is limited to three to four items. While numerous theories have addressed this limited capacity of VWM, it is unclear how processing bottlenecks in the initial selection and perception of visual information affect the number or precision of representations that can be maintained in VWM. The purpose of this dissertation was to examine whether early competition for resources within the visual system limits the number or precision of representation that can be maintained in VWM. To establish whether competitive interactions affect VWM, Chapters 1 – 4 tested whether performance on VWM tasks was related to the distance between memory items. The results of these experiments reveal that when objects are presented close together in space, VWM performance is impaired relative to when those same objects are presented further apart. Using a three-component model of continuous responses in a recall task, Chapters 3 – 4 demonstrated that the distance between objects primarily affects the precision of responses, and increases the number of non-target errors. Chapter 5 extended these findings to distractors, demonstrating that multiple distractors affect the precision and accuracy of VWM responses. Chapters 6 – 7 tested how attentional selection can bias memory representations, revealing that objects that are given high attentional priority were reported with greater precision. Finally, Chapters 8 and 9 examined bias-signals as a potential source of individual differences in VWM performance, revealing that high-performers have more precise representations of sub-capacity representations than low-performers. Together, these results reveal that VWM performance is limited by competition for representation within the visual system, and that attention plays a critical role in resolving competition and consequently, determining the contents of VWM.

xiii, 143 p. : ill. (some col.) / The cognitive ability to group information into chunks is a well known phenomenon, however, the effects of chunking on visual representations is not well understood. Here we investigate the effects of visual chunking using Gestalt grouping principles in two tasks: visual working memory change detection and multiple object tracking. Though both these tasks have been used to study cognitive functions in the past, including object-based attention, attentional control and working memory capacity, the effect of grouping on mental representations in these tasks has not been well characterized. That is, while researches have measured effects of grouping on behavioral output in similar tasks, there are few studies of the effects of grouping on neurophysiological indices of object representations. Indeed, these current studies are the first to use event-related potentials (ERPs) to elucidate the effect of grouping on active mental representations of visual stimuli. In the visual working memory task, observers remembered either the color or orientation of pacman stimuli across a delay. We manipulated the collinearity of these objects, whether or not they formed a Kanizsa triangle figure, and measured the behavioral and electrophysiological effects. In the multiple object tracking task, a subset of identical stimuli were briefly cued as targets and then their motion was tracked by participants. We manipulated whether and which Gestalt heuristics were used to bind targets together during their motion and measured the effects on behavior and electrophysiology. In both tasks we compared the grouped to ungrouped conditions. We found that across experiments and tasks behavioral performance was enhanced in grouping conditions compared to ungrouped conditions. Furthermore, the waveforms evoked by grouped stimuli were reduced compared to waveforms produced in response to locally identical but ungrouped stimuli. These data suggest that the mental representation of visual objects may be reshaped moment-by-moment by grouping cues or task demand, giving rise to a flexible, active and dynamic yet parsimonious representation of the visual world. / Committee in charge: Edward K. Vogel, Chair; Edward Awh, Member; Ulrich Mayr, Member; Paul van Donkelaar, Outside Member

京都大学 / 0048 / 新制・課程博士 / 博士(人間・環境学) / 甲第19079号 / 人博第732号 / 新制||人||176(附属図書館) / 26||人博||732(吉田南総合図書館) / 32030 / 京都大学大学院人間・環境学研究科共生人間学専攻 / (主査)教授 齋木 潤, 教授 船橋 新太郎, 准教授 月浦 崇 / 学位規則第4条第1項該当 / Doctor of Human and Environmental Studies / Kyoto University / DGAM