The human visual system gathers information about three-dimensional object shape from a wide range of sources. How effectively we can use these sources, and how they are combined to form a consistent and accurate percept of the 3D world is the focus of much research. In complex scenes inter-reflections of light between surfaces (mutual illumination) can occur, creating chromatic illumination gradients. These gradients provide a source of information about 3D object shape, but little research has been conducted into the capabilities of the visual system to use such information. The experiments described here were conducted with the aim of understanding the influence of chromatic gradients from mutual illumination on 3D shape perception. Psychophysical experiments are described that were designed to investigate: If the human visual system takes account of mutual illumination when estimating 3D object shape, and how this might occur; How colour shading cues are integrated with other shape cues; The relative influence on 3D shape perception of achromatic (luminance) shading and chromatic shading from mutual illumination. In addition, one chapter explores a selection of mathematical models of cue integration and their applicability in this case. The results of the experiments suggest that the human visual system is able to quickly assess and take account of colour mutual illuminations when estimating 3D object shape, and use chromatic gradients as an independent and effective cue. Finally, mathematical modelling reveals that the chromatic gradient cue is likely integrated with other shape cues in a way that is close to statistically optimal.

No description available.

The human visual system gathers information about three-dimensional object shape from a wide range of sources. How effectively we can use these sources, and how they are combined to form a consistent and accurate percept of the 3D world is the focus of much research. In complex scenes inter-reflections of light between surfaces (mutual illumination) can occur, creating chromatic illumination gradients. These gradients provide a source of information about 3D object shape, but little research has been conducted into the capabilities of the visual system to use such information. The experiments described here were conducted with the aim of understanding the influence of chromatic gradients from mutual illumination on 3D shape perception. Psychophysical experiments are described that were designed to investigate: If the human visual system takes account of mutual illumination when estimating 3D object shape, and how this might occur; How colour shading cues are integrated with other shape cues; The relative influence on 3D shape perception of achromatic (luminance) shading and chromatic shading from mutual illumination. In addition, one chapter explores a selection of mathematical models of cue integration and their applicability in this case. The results of the experiments suggest that the human visual system is able to quickly assess and take account of colour mutual illuminations when estimating 3D object shape, and use chromatic gradients as an independent and effective cue. Finally, mathematical modelling reveals that the chromatic gradient cue is likely integrated with other shape cues in a way that is close to statistically optimal.

Judgments that demand consideration of pieces of information in the environment occur repeatedly throughout our lives. One professional example is that of a physician that considers multiple symptoms to make a judgment about a patient’s disease. The scientific study of such, so called, multiple-cue judgments that involve multiple pieces of information (cues: e.g., symptoms) and continuous criterion (e.g., blood pressure) has been concerned with the statistical modelling of judgment data (see Brehmer, 1994; Cooksey, 1996; Hammond & Stewart, 2001). In this thesis behavioural experiments, cognitive modelling and brain imaging is used to investigate an adaptive division of labor between multiple memory representations in multiple-cue judgment. It is hypothesized that the additive, independent linear effect of each cue can be explicitly abstracted and integrated by a serial, additive judgment process (Einhorn, Kleinmuntz, & Kleinmuntz, 1979). It is further hypothesized that a variety of sophisticated task properties, like non-additive cue combination, nonlinear relations, and inter-cue correlation, are carried implicitly by exemplar-memory (Medin & Schaffer, 1978; Nosofsky, 1984; Nosofsky & Johansen, 2000). Study I and II investigates the effect of additive versus non-additive cue-combination and verify the predicted shift in cognitive representations as a function of the underlying cue-combination rule. The third study is a review that discusses the nature of these representational shifts; are they contingent upon early perceived learning performance instead of automatic and error-driven? Study IV verifies that this shift is evident also in the neural activity associated with making judgments in additive and non-additive tasks.

During multisensory integration, information from distinct sensory systems that refers to the same physical event is combined. For example, the sound and image that an individual generates as s/he interacts with the world, will provide the nervous system with multiple cues which can be integrated to estimate the individual’s position in the environment. However, the information that is perceived through different sensory pathways/systems can be qualitatively different. The information can be redundant and describe the same property of an event in a common reference frame (i.e., the image and sound referring to the individual’s location), or it can be complementary. Combining complementary information can be advantageous in that it extends the range and richness of the information available to the nervous system, but can also be superfluous and unnecessary to the task at hand – i.e. olfactory cues about the individuals perfume can increase the richness of the representation but not necessarily aid in localisation. Over the last century or so, a large body of research has focused on different aspects of multisensory interactions at both the behavioural and neural levels. It is currently unclear whether the mechanisms underlying multisensory interactions for both type of cue are similar or not. Moreover, the evidence for differences in behavioural outcome, dependent on the nature of the cue, is growing. Such cue property effects possibly reflect a processing heuristic for more efficient parsing of the vast amount of sensory information available to the nervous system at any one time. The present thesis assesses the effects of cue properties (i.e., redundant or complementary) on multisensory processing and reports a series of experiments demonstrating that the nature of the cue, defined by the task of the observer, influences whether the cues compete for representation as a result of interacting, or whether instead multisensory information produces an optimal increase in reliability of the event estimate. Moreover, a bridging series of experiments demonstrate the key role of redundancy in inferring that two signals have a common physical cause and should be integrated, despite conflict in the cues. The experiments provide insights into the different strategies adopted by the nervous system and some tentative evidence for possible, distinct underlying mechanisms.

In recent years, computational neuroscientists have suggested that human behaviour, including perception, occurs in a manner consistent with Bayesian inference. According to the Bayesian ideal observer model, the observer combines cues from multiple sensory streams as a weighted average based on each cue’s reliability. Most cue-combination research has focused on integration of cues between sensory modalities or within the visual modality. Cue combination within the tactile modality has been relatively rarely studied, and it is still not known whether cues from individual digits combine optimally. In this thesis, we use the ideal observer model to determine whether cues from three different digits are combined optimally. We predicted that cues from multiple digits would be combined according to the optimal cue combination model. To test our hypothesis, we devised a two-interval forced choice (2IFC) task where participants had to discriminate the distal/proximal location of a 1-mm thick edge across the fingerpad(s) of the index (D2), middle (D3), and ring (D4) fingers. We used a Bayesian adaptive method, the ψ method, to compute participants’ psychometric functions for single-digit (D2, D3, and D4) and multiple-digit (D23, D24, D34, and D234) conditions. We determined the stimulus level ∆x, the distance (mm) between the distal and proximal stimuli locations, at 76% correct probability. This distance corresponds to a sensitivity index d'=1 and is the σ value of the participant’s stimulus measurement distribution. We then used the single-digit σ values to predict optimal cue combination for the multiple-digits combinations. We did not observer optimal cue-combination between the digits in this study. We outline potential implications the results of this experimental have on determining how the nervous system combines cues between digits, focusing on theoretical and experimental updates to the experiment that might result in the observation of optimal cue combination between digits. / Thesis / Master of Science (MSc)