Multisensory processing in the brain underlies a wide variety of perceptual phenomena, but little is known about the underlying mechanisms of how multisensory neurons are generated and how the neurons integrate sensory information from environmental events. This lack of knowledge is due to the difficulty of biological experiments to manipulate and test the characteristics of multisensory processing. By using a computational model of multisensory processing this research seeks to provide insight into the mechanisms of multisensory processing. From a computational perspective, modeling of brain functions involves not only the computational model itself but also the conceptual definition of the brain functions, the analysis of correspondence between the model and the brain, and the generation of new biologically plausible insights and hypotheses. In this research, the multisensory processing is conceptually defined as the effect of multisensory convergence on the generation of multisensory neurons and their integrated response products, i.e., multisensory integration. Thus, the computational model is the implementation of the multisensory convergence and the simulation of the neural processing acting upon the convergence. Next, the most important step in the modeling is analysis of how well the model represents the target, i.e., brain function. It is also related to validation of the model. One of the intuitive and powerful ways of validating the model is to apply methods standard to neuroscience for analyzing the results obtained from the model. In addition, methods such as statistical and graph-theoretical analyses are used to confirm the similarity between the model and the brain. This research takes both approaches to provide analyses from many different perspectives. Finally, the model and its simulations provide insight into multisensory processing, generating plausible hypotheses, which will need to be confirmed by real experimentation.

Environmental events simultaneously transduced by more than one sensory modality underlie multisensory processing in the CNS. While most studies of multisensory processing examine functional effects, none have evaluated the influence of local or columnar circuitry. The goal of the present study is to examine of local features of the ferret lateral rostral suprasylvian sulcus (LRSS), a multisensory cortex. Immunostaining revealed the cytoarchitectonic features of the LRSS: thick supragranular layers, a narrow layer IV, and moderately stained but differentiated infragranular layers. Golgi-Cox techniques were used with light microscopy and digital reconstruction to document neuronal morphology. Among the 90 reconstructed neurons, 4 distinct forms or pyramidal and 2 types of non-pyramidal neurons were found. Measurement of maximal dendritic spread indicates that a cortical column in the LRSS was 250.9 um in diameter. These results describe local features of the LRSS upon which future experiments of intrinsic circuitry will be based.

UNIQUE FEAUTRES OF ORGANIZATION AND NEURONAL PROPERTIES IN A MULTISENSORY CORTEX Multisensory processing is a ubiquitous sensory effect that underlies a wide variety of behaviors, such as detection and orientation, as well as perceptual phenomena from speech comprehension to binding. Such multisensory perceptual effects are presumed to be based in cortex, especially within areas known to contain multisensory neurons. However, unlike their lower-level/primary sensory cortical counterparts, little is known about the connectional, functional and laminar organization of higher-level multisensory cortex. Therefore, to examine the fundamental features of neuronal processing and organization in the multisensory cortical area of the posterior parietal cortex (PPr) of ferrets, the present experiments utilized a combination of immunohistological, neuroanatomical and multiple single-channel electrophysiological recording techniques. These experiments produced four main results. First, convergence of extrinsic inputs from unisensory cortical areas predominantly in layers 2-3 in PPr corresponded with the high proportion of multisensory neurons in those layers. This is consistent with multisensory responses in this higher-level multisensory region being driven by cortico-cortical, rather than thalamo-cortical connections. Second, the laminar organization of the PPr differed substantially from the pattern commonly observed in primary sensory cortices. The PPr has a reduced layer 4 compared to primary sensory cortices, which does not receive input from principal thalamic nuclei. Third, the distribution of unisensory and multisensory neurons and properties differs significantly by layer. Given the laminar-dependent input-output relationships, this suggests that unisensory and multisensory signals are processed in parallel as they pass through the circuitry of the PPr. Finally, specific functional properties of bimodal neurons differed significantly from those of their unisensory counterparts. Thus, despite their coextensive distribution within cortex, these results differentiate bimodal from unisensory neurons in ways that have never been examined before. Together these experiments represent the first combined anatomical-electrophysiological examination of the laminar organization of a multisensory cortex and the first systematic comparison of the functional properties of bimodal and unisensory neurons. These results are essential for understanding the neural bases of multisensory processing and carry significant implications for the accurate interpretation of macroscopic studies of multisensory brain regions (i.e. fMRI, EEG), because bimodal and unisensory neurons within a given neural region can no longer be assumed to respond similarly to a given external stimulus.

Yes / Background: Our motor actions normally generate sensory events, but how do we know which events were self generated and which have external causes? Here we use temporal adaptation to investigate the processing stage and generality of our sensorimotor timing estimates. Methodology/Principal Findings: Adaptation to artificially-induced delays between action and event can produce a startling percept¿upon removal of the delay it feels as if the sensory event precedes its causative action. This temporal recalibration of action and event occurs in a quantitatively similar manner across the sensory modalities. Critically, it is robust to the replacement of one sense during the adaptation phase with another sense during the test judgment. Conclusions/Significance: Our findings suggest a high-level, supramodal recalibration mechanism. The effects are well described by a simple model which attempts to preserve the expected synchrony between action and event, but only when causality indicates it is reasonable to do so. We further demonstrate that this model successfully characterises related adaptation data from outside the sensorimotor domain.

Os profissionais em design de exposição, em sua responsabilidade enquanto projetistas devem pensar sobre como desenvolver sistemas e/ou recursos para fruição artística que atendam a pessoas com deficiência visual, incluindo a forma como uma pessoa totalmente habilitada utiliza este objeto/ambiente. Assim, apresenta-­‐se a pesquisa intitulada Design para experiência multissensorial em museus: fruição de objetos culturais por pessoas com deficiência visual. Seu objetivo é propor os requisitos para o projeto e desenvolvimento de recursos multimodais táteis e sonoros para fruição de objetos culturais pelo usuário com deficiência visual em museus, segundo uma abordagem integrada com base no design para a experiência. Para tanto busca-­‐se referências em áreas como o Design para Experiência e Design Emocional, Museologia, Fabricação Digital, Comunicação Acessível, Semiótica Social e Multimodalidade, entre outros. O embasamento metodológico ampara-­‐se nas abordagens de pesquisa qualitativa e pesquisa aplicada para coleta e análise de dados referente a aferição da experiência do usuário. Assim, são empregadas as técnicas de observação direta, entrevistas e Engenharia Kansei para caracterizar e compreender as experiências dos sujeitos da pesquisa, bem como para compreender os processos de desenvolvimento e os materiais para produção de recursos para acessibilidade em museus e sua consequente avaliação. A verificação do trabalho ocorreu no Museu de Porto Alegre – Joaquim Felizardo. Os requisitos de projeto para promoção da experiência multissensorial por pessoas com deficiência visual validaram a hipótese elencada em resposta ao problema de pesquisa. Além dos requisitos de projeto, chegou-­‐se a novos instrumentos para o auxílio no projeto de recursos de acessibilidade, tal como: metodologia proposta para o desenvolvimento de recursos multimodais táteis e sonoros segundo uma abordagem integrada; diretrizes para o desenvolvimento de recursos táteis e sonoros para pessoas com deficiência visual em museus; instrumento para o relacionamento de elementos táteis e sonoros segundo uma abordagem integrada; e, instrumento para avaliação da experiência pelo usuário por meio de escalas de diferencial semântico. / Professionals in exhibition design, in their responsibility as designers should think about how to develop systems and/or resources for artistic fruition that meets the visually impaired, including how a fully qualified person uses this object/environment. Thus, presents this research entitled Design for multisensory experience in museums: fruition of cultural objects by visually impaired people. This aims to propose the requirements for the design and development of tactile and audible multimodal resources for the fruition of cultural objects by the users with visual impairment in museums, in an integrated approach based on the design for the experience. Therefore look for references in areas as the Design for Experience and Emotional Design, Museology, Digital Manufacturing, Affordable Communication, Social Semiotics and Multimodality, among others. The methodological basis employs the qualitative research approaches and applied research for data collection and analysis concerning user experience’s. Thus, use of the techniques of direct observation, interviews and Kansei Engineering to characterize and understand the experiences of research subjects, as well as understand the development of processes and materials for the production of resources for accessibility in museums and their consequent evaluation. The work of verification took place at the Museum of Porto Alegre -­‐ Joaquim Felizardo. The project requirements elected for the promotion of multisensory experience for the visually impaired validated the hypothesis in response to the research problem. In addition to the project requirements, came to new tools to aid in the project of accessibility resources, such as: proposed methodology for the development of tactile and audible multimodal resources through an integrated approach; guidelines for the development of tactile and sound resources for visually impaired people in museums; instrument for the relationship of tactile and sound elements through an integrated approach; and, tool for evaluation of the experience by the user by semantic differential scales.

Objectives: Elementary school classrooms that meet the acoustic requirements for near-optimum speech recognition are extremely scarce. Poor classroom acoustics may become a barrier to speech understanding as children enter school. The purpose of this study was threefold: 1) to quantify the extent to which reverberation, lexical difficulty, and presentation mode affect speech recognition in noise, 2) to examine to what extent auditory-visual (AV) integration assists with the recognition of speech in noisy and reverberant environments typical of elementary school classrooms, 3) to understand the relationship between developing mechanisms of multisensory integration and the concurrently developing linguistic and cognitive abilities. Design: Twenty-seven typically developing children and 9 young adults participated. Participants repeated short sentences reproduced by 10 speakers on a 30” HDTV and/or over loudspeakers located around the listener in a simulated classroom environment. Signal-to-noise ratio (SNR) for 70 (SNR70) and 30 (SNR30) percent correct performance were measured using an adaptive tracking procedure. Auditory-visual integration was assessed via the SNR difference between AV and auditory-only (AO) conditions, labeled speech-reading benefit (SRB). Linguistic and cognitive aptitude was assessed using the NIH-Toolbox: Cognition Battery (NIH-TB: CB). Results: Children required more favorable SNRs for equivalent performance when compared to adults. Participants benefited from the reduction in lexical difficulty, and in most cases the reduction in reverberation time. Reverberation affected children’s speech recognition in AO condition and adults in AV condition. At SNR30, SRB was greater than that at SNR70. Adults showed marginally significant increase in AV integration relative to children. Adults also showed increase in SRB for lexically hard versus easy words, at high level of reverberation. Development of linguistic and cognitive aptitude accounts for approximately 35% of the variance in AV integration, with crystalized and fluid cognition composite scores identified as strongest predictors. Conclusions: The results of this study add to the body of evidence in support of children requiring more favorable SNRs to perform the same speech recognition tasks as adults in simulated listening environments akin to school classrooms. Our findings shed light on the development of AV integration for speech recognition in noise and reverberation during the school years, and provide insight into the balance of cognitive and linguistic underpinnings necessary for AV integration of degraded speech.

Multisensory environments facilitate behavioral functioning in humans. The redundant signal effect (RSE) refers to the observation that individuals respond more quickly to stimuli when information is presented as multisensory, redundant stimuli rather than as a single stimulus presented to either modality alone. Our studies show that the disinhibiting effects of alcohol are attenuated when stop signals are multisensory versus unisensory. The present study expanded on this research to test the degree to which multisensory stop signals could also attenuate the disinhibiting effects of alcohol in those with attention-deficit hyperactivity disorder (ADHD), a clinical population characterized by poor impulse control. The study compared young adults with ADHD with healthy controls and examined the acute impairing effect of alcohol on response inhibition to stop signals that were presented as a unisensory stimulus or a multisensory stimulus. For controls, results showed alcohol impaired response inhibition to unisensory stop signals but not to multisensory stop signals. Response inhibition of those with ADHD was impaired by alcohol regardless of whether stop signals were unisensory or multisensory. The failure of multisensory stimuli to attenuate alcohol impairment in those with ADHD highlights a specific vulnerability that could account for heightened sensitivity to the disruptive effects of alcohol.

Tout individu extrait de ses informations sensorielles des invariants directionnels lui permettant de percevoir sa propre situation spatiale ainsi que celle des objets qui l'entourent. Dans les situations habituelles, les traitements relatifs à l'orientation spatiale s'effectuent à partir de référentiels relativement redondants. Cette redondance autorise à la fois la mise en place d'une sensibilité individuelle préférentielle à l'égard d'un de ces référentiels ainsi qu'une large flexibilité dans le choix du référentiel. Lors de ce travail nous avons voulu tester deux hypothèses principales. La 1ère est que les différences interindividuelles observées lors de tâches de perception d'orientation spatiales dépendent du choix du référentiel. La 2nd hypothèse est que l'un des facteurs explicatifs de la variabilité interindividuelle lors de tâches de perception d'orientation spatiale se situe au niveau des règles d'intégration multisensorielle utilisées par les individus. Les résultats montrent que l'ensemble des cadres de références semblent pris en compte par les individus rejetant ainsi la 1ère hypothèse selon laquelle les différences interindividuelles s'expliqueraient par la capacité de certains sujets à exploiter le "bon" cadre de référence. Les résultats montrent que l'un des facteurs explicatifs des différences interindividuelles se situe dans la manière dont le système nerveux central combinerait les différentes informations, certains sujets (DC) semblant incapables de minimiser l'influence des "mauvais" cadre de référence (les moins appropriés, les plus biaisés) en diminuant leur poids respectifs. / Each person extract from sensory information directional invariant which allow him to perceive his spatial situation as well as those of objects surround him. In usual situations, processing relative to spatial orientation is carried out from relatively redundant reference. This redundancy allows both the setting up of preferential individual sensibility regarding those reference and important flexibility in choice of reference. Durant this work, we tested two main hypotheses. The first one is that observed interindividual differences during spatial orientation perception task depend on reference choice. The second one is that one of explanatory factors of interindividual differences during spatial orientation perception task be set in multisensory integration rules level used by the person. Results show that all frame of reference seem to consider by the people rejecting the first hypothesis according to which interindividual differences be explained by the ability of some subjects to exploit the "good" frame of reference. Results show that one of explanatory factors of interindividual differences is set in the way of central nervous system combine several information, some subjects (DC) seems unable to minimize "wrong" frame of reference influence (the least appropriate, the most biased) by reducing respective weight.

Our ability to interact with the environment requires the integration of multisensory information for the construction of spatial representations. The peripersonal space (i.e., the sector of space closely surrounding one's body) and the integrative processes between visual and tactile inputs originating from this sector of space have been at the center of recent years investigations. Neurophysiological studies provided evidence for the presence in the monkey brain of bimodal neurons, which are activated by tactile as well as visual information delivered near to a specific body part (e.g., the hand). Neuropsychological studies on right brain-damaged patients who present extinction and functional neuroimaging findings suggest the presence of similar bimodal systems in the human brain. Studies on the effects of tool-use on visual-tactile interaction revealed similar dynamic properties of the peripersonal space in monkeys and humans. The functional role of the multisensory coding of peripersonal space is, in our hypothesis, that of providing the brain with a sensori-motor interface for body-objects interactions. Thus, not only it could be involved in driving involuntary defensive movements in response to objects approaching the body, but could be also dynamically maintained and updated as a function of manual voluntary actions performed towards objects in the reaching space. We tested the hypothesis of an involvement of peripersonal space in executing both voluntary and defensive actions. To these aims, we joined a well known cross-modal congruency effect between visual and tactile information to a kinematic approach to demonstrate that voluntary grasping actions induce an on-line re-weighting of multisensory interactions in the peripersonal space. We additionally show that this modulation is handcentred. We also used a motor evoked potentials approach to investigate which coordinates system is used to code the peripersonal space during motor preparation if real objects rapidly approach the body. Our findings provide direct evidence for automatic hand-centred coding of visual space and suggest that peripersonal space may also serve to represent rapidly 3 approaching and potentially noxious objects, thus enabling the rapid selection of appropriate motor responses. These results clearly show that peripersonal space is a multisensori-motor interface that might have been selected through evolution for optimising the interactions between the body and the objects in the external world.

The human sensory system processes many modalities simultaneously. It was believed that each modality would be processed individually first, and their combination deferred to higher-level cortical areas. Recent neurophysiological investigations indicate interconnections between early visual and auditory cortices, areas putatively considered unimodal, but the function remains unclear. The present work explores how this cross-modality might contribute to a visual enhancement of auditory performance, using a combined theoretical and experimental approach. The enhancement of sensory performance was studied through a signal detection framework. A model was constructed using principles from signal detection theory and neurophysiology, demonstrating enhancements of roughly 1.8dB both analytically and through simulation. Several experiments were conducted to observe e ects of visual cues on a 2-alternative-forced-choice detection task of an auditory tone in noise. Results of the main experiment showed an enhancement of 1.6dB. Better enhancement also tended to occur for more realistic relationships between audio to visual stimuli.