The human sensory system processes many modalities simultaneously. It was believed that each modality would be processed individually first, and their combination deferred to higher-level cortical areas. Recent neurophysiological investigations indicate interconnections between early visual and auditory cortices, areas putatively considered unimodal, but the function remains unclear. The present work explores how this cross-modality might contribute to a visual enhancement of auditory performance, using a combined theoretical and experimental approach. The enhancement of sensory performance was studied through a signal detection framework. A model was constructed using principles from signal detection theory and neurophysiology, demonstrating enhancements of roughly 1.8dB both analytically and through simulation. Several experiments were conducted to observe e ects of visual cues on a 2-alternative-forced-choice detection task of an auditory tone in noise. Results of the main experiment showed an enhancement of 1.6dB. Better enhancement also tended to occur for more realistic relationships between audio to visual stimuli.

Brain injury is a serious clinical problem. The success of recovery from brain injury involves functional compensation in the affected brain area. We are interested in general mechanisms that underlie compensatory plasticity after brain damage, particularly when multiple brain areas or multiple modalities are included. In this thesis, I studied the function of auditory cortex after recovery from neonatal midbrain damage as a model system that resembles patients with brain damage or sensory dysfunction. I addressed maladaptive changes of auditory cortex after invasion by ectopic visual inputs. I found that auditory cortex contained auditory, visual, and multisensory neurons after it recovered from neonatal midbrain damage (Mao et al. 2011). The distribution of these different neuronal responses did not show any clustering or segregation. As might be predicted from the fact that auditory neurons and visual neurons were intermingled throughout the entire auditory cortex, I found that residual auditory tuning and tonotopy in the rewired auditory cortex were compromised. Auditory tuning curves were broader and tonotopic maps were disrupted in the experimental animals. Because lateral inhibition is proposed to contribute to refinement of sensory maps and tuning of receptive fields, I tested whether loss of inhibition is responsible for the compromised auditory function in my experimental animals. I found an increase rather than a decrease of inhibition in the rewired auditory cortex, suggesting that broader tuning curves in the experimental animals are not caused by loss of lateral inhibition. These results suggest that compensatory plasticity can be maladaptive and thus impair the recovery of the original sensory cortical function. The reorganization of brain areas after recovery from brain damage may require stronger inhibition in order to process multiple sensory modalities simultaneously. These findings provide insight into compensatory plasticity after sensory dysfunction and brain damage and new information about the role of inhibition in cross-modal plasticity. This study can guide further research on design of therapeutic strategies to encourage adaptive changes and discourage maladaptive changes after brain damage, sensory/motor dysfunction, and deafferentation.

The semantic content (i.e., meaning of words) is the essence of retrieved autobiographical memories. In comparison to previous research, which has mainly focused on phenomenological experiences and age distribution of memory events, the present study provides a novel view on the retrieval of event information by addressing the semantic representation of memories. In the present study the semantic representation (i.e., word locations represented by vectors in a high dimensional space) of retrieved memory information were investigated, by analyzing the data with an automatic statistical algorithm. The experiment comprised a cued recall task, where participants were presented with unimodal (i.e., one sense modality) or multimodal (i.e., three sense modalities in conjunction) retrieval cues and asked to recall autobiographical memories. The memories were verbally narrated, recorded and transcribed to text. The semantic content of the memory narrations was analyzed with a semantic representation generated by latent semantic analysis (LSA). The results indicated that the semantic representation of visually evoked memories were most similar to the multimodally evoked memories, followed by auditorily and olfactorily evoked memories. By categorizing the semantic content into clusters, the present study also identified unique characteristics in the memory content across modalities.

Real-time inspection based on machine vision technologies is being widely used in quality control and cost reduction in a variety of application domains. The high demands on the inspection performance and low cost requirements make the algorithm design a challenging task that requires new and innovative methodologies in image processing and fusion. In this research, an integrated approach that combines novel image processing and fusion techniques is proposed for the efficient design of accurate and real-time machine vision-based inspection algorithms with an application to the food processing problem. Firstly, a general methodology is introduced for effective detection of defects and foreign objects that possess certain spectral and shape features. The factors that affect performance metrics are analyzed, and a recursive segmentation and classification scheme is proposed in order to improve the segmentation accuracy. The developed methodology is applied to real-time fan bone detection in deboned poultry meat with a detection rate of 93% and a false alarm rate of 7% from a lab-scale testing on 280 samples. Secondly, a novel snake-based algorithm is developed for the segmentation of vector-valued images. The snakes are driven by the weighted sum of the optimal forces derived from corresponding energy functionals in each image, where the weights are determined based on a novel metric that measures both local contrasts and noise powers in individual sensor images. This algorithm is effective in improving the segmentation accuracy when imagery from multiple sensors is available to the inspection system. The effectiveness of the developed algorithm is verified using (i) synthesized images (ii) real medical and aerial images and (iii) color and x-ray chicken breast images. The results further confirmed that the algorithm yields higher segmentation accuracy than monosensory methods and is able to accommodate a certain amount of registration error. This feature-level image fusion technique can be combined with pixel- and decision- level techniques to improve the overall inspection system performance.

We perceive the world through sensory experience. Sensory information is registered and processed by our brain in a modality specific fashion. Interestingly, studies have shown that the visual cortex of early but not late blind subjects is able to respond to touch or sound (Sadato et al., 1996; Buchel et al., 1998; Weeks et al., 2000; Gougoux et al., 2009). Here, we investigated whether sensory parcellation in adult cortex is innate or is acquired during early postnatal life in an experience-dependent manner. Furthermore, we studied the anatomical substrates and molecular pathways possibly involved in cross-modal activation and its plasticity. First, mice were reared from birth in total darkness until adulthood (DR) to replicate the human blind condition. Cross-modal activity and the underlying circuitry were analyzed. We found that DR visual cortex was strongly activated by sound stimulation using functional imaging, single-unit recording, and c-Fos immunohistochemistry. Functional analysis was followed by anatomical tracing studies, which showed ectopic projections from the auditory thalamus and auditory cortex into the secondary visual area in DR animals. The second half of our study looked at how visual experience affects cross-modal plasticity. We found that cross-modal activity and ectopic connectivity is present in normally reared young mice (25 postnatal days: P25). Normal sensory experience through the first two months of postnatal life was sufficient to decrease the number of ectopic inputs. Interestingly, exposing DR mice to visual experience as adults established transient functional sensory specificity in the visual cortex without eliminating the ectopic anatomical inputs. Lastly, we tested several molecular pathways that can potentially regulate cross-modal plasticity. We found that myelin signaling and cholinergic modulation controls the duration of cross-modal plasticity and consolidates sensory modularization. Overall, our work proposes a model of how cross-modal inputs into early sensory areas are pruned or retained depending on early life experience. This study provides insight into how the cortex develops functional specificity, and help approach disorders that exhibit abnormal sensory integration and disrupted neuronal connectivity such as Autism Spectrum Disorder.

One of every 166 children born today could be diagnosed with Autism Spectrum Disorder (ASD) (CDC 2006). Growing bodies of evidence show sensory integration issues may be at the root of many of the symptoms children with ASD exhibit. Sensory integration is defined as the ability to feel, understand, and organize sensory information from the body and environment. The issues surrounding sensory integration are reflected in both hypersensitive and hyposensitive reactions by children with ASD to the vestibular, proprioception, visual, audio, tactile, and olfactory senses.The goal of this paper is to address the sensory integration issues of children with ASD by creating a sensory garden which would allow them to focus on therapeutic and diagnostic interventions. By using the principles and elements of design, guidelines for this garden focused on producing calming effects for hyper reactive children with ASD and stimulating effects for hypo reactions.

Most events that we encounter in everyday life provide our different senses with correlated information, and audiovisual speech perception is a familiar instance of multisensory integration. Several approaches will be used to further examine the role of cognitive factors on audiovisual speech perception. The main focuses of this thesis will be to examine the influences of cognitive load and selective attention on audiovisual speech perception, as well as the integration of auditory and visual information in talking distractor faces. The influence of cognitive factors on the temporal integration of auditory and visual speech, and gaze behaviour during audiovisual speech will also be addressed. The overall results of the experiments presented here suggest that the integration of auditory and visual speech information is quite robust to various attempts to modulate the integration. Adding a cognitive load task shows minimal disruption of the integration of auditory and visual speech information. Changing attentional instructions to get subjects to selectively attend to either the auditory or visual speech information also has a rather modest influence on the observed integration of auditory and visual speech information. Generally, the integration of temporally offset auditory and visual information seems rather insensitive to cognitive load or selective attentional manipulations. The processing of visual information from distractor faces seems to be limited. The language of the visually articulating distractors doesn't appear to provide information that is helpful for matching together the auditory and visual speech streams. Audiovisual speech distractors are not really any more distracting than auditory distractor speech paired with a still image, suggesting a limited processing or integration of the visual and auditory distractor information. The gaze behaviour during audiovisual speech perception appears to be relatively unaffected by an increase in cognitive load, but is somewhat influenced by attentional instructions to selectively attend to the auditory and visual information. Additionally, both the congruency of the consonant, and the temporal offset of the auditory and visual stimuli have small but rather robust influences on gaze. / Thesis (Ph.D, Psychology) -- Queen's University, 2011-09-30 23:31:07.754

Mémoire numérisé par la Division de la gestion de documents et des archives de l'Université de Montréal

This interior design project involves investigation into the philosophies of the Slow Food Movement and how they may inspire and inform the design of a social space that fosters a connection between local people, food and place. This socio-cultural connection is implemented through the design of a concentrated, local food centre within the urban environment of Winnipeg, Manitoba. Investigation into interior design strategies which foster social interaction, taste education and sensory engagement contribute to the design of a space where the local food culture of southern Manitoba can be experienced in its authenticity. By combining the public spaces of food which contribute to the contemporary streetscape, with those that exist within the interior environment, this local food centre design aims to promote local identity and facilitate multisensory social engagement that sustains relationships, community, and the environment over time.

This interior design project involves investigation into the philosophies of the Slow Food Movement and how they may inspire and inform the design of a social space that fosters a connection between local people, food and place. This socio-cultural connection is implemented through the design of a concentrated, local food centre within the urban environment of Winnipeg, Manitoba. Investigation into interior design strategies which foster social interaction, taste education and sensory engagement contribute to the design of a space where the local food culture of southern Manitoba can be experienced in its authenticity. By combining the public spaces of food which contribute to the contemporary streetscape, with those that exist within the interior environment, this local food centre design aims to promote local identity and facilitate multisensory social engagement that sustains relationships, community, and the environment over time.