Theories of tactile localization ability are based largely on the study of crossing effects, in which crossing the hands leads to a significant impairment in performance. This work has resulted in a rich literature that establishes tactile localization as inherently multisensory in nature. However, new work suggests that the studies used to date have made incorrect assumptions about the processes underlying performance (Maij et al., 2020) and the perceptual information that is considered (Badde et al., 2019). This thesis proposes the addition of a new parameter to existing theory that allows for these new results to be incorporated into the existing literature—specifically, the influence of cognitive factors on performance. The Introduction provides an overview of the current state of the literature, as well as the novel findings that seem to contradict it. I then propose a framework that highlights the malleability of tactile localization. The empirical work focuses on previously unexplored cognitive influences on tactile localization performance. In Chapter 2 I demonstrate that visual imagery influences performance, and importantly, that individual differences in visual imagery ability influence imagery’s effect on performance. In Chapter 3 I demonstrate that an individual’s attentional set influences performance, and that results previously thought to be due to changes in perceptual signal are likely due to changes in attentional focus. In Chapter 4 I highlight the biases in theory and measurement practice that have limited our understanding of tactile localization more broadly. The General Discussion then provides a detailed discussion about how to incorporate the findings of this thesis with existing literature, which requires a paradigm shift to how we view tactile localization. / Dissertation / Doctor of Philosophy (PhD) / Our ability to localize tactile stimuli is critical to successfully interact with our environment: if we feel something crawling on us, we need to eliminate this unwanted visitor as quickly and accurately as possible. A large body of evidence suggests that tactile localization requires perceptual signals beyond the somatotopic information about where on your skin you feel the tactile stimulus. Just think about how much easier it is to swat at a bug on your arm when you can see it as well as feel it. In this thesis I provide novel empirical evidence that cognitive factors also influence our ability to engage in tactile localization, including visual imagery and attention. I then propose an update to existing theory that can account for the influence of these cognitive factors, alongside the traditional approach to the integration of perceptual signals such as vision.

Attended information is perceived quicker than unattended information. This is known as prior entry. When making judgments on the temporal order of two successive stimuli, performance is influenced based on attention. We were interested in whether this same attentional shift would occur when we adopt a crossed hands posture. Typically when making these tactile temporal order judgments, performance declines when the hands are crossed. This may be due to a greater influence of the external environment in the crossed posture. We investigated this by providing an exogenous visual cue at one or both of the hands prior to making judgments about the temporal order of two successive vibrations. This was completed with the hands crossed and uncrossed. In Experiment 1 responses were to which stimulus occurred first. In Experiment 2 participants responded to which stimulus occurred second. Changing the response requirement did not influence overall performance. In both experiments we observed prior entry that was in the same direction for both crossed and uncrossed postures. The size of the prior entry effect was larger when the hands were crossed. We remap tactile information quickly to external coordinates, however we are less certain of the hand’s location. / Thesis / Master of Science (MSc)

The aim of the experiments reported in this thesis was to investigate the multisensory interactions taking place between vision and audition. The focus is on the modulatory role of the temporal coincidence and semantic congruency of pairs of auditory and visual stimuli. With regards to the temporal coincidence factor, whether, and how, the presentation of a simultaneous sound facilitates visual target perception was tested using the equivalent noise paradigm (Chapter 3) and the backward masking paradigm (Chapter 4). The results demonstrate that crossmodal facilitation can be observed in both visual detection and identification tasks. Importantly, however, the results also reveal that the sound not only had to be presented simultaneously, but also reliably, with the visual target. The suggestion is made that the reliable co-occurrence of the auditory and visual stimuli provides observers with the statistical regularity needed to assume that the visual and auditory stimuli likely originate from the same perceptual event (i.e., that they in some sense 'belong together'). The experiments reported in Chapters 5 through 8 were designed to investigate the role of semantic congruency on audiovisual interactions. The results of the experiments reported in Chapter 5 revealed that the semantic context provided by the soundtrack that a person happens to be listening to can modulate his/her visual conscious perception in the binocular rivalry situation. In Chapters 6-8, the timecourse of audiovisual semantic interactions were investigated using categorization, detection, and identification tasks on visual pictures. The results suggested that when the presentation of the sound leads the presentation of a picture by more than 240 ms, it induces a crossmodal semantic priming effect. In addition, when the presentation of the sound lags a semantically-congruent picture by about 300 ms, it enhances performance, presumably by helping to maintain the visual representation in short-term memory. The results indicate that audiovisual semantic interactions constitute a heterogeneous group of phenomena. A crossmodal type-token binding framework is proposed to account for the parallel processing of the spatiotemporal and semantic interactions of multisensory inputs. The suggestion is that the congruent information in the type and token representation systems would integrate, and they finally bind into a unified multisensory object representation.

En un contexto dado, nuestra percepción de un estímulo (en el presente marco, un producto alimenticio) se ve afectada, en primer lugar, por sus diversas propiedades sensoriales (extrínsecas e intrínsecas). Las características intrínsecas de un alimento (por ejemplo su sabor, olor, color, textura, emisión de sonido al romperse, etc.) obviamente desempañan un papel esencial en su evaluación por parte de los consumidores. Sin embargo, el impacto que puede llegar a producir tanto los envases como la cubertería o vajilla empleada en la evaluación de un alimento, a un nivel tanto sensorial como afectivo, ha sido poco explorado desde un enfoque multisensorial. La presente tesis tiene como finalidad investigar cómo los distintos sentidos interactúan durante el consumo de un alimento o bebida, y más precisamente, cómo la información sensorial que recibimos a través de los elementos externos al alimento influye en nuestra percepción gustativa y hedónica del mismo. Cada estudio comprendido en la tesis explora nuestra percepción multisensorial de los alimentos analizando la interacción entre distintas modalidades principalmente entre la visión, el tacto y el gusto que intervienen en el proceso. Para ello se han empleado diversas técnicas provenientes de disciplinas como la psicofísica, sociología, ciencia sensorial y marketing (por ejemplo, tarea de asociaciones implícitas (IAT), asociación libre de palabras, distintos tipos de cuestionarios, etc.). Los resultados obtenidos se explican en un marco interdisciplinar, combinando conocimientos y teorías de áreas como la psicología perceptiva, la ciencia sensorial, la investigación con consumidores, y el diseño. En conjunto, los resultados demuestran la complejidad de la percepción multisensorial de un alimento (junto con los elementos que lo acompañan durante el consumo), así como la de su medida e interpretación. Esta tesis doctoral se ha elaborado de acuerdo con los requisitos establecidos por la UPV. / Piqueras Fiszman, B. (2012). More than meets the mouth: Assessing the impact of the extrinsic factors on the multisensory perception of food products [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/18104 / Palancia

A key goal for humans and artificial intelligence systems is to develop an accurate and unified picture of the outside world based on the data from any sense(s) that may be available. The availability of multiple senses presents the perceptual system with new opportunities to fulfil this goal, but exploiting these opportunities first requires the solution of two related tasks. The first is how to make the best use of any redundant information from the sensors to produce the most accurate percept of the state of the world. The second is how to interpret the relationship between observations in each modality; for example, the correspondence problem of whether or not they originate from the same source. This thesis investigates these questions using ideal Bayesian observers as the underlying theoretical approach. In particular, the latter correspondence task is treated as a problem of Bayesian model selection or structure inference in Bayesian networks. This approach provides a unified and principled way of representing and understanding the perceptual problems faced by humans and machines and their commonality. In the domain of machine intelligence, we exploit the developed theory for practical benefit, developing a model to represent audio-visual correlations. Unsupervised learning in this model provides automatic calibration and user appearance learning, without human intervention. Inference in the model involves explicit reasoning about the association between latent sources and observations. This provides audio-visual tracking through occlusion with improved accuracy compared to standard techniques. It also provides detection, verification and speech segmentation, ultimately allowing the machine to understand ``who said what, where?'' in multi-party conversations. In the domain of human neuroscience, we show how a variety of recent results in multimodal perception can be understood as the consequence of probabilistic reasoning about the causal structure of multimodal observations. We show this for a localisation task in audio-visual psychophysics, which is very similar to the task solved by our machine learning system. We also use the same theory to understand results from experiments in the completely different paradigm of oddity detection using visual and haptic modalities. These results begin to suggest that the human perceptual system performs -- or at least approximates -- sophisticated probabilistic reasoning about the causal structure of observations under the hood.

Our ability to interact with the environment requires the integration of multisensory information for the construction of spatial representations. The peripersonal space (i.e., the sector of space closely surrounding one's body) and the integrative processes between visual and tactile inputs originating from this sector of space have been at the center of recent years investigations. Neurophysiological studies provided evidence for the presence in the monkey brain of bimodal neurons, which are activated by tactile as well as visual information delivered near to a specific body part (e.g., the hand). Neuropsychological studies on right brain-damaged patients who present extinction and functional neuroimaging findings suggest the presence of similar bimodal systems in the human brain. Studies on the effects of tool-use on visual-tactile interaction revealed similar dynamic properties of the peripersonal space in monkeys and humans. The functional role of the multisensory coding of peripersonal space is, in our hypothesis, that of providing the brain with a sensori-motor interface for body-objects interactions. Thus, not only it could be involved in driving involuntary defensive movements in response to objects approaching the body, but could be also dynamically maintained and updated as a function of manual voluntary actions performed towards objects in the reaching space. We tested the hypothesis of an involvement of peripersonal space in executing both voluntary and defensive actions. To these aims, we joined a well known cross-modal congruency effect between visual and tactile information to a kinematic approach to demonstrate that voluntary grasping actions induce an on-line re-weighting of multisensory interactions in the peripersonal space. We additionally show that this modulation is handcentred. We also used a motor evoked potentials approach to investigate which coordinates system is used to code the peripersonal space during motor preparation if real objects rapidly approach the body. Our findings provide direct evidence for automatic hand-centred coding of visual space and suggest that peripersonal space may also serve to represent rapidly 3 approaching and potentially noxious objects, thus enabling the rapid selection of appropriate motor responses. These results clearly show that peripersonal space is a multisensori-motor interface that might have been selected through evolution for optimising the interactions between the body and the objects in the external world.

Submitted by Luciana Ferreira (lucgeral@gmail.com) on 2017-11-20T09:57:49Z No. of bitstreams: 2 Dissertação - Rosângela Furtado Mac Dowell - 2017.pdf: 8794959 bytes, checksum: abed66149534fa03dfce2c141e637362 (MD5) license_rdf: 0 bytes, checksum: d41d8cd98f00b204e9800998ecf8427e (MD5) / Approved for entry into archive by Luciana Ferreira (lucgeral@gmail.com) on 2017-11-20T09:58:14Z (GMT) No. of bitstreams: 2 Dissertação - Rosângela Furtado Mac Dowell - 2017.pdf: 8794959 bytes, checksum: abed66149534fa03dfce2c141e637362 (MD5) license_rdf: 0 bytes, checksum: d41d8cd98f00b204e9800998ecf8427e (MD5) / Made available in DSpace on 2017-11-20T09:58:14Z (GMT). No. of bitstreams: 2 Dissertação - Rosângela Furtado Mac Dowell - 2017.pdf: 8794959 bytes, checksum: abed66149534fa03dfce2c141e637362 (MD5) license_rdf: 0 bytes, checksum: d41d8cd98f00b204e9800998ecf8427e (MD5) Previous issue date: 2017-09-28 / With the passage from the mechanical era to the digital era, issues related to the architectural space in relation to the symbolic, technological and functional field have allowed the emergence of new skills and capacities to use new technologies that allow the architect to propose different and relevant alternatives to creative process. Powered by the continuous evolution of Information and Communication Technologies (ICTs), architects offer architects unprecedented possibilities and varied resources capable of producing a performance architecture challenging diversity, differentiation, discontinuity and constant dynamic evolution, thus formatting a new symbiosis between design And product. The way it is conceived, developed and built, contemplates new concepts of space, new dynamic and interactive forms that produce new categories of projects. Thus, the architect as facilitator of multisensory experiences modifies and impregnates space with other qualities that only a living and affective being can contain, the emotional meanings. Based on this observation, we try to make a brief reflection in the light of the thought of some theorists, about architecture and its relationship with value issues inserted in the contemporary context, its interaction with the spheres of the senses, with the image and the imaginary; Relating the multisensory perception of the subject that dialogues and interacts with the architectural space. For this, it is sought as a base for the research some works of architecture relevant in the field of performance and inserted in the temporal cut, from the period from the end of the 20th century to the 21st century. / Com a passagem da era mecânica para a era digital, questões pertinentes ao espaço arquitetônico em relação ao campo simbólico, tecnológico e funcional possibilitaram o surgimento de novas competências e capacidades de utilização de novas tecnologias que permitem ao arquiteto a proposição de diversas e relevantes alternativas ao processo criativo. A contínua evolução das Tecnologias da Informação e Comunicação (TICs) coloca à disposição dos arquitetos possibilidades inéditas e variados recursos capazes de produzir uma arquitetura performática, desafiando a diversidade, a diferenciação, a descontinuidade e a constante evolução dinâmica, formatando uma nova simbiose entre projeto e produto. A maneira como o projeto é concebido, desenvolvido e edificado contempla novos conceitos de espaço, novas formas dinâmicas e interativas que produzem novas categorias projetuais. Assim, o arquiteto, enquanto facilitador de experiências multissensoriais, modifica e impregna o espaço com outras qualidades que somente um ser vivo e afetivo pode conter: os significados emocionais. Partindo dessa constatação, esta dissertação reflete, de forma breve e a partir de alguns teóricos, sobre a arquitetura e a sua relação com questões de valor inseridas no contexto contemporâneo, sua interação com as esferas dos sentidos, com a imagem e o imaginário, relacionando-as à percepção multissensorial do sujeito que dialoga e interage com o espaço arquitetônico. Como base da pesquisa, apresentam-se algumas obras de arquitetura relevantes no campo da performance e inseridas no recorte temporal do período compreendido entre o final do século XX e o início do século XXI.

Studien som redovisas i denna D-uppsats undersöker den metaforiska betydelsen av kropp i vinprovningssammanhang. Undersökningen hämtar sitt material från intervjuer med vinkonnässörer, vinlitteratur med undervisande syfte och texter från vinprovarwebsidor. Konceptuell Metaforteori och  Herdenstams modell "Det dubbla greppet" används som metod. Resultatet visar att metaforen kropp bekriver en multisensorisk upplevelse i smakarens mun, liknande fyllighet. / This study examines the metaphorical meaning of body in wine tasting context. The survey draws its material from interviews with experts in wine and taste, literature with educational purposes, and contemporary texts from wine tasting websites. Conceptual Metaphor Theory and Herdenstam's model “The double grip" are used as methods. The results shows that the metaphor body denotes a multisensory experience in the mouth of the taster, similar to fullness.

Die sensorischen und hedonischen Eigenschaften von Essen sind wichtige Einflussfaktoren für die Nahrungsauswahl und –aufnahme. Was macht die Anziehungskraft von Nahrungsreizen aus? Die sensorischen und hedonsichen Eigenschaften von Nahrungsreizen werden mit allen Sinnen, oftmals sogar gleichzeitig, verarbeitet. Nahrungswahrnehmung ist damit ein mutlisensorisches Phänomen. Der Geruch, der Anblick, der Tasteindruck oder Geräusche können bereits vor der Nahrungsaufnahme wahrgenommen werden und Erwartungen hinsichtlich des Geschmacks auslösen. Diese prä-ingestiven Wahrnehmungseindrücke spielen daher auch eine maßgebliche Rolle bei der Entstehung von Verlangen und Gelüsten. Während der Nahrungsaufnahme, beim Kauen und Schlucken, spielen die chemischen Sinne, Schmecken und Riechen, eine besondere Rolle. Der Gesamtsinneseindruck aus den chemischen Sinneskanälen wird auch als Flavor bezeichnet. Wobei angemerkt sein soll, dass auch nicht-chemische Sinne, Sehen, Hören und Tasten, in die Flavordefinition einbezogen werden können. Zweifelsohne stellt die Nahrungsaufnahme ein komplexes Verhalten dar, das perzeptuelle, kognitive und Stoffwechselprozesse gleichermaßen umfasst. Die vorliegende Habilitationsschrift widmet sich der Untersuchung der neurokognitiven Mechanismen der visuellen, gustatorischen und flavour Wahrnehmung von Nahrungsobjekten und umfasst Untersuchungen zur Vulnerabilität der neuronalen Repräsentationen durch kontextuelle Reize. Zusammenfassend schließt die Arbeit mit der Feststellung, dass ein umfassendes Verständnis der psychophysiologischen Mechanismen der sensorischen und hedonischen Verarbeitung von Nahrungsreizen über alle Sinne die perzeptuelle Grundlage für nahrungsbezogenes Urteilen und Entscheiden darstellt. / What characterizes food and makes it so tempting? Sensory and hedonic information about food is conveyed by all senses, activated more or less simultaneously, having led to the notion that food perception is a multisensory experience. The smell, sight, touch or sound of a food can be experienced before ingestion and elicit expectations about the "taste" of that food based on previous encounters. It is, therefore, not surprising that these so-called pre-ingestive sensory experiences play a role in the formation of cravings and the elicitation of appetitive responses. Only during consumption, the chemical senses, smell, taste and oral touch and irritation, are experienced in the oral cavity as food is masticated and swallowed and gives rise to the overall experience commonly referred to as taste. While the term taste is, strictly speaking, incorrect as it does not refer to the gustatory perception, many languages including German lack an appropriate term for the holistic flavor experience arising from the food-induced stimulation of the chemical senses, gustation, olfaction and oral somatosensation in a minimalist interpretation, or, in a broader sense, of all our senses, including hearing and vision. Undoubtedly, feeding behavior is characterized by a complex interplay of perceptual, cognitive and metabolic processes and research on the mechanisms by which these processes regulate food intake behavior is only in its infancy. In this thesis, I present a series of studies aiming to elucidate the cortical representations of the visual, gustatory and flavor components of food objects along with evidence for the vulnerability of these presentations to contextual information. Together, I reckon that an understanding of the psychophysiological mechanisms of the sensory and affective processing of food objects mediated by our senses, seeing, smelling, tasting, feeling and hearing, represents the perceptual basis of food-related decision making.

Herta Müller represents physical suffering and repression in her works, often reflecting on the regime of Nicolae Ceaușescu, and her constant interest in language and reflexivity towards writing have led her to develop sophisticated metaphors that she uses to illuminate language and its functioning under such subjugation. With reference to her fiction and non-fiction, I demonstrate how she uses concrete ideas to understand linguistic phenomena. She evokes injury, destruction, force, life, space, touch, silence, and other bodily experiences to make sense of language in the condition of suffering from social oppression. Drawing on conceptual metaphor theory within the framework of cognitive literary studies, I argue that Müller both relies on and estranges the ways in which people speak and think about language. Language is imagined differently depending on the circumstances and in close relationship with various sensory experiences. The complexity of the relationship between language and thought problematises the process of metaphor building and makes it difficult to identify its key aspects across different contexts and sensory modalities. Müller’s tropes are easy to experience, but difficult to analyse. The idea of language does not exist as a stable concept and is regularly reimagined in her texts; but its meaning is not arbitrary and depends on bodily experience. While Müller evokes such experience to understand language in the condition of suffering, she can also use linguistic concepts to elucidate more abstract ideas. Language can be regarded as an abstract or concrete phenomenon depending on the relevant bodily, linguistic, and cultural contexts. This project contributes to the study of Müller’s poetics as well as to the literary critical interpretation of embodied cognition, and develops the use of conceptual metaphor theory for literary analysis. It also seeks to develop understanding of the role of bodily experience in the metaphorical conceptualisation of language.