Avanços tecnológicos recentes inauguraram novas possibilidades estéticas devido ao potencial de integração de diversos meios, códigos e linguagens. Consequentemente, as tradicionais fronteiras entre as áreas artísticas dissolveram-se e estímulos simultâneos a vários sentidos foram introduzidos. A inclusão do corpo e seus aspectos cognitivos em interação direta com códigos computacionais é fato irreversível na arte contemporânea e seus aspectos e consequências ainda não foram completamente compreendidos. Este trabalho tem como objetivo investigar, sistematizar e utilizar os conhecimentos sobre sinestesia na arte, destacando o caso das obras artísticas interativas que correlacionam voz e imagem. Em função disto, foram propostas: a) a realização de estudos sobre o fenômeno da sinestesia no contexto da ciência e da arte; b) a busca de uma definição para a voz; c) a pesquisa por referências estéticas que abordaram a questão; e d) a realização de trabalhos experimentais correlacionados ao tema. Os métodos de pesquisa utilizados foram: revisão bibliográfica - a partir de um enfoque multidisciplinar, englobando referências tanto da neurociência quanto de outras áreas -, seleção e análise de trabalhos artísticos interativos contemporâneos que abordaram a questão da sinestesia e da interação entre voz e imagem, e investigação estética a partir da realização de propostas poéticas experimentais. Os principais resultados obtidos foram: a compreensão da trajetória dos estudos sobre sinestesia na ciência e a atualização dos conhecimentos sobre o assunto; apontamentos sobre a contextualização da sinestesia na arte; a definição da voz como corporificada e relacionada a todos os sentidos incluídos na unidade do corpo; e a realização de três trabalhos experimentais, dois coletivos sobre sinestesia e um individual que explora a relação entre voz e visualidade. / Recent technological advances opened new aesthetic possibilities in art, due to its potentiality to integrate several medias, codes and languages. Consequently, the traditional boundaries between artistic areas were dissolved and simultaneous stimuli to various senses were introduced. The occurence of the body and its cognitive aspects in interaction with computer codes is an irreversible fact in contemporary art, and its aspects and consequences have not been fully understood. This work aims to investigate, systematize and use the information about synesthesia in arts, highlighting the case of interactive art works connecting voice and image. Therefore, it was proposed: a) the study of the synesthetic phenomenon and how it is treated in both science and arts; b) the definition of voice; c) the search for aesthetic references in interactive arts addressing this issue; and d) the development of experimental works related to this subject. The adopted methodology consisted of: bibliographical review - with a multidisciplinary approach, encompassing references from neuroscience and other areas - , selection and analysis of contemporary interactive artworks adressesing aspects of synesthesia and the voice-image interaction, and aesthetic research based on experimental artworks. Our main results were: understanding the trajectory of the studies about synesthesia in science and updating the knowledge on the subject; notes on the context of synesthesia in art; the definition of voice as an embodied entity, related to all the senses included in the body unit; and the development of three experimental works of art, two collective works about synesthesia, and an individual one that explores the relationship between voice and visuality.

Orientador: Adenil Alfeu Domingos / Banca: Luciano Guimarães / Banca: Lúcia Helena Ferraz Sant'Agostino / Resumo: Partindo da premissa de que o homem é um ser de percepção e comunicação sinestésicas, procede-se a uma investigação que visa a tangenciar os mais diversos aspectos da sinestesiologia - termo cunhado para, neste trabalho de pesquisa, designar o estudo da sinestesia como um dos princípios de comunicação, de modo abrangente. Os escritos de C. S. Peirce, em diálogo com outras áreas do conhecimento, destacando-se entre elas a comunicação, a biologia e as neurociências, portam-se como as principais bases para a fundamentação teórica das hipóteses e proposições formuladas sobre o fenômeno da sinestesia, como aqui esse tema foi tratado. Mediante um método lógico-hipotético, respaldado na visão pragmática peirceana, tecem-se conjecturas e inferências, que, embora lógicas, não têm o objetivo de postular, mas de oferecer novas considerações sobre a sinestesiologia, de modo a contribuir, cientificamente, para a amplitude investigativa desse tema. O homem, sinesteta em sua protopercepção, submete seus sentidos a um condicionamento cultural, processo que implica em mudança de hábitos e aquisição de novos hábitos. Conforme, porém, a doutrina sinequista, pela regência da lei da continuidade, defende-se aqui a potencialidade da percepção sinestésica, que, embora, em um estado anestésico, permanece vívida sob as espessas camadas da habituação. Entendese, portanto, que a percepção sinestésica original pode ser resgatada, não apenas pela sua latência no continuum da consciência, mas pela concepção de mente como matéria dotada de arbitrariedade e variabilidade... (Resumo completo, clicar acesso eletrônico abaixo) / Abstract: Departing from the standing point that man is a synesthetic perception and communication oriented being, we proceed a investigation that aims to deal with the most diverse aspects of synesthesiology - a term coined in this research paper to designate the study of synesthesia as one of the communication principles, in a comprehensive way. The works of C. S. Peirce, in a dialogue with other fields of knowledge, focusing among them communication, biology and neural-sciences, present themselves as the main basis for the theoretical foundations of the hypotheses and propositions formulated about the synesthesia phenomenon, as this theme has been treated herein. By means of a logical/hypothetical method, supported by the pragmatic peircean view, we have woven conjectures and inferences that, although logical, do not aim to postulate, but rather to offer new considerations about synesthesiology, so as to scientifically contribute to the investigative extent of that theme. The synesthetic man in his proto-perception submits his senses to a cultural conditioning, a process that implies a change of habits and the acquisition of new ones. However, according to the synechist doctrine, by means of the ruling of the continuity law, we defend herein the potentiality of synesthetic perception that, although, in an anesthetic state, remains vivid under the thick layers of habit. It is understood, therefore, that the original synesthetic perception may be rescued, not only because of its latency in the continuum of conscience, but also because of the conception of mind as a matter provided with arbitrariness and variability... (Complete abstract, click electronic address below) / Mestre

Avanços tecnológicos recentes inauguraram novas possibilidades estéticas devido ao potencial de integração de diversos meios, códigos e linguagens. Consequentemente, as tradicionais fronteiras entre as áreas artísticas dissolveram-se e estímulos simultâneos a vários sentidos foram introduzidos. A inclusão do corpo e seus aspectos cognitivos em interação direta com códigos computacionais é fato irreversível na arte contemporânea e seus aspectos e consequências ainda não foram completamente compreendidos. Este trabalho tem como objetivo investigar, sistematizar e utilizar os conhecimentos sobre sinestesia na arte, destacando o caso das obras artísticas interativas que correlacionam voz e imagem. Em função disto, foram propostas: a) a realização de estudos sobre o fenômeno da sinestesia no contexto da ciência e da arte; b) a busca de uma definição para a voz; c) a pesquisa por referências estéticas que abordaram a questão; e d) a realização de trabalhos experimentais correlacionados ao tema. Os métodos de pesquisa utilizados foram: revisão bibliográfica - a partir de um enfoque multidisciplinar, englobando referências tanto da neurociência quanto de outras áreas -, seleção e análise de trabalhos artísticos interativos contemporâneos que abordaram a questão da sinestesia e da interação entre voz e imagem, e investigação estética a partir da realização de propostas poéticas experimentais. Os principais resultados obtidos foram: a compreensão da trajetória dos estudos sobre sinestesia na ciência e a atualização dos conhecimentos sobre o assunto; apontamentos sobre a contextualização da sinestesia na arte; a definição da voz como corporificada e relacionada a todos os sentidos incluídos na unidade do corpo; e a realização de três trabalhos experimentais, dois coletivos sobre sinestesia e um individual que explora a relação entre voz e visualidade. / Recent technological advances opened new aesthetic possibilities in art, due to its potentiality to integrate several medias, codes and languages. Consequently, the traditional boundaries between artistic areas were dissolved and simultaneous stimuli to various senses were introduced. The occurence of the body and its cognitive aspects in interaction with computer codes is an irreversible fact in contemporary art, and its aspects and consequences have not been fully understood. This work aims to investigate, systematize and use the information about synesthesia in arts, highlighting the case of interactive art works connecting voice and image. Therefore, it was proposed: a) the study of the synesthetic phenomenon and how it is treated in both science and arts; b) the definition of voice; c) the search for aesthetic references in interactive arts addressing this issue; and d) the development of experimental works related to this subject. The adopted methodology consisted of: bibliographical review - with a multidisciplinary approach, encompassing references from neuroscience and other areas - , selection and analysis of contemporary interactive artworks adressesing aspects of synesthesia and the voice-image interaction, and aesthetic research based on experimental artworks. Our main results were: understanding the trajectory of the studies about synesthesia in science and updating the knowledge on the subject; notes on the context of synesthesia in art; the definition of voice as an embodied entity, related to all the senses included in the body unit; and the development of three experimental works of art, two collective works about synesthesia, and an individual one that explores the relationship between voice and visuality.

In printed text, sound is a somewhat forgotten aspect. The recitation of text is vital in areas like poetry and oratory, but still has no distinct technique of being communicated. Emphasis of a word might be expressed through italics, but no canonic character set or system for vocal delivery of texts exists. With this essay we are creating a foundation for a development of such a character set or system. By studying existing visualisations of sound and comparing these with semiotic and perception-based theories we obtain useful insights for a prospective system for visualisations of vocal sounds. We conclude that the aspects of sound one wants to visualise must be carefully defined since the viewer cannot process infinite amounts of information. A sound visualisation system does not have to consist of multiple characters or signs to be effective, but had better be built around a strong code to manage the signs into a working system. Creating a context for the signs is also recommended in order to be able to compare signs with eachother.

This thesis aims to construct a semantic system for the disparate uses of peh in TSM and peh¡¦s pragmatic functions in actual language uses in terms of cognitive accounts. Firstly, based on the data collected from dictionaries of the Ministry of Education, the present study divided the various meanings of peh into two main types, i.e., prototypical meaning and extended meanings. Prototypical meaning is further classified into perception-based type. How peh is perceived in perception-based type is in close relationship with the actual environment where we interact with. On the other hand, the main category under extended meanings is conception-based type, which is divided into four subcategories of cognitive mechanisms. That is, the diverse meanings of peh in conception-based type will be interpreted in the perspective of four subcategories of cognitive mechanisms. They are metonymic extension, metaphorical extension, the interaction of metaphor and metonymy, and culturally-related extension. Then, to better understand peh¡¦s pragmatic functions in TSM, the study shifts its foucus to the examination of the interrelationship between peh and the shared Taiwanese cultural background knowledge in actual language uses. Basically, the function which peh serves in discourse depends mainly on the context where it is used, and the world or the community will determine and pick up the most appropriate wordings for us (Mey, 2008). There is no definite answer as to which linguistic expression bears which meaning or interpretation when we try to sort out all possible interpretations and meanings for each linguistic expression containing peh. We need both contextual clues and cultural background knowledge to decipher peh¡¦s underlying meanings. The encodings of color perception do not lie in our biology; instead, it is structured socially (Lucy, 1997). In addition, certain uses of peh, e.g., pe̍h-pau (¥Õ¥]), pe̍h-thiap-á (¥Õ©«¥J), etc., in real life discourse make the function of peh more than a representation of the color itself but bridge the gap and create interpersonal relationship between their language users at the same time. On the whole, this research may shed light on the cognitive understanding of peh in TSM not only semantically but also pragmatically.

Thesis (MTech. degree in Architecture: Professional)--Tshwane University of Technology, 2012. / This thesis presents the proposal of a Synaesthesia Research and Sensory Substitution facility in Pretoria. The facility's primary research will be on the phenomenon of synaesthesia and the effect it can have on people with impaired senses. The envisaged site, Prince's Park, is located on the western edge of the Pretoria CBD and is presently used as an urban park. This thesis will explore the human being's perception of the built environment and how we experience the language of materials to create a rich sensual journey: namely, to move away from a singularly visual architecture in favour of a design approach that experientially accommodates all the senses. In other words, seeing is not believing, feeling is believing; the aim is to create architecture through participation - not just observation.

Synaisthe is an intermedia dance performance that premiered in March 2011 at the University Co-op Presents the Cohen New Works Festival, at the University of Texas at Austin. The context of Synaisthe is two-fold. In one aspect, the work lives in the world of theatrical design, emerging technologies, storytelling practices, and performance theory. In its subject matter, Synaisthe is in conversation with cognitive science, neurology, human perception, and the exploration of individual experience. This event, driven by a technology centered devising process, included; dance, a live band, an infra-red motion tracking floor projection system, manipulation of media through midi-instrument control, a walkman converted into a wearable sonic-fabric costume reading device, and an audience inclusive dance party. The creation of Synaisthe came about as a result of three major aims. A design and technology centered devising process, immersive interaction, and exploration of non-traditional collaborative structures. I endeavor to create a performance as means of researching the influence of media on performance practice. / text

The current program of research tests the following main hypotheses: 1) Synesthetic correspondence is an amodal property that serves to bind intersensory signals and manipulating this correspondence between pairs of audiovisual signals will affect performance on a temporal order judgment (TOJ) task; 2) Manipulating emphasis during a TOJ task from spatial to temporal aspects will strengthen the influence of task-irrelevant auditory signals; 3) The degree of dimensional overlap between audiovisual pairs will moderate the effect of synesthetic correspondence on the TOJ task; and 4) There are gaps in current perceptual theory due to the fact that synesthetic correspondence is a potential confound that has not been sufficiently considered in the design of perception research. The results support these main hypotheses. Finally, potential applications for the findings presented here are discussed.

This thesis is concerned with the cross-modal and synaesthetic perception of musical and visual stimuli. Each of these types of perception has been researched separately, and a hypothesis is presented here that accounts for both cross-modal matching and the development of synaesthesia. This hypothesis claims that sensory information can be evaluated in another modality by using a scale of comparison in that modality. The first set of experiments examines normal subjects performing cross-modal matching with coloured circles and auditory stimuli that vary in complexity. It is shown that subjects use a variety of scales of comparison from both visual and auditory modalities to form matches. As the stimuli increase in complexity, the individual variation in cross-modal matching also increases. The second set of experiments examines matching performance using higher order stimuli, by having subjects evaluate fragments of melodies and complete melodies on affective and descriptive adjective scales. Melodies were also matched with landscape scenes to examine if subjects could form matches between two highly complex sets of stimuli. The final experiments examine synaesthetic associations with colour, evoked from music, letters, numbers, and other categorical information. Common features of synaesthesia from a population of synaesthetes are identified, and experiments performed to test the interference of the synaesthetic associations. Additional experiments are presented that explore the superior short-term memory of one synaesthete, and the role of his associations as a mnemonic device.

Thesis (Ph. D.)--University of Chicago, Dept. of Germanic Studies, June 1999. / Includes bibliographical references. Also available on the Internet.