Propolis – Creating and using an immersive 2D honey bee environment to educate children between the ages of 3-10, about the importance of honey bees. This capstone project will showcase this researcher's skills in graphic design, spatial sound, architecture, exhibition design, and character illustration. Within this structure, individuals will be able to observe, determine, and make comparisons between a healthy bee colony and one that is fragile and dying. / Master of Fine Arts / "Honey bees have been on the decline. As of 2016, eight species of bees made it on the United States (U.S.) endangered species list. It reported that the decline of pollinators is due to industrial agriculture, parasites, pathogens, and climate change" (Contributors 2020). The loss of habitat and lack of nectar is due to monocultures and bee-killing pesticides. These are significant threats to honey bees and wild pollinators. There is a need for good, clean food providing for a healthy existence for the honey bee. When bees have proper nutrients, the result is a healthy hive that gives mature bees a better chance to fight off diseases. To showcase the life of a honey bee, this researcher created an immersive environment that simulates a hive structure. The structure, along with the motion graphics and critical takeaway printed pieces, is to help children between the ages of 3-10 understand the importance of honey bees. By comparing healthy and unhealthy bees in their natural habitat in an illustration format, the children could empathize with the honey bees. Hopefully, they will one day change the fate of these amazing creatures. The immersive environment will emulate a spatial audio, 2D character illustration, and animation projected onto a honeycomb structure. Custom seed packets and a brochure will accompany the immersive environment to allow the children to plant a bee-friendly garden and to learn more about the honey bees. This capstone project will showcase the researcher's skills in graphic design, spatial sound, architecture, exhibition design, and character illustration. This structure will allow individuals to observe, determine, and make comparisons between a healthy colony and one that is fragile and dying.

[ES] La reproducción de las propiedades espaciales del sonido es una cuestión cada vez más importante en muchas aplicaciones inmersivas emergentes. Ya sea en la reproducción de contenido audiovisual en entornos domésticos o en cines, en sistemas de videoconferencia inmersiva o en sistemas de realidad virtual o aumentada, el sonido espacial es crucial para una sensación de inmersión realista. La audición, más allá de la física del sonido, es un fenómeno perceptual influenciado por procesos cognitivos. El objetivo de esta tesis es contribuir con nuevos métodos y conocimiento a la optimización y simplificación de los sistemas de sonido espacial, desde un enfoque perceptual de la experiencia auditiva. Este trabajo trata en una primera parte algunos aspectos particulares relacionados con la reproducción espacial binaural del sonido, como son la escucha con auriculares y la personalización de la Función de Transferencia Relacionada con la Cabeza (Head Related Transfer Function - HRTF). Se ha realizado un estudio sobre la influencia de los auriculares en la percepción de la impresión espacial y la calidad, con especial atención a los efectos de la ecualización y la consiguiente distorsión no lineal. Con respecto a la individualización de la HRTF se presenta una implementación completa de un sistema de medida de HRTF y se introduce un nuevo método para la medida de HRTF en salas no anecoicas. Además, se han realizado dos experimentos diferentes y complementarios que han dado como resultado dos herramientas que pueden ser utilizadas en procesos de individualización de la HRTF, un modelo paramétrico del módulo de la HRTF y un ajuste por escalado de la Diferencia de Tiempo Interaural (Interaural Time Difference - ITD). En una segunda parte sobre reproducción con altavoces, se han evaluado distintas técnicas como la Síntesis de Campo de Ondas (Wave-Field Synthesis - WFS) o la panoramización por amplitud. Con experimentos perceptuales se han estudiado la capacidad de estos sistemas para producir sensación de distancia y la agudeza espacial con la que podemos percibir las fuentes sonoras si se dividen espectralmente y se reproducen en diferentes posiciones. Las aportaciones de esta investigación pretenden hacer más accesibles estas tecnologías al público en general, dada la demanda de experiencias y dispositivos audiovisuales que proporcionen mayor inmersión. / [CA] La reproducció de les propietats espacials del so és una qüestió cada vegada més important en moltes aplicacions immersives emergents. Ja siga en la reproducció de contingut audiovisual en entorns domèstics o en cines, en sistemes de videoconferència immersius o en sistemes de realitat virtual o augmentada, el so espacial és crucial per a una sensació d'immersió realista. L'audició, més enllà de la física del so, és un fenomen perceptual influenciat per processos cognitius. L'objectiu d'aquesta tesi és contribuir a l'optimització i simplificació dels sistemes de so espacial amb nous mètodes i coneixement, des d'un criteri perceptual de l'experiència auditiva. Aquest treball tracta, en una primera part, alguns aspectes particulars relacionats amb la reproducció espacial binaural del so, com són l'audició amb auriculars i la personalització de la Funció de Transferència Relacionada amb el Cap (Head Related Transfer Function - HRTF). S'ha realitzat un estudi relacionat amb la influència dels auriculars en la percepció de la impressió espacial i la qualitat, dedicant especial atenció als efectes de l'equalització i la consegüent distorsió no lineal. Respecte a la individualització de la HRTF, es presenta una implementació completa d'un sistema de mesura de HRTF i s'inclou un nou mètode per a la mesura de HRTF en sales no anecoiques. A mès, s'han realitzat dos experiments diferents i complementaris que han donat com a resultat dues eines que poden ser utilitzades en processos d'individualització de la HRTF, un model paramètric del mòdul de la HRTF i un ajustament per escala de la Diferencià del Temps Interaural (Interaural Time Difference - ITD). En una segona part relacionada amb la reproducció amb altaveus, s'han avaluat distintes tècniques com la Síntesi de Camp d'Ones (Wave-Field Synthesis - WFS) o la panoramització per amplitud. Amb experiments perceptuals, s'ha estudiat la capacitat d'aquests sistemes per a produir una sensació de distància i l'agudesa espacial amb que podem percebre les fonts sonores, si es divideixen espectralment i es reprodueixen en diferents posicions. Les aportacions d'aquesta investigació volen fer més accessibles aquestes tecnologies al públic en general, degut a la demanda d'experiències i dispositius audiovisuals que proporcionen major immersió. / [EN] The reproduction of the spatial properties of sound is an increasingly important concern in many emerging immersive applications. Whether it is the reproduction of audiovisual content in home environments or in cinemas, immersive video conferencing systems or virtual or augmented reality systems, spatial sound is crucial for a realistic sense of immersion. Hearing, beyond the physics of sound, is a perceptual phenomenon influenced by cognitive processes. The objective of this thesis is to contribute with new methods and knowledge to the optimization and simplification of spatial sound systems, from a perceptual approach to the hearing experience. This dissertation deals in a first part with some particular aspects related to the binaural spatial reproduction of sound, such as listening with headphones and the customization of the Head Related Transfer Function (HRTF). A study has been carried out on the influence of headphones on the perception of spatial impression and quality, with particular attention to the effects of equalization and subsequent non-linear distortion. With regard to the individualization of the HRTF a complete implementation of a HRTF measurement system is presented, and a new method for the measurement of HRTF in non-anechoic conditions is introduced. In addition, two different and complementary experiments have been carried out resulting in two tools that can be used in HRTF individualization processes, a parametric model of the HRTF magnitude and an Interaural Time Difference (ITD) scaling adjustment. In a second part concerning loudspeaker reproduction, different techniques such as Wave-Field Synthesis (WFS) or amplitude panning have been evaluated. With perceptual experiments it has been studied the capacity of these systems to produce a sensation of distance, and the spatial acuity with which we can perceive the sound sources if they are spectrally split and reproduced in different positions. The contributions of this research are intended to make these technologies more accessible to the general public, given the demand for audiovisual experiences and devices with increasing immersion. / Gutiérrez Parera, P. (2020). Optimization and improvements in spatial sound reproduction systems through perceptual considerations [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/142696

Este trabalho apresenta diversas propostas associadas ao uso ótimo de funções de transferência relativas à cabeça (HRTFs) em sistemas de reprodução acústica virtual por fones de ouvido. Estas propostas permitem personalizar a HRTF a indivíduos particulares, tomando como base uma combinação da modelagem estrutural e morfológica de HRTFs. Dentro do contexto da modelagem estrutural, o presente trabalho se concentrou no estudo da contribuição do pinna à HRTF. O pinna é a estrutura anatômica responsável pela percepção de elevação. Assim, o primeiro passo foi extrair um conjunto de funções de transferência relativas ao pinna (PRTFs) das HRTFs de uma base de dados. Para tanto, foram usadas diversas técnicas como análise preditiva linear para rastrear as ressonâncias, janelamento para eliminar a influência do torso, funções de autocorrelação e de atraso de grupo para salientar as antirressonâncias, e outros algoritmos para combinar ressonâncias e antirressonâncias em apenas uma magnitude espectral. Usando essa nova base de dados de PRTFs e parâmetros antropométricos propostos mais outros registrados na base de dados, um espaço vetorial correspondente à antropometria do pinna foi mapeado linearmente em um espaço vetorial correspondente às características espectrais da PRTF, calculando-se assim várias transformações lineares para estimação de novas PRTFs fora da base de dados. A estimação atingiu 66% de reconstrução no grupo de treino. O trabalho está orientado à exploração das características espectrais importantes na percepção de elevação, portanto, está limitado ao plano médio do hemisfério frontal, onde não existem diferenças interaurais significativas nem efeitos difrativos da cabeça. Finalmente é proposto um sistema de testes de localização de fonte sonora para validar o modelo. / This work presents several proposals associated with the optimal use of head-related transfer functions (HRTF) in virtual auditory spaces presented via headphones. These proposals lead to personalization of the HRTF to particular individuals, using a combination of the structural and morphological modeling techniques. In the context of structural modeling, this work focuses on modeling the contribution of the pinna to the HRTF. The pinna is the anatomical structure responsible for vertical sound localization. Thus, the first step was to extract a set of pinna-related transfer functions (PRTFs) from HRTFs published in a database. This was accomplished using several techniques like linear prediction analysis for tracking the resonances, windowing for eliminating the torso influence, autocorrelation and group delay functions for emphasizing the notches and other algorithms for combining resonances and notches in only one magnitude response. Using this novel database of PRTFs and a set of proposed anthropometric parameters plus some others registered in the database, a vector space corresponding to pinna anthropometry is linearly mapped into a vector space corresponding to spectral features of the PRTF, being calculated, in this way, several linear transformations for estimation of new PRTFs, outside the database. The estimation attains 66% of reconstruction in the training group. The work focuses on the exploration of spectral characteristics important for elevation perception, therefore, it is limited to the median plane where there are no meaningful interaural differences nor head diffraction effects. Finally, a system for sound localization tests is proposed in order to validate the model.

Este trabalho apresenta diversas propostas associadas ao uso ótimo de funções de transferência relativas à cabeça (HRTFs) em sistemas de reprodução acústica virtual por fones de ouvido. Estas propostas permitem personalizar a HRTF a indivíduos particulares, tomando como base uma combinação da modelagem estrutural e morfológica de HRTFs. Dentro do contexto da modelagem estrutural, o presente trabalho se concentrou no estudo da contribuição do pinna à HRTF. O pinna é a estrutura anatômica responsável pela percepção de elevação. Assim, o primeiro passo foi extrair um conjunto de funções de transferência relativas ao pinna (PRTFs) das HRTFs de uma base de dados. Para tanto, foram usadas diversas técnicas como análise preditiva linear para rastrear as ressonâncias, janelamento para eliminar a influência do torso, funções de autocorrelação e de atraso de grupo para salientar as antirressonâncias, e outros algoritmos para combinar ressonâncias e antirressonâncias em apenas uma magnitude espectral. Usando essa nova base de dados de PRTFs e parâmetros antropométricos propostos mais outros registrados na base de dados, um espaço vetorial correspondente à antropometria do pinna foi mapeado linearmente em um espaço vetorial correspondente às características espectrais da PRTF, calculando-se assim várias transformações lineares para estimação de novas PRTFs fora da base de dados. A estimação atingiu 66% de reconstrução no grupo de treino. O trabalho está orientado à exploração das características espectrais importantes na percepção de elevação, portanto, está limitado ao plano médio do hemisfério frontal, onde não existem diferenças interaurais significativas nem efeitos difrativos da cabeça. Finalmente é proposto um sistema de testes de localização de fonte sonora para validar o modelo. / This work presents several proposals associated with the optimal use of head-related transfer functions (HRTF) in virtual auditory spaces presented via headphones. These proposals lead to personalization of the HRTF to particular individuals, using a combination of the structural and morphological modeling techniques. In the context of structural modeling, this work focuses on modeling the contribution of the pinna to the HRTF. The pinna is the anatomical structure responsible for vertical sound localization. Thus, the first step was to extract a set of pinna-related transfer functions (PRTFs) from HRTFs published in a database. This was accomplished using several techniques like linear prediction analysis for tracking the resonances, windowing for eliminating the torso influence, autocorrelation and group delay functions for emphasizing the notches and other algorithms for combining resonances and notches in only one magnitude response. Using this novel database of PRTFs and a set of proposed anthropometric parameters plus some others registered in the database, a vector space corresponding to pinna anthropometry is linearly mapped into a vector space corresponding to spectral features of the PRTF, being calculated, in this way, several linear transformations for estimation of new PRTFs, outside the database. The estimation attains 66% of reconstruction in the training group. The work focuses on the exploration of spectral characteristics important for elevation perception, therefore, it is limited to the median plane where there are no meaningful interaural differences nor head diffraction effects. Finally, a system for sound localization tests is proposed in order to validate the model.

La synthèse de champs sonores est un domaine de recherche actif trouvant de nombreuses applications musicales, multimédias ou encore industrielles. Dans ce dernier cas, la reconstruction précise du champ sonore est souhaitée, ce qui implique de répondre à un certains nombre de questionnements scientifiques. À l'aide de réseaux de microphones et de haut-parleurs, la captation, la synthèse et la reconstruction précise de champs sonores sont théoriquement possibles. Seulement, pour des applications pratiques, la disposition des haut-parleurs et l'influence acoustique du lieu de restitution sont des facteurs cruciaux à prendre en compte pour s'assurer de la bonne reconstruction du champ sonore.Dans ce contexte, cette thèse de doctorat propose des méthodes et des techniques pour la captation, la transformation et la reconstruction précise de champs sonores en trois dimensions en se basant sur la méthode ambisonique d'ordre élevé. Une configuration sphérique pour le réseau de microphones et de haut-parleurs est proposée. Elle suit un maillage de Lebedev à cinquante points qui permet la captation et la reconstruction du champ sonore jusqu'à l'ordre 5 avec le formalisme ambisonique. Les limitations de cette approche, tel le repliement spatial, sont étudiés en détails.De plus, une opération de transformation du champ sonore est présentée. Elle est établie dans le domaine des harmoniques sphériques et permet d'effectuer un filtrage directionnel avant le décodage pour privilégier certaines directions dans le champ sonore, suivant une fonction de directivité choisie.Pour la reconstruction, une approche originale, également établie dans le domaine des harmoniques sphériques, permet de prendre en compte l'influence acoustique du lieu de restitution, ainsi que les défauts du système de restitution. Ce traitement permet alors d'adapter la synthèse de champs sonores au lieu de restitution, en conservant le formalisme théorique établi en champ libre.Finalement, une validation expérimentale des méthodes et des techniques développées au cours de la thèse est faite. Dans ce contexte, une suite logicielle de synthèse et traitement en temps-réel des champs sonore est développée. / Sound field synthesis is an active research domain with various musical, multimedia or industrial applications. In the latter case, the accurate reconstruction of the sound field is targeted, which involves answering several scientific questions. Using arrays of microphones and loudspeakers, the capture, synthesis and accurate reconstruction of sound fields are theoretically possible. However, for practical applications, the arrangement of the loudspeakers and the acoustic influence of the restitution room are critical factors to consider in order to ensure the accurate reconstruction of the sound field.In this context, this thesis proposes methods and techniques for the capture, transformations and accurate reconstruction of sound fields in three dimensions based on the Higher Order Ambisonics (HOA) method. A spherical configuration for the array of microphones and loudspeakers is proposed. It follows a fifty-node Lebedev grid that enables the capture and reconstruction of the sound field up to order 5 with HOA formalism. The limitations of this approach, such as the spatial aliasing, are studied in detail.A transformation operation of the sound field is also proposed. The formulation is established in the spherical harmonics domain and enables a directional filtering on the sound field prior to the decoding step.For the reconstruction of the sound field, and original approach, also established in the spherical harmonics domain, can take into account the acoustic influence of the restitution room and the defects of the playback system. This treatment then adapts the synthesis of sound fields to the restitution room, maintaining the theoretical formalism established in free field.Finally, an experimental validation of methods and techniques developed in the thesis is made. In this context, a digital signal processing toolkit is developed. It process in real-time the microphones, ambisonics, and loudspeaker signals for the sound field capture, transformations, and decoding.

Detta kandidatarbete handlar om hur implementation av binauralt ljud kan användas för att öka precisionen för spelares ljudlokalisering i datorspel. I detta arbete skapades en artefakt som testade spelares förmåga att lokalisera ljudkällor på olika vertikala nivåer. Resultatet analyserades och problematiserades med kvantitativa- och kvalitativa undersökningsmetoder. Arbetet kopplas till tidigare forskning inom binauralt ljud och hur det används inom digitala medier. Studien tar även upp ämnen som involverar den mänskliga hörseln och ljudimplementation i datorspel.

Virtual Reality (VR) is gaining more and more popularity as a research tool in the field of Human-Robot Interaction (HRI). To fully deploy the potential of VR and benefit HRI studies, we need to establish the basic understanding of the relationship between the physical, real-world interaction (Live) and VR. This study compared Live and VR HRI with a focus on proxemics, as proxemics preference can reflect comprehensive human intuition, making it suitable to be used to compare Live and VR. To evaluate the influence of different modalities in VR, virtual scenes with different visual familiarity and spatial sound were compared as well. Lab experiments were conducted with a physical Pepper robot and its virtual copy. In both Live and VR, proxemics preferences, the perception of the robot (competence and discomfort) and the feeling of presence were measured and compared. Results suggest that proxemic preferences do not remain consistent in Live and in VR, which could be influenced by the perception of the robot. Therefore, when conducting HRI experiments in VR, the perceptions of the robot need be compared before the experiments. Results also indicate freedom within VR HRI as different VR settings are consistent with each other.

Multichannel acoustic signal processing has undergone major development in recent years due to the increased complexity of current audio processing applications. People want to collaborate through communication with the feeling of being together and sharing the same environment, what is considered as Immersive Audio Schemes. In this phenomenon, several acoustic e ects are involved: 3D spatial sound, room compensation, crosstalk cancelation, sound source localization, among others. However, high computing capacity is required to achieve any of these e ects in a real large-scale system, what represents a considerable limitation for real-time applications. The increase of the computational capacity has been historically linked to the number of transistors in a chip. However, nowadays the improvements in the computational capacity are mainly given by increasing the number of processing units, i.e expanding parallelism in computing. This is the case of the Graphics Processing Units (GPUs), that own now thousands of computing cores. GPUs were traditionally related to graphic or image applications, but new releases in the GPU programming environments, CUDA or OpenCL, allowed that most applications were computationally accelerated in elds beyond graphics. This thesis aims to demonstrate that GPUs are totally valid tools to carry out audio applications that require high computational resources. To this end, di erent applications in the eld of audio processing are studied and performed using GPUs. This manuscript also analyzes and solves possible limitations in each GPU-based implementation both from the acoustic point of view as from the computational point of view. In this document, we have addressed the following problems: Most of audio applications are based on massive ltering. Thus, the rst implementation to undertake is a fundamental operation in the audio processing: the convolution. It has been rst developed as a computational kernel and afterwards used for an application that combines multiples convolutions concurrently: generalized crosstalk cancellation and equalization. The proposed implementation can successfully manage two di erent and common situations: size of bu ers that are much larger than the size of the lters and size of bu ers that are much smaller than the size of the lters. Two spatial audio applications that use the GPU as a co-processor have been developed from the massive multichannel ltering. First application deals with binaural audio. Its main feature is that this application is able to synthesize sound sources in spatial positions that are not included in the database of HRTF and to generate smoothly movements of sound sources. Both features were designed after di erent tests (objective and subjective). The performance regarding number of sound source that could be rendered in real time was assessed on GPUs with di erent GPU architectures. A similar performance is measured in a Wave Field Synthesis system (second spatial audio application) that is composed of 96 loudspeakers. The proposed GPU-based implementation is able to reduce the room e ects during the sound source rendering. A well-known approach for sound source localization in noisy and reverberant environments is also addressed on a multi-GPU system. This is the case of the Steered Response Power with Phase Transform (SRPPHAT) algorithm. Since localization accuracy can be improved by using high-resolution spatial grids and a high number of microphones, accurate acoustic localization systems require high computational power. The solutions implemented in this thesis are evaluated both from localization and from computational performance points of view, taking into account different acoustic environments, and always from a real-time implementation perspective. Finally, This manuscript addresses also massive multichannel ltering when the lters present an In nite Impulse Response (IIR). Two cases are analyzed in this manuscript: 1) IIR lters composed of multiple secondorder sections, and 2) IIR lters that presents an allpass response. Both cases are used to develop and accelerate two di erent applications: 1) to execute multiple Equalizations in a WFS system, and 2) to reduce the dynamic range in an audio signal. / Belloch Rodríguez, JA. (2014). PERFORMANCE IMPROVEMENT OF MULTICHANNEL AUDIO BY GRAPHICS PROCESSING UNITS [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/40651 / Premios Extraordinarios de tesis doctorales