abstract: This study consisted of several related projects on dynamic spatial hearing by both human and robot listeners. The first experiment investigated the maximum number of sound sources that human listeners could localize at the same time. Speech stimuli were presented simultaneously from different loudspeakers at multiple time intervals. The maximum of perceived sound sources was close to four. The second experiment asked whether the amplitude modulation of multiple static sound sources could lead to the perception of auditory motion. On the horizontal and vertical planes, four independent noise sound sources with 60° spacing were amplitude modulated with consecutively larger phase delay. At lower modulation rates, motion could be perceived by human listeners in both cases. The third experiment asked whether several sources at static positions could serve as "acoustic landmarks" to improve the localization of other sources. Four continuous speech sound sources were placed on the horizontal plane with 90° spacing and served as the landmarks. The task was to localize a noise that was played for only three seconds when the listener was passively rotated in a chair in the middle of the loudspeaker array. The human listeners were better able to localize the sound sources with landmarks than without. The other experiments were with the aid of an acoustic manikin in an attempt to fuse binaural recording and motion data to localize sounds sources. A dummy head with recording devices was mounted on top of a rotating chair and motion data was collected. The fourth experiment showed that an Extended Kalman Filter could be used to localize sound sources in a recursive manner. The fifth experiment demonstrated the use of a fitting method for separating multiple sounds sources. / Dissertation/Thesis / Doctoral Dissertation Speech and Hearing Science 2015

Dans cette thèse, nous étudions le problème de la localisation de sources sismo-volcanique, à partir des données enregistrées par des réseaux de capteurs composés de nouveaux sismomètres à trois composantes (3C). Nous nous concentrerons sur le volcan Ubinas, l'un des plus actifs au Pérou. Nous développons une nouvelle approche (MUSIC-3C) basée sur la méthode MUSIC permetant de retourner les 3 paramètres utiles (lenteur, azimut et incidence). Pour valider notre méthodologie, nous analysons des sources synthétiques propagées en tenant compte de la topographie du volcan Ubinas. Dans cette expérience, les données synthétiques ont été générées pour plusieurs sources situées à différentes profondeurs sous le cratère Ubinas. Nous utilisons l'algorithme MUSIC-3C pour les relocaliser. Nous traitons également des données réelles provenant d'une expérience de terrain menée sur le volcan Ubinas (Pérou) en 2009 par les équipes de recherche de l'IRD-France (Institut de Recherche pour le Déveleppment), UCD l'Irlande (projet VOLUME) et l'Institut de Géophysique du Pérou (IGP). Nous utilisons l'algorithme MUSIC-3C pour localiser les événements explosifs (type vulcanien), ce qui nous permet d'identifier et d'analyser les processus physiques de ces événements, à la suite de cette analyse, nous avons trouvé deux sources pour chaque explosion situées à 300 m et 1100 m en dessous du fond du cratère actif. Basé sur les mécanismes éruptifs proposés pour d'autres volcans du même type, nous interprétons la position de ces sources ainsi que les limites du conduit éruptif impliqué dans le processus de fragmentation. / In this thesis, we study the seismo-volcanic source localization using data recorded by new sensor arrays composed of three-component (3C) seismometers deployed on Ubinas stratovolcano (Peru). We develop a new framework (MUSIC-3C) of source localization method based on the well-known MUSIC algorithm. To investigate the performance of the MUSIC-3C method, we use synthetic datasets designed from eight broadband isotropic seismic sources located beneath the crater floor at different depths. The fundamental scheme of the MUSIC-3C method exploits the fact of the cross-spectral matrix of 3C array data, corresponding to the first seismic signal arrivals, provides of useful vector components (slowness, back-azimuth and incidence angle) from the seismic source. Application of the MUSIC-3C method on synthetic datasets shows the recovery of source positions. Real data used in this study was collected during seismic measurements with two seismic antennas deployed at Ubinas volcano in 2009, whose experiment conduced by volcanic teams of IRD-France (l'Institute de Recherche pour le Déveleppment), Geophysics group University College Dublin Ireland and Geophysical Institute of Peru (IGP). We apply the MUSIC-3C algorithm to investigate wave fields associated with the magmatic activity of Ubinas volcano. These analysis evidence a complex mechanism of vulcanian eruptions in which their seismic sources are found at two separated sources located at depths of 300 m and 1100 m beneath the crater floor. This implies the reproduction of similar mechanisms into the conduit. Based on the eruptive mechanisms proposed for other volcanoes of the same type, we interpret the position of this sources as the limits of the conduit portion that was involved in the fragmentation process.

For a robot to succeed at speech recognition, it is advantageous to have a strong and clear signal tointerpret. To facilitate this the robot can steer and aim for the sound source to get a clearer signal, todo this a sound source localization system is required. If the robot turns towards the speaker thisalso gives a more natural feeling when a human interacts with the robot. To determine where thesound source is positioned, an angle relative to the microphone pair is calculated using theinteraural time difference (ITD), which is the difference in time of arrival of the sound between thepair of microphones. To achieve good result the microphone signals needs to be preprocessed andthere are also different algorithms for calculating the time difference which are investigated in thisthesis. The results presented in this work are from tests, with an emphasis on focusing at real-timesystems, involving noisy environment and response time. The results show the complexity of thebalance between computational time and precision. / För att en robot ska lyckas med taleigenkänning, är det fördelaktigt att ha en stark och tydlig signalatt tolka. För att underlätta detta kan roboten styra och rikta in sig mot ljudkällan för att få entydligare signal och för att detta skall vara möjligt krävs ett system för lokalisering av ljudkällan.Om roboten vänder sig mot talaren ger detta även en mer naturlig känsla när en människainteragerar med roboten. För att avgöra var ljudkällan är placerad, beräknas en vinkel i förhållandetill mikrofonparet med hjälp av interaurala tidsskillnaden (ITD), vilket är skillnaden i ankomsttid avljudet mellan mikrofonparet. För att uppnå bra resultat måste mikrofonsignalerna förbehandlas ochdet finns också olika algoritmer för att beräkna tidsskillnaden som undersöks i detta examensarbete.Det resultat som presenteras i detta arbete kommer från tester, med tonvikt på att fokusera pårealtidssystem, som inbegriper bullrig miljö och svarstid. Resultaten visar komplexiteten i balansenmellan beräknings tid och precision.

Human cognition involves dynamic neural activities in distributed brain areas. For studying such neural mechanisms, magnetoencephalography (MEG) and electroencephalography (EEG) are two important techniques, as they non-invasively detect neural activities with a high temporal resolution. Recordings by MEG/EEG sensors can be approximated as a linear transformation of the neural activities in the brain space (i.e., the source space). However, we only have a limited number sensors compared with the many possible locations in the brain space; therefore it is challenging to estimate the source neural activities from the sensor recordings, in that we need to solve the underdetermined inverse problem of the linear transformation. Moreover, estimating source activities is typically an intermediate step, whereas the ultimate goal is to understand what information is coded and how information flows in the brain. This requires further statistical analysis of source activities. For example, to study what information is coded in different brain regions and temporal stages, we often regress neural activities on some external covariates; to study dynamic interactions between brain regions, we often quantify the statistical dependence among the activities in those regions through “connectivity” analysis. Traditionally, these analyses are done in two steps: Step 1, solve the linear problem under some regularization or prior assumptions, (e.g., each source location being independent); Step 2, do the regression or connectivity analysis. However, biases induced in the regularization in Step 1 can not be adapted in Step 2 and thus may yield inaccurate regression or connectivity results. To tackle this issue, we present novel one-step methods of regression or connectivity analysis in the source space, where we explicitly modeled the dependence of source activities on the external covariates (in the regression analysis) or the cross-region dependence (in the connectivity analysis), jointly with the source-to-sensor linear transformation. In simulations, we observed better performance by our models than by commonly used two-step approaches, when our model assumptions are reasonably satisfied. Besides the methodological contribution, we also applied our methods in a real MEG/EEG experiment, studying the spatio-temporal neural dynamics in the visual cortex. The human visual cortex is hypothesized to have a hierarchical organization, where low-level regions extract low-level features such as local edges, and high-level regions extract semantic features such as object categories. However, details about the spatio-temporal dynamics are less understood. Here, using both the two-step and our one-step regression models in the source space, we correlated neural responses to naturalistic scene images with the low-level and high-level features extracted from a well-trained convolutional neural network. Additionally, we also studied the interaction between regions along the hierarchy using the two-step and our one-step connectivity models. The results from the two-step and the one-step methods were generally consistent; however, the one-step methods demonstrated some intriguing advantages in the regression analysis, and slightly different patterns in the connectivity analysis. In the consistent results, we not only observed an early-to-late shift from low-level to high-level features, which support feedforward information flow along the hierarchy, but also some novel evidence indicating non-feedforward information flow (e.g., topdown feedback). These results can help us better understand the neural computation in the visual cortex. Finally, we compared the empirical sensitivity between MEG and EEG in this experiment, in detecting dependence between neural responses and visual features. Our results show that the less costly EEG was able to achieve comparable sensitivity with that in MEG when the number of observations was about twice of that in MEG. These results can help researchers empirically choose between MEG and EEG when planning their experiments with limited budgets.

A recording of the defense presentation for this dissertation is available at: http://hdl.handle.net/10754/625197 / Steady-state elliptic partial differential equations (PDEs) are frequently used to model a diverse range of physical phenomena. The source and boundary data estimation problems for such PDE systems are of prime interest in various engineering disciplines including biomedical engineering, mechanics of materials and earth sciences. Almost all existing solution strategies for such problems can be broadly classified as optimization-based techniques, which are computationally heavy especially when the problems are formulated on higher dimensional space domains. However, in this dissertation, feedback based state estimation algorithms, known as state observers, are developed to solve such steady-state problems using one of the space variables as time-like. In this regard, first, an iterative observer algorithm is developed that sweeps over regular-shaped domains and solves boundary estimation problems for steady-state Laplace equation. It is well-known that source and boundary estimation problems for the elliptic PDEs are highly sensitive to noise in the data. For this, an optimal iterative observer algorithm, which is a robust counterpart of the iterative observer, is presented to tackle the ill-posedness due to noise. The iterative observer algorithm and the optimal iterative algorithm are then used to solve source localization and estimation problems for Poisson equation for noise-free and noisy data cases respectively. Next, a divide and conquer approach is developed for three-dimensional domains with two congruent parallel surfaces to solve the boundary and the source data estimation problems for the steady-state Laplace and Poisson kind of systems respectively. Theoretical results are shown using a functional analysis framework, and consistent numerical simulation results are presented for several test cases using finite difference discretization schemes.

L'électroencéphalographie (EEG) est une technique qui est couramment utilisée pour le diagnostic et le suivi de l'épilepsie. L'objectif de cette thèse consiste à fournir des algorithmes pour l'extraction, la séparation, et la localisation de sources épileptiques à partir de données EEG. D'abord, nous considérons deux étapes de prétraitement. La première étape vise à éliminer les artéfacts musculaires à l'aide de l'analyse en composantes indépendantes (ACI). Dans ce contexte, nous proposons un nouvel algorithme par déflation semi-algébrique qui extrait les sources épileptiques de manière plus efficace que les méthodes conventionnelles, ce que nous démontrons sur données EEG simulées et réelles. La deuxième étape consiste à séparer des sources corrélées. A cette fin, nous étudions des méthodes de décomposition tensorielle déterministe exploitant des données espace-temps-fréquence ou espace-temps-vecteur-d'onde. Nous comparons les deux méthodes de prétraitement à l'aide de simulations pour déterminer dans quels cas l'ACI, la décomposition tensorielle, ou une combinaison des deux approches devraient être utilisées. Ensuite, nous traitons la localisation de sources distribuées. Après avoir présenté et classifié les méthodes de l'état de l'art, nous proposons un algorithme pour la localisation de sources distribuées qui s'appuie sur les résultats du prétraitement tensoriel. L'algorithme est évalué sur données EEG simulées et réelles. En plus, nous apportons quelques améliorations à une méthode de localisation de sources basée sur la parcimonie structurée. Enfin, une étude des performances de diverses méthodes de localisation de sources est conduite sur données EEG simulées. / Electroencephalography (EEG) is a routinely used technique for the diagnosis and management of epilepsy. In this context, the objective of this thesis consists in providing algorithms for the extraction, separation, and localization of epileptic sources from the EEG recordings. In the first part of the thesis, we consider two preprocessing steps applied to raw EEG data. The first step aims at removing muscle artifacts by means of Independent Component Analysis (ICA). In this context, we propose a new semi-algebraic deflation algorithm that extracts the epileptic sources more efficiently than conventional methods as we demonstrate on simulated and real EEG data. The second step consists in separating correlated sources that can be involved in the propagation of epileptic phenomena. To this end, we explore deterministic tensor decomposition methods exploiting space-time-frequency or space-time-wave-vector data. We compare the two preprocessing methods using computer simulations to determine in which cases ICA, tensor decomposition, or a combination of both should be used. The second part of the thesis is devoted to distributed source localization techniques. After providing a survey and a classification of current state-of-the-art methods, we present an algorithm for distributed source localization that builds on the results of the tensor-based preprocessing methods. The algorithm is evaluated on simulated and real EEG data. Furthermore, we propose several improvements of a source imaging method based on structured sparsity. Finally, a comprehensive performance study of various brain source imaging methods is conducted on physiologically plausible, simulated EEG data.

Les technologies nomades (smartphones, tablettes, . . . ) étant actuellement très répandues,nous avons souhaité, dans le cadre de cette thèse, les utiliser comme vecteur pour proposer au grand public des outils de spatialisation sonore. La taille et le nombre de transducteurs utilisés pour la captation et la restitution sonore spatialisée sont à ce jour la limitation principale pour une utilisation nomade. Dans une première étape, la captation d’un opéra pour une restitution sur des tablettes tactiles nous a permis d’évaluer les technologies audio 3D disponibles aujourd’hui. Les résultats de cette évaluation ont révélé que l’utilisation des quatre capteurs du microphone Soundfield donne de bons résultats à condition d’effectuer un décodage binaural adapté pour une restitution sur casque. Selon une approche inspirée des méthodes de localisation de source et le concept de format « objet », un prototype de prise de son 3D léger et compact a été développé. Le dispositif microphonique proposé se compose de trois capsules microphoniques cardioïdes. A partir des signaux microphoniques, un algorithme de post-traitement spatial est capable, d’une part, de déterminer la direction des sources et, d’autre part, d’extraire un signal sonore représentatif de la scène spatiale. Ces deux informations permettent ainsi de caractérisercomplètement la scène sonore 3D en fournissant un encodage spatial offrant le double avantage d’une compression de l’information audio et d’une flexibilité pour le choix du système de reproduction. En effet, la scène sonore ainsi encodée peut être restituée en utilisant un décodage adapté sur n’importe quel type de dispositif.Plusieurs méthodes de localisation et différentes configurations microphoniques (géométrie et directivité) ont été étudiées.Dans une seconde étape, l’algorithme d’extraction de l’information spatiale a été modifié pour prendre en compte les caractéristiques réelles in situ des microphones.Des méthodes pour compléter la chaîne acoustique sont proposées permettant la restitution binaurale ainsi que sur tout autre dispositif de restitution. Elles proposent l’utilisation de capteurs de localisation présents sur les terminaux mobiles afin d’exploiter les capacités qu’ils offrent aujourd’hui. / Mobile technologies (such as smartphones and tablets) are now common devices of the consumer market. In this PhD we want to use those technologies as the way to introduce tools of sound spatialization into the mass market. Today the size and the number of traducers used to pick-up and to render a spatial sound scene are the main factors which limit the portability of those devices. As a first step, a listening test, based on a spatial audio recording of an opera, let us to evaluate the 3D audio technologies available today for headphone rendering. The results of this test show that, using the appropriate binaural decoding, it is possible to achieve a good binaural rendering using only the four sensors of the Soundfield microphone.Then, the steps of the development of a 3D sound pick-up system are described. Several configurations are evaluated and compared. The device, composed of 3 cardioid microphones, was developed following an approach inspired by the sound source localization and by the concept of the "object format encoding". Using the microphone signals and an adapted post-processing it is possible to determine the directions of the sources and to extract a sound signal which is representative of the sound scene. In this way, it is possible to completely describe the sound scene and to compress the audio information.This method offer the advantage of being cross platform compatible. In fact, the sound scene encoded with this method can be rendered over any reproduction system.A second method to extract the spatial information is proposed. It uses the real in situ characteristics of the microphone array to perform the sound scene analysis.Some propositions are made to complement the 3D audio chain allowing to render the result of the sound scene encoding over a binaural system or any king of speaker array using all capabilities of the mobile devices.

In the ocean, light from the surface dissipates quickly leaving sound the only way to see at a distance. Different sediment types on the ocean floor and water properties like salinity, temperature, and ocean depth all change how sound travels across long distances. Hard sediment types, such as sand and bedrock, are highly reflective while softer sediment types, such as mud, are more absorptive and change the received sound upon arrival. Unfortunately, the vast majority of the ocean floor is not mapped and the expenses involved in creating such a map are far too great. Traditional signal processing methods in underwater acoustics attempt to localize sources and estimate seabed properties, but require a priori decisions and fall victim to ill conditioning and non-linear relationships between the unknowns and are computationally expensive. To address these problems, a deep learning method is proposed to distinguish between seabed types while also predicting source parameters such as source-receiver range from simulated training data. In this thesis, several studies are presented that explore the effectiveness of convolutional neural networks to make predictions from two types of sounds that propagated through the ocean: impulsive explosions and ship noise. These studies show that time-series signals and spectrograms contain sufficient information for deep learning, and additional preprocessing for feature extraction is not necessary. Training data considerations, such as randomness in the network weights and inclusion of representative variability are also explored. In all, this study shows that deep learning is a useful tool in underwater acoustics and has significant potential for seabed parameter estimation.

Sound source localization (SSL) is a broad field, with many important application areas. In outdoor environments SSL systems can, among other things, be used to increase citizens’ safety by detecting and locating abnormal sounds such as gunshots or screams. Localization is a complex field, in the case of an outdoor setting, the sound signal is affected by weather conditions, noise, and objects blocking the propagation path. Furthermore, challenges concerning implementing cost-effective algorithms, robustness, accuracy, and balancing trade-offs, still remain. SSL is a field of intense research, and new studies are continuously published. However, to the best of the authors knowledge, there are no recent reviews of state of the art SSL solutions, applicable in an outdoor urban setting. Hence, this study provides a knowledge base concerning current SSL approaches, intended for the aforesaid environment, and to this end a systematic literature review was performed. The review consisted of a total of 43 studies, published between 2017-2021. From the extracted data, a taxonomy of currently seen design principles was developed. Additionally, both the applied measurement techniques and the positioning methods were defined. It can be seen from the result that classical methods such as direction of arrival and time difference of arrival still are the most used principles in research. However, learning-based approaches have seemingly started to attract more attention. Furthermore, a general description of the SSL approaches has been presented. Thus, the knowledge base provided by this study contains both information on what current state of the art techniques are most commonly adopted as well as the basic ideas behind these principles.

京都大学 / 0048 / 新制・課程博士 / 博士(情報学) / 甲第17928号 / 情博第510号 / 新制||情||90(附属図書館) / 30748 / 京都大学大学院情報学研究科知能情報学専攻 / (主査)教授 奥乃 博, 教授 河原 達也, 教授 山本 章博 / 学位規則第4条第1項該当 / Doctor of Informatics / Kyoto University / DFAM