L'homme ne perçoit pas l'espace de manière homogène : le cerveau code l'espace proche du corps différemment de l'espace lointain. Cette distinction joue un rôle primordial notre comportement social : l'espace proche du corps, appelé espace péripersonnel (EPP), serait une zone de protection du corps, où la présence d'un individu est perçue comme une menace. L'EPP a été initialement décrit par la psychologie sociale et l'anthropologie, comme un facteur de la communication humaine. L'EPP a été plus tard décrit chez le singe par des études de neurophysiologie comme un espace codé par des neurones multisensoriels. Ces neurones déchargent uniquement en réponse à des évènements sensoriels situés à une distance limitée du corps du singe (qu'ils soient tactiles, visuels ou auditifs). L'ensemble de ces neurones multisensoriels code ainsi l'EPP tout autour du corps. Ce codage exclusif de l'EPP est crucial pour interagir avec le monde extérieur, car c'est dans cet espace que sont réalisées les actions visant à protéger le corps ou visant à atteindre des objets autour de soi. Le codage mutlisensoriel de l'EPP pendant des interactions sociales est à ce jour peu étudié. Dans ce travail de recherche, nous avons réalisé plusieurs études en vu d'identifier des facteurs contribuant à la perméabilité de l'EPP et ses aspects adaptatifs. Une première étude a examiné les frontières latérales de l'EPP chez des individus seuls, en mesurant l'interaction d'une source sonore dynamique s'approchant du corps avec le temps de détection de stimulations tactiles. Cette étude a montré des différences dans la taille de l'EPP entre les deux hémi-espaces, qui seraient liées à la latéralité manuelle. Une seconde étude a exploré les modulations de l'EPP dans des contextes sociaux. Elle a montré que l'EPP est modifié lorsque des individus réalisent une tâche en collaboration. La troisième étude est une recherche méthodologique qui vise à dépasser les limitations des paradigmes comportementaux utilisés actuellement pour mesurer l'EPP. Elle propose de nouvelles pistes pour évaluer comment les stimuli approchant le corps sont intégrés en fonction de leur distance et du contexte multisensoriel dans lequel ils sont traités. L'ensemble de ces travaux montre l'intérêt d'étudier l'intégration multisensorielle autour du corps dans l'espace 3D pour comprendre pleinement l'EPP, et les impacts potentiels de facteurs sociaux sur les processus multisensoriels de bas-niveaux. De plus, ces études soulignent l'importance pour les neurosciences sociales de développer des protocoles expérimentaux réellement sociaux, à plusieurs participants. / The space near the body, called peripersonal space (PPS), was originally studied in social psychology and anthropology as an important factor in interpersonal communication. It was later described by neurophysiological studies in monkeys as a space mapped with multisensory neurons. Those neurons discharge only when events are occurring near the body (be it tactile, visual or audio information), delineating the space that people consider as belonging to them. The human brain also codes events that are near the body differently from those that are farther away. This dedicated brain function is critical to interact satisfactorily with the external world, be it for defending oneself or to reach objects of interest. However, little is known about how this function is impacted by real social interactions. In this work, we have conducted several studies aiming at understanding the factors that contribute to the permeability and adaptive aspects of PPS. A first study examined lateral PPS for individuals in isolation, by measuring reaction time to tactile stimuli when an irrelevant sound is looming towards the body of the individual. It revealed an anisotropy of reaction time across hemispaces, that we could link to handedness. A second study explored the modulations of PPS in social contexts. It was found that minimal social instructions could influence the shape of peripersonal space, with a complex modification of behaviors in collaborative tasks that outreaches the handedness effect. The third study is a methodological investigation attempting to go beyond the limitations of the behavioral methods measuring PPS, and proposing a new direction to assess how stimuli coming towards the body are integrated according to their distance and the multisensory context in which they are processed. Taken together, our work emphasizes the importance of investigating multisensory integration in 3D space around the body to fully capture PPS mechanisms, and the potential impacts of social factors on low-level multisensory processes. Moreover, this research provides evidence that neurocognitive social investigations, in particular on space perception, benefit from going beyond the traditional isolated individual protocols towards actual live social interactive paradigms.

Dans le cadre de la classification de sons,cette thèse construit des représentations du signal qui vérifient des propriétés d’invariance et de variabilité inter-classe. D’abord, nous étudions le scattering temps- fréquence, une représentation qui extrait des modulations spectrotemporelles à différentes échelles. Enclassification de sons urbains et environnementaux, nous obtenons de meilleurs résultats que les réseaux profonds à convolutions et les descripteurs à court terme. Ensuite, nous introduisons le scattering en spirale, une représentation qui combine des transformées en ondelettes selon le temps, selon les log-fréquences, et à travers les octaves. Le scattering en spirale suit la géométrie de la spirale de Shepard, qui fait un tour complet à chaque octave. Nous étudions les sons voisés avec un modèle source-filtre non stationnaire dans lequel la source et le filtre sont transposés au cours du temps, et montrons que le scattering en spirale sépare et linéarise ces transpositions. Le scattering en spirale améliore lesperformances de l’état de l’art en classification d’instruments de musique. Outre la classification de sons, le scattering temps-fréquence et le scattering en spirale peuvent être utilisés comme des descripteurspour la synthèse de textures audio. Contrairement au scattering temporel, le scattering temps-fréquence est capable de capturer la cohérence de motifs spectrotemporels en bioacoustique et en parole, jusqu’à une échelle d’intégration de 500 ms environ. À partir de ce cadre d’analyse-synthèse, une collaboration artscience avec le compositeur Florian Hecker / This dissertation addresses audio classification by designing signal representations which satisfy appropriate invariants while preserving inter-class variability. First, we study time-frequencyscattering, a representation which extract modulations at various scales and rates in a similar way to idealized models of spectrotemporal receptive fields in auditory neuroscience. We report state-of-the-artresults in the classification of urban and environmental sounds, thus outperforming short-term audio descriptors and deep convolutional networks. Secondly, we introduce spiral scattering, a representationwhich combines wavelet convolutions along time, along log-frequency, and across octaves. Spiral scattering follows the geometry of the Shepard pitch spiral, which makes a full turn at every octave. We study voiced sounds with a nonstationary sourcefilter model where both the source and the filter are transposed through time, and show that spiral scattering disentangles and linearizes these transpositions. Furthermore, spiral scattering reaches state-of-the-art results in musical instrument classification ofsolo recordings. Aside from audio classification, time-frequency scattering and spiral scattering can be used as summary statistics for audio texture synthesis. We find that, unlike the previously existing temporal scattering transform, time-frequency scattering is able to capture the coherence ofspectrotemporal patterns, such as those arising in bioacoustics or speech, up to anintegration scale of about 500 ms. Based on this analysis-synthesis framework, an artisticcollaboration with composer Florian Hecker has led to the creation of five computer music

L'objectif de cette thèse est de concevoir des algorithmes qui permettent la détection robuste d’objets et d’événements dans des vidéos en s’appuyant sur une analyse conjointe de données audio et visuelle. Ceci est inspiré par la capacité remarquable des humains à intégrer les caractéristiques auditives et visuelles pour améliorer leur compréhension de scénarios bruités. À cette fin, nous nous appuyons sur deux types d'associations naturelles entre les modalités d'enregistrements audiovisuels (réalisés à l'aide d'un seul microphone et d'une seule caméra), à savoir la corrélation mouvement/audio et la co-occurrence apparence/audio. Dans le premier cas, nous utilisons la séparation de sources audio comme application principale et proposons deux nouvelles méthodes dans le cadre classique de la factorisation par matrices non négatives (NMF). L'idée centrale est d'utiliser la corrélation temporelle entre l'audio et le mouvement pour les objets / actions où le mouvement produisant le son est visible. La première méthode proposée met l'accent sur le couplage flexible entre les représentations audio et de mouvement capturant les variations temporelles, tandis que la seconde repose sur la régression intermodale. Nous avons séparé plusieurs mélanges complexes d'instruments à cordes en leurs sources constituantes en utilisant ces approches.Pour identifier et extraire de nombreux objets couramment rencontrés, nous exploitons la co-occurrence apparence/audio dans de grands ensembles de données. Ce mécanisme d'association complémentaire est particulièrement utile pour les objets où les corrélations basées sur le mouvement ne sont ni visibles ni disponibles. Le problème est traité dans un contexte faiblement supervisé dans lequel nous proposons un framework d’apprentissage de représentation pour la classification robuste des événements audiovisuels, la localisation des objets visuels, la détection des événements audio et la séparation de sources.Nous avons testé de manière approfondie les idées proposées sur des ensembles de données publics. Ces expériences permettent de faire un lien avec des phénomènes intuitifs et multimodaux que les humains utilisent dans leur processus de compréhension de scènes audiovisuelles. / The goal of this thesis is to design algorithms that enable robust detection of objectsand events in videos through joint audio-visual analysis. This is motivated by humans’remarkable ability to meaningfully integrate auditory and visual characteristics forperception in noisy scenarios. To this end, we identify two kinds of natural associationsbetween the modalities in recordings made using a single microphone and camera,namely motion-audio correlation and appearance-audio co-occurrence.For the former, we use audio source separation as the primary application andpropose two novel methods within the popular non-negative matrix factorizationframework. The central idea is to utilize the temporal correlation between audio andmotion for objects/actions where the sound-producing motion is visible. The firstproposed method focuses on soft coupling between audio and motion representationscapturing temporal variations, while the second is based on cross-modal regression.We segregate several challenging audio mixtures of string instruments into theirconstituent sources using these approaches.To identify and extract many commonly encountered objects, we leverageappearance–audio co-occurrence in large datasets. This complementary associationmechanism is particularly useful for objects where motion-based correlations are notvisible or available. The problem is dealt with in a weakly-supervised setting whereinwe design a representation learning framework for robust AV event classification,visual object localization, audio event detection and source separation.We extensively test the proposed ideas on publicly available datasets. The experimentsdemonstrate several intuitive multimodal phenomena that humans utilize on aregular basis for robust scene understanding.

The main goal of this work is to design and to implement a mobile application for drums training. The application must be capable of displaying drum notation of different grooves from various music styles, playing audio examples of those grooves and also analyze and evaluate drumming skills of drummers. The main method for audio processing is discrete wavelet transform. Results show true value rate above 96%.

The aim of this work is to design an architecture of the embedded device that will be used for controlling DAW software in recording studio. First of all, attention is given to a brief summary of the necessary knowledge which is needed to design such kind of device. Af- ter that follows short survey of the existing solutions and description of protocols which can be used for communication with the recording software. Then, subsequent part of the thesis builds upon these foundations and further elaborates the device architecture by me- ans of decomposing it into several modules. In fact, two hardware modules are designed and manufactured, when each of them is conceived on a separate PCB with its own microcon- troller. Then the control firmware has been implemented for each of the modules. At the end of the work an aluminium enclosure, which holds both modules, is designed. The result of this work is a functional prototype of the assembled controller which can be used for the purpose of controlling DAW software.

The thesis deals with modern methods of audio denoising. Reconstruction of the audiosignal is primarly based on utilization of phase information of signals and phase derivatives. Denoising methods also use sparse signal representations. In thesis is described the way of searching sparse coefficients using proximal Condat algorithm and following computation of reconstructed signal using this coefficients. The reconstruction algorithms are implemented in the MATLAB software with toolbox LTFAT included. Results of the reconstruction are compared using objective evaluation method Signal-to-Noise Ratio (SNR) and also by subjective evaluation.

This diploma work deals with music genre recognition using the techniques of Music Information Retrieval. It contains a brief description of the principle of this research area and its subfield called Music Genre Recognition. The following chapter includes selection of the most suitable parameters for describing music genres. This work further characterizes machine learning methods used in this field of research. The next chapter deals with the descriptions of music datasets created for genre classification studies. Subsequently, there is a draft and evaluation of the system for music genre recognition. The last part of this work describes the results of partial parameter analysis, dependence of genre classification accuracy on the amount of parameters and contains a discussion on the causes of classification accurancy for the individual genres.

The first part of this Master’s Thesis deals with the theory of digital signal processing and describes the JUCE library. In this part some basic operations are explained with digital audio signals, such as polarity inversion, delay and linear interpolation of signal samples. The creation of new audio applications using the JUCE library in the C++ programming language is explained too. The next parts of this thesis describe the implemented audio application that allows the user to provide the described basic operations with digital audio signals in real time. For multiple channel audio signals the channels can be processed independently.

This master’s thesis deals with designs and implementation of systems for music cover recognition. The introduction part is devoted to the calculation parameters from audio signal using Music Information Retrieval techniques. Subsequently, various forms of cover versions and musical aspects that cover versions share are defined. The thesis also deals in detail with the creation and distribution of a database of cover versions. Furthermore, the work presents methods and techniques for comparing and processing the calculated parameters. Attention is then paid to the OTI method, CSM calculation and methods dealing with parameter selection. The next part of the thesis is devoted to the design of systems for recognizing cover versions. Then there are compared systems already designed for recognizing cover versions. Furthermore, the thesis describes machine learning techniques and evaluation methods for evaluating the classification with a special emphasis on artificial neural networks. The last part of the thesis deals with the implementation of two systems in MATLAB and Python. These systems are then tested on the created database of cover versions.

In this work is presented technique for embedding digital watermark in digital audio signals. Digital watermark must be imperceptible and should be robust against attacks and other types of distortion. Algorithm is implemented for embedding digital watermark using technique spread-spectrum and psychoacoustic model ISO-MPEG I layer I. Robustness was tested for filtering signal, MP3 compression and resample method.