The present research sought to investigate the correspondence between auditory and visual feature dimensions and to utilise this knowledge in order to inform the design of audio-visual mappings for visual control of sound synthesis. The first stage of the research involved the design and implementation of Morpheme, a novel interface for interaction with corpus-based concatenative synthesis. Morpheme uses sketching as a model for interaction between the user and the computer. The purpose of the system is to facilitate the expression of sound design ideas by describing the qualities of the sound to be synthesised in visual terms, using a set of perceptually meaningful audio-visual feature associations. The second stage of the research involved the preparation of two multidimensional mappings for the association between auditory and visual dimensions. The third stage of this research involved the evaluation of the Audio-Visual (A/V) mappings and of Morpheme's user interface. The evaluation comprised two controlled experiments, an online study and a user study. Our findings suggest that the strength of the perceived correspondence between the A/V associations prevails over the timbre characteristics of the sounds used to render the complementary polar features. Hence, the empirical evidence gathered by previous research is generalizable/ applicable to different contexts and the overall dimensionality of the sound used to render should not have a very significant effect on the comprehensibility and usability of an A/V mapping. However, the findings of the present research also show that there is a non-linear interaction between the harmonicity of the corpus and the perceived correspondence of the audio-visual associations. For example, strongly correlated cross-modal cues such as size-loudness or vertical position-pitch are affected less by the harmonicity of the audio corpus in comparison to weaker correlated dimensions (e.g. texture granularity-sound dissonance). No significant differences were revealed as a result of musical/audio training. The third study consisted of an evaluation of Morpheme's user interface were participants were asked to use the system to design a sound for a given video footage. The usability of the system was found to be satisfactory. An interface for drawing visual queries was developed for high level control of the retrieval and signal processing algorithms of concatenative sound synthesis. This thesis elaborates on previous research findings and proposes two methods for empirically driven validation of audio-visual mappings for sound synthesis. These methods could be applied to a wide range of contexts in order to inform the design of cognitively useful multi-modal interfaces and representation and rendering of multimodal data. Moreover this research contributes to the broader understanding of multimodal perception by gathering empirical evidence about the correspondence between auditory and visual feature dimensions and by investigating which factors affect the perceived congruency between aural and visual structures.

Cette thèse présente l’étude d’une stratégie de coordination hybride d’un groupe de robots sous-marins pour la recherche d’objets de petites dimensions ou de singularités sur les fonds marins. Chaque robot est équipé d’un module de perception utilisant la librairie de traitement d’image OpenCV qui lui permet d’apercevoir les autres éléments de la meute ainsi que l’environnement d’évolution de la mission.Cette stratégie hybride est constituée de deux phases : une phase de mise en formation géométrique et une phase d’acquisition des données vidéo. La première phase s’appuie sur des algorithmes de type "essaims" alors que la seconde se fonde sur une méthode hiérarchique de coordination. En cas de perte de la formation, le groupe de robots quitte le mode hiérarchique et reprend le mode essaim pour se reformer. Ces changements de modes sont contrôlés par une machine à états finis. Avant d’entamer une expérimentation en grandeur nature, la méthodologie et les algorithmes de coordination doivent être testés et validés par simulation.Dans ce contexte, un simulateur basé sur le logiciel Blender a été conçu de façon à ce qu’il tienne compte des différentes contraintes liées à l’évolution des robots dans l’environnement sous-marin. Les résultats de simulation d’une meute de 3 AUVs montrent la capacité de notre stratégie à optimiser l’exécution d’une mission d’acquisition vidéo par un groupe de robots autonomes contrôlés par la vision et coordonnés par une stratégie hybride. / In the underwater environment, the needs of data acquisition have significantly increased over the last decades. As electromagnetic waves show poor propagation in sea water, acoustical sensing is generally preferred. However, the emergence of small and low cost autonomous underwater vehicles (AUV) allow for rethinking the underwater use of optical sensors as their small coverage can be significantly improved by using a fleet of coordinated underwater robots.This paper presents a strategy to coordinate the group of robots in order to systematically survey the seabed to detect small objects or singularities. The proposed hybrid coordination strategy is defined by two main modes. The first mode relies on a swarm algorithm to organize the team in geometrical formation. In the second mode, the robot formation is maintained using a hierarchical coordination. A finite state machine controls the high level hybrid strategy by defining the appropriate coordination mode according to the evolution of the mission. Before sea validation, the behavior and the performance of the hybrid coordination strategy are first assessed in simulation. The control of individual robots relies on visual servoing, implemented with the OpenCV library, and the simulation tool is based on Blender software.The dynamics of the robots has been implemented in a realistic way in Blender by using the Bullet solver and the hydrodynamic coeficcients estimated on the actual robot. First results of the hybrid coordination strategy applied on a fleet of 3 AUV’s, show execution of a video acquisition task by a group of autonomous robots controlled by vision and coordinated by a hybrid strategy.

This thesis describes the design and realization of a mobile robot with autonomous audio-visual control. This robot is able of movement based on sensors consisting of camera and microphone. The mechanical part consists of components made with 3D print technology and omnidirectional Mecanum wheels. Software utilizes OpenCV library for image processing and computes MFCC a DTW for voice command detection.

Title: The impact of the exclusion of the visual analyzer on the swimmer's ability to swim straight while doing the crawl Goals: The goal of this work is to determine the impact of the exclusion of the visual analyzer on one's ability to swim straight while doing the crawl; to find and analyze the angle of the deviation from the set central line, if any, using measurable variables; to find the critical spots in the fulfillment of a set movement assignment where the swimmer deviated from his straight swimming direction, to make a qualitative analysis of these spots in relation to the individually optimal technique, and to propose potential corrections of the time and space of movements of the upper limbs work and the body. Methods: A qualitative analysis with a manual measurement of the deviation from the straight direction of swimming per constant distance. Analysis of a video- recording to find the critical points in the crawl technique. Questionnaire to deal with participants' upper limbs laterality. Results: We managed to determine some causes of the deviation from the straight swimming direction without visual control. The achieved results proved that breathing was an important aspect when doing the crawl having influence on the ability to swim straight: The impact of laterality on the...