A expressividade é um elemento chave para o transporte de emoções em música, e seu modelamento, vital para a concepção de sistemas de síntese mais realistas. Gestos musicais executados durante a interpretação usualmente portam a informação responsável pela expressividade percebida, e podem ser rastreados por meio de padrões sônicos a eles associados em diversas escalas de resolução. Um conjunto relevante de gestos musicais expressivos foi estudado através de uma análise em multiresolução utilizando-se a transformada wavelet. A escolha deve-se principalmente à capacidade natural desta ferramenta em realizar análises de tempo-escala/frequência, e suas semelhanças com o processamento dos estágios primários do sistema auditivo. Vinte e sete eventos musicais foram capturados em interpretações de violino e flauta, e analisados com o objetivo de avaliar a aplicabilidade desta ferramenta na identificação e segregação de padrões sônicos associados a gestos musicais expressivos. Os algoritmos wavelet foram implementados na plataforma MATLAB utilizando-se bancos de filtros organizados em esquema piramidal. Rotinas para análises gráfica e sônica e uma interface ao usuário foram também implementadas. Verificou-se que as wavelets permitem a identificação de padrões sônicos associados a gestos expressivos exibindo diferentes propriedades em níveis diferentes da análise. A técnica mostrou-se útil para isolar ruídos oriundos de fontes diversas, extrair transientes associados a gestos súbitos e/ou intensos, e para segregar a estrutura harmônica de tons musicais, entre outras potencialidades não menos importantes. Particularidades da técnica e efeitos secundários observados são discutidos, e os padrões sônicos observados nos níveis wavelets são correlacionados com os gestos musicais que lhes deram origem. São propostos trabalhos futuros objetivando a investigação de certos eventos musicais e fenômenos verificados, bem como o estudo de implementações alternativas. / Expressiveness is a key element for emotion transportation in music, and its modeling necessary to conceive more realistic synthesis systems. Musical gestures executed during a performance carry the information answering for expressiveness, and may be tracked by means of sonic patterns associated to them within several resolution scales. A relevant set of musical gestures was studied through a multiresolution analysis using the wavelet transform. The choice for this tool is mainly due to its natural ability to perform time-scale/frequency analysis, and for its similarities with early auditory processing stages. Twenty seven musical events were captured from violin and flute performances, and analyzed in order to evaluate the applicability of this tool for identification and segregation of sonic patterns associated with expressive musical gestures. The wavelet algorithms were implemented on the MATLAB platform, employing filter banks organized in a pyramidal scheme. Graphical and sonic analysis routines and a user interface were carried out over the same platform. It was verified that wavelets enable the identification of sonic patterns associated to musical gestures revealing different properties on different levels of the analysis. The technique showed up useful to isolate noise from different sources, extract transients associated to sudden and/or intense gestures, and segregate the tonal harmonic structure, among other important features. Particularities of the technique and secondary effects observed are discussed, and sonic patterns on wavelet levels are correlated with the musical gestures which produced them. Future works are proposed addressing further investigation of certain musical events and phenomena observed, as well as the study of alternative implementations.

L'inversion acoustique-articulatoire de la parole consiste à récupérer la forme du conduit vocal à partir d'un signal de parole. Ce problème est abordé à l'aide d'une méthode d'analyse par synthèse reposant sur un modèle physique de production de la parole contrôlé par un petit nombre de paramètres décrivant la forme du conduit vocal : l'ouverture de la mâchoire, la forme et la position de la langue et la position des lèvres et du larynx. Afin de s'approcher de la géométrie de notre locuteur, le modèle articulatoire est construit à l'aide de contours articulatoires issus d'images cinéradiographiques présentant une vue sagittale du conduit vocal. Ce synthétiseur articulatoire nous permet de créer une table formée de couples associant un vecteur articulatoire au vecteur acoustique correspondant. Nous n'utiliserons pas les formants (fréquences de résonance du conduit vocal) comme vecteur acoustique car leur extraction n'est pas toujours fiable provoquant des erreurs lors de l'inversion. Les coefficients cepstraux sont utilisés comme vecteur acoustique. De plus, l'effet de la source et les disparités entre le conduit vocal du locuteur et le modèle articulatoire sont pris en compte explicitement en comparant les spectres naturels à ceux produits par le synthétiseur car nous disposons des deux signaux / The acoustic-to-articulatory inversion of speech consist in the recovery of the vocal tract shape from the speech signal. This problem is tackled with an analysis-by-synthesis method depending on a physical model of speech production controlled by a small number of parameters describing the vocal tract shape: the jaw opening, the shape and the position of the tongue and the position of lips and larynx. In order to approach the geometry of the speaker, the articulatory model is built with articulatory contours from cineradiographic images of the sagittal view of the vocal tract. This articulatory synthesizer allows us to create a table made up with couples associating a articulatory vector with the corresponding acoustic vector. The formants (resonance frequency of the vocal tract shape) are not used as acoustic vector because their extraction is not always reliable causing errors during inversion. The cepstral coefficients are used as acoustic vector. Moreover, the source effect and the mismatch between the speaker vocal tract and the articulatory model are considered explicitly comparing the natural spectrum with those produced by the synthesizer because we have the both signals

<p>This thesis covers topics relevant to model-based coding. Model-based coding is a promising very low bit rate video coding technique. The idea behind this technique is to parameterize a talking head and to extract and transmit the parameters describing facial movements. At the receiver, the parameters are used to reconstruct the talking head. Since only high-level animation parameters are transmitted, very high compression can be achieved with this coding scheme. This thesis covers the following three key problems.</p><p>Although it is a fundamental problem, the initialization problem, has been neglected some extent in the literature. In this thesis, we pay particular attention to the study of this problem. We propose a pseudo-automatic initialization scheme: an Analysis-by-Synthesis scheme based on Simulated Annealing. It has been proved to be an efficient scheme.</p><p>Owing to technical advance today and the newly emerged MPEG-4 standard, new schemes of performing texture mapping and motion estimation are suggested which use sample based direct texture mapping; the feasibility of using active motion estimation is explored which proves to be able to give more than 10 times tracking resolution. Based on the matured face detection technique, Dynamic Programming is introduced to face detection module and work for face tracking.</p><p>Another important problem addressed in this thesis is how to evaluate the face tracking techniques. We studied the evaluation problems by examining the commonly used method, which employs a physical magnetic sensor to provide "ground truth". In this thesis we point out that it is quite misleading to use such a method.</p>

This thesis covers topics relevant to model-based coding. Model-based coding is a promising very low bit rate video coding technique. The idea behind this technique is to parameterize a talking head and to extract and transmit the parameters describing facial movements. At the receiver, the parameters are used to reconstruct the talking head. Since only high-level animation parameters are transmitted, very high compression can be achieved with this coding scheme. This thesis covers the following three key problems. Although it is a fundamental problem, the initialization problem, has been neglected some extent in the literature. In this thesis, we pay particular attention to the study of this problem. We propose a pseudo-automatic initialization scheme: an Analysis-by-Synthesis scheme based on Simulated Annealing. It has been proved to be an efficient scheme. Owing to technical advance today and the newly emerged MPEG-4 standard, new schemes of performing texture mapping and motion estimation are suggested which use sample based direct texture mapping; the feasibility of using active motion estimation is explored which proves to be able to give more than 10 times tracking resolution. Based on the matured face detection technique, Dynamic Programming is introduced to face detection module and work for face tracking. Another important problem addressed in this thesis is how to evaluate the face tracking techniques. We studied the evaluation problems by examining the commonly used method, which employs a physical magnetic sensor to provide "ground truth". In this thesis we point out that it is quite misleading to use such a method.

This thesis presents a new technique for performing image analysis, synthesis, and modification using a generalized Gaussian model. The joint time-frequency characteristics of a generalized Gaussian are combined with the flexibility of the analysis-by-synthesis (ABS) decomposition technique to form the basis of the model. The good localization properties of the Gaussian make it an appealing basis function for image analysis, while the ABS process provides a more flexible representation with enhanced functionality. ABS was first explored in conjunction with sinusoidal modeling of speech and audio signals [George87]. A 2D extension of the ABS technique is developed here to perform the image decomposition. This model forms the basis for new approaches in image analysis and enhancement. The major contribution is made in the resolution enhancement of images generated using coherent imaging modalities such as Synthetic Aperture Radar (SAR) and ultrasound. The ABS generalized Gaussian model is used to decouple natural image features from the speckle and facilitate independent control over feature characteristics and speckle granularity. This has the beneficial effect of increasing the perceived resolution and reducing the obtrusiveness of the speckle while preserving the edges and the definition of the image features. A consequence of its inherent flexibility, the model does not preclude image processing applications for non-coherent image data. This is illustrated by its application as a feature extraction tool for a FLIR imagery complexity measure.

Les plantes sont des éléments essentiels du monde qui nous entoure. Ainsi, si l’on veut créer des environnements virtuels qui soient à la fois agréables et réalistes, un effort doit être fait pour modéliser les plantes. Malgré les immenses progrès en vision par ordinateur pour reconstruire des objets de plus en plus compliqués, les plantes restent difficiles à reconstruire à cause de la complexité de leur topologie. Cette thèse se divise en deux grandes parties. La première partie s’intéresse à la modélisation de plantes, biologiquement réalistes, à partir d’une seule image. Nous générons un modèle de plante respectant les contraintes biologiques de son espèce et tel que sa projection soit la plus fidèle possible à l’image. La première étape consiste à extraire de l’image le squelette de la plante. Dans la plupart de nos images, aucune branche n’est visible et les images peuvent être de qualité moyenne. Notre première contribution consiste en une méthode de squelettisation basée sur les champs de vecteurs. Le squelette est extrait suite à un partitionnement non déterministe du feuillage de l’image assurant son réalisme. Dans un deuxième temps, la plante est modélisée en 3D. Notre deuxième contribution est la création de modèles pour différents types de plantes, basée sur les L-systèmes. Puis, un processus d’analyse-par-synthèse permet de choisir le modèle 3D final : plusieurs propositions de squelette sont générées et un processus bayésien permet d’extraire le modèle maximisant le critère a posteriori. Le terme d’attache aux données (vraisemblance) mesure la similarité entre l’image et la reprojection du modèle, la probabilité a priori mesure le réalisme du modèle. Après avoir généré des modèles de plantes, des modèles de fruits doivent être créés. Ayant travaillé principalement sur les pieds de vigne, nous avons développé une méthode pour reconstruire une grappe de raisin à partir d’au moins deux vues. Chaque baie est assimilée à un ellipsoïde de révolution. La méthode obtenue peut être plus généralement adaptée à tout type de fruits assimilables à une quadrique de révolution. La seconde partie de cette thèse s’intéresse à la reconstruction de quadriques de révolution à partir d’une ou plusieurs vues. La reconstruction de quadriques et, en général, la reconstruction de surfaces 3D est un problème très ancien en vision par ordinateur qui a donné lieu à de nombreux travaux. Nous rappelons les notions nécessaires de géométrie projective des quadriques, et de vision par ordinateur puis, nous présentons un état de l’art sur les méthodes existantes sur la reconstruction de surfaces quadratiques. Nous détaillons un premier algorithme permettant de retrouver les images des foyers principaux d’une quadrique de révolution à partir d’une vue « calibrée », c’est-à-dire pour laquelle les paramètres intrinsèques de la caméra sont connus. Puis, nous détaillons comment utiliser ce résultat pour reconstruire, à partir d’un schéma de triangulation linéaire, tout type de quadriques de révolution à partir d’au moins deux vues. Enfin, nous montrons comment il est possible de retrouver la pose 3D d’une quadrique de révolution dont on connaît les paramètres à partir d’un seul contour occultant. Nous évaluons les performances de nos méthodes et montrons quelques applications possibles. / Plants are essential elements of our world. Thus, 3D plant models are necessary to create realistic virtual environments. Mature computer vision techniques allow the reconstruction of 3D objects from images. However, due to the complexity of the topology of plants, dedicated methods for generating 3D plant models must be devised. This thesis is divided into two parts. The first part focuses on the modeling of biologically realistic plants from a single image. We propose to generate a 3D model of a plant, using an analysis-by-synthesis method considering both a priori information of the plant species and a single image. First, a dedicated 2D skeletonisation algorithm generates possible branching structures from the foliage segmentation. Then, we built a 3D generative model based on a parametric model of branching systems taking into account botanical knowledge. The resulting skeleton follows the hierarchical organisation of natural branching structures. Varying parameter values of the generative model (main branching structure of the plant and foliage), we produce a series of candidate models. A Bayesian model optimizes a posterior criterion which is composed of a likelihood function which measures the similarity between the image and the reprojected 3D model and a prior probability measuring the realism of the model. After modeling plant models branching systems and foliage, we propose to model the fruits. As we mainly worked on vines, we propose a method for reconstructing a vine grape from at least two views. Each bay is considered to be an ellipsoid of revolution. The resulting method can be adapted to any type of fruits with a shape similar to a quadric of revolution. The second part of this thesis focuses on the reconstruction of quadrics of revolution from one or several views. Reconstruction of quadrics, and in general, 3D surface reconstruction is a very classical problem in computer vision. First, we recall the necessary background in projective geometry quadrics and computer vision and present existing methods for the reconstruction of quadrics or more generally quadratic surfaces. A first algorithm identifies the images of the principal foci of a quadric of revolution from a "calibrated" view (that is, the intrinsic parameters of the camera are given). Then we show how to use this result to reconstruct, from a linear triangulation scheme, any type of quadrics of revolution from at least two views. Finally, we show that we can derive the 3D pose of a given quadric of revolution from a single occluding contour. We evaluate the performance of our methods and show some possible applications.

La manipulation dextre est un sujet important dans la recherche en robotique et dans lequel peu de travaux ont abordé la manipulation d'objets déformables. De nouvelles applications en chirurgie, en industrie agroalimentaire ou encore dans les services à la personne nécessitent la maîtrise de la saisie et la manipulation d'objets déformables. Cette thèse s’intéresse à la manipulation d’objets déformables par des préhenseurs mécaniques anthropomorphiques tels que des mains articulées à plusieurs doigts. Cette tâche requière une grande expertise en modélisation mécanique et en commande : modélisation des interactions, perception tactile et par vision, contrôle des mouvements des doigts en position et en force pour assurer la stabilité de la saisie. Les travaux présentés dans cette thèse se focalisent sur la modélisation de la saisie d'objets déformables. Pour cela, nous avons utilisé une discrétisation par des systèmes masses-ressorts non-linéaires pour modéliser des corps déformables en grands déplacements et déformations tout en ayant un coût calculatoire faible. Afin de prédire les forces d’interactions entre main robotique et objet déformable, nous avons proposé une approche originale basée sur un modèle rhéologique visco-élasto-plastique pour évaluer les forces tangentielles de contact et décrire la transition entre les modes d’adhérence et de glissement. Les forces de contact sont évaluées aux points nodaux en fonction des mouvements relatifs entre les bouts des doigts et les facettes du maillage de la surface de l’objet manipulé. Une autre contribution de cette thèse consiste à utiliser de cette modélisation dans la planification des tâches de manipulation d’objets déformables 3D. Cette planification consiste à déterminer la configuration optimale de la main pour la saisie de l’objet ainsi que les trajectoires à suivre et les efforts à appliquer par les doigts pour contrôler la déformation de l’objet tout en assurant la stabilité de l’opération. La validation expérimentale de ces travaux a été réalisée sur deux plateformes robotiques : une main Barrett embarquée sur un bras manipulateur Adept S1700D et une main Shadow embarquée sur un bras manipulateur Kuka LWR4+. / Dexterous manipulation is an important issue in robotics research in which few works have tackled deformable object manipulation. New applications in surgery, food industry or in service robotics require mastering the grasping and manipulation of deformable objects. This thesis focuses on deformable object manipulation by anthropomorphic mechanical graspers such as multi-fingered articulated hands. This task requires a great expertise in mechanical modeling and control: interaction modeling, tactile and vision perception, force / position control of finger movements to ensure stable grasping. The work presented in this thesis focuses on modeling the grasping of deformable objects. To this end, we used a discretization by non-linear mass-spring systems to model deformable bodies in large displacements and deformations while having a low computational cost. To predict the interaction forces between robot hand and deformable object, we proposed an original approach based on a visco-elasto-plastic rheological model to evaluate tangential contact forces and describe the transition between the sticking and slipping modes. The contact forces are evaluated at nodes as function of the relative movements between the fingertips and the surface mesh facets of the manipulated object. Another contribution of this thesis is the use of this model in the planning of 3D deformable object manipulation tasks. This planning consists in determining the optimal configuration of the hand for grasping the objects as well as the paths to track and the efforts to be applied by the fingers to control the deformation of the object while ensuring the stability of the operation. The experimental validation of this work has been carried out on two robotic platforms: a Barrett hand embedded on a Adept S1700D ® manipulator and a Shadow hand embedded on a Kuka LWR4+® manipulator.

<p>Automatic brain tumor segmentation in Magnetic Resonance Imaging scans is essential for the diagnosis, treatment, and surgery of cancerous tumors. However, identifying the hardly detectable tumors poses a considerable challenge, which are usually of different sizes, irregular shapes, and vague invasion areas. Current advancements have not yet fully leveraged the dynamics in the multiple modalities of MRI, since they usually treat multi-modality as multi-channel, and the early channel merging may not fully reveal inter-modal couplings and complementary patterns. In this thesis, we propose a novel deep cross-attention learning algorithm that maximizes the subtle dynamics mining from each of the input modalities and then boosts feature fusion capability. More specifically, we have designed a Multimodal Cross-Attention Module (MM-CAM), equipped with a 3D Multimodal Feature Rectification and Feature Fusion Module. Extensive experiments have shown that the proposed novel deep learning architecture, empowered by the innovative MM- CAM, produces higher-quality segmentation masks of the tumor subregions. Further, we have enhanced the algorithm with image matting refinement techniques. We propose to integrate a Progressive Refinement Module (PRM) and perform Cross-Subregion Refinement (CSR) for the precise identification of tumor boundaries. A Multiscale Dice Loss was also successfully employed to enforce additional supervision for the auxiliary segmentation outputs. This enhancement will facilitate effectively matting-based refinement for medical image segmentation applications. Overall, this thesis, with deep learning, transformer-empowered pattern mining, and sophisticated architecture designs, will greatly advance deep brain dynamics and images mining for tumor detection and precision medicine.</p>

Les carottes, extraites lors des forages de puits de pétrole, font partie des éléments les plus importants dans la chaîne de caractérisation de réservoir. L’acquisition de celles-ci à travers un scanner médical permet d’étudier de façon plus fine les variations des types de dépôts. Le but de cette thèse est d’établir les liens entre les imageries scanners 3D de carottes, et les différentes propriétés pétrophysiques et géologiques. Pour cela la phase de modélisation des images, et plus particulièrement des textures, est très importante et doit fournir des descripteurs extraits qui présentent un assez haut degrés de confiance. Une des solutions envisagée pour la recherche de descripteurs a été l’étude des méthodes paramétriques permettant de valider l’analyse faite sur les textures par un processus de synthèse. Bien que ceci ne représente pas une preuve pour un lien bijectif entre textures et paramètres, cela garantit cependant au moins une confiance en ces éléments. Dans cette thèse nous présentons des méthodes et algorithmes développés pour atteindre les objectifs suivants : 1. Mettre en évidence les zones d’homogénéités sur les zones carottées. Cela se fait de façon automatique à travers de la classification et de l’apprentissage basés sur les paramètres texturaux extraits. 2. Établir les liens existants entre images scanners et les propriétés pétrophysiques de la roche. Ceci se fait par prédiction de propriétés pétrophysiques basées sur l’apprentissage des textures et des calibrations grâce aux données réelles. . / Cores extracted, during wells drilling, are essential data for reservoirs characterization. A medical scanner is used for their acquisition. This feature provide high resolution images improving the capacity of interpretation. The main goal of the thesis is to establish links between these images and petrophysical data. Then parametric texture modelling can be used to achieve this goal and should provide reliable set of descriptors. A possible solution is to focus on parametric methods allowing synthesis. Even though, this method is not a proven mathematically, it provides high confidence on set of descriptors and allows interpretation into synthetic textures. In this thesis methods and algorithms were developed to achieve the following goals : 1. Segment main representative texture zones on cores. This is achieved automatically through learning and classifying textures based on parametric model. 2. Find links between scanner images and petrophysical parameters. This is achieved though calibrating and predicting petrophysical data with images (Supervised Learning Process).

<p dir="ltr">Both diet and physical activity are known to be risk factors for obesity and chronic diseases such as diabetes and metabolic syndrome. We explore a distance-based approach for clustering daily physical activity time series to find temporal physical activity patterns among U.S. adults (ages 20-65). We further extend this approach to integrate both diet and physical activity, and find joint temporal diet and physical activity patterns. Our experiments indicate that the integration of diet, physical activity, and time has the potential to discover joint patterns with association to health. </p><p dir="ltr">Unsupervised domain adaptive (UDA) person re-identification (re-ID) aims to learn identity information from labeled images in source domains and apply it to unlabeled images in a target domain. We propose a deep learning architecture called Synthesis Model Bank (SMB) to deal with illumination variation in unsupervised person re-ID. From our experiments, the proposed SMB outperforms other synthesis methods on several re-ID benchmarks. </p><p dir="ltr">Recent technology advancement introduced modern high-throughput methodologies such as Unmanned Aerial Vehicles (UAVs) to replace the traditional, labor-intensive phenotyping. For many UAV phenotyping analysis, the first step is to extract the smallest groups of plants called “plots” that have the same genotype. We propose an optimization-based, rotation-adaptive approach for extracting plots in a UAV RGB orthomosaic image. From our experiments, the proposed method achieves better plot extraction accuracy compared to existing approaches, and does not require training data.</p>