Constructivism remains one of the most influential views of understanding how children learn science today. Research investigating learning from within this viewpoint has led to the development of a range of theoretical models, most of which aim to explain the underlying processes associated with conceptual change. Such models range in depth and scope with some attributing change to purely cognitive processes while others suggest a role for social factors. Contemporary research has also begun to explore links between the role of practical activity, skills development and language. This study utilises a cross-sectional design in order to investigate the development of children’s ideas and concepts related to two areas of the English National Curriculum for Science: ‘electricity’ and ‘floating and sinking’. A new and innovative multimodal methodology combining practical science activities and traditional / conventional perspectives alongside interview and observational protocols is presented. Multimodal research proposes that knowledge and meaning are transmitted through a range of responses types including language, drawings and gesture. The participants in this study were children aged 7, 11 and 14 years attending four schools in the East Midlands region. Results demonstrate that the children’s ideas could be developed using conceptual challenge tasks. The gestures that the children produced were categorised according to five different forms: referential, representative, expressive, thinking and social, often containing information about their science ideas that was not included in other response types. The results also begin to uncover how meaning is socially constructed and supported. These results form the basis of a critique of methodology intended to re-evaluate and inform debate arising from different models of conceptual change. The potential importance of studying children’s gestures in classroom settings for providing important cues and clues to underlying thoughts that may not be present in verbal or other more conventional responses alone is highlighted.

The video-based computational analyses of human face and gesture signals encompass a myriad of challenging research problems involving computer vision, machine learning and human computer interaction. In this thesis, we focus on the following challenges: a) the classification of hand and body gestures along with the temporal localization of their occurrence in a continuous stream, b) the recognition of facial expressivity levels in people with Parkinson's Disease using multimodal feature representations, c) the prediction of student learning outcomes in intelligent tutoring systems using affect signals, and d) the personalization of machine learning models, which can adapt to subject and group-specific nuances in facial and gestural behavior. Specifically, we first conduct a quantitative comparison of two approaches to the problem of segmenting and classifying gestures on two benchmark gesture datasets: a method that simultaneously segments and classifies gestures versus a cascaded method that performs the tasks sequentially. Second, we introduce a framework that computationally predicts an accurate score for facial expressivity and validate it on a dataset of interview videos of people with Parkinson's disease. Third, based on a unique dataset of videos of students interacting with MathSpring, an intelligent tutoring system, collected by our collaborative research team, we build models to predict learning outcomes from their facial affect signals. Finally, we propose a novel solution to a relatively unexplored area in automatic face and gesture analysis research: personalization of models to individuals and groups. We develop hierarchical Bayesian neural networks to overcome the challenges posed by group or subject-specific variations in face and gesture signals. We successfully validate our formulation on the problems of personalized subject-specific gesture classification, context-specific facial expressivity recognition and student-specific learning outcome prediction. We demonstrate the flexibility of our hierarchical framework by validating the utility of both fully connected and recurrent neural architectures.

Recentemente, tem se percebido um interesse maior da área de computação pela pesquisa em análise de gestos. Parte dessas pesquisas visa dar suporte aos pesquisadores da área de \"estudos dos gestos\", que estuda o uso de partes do corpo para fins comunicativos. Pesquisadores dessa área analisam gestos a partir de transcrições de conversas ou discursos gravados em vídeo. Para a transcrição dos gestos, geralmente realiza-se a sua segmentação em unidades gestuais e fases. O presente trabalho tem por objetivo desenvolver estratégias para segmentação automatizada das unidades gestuais e das fases dos gestos contidos em um vídeo no contexto de contação de histórias, formulando o problema como uma tarefa de classificação supervisionada. As Máquinas de Vetores Suporte foram escolhidas como método de classificação, devido à sua capacidade de generalização e aos bons resultados obtidos para diversos problemas complexos. Máquinas de Vetores Suporte, porém, não consideram os aspectos temporais dos dados, características que são importantes na análise dos gestos. Por esse motivo, este trabalho investiga métodos de representação temporal e variações das Máquinas de Vetores Suporte que consideram raciocínio temporal. Vários experimentos foram executados neste contexto para segmentação de unidades gestuais. Os melhores resultados foram obtidos com Máquinas de Vetores Suporte tradicionais aplicadas a dados janelados. Além disso, três estratégias de classificação multiclasse foram aplicadas ao problema de segmentação das fases dos gestos. Os resultados indicam que um bom desempenho para a segmentação de gestos pode ser obtido ao realizar o treinamento da estratégia com um trecho inicial do vídeo para obter uma segmentação automatizada do restante do vídeo. Assim, os pesquisadores da área de \"estudos dos gestos\" poderiam segmentar manualmente apenas um trecho do vídeo, reduzindo o tempo necessário para realizar a análise dos gestos presentes em gravações longas. / Recently, it has been noted an increasing interest from computer science for research on gesture analysis. Some of these researches aims at supporting researchers from \"gesture studies\", which studies the use of several body parts for communicative purposes. Researchers of \"gesture studies\" analyze gestures from transcriptions of conversations and discourses recorded in video. For gesture transcriptions, gesture unit segmentation and gesture phase segmentation are usually employed. This study aims to develop strategies for automated segmentation of gestural units and phases of gestures contained in a video in the context of storytelling, formulating the problem as a supervised classification task. Support Vector Machines were selected as classification method, because of its ability to generalize and good results obtained for many complex problems. Support Vector Machines, however, do not consider the temporal aspects of data, characteristics that are important for gesture analysis. Therefore, this paper investigates methods of temporal representation and variations of the Support Vector machines that consider temporal reasoning. Several experiments were performed in this context for gesture units segmentation. The best results were obtained with traditional Support Vector Machines applied to windowed data. In addition, three strategies of multiclass classification were applied to the problem of gesture phase segmentation. The results indicate that a good performance for gesture segmentation can be obtained by training the strategy with an initial part of the video to get an automated segmentation of the rest of the video. Thus, researchers in \"gesture studies\" could manually segment only part of the video, reducing the time needed to perform the analysis of gestures contained in long recordings.

Recentemente, tem se percebido um interesse maior da área de computação pela pesquisa em análise de gestos. Parte dessas pesquisas visa dar suporte aos pesquisadores da área de \"estudos dos gestos\", que estuda o uso de partes do corpo para fins comunicativos. Pesquisadores dessa área analisam gestos a partir de transcrições de conversas ou discursos gravados em vídeo. Para a transcrição dos gestos, geralmente realiza-se a sua segmentação em unidades gestuais e fases. O presente trabalho tem por objetivo desenvolver estratégias para segmentação automatizada das unidades gestuais e das fases dos gestos contidos em um vídeo no contexto de contação de histórias, formulando o problema como uma tarefa de classificação supervisionada. As Máquinas de Vetores Suporte foram escolhidas como método de classificação, devido à sua capacidade de generalização e aos bons resultados obtidos para diversos problemas complexos. Máquinas de Vetores Suporte, porém, não consideram os aspectos temporais dos dados, características que são importantes na análise dos gestos. Por esse motivo, este trabalho investiga métodos de representação temporal e variações das Máquinas de Vetores Suporte que consideram raciocínio temporal. Vários experimentos foram executados neste contexto para segmentação de unidades gestuais. Os melhores resultados foram obtidos com Máquinas de Vetores Suporte tradicionais aplicadas a dados janelados. Além disso, três estratégias de classificação multiclasse foram aplicadas ao problema de segmentação das fases dos gestos. Os resultados indicam que um bom desempenho para a segmentação de gestos pode ser obtido ao realizar o treinamento da estratégia com um trecho inicial do vídeo para obter uma segmentação automatizada do restante do vídeo. Assim, os pesquisadores da área de \"estudos dos gestos\" poderiam segmentar manualmente apenas um trecho do vídeo, reduzindo o tempo necessário para realizar a análise dos gestos presentes em gravações longas. / Recently, it has been noted an increasing interest from computer science for research on gesture analysis. Some of these researches aims at supporting researchers from \"gesture studies\", which studies the use of several body parts for communicative purposes. Researchers of \"gesture studies\" analyze gestures from transcriptions of conversations and discourses recorded in video. For gesture transcriptions, gesture unit segmentation and gesture phase segmentation are usually employed. This study aims to develop strategies for automated segmentation of gestural units and phases of gestures contained in a video in the context of storytelling, formulating the problem as a supervised classification task. Support Vector Machines were selected as classification method, because of its ability to generalize and good results obtained for many complex problems. Support Vector Machines, however, do not consider the temporal aspects of data, characteristics that are important for gesture analysis. Therefore, this paper investigates methods of temporal representation and variations of the Support Vector machines that consider temporal reasoning. Several experiments were performed in this context for gesture units segmentation. The best results were obtained with traditional Support Vector Machines applied to windowed data. In addition, three strategies of multiclass classification were applied to the problem of gesture phase segmentation. The results indicate that a good performance for gesture segmentation can be obtained by training the strategy with an initial part of the video to get an automated segmentation of the rest of the video. Thus, researchers in \"gesture studies\" could manually segment only part of the video, reducing the time needed to perform the analysis of gestures contained in long recordings.

Tout énoncé consiste en un message, explicite ou implicite, qui met en relief les objectifs du locuteur et aiguille l’interlocuteur vers une réponse ou une interprétation attendue par le locuteur. Dans le cadre du discours politique, la parole du personnage politique a pour objectif de mettre l’interlocuteur et le spectateur de son côté. Le langage employé est donc très souvent persuasif. Les études montrent que la communication ne passe pas exclusivement par la communication verbale. Elles mettent en évidence la place occupée par la cognition dans toute communication. L’approche formaliste ne tient pas compte de la dimension psychologique de la communication, tandis qu’une approche psychologique ne tient pas compte de la cognition sans placer la communication dans une structure suffisamment élaborée. Dans le cadre théorique que nous avons choisi, le domaine de la cognition s’inscrit dans une approche conceptuelle. Par conséquent, il conviendra d’analyser non seulement la communication verbale mais aussi la communication non-verbale. Dans quelle mesure le message verbal passe-t-il par d’autres voies ? L’analyse procédera à la mise en relief de la dimension paralinguistique tant dans l’espace que dans le temps : geste, regard, posture.La montée au pouvoir du Parti conservateur au Royaume-Uni en 2010 a donné lieu à la première coalition britannique depuis 1945. C’est la participation de son adversaire, les Liberal Democrats, qui a suscité de nombreux débats sur la faisabilité d’une coalition en raison d’un décalage idéologique important. Il sera donc indispensable d’analyser le contexte politique par rapport à la nature du discours. Pour ce faire, il conviendra d’adopter une approche cognitive de l’analyse du discours. Dans quelle mesure les phénomènes linguistiques et paralinguistiques sont-ils volontaires ou involontaires ? Dans quelle mesure existe-t-il une concordance entre les différents phénomènes observés ? L’approche multimodale permettra de montrer le rapport entre sémantique, phonologie, lexique et grammaire afin de mesurer la pertinence du discours. Les théories cognitivistes élaborées par Leonard TALMY et Ronald LANGACKER seront le point de départ de cette approche conceptuelle. Celle-ci permettra d’aboutir à une analyse de tous les éléments linguistiques et paralinguistiques en fonction de la situation d’énonciation : discours, entretien, débat. / Every utterance is composed of an implicit or an explicit message which highlights the speaker’s aims and steers the interlocutor towards a response or an interpretation intended by the speaker. In the field of political discourse, the objective of the politician’s discourse is to make the interlocutor and the spectator take a partisan view. Consequently, the language used is often persuasive. Studies have shown that communication is not exclusively verbal. They reveal the role of cognition in all forms of communication. A formalist approach does not take into account the psychological dimension of communication, while a psychological approach fails to account for cognition by not addressing communication within a sufficiently detailed linguistic context. In the framework that we have chosen, the cognitive domain is located within a conceptual approach. As a result, it is necessary to analyse both verbal and non-verbal communication. To what extent is the spoken message transmitted through other means? Our analysis will highlight the paralinguistic dimension of language both in space and time – gesture, gaze and posture. The electoral success of the Conservative Party in the United Kingdom in 2010 gave rise to the first British coalition government since 1945. The role played by the party’s opponent, the Liberal Democrats, has led to many debates about the feasibility of a coalition due to the ideological divide. It is therefore essential to examine the political context with regard to the type of discourse. In order to achieve this, we have chosen a cognitive approach to discourse analysis. To what extent are the linguistic and paralinguistic phenomena intended or unintended? To what extent is there any coherence between these different observable phenomena? A multimodal approach will help us to identify the link between semantics, lexis and grammar in order to determine the relevance of the discourse. A number of theories of cognitive linguistics and Critical Discourse Analysis form the basis of our conceptual approach, with particular emphasis on the cognitive dimension put forward by Leonard Talmy. This will enable us to analyse all the linguistic and paralinguistic features of discourse, depending on different situations which include speeches, interviews and debates.

Les robots industriels ont évolué fondamentalement ces dernières années pour répondre aux exigences industrielles de machines et mécanismes toujours plus performants. Ceci se traduit aujourd’hui par de nouveaux robots anthropomorphes plus adaptés laissant entrevoir la réalisation de tâches plus complexes comme la découpe d’objets déformables telle que la découpe de viande ou soumis à de fortes sollicitations comme l’usinage. L’étude du comportement des robots anthropomorphes, à structures parallèles ou hybrides montre une anisotropie aussi bien cinématique, que dynamique, impactant la précision attendue. Ces travaux de thèse étudient l’intégration des redondances cinématiques qui permettent de pallier en partie ce problème en positionnant au mieux la tâche à réaliser dans un espace de travail compatible avec les capacités attendues. Ces travaux ont suivi une démarche en trois étapes : la modélisation analytique de cellules robotiques par équivalent sériel basée sur la méthode TCS, la formalisation des contraintes des processus de découpe de viande et d’usinage et une résolution par optimisation multicritère. Une première originalité de ces travaux réside en le développement d’un modèle à 6 degrés de liberté permettant d’analyser les gestes de l’opérateur qui optimise naturellement le comportement de son bras pour garantir la tâche qu’il réalise. La seconde originalité concerne le placement optimisé des redondances structurales (cellules à 9 ddls) où les paramètres de positionnement sont incorporés comme des variables pilotables (cellule à 11 ddls). Ainsi, ces travaux de thèse apportent des contributions à : - la définition de critères adaptés à la réalisation de tâches complexes et sollicitantes pour la gestion des redondances cinématiques ; - l’identification du comportement des structures sous sollicitations par moyen métrologique (Laser tracker) et l’auto-adaptation des trajectoires par l’utilisation d’une commande en effort industrielle ; - l’optimisation du comportement permettant l’amélioration de la qualité de réalisation des différents processus de coupe (découpe de viande et usinage). / Industrial robots have evolved fundamentally in recent years to reach the industrial requirements. We now find more suitable anthropomorphic robots leading to the realization of more complex tasks like deformable objects cutting such as meat cutting or constrained to high stresses as machining. The behavior study of anthropomorphic robots, parallel or hybrid one highlights a kinematic and dynamic anisotropy, which impacts the expected accuracy. This thesis studied the integration of the kinematic redundancy that can partially overcome this problem by well setting the task to achieve it in a space compatible with the expected capacity. This work followed a three-step approach: analytical modeling of robotic cells by serial equivalent based on the TCS method, formalizing the constraints of meat cutting process and machining process and a multicriteria optimization.The first originality of this work focuses on the development of a 6 DoFs model to analyze the operator actions who naturally optimizes his arm behavior to ensure the task it performs. The second originality concerns the optimized placement of structural redundancy (9 DoFs robotic cell) where positioning parameters are incorporated as controllable variables (11 DoFs robotic cell). Thus, the thesis makes contributions to : - the definition of criteria adapted to the realization of complex and under high stress task for the management of the kinematic redundancy; - the structural behavior identification, under stress, by metrology tools (Laser tracker ) and the self- adaptation paths by using an industrial force control; - the behavior optimization to improve the cutting process quality (meat cutting and machining).

Mémoire en recherche-création / Creative dissertation / Cette thèse de maîtrise traite du processus créatif et de la recherche qui y est associée afin de produire deux performances en direct dans le domaine de la musique électroacoustique. À l’aide de ces deux œuvres, mon intention était de concevoir une pratique artistique qui réunit plusieurs modes autour de la gestuelle et du son, influencée par des algorithmes. Une tentative approfondie d’extraire un processus de composition à partir des réactions de la matière en vibration englobe une grande partie de la recherche. Cette recherche découle de ma transition d’une pratique artistique basée sur la représentation (soucieuse des haut-parleurs, manettes et boutons) vers une pratique imprégnée par la performance (soucieuse de la transformation continuelle du son en relation avec les modulations de la matière). Tout au long de cette recherche, j’ai mené un examen approfondi des rythmes au-delà de la pure mesure de leurs expressions musicales, pour considérer les nombreuses notions du rythme qui se dévoilent dans les interactions quotidiennes de l’expérience vécue. Les micro-rythmes perçus par l’oreille comme des textures, les gestes répétitifs perçus par l’œil comme un mouvement linéaire et les rythmes observés lors de circonstances sociales communes, comme la cadence de la conversation sont, parmi les caractéristiques du rythme qui ont suscité mon intérêt. Le tout se situe dans un récit historique éclairé qui étudie l’influence de l’algorithme et de la matière tout au long de la musique et de l’art sonore du XXe siècle. La recherche conceptuelle est enrichie par des expériences exhaustives en composition algorithmique, analyse gestuelle et modélisation gestuelle. Dans chacun de ces domaines, bien que soutenue par des lectures fondamentales en philosophie et en art, une approche primaire de la création s’est faite dans un processus « réfléchir-en-faisant » qui ont généré de nombreuses ex- périences tant avec la matière physique qu’avec la conception d’instruments numériques. Au- delà de la création des performances qui constituent la base des résultats de cette recherche, un vaste ensemble d’outils interopérables d’analyse gestuelle en temps réel, de modélisation, de composition algorithmique et de traitement du son a été développé et publié pour l’environnement Max/MSP. / This master’s thesis concerns the creative process and related research for the production of two live performances in the domain of electroacoustic music. Across the creation of the two works, my intention has been to develop a unified multi-modal gesture, sound, and algorithm influenced performance practise. Encompassing the largest portion of the research is an earnest attempt to derive compositional process from the behavior of vibrating matter. This research is precipitated by my movement from an artistic practice based on representation (concerned with speakers, knobs, and buttons) towards a practice steeped in performance (concerned with the continuous transformation of sound correlated to material modulation). Across this research, an in-depth investigation was conducted into rhythms beyond their purely metric musical manifestations, and into the numerous alternative notions of rhythm which are revealed through daily interactions and lived experience. Rhythmic artifacts of interest have included micro-rhythms perceived by the ear as textures, repetitive gestures perceived by the eye as linear motion, and rhythms observed in ordinary social situations such as the cadence of conversation. This is all situated within an informed historical narrative which considers the influence of the algorithm and material primarily across 20th century music and sound art. The conceptual research is augmented by extensive experiments in algorithmic composition, gesture analysis, and gesture mapping. In each of these areas, though tied to fundamental readings in philosophy and art, a primary approach to creation has been thinking-through-making, which has led to extensive experimentation with both physical materials and digital instrument design. Beyond the performance creations which form the basis of this research output, a large set of interoperable tools for real-time gesture analysis, mapping, algorithmic composition, and sound processing was developed and published for the Max/MSP environment.

La présente thèse a pour but d’approfondir l’étude du geste dans le cadre de l’interaction Homme Machine. Il s’agit de créer de nouveaux paradigmes d’interaction qui offrent à l’utilisateur de plus amples possibilités d’expression basées sur le geste. Un des vecteurs d’expression du geste, très rarement traité en Interaction Homme Machine, qui lui confère sa coloration et son aspect, est ce que les théoriciens et praticiens de la danse appellent « les qualités de mouvement ». Nous mettons à profit des collaborations avec le domaine de la danse pour étudier la notion de qualités de mouvement et l’intégrer à des paradigmes d’interaction gestuelle. Notre travail analyse les apports de l’intégration des qualités de mouvement comme modalité d’interaction, fournit les outils propices à l’élaboration de cette intégration (en termes de méthodes d’analyse, de visualisation et de contrôle gestuel), en développe et évalue certaines techniques d’interaction.Les contributions de la thèse se situent d’abord dans la formalisation de la notion de qualités de mouvement et l’évaluation de son intégration dans un dispositif interactif en termes d’expérience utilisateur. Sur le plan de la visualisation des qualités de mouvement, les travaux menés pendant la thèse ont permis de démontrer que les modèles physiques masses-ressorts offrent de grandes possibilités de simulation de comportements dynamiques et de contrôle en temps réel. Sur le plan de l’analyse, la thèse a permis de développer des approches novatrices de reconnaissance automatique des qualités de mouvement de l’utilisateur. Enfin, à partir des approches d’analyse et de visualisation des qualités de mouvement, la thèse a donné lieu à l’implémentation d’un ensemble de techniques d’interaction. Elle a appliqué et évalué ses techniques dans le contexte de la pédagogie de la danse et de la performance. / The thesis studies gesture in the context of Human-Computer interaction. It aims at creating new interaction paradigms that offer the user further expressive possibilities based on gestures. The theorists and practitioners of the dance call "movement qualities” (MQ), a notion that conveys expressive content describing the way a gesture is performed. This notion has been rarely taken into consideration in the field of HCI. Our work draws on collaborations with the field of dance to explore the notion of movement qualities and to integrate it as interaction modality. 

The contributions of the thesis are in the formalism of the notion of movement qualities and evaluation of its integration as interaction modality in terms of user experience. 

We also provide computational tools for considering MQ in interactive systems in terms of analysis, representation and gesture control methods. On the representational level, our work have demonstrated that physical models based on masses and springs systems offer great opportunities for simulating dynamics related to MQs and for real-time gesture control. On the analysis level, we developed innovative approaches to automatic real time recognition of movement qualities. Finally, we implemented of a set of interaction techniques based on movement qualities that we applied and evaluated in the context of dance pedagogy and performance.