Cette recherche propose une analyse multimodale du positionnement interactionnel ou stance-taking. Le corpus de travail, filmé, transcrit et annoté par nos soins dans trois logiciels compatibles (CLAN, PRAAT, ELAN), est une collection de discussions semi-guidées sur le thème de l’environnement (2h 20 min). Les 16 locuteurs sont des étudiants locuteurs natifs d’anglais britannique qui discutent par deux et entre amis. Dans cette recherche, nous adoptons une définition large du « langage », en y incluant l’ensemble des ressources sémiotiques verbales et non-verbales mobilisées pour la co-construction dynamique et intersubjective du sens au cours de l’interaction orale. Nous montrons que les locuteurs intègrent une grande variété de ressources verbales (segments, énoncés), mais aussi vocales (intonation) et mimo-posturo-gestuelles (gestes, expressions du visage), en les synchronisant de manière tant simultanée que séquentielle, pour prendre position vis-à-vis de leur interlocuteur. Au plan théorique, notre approche multi-niveaux et multimodale tisse des liens entre théories françaises de l’énonciation (Benveniste, 1966, Morel et Danon-Boileau, 1998), théorie discursive-fonctionnelle du stance-taking (Kärkkäinen, 2006, Du Bois, 2007), analyse conversationnelle multimodale (C. Goodwin et M.H. Goodwin, 1992, Mondada, 2007), anthropologie linguistique (Ochs, 1996), et étude de la gestualité (Kendon, 2004, Müller, 2004, Streeck, 2009) ; au plan méthodologique, nous combinons analyse qualitative et codage systématique. Notre thèse pose d’abord les bases théoriques et méthodologiques d’une étude multimodale des stances (Partie 1), puis propose la possibilité d’un marquage visuel du positionnement intersubjectif (Partie 2), avant de montrer comment les locuteurs intègrent mots et syntaxe, voix, visage et corps pour prendre position en interaction (Partie 3). / In this research, we propose a multimodal analysis of stance-taking based a collection of semi-guided discussions between pairs of friends who discuss environmental issues (2h 20 min). All 16 speakers are university students who are native speakers of British English. We filmed, transcribed and annotated this video corpus in three compatible software tools, CLAN, PRAAT and ELAN. In this research, we defend a broad understanding of “language”, defined as encompassing all verbal and non-verbal semiotic resources involved in the dynamic and intersubjective co-construction of meaning during spoken interaction. We show that speakers integrate a wide range of verbal resources (segments, utterances) as well as vocal (intonation) and visual ones (gestures, postures and facial expressions), and synchronize these resources simultaneously and sequentially so as to take stances with respect to their interlocutors. On a theoretical level, our multi-level, multimodal approach brings together French utterer-centred approaches to language (Benveniste, 1966, Morel and Danon-Boileau, 1998), discursive-functional theories of stance-taking (Kärkkäinen, 2006, Du Bois, 2007), multimodal conversation analysis (C. Goodwin and M.H. Goodwin, 1992, Mondada, 2007), linguistic anthropology (Ochs, 1996) and gesture studies (Kendon, 2004, Müller, 2004, Streeck, 2009); our methodology combines qualitative analysis with systematic coding. This thesis starts with laying the theoretical and methodological bases for a multimodal study of stance-taking (Part 1); it then proposes that some gestures and facial expressions can be used as intersubjective visual stance markers (Part 2), before showing how speakers integrate words and syntax, voice, facial expressions, gestures and physical posture to take stance in interaction (Part 3).

xv, 256 p. : ill. / This dissertation stages a reciprocal critique between traditional and marginal philosophical approaches to language on the one hand and interdisciplinary studies of speech-accompanying hand gestures on the other. Gesturing with the hands while speaking is a ubiquitous, cross-cultural human practice. Yet this practice is complex, varied, conventional, nonconventional, and above all under-theorized. In light of the theoretical and empirical treatments of language and gesture that I engage in, I argue that the hand gestures that spontaneously accompany speech are a part of language; more specifically, they are enactments of linguistic meaning. They are simultaneously (acts of) cognition and communication. Human communication and cognition are what they are in part because of this practice of gesturing. This argument has profound implications for philosophy, for gesture studies, and for interdisciplinary work to come. As further, strong proof of the pervasively embodied way that humans make meaning in language, reflection on gestural phenomena calls for a complete re-orientation in traditional analytic philosophy of language. Yet philosophical awareness of intersubjectivity and normativity as conditions of meaning achievement is well-deployed in elaborating and refining the minimal theoretical apparatus of present-day gesture studies. Triangulating between the most social, communicative philosophies of meaning and the most nuanced, reflective treatments of co-speech hand gesture, I articulate a new construal of language as embodied, world-embedded, intersubjectively normative, dynamic, multi-modal enacting of appropriative disclosure. Spontaneous co-speech gestures, while being indeed spontaneous, are nonetheless informed in various ways by conventions that they appropriate and deploy. Through this appropriation and deployment speakers enact, rather than represent, meaning, and they do so in various linguistic modalities. Seen thusly, gestures provide philosophers with a unique new perspective on the paradoxical determined-yet-free nature of all human meaning. / Committee in charge: Mark Johnson, Chairperson; Ted Toadvine, Member; Naomi Zack, Member; Eric Pederson, Outside Member

A investigação da multimodalidade nas interações tem sido contemplada em campos de pesquisa cada vez mais diversos. No entanto, ainda é incipiente o entendimento de como os significados são construídos pela mobilização de múltiplas dimensões semióticas, atuando concomitantemente dentro do evento interacional. Esta tese busca contribuir para expandir esse entendimento, sobretudo, em episódios de desacordo. A Análise da Conversa, de base etnometodológica, e os Estudos dos Gestos são as linhas teóricas que balizam esta tese. Pelas contribuições da Análise da Conversa, pudemos observar como os próprios participantes se orientam uns em relação aos outros, revelando-nos os recursos mobilizados como práticas interacionais voltadas à realização e coordenação de suas ações durante a interação. Das contribuições dos Estudos dos Gestos, pudemos explorar as relações entre língua e ações gestuais e verificar o importante papel da gestualidade tanto para o estabelecimento de práticas interacionais quanto para o enquadramento das ações que estão em processo de formação. Os dados desta pesquisa compreendem vários tipos de interação realizados por falantes nativos de português brasileiro, perfazendo aproximadamente 15 horas de registros audiovisuais, coletados entre julho de 2012 e agosto de 2015. Desse material, selecionamos excertos para exemplificação teórica e 6 excertos para análise minuciosa, realizada dentro do programa ELAN, das sequências de desacordo. As análises mostram que os participantes se valem das ações gestuais como um recurso dinâmico e flexível que eles utilizam situadamente; que as ações faciais auxiliam os participantes a demonstrar suas posições e entendimentos a respeito das ações uns dos outros; que as ações gestuais dos coparticipantes produzidas em possíveis pontos de completude do turno do falante parecem ser tomadas como as primeiras manifestações das discordâncias que estão em processo de desenvolvimento; que o falante parece considerar essas ações do coparticipante para produzir seus enunciados subsequentes; que tal prática também parece servir ao propósito de oferecer ao falante uma oportunidade para promover ajustes interacionais antes que os desacordos verbais do coparticipante venham a ser produzidos. Algumas das contribuições desta tese são: (1) mostrar a necessidade de se estabelecer um padrão de transcrição multimodal que possa contemplar as várias dimensões semióticas envolvidas na interação, revelando toda a complexidade do que está acontecendo momento a momento; (2) mostrar como uma análise multimodal minuciosa elucida detalhes interacionais que ficam apagados na análise verbo-vocal; (3) apresentar uma mudança na compreensão das ações gestuais produzidas pelo coparticipante durante o turno do falante, mostrando como essas ações não devem ser entendidas como pré-ações antecipatórias de desacordos verbais, mas como primeiras manifestações da discordância em processo de formação; (4) mostrar como as ações gestuais emergem como parte de uma organização sequencial no desenrolar da interação, agregando informações a respeito do tipo de ação em desenvolvimento; e (5) mostrar como as ações gestuais são práticas recorrentes que podem ser definidas dentro da sequência interacional quando se analisa a sua progressão temporal. / Research on the multimodality of interaction is being carried out in increasingly diverse fields of studies. However, comprehension of how meanings are constructed through the deployment of multiple semiotic dimensions acting concomitantly within the interactional event is still in its infancy. This dissertation contributes to expanding this understanding specifically in episodes of disagreement. Conversation Analysis and Gesture Studies are the theoretical bases for this dissertation. By applying Conversation Analysis, we are able to observe how participants orient towards each other, revealing how resources are mobilized in interactional practices that help them coordinate and accomplish their actions during the interaction. By applying the findings of Gesture Studies, we are able to explore the relations between language and bodily actions and to verify the important role of bodily action both for the establishment of interactional practices and for the framing of action formation. Varied types of interaction, approximately 15 hours of audiovisual records, collected between July 2012 and August 2015, and performed by native speakers of Brazilian Portuguese compose the data of this research. From this material, using ELAN, we selected excerpts for theoretical exemplification and 6 excerpts for detailed analysis of sequences of disagreement. The analyses show that participants use bodily actions as a dynamic and flexible resource, locally performed; that facial actions help participants demonstrate their positions and understandings about one another\'s actions; that coparticipants bodily actions, produced at transitional-relevance places in the speaker\'s turn seem to be taken as first manifestations of disagreements that are in the process of developing; that the speaker appears to consider these coparticipant actions in producing his subsequent utterances. Such a practice also seems to serve the purpose of giving the speaker an opportunity to promote interactive adjustments before verbal disagreements of the co-participant would be produced. Some of the contributions of this dissertation are: (1) to show the need to establish a multimodal transcription pattern that can contemplate the multiple semiotic dimensions involved in the interaction, revealing all the complexity of what is happening moment by moment; (2) to show how a detailed multimodal analysis elucidates interactional details that are \"erased\" in the verb-vocal analysis; (3) to present a change in the understandings of the bodily actions produced by the coparticipant during the speaker\'s turn, showing how these actions should not be understood as anticipatory pre-actions of verbal disagreements, but as the first manifestations of the disagreement that is in the formation process; (4) to show how bodily actions emerge as part of a sequential organization in the course of the interaction, adding information about the type of action in development; and (5) to show how bodily actions are recurrent practices that can be defined within the interactional sequence when analyzing its temporal progression.

A investigação da multimodalidade nas interações tem sido contemplada em campos de pesquisa cada vez mais diversos. No entanto, ainda é incipiente o entendimento de como os significados são construídos pela mobilização de múltiplas dimensões semióticas, atuando concomitantemente dentro do evento interacional. Esta tese busca contribuir para expandir esse entendimento, sobretudo, em episódios de desacordo. A Análise da Conversa, de base etnometodológica, e os Estudos dos Gestos são as linhas teóricas que balizam esta tese. Pelas contribuições da Análise da Conversa, pudemos observar como os próprios participantes se orientam uns em relação aos outros, revelando-nos os recursos mobilizados como práticas interacionais voltadas à realização e coordenação de suas ações durante a interação. Das contribuições dos Estudos dos Gestos, pudemos explorar as relações entre língua e ações gestuais e verificar o importante papel da gestualidade tanto para o estabelecimento de práticas interacionais quanto para o enquadramento das ações que estão em processo de formação. Os dados desta pesquisa compreendem vários tipos de interação realizados por falantes nativos de português brasileiro, perfazendo aproximadamente 15 horas de registros audiovisuais, coletados entre julho de 2012 e agosto de 2015. Desse material, selecionamos excertos para exemplificação teórica e 6 excertos para análise minuciosa, realizada dentro do programa ELAN, das sequências de desacordo. As análises mostram que os participantes se valem das ações gestuais como um recurso dinâmico e flexível que eles utilizam situadamente; que as ações faciais auxiliam os participantes a demonstrar suas posições e entendimentos a respeito das ações uns dos outros; que as ações gestuais dos coparticipantes produzidas em possíveis pontos de completude do turno do falante parecem ser tomadas como as primeiras manifestações das discordâncias que estão em processo de desenvolvimento; que o falante parece considerar essas ações do coparticipante para produzir seus enunciados subsequentes; que tal prática também parece servir ao propósito de oferecer ao falante uma oportunidade para promover ajustes interacionais antes que os desacordos verbais do coparticipante venham a ser produzidos. Algumas das contribuições desta tese são: (1) mostrar a necessidade de se estabelecer um padrão de transcrição multimodal que possa contemplar as várias dimensões semióticas envolvidas na interação, revelando toda a complexidade do que está acontecendo momento a momento; (2) mostrar como uma análise multimodal minuciosa elucida detalhes interacionais que ficam apagados na análise verbo-vocal; (3) apresentar uma mudança na compreensão das ações gestuais produzidas pelo coparticipante durante o turno do falante, mostrando como essas ações não devem ser entendidas como pré-ações antecipatórias de desacordos verbais, mas como primeiras manifestações da discordância em processo de formação; (4) mostrar como as ações gestuais emergem como parte de uma organização sequencial no desenrolar da interação, agregando informações a respeito do tipo de ação em desenvolvimento; e (5) mostrar como as ações gestuais são práticas recorrentes que podem ser definidas dentro da sequência interacional quando se analisa a sua progressão temporal. / Research on the multimodality of interaction is being carried out in increasingly diverse fields of studies. However, comprehension of how meanings are constructed through the deployment of multiple semiotic dimensions acting concomitantly within the interactional event is still in its infancy. This dissertation contributes to expanding this understanding specifically in episodes of disagreement. Conversation Analysis and Gesture Studies are the theoretical bases for this dissertation. By applying Conversation Analysis, we are able to observe how participants orient towards each other, revealing how resources are mobilized in interactional practices that help them coordinate and accomplish their actions during the interaction. By applying the findings of Gesture Studies, we are able to explore the relations between language and bodily actions and to verify the important role of bodily action both for the establishment of interactional practices and for the framing of action formation. Varied types of interaction, approximately 15 hours of audiovisual records, collected between July 2012 and August 2015, and performed by native speakers of Brazilian Portuguese compose the data of this research. From this material, using ELAN, we selected excerpts for theoretical exemplification and 6 excerpts for detailed analysis of sequences of disagreement. The analyses show that participants use bodily actions as a dynamic and flexible resource, locally performed; that facial actions help participants demonstrate their positions and understandings about one another\'s actions; that coparticipants bodily actions, produced at transitional-relevance places in the speaker\'s turn seem to be taken as first manifestations of disagreements that are in the process of developing; that the speaker appears to consider these coparticipant actions in producing his subsequent utterances. Such a practice also seems to serve the purpose of giving the speaker an opportunity to promote interactive adjustments before verbal disagreements of the co-participant would be produced. Some of the contributions of this dissertation are: (1) to show the need to establish a multimodal transcription pattern that can contemplate the multiple semiotic dimensions involved in the interaction, revealing all the complexity of what is happening moment by moment; (2) to show how a detailed multimodal analysis elucidates interactional details that are \"erased\" in the verb-vocal analysis; (3) to present a change in the understandings of the bodily actions produced by the coparticipant during the speaker\'s turn, showing how these actions should not be understood as anticipatory pre-actions of verbal disagreements, but as the first manifestations of the disagreement that is in the formation process; (4) to show how bodily actions emerge as part of a sequential organization in the course of the interaction, adding information about the type of action in development; and (5) to show how bodily actions are recurrent practices that can be defined within the interactional sequence when analyzing its temporal progression.

Cette thèse porte sur l'acquisition et le développement de la négation chez deux enfants monolingues anglaise et française, filmées entre 10 mois et 4 ans et 2 mois (66h) en interactions naturelles avec leur mère. Nous adoptons une perspective constructiviste et fonctionnaliste de la langue (Tomasello 2003) en tissant des liens avec la théorie des opérations énonciatives, la socialisation langagière et avec les études sur la gestualité. Notre définition du langage est large car nous analysons toutes les ressources sémiotiques dont le locuteur dispose pour se positionner en interaction. À l'aide d'un système de codage multimodal qui repose sur l'utilisation de logiciels compatibles, nous menons des analyses qualitatives et quantitatives de l'usage des modalités verbales et non-verbales pour l'expression de la négation chez l'enfant avant 4 ans.Après avoir présenté l'ancrage théorique (partie 1) et notre méthode (partie 2), nous montrons que la négation correspond à un grand nombre de fonctions pragmatiques qui sont exprimées à l'aide de la synchronisation de modalités distinctes (partie 3). Les résultats indiquent que distinguer le rôle des modalités dans la construction de l’énoncé permet de travailler sur la complexité du langage. Concernant la négation, nous observons qu’il s’agit d’une opération énonciative qui ne repose pas systématiquement sur les mêmes formes selon la fonction exprimée.Cette recherche montre que l'usage synchronisé de plusieurs modalités en contexte de négation est une compétence linguistique et cognitive. En outre, les formes négatives s’enrichissent et se spécialisent après 3 ans pour permettre l’expression d’intentions communicatives variées. / This research focuses on the acquisition and the development of negation in two monolingual French and English children filmed from 10 months to 4 years and 2 months old (66 hours) in natural mother-child dyadic interactions.We use a functionalist and constructivist theoretical approach (Tomasello 2003) but we also bring together French utter-centred approach to language, language socialisation and gesture studies. Our definition of language encompasses all verbal and non-verbal means of expression speakers use to position themselves within interaction. We developed a multimodal coding system relying on the use of several compatible programs to combine qualitative and quantitative analyses. This method offers the opportunity to investigate the expression of negation in verbal and non-verbal modalities in children under 4.After laying the theoretical background (Part 1), we will present our methodology (Part 2). Results show that negation refers to a vast range of pragmatic functions whose expression is fully embodied because it is conveyed through the synchronisation of several modalities of expression (Part 3). Our analysis of the interplay of modalities in the construction of meaning happens to be a great locus to account for the complexity of language. We also observe that negation is a meta-category which can be expressed by a variety of forms.Our research shows that the usage of synchronised modalities in negative contexts can be considered a linguistic and cognitive skill. Moreover, the set of forms for negation develops and specialises after 3 years and helps the child express various communicative intentions linked to negation.

Cette thèse propose une analyse multimodale de l’expression des émotions chez deux enfants américaines et leurs parents monolingues. Les enfants ont été filmées entre 11 mois et 3 and et 10 mois pour l’une et entre 1 an et 1 mois et 4 ans pour l’autre au cours d’interactions spontanées en milieu familial. Nous adoptons une définition du langage large car toutes les ressources sémiotiques sont à prendre en compte : ressources verbales (lexique, marqueurs grammaticaux), vocales (vocalisations), gestuelles et corporelles (gestes, expressions faciales, actions).Nous nous concentrons sur l’acquisition et le développement des marqueurs verbaux et non verbaux exprimant les émotions chez l’enfant et sur l’usage de ces marqueurs chez l’adulte. Nous montrons que des profils expressifs bien précis et distincts semblent déjà émerger chez les enfants, grandement influencés par l’input auquel ils sont exposés chaque jour.Au plan théorique, notre recherche s’inscrit dans une approche constructiviste et fonctionnaliste de la langue (Tomasello, 2003) et nous analysons les données à l’aune de la socialisation langagière, et des études sur la gestualité et les expressions faciales comme vecteurs d’informations communicationnelles. Au plan méthodologique, nous réalisons des analyses quantitatives et qualitatives afin d’éclairer les comportements propres à chaque locuteur.Après avoir exposé notre socle théorique et notre méthodologie (partie I), nous révélons nos résultats sur l’expression de trois émotions (colère, joie, et tristesse) chez les locuteurs adultes et enfants (partie II). Nos résultats suggèrent que le développement linguistique des enfants n’a pas d’incidence sur l’expression de leurs émotions, mais que l’input et les attitudes parentales jouent un rôle majeur dans l’acquisition et le développement de chaque modalité et dans la transmission de modèles expressifs. / This research provides a multimodal analysis of the expression of emotion in two monolingual American children and their parents. The children were filmed in natural interactions in a family setting from the ages of 11 months to 3 years 10 months, and from 1 year 1 month to 4 years.We adopted a broad definition of language in this research which encompasses various semiotic resources – from verbal resources (lexicon and grammatical features), to nonverbal (vocalizations, facial expressions, and gestures). We focus on the children’s acquisition and development of these verbal and nonverbal markers and on how they are used by their parents. Our research shows that children develop specific and distinct communicational patterns, which are greatly influenced by the input to which they are exposed.From a theoretical perspective, our research draws from a constructivist and functionalist approach (Tomasello, 2003), and our data is analyzed in light of language socialization and of studies which have shown that facial expressions and gestures are used as communicational signals in face-to-face dialogue. Our methodology combines quantitative and qualitative methods to investigate each speaker’s verbal and nonverbal behavior when expressing emotions.Having outlined our theoretical and methodological foundation (Part I), we present our results on the expression of three emotions (happiness, sadness, and anger) in children and adults (Part II). Our research suggests that while children’s linguistic development has little impact on the richness of their emotional expression parental input and attitudes both play a crucial role in the acquisition of each modality and in the transmission of communicational patterns.