The preliminary design phase occurs near the launch of an engineering project, normally after an initial requirements gathering phase. Through a series of meetingswhich gathers the key actors of a project, effective preliminary design involves discussion and decision-making punctuated by group creativity techniques. These activities are designed to explore the potential solutions of the problem, such asbrainstorming or causal analysis, or to address the project itself, such as collaborative project planning. Such activities are usually conducted in traditional meeting rooms with pen and paper media, which requires significant time and effort to prepare, perform, and later render into a digitally exploitable format. These processes have resisted previous attempts of computer-supported solutions, because any additional instruments risk obstructing the natural collaboration and workflow that make these activities so beneficial. Over the past decade, technologies such as interactive table tops, interactive wall displays, speech recognition software, 3D motion sensing cameras, and handheld tablets and smartphones have experienced significant advances in maturity. Theirform factors resemble the physical configuration of traditional pen-and-paper environments,while their "natural" input devices (based on multi-touch, gestures, voice, tangibles, etc.) allow them to leverage a user's pre-existing verbal, spatial,social, motor and cognitive skills. Researchers hypothesize that having these devices working in concert inside interactive spaces could augment collaboration forco-located (i.e. physically present) groups of users.There currently exist several interactive spaces in the literature, illustrating awide range of potential hardware configurations and interaction techniques. The goal of this thesis is first to explore what qualities these interactive spaces should exhibit in their interaction design, particularly with regard to preliminary designactivities, and second, to investigate how their heterogeneous and distributed computing devices can be unified into a flexible and extensible distributed computing architecture. The first main contribution of this thesis is an extensive presentation of an interactive space, which at its core uses a configuration not yet fully explored inprevious literature : a large multitouch table top and a large multitouch interactive Abstract board display. The design of this interactive space is driven by observations o fgroups engaged in preliminary design activities in traditional environments and a literature review aimed at extracting user-centered design guide lines. Special consideration is given to the user interface as it extends across multiple shared displays, and maintains a separation of concerns regarding personal and group work. Finally, evaluations using groups of five and six users show that using such an interactive space, coupled with our proposed multi-display interaction techniques, leads to a more effective construction of the digital artifacts used in preliminary design.The second main contribution of this thesis is a multi-agent infrastructure forthe distributed computing environment which effectively accommodates a widerange of platforms and devices in concerted interaction. By using agent-oriented programming and by establishing a common content language for messaging, the infrastructure is especially tolerant of network faults and suitable for rapid prototyping of heterogeneous devices in the interactive space.

Made available in DSpace on 2016-06-02T19:07:09Z (GMT). No. of bitstreams: 1 LEME_Ricardo_2015.pdf: 7202991 bytes, checksum: e01ea6e52b9be5ca7ceca1056abb2bcc (MD5) Previous issue date: 2015-01-23 / The use of small mobile devices such as smartphones and tablets has been growing in various public in Brazil. Even though many devices have features such as touch, gestures, voice, many applications do not adopt such resources to support different ways of interaction. Among the various audiences that have been using the mobile devices in Brazil, it can highlight the elderly person. To understand the needs of elderly audience is important not only observe their actions as well as brings them to inside of the process of development. User Centered Design (UCD) is a methodology that uses models, methods and processes for software design aiming to fulfill the user needs, composed by the steps of research, development and evaluation. Based on the UCD principals, this work presents an approach to support the design of interaction which besides being to the user, is also developed with the elderly user adding to the UCD cycle the techniques of participatory design and of personas. The application developed by the approach allows the elderly users to use the multimodal interaction according to their preferences and to the context of use. The methodology of this work was composed by three steps. First, a systematic review gave the support to the fundamentals and to the delimitation of the work scope. After that, the approach was designed based on UCD method including the participatory aspects. The proposal verification was conducted following the approach phases with the participation of a total of 279 elderly users. The results show that the elderly user, as a active member of the development process, aiding on the identification of real interaction aspects of the application. / O uso de dispositivos móveis, como smartphones e tablets vem crescendo junto a vários públicos no Brasil. Mesmo que muitos dispositivos possuam características tais como toque, gestos, voz, muitas aplicações não adotam tais recursos para apoiar diferentes formas de interação. Entre os diversos públicos que utilizam os dispositivos móveis no Brasil, pode-se destacar o usuário da terceira idade. Para entender as necessidades deste público, é importante não somente observar as suas ações, mas trazê-los para o processo de desenvolvimento. O Design Centrado no Usuário (UCD) é uma metodologia que utiliza modelos, métodos e processos de design de software com o objetivo para atender as necessidades do usuário, composto pelas etapas de pesquisa, desenvolvimento e avaliação. Com base nos princípios da UCD, este trabalho apresenta uma abordagem para apoiar o projeto de interação que além de ser para o usuário, também é desenvolvido com o usuário da terceira idade somando-se ao ciclo da UCD as técnicas de design participativo e de personas. A aplicação desenvolvida pela abordagem permite que os usuários idosos utilizem a interação multimodal de acordo com as suas preferências e para o contexto de utilização. A metodologia deste trabalho foi composta por três etapas. Em primeiro lugar, uma revisão sistemática deu o apoio para os fundamentos e para a delimitação do âmbito de trabalho. Depois disso, a abordagem foi concebida com base no método da UCD incluindo os aspectos de participação. A abordagem proposta foi realizada seguindo as fases de aproximação com a participação de um total de 279 usuários idosos. Os resultados mostram que o usuário idoso, como um membro ativo do processo de desenvolvimento, pode auxiliar na identificação de aspectos reais de interação da aplicação.

The thesis project concerns the development and evaluation of a real-time music visualization system aimed at creating a multi-modal perceptual experience of music emotions. The purpose of the project is to provide singers with real-time visual feedback on their singing, to enhance their expression of emotions in the music. Built upon results from previous studies on emotional expression in music, crossmodal correspondences, and associations among sound, shape, color, and emotions, a singing voice visualization system is proposed that generates real-time graphics to reflect the emotional expression in the input singing in an intuitive fashion. A mapping between musical and visual features was established and tested within a user study regarding the setting of its polarities. The singing voice visualization system was developed as as a software system that runs on personal computers, utilizing Pure Data and Unity. This implementation allows for instantaneous feedback to the user during their singing. The mapping was evaluated in a user study, where participants engage in expressive singing to test the system,in order to assess the meaningfulness of the visual feedback and the effectiveness of the mapping, as well as the impact of the polarity. The results show that color as a strong visual cue of emotional expression provided meaningful information on some participants’ expression of typical happiness and sadness. Other cues of the visual feedback possibly enhanced some participants’ emotional expression of singing in an indirect way. The polarity had a noticeable impact on the perception of the visual feedback. The current study is limited by the reliability of the techniques used for the extracting acoustic features from real-time singing, particularly in the detection of attack speed. The evaluation were limited by the broad definition of one of the research questions. The findings of this study suggest potential applications for the singing voice visualization system in fields of music education, art, and entertainment. Additionally, the research highlights the need for further exploration and refinement in the design of the mapping and the evaluation methodology. / Avhandlingsprojektet handlar om utveckling och utvärdering av ett realtidssystem för musikvisualisering som syftar till att skapa en multimodal perceptuell upplevelse av musikkänslor. Syftet med projektet är att ge sångare visuell feedback i realtid på deras sång, för att förstärka deras uttryck av känslor i musiken. Med utgångspunkt i resultat från tidigare studier om känslouttryck i musik, korsmodala korrespondenser och associationer mellan ljud, form, färg och känslor, föreslås ett visualiseringssystem för sångstämmor som genererar realtidsgrafik för att på ett intuitivt sätt återspegla känslouttrycket i den inmatade sången. En mappning mellan musikaliska och visuella funktioner etablerades och testades i en användarstudie avseende inställningen av dess polariteter. Visualiseringssystemet för sångröst utvecklades som ett mjukvarusystem som körs på persondatorer med Pure Data och Unity. Denna implementering möjliggör omedelbar feedback till användaren under dennes sång. Kartläggningen utvärderades i en användarstudie, där deltagarna fick sjunga uttrycksfullt för att testa systemet, för att bedöma meningsfullheten i den visuella feedbacken och kartläggningens effektivitet, samt polaritetens inverkan. Resultaten visar att färg som en stark visuell signal för känslomässiga uttryck gav meningsfull information om vissa deltagares uttryck av typisk lycka och sorg. Andra ledtrådar i den visuella återkopplingen förstärkte möjligen vissa deltagares känslomässiga uttryck av sång på ett indirekt sätt. Polariteten hade en märkbar inverkan på uppfattningen av den visuella feedbacken. Den aktuella studien begränsas av tillförlitligheten hos de tekniker som används för att extrahera akustiska egenskaper från sång i realtid, särskilt när det gäller att upptäcka attackhastighet. Utvärderingen begränsades av den breda definitionen av en av forskningsfrågorna. Resultaten av denna studie visar på potentiella tillämpningar för visualiseringssystemet för sångröster inom musikutbildning, konst och underhållning. Dessutom belyser forskningen behovet av ytterligare utforskning och förfining av kartläggningens utformning och utvärderingsmetodiken.

[pt] A presente tese investiga a multimodalidade em interações sinalizadas, em aulas via Google Meet. Os objetivos principais são descrever e analisar os aspectos verbais e não verbais da Libras. Este estudo está alinhado com as bases teóricas da Análise da Conversa (AC), da Sociolinguística Interacional (SI) e da Linguística Aplicada (LA). Esta pesquisa é de natureza qualitativa e utiliza a Análise da Conversa Multimodal como teoria e método. Utiliza a sequencialidade como teoria e, também, como categoria analítica. Os princípios da sequencialidade auxiliaram na compreensão das ações conjuntas construídas pelos surdos, em aulas virtuais. Os dados apresentados foram gerados de setembro a dezembro de 2020, nos encontros virtuais entre professor surdo e estudantes surdos que participam de um projeto de extensão situado na Zona da Mata Mineira. Para fins de transcrição e de análise de dados, adotaram-se critérios capazes de abarcar a interação multimodal em línguas sinalizadas. Então, utilizaram-se princípios da Tradução Audiovisual para se poder descrever as ações em Libras, aliando-se aos critérios de transcrição multimodal. Através da análise sequencial de turnos de fala, compreendem-se os diferentes enquadres presentes nas aulas virtuais sinalizadas. Nota-se que os aspectos multimodais presentes na interação dão a clave da interação, mostrando contextualmente qual ação os participantes estão performando momento a momento. Assim, as diferentes ações performadas em sala de aula, como a brincadeira, a narração, a explicação, mostram os papéis institucionais presentes nas aulas. Em alguns momentos, as relações tornam-se mais assimétricas, em outros, mais simétricas, dependendo das ações que os participantes estão performando. A compreensão desses significados interacionais só é possível quando conseguimos analisar a situação social como um todo, considerando os aspectos linguísticos, gramaticais, prosódicos, materiais e gestuais presentes nos turnos de fala, ou turnos sinalizados. Ao se olhar para os gestos presentes na Libras, é possível contribuiem mais r uma maneira de m se compreender o para o campo da Linguística Aplicada, pensando b aplicação no campo educacional, pensando abarque a experiência linguística que o surdo possu escolar institucional. / [en] This thesis investigates multimodality in signaled interactions, in classes given through Google Meet. The main objectives are to describe and analyze verbal and non-verbal aspects of Libras. This study is aligned with theoretical bases of Conversation Analysis (CA), Interactional Sociolinguistics (IS) and Applied Linguistics (AL). This research is qualitative in nature and uses Multimodal Conversation Analysis as theory and method. Uses sequentiality as a theory and also as an analytical category. The principles of sequentiality helped to understand the joint actions built by the deaf, in virtual classes. The data presented was generated from September to December 2020, in virtual meetings between deaf professor and deaf students who participate in an extension project located in Zona da Mata Mineira. For transcription and data analysis, criteria capable of encompassing multimodal interaction in signed languages were adopted. Then, principles of Audiovisual Translation were used to be able to describe the actions in Libras, allied to the criteria of multimodal transcription. Through sequential analysis of speech turns, different framings present in the signed virtual classes are understood. It is noted that the multimodal aspects present in the interaction give the key to the interaction, contextually showing which action the participants are performing moment by moment. Thus, different actions performed in the classroom, such as game, narration, explanation, show the institutional roles present in classes. In some moments, relationships become more asymmetrical, in others, more symmetrical, depending on the actions that participants are performing. Understanding these interactional meanings is only possible when we are able to analyze the social situation as a whole, considering linguistic, grammatical, prosodic, material and gestural aspects present in speech turns, or signed turns. When looking at the gestures present in Libras, it is possible to contribute to the field of Applied Linguistics, thinking of another way of understanding deaf bilingualism. There are also possibilities of application in educational field, thinking about educational policies that cover linguistic experience that the deaf has before going to the institutional school space.

Eye tracking and gestures are relatively new input methods, changing the way humans interact with computers. Gestures can be used for games or controlling a computer through an interface. Eye tracking is another way of interacting with computers, often by combining with other inputs such as a mouse or touch pad. Gestures and eye tracking have been used in commercially available products, but seldom combined to create a multimodal interaction. This thesis presents a prototype which combines eye tracking with gestures to interact with a computer. To accomplish this, the report investigates different methods of recognizing hand gestures. The aim is to combine the technologies in such a way that the gestures can be simple, and the location of a user’s gaze will decide what the gesture does. The report concludes by presenting a final prototype where the gestures are combined with eye tracking to interact with a computer. The final prototype uses an IR camera together with an eye tracker. The final prototype is evaluated with regards to learnability, usefulness, and intuitiveness. The evaluation of the prototype shows that usefulness is low, but learnability and intuitiveness are quite high. / Eye tracking och gester är relativt nya inmatningsmetoder, som förändra sättet människor interagerar med datorer. Gester kan användas för till exempel spel eller för att styra en dator via ett gränssnitt. Eye tracking är ett annat sätt att interagera med datorer, ofta med hjälp av genom att kombinera med andra styrenheter såsom en mus eller styrplatta. Gester och eye tracking har använts i kommersiellt tillgängliga produkter, men sällan kombinerats för att skapa en multimodal interaktion. Denna avhandling presenterar en prototyp som kombinerar eye tracking med gester för att interagera med en dator. För att åstadkomma detta undersöker rapporten olika metoder för att känna igen gester. Målet är att kombinera teknologierna på ett sådant sätt att gestern kan vara enkla, och platsen för användarens blick kommer bestämma vad gesten gör. Rapporten avslutas genom att presentera en slutlig prototyp där gester kombineras med eye tracking för att interagera med en dator. Den slutliga prototypen använder en IR kamera och en eye tracker. Den slutliga prototypen utvärderas med avseende på lärbarhet, användbarhet, och intuition. Utvärderingen av prototypen visar att användbarheten är låg, men både lärbarhet och intuition är ganska höga.

No description available.

Multiprofessionella team i rehabiliteringsverksamheter kräver att patienter kontinuerligt uttrycker sina önskningar och behov för att teammedlemmar ska kunna arbeta personcentrerat. För patienter med kommunikativa svårigheter kan det uppstå hinder i vårdsammanhang i form av minskad delaktighet gällande sin vård. För att effektivisera kommunikationen och motverka låg delaktighet behöver anpassningar göras. Anpassningar kan komma i form av alternativ och kompletterande kommunikation (AKK) eller andra strategier som underlättar ett samtal för båda samtalsparter. Föreliggande studie undersöker interaktionen mellan logopeder och patienter med kommunikativa svårigheter med respektive utan AKK under vårdbesök. Studien innefattar dessutom vårdpersonals samt patienters upplevelser avseende AKK och kommunikation.  Materialet bestod av videoinspelningar av två behandlingsbesök med två logopeder och två patienter som använder AKK. I samband med videoinspelningarna intervjuades även patienterna. Därtill inkluderades intervjuer med tio deltagare som representerade fem olika professioner. Det inhämtade videomaterialet analyserades med hjälp av multimodal interaktionsanalys och intervjuerna analyserades tematiskt. Resultatet som framkom i studien indikerade att interaktionen mellan vårdpersonal och patienter generellt förbättras vid AKK-användning. Däremot finns faktorer som påverkar effektiviteten av AKK-användningen, till exempel vad det är för omgivning, patientens motivation, frustration respektive insikt om kommunikationssvårigheterna samt vårdens bemötande och vilka anpassningar som tillämpas. Intervjuerna med vårdpersonalen visade att logopeden har en betydande roll inom teamet i relation till att öka kunskapen och medvetenheten avseende hur man samtalar med patienter med kommunikativa svårigheter. Videoinspelningarna och tillhörande patientintervjuer stöttade flera aspekter som framkom i vårdpersonalintervjuerna samtidigt som de bidrog med ytterligare perspektiv. Slutsatsen som dras i föreliggande studie är behovet av ökad kunskap avseende kommunikation och AKK, vilket kan leda vidare till ett tätare samarbete mellan teammedlemmar som arbetar med patienter med kommunikativa svårigheter. Slutligen är det av vikt att betona att kommunikativa svårigheter inte endast beror på en patients insjuknande, utan de blir mer påtagliga när goda förutsättningar i olika sammanhang inte finns. / Multiprofessional teams within rehabilitation clinics are dependent on patients verbalizing their needs and wishes in order for the team members to work with a person-centered approach. Patients with communication difficulties may experience reduced participation regarding their own healthcare. Therefore, it is important that adjustments are implemented to increase communication efficiency and the patient’s participation. The adjustments can be Augmentative and Alternative Communication (AAC) or other strategies that facilitate a conversation between the participants. The present study investigated interaction between speech and language pathologists (SLPs) and patients with communication difficulties with or without AAC during a medical appointment. The study also investigates experiences of healthcare professionals and patients regarding communication and AAC.  The data consisted of video recordings of two treatment visits between two SLPs and two patients that use AAC. The patients were interviewed after the visit. The data also consisted of interviews with ten participants that represented five different professions that regularly work with patients who use AAC. The video data was analyzed using multimodal interaction analysis and for analysis of the interviews with professionals, thematic analysis was employed. The findings indicated that the interaction between healthcare professionals and patients was improved when AAC was used. There were, however, factors that may affect the effectiveness of AAC, such as the type of environment, the patient’s motivation, frustration and an awareness of the communication difficulties, as well as healthcare providers’ attitudes and adjustments that are implemented. Analysis of the interview conducted with healthcare providers demonstrated the vital role of the SLP in a team in regard to improving knowledge and awareness of how to communicate with patients with communication difficulties. The video recordings with complementary patient interviews supported many aspects that emerged in the interviews with healthcare providers. Video analysis also contributed to additional perspectives.  Results of the present study imply that there is a need to improve the knowledge of communication and AAC in interprofessional teams, which could also lead to enhanced collaboration between team members. Lastly, it is of importance to highlight that communication difficulties are not solely due to a patient’s illness.  Communication difficulties become more pronounced when various contextual conditions are not favorable.

Tesis por compendio / [ES] La Realidad Aumentada (Augmented Reality, AR) y su capacidad para integrar contenido sintético sobre una imagen real proporciona un valor incalculable en diversos campos; no obstante, la industria es uno de estos campos que más se puede aprovechar de ello. Como tecnología clave en la evolución hacia la Industria 4.0 y 5.0, la AR no solo complementa sino que también potencia la interacción humana con los procesos industriales. En este contexto, la AR se convierte en una herramienta esencial que no sustituye al factor humano, sino que lo enriquece, ampliando sus capacidades y facilitando una colaboración más efectiva entre humanos y tecnología. Esta integración de la AR en entornos industriales no solo mejora la eficiencia y precisión de las tareas, sino que también abre nuevas posibilidades para la expansión del potencial humano. Existen numerosas formas en las que el ser humano interactúa con la tecnología, siendo la AR uno de los paradigmas más innovadores respecto a cómo los usuarios acceden a la información; sin embargo, es crucial reconocer que la AR, por sí misma, tiene limitaciones en cuanto a la interpretación del contenido que visualiza. Aunque en la actualidad podemos acceder a diferentes librerías que utilizan algoritmos para realizar una detección de imágenes, objetos, o incluso entornos, surge una pregunta fundamental: ¿hasta qué punto puede la AR comprender el contexto de lo que ve? Esta cuestión se vuelve especialmente relevante en entornos industriales. ¿Puede la AR discernir si una máquina está funcionando correctamente, o su rol se limita a la presentación de indicadores digitales superpuestos? La respuesta a estas cuestiones subrayan tanto el potencial como los límites de la AR, impulsando la búsqueda de innovaciones que permitan una mayor comprensión contextual y adaptabilidad a situaciones específicas dentro de la industria. En el núcleo de esta tesis yace el objetivo de no solo dotar a la AR de una "inteligencia semántica" capaz de interpretar y adaptarse al contexto, sino también de ampliar y enriquecer las formas en que los usuarios interactúan con esta tecnología. Este enfoque se orienta particularmente a mejorar la accesibilidad y la eficiencia de las aplicaciones de AR en entornos industriales, que son por naturaleza restringidos y complejos. La intención es ir un paso más allá de los límites tradicionales de la AR, proporcionando herramientas más intuitivas y adaptativas para los operadores en dichos entornos. La investigación se despliega a través de tres artículos de investigación, donde se ha desarrollado y evaluado una arquitectura multimodal progresiva. Esta arquitectura integra diversas modalidades de interacción usuario-tecnología, como el control por voz, la manipulación directa y el feedback visual en AR. Además, se incorporan tecnologías avanzadas basadas en modelos de aprendizaje automática (Machine Learning, ML) y aprendizaje profundo (Deep Learning, DL) para extraer y procesar información semántica del entorno. Cada artículo construye sobre el anterior, demostrando una evolución en la capacidad de la AR para interactuar de manera más inteligente y contextual con su entorno, y resaltando la aplicación práctica y los beneficios de estas innovaciones en la industria. / [CA] La Realitat Augmentada (Augmented Reality, AR) i la seua capacitat per integrar contingut sintètic sobre una imatge real ofereix un valor incalculable en diversos camps; no obstant això, la indústria és un d'aquests camps que més pot aprofitar-se'n. Com a tecnologia clau en l'evolució cap a la Indústria 4.0 i 5.0, l'AR no només complementa sinó que també potencia la interacció humana amb els processos industrials. En aquest context, l'AR es converteix en una eina essencial que no substitueix al factor humà, sinó que l'enriqueix, ampliant les seues capacitats i facilitant una col·laboració més efectiva entre humans i tecnologia. Esta integració de l'AR en entorns industrials no solament millora l'eficiència i precisió de les tasques, sinó que també obri noves possibilitats per a l'expansió del potencial humà. Existeixen nombroses formes en què l'ésser humà interactua amb la tecnologia, sent l'AR un dels paradigmes més innovadors respecte a com els usuaris accedeixen a la informació; no obstant això, és crucial reconéixer que l'AR, per si mateixa, té limitacions quant a la interpretació del contingut que visualitza. Encara que en l'actualitat podem accedir a diferents llibreries que utilitzen algoritmes per a realitzar una detecció d'imatges, objectes, o fins i tot entorns, sorgeix una pregunta fonamental: fins a quin punt pot l'AR comprendre el context d'allò veu? Esta qüestió esdevé especialment rellevant en entorns industrials. Pot l'AR discernir si una màquina està funcionant correctament, o el seu rol es limita a la presentació d'indicadors digitals superposats? La resposta a estes qüestions subratllen tant el potencial com els límits de l'AR, impulsant la recerca d'innovacions que permeten una major comprensió contextual i adaptabilitat a situacions específiques dins de la indústria. En el nucli d'esta tesi jau l'objectiu de no solament dotar a l'AR d'una "intel·ligència semàntica" capaç d'interpretar i adaptar-se al context, sinó també d'ampliar i enriquir les formes en què els usuaris interactuen amb esta tecnologia. Aquest enfocament s'orienta particularment a millorar l'accessibilitat i l'eficiència de les aplicacions d'AR en entorns industrials, que són de naturalesa restringida i complexos. La intenció és anar un pas més enllà dels límits tradicionals de l'AR, proporcionant eines més intuïtives i adaptatives per als operaris en els entorns esmentats. La recerca es desplega a través de tres articles d'investigació, on s'ha desenvolupat i avaluat una arquitectura multimodal progressiva. Esta arquitectura integra diverses modalitats d'interacció usuari-tecnologia, com el control per veu, la manipulació directa i el feedback visual en AR. A més, s'incorporen tecnologies avançades basades en models d'aprenentatge automàtic (ML) i aprenentatge profund (DL) per a extreure i processar informació semàntica de l'entorn. Cada article construeix sobre l'anterior, demostrant una evolució en la capacitat de l'AR per a interactuar de manera més intel·ligent i contextual amb el seu entorn, i ressaltant l'aplicació pràctica i els beneficis d'estes innovacions en la indústria. / [EN] Augmented Reality (AR) and its ability to integrate synthetic content over a real image provides invaluable value in various fields; however, the industry is one of these fields that can benefit most from it. As a key technology in the evolution towards Industry 4.0 and 5.0, AR not only complements but also enhances human interaction with industrial processes. In this context, AR becomes an essential tool that does not replace the human factor but enriches it, expanding its capabilities and facilitating more effective collaboration between humans and technology. This integration of AR in industrial environments not only improves the efficiency and precision of tasks but also opens new possibilities for expanding human potential. There are numerous ways in which humans interact with technology, with AR being one of the most innovative paradigms in how users access information; however, it is crucial to recognize that AR, by itself, has limitations in terms of interpreting the content it visualizes. Although today we can access different libraries that use algorithms for image, object, or even environment detection, a fundamental question arises: To what extent can AR understand the context of what it sees? This question becomes especially relevant in industrial environments. Can AR discern if a machine functions correctly, or is its role limited to presenting superimposed digital indicators? The answer to these questions underscores both the potential and the limits of AR, driving the search for innovations that allow for greater contextual understanding and adaptability to specific situations within the industry. At the core of this thesis lies the objective of not only endowing AR with "semantic intelligence" capable of interpreting and adapting to context, but also of expanding and enriching the ways users interact with this technology. This approach mainly aims to improve the accessibility and efficiency of AR applications in industrial environments, which are by nature restricted and complex. The intention is to go beyond the traditional limits of AR, providing more intuitive and adaptive tools for operators in these environments. The research unfolds through three articles, where a progressive multimodal architecture has been developed and evaluated. This architecture integrates various user-technology interaction modalities, such as voice control, direct manipulation, and visual feedback in AR. In addition, advanced technologies based on Machine Learning (ML) and Deep Learning (DL) models are incorporated to extract and process semantic information from the environment. Each article builds upon the previous one, demonstrating an evolution in AR's ability to interact more intelligently and contextually with its environment, and highlighting the practical application and benefits of these innovations in the industry. / Izquierdo Doménech, JJ. (2024). Enhancing Industrial Process Interaction Using Deep Learning, Semantic Layers, and Augmented Reality [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/205523 / Compendio

This doctoral dissertation is an empirical qualitative research study on the emergence of learnables in classrooms of Swedish as a second language. It adopts a dialogical and praxeological approach, and analysis is based on video recorded teacher-student interactivities in classrooms. Learnables are taken to be linguistic items or constructs that are displayed as unknown by students, or problematized by students or teachers, and therefore oriented to as explainable, remediable, or improvable. Learnables are introduced in planned or less planned classroom activities, either in passing, while continuing the current main activity, or in sidesequences. In these activities, teachers and students not only talk, but also use other embodied resources (e.g. pointing) or available artifacts (e.g. worksheets) to highlight linguistic learnables. Teachers and students use these resources for achieving and maintaining intersubjectivity as well as contributing learnables to the interactivities. Through manifest embodied practices, abstract linguistic learnables become objectified, and knowledge about them gets organized in and through joint co-operative activities. / Denna avhandling är en empirisk, kvalitativ studie om uppkomsten av s.k. ”learnables” i svenska som andraspråksutbildning. Studien antar ett dialogiskt och praxeologiskt perspektiv, och analysen baseras på video-inspelade lärare-elevinteraktiviteter i klassrummet. ”Learnables” utgörs av språkliga objekt eller konstruktioner, som hanteras som obekanta av elever, eller som problematiseras av elever eller lärare, och därför orienteras emot som objekt som kan förklaras, korrigeras eller förbättras. ”Learnables” kan uppstå i planerade eller mindre planerade klarssrumsaktiviteter, antingen i förbigående, samtidigt som huvudaktiviteten fortsätter utan avbrott, eller i sidosekvenser. I dessa aktiviteter använder lärare och elever inte bara talspråk, utan även andra kroppsliga resurser (t.ex. pekningar) eller tillgängliga artefakter (t.ex. papper) för att fokusera på språkliga ”learnables”. Lärare och elever använder dessa medel för att uppnå och bibehålla intersubjektivitet, samt för att bidra med ”learnables” till interaktiviteterna. Genom konkreta kroppsliga metoder blir abstrakta, språkliga ”learnables” objektifierade och kunskapen om dem organiseras i och genom deltagarnas koordinerade handlingar.

La phase de conception préliminaire est déterminante lors de la réalisation d’un projet industriel. Elle exploite généralement des outils méthodologiques tels que le brainstorming, l’analyse causale et le chronogramme, qui permettent la collaboration entre des participants aux compétences et aux approches différentes. Ces activités se déroulent dans des salles de réunions traditionnelles, autour d'une table ou devant un tableau blanc, avec l’aide de nombreux papiers et Post-it, ce qui rend la préparation, l’exécution, et l’exploitation de ce processus particulièrement difficile. Jusqu’à présent, cette phase de conception préliminaire a résisté à la numérisation, notamment parce que l’addition d’un dispositif informatique au sein de ces activités perturbe la communication et la collaboration naturelles entre participants. Au cours des dix dernières années, de nombreuses avancées technologiques ont été réalisées en ce qui concerne les dispositifs numériques tels que les tables et les tableaux interactifs, les smartphones et les tablettes tactiles. La similarité des configurations physiques de ces dispositifs avec les dispositifs plus traditionnels permet d’exploiter les capacités préexistantes des utilisateurs (l’habileté motrice, le raisonnement spatial, le langage parlé, etc.). Les chercheurs se basent sur l’hypothèse que ces nouveaux dispositifs, travaillant de concert au sein d’espaces interactifs, pourront augmenter la collaboration co-localisée pour les équipes de conception préliminaire. L’objectif de cette thèse est, d’une part, d’étudier la conception d’un espace interactif pour la collaboration co-localisée durant la phase de conception préliminaire, et d’autre part, de proposer une architecture permettant de réunir les dispositifs hétérogènes et distribués composant cet espace.La première contribution consiste en une présentation détaillée d’un espace interactif utilisant une configuration physique encore peu exploitée dans la littérature scientifique : une table et un tableau multi-tactiles de grandes dimensions. La conception de cet espace interactif a été basée sur des observations d’utilisateurs dans un contexte de conception préliminaire traditionnel et sur une revue de la littérature visant à identifier des principes de conception. Lors de la conception de cet espace, une attention particulière a été portée à l’interface utilisateur qui s’étend sur des écrans partagés et qui maintient une séparation entre les activités d’un participant et les activités d’une équipe. Enfin, les évaluations, réalisées avec des groupes de cinq à six participants, démontrent une amélioration dans l’exploitation des outils méthodologiques sur supports numériques par rapport à une utilisation traditionnelle lors de la conception préliminaire.La conception d’une infrastructure distribuée basée sur un système multi-agents constitue la deuxième contribution de cette thèse. Cette infrastructure parvient à rassembler de nombreuses plateformes et des dispositifs hétérogènes. Elle représente une solution intéressante pour les espaces interactifs, en particulier parce qu’elle tolère particulièrement bien la défaillance de réseau et permet un prototypage rapide des dispositifs. / The preliminary design phase occurs near the launch of an engineering project, normally after an initial requirements gathering phase. Through a series of meetingswhich gathers the key actors of a project, effective preliminary design involves discussion and decision-making punctuated by group creativity techniques. These activities are designed to explore the potential solutions of the problem, such asbrainstorming or causal analysis, or to address the project itself, such as collaborative project planning. Such activities are usually conducted in traditional meeting rooms with pen and paper media, which requires significant time and effort to prepare, perform, and later render into a digitally exploitable format. These processes have resisted previous attempts of computer-supported solutions, because any additional instruments risk obstructing the natural collaboration and workflow that make these activities so beneficial. Over the past decade, technologies such as interactive table tops, interactive wall displays, speech recognition software, 3D motion sensing cameras, and handheld tablets and smartphones have experienced significant advances in maturity. Theirform factors resemble the physical configuration of traditional pen-and-paper environments,while their “natural” input devices (based on multi-touch, gestures, voice, tangibles, etc.) allow them to leverage a user’s pre-existing verbal, spatial,social, motor and cognitive skills. Researchers hypothesize that having these devices working in concert inside interactive spaces could augment collaboration forco-located (i.e. physically present) groups of users.There currently exist several interactive spaces in the literature, illustrating awide range of potential hardware configurations and interaction techniques. The goal of this thesis is first to explore what qualities these interactive spaces should exhibit in their interaction design, particularly with regard to preliminary designactivities, and second, to investigate how their heterogeneous and distributed computing devices can be unified into a flexible and extensible distributed computing architecture. The first main contribution of this thesis is an extensive presentation of an interactive space, which at its core uses a configuration not yet fully explored inprevious literature : a large multitouch table top and a large multitouch interactive Abstract board display. The design of this interactive space is driven by observations o fgroups engaged in preliminary design activities in traditional environments and a literature review aimed at extracting user-centered design guide lines. Special consideration is given to the user interface as it extends across multiple shared displays, and maintains a separation of concerns regarding personal and group work. Finally, evaluations using groups of five and six users show that using such an interactive space, coupled with our proposed multi-display interaction techniques, leads to a more effective construction of the digital artifacts used in preliminary design.The second main contribution of this thesis is a multi-agent infrastructure forthe distributed computing environment which effectively accommodates a widerange of platforms and devices in concerted interaction. By using agent-oriented programming and by establishing a common content language for messaging, the infrastructure is especially tolerant of network faults and suitable for rapid prototyping of heterogeneous devices in the interactive space.