Este trabalho aborda o uso de experimentação remota como forma de aprimorar a educação em sistemas de controle e automação. Para isto, uma arquitetura é proposta, onde experimentos reais podem ser combinados com partes simuladas (virtuais) criando um experimento de realidade mista. Um ambiente de ensino à distância, acessível na Internet, engloba o experimento remoto para proporcionar maiores recursos pedagógicos, guias e materiais educacionais teóricos aos estudantes. As diversas tecnologias, presentes no estado da arte, utilizadas para a elaboração da arquitetura são descritas e comparadas para facilitar o entendimento das escolhas realizadas. Trabalhos correlatos são apresentados para descrever o estado da arte dos laboratórios de experiências remotas de diversos tipos. Com base neste estudo, deficiências e vantagens são apontadas para a criação da arquitetura proposta. Para validar a arquitetura são propostos diversos estudos de caso com implementações utilizadas nos cursos de automação e controle da instituição na qual este trabalho está relacionado, ou ainda em cursos de educação à distância para atingir um público mais amplo. Esta dissertação está relacionada como resultado importante do consórcio internacional RExNet, financiado pelo projeto europeu Alfa, do qual a UFRGS faz parte. / This work aims to the use of remotely web-based experiments to improve the learning process of automation and control systems theory courses. An architecture combining virtual learning environments, remote experiments, students guide and experiments analysis is proposed based on a wide state of art study. The validation of the architecture uses state of art technologies and new simple developed programs to implement the case studies presented. All implementations presented in this work uses an Internet accessible virtual learning environment providing educational resources, guides and learning material to create a distance learning course associated with the remote mixed reality experiment. The experiment is integrated in the learning environment and support the mixture of real equipments combined with simulated ones (virtual). This mixture of equipments (components) is automatic controlled by the user and is a new developed technology called interchangeable components. The state of art section of this work presents, compare and describes several technologies used to implement the proposed architecture. Related work of remote experimentation and virtual learning environments is also presented. The architecture was elaborated based on appointed advantages and drawbacks of the related studies. This work is part of the RExNet consortium, supported by the European Alfa project.

Este trabalho aborda o uso de experimentação remota como forma de aprimorar a educação em sistemas de controle e automação. Para isto, uma arquitetura é proposta, onde experimentos reais podem ser combinados com partes simuladas (virtuais) criando um experimento de realidade mista. Um ambiente de ensino à distância, acessível na Internet, engloba o experimento remoto para proporcionar maiores recursos pedagógicos, guias e materiais educacionais teóricos aos estudantes. As diversas tecnologias, presentes no estado da arte, utilizadas para a elaboração da arquitetura são descritas e comparadas para facilitar o entendimento das escolhas realizadas. Trabalhos correlatos são apresentados para descrever o estado da arte dos laboratórios de experiências remotas de diversos tipos. Com base neste estudo, deficiências e vantagens são apontadas para a criação da arquitetura proposta. Para validar a arquitetura são propostos diversos estudos de caso com implementações utilizadas nos cursos de automação e controle da instituição na qual este trabalho está relacionado, ou ainda em cursos de educação à distância para atingir um público mais amplo. Esta dissertação está relacionada como resultado importante do consórcio internacional RExNet, financiado pelo projeto europeu Alfa, do qual a UFRGS faz parte. / This work aims to the use of remotely web-based experiments to improve the learning process of automation and control systems theory courses. An architecture combining virtual learning environments, remote experiments, students guide and experiments analysis is proposed based on a wide state of art study. The validation of the architecture uses state of art technologies and new simple developed programs to implement the case studies presented. All implementations presented in this work uses an Internet accessible virtual learning environment providing educational resources, guides and learning material to create a distance learning course associated with the remote mixed reality experiment. The experiment is integrated in the learning environment and support the mixture of real equipments combined with simulated ones (virtual). This mixture of equipments (components) is automatic controlled by the user and is a new developed technology called interchangeable components. The state of art section of this work presents, compare and describes several technologies used to implement the proposed architecture. Related work of remote experimentation and virtual learning environments is also presented. The architecture was elaborated based on appointed advantages and drawbacks of the related studies. This work is part of the RExNet consortium, supported by the European Alfa project.

Este trabalho aborda o uso de experimentação remota como forma de aprimorar a educação em sistemas de controle e automação. Para isto, uma arquitetura é proposta, onde experimentos reais podem ser combinados com partes simuladas (virtuais) criando um experimento de realidade mista. Um ambiente de ensino à distância, acessível na Internet, engloba o experimento remoto para proporcionar maiores recursos pedagógicos, guias e materiais educacionais teóricos aos estudantes. As diversas tecnologias, presentes no estado da arte, utilizadas para a elaboração da arquitetura são descritas e comparadas para facilitar o entendimento das escolhas realizadas. Trabalhos correlatos são apresentados para descrever o estado da arte dos laboratórios de experiências remotas de diversos tipos. Com base neste estudo, deficiências e vantagens são apontadas para a criação da arquitetura proposta. Para validar a arquitetura são propostos diversos estudos de caso com implementações utilizadas nos cursos de automação e controle da instituição na qual este trabalho está relacionado, ou ainda em cursos de educação à distância para atingir um público mais amplo. Esta dissertação está relacionada como resultado importante do consórcio internacional RExNet, financiado pelo projeto europeu Alfa, do qual a UFRGS faz parte. / This work aims to the use of remotely web-based experiments to improve the learning process of automation and control systems theory courses. An architecture combining virtual learning environments, remote experiments, students guide and experiments analysis is proposed based on a wide state of art study. The validation of the architecture uses state of art technologies and new simple developed programs to implement the case studies presented. All implementations presented in this work uses an Internet accessible virtual learning environment providing educational resources, guides and learning material to create a distance learning course associated with the remote mixed reality experiment. The experiment is integrated in the learning environment and support the mixture of real equipments combined with simulated ones (virtual). This mixture of equipments (components) is automatic controlled by the user and is a new developed technology called interchangeable components. The state of art section of this work presents, compare and describes several technologies used to implement the proposed architecture. Related work of remote experimentation and virtual learning environments is also presented. The architecture was elaborated based on appointed advantages and drawbacks of the related studies. This work is part of the RExNet consortium, supported by the European Alfa project.

Der Einsatz von immersiven Systemen, also Virtual Reality (VR), Augmented Reality (AR) und Mixed Reality (MR) Systemen in der Lehre ist naheliegend. Eigene interaktive Erfahrung einer Tätigkeit ist immer einer reinen rezeptiven Beobachtung bzw. verbalen Erläuterung vorzuziehen. Trotzdem ist heutige Lehre selbst in Praktika und Übungen zum sehr großen Teil passiv, die selbständige Umsetzung, etwa das Bedienen einer Anlage oder die eigenständige Synthese einer Chemikalie, können aus Gründen der Zeit, Verfügbarkeit, Sicherheitsbedenken und Kostengründen oft nur selten eingesetzt werden. Dem Einsatz o.g. neuer immersiven Technologien stand bisher nicht nur der erhebliche Aufwand zur Erstellung entsprechender Simulationen gegenüber. Vor allem aber auch der Hardwareaufwand bei gleichzeitigem nicht optimalem Grad an Immersivität ließ kaum Möglichkeiten offen. Jeden Studenten einzeln ausreichend Zeit in einer teuren und großen Cave-Umgebung zu ermöglichen, damit dieser virtuell technische Anlagen bedient, ist für größere Studentenzahlen untauglich. [... aus der Einleitung]

Due to their programmability, robots have been used instead of traditional automation for a longtime within the industrial sector. However, programming robots is complex and time-consuming,where for example small errors can have major consequences both materially and for the peopleworking next to the robot. Trying to find simpler ways of programming industrial robots is thus ofhigh relevance. The latest technique for simplifying programming is to use either augmented realityor mixed reality, and today, several studies show that these techniques are possible. However, moreresearch on usability and limitations such tools have is necessary. This study investigates how ageneric holographic tool, which utilizes mixed reality, needs to be improved to be usable for robotprogramming in realistic industrial settings. In this study this is done by investigating limitationsof using a holographic tool and then discuss how these limitations can be reduced or eliminated.To find limitations, a prototype has been designed and developed. The prototype can visualize adigital copy of a physical robot, that can be moved into different positions which in turn createsprogram instructions. The prototype was tested at ABB Robotics with the collaborative single armYuMi robot, and with users that had different amount of experience with similar programmingtools. The results showed several possible limitations, mainly linked to potential safety risks andtechnical limitations. Additionally, some perceived physical impact was found. The most importantimprovement stated were the importance of an easy access to an emergency stop. Finally, theprototype developed during this study does not support all functionality that are included in theprogramming tools used today. Therefore, more studies need to be made to see if the result staysthe same.

The exploration of realistic grasping with industrial robots could open gates in the way of understanding the industry, giving robots the ability to adapt to different situations without reprogramming and therefore, become more flexible tools. In order to facilitate this apparently complex task it is important to implement new tools for robot programming. The project presented shows the development of a program that lets the user give specific instructions about ways of approaching new objects in the task of robot grasping using a mixed reality interface. The proposed solution can let the user work in real time and repeat saved paths with familiar objects. This program consists of recording the user’s movement, saving and processing the information to a robot that can replicate the exact same approaches and grasps. The development of this project as a mixed reality interface permits bringing together the real and virtual world with safety of working with a previsualization of the robot and with the certainty of processing the information right in the real world. This solution is a flexible tool that can be changed to different situations and be implemented in any kind of production.

Dieser Beitrag stellt die Mixed-Reality-in-the-Loop Simulation (MRiLS) zur Schulung technischer Fachkräfte im Maschinen- und Anlagenbau vor. Die MRiLS koppelt die aus dem Engineering bereits vorhandenen Modelle der Hardware-in-the-Loop Simulation (HiLS) mit Visualisierungs- und Interaktionsmethoden der Mixed Reality (MR) und integriert dadurch den Nutzenden und dessen Verhalten sowie die reale Umgebung vollständig in den Simulationskreislauf. Der Beitrag thematisiert neben der notwendigen Middleware zur Kopplung der HiLS mit der MR-Umgebung auch die Steuerungsbelastung durch Multiuser-Zugriffe. Die Funktionsfähigkeit des vorgestellten Konzepts wird anhand eines ausgewählten beispielhaften Automatisierungssystems belegt. Für das Automatisierungssystem wird der Aufbau der MRiLS sowie das Konzept für den Ablauf einer Schulung mittels MRiLS vorgestellt.

Two common problems in Remote Guidance applications include the remote guides lack of direct control over their view into the worker’s physical environment and the difficulties that arise with trying to place virtual 3D objects in a real 3D environment,via a moving, shaky, 2D image.The first issue can be called a lack of remote spatial awareness, the guide can see only what the worker enables them to see. In the worst case the guide is rendered blind to the task environment while the worker is unable to use their device. A common occurrence is tasks that require both hands.The second issue arises from the inherent difficulty present in trying to correctly place a 3D object using only a limited perspective. Camera shake and unreliable tracking of the physical environment being depicted only further add to this problem. Studies show that 3D annotations make for much more effective means of communication, especially in 3D task environments. Allowing the guide some measure of control over their own view has also been shown to improve the guides ability to aid their partner. This paper investigates a method of Remote Guidance where the task of environment tracking and object placement are separated. A prototype application is developed and tested against a baseline 2D-annotation Remote Guidance tool. The study finds the prototype to be an effective way of placing virtual 3D objects in a remote environment. Experimental results show that communication is indeed made better by the inclusion of 3D objects into Remote Guidance. This comes at the cost of a slight increase in the timetaken to complete a task as the complexity of the 3D tool is greater than the 2D one. Unfortunately, the experiment performed fails to properly account for remote spatial awareness.

There are different ways to interact with different hardware, therefore it is important to have an understanding about what factors that affect the experience when designing interactions and interfaces. This study focuses on exploring how auditory and vibrotactile feedback are perceived by the users when they interact in a virtual AR environment. An application was developed to the AR glasses Magic Leap with different interactions, both passive and active. An experimental study was conducted with 28 participants that got to interact in this virtual environment. The study included two parts. First the participants interacted in the virtual environment where they did a think aloud. Thereafter they were interviewed. There were a total of three test cases. One with only auditory feedback, one with vibrotactile feedback, and a third that had both auditory and vibrotactile feedback. Seven of the 28 participants acted as a control group that did not have any feedback to their interactions. The study shows that using only vibrotactile feedback creates different impressions depending on earlier experiences with the same AR environment. Using only auditory feedback created an atmosphere that were close to reality. Having both feedbacks active at the same time reduced the noticed feedback and some interactions were here not even noticed at all. Passive interactions were more noticed than active interactions in all cases. / Det finns flera olika sätt att interagera med olika hårdvaror och därför är det viktigt att ha en förståelse kring vilka faktorer som påverkar upplevelsen när man designar för diverse gränssnitt och interaktioner. Den här studien fokuserar på att utforska hur auditiv och vibrationsåterkoppling uppfattas av användaren när de interagerar i en virtuell AR-miljö. En applikation var utvecklad till AR-glasögonen Magic Leap One med olika aktiva och passiva interaktioner.En experimentell studie genomfördes med 28 deltagare som fick interagera i en virtuell miljö. Studien bestod av av två delar. Först fick deltagarna interagera i en virtuell miljö där de gjorde en think aloud. Efter detta blev de intervjuade. Det var totalt tre testfall, ett hade endast auditiv återkoppling, ett hade vibrationsåterkoppling och det sista hade både auditiv och vibrationsåterkoppling. Sju av de 28 deltagarna agerade kontrollgrupp och de hade ingen återkoppling på deras interaktioner.Studien visade att bara använda vibrationsåterkoppling skapade olika intryck beroende på de tidigare erfarenheterna i samma AR-miljö. Att endast använda auditiv återkoppling skapade en atmosfär som vara nära verkligheten. Att ha båda återkopplingarna aktiva samtidigt reducerade den totala märkta återkopplingen och några interaktioner hade inte någon person som noterade någon av dem. Passiva interaktioner var mer uppmärksammade än aktiva interaktioner i alla testfallen.

The aim of this thesis is to understand the impact of virtual reality (VR) on user's performance and experience in practical industrial applications. In particular, I collaborated with Volvo Cars supporting their current engineering design practices. VR has the potential to augment and thereby improve a person's observational analysis. Current VR publications are focused on VR applications and their potential benefits and improvements, mainly towards science, industry, medicine, and education. Yet, no research on the impact of augmenting the exploration and analysis of water contamination events in a simulated car engine has been done. In this thesis, this issue has been explored by performing a task-based controlled user study with two conditions: the Workstation and the Wall. The Workstation is the current system examining contamination events in a video with keyboard and mouse on a normal desktop screen. The Wall is the developed system examining contamination events through interaction with an Xbox-controller and Kinect head-tracking motion control on a 4K screen (400 x 240 cm projection surface) with stereoscopy using passive 3D glasses. This thesis is a pilot to discover meaningful differences and trade-offs for future studies. The result of the study shows significant differences between the two conditions. Where the Workstation is faster, but less precise, the Wall is more precise and gives more options for analysis. Nevertheless, the current version of the Wall has some weaknesses, in particular, lag and discrete steps in the rotation following the head-tracking, which meant increased duration performing the task. Overall, the high level of interaction using the Wall, the details from the 4K screen, and the stereoscopic rendering were seen as helpful in solving the task. However, the participants were not unanimous on which features were most important or whether or not every feature was essential. Therefore, research, beyond the scope of this thesis, needs to be done to examine how these individual factors affect task performance. / Syftet med detta examensarbete är att studera vilken effekt virtual reality (VR) har på användares prestation och upplevelse när de använder industriella applikationer avsedda för att utföra uppgiften i den verkliga världen. Jag samarbetade med Volvo Cars i att stödja deras nuvarande tekniska designmetod. VR har möjligheten att förstärka och därigenom förbättra analysen av en persons observationer. Nuvarande forskning rörande VR fokuserar på potentiella fördelar och möjliga förbättringar i VR applikationer, riktad mot vetenskap, industri, medicin, utbildning m.m. Emellertid har inte någon forskning gjorts över effekterna av att förstärka sättet att utforska och analysera vattenföroreningshändelser i en simulerad bilmotor. I detta examensarbete har denna fråga studerats genom en uppgiftsbaserad användarstudie med två olika metoder, nämligen Workstation och Wall. Workstation är det system som idag används för att leta efter vattenföroreningshändelser, vilket sker genom en video med tangentbord och mus på en vanlig datorskärm. Wall är ett mer utvecklat system där förekomsten av vattenföroreningshändelser kontrolleras med hjälp av en Xbox-spelkontroll och Kinect "head-tracking" rörelsekontroller på en 4K skärm (400 x 240 cm projektionsyta) med stereoskopi med 3D-glasögon. Examensarbetet är en pilotstudie för att upptäcka skillnader mellan dessa metoder i syfte att hitta infallsvinklar inför framtida studier. Resultatet av studien visar en signifikant skillnad mellan de två metoderna. Workstation är snabbare men mindre exakt medan Wall är mer noggrann och ger fler analysmöjligheter. Den nuvarande versionen av Wall har dock problem med bl.a. fördröjningar som gör att tidsåtgången för att utföra en specifik uppgift förlängs. Sammanfattningsvis ansågs interaktionen i Wall, detaljerna från 4K skärmen och 3D som hjälpsamma för att lösa uppgiften, men deltagarna var inte eniga vilken funktion som var viktigast eller huruvida varje funktion var nödvändig. Ytterligare forskning behöver därför utföras för att utröna vilka enskilda faktorer som har betydelse för deltagarnas prestation.