The ability to visualize and manipulate an airspace is an important tool for an instructor controlling a flight simulator mission. Usually, airspaces are observed and manipulated through 2D and 3D views on a computer screen. A problem with using computer screen is spatial perception, which is significantly limited when observing a 3D space. Technologies such as AR and VR provide new possibilities for presenting and manipulating objects and spaces in 3D which could be used to improve spatial perception. Microsoft's HoloLens is a see-through head mounted display which can project 3D holograms into the real world to create an AR experience. This thesis presents a prototype for displaying a 3D map view using HoloLens which has been designed to improve spatial perception while maintaining high usability. The prototype has been evaluated through heuristic and formative evaluations and was well received by its potential users. The prototype has also been used to present suggestions to improve spatial ability for similar applications.

Este estudo tem como objetivo melhorar o processo de ensino-aprendizagem dos conceitos de Eletromagnetismo usando Realidade Aumentada (RA). Eletromagnetismo é a base científica para a compreensão do princípio de funcionamento das máquinas elétricas rotativas. No entanto, verificou-se através de um estudo preliminar que existem dificuldades no processo de ensino-aprendizagem de Eletromagnetismo e que estas dificuldades estão concentradas basicamente em dois fatores: tanto na visualização da representação correta do campo magnético quanto na base matemática para trabalhar com esses conceitos, os quais permitiriam a aquisição de novos conceitos na área de Eletromagnetismo. O estudo preliminar foi realizado através de questionários aplicados aos alunos e aos professores da disciplina Eletricidade, que aborda os referidos conceitos. O resultado do estudo revelou que 85% dos professores percebem as dificuldades dos alunos em aprender e, confirmada esta percepção pelos próprios alunos, dos quais apenas 39,5% demonstraram ter aprendido estes conceitos. Este trabalho foi baseado na teoria da Aprendizagem Significativa, que, de acordo com Ausubel, ocorre quando um conceito está relacionado de forma significativa e não arbitrária, com conceitos preexistentes na estrutura cognitiva do indivíduo. Na busca de nivelar os alunos para que todos tivessem os subsunçores necessários que permitissem melhorar o processo de ensino-aprendizagem de novos conceitos, os subsunçores de outros conceitos foram definidos e desenvolvidos três Objetos de Aprendizagem. A descoberta dos subsunçores e a ordem do uso dos Objetos de Aprendizagem permitiram melhorar o processo de ensino-aprendizagem de novos conceitos com o objetivo de atingir o conceito principal de campo girante fundamental no processo de ensinoaprendizagem de Máquinas Elétricas. Neste trabalho, foram utilizados três Objetos de Aprendizagem como ferramentas no processo de ensino-aprendizagem de conceitos de Eletromagnetismo para se superar o obstáculo da visualização dos campos magnéticos. Estes Objetos de Aprendizagem foram testados com os alunos dos cursos técnicos do IFF – campus Itaperuna, onde o aluno pode visualizar e interagir com os mesmos. A partir dos resultados obtidos com a interação dos alunos com os Objetos de Aprendizagem foi possível fazer análises e avaliações que permitiram concluir o presente trabalho. / This study aims to improve teaching-learning process of the concepts of Electromagnetism using Augmented Reality (AR). Electromagnetism is the scientific bases for understanding the operating principle of rotating electrical machines. However, it was found through a preliminary study, that there are difficulties in the teaching-learning process of Electromagnetism and these difficulties are concentrated primarily on two factors: the correct view of the representation of the magnetic field and in the mathematical basis to work with these concepts, concepts that allow the acquisition of new concepts in Electromagnetism area. The preliminary study was realized through questionnaires to students and professors of Electricity that addresses these concepts. The study results revealed that 85% of teachers perceive students’ difficulties in learning and confirming this perception by the students, where only 39,5% of these demonstrate to have learned these concepts. This work was based on the Meaningful Learning theory, which, according to Ausubel, occurs when a concept is related significantly and not arbitrary, with pre-existing concepts in cognitive structure of the individual. Aiming to put all students at the same level, there was necessary subsumers for improving the teaching-learning of new concepts, another subsumers about other concepts were defined and were developed three Learning Objects. The discovery of subsumers and the order the use of the Learning Objects allowed to improve the teaching-learning process of new concepts in order to achieve the main concept of the rotating fields, key in the teaching-learning process of Electrical Machines. In this study, there were employed three Learning Objects as tools in the teaching-learning process of Electromagnetism concepts to overcome the obstacle of the visualization of the magnetic fields. These Learning Objects have been tested with students of technical courses of IFF – campus Itaperuna, where students could view and interact with them. From the results, analysis and evaluations were made to complete the work.

This thesis explores a way of increasing the perception of reality within an ''Augmented Reality'' application by making real objects able to obstruct the view of virtual objects. This mimics how real opaque objects occlude each other and thus making virtual objects behave the same way will improve the user experience of Augmented Reality users. The solution uses Unity as the engine with plugins for ARKit and OpenCV. ARKit provides the Augmented Reality experience and can detect real world flat surfaces on which virtual objects can be placed. OpenCV is used for image processing to detect real world objects which can then be translated into virtual silhouettes within Unity that can interact with, and occlude, the virtual objects. The end result is a system that can handle the occlusion in real time, while allowing both the real and virtual objects to translate and rotate within the scene while still maintaining the occlusion. The big drawback of the solution is that it requires a well defined environment without visual clutter and with even lighting to work as intended. This makes it unsuitable for outdoor usage.

abstract: The medical field is constantly looking for technological solutions to reduce user-error and improve procedures. As a potential solution for healthcare environments, Augmented Reality (AR) has received increasing attention in the past few decades due to advances in computing capabilities, lower cost, and better displays (Sauer, Khamene, Bascle, Vogt, & Rubino, 2002). Augmented Reality, as defined in Ronald Azuma’s initial survey of AR, combines virtual and real-world environments in three dimensions and in real-time (Azuma, 1997). Because visualization displays used in AR are related to human physiologic and cognitive constraints, any new system must improve on previous methods and be consistently aligned with human abilities in mind (Drascic & Milgram, 1996; Kruijff, Swan, & Feiner, 2010; Ziv, Wolpe, Small, & Glick, 2006). Based on promising findings from aviation and driving (Liu & Wen, 2004; Sojourner & Antin, 1990; Ververs & Wickens, 1998), this study identifies whether the spatial proximity affordance provided by a head-mounted display or alternative heads up display might benefit to attentional performance in a simulated routine medical task. Additionally, the present study explores how tasks of varying relatedness may relate to attentional performance differences when these tasks are presented at different spatial distances. / Dissertation/Thesis / Masters Thesis Engineering 2017

Orientador: Antonio Carlos Sementille / Banca: Silvio Ricardo Rodrigues Sanches / Banca: Marcos Américo / Resumo: O estúdio virtual é um sistema para a criação de cenas com objetos virtuais integrados digitalmente - e em tempo real - a imagens capturadas em estúdio. Seu emprego flexibiliza a produção audiovisual, permitindo a utilização de objetos e efeitos difíceis de serem recriados fisicamente. Diferente da forma clássica de produção, em que os efeitos são inseridos somente na pós-produção, o estúdio virtual insere o conteúdo digital na fase on set, facilitando direção e fotografia. Considerando este contexto, o objeto deste trabalho é a interação de atores com objetos virtuais utilizando técnicas de Realidade Aumentada em estúdios virtuais. O objetivo principal deste trabalho consiste na verificação da interação de atores com elementos virtuais 3D sob dois aspectos: o primeiro trata da aplicação de técnicas baseadas em interfaces tangíveis associadas ao uso de marcadores fiduciais; e o segundo foca-se na utilização da interação via gestos por meio de dispositivo de detecção de profundidade (Kinect). Para a comprovação do segundo aspecto procedeu-se na implementação, na forma de um protótipo, de um Módulo de Interação para o ambiente do ARSTUDIO, um ambiente de estúdio virtual que está em desenvolvimento na Unesp/Bauru, o qual permite a geração de cenas com Realidade Aumentada e associação de objetos virtuais por meio de marcadores fiduciais. / Abstract: The virtual studio is a system for creating scenes with digitally integrated virtual objects - in real time - to images captured in the studio. Its use eases the audiovisual production, allowing the use of difficult physically rebuilt objects and effects. Unlike the classic production process, in which inserted effects just in post-production, the virtual studio inserts the digital content in the on set phase, facilitating direction and photography. Considering this context, the object of this research is the interaction of actors with virtual objects using Augmented Reality techniques in virtual studios. The main goal is to verify interaction between actors and 3D elements under two aspects: the first is about the application of based tangible interfaces and markers techniques; and the second is about gesture interaction by depth camera device (Kinect). To prove the second aspect it was made the implementation, in prototype, of a Interaction Module for the ARSTUDIO, a virtual studio environment under development in Unesp/Bauru, that allows generating of Augmented Reality scenes and association of virtual objects with markers. / Mestre

Este estudo tem como objetivo melhorar o processo de ensino-aprendizagem dos conceitos de Eletromagnetismo usando Realidade Aumentada (RA). Eletromagnetismo é a base científica para a compreensão do princípio de funcionamento das máquinas elétricas rotativas. No entanto, verificou-se através de um estudo preliminar que existem dificuldades no processo de ensino-aprendizagem de Eletromagnetismo e que estas dificuldades estão concentradas basicamente em dois fatores: tanto na visualização da representação correta do campo magnético quanto na base matemática para trabalhar com esses conceitos, os quais permitiriam a aquisição de novos conceitos na área de Eletromagnetismo. O estudo preliminar foi realizado através de questionários aplicados aos alunos e aos professores da disciplina Eletricidade, que aborda os referidos conceitos. O resultado do estudo revelou que 85% dos professores percebem as dificuldades dos alunos em aprender e, confirmada esta percepção pelos próprios alunos, dos quais apenas 39,5% demonstraram ter aprendido estes conceitos. Este trabalho foi baseado na teoria da Aprendizagem Significativa, que, de acordo com Ausubel, ocorre quando um conceito está relacionado de forma significativa e não arbitrária, com conceitos preexistentes na estrutura cognitiva do indivíduo. Na busca de nivelar os alunos para que todos tivessem os subsunçores necessários que permitissem melhorar o processo de ensino-aprendizagem de novos conceitos, os subsunçores de outros conceitos foram definidos e desenvolvidos três Objetos de Aprendizagem. A descoberta dos subsunçores e a ordem do uso dos Objetos de Aprendizagem permitiram melhorar o processo de ensino-aprendizagem de novos conceitos com o objetivo de atingir o conceito principal de campo girante fundamental no processo de ensinoaprendizagem de Máquinas Elétricas. Neste trabalho, foram utilizados três Objetos de Aprendizagem como ferramentas no processo de ensino-aprendizagem de conceitos de Eletromagnetismo para se superar o obstáculo da visualização dos campos magnéticos. Estes Objetos de Aprendizagem foram testados com os alunos dos cursos técnicos do IFF – campus Itaperuna, onde o aluno pode visualizar e interagir com os mesmos. A partir dos resultados obtidos com a interação dos alunos com os Objetos de Aprendizagem foi possível fazer análises e avaliações que permitiram concluir o presente trabalho. / This study aims to improve teaching-learning process of the concepts of Electromagnetism using Augmented Reality (AR). Electromagnetism is the scientific bases for understanding the operating principle of rotating electrical machines. However, it was found through a preliminary study, that there are difficulties in the teaching-learning process of Electromagnetism and these difficulties are concentrated primarily on two factors: the correct view of the representation of the magnetic field and in the mathematical basis to work with these concepts, concepts that allow the acquisition of new concepts in Electromagnetism area. The preliminary study was realized through questionnaires to students and professors of Electricity that addresses these concepts. The study results revealed that 85% of teachers perceive students’ difficulties in learning and confirming this perception by the students, where only 39,5% of these demonstrate to have learned these concepts. This work was based on the Meaningful Learning theory, which, according to Ausubel, occurs when a concept is related significantly and not arbitrary, with pre-existing concepts in cognitive structure of the individual. Aiming to put all students at the same level, there was necessary subsumers for improving the teaching-learning of new concepts, another subsumers about other concepts were defined and were developed three Learning Objects. The discovery of subsumers and the order the use of the Learning Objects allowed to improve the teaching-learning process of new concepts in order to achieve the main concept of the rotating fields, key in the teaching-learning process of Electrical Machines. In this study, there were employed three Learning Objects as tools in the teaching-learning process of Electromagnetism concepts to overcome the obstacle of the visualization of the magnetic fields. These Learning Objects have been tested with students of technical courses of IFF – campus Itaperuna, where students could view and interact with them. From the results, analysis and evaluations were made to complete the work.

Este estudo tem como objetivo melhorar o processo de ensino-aprendizagem dos conceitos de Eletromagnetismo usando Realidade Aumentada (RA). Eletromagnetismo é a base científica para a compreensão do princípio de funcionamento das máquinas elétricas rotativas. No entanto, verificou-se através de um estudo preliminar que existem dificuldades no processo de ensino-aprendizagem de Eletromagnetismo e que estas dificuldades estão concentradas basicamente em dois fatores: tanto na visualização da representação correta do campo magnético quanto na base matemática para trabalhar com esses conceitos, os quais permitiriam a aquisição de novos conceitos na área de Eletromagnetismo. O estudo preliminar foi realizado através de questionários aplicados aos alunos e aos professores da disciplina Eletricidade, que aborda os referidos conceitos. O resultado do estudo revelou que 85% dos professores percebem as dificuldades dos alunos em aprender e, confirmada esta percepção pelos próprios alunos, dos quais apenas 39,5% demonstraram ter aprendido estes conceitos. Este trabalho foi baseado na teoria da Aprendizagem Significativa, que, de acordo com Ausubel, ocorre quando um conceito está relacionado de forma significativa e não arbitrária, com conceitos preexistentes na estrutura cognitiva do indivíduo. Na busca de nivelar os alunos para que todos tivessem os subsunçores necessários que permitissem melhorar o processo de ensino-aprendizagem de novos conceitos, os subsunçores de outros conceitos foram definidos e desenvolvidos três Objetos de Aprendizagem. A descoberta dos subsunçores e a ordem do uso dos Objetos de Aprendizagem permitiram melhorar o processo de ensino-aprendizagem de novos conceitos com o objetivo de atingir o conceito principal de campo girante fundamental no processo de ensinoaprendizagem de Máquinas Elétricas. Neste trabalho, foram utilizados três Objetos de Aprendizagem como ferramentas no processo de ensino-aprendizagem de conceitos de Eletromagnetismo para se superar o obstáculo da visualização dos campos magnéticos. Estes Objetos de Aprendizagem foram testados com os alunos dos cursos técnicos do IFF – campus Itaperuna, onde o aluno pode visualizar e interagir com os mesmos. A partir dos resultados obtidos com a interação dos alunos com os Objetos de Aprendizagem foi possível fazer análises e avaliações que permitiram concluir o presente trabalho. / This study aims to improve teaching-learning process of the concepts of Electromagnetism using Augmented Reality (AR). Electromagnetism is the scientific bases for understanding the operating principle of rotating electrical machines. However, it was found through a preliminary study, that there are difficulties in the teaching-learning process of Electromagnetism and these difficulties are concentrated primarily on two factors: the correct view of the representation of the magnetic field and in the mathematical basis to work with these concepts, concepts that allow the acquisition of new concepts in Electromagnetism area. The preliminary study was realized through questionnaires to students and professors of Electricity that addresses these concepts. The study results revealed that 85% of teachers perceive students’ difficulties in learning and confirming this perception by the students, where only 39,5% of these demonstrate to have learned these concepts. This work was based on the Meaningful Learning theory, which, according to Ausubel, occurs when a concept is related significantly and not arbitrary, with pre-existing concepts in cognitive structure of the individual. Aiming to put all students at the same level, there was necessary subsumers for improving the teaching-learning of new concepts, another subsumers about other concepts were defined and were developed three Learning Objects. The discovery of subsumers and the order the use of the Learning Objects allowed to improve the teaching-learning process of new concepts in order to achieve the main concept of the rotating fields, key in the teaching-learning process of Electrical Machines. In this study, there were employed three Learning Objects as tools in the teaching-learning process of Electromagnetism concepts to overcome the obstacle of the visualization of the magnetic fields. These Learning Objects have been tested with students of technical courses of IFF – campus Itaperuna, where students could view and interact with them. From the results, analysis and evaluations were made to complete the work.

Monteringsinstruktioner och dokumentation i förstärkt verklighet är en potentiell revolution för industriellt monteringsarbete. Att montören kan få virtuella instruktioner i en verklig kontext, samtidigt som montörens händer hålls fria för montering, skulle bland annat kunna bidra till kortare monteringstider, mer effektiv upplärning, bättre dokumentation och säkrare arbetsplatser. Syftet med examensarbetet som beskrivs i den här rapporten var att studera om och hur Combitech bör använda förstärkt verklighet, för kommunikation av monteringsinstruktioner och dokumentation för komplex montering av kablage till flygplan, samt vilka möjligheter och risker detta medför. I examensarbetet som beskrivs i den här rapporten genomfördes först en litteraturstudie som syftade till att identifiera hur företaget Combitech bör arbeta med förstärkt verklighet vid kabelmontering och dokumentation, vad som har gjorts tidigare inom området samt vilka möjligheter och utmaningar som finns inom området. Efter genomförd litteraturstudie tillämpades resultaten av litteraturstudien på ett verkligt fall. Fallstudien bestod av att implementera och utvärdera prototyper, för monteringsinstruktioner och dokumentation, för komplex montering av kablage till flygplan. Tre olika prototyper skapades och utvärderades i fallstudien, varav en skulle motsvara traditionella monteringsinstruktioner och de två andra skapades för förstärkt verklighet. Den första av två prototyper som skapades för förstärkt verklighet implementerades för Microsoft Hololens och den andra för projicerade monteringsinstruktioner. Utvärderingen genomfördes med sex stycken testpersoner med olika monteringserfarenhet. Resultaten från utvärderingen visade att monteringsinstruktionerna för Microsoft Hololens gav flest antal fel och längst monteringstid samt att de projicerade monteringsinstruktionerna gav minst antal fel och kortast monteringstid. En jämförelse mellan de tre prototyperna som utvärderades visar att de projicerade monteringsinstruktionerna i förstärkt verklighet gav minst antal fel och kortast monteringstid, hade bäst ergonomi, var roligast samt enklast att använda. Valet av metod för monteringsinstruktioner i förstärkt verklighet visade sig vara helt avgörande när resultaten från utvärderingen i fallstudien studerades. I jämförelsen mellan monteringsinstruktionerna i Microsoft Hololens och projicerade monteringsinstruktioner blir rekommendationen enkel, baserat på resultaten från utvärderingen av prototyperna. Väljer Combitech att ta steget till förstärkt verklighet bör de använda sig av projicerade monteringsinstruktioner, för komplex montering av kablage till flygplan i förstärkt verklighet.

Cette thèse a été réalisée dans le cadre d’un projet intitulé Localisation et Réalité Augmentée (LRA). Mettant en relation plusieurs structures industrielles et universitaires, ce projet était appliqué à la fois au domaine ferroviaire et aux voitures autonomes. Elle aborde trois questions principales relatives à la conception des interfaces Humain-Machine dans les voitures autonomes : quelles informations faut-il présenter au conducteur ? Sous quelle forme ? À quel moment ou dans quel contexte ? Elle vise, au travers des réponses apportées à ces questions, à rendre le système « transparent », de façon à susciter une calibration de la confiance appropriée du conducteur en la voiture autonome mais aussi à susciter une expérience utilisateur satisfaisante. Nous nous sommes focalisés en particulier sur la tâche de changement de voie réalisée entièrement réalisée par la voiture autonome. Pour atteindre cet objectif, nous avons déployé une méthodologie comportant cinq étapes. Sur la base du modèle de LYONS (2013), des principes généraux de transparence ont été définis, et ont ensuite été opérationnalisées grâce à la démarche Cognitive Work Analysis. La forme des informations utiles ou potentiellement utiles au conducteur a été définie lors de de séances de créativité et en utilisant la Réalité Augmentée qui est au cœur du projet LRA. Ces informations ont été catégorisées selon les différentes fonctions dont elles résultent (prise d’information, analyse de l’information, prise de décision, implémentation de l’action). Cinq interfaces ont été conçues ; elles présentent des informations relevant de tout ou partie de ces 4 catégories. Elles représentent donc des niveaux de transparence plus ou moins élevés. La validité des principes de transparence a été éprouvée grâce à une expérimentation menée sur simulateur, auprès d’un échantillon de 45 personnes, durant laquelle différents indicateurs des activités cognitives et de l’Expérience Utilisateur ont été mesurés. L’analyse de ces données conduit à mettre en évidence des différences entre les cinq interfaces. L’interface présentant les informations issues des fonctions « prise d’information » et « implémentation de l’action » facilite les activités cognitives du conducteur/ Superviseur. Du point de vue de l’Expérience Utilisateur, c’est l’interface présentant toutes les catégories d’informations qui s’est distinguée des autres. / This doctoral thesis was conducted under the canopy of the Localization and Augmented Reality (LAR) project. The research project was focused on railway yard and autonomous vehicles. The thesis provides answers to three main questions about the Human-Machine interface design in autonomous vehicles: Which information should be conveyed to the human agent? In which form? And when? Answers will enable an appropriate trust calibration of the human agent in the autonomous vehicle and improve driver’s experience by making automation “transparent”. We focus especially on the lane changing task entirely realized by the autonomous vehicle. The aim and the objectives were achieved by a five-steps methodology. Some general principles of transparency have been redefined on the LYONS (2013) model. These principles have been then operationalized by means of Cognitive Work Analysis. Graphical representation of useful information or potentially useful information was defined during creative sessions, by using Augmented Reality that lies at the heart of the LAR project. This information was categorized according to the functions from which it results: information acquisition, information analysis, decision making and action execution. Five interfaces were designed. Each of these interfaces presents information from some of these functions. Therefore, these interfaces corresponded to different transparency configurations more or less high. The validity of transparency principles was tested through an experiment on driving simulator with a sample of 45 participants. In this experiment, some indicators of cognitive activities and User Experience were measured. Data analysis has led to some differences between 5 interfaces. Indeed, the interface with related information to “information acquisition” and “action execution” functions improves the cognitive activities of the human agent. Considering the User Experience, it is the interface with the information related from the 4 functions which provides the best User Experience.

abstract: Underground infrastructure is a critical part of the essential utility services provided to society and the backbone of modern civilization. However, now more than ever before, the disastrous events of a striking underground utilities cost billions of dollars each year in societal damages. Advanced technology and sophisticated visualization techniques such as augmented reality (AR) now play a significant role in mitigating such devastating consequences. Therefore, it is vitally important to coordinate resources, share information, and ensure efficient communication between construction personnel and utility owners. Besides, geographic information systems (GIS) provide a solution for interoperability in the construction industry. Applying such technologies in the field of underground construction requires accurate and up-to-date information. However, there is currently limited research that has integrated AR and GIS and evaluated the effectiveness and usability of the combination in this domain. The main objective of this research was to develop an integrated AR-GIS for mapping and capturing underground utilities using a mobile device. To achieve these objectives, a design research approach utilized to develop and evaluate a mobile extended-reality (XR-GIS) application. This research has produced an efficient solution for data collection and sharing among stakeholders in the underground construction industry. The main challenge in creating a reliable and adaptive outdoor AR system is the accurate registration of virtual objects in the real world. Due to the limited accuracy of smartphones, this study used an external Global Positioning System (GPS) devices to reduce positional error. The primary motivation behind this research is to make the construction industry more aware of the benefits of leveraging AR to prevent utility strikes and enhance public safety. This dissertation fills the gap in the knowledge regarding applying Augmented Reality (AR) in the underground infrastructure mapping. This study’s three research objectives are: (1) Identify the challenges and barriers facing the underground construction industry when applying AR. (2) Develop an integrated AR-GIS for mapping and capturing underground utilities using a mobile device. (3) Evaluate the horizontal accuracy of the captured data used by the AR phone application XR-GIS that has been developed by the author. / Dissertation/Thesis / Doctoral Dissertation Civil, Environmental and Sustainable Engineering 2020