Thesis (Ph.D)--Computing, Georgia Institute of Technology, 2009. / Committee Chair: MacIntyre, Blair; Committee Member: Bolter, Jay; Committee Member: Guzdial, Mark; Committee Member: Mateas, Michael; Committee Member: Mynatt, Elizabeth. Part of the SMARTech Electronic Thesis and Dissertation Collection.

<p>Emerging technologies in the classroom are paving the way towards high-quality, hands-on distance learning. Augmented Reality (AR), which overlays virtual information into the physical world, provides a promising solution for the development and delivery of collaborative educational content. Frameworks such as ARkit, ARCore, have enabled AR experiences to become available to a wider audience. However, there are still several challenges to implementing an AR-based curriculum in classrooms, such as difficulty to create AR content, lack of an architecture capable of supporting collaboration between users, and questions about the user experience. This thesis introduces the MetaAR project, a series of solutions to enable instructors and designers to prototype AR experiences in collaborative and distant classrooms. We designed and tested interactive systems, each targeted towards solving a different problem: (1) MetaAR, an augmented reality authoring platform for instructors and students; (2) RobotAR, a robotics toolkit to create augmented reality-based makerspaces; (3) ColabAR, a toolkit for quick-prototyping of Tangible Augmented Reality (TAR) laboratories; (4) Grove-Blockly, a website with a STEAM curriculum involving IoTs, crafting and coding aimed at middle-schoolers; (5) Towards Modeling of Human Skilling for Electrical</p> <p>Circuitry using Augmented Reality Applications, which provides a model to cluster microskills found in AR (perceptual, cognitive, motor) and aligns them to educational content design for AR. Our preliminary results, obtained from user studies involving more than 120 participants, provide evidence of the sustainability and the positive reception of our prototypes in learning environments. We demonstrated an improvement in several of students’ key competencies and in the overall user experience for both instructors and students. Our hope is that this thesis provides a pathway towards more natural interactions and advances in our understanding of distance learning technology, which is becoming increasingly important in today's society.</p>

Virtual Reality (VR) und Augmented Reality (AR) sind innovative Technologien, die in der modernen Entwicklung, Herstellung und Nutzung von Produkten zum Einsatz kommen. Bisher werden beide Technologien nicht gemeinsam genutzt, obwohl eine Kombination in bestimmten Fällen erhebliches Potenzial zur Kosteneinsparung besitzt. Die VR-Technologie wird vorrangig In der Produktentwicklung eingesetzt um Kosten für physische Prototypen einzusparen. Bei der Montage oder der Wartung komplexer Produkte hingegen kommt die AR-Technologie zum Einsatz. Dabei wird der Servicetechniker durch Arbeitsunterlagen, die über ein Display in sein Sichtfeld projiziert werden, bei seiner Tätigkeit unterstützt. Um die Qualität der Arbeitsunterlagen für AR-Systeme schon während der Produktentwicklung zu sichern und einen Schulungsvorlauf beim Servicepersonal zu erreichen, ist die Evaluierung dieser Arbeitsunterlagen bereits am virtuellen Prototyp eines künftigen Produktes sinnvoll. Mit der Kombination von AR und VR Technologie in einem integrierten System sollen für diesen Ansatz die Voraussetzung geschaffen werden. Der Beitrag beschreibt die notwendigen Grundlagen und stellt die Entwicklung eines Systems vor, welches die Wahrnehmung von AR-Informationen am virtuellen Prototyp ermöglicht. Anhand eines gewählten Wartungsszenarios wird das notwendige Vorgehen zum Erstellen von virtuellen Prototyp und AR-Arbeitsunterlagen erläutert und Gestaltungsparameter beschrieben. Basierend auf diesem Szenario wird das entwickelte System in einer Benutzerstudie getestet und Vorschläge für die weitere Entwicklung abgeleitet.

<div>With rapidly increasing mobile computing devices and high speed networks, large amounts of digital information and intelligence from the surrounding environment have been introduced into our everyday life. However, much of the context and content is in textual and in 2D. To access the digital contents spontaneously, augmented reality~(AR) has become a promising surrogate to bridge the physical with the digital world. Thanks to the vast improvement to the personal computing devices, AR technologies are emerging in realistic scenarios. Commercially available software development kits~(SDKs) and hardware platforms have started to expose AR applications to a large population. </div><div> </div><div>In a broader level, this thesis focuses on investigating suitable interactions metaphors for the evolving AR. In particular, this work leverages the spatial awareness in AR environment to enable spatially-aware interactions. This work explores (i) spatial inputs around AR devices using the local spatial relationship between the AR devices and the scene, (ii) spatial interactions within the surrounding environment exploiting the global spatial relationship among multiple users as well as between the users and the environment. In this work, I mainly study four spatially-aware AR interactions: (i) 3D tangible interactions by directly mapping input to the continuous and discrete volume around the device, (ii) 2D touch input in 3D context by projecting the screen input to the real world, (iii) location aware interactions which use the locations of the real/virtual objects in the AR scene as spatial references, and (iv) collaborative interactions referring to a commonly shared AR scene. This work further develop the enabling techniques including a magnetic sensing based 3D tracking of tangible devices relative to a handheld AR device, a projection based 3D sketching technique for in-situ AR contents creation, a localization method for spatially mapping the smart devices into the AR scene, and a registration approach for resolving the transformations between multiple SLAM AR devices. Moreover, I build systems towards allowing pervasive AR experiences. Primarily, I develop applications for increasing the flexibility of AR contents manipulation, creation and authoring, intuitively interacting with the smart environment, and spontaneously collaborating within a co-located AR scene.</div><div> </div><div>The main body of the research has contributed to multiple on-going collaborative projects. I briefly discuss the key results and visions from these projects including (i) autonomous robotic exploration and mapping of smart environment where the spatial relationship between the robot and the smart devices is resolved, and (ii) human-robot-interaction in AR where the spatial intelligence can be seamlessly exchanged between the human and the robot. Further, I suggest future research projects leveraging three critical features from AR, namely situatedness, mobility, and the capability to support spatial collaborations.</div>

Augmented Reality (AR) is a technology which blends virtual and real worlds. The technology has various potential application domains such as broadcasting, architecture, manufacturing, and entertainment. As the tempting developments in AR technology continues, the solutions for rapid creation of AR applications become crucial. This thesis presents an AR application development framework with scripting capability as a solution for rapid application development and rapid prototyping in AR. The proposed AR framework shares several components with game engines. Thus, the framework is designed as an extension of a game engine. The components of the framework are designed to be changable in a plug-in system. The proposed framework provides the developers with the ability of agile coding through the scripting language. Our solution embeds a dynamic scripting programming language (Python) in a strictly typed static programming language (C++) in order to achieve both agility and performance. The communication between the AR framework components and the scripting programming language is established through a messaging mechanism.

Thesis (M.S.)--Mississippi State University. Department of Computer Science and Engineering. / Title from title screen. Includes bibliographical references.

Dieser Beitrag stellt die Mixed-Reality-in-the-Loop Simulation (MRiLS) zur Schulung technischer Fachkräfte im Maschinen- und Anlagenbau vor. Die MRiLS koppelt die aus dem Engineering bereits vorhandenen Modelle der Hardware-in-the-Loop Simulation (HiLS) mit Visualisierungs- und Interaktionsmethoden der Mixed Reality (MR) und integriert dadurch den Nutzenden und dessen Verhalten sowie die reale Umgebung vollständig in den Simulationskreislauf. Der Beitrag thematisiert neben der notwendigen Middleware zur Kopplung der HiLS mit der MR-Umgebung auch die Steuerungsbelastung durch Multiuser-Zugriffe. Die Funktionsfähigkeit des vorgestellten Konzepts wird anhand eines ausgewählten beispielhaften Automatisierungssystems belegt. Für das Automatisierungssystem wird der Aufbau der MRiLS sowie das Konzept für den Ablauf einer Schulung mittels MRiLS vorgestellt.

No

[pt] O objetivo desta pesquisa é projetar serviços da propedêutica clínica considerando o uso da tecnologia de visualização tridimensional como linguagem e fonte de dados para apoiar o raciocínio das decisões clínicas. Visando projetar esse serviço, avaliamos o potencial de algumas tecnologias de visualização tridimensional que poderiam ser úteis para apoiar a propedêutica: visualização de modelos em ambientes virtuais (imersão em Realidade Virtual ); projeção de modelos tridimensionais na realidade visualizados por smartphones (Realidade Aumentada); modelos impressos em 3D. Optamos por projetar um produto que seria o objeto de investigação aprofundada nesta pesquisa: um ambiente virtual para a visualização de casos clínicos reais por meio de óculos de realidade aumentada. Para realizar esta pesquisa um artefato para a visualização de casos clínicos reais por meio de Realidade Virtual (RV) e Realidade Aumentada (RA) foi projetado. O artefacto foi nomeado como ProVis3D e foi desenvolvido para que os médicos assistentes visualizassem, de forma tridimensional, imagens médicas em posição anatômica, mantendo as relações espaciais entre órgãos e vasos de forma fidedigna, para o que é proposto como precisão pelo método, e correspondendo a sua posição no mundo real, ou seja, como estas estão dispostas no interior do corpo humano. Além de poderem ser visualizadas, há possibilidade de interação com as imagens, simulando a percepção táctil, e de produzir sua movimentação: tocar em órgãos segmentados, vasos e vísceras, separar estas estruturas, aumentar seu tamanho e fazer a rotação do objeto em torno de seu eixo. Uma avaliação empírica foi realizada nesta pesquisa, na perspectiva epistemológica-metodológica projetiva Design Science Research (DSR), utilizando como metodologia projetando diferentes casos, com cinco unidades de análise (cada médico convidado para usar o artefato da pesquisa consiste em uma unidade de análise nesta pesquisa). A abordagem foi observacional e interpretativa, com a finalidade de compreender e refletir sobre o comportamento do médico, especialmente a sua tomada de decisão, quando utiliza o artefato desenvolvido nesta pesquisa. Foram projetadas diferentes cenas para possibilitar a observação das experiências dos usuários do ProVis3D (médicos especialistas e não especialistas) ao visualizar imagens médicas em contextos anatômicos reais (reconstrução 3D a partir de exames de tomografia computadorizada e modelos 3D coloridos com todas as estruturas nas suas posições reais) e ao interagir com essas imagens por meio de Realidade Aumentada. A cena virtual consiste em uma imagem tridimensional dinâmica, que pode ser manipulada para apoiar um cirurgião no seu planejamento cirúrgico ou apoiar um radiologista a complementar seu laudo na estação de trabalho (workstation). Os médicos visualizam a cena inicialmente em RV e posteriormente em RA. Duas questões foram elaboradas para o estudo: 1) Será que o artefato realmente possibilita obter informação de valor para a tomada de decisão clínica?; 2) Será que essa tecnologia de visualização realmente cria uma linguagem que possibilita aos médicos produzirem novos conhecimentos? A partir desse estudo, foi demonstrado que artefato ProVis3D tem potencial para apoiar a decisão clínica e que a tecnologia interativa de visualização tridimensional, em Realidade Aumentada, cria uma linguagem, faz os médicos conversarem de um modo diferente, sobre coisas que não estão habituados, possibilitando aos médicos produzirem novos conhecimentos. / [en] The objective of this research is to design a clinical propaedeutic services considering the use of three-dimensional visualization technology as a language and data source to support the reasoning of clinical decisions. In order to design this service, we evaluated the potential of some three-dimensional visualization technologies that could be useful to support the propaedeutics: visualization of models in virtual environments (immersion in Virtual Reality); projection of three-dimensional models in reality viewed by smartphones (Augmented Reality); 3D printed models. We opt to design a product that should be the object of further investigation in this research: a virtual environment for viewing real clinical cases through augmented reality glasses. In order to accomplish this research, an artifact for visualization of real clinical cases through Virtual Reality (VR) and Augmented Reality (AR) was designed The artifact was named as ProVis3D and was designed for assisting physicians to view medical images in an anatomical position and three-dimensional manner, maintaining spatial relationships between organs and vessels reliably as corresponding to their position in the real world and arranged within the human body. Besides being able to be visualized, there is a possibility for interact with the images, simulating the tactile perception, and producing its movement: touching segmented organs, vessels and viscera, separating these structures, increasing their size and rotating the object around them. its axis. An empirical evaluation was performed in this research, in the projective epistemological-methodological perspective Design Science Research (DSR), designing different cases with five units of analysis (each physician invited to use the research artifact consists of one unit of analysis in this research). The approach was observational and interpretative, with the purpose of understanding and reflecting on the physician s behavior, especially his decision making, when using the artifact developed in this research. Different scenes have been designed in order to observe experiences on ProVis3D s users (medical specialists and non-specialists) when viewing medical images in real anatomical contexts (3D reconstruction from CT scans and full-color 3D models with all structures in place) and interacting with these images through augmented reality. The virtual scene consists of a dynamic three-dimensional image that can be manipulated to support a surgeon on his surgical planning or support a radiologist to complement his workstation report. Doctors view the scene initially in RV and later in RA. Two questions were elaborated for the study: 1) Does the artifact really make it possible to obtain valuable information for clinical decision making? 2) Does this visualization technology really create a language that enables doctors to produce new knowledge? It has been demonstrated that ProVis3D artifact has the potential to support clinical decision making and that the augmented reality interactive three-dimensional visualization technology creates a language; it makes doctors talk differently about things they are not used to, enabling doctors to produce new knowledge.

<p>Развој&nbsp; проширене&nbsp; и&nbsp; виртуелне&nbsp; реалности&nbsp; у&nbsp; последњих неколико година створио је могућности за укључивање и примену ових технологија на свим нивоима образовања. Бројна истраживања истичу да&nbsp; имерзивне&nbsp; технологије&nbsp; могу&nbsp; позитивно&nbsp; утицати&nbsp; на исходе&nbsp; учења&nbsp; и&nbsp; мотивисаност&nbsp; ученика.&nbsp; Такође, географија&nbsp; се&nbsp; често&nbsp; издваја&nbsp; као&nbsp; предмет&nbsp; који посебно&nbsp; може&nbsp; да&nbsp; искористи&nbsp; те&nbsp; потенцијале&nbsp; за унапређење&nbsp; и&nbsp; осавремењавање&nbsp; наставне&nbsp; праксе. Међутим,&nbsp; поставља&nbsp; се&nbsp; питање&nbsp; спремности наставника&nbsp; и&nbsp; студената&nbsp; географије&nbsp; (наставног усмерења)&nbsp; да&nbsp; организују&nbsp; наставу&nbsp; са&nbsp; мобилним&nbsp; и имерзивним&nbsp; технологијама.&nbsp; Притом,&nbsp; неопходно&nbsp; је свеобухватно&nbsp; сагледати&nbsp; позитивне&nbsp; и&nbsp; негативне факторе који утичу или могу утицати на интеграцију ових технологија у географско образовање. Сходно<br />наведеном,&nbsp; ова&nbsp; дисертација&nbsp; разматра&nbsp; како,&nbsp; када&nbsp; и зашто користити проширену и виртуелну реалност у настави и учењу географије.</p> / <p>Razvoj&nbsp; proširene&nbsp; i&nbsp; virtuelne&nbsp; realnosti&nbsp; u&nbsp; poslednjih nekoliko godina stvorio je mogućnosti za uključivanje i primenu ovih tehnologija na svim nivoima obrazovanja. Brojna istraživanja ističu da&nbsp; imerzivne&nbsp; tehnologije&nbsp; mogu&nbsp; pozitivno&nbsp; uticati&nbsp; na ishode&nbsp; učenja&nbsp; i&nbsp; motivisanost&nbsp; učenika.&nbsp; Takođe, geografija&nbsp; se&nbsp; često&nbsp; izdvaja&nbsp; kao&nbsp; predmet&nbsp; koji posebno&nbsp; može&nbsp; da&nbsp; iskoristi&nbsp; te&nbsp; potencijale&nbsp; za unapređenje&nbsp; i&nbsp; osavremenjavanje&nbsp; nastavne&nbsp; prakse. Međutim,&nbsp; postavlja&nbsp; se&nbsp; pitanje&nbsp; spremnosti nastavnika&nbsp; i&nbsp; studenata&nbsp; geografije&nbsp; (nastavnog usmerenja)&nbsp; da&nbsp; organizuju&nbsp; nastavu&nbsp; sa&nbsp; mobilnim&nbsp; i imerzivnim&nbsp; tehnologijama.&nbsp; Pritom,&nbsp; neophodno&nbsp; je sveobuhvatno&nbsp; sagledati&nbsp; pozitivne&nbsp; i&nbsp; negativne faktore koji utiču ili mogu uticati na integraciju ovih tehnologija u geografsko obrazovanje. Shodno<br />navedenom,&nbsp; ova&nbsp; disertacija&nbsp; razmatra&nbsp; kako,&nbsp; kada&nbsp; i zašto koristiti proširenu i virtuelnu realnost u nastavi i učenju geografije.</p> / <p>In&nbsp; recent&nbsp; years,&nbsp; the&nbsp; development&nbsp; of&nbsp; augmented&nbsp; and&nbsp; virtual&nbsp; reality&nbsp; has&nbsp; created&nbsp; opportunities&nbsp; for&nbsp; integration and use of these technologies at all levels of&nbsp; education.Numerous&nbsp; studies&nbsp; showed&nbsp; that&nbsp; immersive&nbsp; technologies have the potential to improve student learning outcomes and&nbsp; motivation.&nbsp; Geography&nbsp; is&nbsp; also&nbsp; often&nbsp; indicated&nbsp; as&nbsp; a subject&nbsp; that&nbsp; can&nbsp; utilize&nbsp; these&nbsp; potentials&nbsp; to&nbsp; improve&nbsp; and modernize&nbsp; teaching&nbsp; practice.&nbsp; However,&nbsp; the&nbsp; integration raises the question of the readiness of geography pre- and in-service&nbsp; teachers&nbsp; to&nbsp; organize&nbsp; classes&nbsp; with&nbsp; mobile&nbsp; and immersive technologies. Also, a comprehensive overlook regarding the positive and negative factors that influence or&nbsp; may influence the integration of these technologies in geographic&nbsp; education&nbsp; is&nbsp; necessary.&nbsp; Accordingly,&nbsp; this dissertation&nbsp; examines&nbsp; how,&nbsp; when,&nbsp; and&nbsp; why&nbsp; to&nbsp; use augmented and virtual reality in geography teaching and&nbsp; learning.</p>