Approved for public release, distribution is unlimited / This thesis addresses the need of Naval Aviation Maintenance to streamline and more effectively manage the process of technical consultation aboard deployed aircraft carriers. The current process involves the physical transportation of an appropriate technician to the carrier to perform required maintenance and/or repairs. In light of the technology currently available this process becomes obviously obsolete, overly costly and needlessly time consuming. By implementing wireless technology in combination with advanced software allowing the virtual collaboration of parties widely separated by geographical distance the Navy can establish a "virtual technical presence" onboard aircraft carriers wherever they may be in the world. This thesis will describe how the fusion of wearable computing, augmented reality, intelligent agents coupled with CoABS, and a modern collaborative software application can revolutionize Naval aviation maintenance as we know it. The technology is there - it only remains for the Navy to leverage it and take advantage of the significant returns that it will provide. The implementation of this technology will allow maintainers onboard deployed aircraft carriers to consult in an augmented virtual environment with technical assets on the shore. These shore-based assets will then be able to "walk" deployed personnel through complicated repair procedures in a matter of minutes or hours as opposed to the previous need to wait for days for the technician to arrive. This is a bold and innovative new concept that will allow commands at sea to increase their levels of combat readiness and allow them the ability to respond to ever changing mission needs. Turn around times for the repair of critical parts and assemblies will be reduced and readiness levels elevated. The ultimate goal of any command is mission accomplishment. This system will aid commands in achieving that all important goal. / Lieutenant, United States Navy

Approved for public release, distribution is unlimited / Current network administrators use network management software to monitor and control elements within a network. This is largely a manual process since managers must interrogate devices individually and evaluate performance statistics manually. The systems provide multiple views on network data but lack capabilities that allow operators to visualize network performance. Since personnel are required to identify problems, interpret potential solutions, and decide on appropriate corrective measures without automatic assistance, maintaining and solving problems for a network can be time-consuming and complex significantly reducing network efficiency. Since FORCENET is a heterogeneous concept that combines various C4I networks, sensors, weapon systems, and platforms, a new model must be developed for network operations. This paper researches an improved model for fleet network operations management for distributed sea-based forces using existing technologies. Combining a collaborative tool, Decision Support System (DSS), and Augmented Reality (AR) imagery transforms Naval information network management from a "minimum threshold" to an "operations fusion" perspective. Little is known about AR technologies, but the potential exists for virtual network operations centers that can remotely direct networks for sea and shore assets through collaborative efforts. The product of this paper will serve as a baseline for network operations in the network centric environment. / Lieutenant, United States Naval Reserve

With the advent of ever smaller and more powerful portable computing devices, and ever smaller cameras, wearable computing is becoming more feasible. The ever increasing numbers of augmented reality applications are allowing users to view additional data about their world overlaid on their world using portable computing devices. The main aim of this research is to enable a user of a wearable robot to explore large environments automatically viewing augmented reality at locations and on objects of interest. To implement this research a wearable visual robotic assistant is designed and constructed. Evaluation of the different technologies results in a final design that combines a shoulder mounted self stabilizing active camera, and a hand held magic lens into a single portable system. To enable the wearable assistant to locate known objects, a system is designed that combines an established method for appearance-based recognition with one for simultaneous localization and mapping using a single camera. As well as identifying planar objects, the objects are located relative to the camera in 3D by computing the image-to-database homography. The 3D positions of the objects are then used as additional measurements in the SLAM process, which routinely uses other point features to acquire and maintain a map of the surroundings, irrespective of whether objects are present or not. The monocular SLAM system is then replaced with a new method for building maps and tracking. Instead of tracking and mapping in a linear frame-rate driven manner, this adopted method separates the mapping from the tracking. This allows higher density maps to be constructed, and provides more robust tracking. The flexible framework provided by this method is extended to support multiple independent cameras, and multiple independent maps, allowing the user of the wearable two-camera robot to escape the confines of the desk top and explore arbitrarily sized environments. The final part of the work brings together the parallel tracking and multiple mapping system with the recognition and localization of planar objects from a database. The method is able to build multiple feature rich maps of the world and simultaneously recognize, reconstruct and localize objects within these maps. The object reconstruction process uses the spatially separated keyframes from the tracking and mapping processes to recognize and localize known objects in the world. These are then used for augmented reality overlays related to the objects.

Today our society is greatly built on modern technologies. In most parts of the world itis a social standard to have access to a smartphone and the Internet. Being able to keepin touch with others, consuming information, or being entertained independent of placeand time are assumed standards. From an entertainment perspective, the interactivemedium has changed in the modern society, the gaming community is still growing, andfor many it is a daily routine to play some sort of game, either on the computer, tablet orsmartphone. However, new ways to engage the user in fun activities are still beingexplored. At the same time, the consumption of media and use of digital technologiesraises concerns towards health issues and time spend outside in the nature, according tocritics. Pokemon Go, a smartphone application released in 2016, presented a possiblesolution to this problem, encouraging users to go outside and catch Pokemons due to thelocation-based nature of the application. The need to physically move to certain placesin order to catch a certain Pokemon facilitated not just people’s exercise, but also socialaspects as one would likely meet other players trying to catch Pokemons. While this is astep in the right direction, I strongly believe that this technological advancement couldalso be used for other entertaining and beneficial needs to people. This thesis investigates an approach of using similar technologies compared to thoseused in Pokemon Go, combined with narrative elements and a linear task system, inorder to create an experience with the purpose to engage users with local places ofhistorical value. By using geolocation-based technology, the application guides the userto places of historical value, while augmented reality (AR) elements are used to raisethe user’s interest, and further engage them in an interactive way. The developedapplication allows the user to solve tasks, which require interaction and exploration withthe real world environment in order to gain the necessary information, related to thehistorical value of the place, to solve the task. To provide a more meaningful scenario,all this is accompanied with a complete narrative. Design and concept have beenevaluated by four experts within explorative discussions, which lead to the laterimplementation of a digital prototype. This prototype was then tested and evaluated,providing practical feedback and insights. The user interaction study, conducted withnine participants around the Linnéparken area in Växjö, suggests that the users overallfelt engaged interacting with the application and the real world environment. They alsoseemed to have developed a better understanding about the local history by gaining newinsights about the information and areas presented to them. Users also stated that duringprevious visits to the area they had not payed attention to the statues and otherinformation signs, thus resulting in a new experience visiting the area facilitated throughthe use of the developed application. The users were positive about exploring even morepoints of interest in the future if presented this way, indicating their acceptance towardsthe developed application. Some users also stated that the narrative elements made theexperience interesting and motivating. While both application design and concept wereunderstandable, the applied navigation system would need further improvements,although all users were successfully able to navigate to the real world points of interest.The AR-features seemed to increase the user’s enjoyment, while being useful at thesame time, as the AR-markers provided necessary clues considering where to findanswers to the presented questions as well as a sense of progression with the narrative. / Idag är vårt samhälle starkt byggt på modern teknik. I de flesta delar av världen är deten social standard att ha tillgång till en smartphone och Internet. Att kunna hållakontakten med andra, konsumera information, eller underhållas oberoende av plats ochtid, antas vara normer. Ur ett underhållningsperspektiv har det interaktiva medietförändrats i samhället, spelindustrin växer fortfarande och för många är det en dagligrutin att spela något slags spel, antingen på datorn, surfplattan eller smartphonen. Mennya sätt att engagera användaren i roliga aktiviteter undersöks fortfarande. Samtidigt,enligt kritiker, ger konsumtionen av media och användningen av digital teknikanledning till oro för hälsoproblem samt tid spenderad ute i naturen. Pokemon Go, ensmartphone-applikation som släpptes 2016, presenterade en möjlig lösning på dessaproblem, och uppmuntrade användarna till att gå ut för att fånga Pokemons tack vareapplikationens platsbaserade karaktär. Behovet av att fysiskt flytta sig till olika platserför att fånga Pokemons underlättad inte bara människors motionering, men bidrogockså med sociala aspekter då man sannolikt kommer möta andra spelare ute iverkligheten. Även om detta är ett steg i rätt riktning, tror jag starkt att dessa tekniskaframsteg också kan användas för andra underhållande och fördelaktiga syften. Denna studie undersöker ett sätt att använda liknande tekniker jämfört med de somanvänds i Pokemon Go, kombinerat med narrativ och ett linjärt uppdragssystem, för attskapa en upplevelse med syfte att engagera användare mot lokalhistoriska platser.Genom att använda geolocation-baserad teknik, guidar applikationen användaren tilllokalhistoriska platser, medans elementen inom förstärkt verklighet (AR) används föratt höja användarens intresse och engagera dem på ett interaktivt sätt. Den utveckladeapplikationen låter användaren lösa uppgifter som kräver interaktion och utforskning avden verkliga världsmiljön, för att få den information som krävs för att lösa uppgiften.För att ge ett mer meningsfullt scenario kompletteras detta med ett narrativ. Design ochkoncept har utvärderats av fyra experter genom utforskande diskussioner, vilket senareledde till implementering av en digital prototyp. Denna prototyp testades ochutvärderades mot användare, vilket gav feedback och insikter. Interaktionsstudier medanvändare, vilken genomfördes med nio deltagare vid Linnéparken i Växjö, föreslår attanvändarna i allmänhet kände sig engagerade under interaktion med applikationen ochvärldsmiljön. De verkade också ha utvecklat en bättre förståelse om lokalhistoriagenom att få nya insikter om informationen som presenteras för dem. Användareuppgav också att de under tidigare besök i området inte hade uppmärksammat statyernaeller andra informationsskyltar, vilket resulterade i en ny upplevelse genom att besökaområdet tack vare applikationen. Användarna var positiva till att utforska flerintressanta platser i framtiden om de presenterades på detta sätt, vilket indikerar derasacceptans mot den utvecklade applikationen. Vissa användare uppgav också att denarrativa elementen gjorde upplevelsen intressant och motiverande. Medans bådeapplikationsdesign och koncept var förståeligt skulle det tillämpadenavigationssystemet behöva ytterligare förbättringar, även om användarna lyckadesnavigera till platserna i den verkliga världen. AR-funktionen verkade öka användarensnöje samtidigt som den var användbar eftersom att AR-markörerna tillhandahöllnödvändiga ledtrådar åt vart användaren kunde hitta svaren på presenterade frågor.

The principal aim of this thesis is to assess the state of the mobile marketing as of the first quarter of 2011 and to discuss various scenarios of the future development. This thesis defines the terms "mobile marketing" and "mobile advertising" and identifies the main players in the industry. It explores the main categories of mobile advertising such as mobile messaging, in-content and mobile internet advertising. Later, it analyzes the latest trends in the industry and describes in detail the advancements in QR tagging, location based advertising and augmented reality.

Enligt en rapport från trafikverket förekommer kollisioner vid väghållningsarbeten mellan väghållningsfordon och objekt täckta av väglag. Dessa trafikolyckor kan leda till personskador, skador på infrastruktur och skador på de inblandade väghållnings-fordonen. Som lösning på detta problem har möjligheten att visuellt markera de dolda objekten med förstärkt verklighet undersökts. Olika implementationer och metoder för användning av förstärkt verklighet studerades först för att få en bra upp-fattning om ämnet. En applikation utvecklades därefter till mobiltelefon med opera-tivsystemet Android i utvecklingsmiljön Unity. För förstärkt verklighet i applikat-ionen användes en natural feature tracking implementation av ARCore. Positioner-ingstester utfördes med Mobiltelefonens inbyggda GPS vilket visade en stor felmar-ginal. Applikationen utgår därför från en fast startpunkt inmätt med geodetisk mät-utrustning för ökad positioneringsnoggrannhet. För markering av de dolda objekten användes en lösning med cirkelbaserat skapande av tredimensionella objektmarkörer där cirklarnas radie utgick från startpositionen. Användning av applikationen gav va-rierande resultat beroende på den trafiksituation den används i och avståndet från startpositionen. Objektens position markeras tydligast i situationer där mobiltelefo-nen närmar sig det objekt som ska markeras på en väg i en riktning parallellt med cirkelns radie. Sämre tydlighet uppnåddes i situationer där mobiltelefonen närmar sig på en väg med en riktning parallell med cirkelns tangent eller på öppna ytor likt parkeringsplatser. Positioner för olika testobjekt mättes också med geodetisk mätut-rustning för hög precision för att få tillgång till en testmiljö där pålitligt data kunde hämtas upprepade gånger. Resultaten visar att den implementation av ARCore som finns i applikationen är känslig för avbrott i spårning av omgivningen och kan leda till fel i positionering. Ett fel i avståndsberäkningen finns också i programmet. Där-för är vidareutveckling och mer tester ett krav innan applikationen kan fungera i skarpt läge. / According to a report from Trafikverket collisions occur in road maintenance work with road maintenance vehicles and objects hidden by road conditions. These traffic accidents can lead to personal injury, damage to infrastructure and damage to the road maintenance vehicles involved. As a solution to this problem the possibility of visually marking the hidden objects with the use of augmented reality has been ex-amined. Different implementations and methods for use of augmented reality were first studied to get a good understanding of the subject. An application for mobile telephone was developed with the development platform Unity. The targeted oper-ating system for the mobile telephone was Android. A natural feature tracking imple-mentation of ARCore was used to bring features of augmented reality to the applica-tion. Tests were done to measure the precision of the mobile telephone’s GPS which showed a large margin for error. The application therefore uses a fixed start-ing location which has been measured with geodetic-measuring equipment for in-creased positioning accuracy. To visualize the position of the hidden objects a solu-tion with circle based creation of three-dimensional object markers was used where the radius of the circles originated from the starting position. Use of the application led to varied results which depended on the traffic situation and distance from the start position. The positions of the objects are marked more clearly in situations where the mobile phone is approaching the objects on a road and in a direction par-allel to the circle’s radius. Lower accuracy was generally achieved in situations where the mobile phone was approaching the objects on a road in a direction paral-lel to the circle’s tangent or in open areas like a parking lot. Positions for different test objects were also measured to achieve a test-environment where reliable data could be extracted repeatedly. The results of this study showed that the implemen-tation of ARCore used in this application is sensitive to disruption in the real-world registration of the mobile telephone’s position, which can lead to errors in position-ing. An error in calculating the distance to the objects also exists in the application. Therefore, before the application can be used in a real-life situations further devel-opment and tests are required.

[pt] Nesta dissertação são investigadas situações em que se faz necessária a sobreposição de informações digitais ao ambiente físico em que o usuário está, uma condição proporcionada pela utilização do recurso da Realidade Aumentada (RA). São consideradas, na pesquisa, aplicações da tecnologia que empregam a técnica de Video See-Through (VST) para utilização em celulares. São apresentadas diferentes definições de RA, delimitando-se conceitos tangentes à tecnologia, tais como Realidade Virtual e Computação Ubíqua, para promover uma discussão acerca das definições levantadas. Em seguida, se apresentam algumas soluções, mais encontradas no mercado, de aplicativos de RA para celular, analisando-se os impactos da utilização de ferramentas específicas no seu processo de criação e desenvolvimento. Considerando-se ainda aspectos sociais e filosóficos inerentes à tecnologia, busca-se uma melhor compreensão dos conceitos-base de espaço, tempo e realidade a que a RA recorre para operar, a fim de subsidiar a discussão acerca da tecnologia e criar o embasamento teórico para as análises subsequentes. / [en] This dissertation investigates situations that require the overlay of digital information to the user environment, a condition provided by the use of Augmented Reality (AR) technology. In the research, there are considered applications of the technology that uses the Video See-Through (VST) technique on mobile phones. The dissertation presents different definitions of AR, delimiting tangent concepts such as Virtual Reality and Ubiquitous Computing, to arouse a discussion about the definitions. Then, it presents some of the mobile applications most used in the market, analyzing the impact on using some specific tools in its creation and development process. Considering further social and philosophical aspects inherent to the technology, it seeks to better understand the basic concepts of space, time and reality that the AR uses to operate, in order to support the discussion about the technology and to create the theoretical basis for subsequent analysis.

De nos jours, les images augmentées font partie du quotidien. Du cinéma aux jeux vidéo en passant par l'architecture ou le design, nombreuses sont les applications qui ont besoin d'afficher des objets synthétiques dans un contexte réel. Cependant, le processus permettant d'intégrer ces objets de manière cohérente dans leur environnement peut rapidement devenir très difficile à mettre en œuvre. Lorsque l'environnement à augmenter est de grande taille ou présente une géométrie ou un éclairage complexe, sa modélisation devient alors fastidieuse et l'utilisation de ces modèles dans le rendu d'images augmentées réalistes est très coûteuse en ressources. D'un autre côté, des applications telles que la réalité augmentée ont besoin de méthodes de rendu efficaces pour fonctionner en temps réel. Elles doivent, par ailleurs, pouvoir s'adapter automatiquement à des environnements a priori inconnus avec pour seule source d'informations les images acquises progressivement dans ces derniers. Dans cette thèse, nous nous sommes appuyés sur les méthodes développées en vision par ordinateur, en modélisation à partir d'images et en synthèse d'images pour proposer une approche globale au problème d'augmentation cohérente d'environnements complexes et progressivement découverts. Nous y développons de nouvelles méthodes d'acquisition permettant d'obtenir des images RGB+Z avec une grande dynamique et localisées dans l'environnement. Nous présentons ensuite comment exploiter cette source d'information pour construire incrémentalement des représentations de la géométrie et de l'éclairement de la scène à augmenter. Enfin, nous apportons de nouvelles approches de rendu adaptées à ces modélisations et permettant une génération rapide d'images augmentées où l'éclairement des objets synthétiques reste cohérent avec celui de l'environnement / Today, augmented images are parts of our daily life. From movie industry to video games through architecture and object design, many applications need to display synthetic objects into a real context. However, coherently integrating objects in their environment may be a difficult task. When the environment is vast or includes complex geometry or lighting, its modelling is tedious and using its model to render augmented images is resource-consuming. Moreover, applications like augmented reality need efficient real-time rendering. They also have to automatically adapt to unmodelled environments, while progressively acquiring data from incoming images. In this thesis, we based our work on computer vision, image-based modelling and rendering methods to propose a global approach to the problem of progressively discovered and complex environment coherent augmentation. We first develop new acquisition methods to get high dynamic range RGB+Z registered images of the environment. Then we explain how to use these informations to incrementally build models of scene geometry and lighting. Finally, we provide new rendering approaches using these models and suitable for an efficient and photometrically coherent image augmentation

Over the history of interaction design, concepts and methods of prototyping have been significantly developed to address new challenges faced by the field and the ever-going advancement of new technologies. Some of the important developments are related to the notions of mixed-fidelity prototyping, experience prototyping, and prototypes as filters. Building upon these developments, this study explores a multiple-fidelity prototyping approach, that is, using progressively higher fidelity prototypes in iteration. The study focused particularly on the design of user engagement. It documented, and contributed to, an actual design process of Xplore, an augmented reality game-based learning application. Three participatory design-style workshops employing low-, mixed-, and high-fidelity prototypes were conducted respectively. Thematic analyses of participants’ interactions revealed sets of themes in the workshops, which reflected the changing focus and scope of the design space. The implications of the results obtained in the study highlighted the role of prototypes of each fidelity level in engaging users in a design process, as well as the potential benefits of combining multiple fidelity prototypes when designing for user engagement.

Ces travaux de recherche présentés dans cette thèse s’intègrent dans le cadre d’un partenariat entre Generix Group, éditeur de logiciels collaboratifs pour l’écosystème du commerce, et l’École Centrale de Lille portant sur la réalisation d’un système d’aide au déplacement des opérateurs intégrant la Réalité Augmentée (RA) dans le domaine de la supply chain. Dans la gestion des entrepôts, la préparation des commandes représente un processus important. Avoir une gestion optimisée des entrepôts en aidant les opérateurs à travailler dans des meilleures conditions est un enjeu majeur. Le but de cette thèse est de proposer un Système d’Aide à la Décision (SAD) dans les entrepôts pour l’optimisation des processus de picking et putting. L’aspect dynamique et ouvert du problème nous a conduits à adopter une modélisation multi-agent. Le système multi-agent proposé s’appuie sur les méta heuristiques pour gérer l’affectation aux opérateurs des chemins optimisés de préparation de commandes. Le système d’Alliance entre l’Optimisation et les Systèmes Multi-agent (AOSMA) proposé est basé sur une approche de modélisation, optimisation et simulation orientée agent intégrant la technologie des lunettes à RA. En effet, les lunettes connectées permettent d’afficher d’une manière confortable dans le champ de vision de l’opérateur les informations nécessaires afin d’améliorer l’efficacité et le rendement et de réduire les erreurs de picking et putting. Les résultats expérimentaux présentés dans cette thèse justifient l’alliance entre les Systèmes Multi-Agent et l’optimisation tout en intégrant la nouvelle technologie de RA pour assurer le pilotage des parcours de picking et putting / The research presented in this thesis belongs to a partnership between Generix Group, collaborative software vendor for Retail ecosystem, and the Ecole Centrale of Lille which aims to implement a Support System for Travel (SST) distance of pickers integrating Augmented Reality (AR) in the area of the supply chain. In warehouse management, order picking is an important process. Having an optimized warehouse management by helping order pickers to work in better conditions is a major issues. The aim of this thesis is to propose a Decision Support System (DSS) in warehouses to optimize picking and putting processes. The dynamic and open aspect of the problem has led us to adopt a multi-agent modelling approach. The proposed multi-agent system is based on metaheuristics to manage the optimized paths allocation to order pickers. The Alliance between the Optimization and Multi-Agent System (AOMAS) proposed is based on a modeling approach, optimization and agent-oriented simulation integrating Augmented Reality (AR) Smart Glasses. Indeed, the connected glasses can display in the operator's field of vision the necessary information to improve efficiency and effectiveness and reduce errors in picking and putting. The experimental results presented in this thesis, justify the alliance between the multi-agent systems and optimization integrating the new AR technology to ensure the piloting of picking and putting path.