With the advent of ever smaller and more powerful portable computing devices, and ever smaller cameras, wearable computing is becoming more feasible. The ever increasing numbers of augmented reality applications are allowing users to view additional data about their world overlaid on their world using portable computing devices. The main aim of this research is to enable a user of a wearable robot to explore large environments automatically viewing augmented reality at locations and on objects of interest. To implement this research a wearable visual robotic assistant is designed and constructed. Evaluation of the different technologies results in a final design that combines a shoulder mounted self stabilizing active camera, and a hand held magic lens into a single portable system. To enable the wearable assistant to locate known objects, a system is designed that combines an established method for appearance-based recognition with one for simultaneous localization and mapping using a single camera. As well as identifying planar objects, the objects are located relative to the camera in 3D by computing the image-to-database homography. The 3D positions of the objects are then used as additional measurements in the SLAM process, which routinely uses other point features to acquire and maintain a map of the surroundings, irrespective of whether objects are present or not. The monocular SLAM system is then replaced with a new method for building maps and tracking. Instead of tracking and mapping in a linear frame-rate driven manner, this adopted method separates the mapping from the tracking. This allows higher density maps to be constructed, and provides more robust tracking. The flexible framework provided by this method is extended to support multiple independent cameras, and multiple independent maps, allowing the user of the wearable two-camera robot to escape the confines of the desk top and explore arbitrarily sized environments. The final part of the work brings together the parallel tracking and multiple mapping system with the recognition and localization of planar objects from a database. The method is able to build multiple feature rich maps of the world and simultaneously recognize, reconstruct and localize objects within these maps. The object reconstruction process uses the spatially separated keyframes from the tracking and mapping processes to recognize and localize known objects in the world. These are then used for augmented reality overlays related to the objects.

Today our society is greatly built on modern technologies. In most parts of the world itis a social standard to have access to a smartphone and the Internet. Being able to keepin touch with others, consuming information, or being entertained independent of placeand time are assumed standards. From an entertainment perspective, the interactivemedium has changed in the modern society, the gaming community is still growing, andfor many it is a daily routine to play some sort of game, either on the computer, tablet orsmartphone. However, new ways to engage the user in fun activities are still beingexplored. At the same time, the consumption of media and use of digital technologiesraises concerns towards health issues and time spend outside in the nature, according tocritics. Pokemon Go, a smartphone application released in 2016, presented a possiblesolution to this problem, encouraging users to go outside and catch Pokemons due to thelocation-based nature of the application. The need to physically move to certain placesin order to catch a certain Pokemon facilitated not just people’s exercise, but also socialaspects as one would likely meet other players trying to catch Pokemons. While this is astep in the right direction, I strongly believe that this technological advancement couldalso be used for other entertaining and beneficial needs to people. This thesis investigates an approach of using similar technologies compared to thoseused in Pokemon Go, combined with narrative elements and a linear task system, inorder to create an experience with the purpose to engage users with local places ofhistorical value. By using geolocation-based technology, the application guides the userto places of historical value, while augmented reality (AR) elements are used to raisethe user’s interest, and further engage them in an interactive way. The developedapplication allows the user to solve tasks, which require interaction and exploration withthe real world environment in order to gain the necessary information, related to thehistorical value of the place, to solve the task. To provide a more meaningful scenario,all this is accompanied with a complete narrative. Design and concept have beenevaluated by four experts within explorative discussions, which lead to the laterimplementation of a digital prototype. This prototype was then tested and evaluated,providing practical feedback and insights. The user interaction study, conducted withnine participants around the Linnéparken area in Växjö, suggests that the users overallfelt engaged interacting with the application and the real world environment. They alsoseemed to have developed a better understanding about the local history by gaining newinsights about the information and areas presented to them. Users also stated that duringprevious visits to the area they had not payed attention to the statues and otherinformation signs, thus resulting in a new experience visiting the area facilitated throughthe use of the developed application. The users were positive about exploring even morepoints of interest in the future if presented this way, indicating their acceptance towardsthe developed application. Some users also stated that the narrative elements made theexperience interesting and motivating. While both application design and concept wereunderstandable, the applied navigation system would need further improvements,although all users were successfully able to navigate to the real world points of interest.The AR-features seemed to increase the user’s enjoyment, while being useful at thesame time, as the AR-markers provided necessary clues considering where to findanswers to the presented questions as well as a sense of progression with the narrative. / Idag är vårt samhälle starkt byggt på modern teknik. I de flesta delar av världen är deten social standard att ha tillgång till en smartphone och Internet. Att kunna hållakontakten med andra, konsumera information, eller underhållas oberoende av plats ochtid, antas vara normer. Ur ett underhållningsperspektiv har det interaktiva medietförändrats i samhället, spelindustrin växer fortfarande och för många är det en dagligrutin att spela något slags spel, antingen på datorn, surfplattan eller smartphonen. Mennya sätt att engagera användaren i roliga aktiviteter undersöks fortfarande. Samtidigt,enligt kritiker, ger konsumtionen av media och användningen av digital teknikanledning till oro för hälsoproblem samt tid spenderad ute i naturen. Pokemon Go, ensmartphone-applikation som släpptes 2016, presenterade en möjlig lösning på dessaproblem, och uppmuntrade användarna till att gå ut för att fånga Pokemons tack vareapplikationens platsbaserade karaktär. Behovet av att fysiskt flytta sig till olika platserför att fånga Pokemons underlättad inte bara människors motionering, men bidrogockså med sociala aspekter då man sannolikt kommer möta andra spelare ute iverkligheten. Även om detta är ett steg i rätt riktning, tror jag starkt att dessa tekniskaframsteg också kan användas för andra underhållande och fördelaktiga syften. Denna studie undersöker ett sätt att använda liknande tekniker jämfört med de somanvänds i Pokemon Go, kombinerat med narrativ och ett linjärt uppdragssystem, för attskapa en upplevelse med syfte att engagera användare mot lokalhistoriska platser.Genom att använda geolocation-baserad teknik, guidar applikationen användaren tilllokalhistoriska platser, medans elementen inom förstärkt verklighet (AR) används föratt höja användarens intresse och engagera dem på ett interaktivt sätt. Den utveckladeapplikationen låter användaren lösa uppgifter som kräver interaktion och utforskning avden verkliga världsmiljön, för att få den information som krävs för att lösa uppgiften.För att ge ett mer meningsfullt scenario kompletteras detta med ett narrativ. Design ochkoncept har utvärderats av fyra experter genom utforskande diskussioner, vilket senareledde till implementering av en digital prototyp. Denna prototyp testades ochutvärderades mot användare, vilket gav feedback och insikter. Interaktionsstudier medanvändare, vilken genomfördes med nio deltagare vid Linnéparken i Växjö, föreslår attanvändarna i allmänhet kände sig engagerade under interaktion med applikationen ochvärldsmiljön. De verkade också ha utvecklat en bättre förståelse om lokalhistoriagenom att få nya insikter om informationen som presenteras för dem. Användareuppgav också att de under tidigare besök i området inte hade uppmärksammat statyernaeller andra informationsskyltar, vilket resulterade i en ny upplevelse genom att besökaområdet tack vare applikationen. Användarna var positiva till att utforska flerintressanta platser i framtiden om de presenterades på detta sätt, vilket indikerar derasacceptans mot den utvecklade applikationen. Vissa användare uppgav också att denarrativa elementen gjorde upplevelsen intressant och motiverande. Medans bådeapplikationsdesign och koncept var förståeligt skulle det tillämpadenavigationssystemet behöva ytterligare förbättringar, även om användarna lyckadesnavigera till platserna i den verkliga världen. AR-funktionen verkade öka användarensnöje samtidigt som den var användbar eftersom att AR-markörerna tillhandahöllnödvändiga ledtrådar åt vart användaren kunde hitta svaren på presenterade frågor.

The principal aim of this thesis is to assess the state of the mobile marketing as of the first quarter of 2011 and to discuss various scenarios of the future development. This thesis defines the terms "mobile marketing" and "mobile advertising" and identifies the main players in the industry. It explores the main categories of mobile advertising such as mobile messaging, in-content and mobile internet advertising. Later, it analyzes the latest trends in the industry and describes in detail the advancements in QR tagging, location based advertising and augmented reality.

Enligt en rapport från trafikverket förekommer kollisioner vid väghållningsarbeten mellan väghållningsfordon och objekt täckta av väglag. Dessa trafikolyckor kan leda till personskador, skador på infrastruktur och skador på de inblandade väghållnings-fordonen. Som lösning på detta problem har möjligheten att visuellt markera de dolda objekten med förstärkt verklighet undersökts. Olika implementationer och metoder för användning av förstärkt verklighet studerades först för att få en bra upp-fattning om ämnet. En applikation utvecklades därefter till mobiltelefon med opera-tivsystemet Android i utvecklingsmiljön Unity. För förstärkt verklighet i applikat-ionen användes en natural feature tracking implementation av ARCore. Positioner-ingstester utfördes med Mobiltelefonens inbyggda GPS vilket visade en stor felmar-ginal. Applikationen utgår därför från en fast startpunkt inmätt med geodetisk mät-utrustning för ökad positioneringsnoggrannhet. För markering av de dolda objekten användes en lösning med cirkelbaserat skapande av tredimensionella objektmarkörer där cirklarnas radie utgick från startpositionen. Användning av applikationen gav va-rierande resultat beroende på den trafiksituation den används i och avståndet från startpositionen. Objektens position markeras tydligast i situationer där mobiltelefo-nen närmar sig det objekt som ska markeras på en väg i en riktning parallellt med cirkelns radie. Sämre tydlighet uppnåddes i situationer där mobiltelefonen närmar sig på en väg med en riktning parallell med cirkelns tangent eller på öppna ytor likt parkeringsplatser. Positioner för olika testobjekt mättes också med geodetisk mätut-rustning för hög precision för att få tillgång till en testmiljö där pålitligt data kunde hämtas upprepade gånger. Resultaten visar att den implementation av ARCore som finns i applikationen är känslig för avbrott i spårning av omgivningen och kan leda till fel i positionering. Ett fel i avståndsberäkningen finns också i programmet. Där-för är vidareutveckling och mer tester ett krav innan applikationen kan fungera i skarpt läge. / According to a report from Trafikverket collisions occur in road maintenance work with road maintenance vehicles and objects hidden by road conditions. These traffic accidents can lead to personal injury, damage to infrastructure and damage to the road maintenance vehicles involved. As a solution to this problem the possibility of visually marking the hidden objects with the use of augmented reality has been ex-amined. Different implementations and methods for use of augmented reality were first studied to get a good understanding of the subject. An application for mobile telephone was developed with the development platform Unity. The targeted oper-ating system for the mobile telephone was Android. A natural feature tracking imple-mentation of ARCore was used to bring features of augmented reality to the applica-tion. Tests were done to measure the precision of the mobile telephone’s GPS which showed a large margin for error. The application therefore uses a fixed start-ing location which has been measured with geodetic-measuring equipment for in-creased positioning accuracy. To visualize the position of the hidden objects a solu-tion with circle based creation of three-dimensional object markers was used where the radius of the circles originated from the starting position. Use of the application led to varied results which depended on the traffic situation and distance from the start position. The positions of the objects are marked more clearly in situations where the mobile phone is approaching the objects on a road and in a direction par-allel to the circle’s radius. Lower accuracy was generally achieved in situations where the mobile phone was approaching the objects on a road in a direction paral-lel to the circle’s tangent or in open areas like a parking lot. Positions for different test objects were also measured to achieve a test-environment where reliable data could be extracted repeatedly. The results of this study showed that the implemen-tation of ARCore used in this application is sensitive to disruption in the real-world registration of the mobile telephone’s position, which can lead to errors in position-ing. An error in calculating the distance to the objects also exists in the application. Therefore, before the application can be used in a real-life situations further devel-opment and tests are required.

[pt] Nesta dissertação são investigadas situações em que se faz necessária a sobreposição de informações digitais ao ambiente físico em que o usuário está, uma condição proporcionada pela utilização do recurso da Realidade Aumentada (RA). São consideradas, na pesquisa, aplicações da tecnologia que empregam a técnica de Video See-Through (VST) para utilização em celulares. São apresentadas diferentes definições de RA, delimitando-se conceitos tangentes à tecnologia, tais como Realidade Virtual e Computação Ubíqua, para promover uma discussão acerca das definições levantadas. Em seguida, se apresentam algumas soluções, mais encontradas no mercado, de aplicativos de RA para celular, analisando-se os impactos da utilização de ferramentas específicas no seu processo de criação e desenvolvimento. Considerando-se ainda aspectos sociais e filosóficos inerentes à tecnologia, busca-se uma melhor compreensão dos conceitos-base de espaço, tempo e realidade a que a RA recorre para operar, a fim de subsidiar a discussão acerca da tecnologia e criar o embasamento teórico para as análises subsequentes. / [en] This dissertation investigates situations that require the overlay of digital information to the user environment, a condition provided by the use of Augmented Reality (AR) technology. In the research, there are considered applications of the technology that uses the Video See-Through (VST) technique on mobile phones. The dissertation presents different definitions of AR, delimiting tangent concepts such as Virtual Reality and Ubiquitous Computing, to arouse a discussion about the definitions. Then, it presents some of the mobile applications most used in the market, analyzing the impact on using some specific tools in its creation and development process. Considering further social and philosophical aspects inherent to the technology, it seeks to better understand the basic concepts of space, time and reality that the AR uses to operate, in order to support the discussion about the technology and to create the theoretical basis for subsequent analysis.

De nos jours, les images augmentées font partie du quotidien. Du cinéma aux jeux vidéo en passant par l'architecture ou le design, nombreuses sont les applications qui ont besoin d'afficher des objets synthétiques dans un contexte réel. Cependant, le processus permettant d'intégrer ces objets de manière cohérente dans leur environnement peut rapidement devenir très difficile à mettre en œuvre. Lorsque l'environnement à augmenter est de grande taille ou présente une géométrie ou un éclairage complexe, sa modélisation devient alors fastidieuse et l'utilisation de ces modèles dans le rendu d'images augmentées réalistes est très coûteuse en ressources. D'un autre côté, des applications telles que la réalité augmentée ont besoin de méthodes de rendu efficaces pour fonctionner en temps réel. Elles doivent, par ailleurs, pouvoir s'adapter automatiquement à des environnements a priori inconnus avec pour seule source d'informations les images acquises progressivement dans ces derniers. Dans cette thèse, nous nous sommes appuyés sur les méthodes développées en vision par ordinateur, en modélisation à partir d'images et en synthèse d'images pour proposer une approche globale au problème d'augmentation cohérente d'environnements complexes et progressivement découverts. Nous y développons de nouvelles méthodes d'acquisition permettant d'obtenir des images RGB+Z avec une grande dynamique et localisées dans l'environnement. Nous présentons ensuite comment exploiter cette source d'information pour construire incrémentalement des représentations de la géométrie et de l'éclairement de la scène à augmenter. Enfin, nous apportons de nouvelles approches de rendu adaptées à ces modélisations et permettant une génération rapide d'images augmentées où l'éclairement des objets synthétiques reste cohérent avec celui de l'environnement / Today, augmented images are parts of our daily life. From movie industry to video games through architecture and object design, many applications need to display synthetic objects into a real context. However, coherently integrating objects in their environment may be a difficult task. When the environment is vast or includes complex geometry or lighting, its modelling is tedious and using its model to render augmented images is resource-consuming. Moreover, applications like augmented reality need efficient real-time rendering. They also have to automatically adapt to unmodelled environments, while progressively acquiring data from incoming images. In this thesis, we based our work on computer vision, image-based modelling and rendering methods to propose a global approach to the problem of progressively discovered and complex environment coherent augmentation. We first develop new acquisition methods to get high dynamic range RGB+Z registered images of the environment. Then we explain how to use these informations to incrementally build models of scene geometry and lighting. Finally, we provide new rendering approaches using these models and suitable for an efficient and photometrically coherent image augmentation

Over the history of interaction design, concepts and methods of prototyping have been significantly developed to address new challenges faced by the field and the ever-going advancement of new technologies. Some of the important developments are related to the notions of mixed-fidelity prototyping, experience prototyping, and prototypes as filters. Building upon these developments, this study explores a multiple-fidelity prototyping approach, that is, using progressively higher fidelity prototypes in iteration. The study focused particularly on the design of user engagement. It documented, and contributed to, an actual design process of Xplore, an augmented reality game-based learning application. Three participatory design-style workshops employing low-, mixed-, and high-fidelity prototypes were conducted respectively. Thematic analyses of participants’ interactions revealed sets of themes in the workshops, which reflected the changing focus and scope of the design space. The implications of the results obtained in the study highlighted the role of prototypes of each fidelity level in engaging users in a design process, as well as the potential benefits of combining multiple fidelity prototypes when designing for user engagement.

Ces travaux de recherche présentés dans cette thèse s’intègrent dans le cadre d’un partenariat entre Generix Group, éditeur de logiciels collaboratifs pour l’écosystème du commerce, et l’École Centrale de Lille portant sur la réalisation d’un système d’aide au déplacement des opérateurs intégrant la Réalité Augmentée (RA) dans le domaine de la supply chain. Dans la gestion des entrepôts, la préparation des commandes représente un processus important. Avoir une gestion optimisée des entrepôts en aidant les opérateurs à travailler dans des meilleures conditions est un enjeu majeur. Le but de cette thèse est de proposer un Système d’Aide à la Décision (SAD) dans les entrepôts pour l’optimisation des processus de picking et putting. L’aspect dynamique et ouvert du problème nous a conduits à adopter une modélisation multi-agent. Le système multi-agent proposé s’appuie sur les méta heuristiques pour gérer l’affectation aux opérateurs des chemins optimisés de préparation de commandes. Le système d’Alliance entre l’Optimisation et les Systèmes Multi-agent (AOSMA) proposé est basé sur une approche de modélisation, optimisation et simulation orientée agent intégrant la technologie des lunettes à RA. En effet, les lunettes connectées permettent d’afficher d’une manière confortable dans le champ de vision de l’opérateur les informations nécessaires afin d’améliorer l’efficacité et le rendement et de réduire les erreurs de picking et putting. Les résultats expérimentaux présentés dans cette thèse justifient l’alliance entre les Systèmes Multi-Agent et l’optimisation tout en intégrant la nouvelle technologie de RA pour assurer le pilotage des parcours de picking et putting / The research presented in this thesis belongs to a partnership between Generix Group, collaborative software vendor for Retail ecosystem, and the Ecole Centrale of Lille which aims to implement a Support System for Travel (SST) distance of pickers integrating Augmented Reality (AR) in the area of the supply chain. In warehouse management, order picking is an important process. Having an optimized warehouse management by helping order pickers to work in better conditions is a major issues. The aim of this thesis is to propose a Decision Support System (DSS) in warehouses to optimize picking and putting processes. The dynamic and open aspect of the problem has led us to adopt a multi-agent modelling approach. The proposed multi-agent system is based on metaheuristics to manage the optimized paths allocation to order pickers. The Alliance between the Optimization and Multi-Agent System (AOMAS) proposed is based on a modeling approach, optimization and agent-oriented simulation integrating Augmented Reality (AR) Smart Glasses. Indeed, the connected glasses can display in the operator's field of vision the necessary information to improve efficiency and effectiveness and reduce errors in picking and putting. The experimental results presented in this thesis, justify the alliance between the multi-agent systems and optimization integrating the new AR technology to ensure the piloting of picking and putting path.

O uso de dispositivos móveis ou handheld devices na implementação de um sistema de Realidade Aumentada (RA) é denominado Handheld Augmented Reality ou simplesmente Handheld AR. Aplicações do tipo Handheld AR utilizam a câmera integrada no dispositivo móvel para reconhecer marcadores (inseridos no ambiente), o que permite a entrada de dados para o sistema de RA. As informações geradas nestas aplicações são apresentadas como imagens virtuais (gráficos 3D) nas telas dos dispositivos móveis. Já aplicações do tipo Handheld AR mais complexas e recentes, utilizam dispositivos sensores (com interfaces de comunicação sem fio) como RFID tags, Redes de Sensores Sem Fio (RSSF) e sistemas de GPS (Global Positioning System), como fontes de entrada de dados dinâmicos para o sistema de RA, junto à câmera e os marcadores. Aplicações deste tipo apresentam características interessantes para áreas de aplicação como: sensoriamento, monitoramento, visualização e interação em ambientes inteligentes e ubíquos. Porém, a carência de documentação e a complexidade no desenvolvimento destas aplicações geram a necessidade de metodologias e ferramentas que colaborem para um rápido e simples processo de desenvolvimento. Neste sentido, esta dissertação apresenta uma Metodologia de Desenvolvimento para aplicações de realidade aumentada em dispositivos móveis, a qual visa ajudar no projeto e facilitar o processo de desenvolvimento de aplicações do tipo Handheld AR, em especial as que utilizem telefones celulares e dispositivos sensores com interfaces de comunicação sem fio, apresentando uma documentação bem definida e um roteiro claro sobre os requisitos e características no desenvolvimento de aplicações deste tipo. Já a nossa metodologia proposta apresenta um cenário geral de aplicação, uma arquitetura e orientações (guidelines) de desenvolvimento para aplicações do tipo Handheld AR. Além disso, são apresentados detalhes de implementação relativos: à aquisição dos dados gerados pelos dispositivos sensores (neste caso uma RSSF), a uma modelagem semântica destes dados (como parte da solução para facilitar a aquisição, armazenamento e gerenciamento de dados), à criação de perfis de usuário (para prover interfaces de interação intuitivas) e à disponibilização destas informações utilizando técnicas de RA, apresentando as informações geradas como objetos virtuais (imagens virtuais, efeitos de vibração e efeitos sonoros), os quais são sobrepostos em um cenário real utilizando o telefone celular como dispositivo de interação com o usuário. Finalmente, nós apresentamos um estudo de caso que desenvolve uma aplicação do tipo Handheld AR (protótipo) para o monitoramento da temperatura na sala de servidores do PAD (Grupo de Sistemas Pervasivos e de Alto Desempenho). Neste estudo de caso, são descritos os materiais, o cenário, os serviços oferecidos, o fluxo de desenvolvimento e as avaliações do protótipo. / Handheld Augmented Reality or Handheld AR is the implementation of Augmented Reality (AR) systems into handheld devices. Handheld AR applications use the handheld device\'s camera to recognize markers (inserted into the environment). This process enables data entry into AR system. The response generated for these applications are presented as virtual images (3D graphics) on the handheld device\'s display. New Handheld AR applications use a mobile phone camera, markers and wireless sensors (inserted into the environment) as elements for dynamic data entry into the AR system. These applications show interesting characteristics for: sensing, visualization and interaction in ambient intelligence (AmI) and ubiquitous environments. But, the lack of documentation and complexity development process produce the need for methodologies and tools for a fast and simple development model. For these reason, this work presents a Methodology for Design and development Handheld AR applications, which present an especial support for mobile phones and wireless sensor networks, making a description and clear roadmap of the requirements and characteristics in the Handheld AR development process model. Our methodology proposed presents a general scenario of application, a software architecture and application development guidelines. Moreover, we present implementation details over: acquisition data process, semantic modeling data, development of user profiles and visualization of the information using AR techniques, showing these information as virtual objects (virtual images, vibration effects and sound effects). These virtual objects use the handheld device as interaction interface between the user and real world. Finally, we present a case study that develops a Handheld AR application (prototype) to monitoring the temperature in the PAD (Pervasive and Distributed Computing Group) server\'s room. In this case study, we describe the materials, the scenario, the services offered, the development process and prototype evaluations.

La réalité augmentée (RA) consiste en l’insertion d’éléments virtuels dans une scène réelle, observée à travers un écran ou en utilisant un système de projection sur la scène ou l’objet d’intérêt. Les systèmes de réalité augmentée peuvent prendre des différentes formes pour obtenir l’équilibre désiré entre trois critères : précision, latence et robustesse. Il est possible d’identifier trois composants principaux à ces systèmes : localisation, reconstruction et affichage. Les contributions de cette thèse se concentrent essentiellement sur l’affichage et plus particulièrement le rendu des applications de réalité augmentée. À l’opposé des récentes avancées dans le domaine de la localisation et de la reconstruction, l’insertion d’éléments virtuels de façon plausible et esthétique reste une problématique compliquée, mal-posée et peu adaptée à un contexte temps réel. En effet, cette insertion requiert une reconstruction de l’illumination de la scène afin d’appliquer les conditions lumineuses adéquates à l’objet inséré. L’illumination de la scène peut être divisée en plusieurs catégories. Nous pouvons modéliser l’environnement de façon à décrire l’interaction de la lumière incidente et réfléchie pour chaque point 3D d’une surface. Il est également possible d’expliciter l’environnement en calculant la position des sources de lumière, leur type (lampe de bureau, néon, ampoule, ….), leur intensité et leur couleur. Pour insérer un objet de façon cohérente et réaliste, il est primordial d’avoir également une connaissance de la surface recevant l’illumination. Cette interaction lumière/matériaux est dépendante de la géométrie de la surface, de sa composition chimique (matériau) et de sa couleur. Pour tous ces aspects, le problème de reconstruction de l’illumination est difficile, car il est très complexe d’isoler l’illumination sans connaissance a priori de la géométrie, des matériaux de la scène et de la pose de la caméra observant la scène. De manière générale, sur une surface, une source de lumière laisse plusieurs traces telles que les ombres, qui sont créées par l’occultation de rayons lumineux par un objet, et les réflexions spéculaires ou spécularités qui se manifestent par la réflexion partielle ou totale de la lumière. Bien que ces spécularités soient souvent considérées comme des éléments parasites dans les applications de localisation de caméra, de reconstruction ou encore de segmentation, ces éléments donnent des informations cruciales sur la position et la couleur de la source lumineuse, mais également sur la géométrie de la surface et la réflectance du matériau où elle se manifeste. Face à la difficulté du problème de modélisation de la lumière et plus particulièrement du calcul de l’ensemble des paramètres de la lumière, nous nous sommes focalisés, dans cette thèse, sur l’étude des spécularités et sur toutes les informations qu’elles peuvent fournir pour la compréhension de la scène. Plus particulièrement, nous savons qu’une spécularité est définie comme la réflexion d’une source de lumière sur une surface réfléchissante. Partant de cette remarque, nous avons exploré la possibilité de considérer la spécularité comme étant une image issue de la projection d’un objet 3D dans l’espace. Nous sommes partis d’un constat simple, mais peu traité par la littérature qui est que les spécularités présentent une forme elliptique lorsqu’elles apparaissent sur une surface plane. À partir de cette hypothèse, pouvons-nous considérer un objet 3D fixe dans l’espace tel que sa projection perspective dans l’image corresponde à la forme de la spécularité ? Plus particulièrement, nous savons qu’un ellipsoïde projeté perspectivement donne une ellipse. Considérer le phénomène de spécularité comme un phénomène géométrique a de nombreux avantages. (...) / Augmented Reality (AR) consists in inserting virtual elements in a real scene, observed through a screen or a projection system on the scene or the object of interest. The augmented reality systems can take different forms to obtain a balance between three criteria: precision, latency and robustness. It is possible to identify three main components to these systems: localization, reconstruction and display. The contributions of this thesis focus essentially on the display and more particularly the rendering of augmented reality applications. Contrary to the recent advances in the field of localization and reconstruction, the insertion of virtual elements in a plausible and aesthetic way remains a complicated problematic, ill-posed and not adapted to a real-time context. Indeed, this insertion requires a good understanding of the lighting conditions of the scene. The lighting conditions of the scene can be divided in several categories. First, we can model the environment to describe the interaction between the incident and reflected light pour each 3D point of a surface. Secondly, it is also possible to explicitly the environment by computing the position of the light sources, their type (desktop lamps, fluorescent lamp, light bulb, . . . ), their intensities and their colors. Finally, to insert a virtual object in a coherent and realistic way, it is essential to have the knowledge of the surface’s geometry, its chemical composition (material) and its color. For all of these aspects, the reconstruction of the illumination is difficult because it is really complex to isolate the illumination without prior knowledge of the geometry, material of the scene and the camera pose observing the scene. In general, on a surface, a light source leaves several traces such as shadows, created from the occultation of light rays by an object, and the specularities (or specular reflections) which are created by the partial or total reflection of the light. These specularities are often described as very high intensity elements in the image. Although these specularities are often considered as outliers for applications such as camera localization, reconstruction or segmentation, these elements give crucial information on the position and color of the light source but also on the surface’s geometry and the material’s reflectance where these specularities appear. To address the light modeling problem, we focused, in this thesis, on the study of specularities and on every information that they can provide for the understanding of the scene. More specifically, we know that a specularity is defined as the reflection of the light source on a shiny surface. From this statement, we have explored the possibility to consider the specularity as the image created from the projection of a 3D object in space.We started from the simple but little studied in the literature observation that specularities present an elliptic shape when they appear on a planar surface. From this hypothesis, can we consider the existence of a 3D object fixed in space such as its perspective projection in the image fit the shape of the specularity ? We know that an ellipsoid projected perspectivally gives an ellipse. Considering the specularity as a geometric phenomenon presents various advantages. First, the reconstruction of a 3D object and more specifically of an ellipsoid, has been the subject to many publications in the state of the art. Secondly, this modeling allows a great flexibility on the tracking of the state of the specularity and more specifically the light source. Indeed, if the light is turning off, it is easy to visualize in the image if the specularity disappears if we know the contour (and reciprocally of the light is turning on again). (...)