Ces travaux de recherche présentés dans cette thèse s’intègrent dans le cadre d’un partenariat entre Generix Group, éditeur de logiciels collaboratifs pour l’écosystème du commerce, et l’École Centrale de Lille portant sur la réalisation d’un système d’aide au déplacement des opérateurs intégrant la Réalité Augmentée (RA) dans le domaine de la supply chain. Dans la gestion des entrepôts, la préparation des commandes représente un processus important. Avoir une gestion optimisée des entrepôts en aidant les opérateurs à travailler dans des meilleures conditions est un enjeu majeur. Le but de cette thèse est de proposer un Système d’Aide à la Décision (SAD) dans les entrepôts pour l’optimisation des processus de picking et putting. L’aspect dynamique et ouvert du problème nous a conduits à adopter une modélisation multi-agent. Le système multi-agent proposé s’appuie sur les méta heuristiques pour gérer l’affectation aux opérateurs des chemins optimisés de préparation de commandes. Le système d’Alliance entre l’Optimisation et les Systèmes Multi-agent (AOSMA) proposé est basé sur une approche de modélisation, optimisation et simulation orientée agent intégrant la technologie des lunettes à RA. En effet, les lunettes connectées permettent d’afficher d’une manière confortable dans le champ de vision de l’opérateur les informations nécessaires afin d’améliorer l’efficacité et le rendement et de réduire les erreurs de picking et putting. Les résultats expérimentaux présentés dans cette thèse justifient l’alliance entre les Systèmes Multi-Agent et l’optimisation tout en intégrant la nouvelle technologie de RA pour assurer le pilotage des parcours de picking et putting / The research presented in this thesis belongs to a partnership between Generix Group, collaborative software vendor for Retail ecosystem, and the Ecole Centrale of Lille which aims to implement a Support System for Travel (SST) distance of pickers integrating Augmented Reality (AR) in the area of the supply chain. In warehouse management, order picking is an important process. Having an optimized warehouse management by helping order pickers to work in better conditions is a major issues. The aim of this thesis is to propose a Decision Support System (DSS) in warehouses to optimize picking and putting processes. The dynamic and open aspect of the problem has led us to adopt a multi-agent modelling approach. The proposed multi-agent system is based on metaheuristics to manage the optimized paths allocation to order pickers. The Alliance between the Optimization and Multi-Agent System (AOMAS) proposed is based on a modeling approach, optimization and agent-oriented simulation integrating Augmented Reality (AR) Smart Glasses. Indeed, the connected glasses can display in the operator's field of vision the necessary information to improve efficiency and effectiveness and reduce errors in picking and putting. The experimental results presented in this thesis, justify the alliance between the multi-agent systems and optimization integrating the new AR technology to ensure the piloting of picking and putting path.

O uso de dispositivos móveis ou handheld devices na implementação de um sistema de Realidade Aumentada (RA) é denominado Handheld Augmented Reality ou simplesmente Handheld AR. Aplicações do tipo Handheld AR utilizam a câmera integrada no dispositivo móvel para reconhecer marcadores (inseridos no ambiente), o que permite a entrada de dados para o sistema de RA. As informações geradas nestas aplicações são apresentadas como imagens virtuais (gráficos 3D) nas telas dos dispositivos móveis. Já aplicações do tipo Handheld AR mais complexas e recentes, utilizam dispositivos sensores (com interfaces de comunicação sem fio) como RFID tags, Redes de Sensores Sem Fio (RSSF) e sistemas de GPS (Global Positioning System), como fontes de entrada de dados dinâmicos para o sistema de RA, junto à câmera e os marcadores. Aplicações deste tipo apresentam características interessantes para áreas de aplicação como: sensoriamento, monitoramento, visualização e interação em ambientes inteligentes e ubíquos. Porém, a carência de documentação e a complexidade no desenvolvimento destas aplicações geram a necessidade de metodologias e ferramentas que colaborem para um rápido e simples processo de desenvolvimento. Neste sentido, esta dissertação apresenta uma Metodologia de Desenvolvimento para aplicações de realidade aumentada em dispositivos móveis, a qual visa ajudar no projeto e facilitar o processo de desenvolvimento de aplicações do tipo Handheld AR, em especial as que utilizem telefones celulares e dispositivos sensores com interfaces de comunicação sem fio, apresentando uma documentação bem definida e um roteiro claro sobre os requisitos e características no desenvolvimento de aplicações deste tipo. Já a nossa metodologia proposta apresenta um cenário geral de aplicação, uma arquitetura e orientações (guidelines) de desenvolvimento para aplicações do tipo Handheld AR. Além disso, são apresentados detalhes de implementação relativos: à aquisição dos dados gerados pelos dispositivos sensores (neste caso uma RSSF), a uma modelagem semântica destes dados (como parte da solução para facilitar a aquisição, armazenamento e gerenciamento de dados), à criação de perfis de usuário (para prover interfaces de interação intuitivas) e à disponibilização destas informações utilizando técnicas de RA, apresentando as informações geradas como objetos virtuais (imagens virtuais, efeitos de vibração e efeitos sonoros), os quais são sobrepostos em um cenário real utilizando o telefone celular como dispositivo de interação com o usuário. Finalmente, nós apresentamos um estudo de caso que desenvolve uma aplicação do tipo Handheld AR (protótipo) para o monitoramento da temperatura na sala de servidores do PAD (Grupo de Sistemas Pervasivos e de Alto Desempenho). Neste estudo de caso, são descritos os materiais, o cenário, os serviços oferecidos, o fluxo de desenvolvimento e as avaliações do protótipo. / Handheld Augmented Reality or Handheld AR is the implementation of Augmented Reality (AR) systems into handheld devices. Handheld AR applications use the handheld device\'s camera to recognize markers (inserted into the environment). This process enables data entry into AR system. The response generated for these applications are presented as virtual images (3D graphics) on the handheld device\'s display. New Handheld AR applications use a mobile phone camera, markers and wireless sensors (inserted into the environment) as elements for dynamic data entry into the AR system. These applications show interesting characteristics for: sensing, visualization and interaction in ambient intelligence (AmI) and ubiquitous environments. But, the lack of documentation and complexity development process produce the need for methodologies and tools for a fast and simple development model. For these reason, this work presents a Methodology for Design and development Handheld AR applications, which present an especial support for mobile phones and wireless sensor networks, making a description and clear roadmap of the requirements and characteristics in the Handheld AR development process model. Our methodology proposed presents a general scenario of application, a software architecture and application development guidelines. Moreover, we present implementation details over: acquisition data process, semantic modeling data, development of user profiles and visualization of the information using AR techniques, showing these information as virtual objects (virtual images, vibration effects and sound effects). These virtual objects use the handheld device as interaction interface between the user and real world. Finally, we present a case study that develops a Handheld AR application (prototype) to monitoring the temperature in the PAD (Pervasive and Distributed Computing Group) server\'s room. In this case study, we describe the materials, the scenario, the services offered, the development process and prototype evaluations.

La réalité augmentée (RA) consiste en l’insertion d’éléments virtuels dans une scène réelle, observée à travers un écran ou en utilisant un système de projection sur la scène ou l’objet d’intérêt. Les systèmes de réalité augmentée peuvent prendre des différentes formes pour obtenir l’équilibre désiré entre trois critères : précision, latence et robustesse. Il est possible d’identifier trois composants principaux à ces systèmes : localisation, reconstruction et affichage. Les contributions de cette thèse se concentrent essentiellement sur l’affichage et plus particulièrement le rendu des applications de réalité augmentée. À l’opposé des récentes avancées dans le domaine de la localisation et de la reconstruction, l’insertion d’éléments virtuels de façon plausible et esthétique reste une problématique compliquée, mal-posée et peu adaptée à un contexte temps réel. En effet, cette insertion requiert une reconstruction de l’illumination de la scène afin d’appliquer les conditions lumineuses adéquates à l’objet inséré. L’illumination de la scène peut être divisée en plusieurs catégories. Nous pouvons modéliser l’environnement de façon à décrire l’interaction de la lumière incidente et réfléchie pour chaque point 3D d’une surface. Il est également possible d’expliciter l’environnement en calculant la position des sources de lumière, leur type (lampe de bureau, néon, ampoule, ….), leur intensité et leur couleur. Pour insérer un objet de façon cohérente et réaliste, il est primordial d’avoir également une connaissance de la surface recevant l’illumination. Cette interaction lumière/matériaux est dépendante de la géométrie de la surface, de sa composition chimique (matériau) et de sa couleur. Pour tous ces aspects, le problème de reconstruction de l’illumination est difficile, car il est très complexe d’isoler l’illumination sans connaissance a priori de la géométrie, des matériaux de la scène et de la pose de la caméra observant la scène. De manière générale, sur une surface, une source de lumière laisse plusieurs traces telles que les ombres, qui sont créées par l’occultation de rayons lumineux par un objet, et les réflexions spéculaires ou spécularités qui se manifestent par la réflexion partielle ou totale de la lumière. Bien que ces spécularités soient souvent considérées comme des éléments parasites dans les applications de localisation de caméra, de reconstruction ou encore de segmentation, ces éléments donnent des informations cruciales sur la position et la couleur de la source lumineuse, mais également sur la géométrie de la surface et la réflectance du matériau où elle se manifeste. Face à la difficulté du problème de modélisation de la lumière et plus particulièrement du calcul de l’ensemble des paramètres de la lumière, nous nous sommes focalisés, dans cette thèse, sur l’étude des spécularités et sur toutes les informations qu’elles peuvent fournir pour la compréhension de la scène. Plus particulièrement, nous savons qu’une spécularité est définie comme la réflexion d’une source de lumière sur une surface réfléchissante. Partant de cette remarque, nous avons exploré la possibilité de considérer la spécularité comme étant une image issue de la projection d’un objet 3D dans l’espace. Nous sommes partis d’un constat simple, mais peu traité par la littérature qui est que les spécularités présentent une forme elliptique lorsqu’elles apparaissent sur une surface plane. À partir de cette hypothèse, pouvons-nous considérer un objet 3D fixe dans l’espace tel que sa projection perspective dans l’image corresponde à la forme de la spécularité ? Plus particulièrement, nous savons qu’un ellipsoïde projeté perspectivement donne une ellipse. Considérer le phénomène de spécularité comme un phénomène géométrique a de nombreux avantages. (...) / Augmented Reality (AR) consists in inserting virtual elements in a real scene, observed through a screen or a projection system on the scene or the object of interest. The augmented reality systems can take different forms to obtain a balance between three criteria: precision, latency and robustness. It is possible to identify three main components to these systems: localization, reconstruction and display. The contributions of this thesis focus essentially on the display and more particularly the rendering of augmented reality applications. Contrary to the recent advances in the field of localization and reconstruction, the insertion of virtual elements in a plausible and aesthetic way remains a complicated problematic, ill-posed and not adapted to a real-time context. Indeed, this insertion requires a good understanding of the lighting conditions of the scene. The lighting conditions of the scene can be divided in several categories. First, we can model the environment to describe the interaction between the incident and reflected light pour each 3D point of a surface. Secondly, it is also possible to explicitly the environment by computing the position of the light sources, their type (desktop lamps, fluorescent lamp, light bulb, . . . ), their intensities and their colors. Finally, to insert a virtual object in a coherent and realistic way, it is essential to have the knowledge of the surface’s geometry, its chemical composition (material) and its color. For all of these aspects, the reconstruction of the illumination is difficult because it is really complex to isolate the illumination without prior knowledge of the geometry, material of the scene and the camera pose observing the scene. In general, on a surface, a light source leaves several traces such as shadows, created from the occultation of light rays by an object, and the specularities (or specular reflections) which are created by the partial or total reflection of the light. These specularities are often described as very high intensity elements in the image. Although these specularities are often considered as outliers for applications such as camera localization, reconstruction or segmentation, these elements give crucial information on the position and color of the light source but also on the surface’s geometry and the material’s reflectance where these specularities appear. To address the light modeling problem, we focused, in this thesis, on the study of specularities and on every information that they can provide for the understanding of the scene. More specifically, we know that a specularity is defined as the reflection of the light source on a shiny surface. From this statement, we have explored the possibility to consider the specularity as the image created from the projection of a 3D object in space.We started from the simple but little studied in the literature observation that specularities present an elliptic shape when they appear on a planar surface. From this hypothesis, can we consider the existence of a 3D object fixed in space such as its perspective projection in the image fit the shape of the specularity ? We know that an ellipsoid projected perspectivally gives an ellipse. Considering the specularity as a geometric phenomenon presents various advantages. First, the reconstruction of a 3D object and more specifically of an ellipsoid, has been the subject to many publications in the state of the art. Secondly, this modeling allows a great flexibility on the tracking of the state of the specularity and more specifically the light source. Indeed, if the light is turning off, it is easy to visualize in the image if the specularity disappears if we know the contour (and reciprocally of the light is turning on again). (...)

Este trabalho apresenta uma pesquisa desenvolvida em uma instituição de educação básica e investiga práticas que usam ou refletem sobre o uso das tecnologias digitais na formação docente e discente. Para tanto, utilizou-se das contribuições de autores de diversas áreas do conhecimento como da educação, comunicação e linguística, além de autores que discutem sobre os novos letramentos e a contemporaneidade. A introdução deste trabalho apresenta as justificativas e questões de investigação, as escolhas metodológicas e o campo investigado. Os dados da pesquisa foram organizados a partir de práticas de letramento digital que tinham como foco o estudo sobre as tecnologias; ações que se utilizaram dos recursos e ambientes virtuais como apoio à prática pedagógica e atividades cujo objetivo era a reflexão sobre o tempo presente e a ubiquidade das tecnologias. No capítulo 1 analiso algumas concepções teóricas usadas na análise dos dados e relato uma sequência de aulas de história, cujo tema de estudos é a globalização. Parte dos aspectos discutidos se relaciona com a chamada pós-modernidade, a circulação de informação na Internet e o uso das tecnologias digitais nos movimentos sociais. No segundo relato e respectiva análise (capítulo 2), os alunos exploram recursos digitais e constroem um jogo de caça ao tesouro com realidade aumentada; trata-se aqui do conceito de apropriação das tecnologias numa perspectiva crítica. Ao final, no capítulo 3, são relatadas e analisadas práticas de formação docente a partir de oficinas técnico pedagógicas, visando a apropriação e a reflexão sobre o uso de recursos digitais; atividades que usam ferramentas de apoio à prática pedagógica e que promovem a reflexão sobre aspectos éticos do uso dos ambientes virtuais no ensino fundamental. Ainda neste capítulo, analiso as mudanças de procedimentos de estudos com o uso das tecnologias digitais a partir de práticas investigadas. / This study presents the research developed in a primary and secondary educational institution and investigates practices which use and reflect upon digital technologies in the development of teachers and students. For this, the contributions of authors from diverse areas of knowledge were used, such as education, communication and linguistics, as well as authors who discuss approaches to new literacies and contemporary society. The introduction of this study presents the investigation justifications and questions, methodological choices and the investigated field. The data from the research were collected from the digital literacy practices which focused on the study of technology; actions which used virtual resources and environments as support for the pedagogical practices and activities which had as an objective the reflection on the present times and the ubiquity of technology. In chapter 1, I analyze some theoretic conceptions used in the data analysis and I report a sequence of history classes, where the object of study is globalization. Part of the aspects discussed are related to post-modernity, the circulation of information on the internet and the use of digital technologies in social movements. In the second chapter, the students explore digital resources to build a treasure-hunt game with augmented reality; based on the concept of appropriation of technologies from a critical perspective. Finally, in chapter 3, teacher development practices are described and analyzed during technical-pedagogical workshops, focusing on the appropriation and reflection on the use of digital resources; activities that use the tools to support pedagogical practices and promote reflection on ethical aspects and the use of virtual environments in primary education. Still in this chapter, I analyze the change in study procedures with the use of digital technologies from the investigated practices.

Augmented reality (AR) is constantly moving forward and pushing its bound- aries forward. New applications and frameworks for mobile devices are rapidly developing. Head mounted displays are evolving and making an impact on in- dustries and people. In this thesis, we are going to evaluate the concept of how to make use of the environmental space in augmented reality. Within the environmental space, we are going to focus on secondary elements — elements and objects that not are in the focus of the users. Both augmented reality in smartphones and head-mounted displays are going to be considered. Through an evaluation conducted with four participants during a week, we are going to find use cases and scenarios where this type of concept could be used and where it can be applied. The results of this thesis shows where and how the can be use for a concept such as this.

Context. Volvo Construction Equipment is planning to use Microsoft Hololens as a tool for the on-site manager to keep a track on the automotive machines and obtain their corresponding work information. For that, a miniature site has been build at PDRL BTH consisting of three different automotive vehicles. We are developing Augmented Reality applications for Microsoft Hololens to recognize these automotive vehicles. There is a need to identify the most feasible recognition method that can be implemented using Microsoft Hololens. Objectives. In this study, we investigate which among the Vuforia 3-D recognition method and the feasible method is best suited for the Microsoft Hololens and we also find out the maximum distance at which an automotive vehicle can be recognized by the Microsoft Hololens. Methods. In this study, we conducted a literature review and the number of articles has been reviewed for IEEE Xplore, ACM Digital Library, Google Scholar and Scopus sources. Seventeen articles were selected for review after reading their titles and abstracts of articles obtained from the search. Two experiments were performed to find out the best recognition method of the Microsoft Hololens and the maximum distance at which an automotive vehicle can be recognized by the Microsoft Hololens. Results. QR-code recognition method is the best recognition method to be used by Microsoft Hololens for recognizing automotive vehicles in the range of one to two feet and Vuforia 3-D recognition method is recommended for more than two feet distance. Conclusions. We conclude that the QR-code recognition method is suitable for recognizing vehicles in the close range (1-2 feet) and Vuforia 3-D object recognition is suitable for recognition for distance over two feet. These two methods are different from each other. One used the 3-D scan of the vehicle to recognize the vehicle and the other uses image recognition (using unique QR-codes). We covered effect of distance on the recognition capability of the application and a lot of work has to be done in terms of how does the QR-code size effects the maximum distance at which an automotive vehicle can be recognized. We conclude that there is a need for further experimentation in order to find out the impact of QR-code size on the maximum recognition distance.

Made available in DSpace on 2016-04-26T18:10:52Z (GMT). No. of bitstreams: 1 Cesar Augusto Baio Santos.pdf: 2736898 bytes, checksum: 06ae90a48c2064c02ad0d62ffdd329b3 (MD5) Previous issue date: 2011-05-16 / Conselho Nacional de Desenvolvimento Científico e Tecnológico / This work investigates the transformations in processes of intermediation of sense in digital image and its aesthetic unfoldings. In order to achieve that, an update of theories on dispositif is suggested (BAUDRY, FOUCAULT, DELEUZE), mainly based on the systemic phenomenology of Vilém Flusser, to develop thinking on a non-immersive condition of image. This hypothesis is taken forward with the problematization of neoplatonic conceptions regarding virtual environments (HEIM, FRIEDBERG, GRAU) which establish sense regimes based on the abduction of the individual in the parallel worlds of immersive environments. The main issues raised are related to the validity of these theories towards the dispositifs created in a context of Augmented Reality, ubiquitous computing (WEISER), cybrid condition of space (ANDERS) and networks (BEIGUELMAN); and, above all, of interfaces based on physical computing and algorithms of computer vision. Interpreted as a projection of conceptual abstraction towards concreteness of experience, image stops being understood for dualities between what's "real" and "virtual" or "physical" and "informational", to present itself as a phenomenon that is projected from the dispositif to establish new relations with phenomenons of other natures which constitute reality. Considering this project's condition, image starts to give itself to experience through processes of objectification and otherization, installing new aesthetic experiences of image. Being integrated to the coded world (FLUSSER), image becomes an object of a specific genesis among many others which surround us. However, the processes of otherization approximate the condition of image and of the individual to that one assigned to the body in performance aesthetics (COHEN, LEHMANN, FISCHER-LICHTE), demanding from both a performative posture based on the significant value of presence (GUMBRECHT, ZUMTHOR) and gesture (FLUSSER). Those transformations in the intermediation processes between piece and individual launch new aesthetic vectors to the understanding of media art. This scenery is outlined in this research considering the analysis of audiovisual pieces produced in the last two decades, which present themselves as technical dispositifs that are open to someone's interaction. The immersive environments in Virtual Reality, CAVEʼs and digital panoramas are analyzed against cybernetic dispositifs, pieces that use techniques of augmented reality and others that put image and public in performative conditions. The analysis goes through theories of media art (BELLOUR, DUBOIS, MACHADO) and of new media (MURAY, MANOVICH, COUCHOT, WEIBEL, HANSEN) identifying displacements in the condition of image and of subjectivity's figures, this way demanding other postures and sensibilities, both from creators and from those who place themselves before the image / Este trabalho investiga as transformações nos processos de agenciamento de sentido da imagem digital e seus desdobramentos estéticos. Para tanto é proposta uma atualização das teorias do dispositivo (BAUDRY, FOUCAULT, DELEUZE), principalmente a partir da fenomenologia sistêmica de Vilém Flusser, para pensar uma condição não imersiva da imagem. Esta hipótese é levada adiante com a problematização das concepções neoplatônicas dos ambientes virtuais (HEIM, FRIEDBERG, GRAU), que estabelecem regimes de sentido fundados na abdução do sujeito nos mundos paralelos dos ambientes imersivos. As principais questões levantadas se referem à validade de tais teorias frente aos dispositivos criados no contexto da Realidade Aumentada, da ubiquidade computacional (WEISER), da condição cíbrida do espaço (ANDERS) e das redes (BEIGUELMAN); e, sobretudo, das interfaces baseadas na computação física e nos algoritmos de computer vision. Entendida como uma projeção da abstração conceitual rumo à concretude da experiência, a imagem deixa de ser compreendida por dualidades entre real e virtual ou físico e informacional para se apresentar com um fenômeno que se projeta do dispositivo para estabelecer relações com fenômenos de outras naturezas que constituem a realidade. Nessa condição de projeto, a imagem passa a se dar à experiência por meio de processos de objetivação e outrificação, instalando novas experiências estéticas da imagem. Integrada ao mundo codificado (FLUSSER), a imagem torna-se um objeto de gênese específica dentre tantos outros que nos cercam. No entanto, os processos de outrificação aproximam a condição da imagem e do sujeito àquela conferida ao corpo na estética da performance (COHEN, LEHMANN, FISCHERLICHTE), demandando de ambos uma postura performativa pautada no valor significante da presença (GUMBRECHT, ZUMTHOR) e do gesto (FLUSSER). Tais transformações nos processos de agenciamento entre obra e sujeito lançam assim novos vetores estéticos para a compreensão da media art. Esse cenário é delineado nesta pesquisa a partir da análise de obras audiovisuais produzidas nas últimas duas décadas que se apresentam elas mesmas como dispositivos técnicos abertos à interação do sujeito. Os ambientes imersivos em Realidade Virtual, CAVEʼs e panoramas digitais são analisados em contraposição aos dispositivos da cibernética, aos trabalhos que utilizam técnicas de realidade aumentada e outros que colocam imagem e público em condições performativas. As análises atravessam as teorias da media art (BELLOUR, DUBOIS, MACHADO) e da nova mídia (MURRAY, MANOVICH, COUCHOT, WEIBEL, HANSEN) identificando deslocamentos na condição da imagem e das figuras de subjetividade, demandando assim outras posturas e sensibilidades, tanto dos criadores quanto daquele que se posiciona diante da imagem

Dans ce travail, méthodes d'estimation basées sur des images, également connu sous le nom de méthodes directes, sont étudiées qui permettent d'éviter l'extraction de caractéristiques et l'appariement complètement. L'objectif est de produire pose 3D précis et des estimations de la structure. Les fonctions de coût présenté minimiser l'erreur du capteur, car les mesures ne sont pas transformés ou modifiés. Dans la caméra photométrique estimation de la pose, rotation 3D et les paramètres de traduction sont estimées en minimisant une séquence de fonctions de coûts à base d'image, qui sont des non-linéaires en raison de la perspective projection et la distorsion de l'objectif. Dans l'image la structure basée sur le raffinement, d'autre part, de la structure 3D est affinée en utilisant un certain nombre de vues supplémentaires et un coût basé sur l'image métrique. Les principaux domaines d'application dans ce travail sont des reconstitutions d'intérieur, la robotique et la réalité augmentée. L'objectif global du projet est d'améliorer l'image des méthodes d'estimation fondées, et pour produire des méthodes de calcul efficaces qui peuvent être accueillis dans des applications réelles. Les principales questions pour ce travail sont : Qu'est-ce qu'une formulation efficace pour une image 3D basé estimation de la pose et de la structure tâche de raffinement ? Comment organiser calcul afin de permettre une mise en œuvre efficace en temps réel ? Quelles sont les considérations pratiques utilisant l'image des méthodes d'estimation basées sur des applications telles que la réalité augmentée et la reconstruction 3D ? / In this work, image based estimation methods, also known as direct methods, are studied which avoid feature extraction and matching completely. Cost functions use raw pixels as measurements and the goal is to produce precise 3D pose and structure estimates. The cost functions presented minimize the sensor error, because measurements are not transformed or modified. In photometric camera pose estimation, 3D rotation and translation parameters are estimated by minimizing a sequence of image based cost functions, which are non-linear due to perspective projection and lens distortion. In image based structure refinement, on the other hand, 3D structure is refined using a number of additional views and an image based cost metric. Image based estimation methods are usable whenever the Lambertian illumination assumption holds, where 3D points have constant color despite viewing angle. The main application domains in this work are indoor 3D reconstructions, robotics and augmented reality. The overall project goal is to improve image based estimation methods, and to produce computationally efficient methods which can be accomodated into real applications. The main questions for this work are : What is an efficient formulation for an image based 3D pose estimation and structure refinement task ? How to organize computation to enable an efficient real-time implementation ? What are the practical considerations of using image based estimation methods in applications such as augmented reality and 3D reconstruction ?

Com a popularização de ferramentas computacionais aplicadas ao ensino e o advento da Web 2.0, os Ambientes Virtuais de Aprendizagem (AVAs) evoluíram com os recursos proporcionados pelas tecnologias de Realidade Virtual (RV) e Realidade Aumentada (RA), surgindo os AVAs tridimensionais (3D), amplamente adotados em diversos domínios de conhecimento. Aliado a este avanço, o processo de aquisição de conhecimento em AVAs 3D tem sido alvo de pesquisas a fim de que tal aspecto seja considerado na concepção dos ambientes virtuais. Uma discussão que é delineada neste contexto se refere à contribuição que os AVAs 3D efetivamente oferecem para a aquisição de conhecimento do aprendiz e como é possível avaliar tal aprendizado. Estudos têm demonstrado que os métodos amplamente aplicados para avaliação da cognição nos AVAs 3D têm feito uso intensivo de preenchimento de questionários, aplicados fora dos AVAs. É possível perceber ainda que os instrumentos de avaliação aplicados nos AVAs 3D não incorporam no processo de avaliação as próprias tecnologias interativas disponíveis no ambiente. Sendo assim, o propósito inicial desta pesquisa é identificar o processo de aquisição de conhecimento nos AVAs 3D, como também os métodos existentes para avaliar se ocorre de fato a aquisição de conhecimento e em que grau ela ocorre. A partir deste estudo, o objetivo deste trabalho é definir uma metodologia para avaliar a aquisição de conhecimento, usando as próprias tecnologias interativas disponíveis nos AVAs 3D, de forma inovadora, em complementação aos questionários amplamente empregados como instrumentos de avaliação. Além da proposição do método conceitual, foi definida uma arquitetura e implementado um sistema computacional capaz de registrar as interações dos usuários em AVAs 3D, de forma não intrusiva à aplicação. Também foi sugerido um modelo matemático que propõe a avaliação da aprendizagem sem a necessidade de execução de provas de conhecimento. Experimentos foram conduzidos para avaliar cada uma das fases do modelo. Os resultados dos experimentos mostraram que é factível avaliar a aquisição de conhecimento dentro de contextos pré-determinados e que há indícios de que alguns tipos de interações do usuário com os AVAs mantêm correlação com o nível de aprendizado do usuário. / The popularization of computational tools applied to teaching and the advent of Web 2.0 have led to the advancement of Virtual Learning Environments (VLE) through resources rendered by Virtual Reality (VR) and Augmented Reality (AR) technologies, leading to the emergence of Tridimensional VLEs (3D), which are widely employed in diverse fields of knowledge. In alliance with this advancement, the 3D VLE knowledge acquisition process has been the object of research studies which consider the conception of virtual environments. One of the discussions outlined in this context refers to the contribution that 3D VLEs effectively offer for knowledge acquisition and how it is possible to evaluate this learning process. Studies have demonstrated that methods widely employed for assessing cognition in 3D VLEs have made intense use of questionnaire application outside VLEs. It is possible to perceive that, in the evaluation process, the assessment tools applied in 3D VLEs do not incorporate the same interactive technologies available in the environment. Thus, the initial aim of this research is to identify the knowledge acquisition process in 3D VLEs, as well as existing methods for assessing if knowledge acquisition actually occurs and to which degree it occurs. From this study, the aim of the research is to define a method for assessing knowledge acquisition employing the same interactive technologies available in 3D VLEs, in an innovative manner, complementing widely used questionnaires as assessment tools. In addition to the suggestion of the conceptual method, an architecture was defined and a computational system capable of registering user interactions in 3D VLEs was implemented, in a non intrusive form of application. A mathematical model that offers learning assessment without the need of knowledge tests was also suggested. Experiments were conducted to evaluate each phase of the model. The results of the experiments show that it is feasible to evaluate knowledge acquisition within predetermined contexts and that there is evidence that some types of user interactions with VLEs are correlated with the user\'s learning level.

O uso de dispositivos móveis ou handheld devices na implementação de um sistema de Realidade Aumentada (RA) é denominado Handheld Augmented Reality ou simplesmente Handheld AR. Aplicações do tipo Handheld AR utilizam a câmera integrada no dispositivo móvel para reconhecer marcadores (inseridos no ambiente), o que permite a entrada de dados para o sistema de RA. As informações geradas nestas aplicações são apresentadas como imagens virtuais (gráficos 3D) nas telas dos dispositivos móveis. Já aplicações do tipo Handheld AR mais complexas e recentes, utilizam dispositivos sensores (com interfaces de comunicação sem fio) como RFID tags, Redes de Sensores Sem Fio (RSSF) e sistemas de GPS (Global Positioning System), como fontes de entrada de dados dinâmicos para o sistema de RA, junto à câmera e os marcadores. Aplicações deste tipo apresentam características interessantes para áreas de aplicação como: sensoriamento, monitoramento, visualização e interação em ambientes inteligentes e ubíquos. Porém, a carência de documentação e a complexidade no desenvolvimento destas aplicações geram a necessidade de metodologias e ferramentas que colaborem para um rápido e simples processo de desenvolvimento. Neste sentido, esta dissertação apresenta uma Metodologia de Desenvolvimento para aplicações de realidade aumentada em dispositivos móveis, a qual visa ajudar no projeto e facilitar o processo de desenvolvimento de aplicações do tipo Handheld AR, em especial as que utilizem telefones celulares e dispositivos sensores com interfaces de comunicação sem fio, apresentando uma documentação bem definida e um roteiro claro sobre os requisitos e características no desenvolvimento de aplicações deste tipo. Já a nossa metodologia proposta apresenta um cenário geral de aplicação, uma arquitetura e orientações (guidelines) de desenvolvimento para aplicações do tipo Handheld AR. Além disso, são apresentados detalhes de implementação relativos: à aquisição dos dados gerados pelos dispositivos sensores (neste caso uma RSSF), a uma modelagem semântica destes dados (como parte da solução para facilitar a aquisição, armazenamento e gerenciamento de dados), à criação de perfis de usuário (para prover interfaces de interação intuitivas) e à disponibilização destas informações utilizando técnicas de RA, apresentando as informações geradas como objetos virtuais (imagens virtuais, efeitos de vibração e efeitos sonoros), os quais são sobrepostos em um cenário real utilizando o telefone celular como dispositivo de interação com o usuário. Finalmente, nós apresentamos um estudo de caso que desenvolve uma aplicação do tipo Handheld AR (protótipo) para o monitoramento da temperatura na sala de servidores do PAD (Grupo de Sistemas Pervasivos e de Alto Desempenho). Neste estudo de caso, são descritos os materiais, o cenário, os serviços oferecidos, o fluxo de desenvolvimento e as avaliações do protótipo. / Handheld Augmented Reality or Handheld AR is the implementation of Augmented Reality (AR) systems into handheld devices. Handheld AR applications use the handheld device\'s camera to recognize markers (inserted into the environment). This process enables data entry into AR system. The response generated for these applications are presented as virtual images (3D graphics) on the handheld device\'s display. New Handheld AR applications use a mobile phone camera, markers and wireless sensors (inserted into the environment) as elements for dynamic data entry into the AR system. These applications show interesting characteristics for: sensing, visualization and interaction in ambient intelligence (AmI) and ubiquitous environments. But, the lack of documentation and complexity development process produce the need for methodologies and tools for a fast and simple development model. For these reason, this work presents a Methodology for Design and development Handheld AR applications, which present an especial support for mobile phones and wireless sensor networks, making a description and clear roadmap of the requirements and characteristics in the Handheld AR development process model. Our methodology proposed presents a general scenario of application, a software architecture and application development guidelines. Moreover, we present implementation details over: acquisition data process, semantic modeling data, development of user profiles and visualization of the information using AR techniques, showing these information as virtual objects (virtual images, vibration effects and sound effects). These virtual objects use the handheld device as interaction interface between the user and real world. Finally, we present a case study that develops a Handheld AR application (prototype) to monitoring the temperature in the PAD (Pervasive and Distributed Computing Group) server\'s room. In this case study, we describe the materials, the scenario, the services offered, the development process and prototype evaluations.