Abstract Interactive Media and Game development are experiencing an incredible growth the recent year. Digital devices are in constant evolution, and the developers/artist are glad to unleash their full potential by creating innovative content that take advantage of those capacities. Although creation and innovation is not what is limiting this field, the business side to this industry is the biggest challenge to this kind of content. Making a good product needs a team effort, and these employees will obviously need good money for their expertise, which, means the product needs to be funded to make it happen. In this context I find myself in the perfect position for this kind of adventure as I both love games and I like to play the role of the entrepreneur/problem solver. As a Fulbright scholar I feel my self-obligated to help my country back with all the knowledge and the opportunities I got offered by coming here and learning from U.S professionals in the industry This brings us to the core of this thesis, by doing a Co-op job in Petricore, shadowing its CEO and surrounding myself by a huge community of professionals from the industry, researching the best way to start my own startup €œABY€� in the same industry in Tunisia. I will also using auto ethnography and interviews to collect relevant data in an ultimate goal of building a lightweight business plan to guide me in starting this adventure in my home country.

Syftet med denna forskning genom design och kvalitativa undersökning är att skapa en audiovisuell installation vilket mäts av en fokusgrupp för att se om fenomenet Immersion uppnås hos informanterna. Det är informanternas kollektiva och subjektiva upplevelse av installationen som ligger till grund för studiens resultat och berör frågor inom det audiovisuella området. Installationen undersöks i förhållande till rummet den tar plats i som en viktigt påverkande aspekt. Studien ska ses på som ett förslag för hur fenomenet Immersion kan uppnås genom audiovisuella verktyg. Alltså är detta en studie som inte försöker nå en absolut sanning genom undersökningen utan olika resultat kan nås även om genomförandet av installationen skulle försökas återskapas. Resultaten tyder på att Installationen kändes Immersiv och nåddes genom projektionskonstformen och spatialt 3D ljud. Samt att andra aspekter och egenskaper som att installationens fysiska form, vilket bestod av genomskinligt tyg som omringade informanterna, påverkade deras Immersion positivt genom att avskärma deras perception. Det presenteras även andra aspekter och egenskaper som påverkade informanternas upplevelse av Immersion positivt men även resultat som påverkade upplevelsen negativt, vilket är lika viktigt för vidare forskning och som bidrag till det kunskapsområdet. Styrkan i undersökningen är presentationen av installationens genomförande men framförallt resultatet som bidrar med förståelse och perspektiv till ett svårformulerat intresseområde. Svagheten ligger i och med att fokusgruppdiskussioner och kvalitativa underökningar inte går att generaliseras, samt så är validiteten svår att styrka. Ytterligare en svaghet är att bara en fokusgrupp användes för studiens undersökning vilket gör det svårt att säkerställa resultatets giltighet.

This thesis presents a real-time automated building progress monitoring solution for indoor environments using a mobile device. Such a system could prompt accurate and timely assessment of work progress that would allow managers to make adjustments and minimise both time and cost overrun when deviations from the schedule occur. Although many researchers have proposed approaches for progress monitoring in outdoor scenes, these cannot perform in real-time and shift into the complex interior environment. Research efforts for indoor environments are not fully automated and lead to errors in more complex scenes. Systems based on mobile devices could potentially enhance the inspection process and reduce the required time by allowing the inspector to acquire progress data by simply walking around the site. The main challenge of these systems is the tracking of the pose of the camera to achieve accurate alignment between the 3D design model and the real-world scene. Methods for estimating the user’s pose rely on a) tags on each target of interest, which require additional time and cost for installation and maintenance; b) pre-selected user locations, which restricts the user to those locations only; or c) GPS on the augmented reality headset, which only applies to outdoor inspections. Additionally, current mobile-based inspection systems do not perform any comparison between the captured as-built and the as-planned data. In this research, different marker-less Augmented Reality (AR) potential methods were implemented and tested for finding the most robust tracking solution. The Microsoft HoloLens was found to be the top performer for tracking the user’s pose and for overall user-experience. Next, a semi-automated method was developed for initially registering the 3D model to the real environment by exploiting information from detected floor and wall surfaces. Results showed that this method reduces the time of the initial registration by 58%. Having the 3D model aligned to the real environment and knowing the pose of the camera at every moment, an automated method was developed that exploits the captured as-built surface mesh data from the mobile device, compares it against the 3D design model and identifies in real-time whether an object has been built according to plan. Different parameters were tested for finding the optimum combination based on the current quality of mesh data. If quality of mesh data changes, then new parameters should be explored. Finally, the proposed solution was tested in real site conditions resulting in 76.6% precision, 100.0% recall, and 83.5% accuracy.

虛擬現實（VR）為用家提供了一個結合了人工場境和現實世界的練習環境。隨著在醫療，軍事和娛樂行業日益增長的需求，虛擬現實化的設備將會越來越受歡迎。頭戴式顯示器（HMD）是一種利用自由曲面的光學棱鏡，以投射由微型顯示器顯示的圖像，讓用家能夠在近距離觀看和體驗虛擬環境的產品。 / 頭戴式顯示器由兩個基本部分組成，液晶顯示屏和一個光學元件，以能夠在很短的距離顯示圖像到用家的眼睛。光學自由曲面棱鏡在HMD研究中是一個主要的課題。設計的棱鏡需要運用大多數的光學理論和一些商業光學設計軟件的輔助。在市場，只有有限的光學軟件能深入地協助設計光學元件和系統。同時，成本高昂和不容易使用的光學軟件令到光學設計並不普及的原因。在下面的部分將用光學軟件ZEMAX去驗證每個設計。 / 自由曲面棱鏡的幾何形狀設計是為構建一個頭盔顯示器的關鍵問題。通常情況下，頭盔內自由曲面棱鏡由三面特別設計的非球面表面而組成。源圖像經過三個非球面反映後，可參照圖6，將圖像投射到人類視網膜上而形成圖像。棱鏡上的三個面的互相協調將是一個具有挑戰性的問題。 / 跟據現有的產品和研究分析，研發了一種新風格的棱鏡。在這篇論文中，由射線追踪和光學理論的幫忙，建議了一款凸-平棱鏡作HMD上的光學棱鏡。利用光學設計軟件分析和優化以追求一個可穿戴和低像差的光學棱鏡作目標。光學棱鏡中有兩個面為平面，因此光線能在棱鏡中穩步地轉化，從而，降低光學系統的像差。製作了經優化表面定位的原型，雖然原型是成本低，但它們比半透明的鏡子有一個更好的分辨率。在新設計的一個HMD上，一個廣闊和清晰的圖像將顯示。 / Virtual reality (VR) provides an artificial environment to user for practicing under a synthetic environment which merges the real world with artificial scene. In response to the growing demand in medical, military and entertaining application, devices that can simulate VR will become more and more popular. Head mounted display (HMD) is a kind of wearable products that utilize freeform optical prisms to project images generated by microdisplays, enabling users to situate in virtual environments. / HMD has two basic components, a LCD and an optical element, to display image to naked eye in a short distance. An optical freeform prism will be a main research topic in HMD. To design the prism will require substantial knowledge in optical theory and the aid of some commercial optical design software. In the market, there is limited number of software for in depth optical analysis. Meanwhile, costly and not user-friendly software is a problem of unpopularity of optical design. Zemax has been used for verifying the each design in the below sections. / The design of geometric shape for the freeform prism is crucial for constructing a HMD. Typically, the freeform prisms inside HMD comprise of 3 specially designed aspherical surfaces. The source image is reflected by those three surfaces to form image on human retina. The interplay of three surfaces will is the main challenging problem. With the analysis of existing products and researches, a new-style prism is proposed. In this thesis, with the aid of ray tracing and optical theory, a convex-plano prism is developed. / Analyses of optical design software and optimization are utilized to pursue the goal of wearable and low aberration HMD optical prism. Prototypes with the optimized surfaces alignment are fabricated. With two flat surfaces on the prism, the rays translate steadily in the prism. Hence, the optical system’s aberration is reduced. Although the prototypes are low cost, they have a better resolution than half-transparent mirror. Also a wide and clear image will show on the new style see-through HMD. / Detailed summary in vernacular field only. / Detailed summary in vernacular field only. / Detailed summary in vernacular field only. / Detailed summary in vernacular field only. / Ho, Pui Lam. / Thesis (M.Phil.)--Chinese University of Hong Kong, 2012. / Includes bibliographical references (leaves 57-59). / Abstracts also in Chinese. / Abstract --- p.i / Abstract (Chinese) 摘要 --- p.iii / Acknowledgement --- p.iv / List of Figure and Table --- p.vii / Paper published --- p.xi / Chapter Chapter 1 --- Introduction --- p.1 / Chapter 1.1 --- Overview on Near Eye Display (NED) --- p.1 / Chapter 1.2 --- Literature Review on HMD --- p.1 / Chapter 1.3 --- Motivation and Research Objective --- p.7 / Chapter 1.4 --- Thesis Outline --- p.8 / Chapter Chapter 2 --- Fundamental Design Concept --- p.9 / Chapter 2.1 --- Prism --- p.9 / Chapter 2.2 --- Convex Lens --- p.11 / Chapter 2.3 --- Combination of Convex lens and Regular Prism --- p.12 / Chapter Chapter 3 --- Optical Design Analysis by Software Package --- p.16 / Chapter 3.1 --- Surface Alignment --- p.16 / Chapter 3.2 --- Distortion --- p.18 / Chapter 3.3 --- MTF --- p.19 / Chapter 3.4 --- Image Simulation --- p.20 / Chapter 3.5 --- Aberration analysis --- p.21 / Chapter 3.6 --- Aspherical surface --- p.22 / Chapter Chapter 4 --- Design Principle and Procedure --- p.26 / Chapter 4.1 --- Basic Criteria --- p.26 / Chapter 4.2 --- See-throughPurpose --- p.27 / Chapter 4.3 --- Material --- p.28 / Chapter 4.4 --- SurfacesAlignment --- p.29 / Chapter 4.5 --- Compensator --- p.30 / Chapter 4.6 --- Angle Alignment --- p.30 / Chapter Chapter 5 --- System Efficiency --- p.32 / Chapter 5.1 --- System Efficiency --- p.32 / Chapter 5.2 --- System Analysis --- p.35 / Chapter Chapter 6 --- Prototype Fabrication and Testing --- p.38 / Chapter 6.1 --- Display System --- p.38 / Chapter 6.2 --- Monocular Configurations --- p.39 / Chapter 6.3 --- Prototypes Building --- p.40 / Chapter 6.4 --- Advantages --- p.54 / Chapter 6.5 --- Limitation --- p.54 / Chapter Chapter 7 --- Conclusions and Future Work --- p.55 / Chapter 7.1 --- Conclusions --- p.55 / Chapter 7.2 --- Future work --- p.55 / Reference --- p.57 / Appendix --- p.60 / Chapter Appendix 1 --- Background Theory --- p.60 / Chapter A1.1 --- Optical Theory --- p.60 / Chapter A1.2 --- Matrix Method --- p.65 / Chapter A1.3 --- Human Vision --- p.66 / Chapter Appendix 2 --- Simulation of reflection --- p.67 / Chapter Appendix 3 --- Aberration analysis of prisms --- p.70 / Chapter Appendix 4 --- Specification --- p.71

Li Kai Min, Kenneth. / "Architecture Department, Chinese University of Hong Kong, Master of Architecture Programme 2010-2011, design report." / Includes bibliographical references (p. 77-78). / Augmneted Reality: A New challenge that Architecture Face --- p.3 / Chapter 1. --- Ornament / Iconography / Visual Narratives --- p.4 / Chapter 2. --- From Static to Augmented and Interactive --- p.9 / Chapter 3. --- Globalization / Changes in Human Behaviours --- p.14 / Chapter 4. --- Towards A New Type of Configurable Architecture --- p.15 / Chapter 5. --- Use of Interface as Means of Spatial Augmentation --- p.16 / Chapter 6. --- Life in the Era of Augmented Reality --- p.35 / Augmented Surface: The Key for Architectural Design in the New Era --- p.49 / Chapter 7. --- Nature of the Project --- p.50 / Chapter 8. --- Site - Jhe going-to-be Plaza in Tsim Sha Tsui --- p.54 / Chapter 9. --- Programme --- p.56 / Chapter 10. --- Schemes --- p.58 / Chapter 11. --- Studies on Image Resolution & Media / Mediated Facade --- p.60 / Chapter 12. --- Further Development --- p.62 / Chapter 13. --- Project Finalized --- p.65 / Blibiography --- p.77 / Appendix

La plupart des métiers du travail de l’information requièrent maintenant de passer la majeure partie de nos journée devant un écran. Ceci s’ajoute au temps déjà consacré à ce médium dans nos temps libres pour le divertissement et la communication, que ce soit en utilisant des téléphones intelligents, tablettes ou ordinateurs. Alors que les avancées technologiques dans le domaine des écrans tactiles nous ont permis d’interagir avec ces appareils de manière plus expressive, par exemple en utilisant nos doigts pour interagir directement avec le contenu numérique, ce que nous voyons et manipulons sur écran reste “intouchable” ; nos doigts ne pouvant pénétrer la surface de l’écran pour toucher le contenu numérique qui se trouve derrière. Pour que l’interaction avec le monde numérique soit écologique dans le futur, elle doit mettre à profit l’ensemble des différentes capacités de l’humain au lieu de ne se concentrer que sur certaines d’entre elles (comme le toucher et la vision), laissant les autres sens s’atrophier. Une façon de considérer le problème est d’utiliser le monde réel physique comme support pour le monde numérique, permettant d’imaginer un futur où les objets du quotidien auront de riches et expressives fonctions numériques, tout en étant ancrés dans le monde réel. La réalité augmentée spatiale est une modalité permettant d’aller dans cette direction. Cette thèse s’intéresse principalement à deux aspects en lien avec ces objets tangibles augmentés. Dans un premier temps, nous soulevons la question de comment interagir avec du contenu numérique lorsqu’il est supporté par des objets physiques. Comme point de départ de notre investigation, nous avons étudié différentes modalités qui utilisent des dispositifs d’entrée/sortie typiquement retrouvés dans un environnement de bureau. Cette approche est justifiée par le désir d’utiliser au maximum l’expérience que les utilisateurs ont déjà acquise avec leurs outils numériques tout en se dirigeant vers un espace d’interaction comprenant des éléments physiques. Dans un second temps, nous sommes allés au delà du thème de l’interaction avec le contenu numérique pour se questionner sur le potentiel des objets tangibles augmentés comme support pour un médium plus humain. Nous avons investigué comment ces artéfacts augmentés, combinés à différents capteurs physiologiques, pourraient permettre d’améliorer notre conscience des processus internes de notre corps et de notre esprit, pour éventuellement favoriser l’introspection. Cette partie a pris la forme de deux projets où un avatar tangible a été proposé pour laisser les utilisateurs explorer et personnaliser le retour d’information sur leurs propres états internes en temps réel. / Most of our waking hours are now spent staring at a screen. While the advances in touch screens have enabled a more expressive interaction space with our devices, by using our fingers to interact with digital content, what we see and manipulate on screen is still being kept away from us, locked behind a glassy surface. The range of capabilities of the human senses is much richer than what screens can currently offer. In order to be sustainable in the future, interaction with the digital world should leverage these human capabilities instead of letting them atrophy. One way to provide richer interaction and visualization modalities is to rely on the physical world itself as a host for digital content. Spatial Augmented Reality provides a technical mean towards this idea, by using projectors to shed digitally controlled light onto real-world objects to augment them and their environment with features and content. This paves the way to a future where everyday objects will be embedded with rich and expressive capabilities, while still being anchored in the real world. In this thesis, we are interested in two main aspects related to these tangible augmented objects. In a first time, we are raising the question on how to interact with digital content when it is hosted on physical objects. As a basis for our investigation, we studied interaction modalities that leverage traditional input and output devices found in a typical desktop environment. Our rationale for this approach is to leverage the experience of users with traditional digital tools – tools which researchers and developers spent decades to make simpler and more efficient to use – while at the same time steering towards a physically enriched interaction space. In a second time, we go beyond theinteraction with the digital content of augmented objects and reflect on their potential as a humane medium support. We investigate how these augmented artifacts, combined with physiological computing, can be used to raise our awareness of the processes of our own bodies and minds and, eventually, foster introspection activities. This took the form of two different projects where we used tangible avatars to let users explore and customize real-time physiological feedback of their own inner states.

A investigação desta dissertação consiste em definir os conceitos de Realidade Aumentada, seus desafios técnicos e tecnológicos, mapear as várias áreas de sua aplicação e quais as suas contribuições na criação das poéticas visuais. Para essa finalidade, tornou-se necessário o entendimento do processo histórico da Realidade Virtual do qual faz parte como uma linha de pesquisa, com seus desafios e aspectos específicos. Podemos perceber, ao longo da pesquisa, que a área dos jogos eletrônicos, com seus interesses mercadológicos, estimula o desenvolvimento tecnológico do sistema de Realidade Aumentada. E que alguns recursos dos sistemas de Realidade Aumentada têm sido utilizados por artistas visuais nas suas criações poéticas, integrando o observador em suas obras e estimulando novas implicações às tecnologias. / The investigation of this thesis sought to define the concepts of Augmented Reality, its technological and technical challenges, to map its various fields of application and reveal its contribution to the creation of visual poetry. To achieve that, it was necessary to understand the historical process of Virtual Reality, to which it belongs as a line of research with its challenges and specific features. We could realize along the research that the area of electronic games with its increasing interest in the market fosters the technological development of the Augmented Reality system. Moreover, some resources of the Augmented Reality systems have been used by Visual artists in their poetic creations, integrating the observer to their works and encouraging new approaches towards technology.

Acidentes vasculares encefálicos têm causado a incapacidade física em adultos e até mesmo em crianças, em todo o mundo. As deficiências motoras provocadas por essa patologia se caracterizam por paralisia ou fraqueza no lado do corpo oposto ao da lesão, gerando déficits no movimento do membro superior e na marcha. A reabilitação desempenha um papel importante no sentido de corrigir ou diminuir as deficiências motoras e aumentar a independência funcional dos pacientes. Nesta pesquisa, é proposto e implementado o NeuroR, um sistema de realidade aumentada que permite investigar se a percepção visual da movimentação de um braço virtual tridimensional, sobreposto à imagem do paciente e conectado ao ombro substituindo o braço real lesado, pode afetar positivamente a reabilitação motora deste membro comprometido. Relatos clínicos têm mostrado que a percepção visua de conectividade com o corpo pode reduzir a sensação de dor e de desconforto em amputações, o que norteou a adoção deste requisito para o sistema. Para avaliar a aplicabilidade do NeuroR foram realizados testes com pacientes, conduzidos por fisioterapeutas neurofuncionais. Nas intervenções, o fisioterapeuta estimula a prática mental de forma que o paciente realize a tarefa solicitada no ambiente de realidade aumentada, onde ele visualiza a si mesmo e o cenário real ao seu redor, como em um espelho. Também foi realizada a prática física, sendo esta disparada por sinais eletromiográficos captados de um músculo do braço parético do paciente. O sistema foi utilizado durante quatro sessões, por quatro pacientes de ambos os sexos, com lesões no hemisfério esquerdo ou direito, com hemiplegia há mais de um ano. O comprometimento motor de cada paciente foi diagnosticado no início do tratamento através da escala Fugl-Meyer. Utilizando o NeuroR, observou-se que para três dos quatro pacientes, ao serem solicitados a imaginar o movimento visualizando sua imagem com o braço virtual, se esforçaram para executar o movimento fisicamente. No pós-teste, todos apresentaram um aumento na escala Fugl-Meyer, no mínimo de 17% e no máximo de 62%. / Stroke has caused the physical disability in adults and even children in the world. Stroke causes motor impairment characterized by paralysis or weakness on the side of the body opposite the site of injury. They produce deficits in movement of the upper limb and gait. The rehabilitation has an important role to correct or reduce motor impairment and in addition to providing functional independence. This research investigates how the perception of visual connectivity of a virtual arm, drawn from the side the patient\'s body, can affect the upper limb motor rehabilitation. Furthermore, clinical trials have shown that it can reduce the sensation of pain and discomfort for patients with amputations. Therefore, there is evidence that it leads us to investigate it and analyze it. This paper presents a proposal for a system that provides the visual stimulus of the movement of a virtual three-dimensional arm, which replaces the paralyzed arm in a real image of the patient. In the interventions the physiotherapist induces the mental practice of the patient such as he/she does the assigned task in the augmented reality environment, where he/she can visualize himself and surroundings, just like a mirror. The protocol included also physical practice. It was performed using electromyographic signals of a paretic arm muscle. The system was used by four chronic adults of both genders, who suffering lesions in different regions of the central nervous system. The sample included patients with lesions in the left or right hemisphere, which have upper-limb hemiplegia for over a year. This motor impairment was diagnosed before beginning the treatment through the Fugl-Meyer scale for the upper limb. In the trials, it was observed that for three of four patients, when asked to imagine their own movement by looking at their image with the virtual arm in the screen, they struggled to physically execute the movement. In the posttest, all showed an increase in the scale motor Fugl-Meyer, a minimum of 17% and a maximum of 62%, and also had improvements in the range of motion of the shoulder, as measured by photogrammetry.

This thesis describes the process of creating an augmented reality mobile application which allows designers, architects and researchers to retrieve accurate colour information picked from the real-world environment. Specifically, the goal is not only to obtain colour coordinates for the area of interest but to find a set of matching colour samples in an extensive database of colour atlases provided with the application. To properly understand how the camera sensor perceives colour under current lighting conditions, it must be calibrated beforehand using a physical colour chart. Based on this calibration, we can estimate the picked colour coordinates in a standardised colour space. The mobility aspect is the main advantage of the resulting application. Instead of carrying multiple colour atlases, the user can estimate colour matching just using a mobile device and a portable colour chart.

Augmented Reality (AR) is a powerful tool for the visualisation of, and interaction with, digital information, and has been successfully deployed in a number of consumer applications. Despite this, AR has had limited success in industrial applications as the combined precision, accuracy, scalability and robustness of the systems are not up to industry standards. With these characteristics in mind, we present a concept Industrial AR (IAR) framework for use in outdoor environments. Within this concept IAR framework, we focus on the improving the precision and accuracy of consumer level devices by focusing on the issue of localisation, utilising LiDAR based point clouds generated as part of normal surveying and engineering workflow. We evaluate key design points to optimise the localisation solution, including the impact of increased field of view on feature matching performance, the filtering of feature matches between real imagery and an observed point cloud, and how pose can be estimated from 2D to 3D point correspondences. The overall accuracy of this localisation algorithm with respect to ground-truth observations is determined, with unfiltered results indicating an on par horizontal accuracy and significantly improved vertical accuracy with best-case consumer GNSS solutions. When additional filtering is applied, results of localisation show a higher accuracy than best-case consumer GNSS.