In the domain of computer vision, registration is the process of calculating the transformation between a known object, called a marker, and a camera which is viewing it. Registration is the foundation for a number of applications across a range of disciplines such as augmented reality, medical imaging and robotic navigation. In the set of two dimensional planar markers, there are two classes: (1) fiducial, which are designed to be easily recognisable by computers but have little to no semantic meaning to people, and (2) natural features, which have meaning to people, but can still be registered by a computer. As computers become more powerful, natural feature markers are increasingly the more popular choice; however there are still a number of inherent problems with this class of markers. This thesis examines the most common shortcomings of natural feature markers, and proposes and evaluates solutions to these weaknesses. The work starts with a review of the existing planar registration approaches, both fiducial and natural features, with a focus on the strengths and weaknesses of each. From this review, the theory behind planar registration is discussed, from the different coordinate systems and transformations, to the computation of the registration transformation. With a foundation of planar registration, natural feature registration is decomposed into its main stages, and each stage is described in detail. This leads into a discussion of the complete natural feature registration pipeline, highlighting common issues encountered at each step, and discussing the possible solutions for each issue. A new implementation of natural feature registration called the Optical-flow Perspective Invariant Registration Augmentation (OPIRA) is proposed, which provides vast improvements in robustness to perspective, rotation and changes in scale to popular registration algorithms such as SIFT, SURF, and the Ferns classifier. OPIRA is shown to improve perspective invariance on average by 15% for SIFT, 25% for SURF and 20% for the Ferns Classifier, as well as provide complete rotation invariance for the rotation dependent implementations of these algorithms. From the investigation into problems and potential resolutions at each stage during registration, each proposed solution is evaluated empirically against an external ground truth. The results are discussed and a conclusion on the improvements gained by each proposed solution and the feasibility of use in a real natural feature registration application is drawn. Finally, some applications which use the research contained within this thesis are described, as well as some future directions for the research.

This thesis focuses on improving the user experience for computer vision-based Augmented Reality (AR) applications on smartphones. The first part shows our proposed methods to enhance image binarisation. This improves the marker detection results in mobile AR applications. The comparisons of the original ARToolKit binarization method, our proposed histogram-based automatic thresholding and our histogram equalization based thresholding show that the histogram-based automatic thresholding produces a relatively better result under extreme and normal lighting conditions but slightly reduces the ARToolKit framerate. The second part introduces a new fast painterly rendering algorithm which produces an immersive experience for mobile AR users. The proposed algorithm has low complexity and achieves a real-time performance on smartphones. In addition, this study has carried out a preliminary experiment comparing mobile GPU-based image processing algorithms with CPU-based equivalent on smartphones. The result indicates that the GPU-based implementations perform better than the CPU when processing relatively large sized images.

The goal of this research thesis is to explore and evaluate a novel interaction interface performing canonical manipulations in 3D space for Augmented Reality (AR) on handheld devices. Different from current handheld AR applications usually using touch-screen based interaction methods, we developed a 3D gesture based interaction approach for handheld AR using an attached RGB-Depth camera to provide intuitive 3D interaction experience in 3D space. By identifying fingertips and mapping their 3D positions into the coordinate system of AR virtual scene, our proposed method allows users to perform operations on virtual objects using their fingers in midair with six-degrees-of-freedom (6DOF). We applied our methods in two systems: (1) a client-server handheld AR system, and (2) a standalone handheld tablet AR system. In order to evaluate the usability of our gesture-based interface we conducted a user study in which we compared the performance to a 2D touch-based interface. From the results, we concluded that traditional 2D touch-based interface performed faster than our proposed 3D gesture-based interface. However, our method proved a high entertainment value, suggesting great possibilities for leisure applications.

Outdoor tracking and registration are important enabling technologies for mobile augmented reality. Sensor fusion and image processing can be used to improve global tracking and registration for low-cost mobile devices with limited computational power and sensor accuracy. Prior research has confirmed the benefits of this approach with high-end hardware, however the methods previously used are not ideal for current consumer mobile devices. We discuss the development of a hybrid tracking and registration algorithm that combines multiple sensors and image processing to improve on existing work in both performance and accuracy. As part of this, we developed the Transform Flow toolkit, which is one of the first open source systems for developing and quantifiably evaluating mobile AR tracking algorithms. We used this system to compare our proposed hybrid tracking algorithm with a purely sensor based approach, and to perform a user study to analyse the effects of improved precision on real world tracking tasks. Our results show that our implementation is an improvement over a purely sensor fusion based approach; accuracy is improved up to 25x in some cases with only 2-4ms additional processing per frame, in comparison with other algorithms which can take over 300ms.

The two primary objectives for this thesis are (1) to understand the effectiveness of different types of mobile phone based advertisements and (2) to identify if the amount of time users spent viewing an advertisement can be used as a measure of advertising effectiveness. To achieve these objectives, this study first conducted qualitative studies consisting of a focus group with consumers and an interview with a mobile advertising technology provider. Qualitative study results identified the following variables of interest; vividness of the advertisement, product involvement, and consumer goals. Supported by existing literature on advertising, these variables were then used to develop a conceptual model outlining the relationship between the variables and measures of advertising effectiveness. To empirically examine this model, this study conducted a 3x2x4 experiment of high, medium and low advertisement mode vividness, high and low product involvement and four stages of pre-purchase consumer goals. A total of 288 responses were collected from a student sample from the University of Canterbury, New Zealand. The dependence relationships outlined in the conceptual model were then analysed using ANCOVA, logistic regression, linear regression, and various other non-parametric analysis techniques. The results of this study suggest that level of advertisement mode vividness and product involvement both exert a strong influence on the effectiveness of the advertisement. However, results on consumer goals suggest that the effectiveness of the advertisement is only affected by whether a consumer goal existed before viewing the advertisement. This study was unable to identify any relationship between the effectiveness of an advertisement and the amount of time users spent viewing an advertisement on a mobile phone.

We explore the integration of projected imagery with a physical book that acts as a tangible interface to multimedia data. Using a camera and projector pair, a tracking framework is presented wherein the 3D position of planar pages are monitored as they are turned back and forth by a user, and data is correctly warped and projected onto each page at interactive rates to provide the user with an intuitive mixed-reality experience. The book pages are blank, so traditional camera-based approaches to tracking physical features on the display surface do not apply. Instead, in each frame, feature points are independently extracted from the camera and projector images, and matched to recover the geometry of the pages in motion. The book can be loaded with multimedia content, including images and videos. In addition, volumetric datasets can be explored by removing a page from the book and using it as a tool to navigate through a virtual 3D volume.

This thesis presents four case studies from the Finnish museum sector that are using mobile technologies in diverse ways to enhance the museum experience. At the National Museum of Finland, the mobile solution functions as an aid for providing translations in different languages and thus improving the aesthetic appearance of the exhibition. At Tampere Art Museum, the outdoors mobile tour extends the museum visit outside the physical walls of the building. At Helsinki City Museum the mobile phone is perceived as a communication tool. At Luostarinmäki Handicrafts Museum the Augmented Reality game combines digital narrative with real-world events, creating a solution that is both entertaining and informative.   These solutions are analysed in terms of the Contextual Model, developed by Falk and Dierking. The model divides the museum visit into three overlapping and interacting spheres – personal, social and physical. This thesis looks at how mobile solutions may enhance or hinder the museum experience in regards to each of these three spheres. Additionally, the model is compared with the results of a visitor research conducted at the National Museum of Finland in October 2013. The aim of the thesis is to identify the most successful features of these solutions and to explore how the field could be developed in the future.

Recent rapid development of cost-effective, accurate digital imaging sensors, high-speed computational hardware, and tractable design software has given rise to the growing field of augmented reality in the computer vision realm. The system design of a 'Digital Whiteboard' system is presented with the intention of realizing a practical, cost-effective and publicly available spatial augmented reality system. A Microsoft Kinect sensor and a PC projector coupled with a desktop computer form a type of spatial augmented reality system that creates a projection based graphical user interface that can turn any wall or planar surface into a 'Digital Whiteboard'. The system supports two kinds of user inputs consisting of depth and infra-red information. An infra-red collimated light source, like that of a laser pointer pen, serves as a stylus for user input. The user can point and shine the infra-red stylus on the selected planar region and the reflection of the infra-red light source is registered by the system using the infra-red camera of the Kinect. Using the geometric transformation between the Kinect and the projector, obtained with system calibration, the projector displays contours corresponding to the movement of the stylus on the 'Digital Whiteboard' region, according to a smooth curve fitting algorithm. The described projector-based spatial augmented reality system provides new unique possibilities for user interaction with digital content.

The synergy of smartphone, mobile applications (apps) and Augmented Reality (AR) technology has the potential to mediate tourism experiences to great extents. The advent of AR apps on smartphones provides a dynamic solution for tourists by helping convey destinations’ meanings and creates positive experiences via interactive tourist information and services almost anywhere, anytime. As a result, tourists are increasingly using AR travel apps at destinations to create more memorable travel experiences. Despite vast literature on tourists’ experiences, there is limited research focusing on understanding the use of smartphones and AR apps for tourism. A critical review of the literature indicates that there is a need to develop a richer theoretical framework that describes the use of smartphones and AR apps for travel. In addition, there is a need to understand tourists’ experiences with smartphone-mediated technology. In particular, literature on understanding of the use of smartphones and apps for travel is largely established from a quantitative perspective, and it is argued that this perspective cannot provide an in-depth understanding of the mechanisms that affect the use of smartphones and travel apps, which in turn shapes the travel experience. The present qualitative study was designed to understand the current use and possible benefits of smartphone-mediated tourism experiences with AR apps. Specifically, the purpose of this study was to examine the influence of AR apps on tourists’ experiences. The study sought to understand how tourists used AR apps, which specific interactions with the mobile devices were afforded, what emotions were evoked through interaction with the AR technology, and how the technology mediated tourist’s experiences. Based on this notion, the study attempted to generate an inductive middle-range theory on smartphone-mediated tourism experiences using grounded theory method. An iPhone AR app was developed for the Texas A&M University campus to better understand how tourists used the AR app and how this use influenced their travel experiences. Forty-four participants inclusive of students, prospective students and visitors of Texas A&M University were recruited for the study. To aid theory building and enhance the solidity of the smartphone-mediated travel experience theory, the study included a control group, which involve individual, group and guided tours that only use a brochure/campus booklet or listen to a human tour guide. The AR app was tested on 10 individuals and 10 groups. For the control group, 6 individuals and 6 groups used brochure/campus booklet while touring the sites and 6 individuals and 6 groups listen to the tour-guide. This comparison provided detailed understanding of what happens in the absence of technology in travel experience, and a focus on apprehending what AR technology adds. Data were collected through face-to-face in-depth interviews with the participants and then transcribed and imported into ATLAS.TI 7.0 software for analysis. A grounded theory approach was used to analyze the data. The interview data were coded and presented in five major sections representing the research questions. The results of the study provided theoretical contributions in understanding the smartphone-mediated tourism experiences and offered practical implications for app design and interpretative services for tourist sites.

One of the most challenging tasks for augmented reality systems is that of position and attitude determination in outdoor unprepared environments. Augmented reality, a technology that overlays digital information with views of the real world, requires accurate and precise position and attitude determination to operate effectively. For small (often indoor) areas, careful preparation of the environment can allow for augmented reality systems to work successfully. In large outdoor environments, however, such preparation is often impractical, time-consuming and costly. This thesis aims to investigate the development of a position and attitude determination component for augmented reality systems capable of operation in outdoor unprepared environments. The hypothesis tested in this investigation is that the integration of Global Positioning System (GPS), Dead Reckoning (DR) and map matching techniques enables the continuous and accurate real-time visual alignment of three-dimensional data with objects in the perspective view of a user operating in outdoor unprepared environments.