Drivers require a steady stream of relevant but focused visual input to make decisions. Most driving information comes from the surrounding environment so keeping drivers' eyes on the road is paramount. However, important information still comes from in-vehicle displays. With this in mind, there has been renewed recent interest in delivering driving in-formation via head-up display. A head-up display (HUD) can present an image directly on-to the windshield of a vehicle, providing a relatively seamless transition between the display image and the road ahead. Most importantly, HUD use keeps drivers' eyes focused in the direction of the road ahead. The transparent display coupled with a new location make it likely that HUDs provide a fundamentally different driving experience and may change the way people drive, in both good and bad ways. Therefore, the objectives of this work were to 1) understand changes in drivers' glance behaviors when using different types of displays, 2) investigate the impact of HUD position on glance behaviors, and 3) examine the impact of HUD graphic type on drivers' behaviors. Specifically, we captured empirical data regarding changes in driving behaviors, glance behaviors, reported workload, and preferences while driving performing a secondary task using in-vehicle displays. We found that participants exhibited different glance behaviors when using different display types, with participants allocating more and longer glances towards a HUD as compared to a traditional Head-Down Display. However, driving behaviors were not largely affected and participants reported lower workload when using the HUD. HUD location did not cause large changes in glance behaviors, but some driving behaviors were affected. When exam-ining the impact of graphic types on participants, we employed a novel technique for ana-lyzing glance behaviors by dividing the display into three different areas of interest relative to the HUD graphic. This method allowed us to differentiate between graphic types and to better understand differences found in driving behaviors and participant preferences than could be determined with frequently used glance analysis methods. Graphics that were fixed in place rather than animated generally resulted in less time allocated to looking at the graphics, and these changes were likely because the fixed graphics were simple and easy to understand. Ultimately, glance and driving behaviors were affected at some level by the display type, display location, and graphic type as well as individual differences like gender and age. / Ph. D. / Drivers gather most of the information that they need to drive by looking at the world around them and at displays within the vehicle. However, research has shown that looking down at vehicle displays can be distracting to drivers which could be unsafe. Therefore, automotive manufacturers look for new ways to help decrease driver distraction, and one potential solution to this problem is the introduction of head-up displays (HUDs). By displaying a graphic on a see-through surface, like a windshield, we can add information to the world in front of the driver. This means that drivers no longer have to physically look away from the road to gather information, and they may be able to use peripheral vision to help drive while they look at the display. While the technology is promising, it is important that we fully understand other impacts of this technology on drivers before we widely incorporate it into vehicles. Therefore, the purpose of this work is to understand how HUDs change drivers’ ability to drive and their glance patterns as they gather the visual information needed to drive safely. We examined differences between HUDs and traditional displays found in vehicles. We then gathered data regarding the location of HUDs. Finally, we tested different graphics displayed on the HUD. In addition to gathering data about glance and driving behaviors, we also gathered data about drivers’ preferences and experiences with the displays. HUDs may tempt drivers to look away from the road for longer periods of time without negatively affecting their driving behaviors. Different HUD locations did not cause large differences in glance behaviors but did have some impact on driving behaviors. Finally, different graphics resulted in very different glance behaviors without significantly changing driving behaviors. These results suggest that HUDs may capture drivers’ attention and cause drivers to be less observant of other elements around them as they drive. However, because different graphics result in different glance patterns, with careful design we may be able to help drivers keep their eyes on the road while safely gathering necessary information from the vehicle.

The underground limestone mining industry is a small, but growing segment of the U.S. crushed stone industry. However, its fatality rate has been amongst the highest of the mining sector in recent years due to ground control issues related to ground collapses. It is therefore important to improve the engineering design, monitoring and visualization of ground control by utilizing new technologies that can help an underground limestone company maintain a safe and productive operation. Photogrammetry and laser scanning are remote sensing technologies that are useful tools for collecting three-dimensional spatial data with high levels of precision for many types of mining applications. Due to the reality of budget constraints for many underground stone mining operations, this research concentrates on photogrammetry as a more accessible technology for the average operation. Despite the challenging lighting conditions and size of underground limestone mines that has previous hindered photogrammetric surveys in these environments, over 13,000 photographic images were taken over a 3-year period in active mines to compile these models. This research summarizes that work and highlights the many applications of terrestrial close-range photogrammetry, including practical methodologies for implementing the techniques in working operations to better visualize hazards and pragmatic approaches for geotechnical analysis, improved engineering design and monitoring. / M.S. / The underground limestone mining industry is a small, but growing segment of the U.S. crushed stone industry. However, its fatality rate has been amongst the highest of the mining sector in recent years due to ground control issues related to ground collapses. It is therefore important to improve the engineering design, monitoring and visualization of ground control by utilizing new technologies that can help maintain safe and productive underground stone operations. Photogrammetry and laser scanning are remote sensing technologies that are useful tools for collecting three-dimensional spatial data with high levels of precision for many different mining applications. Due to the reality of budget constraints for many mining operations, this research concentrates on photogrammetry as a more accessible technology for the average operation, despite the challenging lighting conditions and expansive size of underground limestone mines that has previous hindered photogrammetric surveys in these environments. This research focuses on the applications of photogrammetry in underground stone mines and practical methodologies for implementing the techniques in working operations to better visualize hazards for improved engineering design and infrastructure management.

The immersive virtual reality (VR) and the optical see-through augmented reality (AR) are expected to revolutionize human lives in work, education, entertainment, healthcare, spatial computing, and digital twins, just to name a few. Next-generation VR/AR devices should exhibit a wide field-of-view (FoV), crisp image without screen-door effect, high dynamic range, compact form factor and lightweight, and low power consumption. Such demanding requirements pose a significant challenge to traditional direct-view display panels. To address these technical challenges, novel approaches need to be proposed. This dissertation is devoted to developing next-generation high-performance display light engines toward high resolution density, high optical efficiency, wide color gamut, and small form factor. These emerging solutions will fuel the growth and accelerate the widespread applications of VR/AR devices. In Chapter 2, we propose practical measurement methods to characterize the halo artifacts of miniature light-emitting diode (mini-LED) backlit liquid crystal displays (LCDs). After measuring and characterizing a high dynamic range (HDR) light engine, we propose and develop field sequential color (FSC) LCDs for high-end virtual reality (VR) devices in Chapter 3. Such an FSC LCD can triple the resolution density and optical efficiency via eliminating color filters. To further mitigate the color breakup (CBU), we also propose to combine mini-LEDs with FSC LCDs to enable progressive emission and achieve a higher frame rate (~ 600 Hz). To quantitatively compare the CBUs corresponding to simultaneous emission, progressive emission, and stencil algorithm, we adopt the CIEDE2000 color difference as a metric. Quantitative simulation results of the CBU indicate that a 600-Hz subframe rate can help mitigate the CBU dramatically. Micro organic light-emitting diode (micro-OLED) exhibiting high-resolution density and high contrast ratio is another type of display for high-end VR devices. More specifically, white micro-OLED is currently employed because it helps ease the manufacturing difficulty. In Chapter 4, we optimize the layer thicknesses to achieve a maximum efficiency while keeping a decent color gamut. We also push the limit of color gamut toward ~ 95% Rec. 2020. Lastly, liquid-crystal-on-silicon (LCoS) offers great potential for achieving high-efficiency and high-resolution waveguide-based AR displays. In Chapter 5, several strategies are proposed and developed to improve the performance of LCoS microdisplays and enable a small pixel size. In Chapter 6, we briefly summarize our major accomplishments.

This study investigates the feasibility and performance of SLAM (Simultaneous Localization and Mapping) as a service (SLAM-as-a-Service) for outdoor augmented reality (AR) applications. Given the rapid advancements in AR technology, integrating lightweight AR glasses with real-time SLAM capabilities poses significant challenges, particularly due to the computational demands of SLAM algorithms and the limited hardware capacity of AR devices. This study proposes a scalable SLAM-as-a-Service framework that offloads intensive computational tasks to remote servers, leveraging cloud and edge computing resources. The ORB-SLAM3 algorithm, known for its robustness and real-time processing capabilities, was adapted and implemented in a service-oriented architecture. The framework was evaluated using the EuRoC dataset to benchmark processing speed, accuracy, and round trip time. The results indicate that while the proposed SLAM-as-a-Service model shows promise in handling high computational loads, several obstacles need to be addressed to achieve minimal round trip time and ensure a seamless AR experience. This thesis contributes to the development of scalable and efficient AR solutions by addressing the limitations of on device processing and highlighting the potential of cloud-based services in enhancing the performance and feasibility of AR applications in dynamic outdoor environments.

Over the past several years, machine learning has enabled incredible progress on many tasks, such as mastering board games, recognizing objects, conversing in natural language, and generating images or videos. Despite these accomplishments, state-of-the-art techniques in artificial intelligence lack the foundations necessary to flexibly and robustly understand and manipulate their three-dimensional spatial surroundings. For instance, before their second birthday, children learn that objects persist during occlusion, they know how containment works, and they are surprised by novel physics. In contrast, a true notion of object permanence has remained elusive for computer vision, despite its vitality in perceiving and interacting with everyday situations. In this thesis, I will outline my work on enhancing spatial reasoning within dynamic scenes, where I have integrated machine learning, intuitive physics, geometry, and world knowledge to create powerful frameworks that can capture, represent, and generate their complex, cluttered visual environment. Specifically, I will present models to reconstruct 4D scenes, track objects through occlusions, and perform dynamic view synthesis, all from a single camera viewpoint, and often successfully generalizing to real-world settings. These capabilities are pivotal for applications in embodied intelligence (such as robotics and self-driving), content creation and editing, or augmented and mixed reality, where machines need to accurately represent their surroundings and deeply understand how they evolve over time.

The future of work is rapidly evolving, particularly in knowledge-based professions such as programming, engineering, and scientific research. These fields traditionally rely on physical monitors in office settings. However, with the rise of hybrid work models fueled by technological advances and the COVID-19 pandemic, there is a growing need for flexible and portable display solutions. Workers can now operate from remote settings, their homes, or mobile scenarios while still requiring substantial screen space to complete their tasks. This dissertation investigates the design and evaluation of virtual displays rendered through head-worn displays (HWDs) as a promising alternative, aiming to optimize them for productive work. These displays offer flexibility, allowing users to achieve large monitor spaces in virtual or augmented reality environments, adaptable to any location. We aim to answer three research questions: (1) ``How does replacing or extending physical monitors with virtual displays using current technology impact the user experience of productivity tasks?'', (2) ``How can we take advantage of the spatial flexibility property of virtual displays to eliminate screen boundaries and increase the amount of space available to users?'', and (3) ``How can we leverage properties of virtual displays to design techniques that minimize overhead in window management tasks without reducing user freedom?'' Through careful interface design and empirical user studies, we seek to understand how to leverage the unique capabilities of HWDs to enhance productivity, preparing the groundwork for future virtual display systems as technology advances. / Doctor of Philosophy / The future of work is rapidly evolving, particularly in knowledge-based professions such as programming, engineering, and scientific research. These fields traditionally rely on physical monitors in office settings. However, with the rise of hybrid work models fueled by technological advances and the COVID-19 pandemic, there is a growing need for flexible and portable display solutions. Workers can now operate from remote settings, their homes, or mobile scenarios while still requiring substantial screen space to complete their tasks. This dissertation investigates the design and evaluation of virtual displays rendered through virtual or augmented reality headsets as a promising alternative, aiming to optimize them for productive work. These displays offer flexibility, allowing users to achieve large monitor spaces while being adaptable to any location. Our research aims to understand how replacing or extending physical monitors with virtual displays impacts productivity, how to maximize the available space and organization, and how to design techniques that make it easier to organize and access windows. Through careful design of these displays and testing with users, we seek to understand how we can take advantage of the capabilities provided by AR/VR headsets to enhance productivity and pave the way for future displays.

Walking is a natural part of our lives and is also becoming increasingly common in mixed reality. Wireless headsets and improved tracking systems allow us to easily navigate real and virtual environments by walking. In spite of the benefits, walking brings challenges to the design of new systems. In particular, designers must be aware of cognitive and motor requirements so that walking does not negatively impact the main task. Unfortunately, those demands are not yet fully understood. In this dissertation, we present new scientific evidence, interaction designs, and analysis of the role of walking in different mixed reality applications. We evaluated the difference in performance of users walking vs. manipulating a dataset during visual analysis. This is an important task, since virtual reality is increasingly being used as a way to make sense of progressively complex datasets. Our findings indicate that neither option is absolutely better: the optimal design choice should consider both user's experience with controllers and user's inherent spatial ability. Participants with reasonable game experience and low spatial ability performed better using the manipulation technique. However, we found that walking can still enable higher performance for participants with low spatial ability and without significant game experience. In augmented reality, specifying points in space is an essential step to create content that is registered with the world. However, this task can be challenging when information about the depth or geometry of the target is not available. We evaluated different augmented reality techniques for point marking that do not rely on any model of the environment. We found that triangulation by physically walking between points provides higher accuracy than purely perceptual methods. However, precision may be affected by head pointing tremors. To increase the precision, we designed a new technique that uses multiple samples to obtain a better estimate of the target position. This technique can also be used to mark points while walking. The effectiveness of this approach was demonstrated with a controlled augmented reality simulation and actual outdoor tests. Moving into the future, augmented reality will eventually replace our mobile devices as the main method of accessing information. Nonetheless, to achieve its full potential, augmented reality interfaces must support the fluid way we move in the world. We investigated the potential of adaptation in achieving this goal. We conceived and implemented an adaptive workspace system, based in the study of the design space and through user contextual studies. Our final design consists in a minimum set of techniques to support mobility and integration with the real world. We also identified a set of key interaction patterns and desirable properties of adaptation-based techniques, which can be used to guide the design of the next-generation walking-centered workspaces. / Ph. D. / Until recently, walking with virtual and augmented reality headsets was restricted by issues such as excessive weight, cables, tracking limitations, etc. As those limits go away, walking is becoming more common, making the user experience closer to the real world. If well explored, walking can also make some tasks easier and more efficient. Unfortunately, walking reduces our mental and motor performance and its consequences in interface design are not fully understood. In this dissertation, we present studies of the role of walking in three areas: scientific visualization in virtual reality, marking points in augmented reality, and accessing information in augmented reality. We show that although walking reduces our ability to perform those tasks, careful design can reduce its impact in a meaningful way.

Current Augmented Reality devices rely heavily on real-time environment mapping to provide convincing world-relative experiences through user interaction with virtual content integrated into the real world. This mapping is obtained and updated through many different algorithms, but often results in holes and other mesh artifacts when generated in less ideal scenarios, like outdoors and with fast movement. In this work, we present the Interaction-Triggered Estimation of AR Object Placement on Indeterminate Meshes, a quick, interaction-triggered method to estimate the normal and position of missing mesh pieces in real-time with low computational overhead. We achieve this by extending the user's hand using a group of additional raycast sample points, aggregating results according to different algorithms, and then using the resulting values to place an object. This thesis will first cover problems with current mapping techniques, thoroughly explain the rationale and algorithms behind our method, and then evaluate our method using a user study. / Master of Science / Augmented Reality (AR) technologies have the potential to change all our lives for the better through tight and seamless integration into our daily lives. Crucial to this seamless integration is the ability for users to manipulate virtual AR objects and interact effortlessly with real-world features around them. In order to facilitate this interaction, AR devices often create 3D maps of the real world to allow the device to recognize and respect the geometry of the world around it. Unfortunately, many AR devices still have trouble creating and maintaining these maps in challenging environments, like outdoors or when moving fast, so the resulting 3D maps of the environments have holes and inaccuracies, causing user interaction with the environment to be unreliable and breaking the seamless integration. While many solutions look toward more advanced algorithms that require more specialized sensors or next-gen AR devices to improve this mapping issue, we see an opportunity to enhance any existing 3D maps using a novel interaction aggregation approach that can theoretically work with any mapping technology. In this work, we present the Interaction-Triggered Estimation of AR Object Placement on Indeterminate Meshes, a work-in-progress application providing a quick, interaction-triggered method to estimate the normal and position of missing mesh in real-time with low computational overhead.

No / In recent times, marketers have started using augmented and virtual reality (AR-VR) to offer unique customer experiences and enhance engagement. While the use of AR-VR in marketing has become the ‘new normal', businesses are still struggling to use these tools to effectively attract and retain customers. Building on the existing literature, this chapter argues that overcoming these challenges requires understanding how to seamlessly integrate these tools throughout the customer journey. This will help businesses in curating AR-VR-enabled spaces that are aligned with customers' experiential preferences at different stages. In this chapter, the authors first summarize the impact of AR-VR technologies on customer engagement. Secondly, the challenges related to integrating these immersive technologies are highlighted. The authors then integrate the literature on AR-VR technologies and the customer journey and propose a framework and encourage academicians and practitioners to focus more on the different stages of the customer journey while designing AR-VR enabled marketing strategies.

Yes / The use of the internet and social media have changed consumer behavior and the ways in which companies conduct their business. Social and digital marketing offers significant opportunities to organizations through lower costs, improved brand awareness and increased sales. However, significant challenges exist from negative electronic word-of-mouth as well as intrusive and irritating online brand presence. This article brings together the collective insight from several leading experts on issues relating to digital and social media marketing. The experts’ perspectives offer a detailed narrative on key aspects of this important topic as well as perspectives on more specific issues including artificial intelligence, augmented reality marketing, digital content management, mobile marketing and advertising, B2B marketing, electronic word of mouth and ethical issues therein. This research offers a significant and timely contribution to both researchers and practitioners in the form of challenges and opportunities where we highlight the limitations within the current research, outline the research gaps and develop the questions and propositions that can help advance knowledge within the domain of digital and social marketing.