A key problem in mobile computing is providing people access to necessary cyber-information associated with their surrounding physical objects. Mobile augmented reality is one of the emerging techniques that address this key problem by allowing users to see the cyber-information associated with real-world physical objects by overlaying that cyber-information on the physical objects's imagery. As a consequence, many mobile augmented reality approaches have been proposed to identify and visualize relevant cyber-information on users' mobile devices by intelligently interpreting users' positions and orientations in 3D and their associated surroundings. However, existing approaches for mobile augmented reality primarily rely on Radio Frequency (RF) based location tracking technologies (e.g., Global Positioning Systems or Wireless Local Area Networks), which typically do not provide sufficient precision in RF-denied areas or require additional hardware and custom mobile devices. To remove the dependency on external location tracking technologies, this dissertation presents a new vision-based context-aware approach for mobile augmented reality that allows users to query and access semantically-rich 3D cyber-information related to real-world physical objects and see it precisely overlaid on top of imagery of the associated physical objects. The approach does not require any RF-based location tracking modules, external hardware attachments on the mobile devices, and/or optical/fiducial markers for localizing a user's position. Rather, the user's 3D location and orientation are automatically and purely derived by comparing images from the user's mobile device to a 3D point cloud model generated from a set of pre-collected photographs. A further challenge of mobile augmented reality is creating 3D cyber-information and associating it with real-world physical objects, especially using the limited 2D user interfaces in standard mobile devices. To address this challenge, this research provides a new image-based 3D cyber-physical content authoring method designed specifically for the limited screen sizes and capabilities of commodity mobile devices. This new approach does not only provide a method for creating 3D cyber-information with standard mobile devices, but also provides an automatic association of user-driven cyber-information with real-world physical objects in 3D. Finally, a key challenge of scalability for mobile augmented reality is addressed in this dissertation. In general, mobile augmented reality is required to work regardless of users' location and environment, in terms of physical scale, such as size of objects, and in terms of cyber-information scale, such as total number of cyber-information entities associated with physical objects. However, many existing approaches for mobile augmented reality have mainly tested their approaches on limited real-world use-cases and have challenges in scaling their approaches. By designing fast direct 2D-to-3D matching algorithms for localization, as well as applying caching scheme, the proposed research consistently supports near real-time localization and information association regardless of users' location, size of physical objects, and number of cyber-physical information items. To realize all of these research objectives, five research methods are developed and validated: 1) Hybrid 4-Dimensional Augmented Reality (HD4AR), 2) Plane transformation based 3D cyber-physical content authoring from a single 2D image, 3) Cached k-d tree generation for fast direct 2D-to-3D matching, 4) double-stage matching algorithm with a single indexed k-d tree, and 5) K-means Clustering of 3D physical models with geo-information. After discussing each solution with technical details, the perceived benefits and limitations of the research are discussed with validation results. / Ph. D.

Common Objects in Context for bridge inspection (COCO-Bridge) was introduced for use by unmanned aircraft systems (UAS) to assist in GPS denied environments, flight-planning, and detail identification and contextualization, but has far-reaching applications such as augmented reality (AR) and other artificial intelligence (AI) platforms. COCO-Bridge is an annotated dataset which can be trained using a convolutional neural network (CNN) to identify specific structural details. Many annotated datasets have been developed to detect regions of interest in images for a wide variety of applications and industries. While some annotated datasets of structural defects (primarily cracks) have been developed, most efforts are individualized and focus on a small niche of the industry. This effort initiated a benchmark dataset with a focus on structural details. This research investigated the required parameters for detail identification and evaluated performance enhancements on the annotation process. The image dataset consisted of four structural details which are commonly reviewed and rated during bridge inspections: bearings, cover plate terminations, gusset plate connections, and out of plane stiffeners. This initial version of COCO-Bridge includes a total of 774 images; 10% for evaluation and 90% for training. Several models were used with the dataset to evaluate model overfitting and performance enhancements from augmentation and number of iteration steps. Methods to economize the predictive capabilities of the model without the addition of unique data were investigated to reduce the required number of training images. Results from model tests indicated the following: additional images, mirrored along the vertical-axis, provided precision and accuracy enhancements; increasing computational step iterations improved predictive precision and accuracy, and the optimal confidence threshold for operation was 25%. Annotation recommendations and improvements were also discovered and documented as a result of the research. / MS / Common Objects in Context for bridge inspection (COCO-Bridge) was introduced to improve a drone-conducted bridge inspection process. Drones are a great tool for bridge inspectors because they bring flexibility and access to the inspection. However, drones have a notoriously difficult time operating near bridges, because the signal can be lost between the operator and the drone. COCO-Bridge is an imagebased dataset that uses Artificial Intelligence (AI) as a solution to this particular problem, but has applications in other facets of the inspection as well. This effort initiated a dataset with a focus on identifying specific parts of a bridge or structural bridge elements. This would allow a drone to fly without explicit direction if the signal was lost, and also has the potential to extend its flight time. Extending flight time and operating autonomously are great advantagesfor drone operators and bridge inspectors. The output from COCO-Bridge would also help the inspectors identify areas that are prone to defects by highlighting regions that require inspection. The image dataset consisted of 774 images to detect four structural bridge elements which are commonly reviewed and rated during bridge inspections. The goal is to continue to increase the number of images and encompass more structural bridge elements in the dataset so that it may be used for all types of bridges. Methods to reduce the required number of images were investigated, because gathering images of structural bridge elements is challenging,. The results from model tests helped build a roadmap for the expansion and best-practices for developing a dataset of this type.

<p dir="ltr">Immersive applications such as augmented reality (AR) and mixed reality (MR) often need to perform latency-critical analytics tasks on every frame captured on camera. These tasks, often powered by deep neural networks (DNNs) for their superior accuracy, necessitate offloading to edge servers with GPUs due to their computational intensity. Achieving high accuracy and efficient AR task offloading faces two fundamental challenges untapped by prior work: (1) In practice, multiple DNN-supported tasks need to offload concurrently to achieve the app functionality -- how to schedule such offloaded tasks on the client which compete for shared edge server resources to maximize the app QoE? (2) Concurrent AR clients from a large user base offload to a cluster of GPU servers -- how to schedule the offloaded tasks on the servers to maximize the number of clients served and lower the operating cost?</p><p dir="ltr">To tackle the first challenge, we design a framework, AccuMO, that balances the offloading frequencies of different tasks by dynamically scheduling the offloading of multiple tasks from an AR client to an edge server, thereby optimizing the overall accuracy across tasks and hence app QoE. Our design employs two novel ideas: (1) task-specific lightweight models that predict offloading accuracy drop as a function of offloading frequency and frame content, and (2) a general two-level control feedback loop that concurrently balances offloading among tasks and adapts between offloading and using local algorithms for each task.</p><p dir="ltr">We tackle the challenge of supporting concurrent AR clients in two steps. We first focus on maximizing the capacity of individual edge servers, where we present ARISE, which untangles the intricate interplay between per-client offloading schedule and batched inference on the server by proactively coordinating offloading requests from different AR clients. In the second step, we focus on a cluster setup of heterogeneous GPU servers which exposes the synergy between diversity in both DNN layers and GPU architectures, manifesting as comparable inference latency for many layers in DNN models when running on low-class and high-class GPUs. We exploit such overlooked capability of low-class GPUs using pipeline parallelism and present a novel inference serving system, IPIPE, that employs pool-based pipeline parallelism with a mixed-integer linear programming (MILP)-based control plane and a data plane that performs resource reservation-based adaptive batching.</p>

Robotic assistance is applied in orthopedic interventions for pedicle screw placement (PSP). While current robots do not act autonomously, they are expected to have higher autonomy under surgeon supervision in the mid-term. Augmented reality (AR) is promising to support this supervision and to enable human–robot interaction (HRI). To outline a futuristic scenario for robotic PSP, the current workflow was analyzed through literature review and expert discussion. Based on this, a hypothetical workflow of the intervention was developed, which additionally contains the analysis of the necessary information exchange between human and robot. A video see-through AR prototype was designed and implemented. A robotic arm with an orthopedic drill mock-up simulated the robotic assistance. The AR prototype included a user interface to enable HRI. The interface provides data to facilitate understanding of the robot’s ”intentions”, e.g., patient-specific CT images, the current workflow phase, or the next planned robot motion. Two-dimensional and three-dimensional visualization illustrated patient-specific medical data and the drilling process. The findings of this work contribute a valuable approach in terms of addressing future clinical needs and highlighting the importance of AR support for HRI.

No / In this chapter, augmented space is described as the layering of media technologies onto the physical space of the city. The approach assesses salient aspects of the experience of space in everyday life, the city and urban space more generally. The chapter discusses these in relation to the deployment of augmenting technologies and modes of display associated with augmented reality, new and digital media: visual or otherwise. Selected work, carried out in relation to culture, leisure and tourism is assessed. These case studies indicate the potential of augmented reality in areas of a) urban design, b) tourism and heritage, and c) the promotion of cycling for health and the creation of alternative transport infrastructure. The main characteristics of AR and augmented space are presented. This is followed by a discussion and development of hybrid research tools and applied in two case studies with a view to providing a potential roadmap for future work in this area.

Purpose: The purpose of this thesis has been to investigate how companies report non-financial information and specifically volatile subjects such as technology. There are several different new technologies that all have different maturities which can affect how they are reported in annual reports. This thesis aims to understand how reporting of technology is affected by innovation, investments and stakeholder expectations. Theoretical perspectives: Stakeholder theory has been used as a primary theoretical background and framework, in conjunction with agency theory to gain a deeper theoretical understanding of non-financial reporting. Methodology: 30 annual reports from five years (2018-2022) and six tech-companies listed on the Swedish stock market have been content analysed with the use of a variation of keyword frequency analysis. The sample companies represent different segments within the tech-industry to provide the thesis with a wider perspective. Furthermore, the method of this thesis has been a mixture between a positivist ontology with constructivist epistemology and the chosen theories has been used in an abductive manner where findings are reflected against the theories. Additionally, the method of data collection has been a mixture between quantitative and qualitative methods, where qualitative data has been quantified and analysed graphically. Findings: The findings indicate that the sample companies seem to be aware of their stakeholders' perception of them as tech-companies, which affect how they report on technology. It is likely that tech-companies increasingly report on new technologies to maintain their perception as tech-companies. Furthermore, the findings suggest that companies report more heavily on technologies when investments into the given technology have been made. The findings are in line with concepts from both agency theory and stakeholder theory.

Yes / This is the twelfth workshop to be organised under the postgraduate programmes in electrical and electronic engineering (EEE). In total, thirty-four papers from forty-nine submissions have been selected for the Proceedings. The Proceedings comprises eleven themes, which reflect today's research agenda.

Augmented reality, place-based games utilize GPS-enabled maps and mobile media recording devices to shift traditional classroom activities into real-world contexts. AR-games for second language learning is a new field of research, and few studies have examined the kinds of face-to-face interactions players engage in during AR-games. Using intensive, multi-camera video data of English language learners playing an AR-game, ChronoOps, this thesis describes how groups start and stop walking during gameplay. The method used is conversation analysis, and this study draws from theories of embodied and distributed cognition, situated learning, and interactional competence Walking to and from various destinations as a group is an important action for accomplishing the ChronoOps game. Thus, starting and stopping are sites where players orient to the tasks and environment of the game. Results show that starts and stops are projectable and accountable actions comprised of multiple semiotic fields including linguistic, gestural, and embodied practices. Furthermore, starts and stops are contingent on players' orientation to their place within the campus and game destinations, but also their place within the locally constructed nature of the AR-game task organization. These findings have implications for future research theories of learning in SLA and AR-games.

Approved for public release, distribution is unlimited / This thesis addresses the need of Naval Aviation Maintenance to streamline and more effectively manage the process of technical consultation aboard deployed aircraft carriers. The current process involves the physical transportation of an appropriate technician to the carrier to perform required maintenance and/or repairs. In light of the technology currently available this process becomes obviously obsolete, overly costly and needlessly time consuming. By implementing wireless technology in combination with advanced software allowing the virtual collaboration of parties widely separated by geographical distance the Navy can establish a "virtual technical presence" onboard aircraft carriers wherever they may be in the world. This thesis will describe how the fusion of wearable computing, augmented reality, intelligent agents coupled with CoABS, and a modern collaborative software application can revolutionize Naval aviation maintenance as we know it. The technology is there - it only remains for the Navy to leverage it and take advantage of the significant returns that it will provide. The implementation of this technology will allow maintainers onboard deployed aircraft carriers to consult in an augmented virtual environment with technical assets on the shore. These shore-based assets will then be able to "walk" deployed personnel through complicated repair procedures in a matter of minutes or hours as opposed to the previous need to wait for days for the technician to arrive. This is a bold and innovative new concept that will allow commands at sea to increase their levels of combat readiness and allow them the ability to respond to ever changing mission needs. Turn around times for the repair of critical parts and assemblies will be reduced and readiness levels elevated. The ultimate goal of any command is mission accomplishment. This system will aid commands in achieving that all important goal. / Lieutenant, United States Navy

Approved for public release, distribution is unlimited / Current network administrators use network management software to monitor and control elements within a network. This is largely a manual process since managers must interrogate devices individually and evaluate performance statistics manually. The systems provide multiple views on network data but lack capabilities that allow operators to visualize network performance. Since personnel are required to identify problems, interpret potential solutions, and decide on appropriate corrective measures without automatic assistance, maintaining and solving problems for a network can be time-consuming and complex significantly reducing network efficiency. Since FORCENET is a heterogeneous concept that combines various C4I networks, sensors, weapon systems, and platforms, a new model must be developed for network operations. This paper researches an improved model for fleet network operations management for distributed sea-based forces using existing technologies. Combining a collaborative tool, Decision Support System (DSS), and Augmented Reality (AR) imagery transforms Naval information network management from a "minimum threshold" to an "operations fusion" perspective. Little is known about AR technologies, but the potential exists for virtual network operations centers that can remotely direct networks for sea and shore assets through collaborative efforts. The product of this paper will serve as a baseline for network operations in the network centric environment. / Lieutenant, United States Naval Reserve