No / This report summarizes results of the first EC/NSF joint Advanced Research Workshop, which identified key research challenges and opportunities in information technology. The group agreed that the first joint research workshop should concentrate on the themes of human-centered computing and VEs. Human-centered computing is perceived as an area of strategic importance because of the move towards greater decentralization and decomposition in the location and provision of computation. The area of VEs is one where increased collaboration should speed progress in solving some of the more intractable problems in building effective applications

The number and types of input devices and related interaction technique types are growing rapidly. Innovative input devices such as game controllers are no longer used just for games, propriety consoles and specific applications, they are also used in many distributed virtual environments, especially the so-called serious virtual environments. In this dissertation a distributed, service based framework is presented to offer context-sensitive interaction interoperability that can support mapping between input devices and suitable application tasks given the attributes (device, applications, users, and interaction techniques) and the current user context without negatively impacting performances of large scale distributed environments. The mapping is dynamic and context sensitive taking into account the context dimensions of both the virtual and real planes. What device or device component to use, how and when to use them depend on the application, task performed, the user and the overall context, including location and presence of other users. Another use of interaction interoperability is as a testbed for input devices, and interaction techniques making it possible to test reality based interfaces and interaction techniques with legacy applications. The dissertation provides a description how the framework provides these affordances and a discussion of motivations, goals and the addressed challenges. Several proof of the concept implementations were developed and an evaluation of the framework performance (in terms of system characteristics) demonstrates viability, scalability and negligible delays. / Ph. D.

Macro Planning of construction projects is among the most challenging tasks faced by the project team. Decisions made during this process have a tremendous impact on the successful execution of the project from its early conceptual phases, through the project construction and completion. For a large majority of construction projects, the current planning practices remain manually based. General and project specific data are communicated among project participants through design drawings in a 2D paper-based format. Due to the interdependence between the different elements and the large amount of information that needs to be manually processed, the current manual implementation approach is very difficult to undertake, and imposes a heavy burden on the project team to carry out the planning process. Various research efforts have been undertaken in an attempt to capture current planning techniques and allow for the development of new innovative and automated ways in planning. The developed planning systems are characterized as responsive decision systems, relying mainly on programmed knowledge and heuristics for decision making, hence reducing or eliminating the role of the human planner. This research presents the framework for a new interactive planning environment called the Virtual Construction Environment (VCE) that supports the thinking process of the project team during the macro planning phase of design-build projects. Unlike previous responsive-type systems developed, the approach utilized in the VCE is supportive to the project team enabling them to be an active participant in the decision making process. The main purpose of the VCE is to assist the project team during decision making, by providing pertinent information necessary for making appropriate decisions in a structured format. This information may be organized, stored, and retrieved by the project team whenever needed during the virtual sessions. The VCE also provides the project team with appropriate tools to test different work execution and site layout planning scenarios early during project development. During the virtual sessions, the project team reconstructs the facility by bringing graphical elements together. The project team's movements and interactions are recorded to capture their thinking process on how to construct the facility (i.e. sequence of major assemblies). Other project participants can retrieve recorded decisions for further review or modification. The project team is also able to specify construction methods, and allocate resources required for the implementation of major assemblies. The VCE guides the project team to perform these interdependent planning functions interactively and concurrently. Using system graphical libraries, major equipment and temporary facilities can be superimposed and displayed as graphical objects for site layout planning. This enables the project team to visually check for space and accessibility conflicts during different virtual construction time intervals. In order to define required information in the VCE, the author has developed a MAcro Planning Information Classification (MAPIC) model under which information required for macro planning decision making could be classified and organized in a structured standardized format. The project team may then retrieve and utilize this information whenever needed during the virtual sessions. A prototype computer tool is developed to illustrate the framework of the VCE. The computer prototype is implemented using available commercial software tools. / Ph. D.

Spatial collaboration refers to collaboration activities involving physical space. It occurs every day as people work together to solve spatial problems, such as rearranging furniture or communicating about an environmental issue. In this work, we investigate how to support spatial collaboration when the collaborators are not colocated. We propose using shared, interactive representations of the space to support distributed, spatial collaboration. Our study examines viewpoint constraint techniques, which determine how the collaborators individually view the representation, and awareness techniques, which enable the collaborators to maintain an understanding of each other's work efforts. Our work consists of four phases, in which we explore a design space for interactive representations and examine the effects of different viewpoint constraint and awareness techniques. We consider situations where the collaborators use the same viewpoints, different viewpoints, and have a choice in viewpoint constraint techniques. In phase 1, we examine current technological support for spatial collaboration and designed two early prototypes. Phase 2 compares various two-dimensional map techniques, with the collaborators using identical techniques. Phase 3 focuses on three-dimensional virtual environment techniques, comparing similar and different frames of reference. The final phase reuses the favorable techniques from the previous studies and presents a novel prototype that combines both two-dimensional and three-dimensional representations. Each phase of this research is limited to synchronous communication activities and non-professional users working together on everyday tasks. Our findings highlight the advantages and disadvantages of the different techniques for spatial collaboration solutions. Also, having conducted multiple evaluations of spatial collaboration prototypes, we offer a common set of lessons with respect to distributed, spatial collaboration activities. This research also highlights the need for continued study to improve on the techniques evaluated and to consider additional spatial collaboration activities. / Ph. D.

Training is one of the most important and widely-used applications of immersive Virtual Reality (VR). Research has shown that Immersive Virtual Environments (IVEs) are beneficial for training motor activities and spatial activities, but it is unclear whether immersive VEs are beneficial for purely mental activities, such as memorizing a procedure. In this thesis, we present two experiments to identify benefits of immersion for a procedural training process. The first experiment is a between-subjects experiment comparing two levels of immersion in a procedural training task. For the higher level of immersion, we used a large L-shaped projection display. We used a typical laptop display for the lower level of immersion. We asked participants to memorize two procedures: one simple and the other complex. We found that the higher level of immersion resulted in significantly faster task performance and reduced error for the complex procedure. As result of the first experiment we performed a controlled second experiment. We compared two within-subjects variables namely environment and location under various treatments formed by combination of three between-subject variables namely Software Field Of View (SFOV), Physical FOV, Field Of Regard (FOR). We found that SFOV is the most essential component for learning a procedure efficiently using IVEs. We hypothesize that the higher level of immersion helped users to memorize the complex procedure by providing enhanced spatial cues, leading to the development of an accurate mental map that could be used as a memory aid. / Master of Science

On average, there are approximately ten fatal haul truck accidents per year in the United States. The most common causes for haul truck accidents include mechanical problems, inadequate training, and insufficient road/berm maintenance. Due to the frequency and magnitude of haul truck accidents, new training methods are being investigated. With the widespread availability of inexpensive and powerful computers and newer information technology, the ability to incorporate computer based training for miners is becoming more of a possibility. Computer based training is as effective in knowledge acquisition as traditional lecture, and computer based training can also lead to a significant increase in the retention of material. Studies have also shown that more engaging training methods lead to much more effective knowledge acquisition. A computer-based virtual environment training system was developed to supplement current new miner training and address the common causes of fatal accidents. The new training system is a virtual pre-shift inspection of a haul truck, and will train the beginner haul truck operator to identify parts which look defective compared to how the parts look normally. The training will increase the operator's ability to recognize problematic parts and correctly identify the corrective action needed. Increasing the quality of training by providing a very engaging simulated hands-on environment will lead to safer behaviors by the trainees, and ultimately fewer accidents and fatalities. / Master of Science

System control—the issuing of commands—is a critical, but largely unexplored task in 3D user interfaces (3DUIs) for immersive virtual environments (IVEs). System control techniques are normally encompassed by complex interfaces that define how these interaction techniques fit together, which we call system control interfaces (SCIs). Creating a testbed to evaluate these SCIs would be beneficial to researchers and would lead to guidelines for choosing a SCI for particular application scenarios. Unfortunately, a major problem in creating such a testbed is the lack of a standard task sequence—the order of operations in a system control task. In this research, we identify various task sequences, such as the Action-Object and Object- Action task sequences, and evaluate the effects that these sequences have on usability, in hopes of establishing a standard task sequence. Two studies were used to estimate the cognitive effort induced by task sequences and, hence, the effects that these sequences have on user performance. We found that sequences similar to the Object-Action task sequence induce less cognitive time than sequences similar to the Action-Object task sequence. A longitudinal study was then used to analyze user preferences for task sequences as novices became experienced users with using an interior design application. We found that novices and experienced users alike prefer sequences like the Object-Action over sequences like the Action-Object task sequence. / Master of Science

We present a resource allocation perspective to Quality of Service in Distributed Virtual Environments. The user of a DVE system will have improved Quality of Service if he/she is allocated the right amount of resources at the right time. Instead of allocating resources on a static basis, we adopt a dynamic need based resource allocation scheme that provides real-time resource allocation. Optimal resource assignments are calculated offline and a neural network is trained with the knowledge of optimal solutions from the offline Operations Research Techniques and it is then used to deliver near-optimal resource allocation decisions in real-time. We also present a case study of network bandwidth allocation and prove the usefulness of the technique. / Master of Science

Virtual Environment (VE) user interfaces do not generally support personalized interaction or the ability to adapt to the user. Our research is developing nuance-oriented interfaces that take advantage of subtle clues that users give in their actions. To that end, we have performed five experiments with the goal of recognizing and applying nuances for the task of selection. We found evidence that users adapt their behavior to the feedback given by the system, rather than using a preconceived mental model of the environment. In addition, users' behavior can be modeled by a reward function. Lastly, users interact by trading off exploration with exploitation. We propose a method of modeling this behavior using machine learning as future work. / Master of Science

The management of online learning environments so that they are effective and efficient presents a significant challenge for institutions and lecturers due to the complexity of requirements in the learning and teaching domain. The use of 3D Multi User Virtual Environments (MUVEs) for education introduces a novel set of management challenges. MUVEs were designed to cater for entertainment and commercial needs and as such do not intrinsically support managed learning. When MUVEs are used for educational purposes, forming 3D Multi User Learning Environments (MULEs), user support for learning management becomes an important factor. This thesis highlights the importance of managed learning in MULEs. It proposes a coordinated approach which accommodates the existing education institutional infrastructure. The research has focused on two very widely used and closely compatible MUVEs, Second Life (SL) and OpenSim. The thesis presents system and user studies that have been carried out on these selected MUVEs. The findings reveal the challenges that academics and students can experience if they do not have sufficient knowhow to manage learning activities in SL/OpenSim. User guidance and training tools were then developed for supporting learning management strategies in the context of SL/OpenSim and demonstrated in exemplar use-case scenarios. The user support models and tools which were developed have been extensively evaluated for their usability and educational value using diverse participant groups. The results validate the efficacy of these contributions, defending the research thesis. These contributions can be used in future research on managing MUVE supported education.