Virtual environment (VE) technology offers a viable training option for developing knowledge, skills and attitudes (KSA) within domains that have limited live training opportunities due to personnel safety and cost (e.g., live fire exercises). However, to ensure these VE training systems provide effective training and transfer, designers of such systems must ensure that training goals and objectives are clearly defined and VEs are designed to support development of KSAs required. Perhaps the greatest benefit of VE training is its ability to provide a multimodal training experience, where trainees can see, hear and feel their surrounding environment, thus engaging them in training scenarios to further their expertise. This work focused on enhancing situation awareness (SA) within a training VE through appropriate use of multimodal cues. The Multimodal Optimization of Situation Awareness (MOSA) model was developed to identify theoretical benefits of various environmental and individual multimodal cues on SA components. Specific focus was on benefits associated with adding cues that activated the haptic system (i.e., kinesthetic/cutaneous sensory systems) or vestibular system in a VE. An empirical study was completed to evaluate the effectiveness of adding two independent spatialized tactile cues to a Military Operations on Urbanized Terrain (MOUT) VE training system, and how head tracking (i.e., addition of rotational vestibular cues) impacted spatial awareness and performance when tactile cues were added during training. Results showed tactile cues enhanced spatial awareness and performance during both repeated training and within a transfer environment, yet there were costs associated with including two cues together during training, as each cue focused attention on a different aspect of the global task. In addition, the results suggest that spatial awareness benefits from a single point indicator (i.e., spatialized tactile cues) may be impacted by interaction mode, as performance benefits were seen when tactile cues were paired with head tracking. Future research should further examine theoretical benefits outlined in the MOSA model, and further validate that benefits can be realized through appropriate activation of multimodal cues for targeted training objectives during training, near transfer and far transfer (i.e., real world performance).

The U.S. military uses pattern recognition training to observe anomalies in human behavior. An examination of the pattern recognition training literature for Warfighters reveals a gap in training to discern patterns of human behavior in live environments. Additionally, the current state of warfare is evolving and requires operations to change. As a result, pattern recognition training must accommodate new practices to improve performance. A technique used to improve memory for identifying patterns in the environment is Kim's game. Kim's game establishes patterns to identify inanimate objects, of which information retains in memory for later recall. The paper discusses the fundamental principles of Kim's game applied to virtual Simulation-Based Training. The virtual version of Kim's game contains customized scenarios for training behavior cue analysis. Virtual agents display kinesic cues that exhibit aggressive (i.e., slap hands and clench fist) and nervous behaviors including wring hands and check six. This research takes a novel approach by animating the kinesics cues in the virtual version of Kim's game for pattern recognition training. Detection accuracy, response time, and false positive detection serve as the performance data for analysis. Additional survey data collected include engagement, flow, and simulator sickness. All collected data was compared to a control condition to examine its effectiveness of behavior cue detection. A series of one-way between subjects design ANOVA's were conducted to examine the differences between Kim's game and control on post-test performance. Although, the results from this experiment showed no significance in post-test performance, the percent change in post-test performance provide further insight into the results of the Kim's game and control strategies. Specifically, participants in the control condition performed better than the Kim's game group on detection accuracy and response time. However, the Kim's game group outperformed the control group on false positive detection. Further, this experiment explored the differences in Engagement, Flow, and Simulator Sickness after the practice scenario between Kim's game group and the control group. The results found no significant difference in Engagement, partial significance for Flow, and significant difference for Simulator Sickness between the Kim's game and control group after the practice scenario. Next, a series of Spearman's rank correlations were conducted to assess the relationships between Engagement, Flow, Simulator Sickness, and post-test performance, as well as examine the relationship between working memory and training performance; resulting in meaningful correlations to explain the relationships and identifying new concepts to explain unrelated variables. Finally, the role of Engagement, Flow, and Simulator Sickness as a predictor of post-test performance was examined using a series of multiple linear regressions. The results highlighted Simulator Sickness as a significant predictor of post-test performance. Overall, the results from this experiment proposes to expand the body of pattern recognition training literature by identifying strategies that enhance behavior cue detection training. Furthermore, it provides recommendations to training and education communities for improving behavior cue analysis. ?

The purpose of simulation is to avoid reality-based constraints by the implemen-tation of a synthetic model. Based on this advantage, interactive simulations have conquered all areas of applications from acquisition, and training, to research. Simulation results are transferred in many ways into reality and conclusions are drawn from the simulation to the application. Many anecdotal observations on human-in-the-loop simulations have shown a significant difference in actor behavior between simulations and reality-based applications. It seems that the factors that makes simulation so attractive, namely the absence of constraints and especially of imminent danger for persons and equipment, influence the behavior and thereby the performance of the user. These differences between simulation and reality may lead to false conclusions based on simulation results. The concept of perceiving a simulation as real and of being in the simulation is called sense of presence. This psychological construct can also be described as level of disbelief towards the simulation. Hence, differences in behavior are based on such users assessment of a simulation and subsequently are supposed to be mediated by a difference in presence. This research established significant differences in presence and performance between a simulation and a miniature-world teleoperation task. Presence and performance changed in identical tasks due to the application type and the connected danger to the robot. Also, the results supported a negative relationship between presence and performance: presence increased in the miniature-world and affected performance so that performance decreased. The causal relationship of application type→ presence→ performance was established and demands the examination of simulation based results with respect to the perceived danger to equipment, before they are transferred into the real application.

Navigation in large spaces is essential in any environment (both the real world and the virtual world) because one of the human fundamental needs is to know the surrounding environment and to freely navigate within the environment. For successful navigation in large-scale virtual environments (VEs), accurate spatial knowledge is required, especially in training and learning application domains. By acquiring accurate spatial knowledge, people can effectively understand spatial layout and objects in environments. In addition, spatial knowledge acquired from a large- scale VE can effectively be transferred to the real world activities. Numerous navigation techniques have been proposed to support successful navigation and effective spatial knowledge acquisition in large-scale VEs. Among them, walking-like navigation techniques have been shown to support spatial knowledge acquisition more effectively in large-scale VEs, compared to non-body-based and non-walking-based navigation techniques. However, walking-like navigation techniques in large-scale VEs still have some issues, such as whole-body fatigue, large-controlled-space and specialized system configuration that make the walking-like navigation techniques less convenient, and consequently less commonly used. Due to these issues, convenient non-walking-like navigation techniques are preferred although they are less effective for spatial learning. While most research and development efforts are centered around walking- like navigation techniques, a fresh approach is needed to effectively and conveniently support for human spatial learning. We propose an action-inspired approach, to design convenient and effective navigation techniques for supporting people to acquire accurate spatial knowledge acquisition or improve spatial learning. The action-inspired approach is based on our insights from learning, neuropsychological and neurophysiological theories. The theories suggest that action and perception are closely related and core elements of learning. Our observations indicated that specific body-parts are not necessarily related to learning. We identified two types of action-inspired approach, body-turn based and action-transferred. Body- turn based approach keeps body-turn but replaces cyclic leg-movements of original walking action with more convenient control to resolve the issues presented from walking-like navigation techniques. Action-transferred approach addresses the design trade-offs between effectiveness and convenience, the core concept of which is grounded in the motor equivalence theory. We provided two navigation techniques, body-turn based and action-transferred based ones, and demonstrated the benefits of our approach by evaluating these two navigation techniques for spatial knowledge acquisition in several empirical studies. We also developed our own walking-like navigation technique, Sensor- Fusion Walking-in-Place (SF-WIP) because we needed a reference navigation technique for estimating the effect of the action-transferred navigation technique on spatial knowledge acquisition compared to that of a walking-like navigation technique. We performed empirical user studies and the experimental results showed that body-turn based navigation technique was more effective for survey knowledge acquisition in a large-scale virtual maze, compared to a wand-joystick based common navigation technique (JS, i.e., non-body-based and non-walking-like navigation technique). However, no significant difference was found for route knowledge acquisition while the SF-WIP was more effective than the JS for both route and survey knowledge acquisition. The results of the SF-WIP were compatible to the results from other studies (using walking-like navigation techniques). The action-transferred navigation technique, named Finger-Walking-in-Place (FWIP), was more effective for both route and survey knowledge acquisition than the JS in the same large-scale, large-extent and visually impoverished virtual maze. In addition, our empirical studies showed that the SF-WIP and the FWIP are similarly effective for route and survey knowledge acquisition, suggesting that human's spatial learning ability is still supported by the transferred action (FWIP) as much as the original action (SF-WIP). Since there was no significant difference between FWIP and SF-WIP but the FWIP showed the better effect than the JS on spatial knowledge acquisition, we can infer that our action-transferred approach is useful for designing convenient and effective navigation techniques for spatial learning. Some design implications are discussed, suggesting that our action-transferred approach is not limited to navigation techniques and can be extensively used to design (general) interaction techniques. In particular, action-transferred design can be more effectively used for the users with disabilities (unable to use of a part of the body) or for fatigue/convenience reasons. Related to our theoretical reasoning, we established another user study to explore if the transferred action is still coupled with the perception that is known as coupled with the original action. Our study results supported that there was a close connection between distance perception and transferred action as literature suggests. Thus, this dissertation successfully supports our theoretical observations and our action-inspired approach to design of convenient and effective navigation techniques for spatial learning through our empirical studies. Although our conclusion is drawn from the empirical studies using a couple of NavTechs (body-turn and FWIP), and is therefore not the direct evidence at the neural level, it should be notable that our action-inspired design approach for effective spatial learning is strongly supported by the theories that have been demonstrated by a number of studies over time. / Ph. D.

Wearing a virtual reality head-mounted display (VR-HMD) disconnects users from the real- world context that they are physically in. While one solution is to have everyone in the room wear VR-HMDs, this is not inclusive for all users. For example, children are not recommended to wear VR-HMDs due to eyesight concerns, and individuals with cybersickness, make-up, or thick hair texture may not want to wear them. In this thesis, we investigated the effects of using motion-tracked tablets as a window through which people can see the virtual world and understand how we can offer a more inclusive and social VR experience. Finally, we explore our in-lab user study to evaluate the usability of such a system, and we compare it against watching the VR-HMD user's egocentric view on a computer screen. Our results show that the tablet-based VR system is highly usable. Because of its agency, the participants felt more present and preferred the tablet-based system over the baseline method. / Master of Science / Wearing a VR-HMD (Virtual Reality Head Mounted Display) to enter VR (Virtual reality) usually cuts the user off from the real-world context around them. One possible solution to this problem is to let everyone wear VR-HMDs. But it is not feasible for everyone. Some individuals experience cybersickness or physical constraints such as glasses, thick hair, or makeup. Additionally, children are not allowed to wear VR-HMD as it may affect their eyesight. We propose a solution to this problem by enabling users to use motion-tracked tablets. Motion-tracked tablets refer to tablets such as iPad, Galaxy Tab, etc., that are tracked inside a Virtual Environment (VE) using trackers such as Vive Tracker. The trackers track the relative location of the tablet inside the VE. We believe motion-tracked tablets will give the users a window to the VE. In this thesis, we investigated the effects of using motion-tracked tablets in VR to understand how we can offer a more inclusive and social VR experience. We explored how useful the users found the system, how situationally aware they were about the VE, and how present they felt in the VE. Our results show a preference for motion-tracked tablets over the egocentric view of the VR-HMD user through a computer screen.

Geometrical modeling is a crucial aspect of simulations involving manufactured objects. Apart from the pure construction of curves and surfaces, the analysis of their quality is equally important in the design and manufacturing process. In computer-aided simulation, the original freeorm surfaces need to be tessellated into triangulated surfaces before the simulation procedure. To concurrently and interactively visualize the results from both simulation and surface interrogation in a virtual environment, I propose two novel surface interrogation algorithms for triangulated surfaces instead of the traditional freeorm surfaces. The novel algorithms are interactive and immersive versions of two well-established surface interrogation techniques ? reflection lines and generalized focal surfaces. These two algorithms have been designed to overcome some limitations of the traditional approaches and make them available for interactive and immersive applications. For reflection lines, the new algorithm maps the triangulated surface onto the light plane so that the computation of distance between reflection ray and light line in three dimensions can be reduced to computation of intersections between light lines and triangle edges in two dimensions, simplifying the computation. For generalized focal surfaces, the new algorithm estimates curvature by simple computation of the derivatives of a3rd degree triangular Bézier patch on each triangle and removes the requirement for a minimum number of neighbor points and implicit requirements on how the neighbor points are distributed. Proposed future work on real-time rendering of surface interrogation using a texture mapping technique is discussed.

No description available.

Haptics is a field of robotics that involves many aspects of engineering, requiring the collaboration of different disciplines like mechanics, electronics, control theory and computer science. Although multi-disciplinarity is an element in common with other robotic application, haptic system has the additional requirement of high performance because of the human perception requirement of 1KHz feedback rate. Such high computing requirement impacts the design of the whole haptic system but it is has particular effects in the design and implementation of haptic rendering algorithms. In the chain of software and hardware components that describe a haptic system the haptic rendering is the element that has the objective of computing the force feedback given the interaction of the user with the device.A variety of haptic rendering algorithms have been proposed in the past for the simulation of three degree of freedom (3DoF) interactions in which a single point touches a complex object as well as 6DoF interactions in which two complex objects interact in multiple points. The use of 3DoF or 6DoF algorithms depends mostly from the type of application and consequently the type of device. For example applications like virtual prototype require 6DoF interaction while many simulation applications have less stringent requirements. Apart the number of degree of freedom haptic rendering algorithms are characterized by the geometrical representation of the objects, by the use of rigid or deformable objects and by the introduction of physical properties of the object surface like friction and texture properties. Given this variety of possibilities and the presence of the human factor in the computation of haptic feedback it is hard to compare different algorithms to asses whether one specific solution performs better than any other previously proposed.The goal of the proposed work is two-fold. First this thesis proposes a framework allowing for more objective comparison of haptic rendering algorithms. Such comparison take into account the perceptual aspect of haptic interaction but tries to remove it from the comparison with the aim of obtaining an objective comparison between algorithms. Second, this thesis proposes a new haptic rendering algorithm for 3DoF interaction and one for 6DoF interaction. The first algorithm for 3DoF interaction provides interaction with rotational friction based on a simulation of the soft finger contact model. The new 6DoF interaction algorithm allows the computation of the haptic feedback of interaction between voxel models.

No / The authors present the Virtual Park (or VPark) system. This includes a networked virtual environment (NVE) system, called W-VLNET (Windows Virtual Life Network) and an Attraction Building System that creates and modifies attractions used in the NVE system. This article details the techniques for communication, scene management, facial and body animation, and general user interaction modules. The authors used VRML97 and MPEG-4 SHNC to stress the compatibility of the system with other similar virtual reality systems. The software provides realistic virtual actors as well as sets of applicable high-level actions in real time. Using this software, users can introduce their own scenario-based applications into a shared virtual environment.

A distributed virtual environment (DVE) is a shared virtual environment where multiple users at their workstations interact with each other. Some of these systems may support a large number of users, e. g. , massive multi-player online games, and these users may be geographically distributed. An important performance measure in a DVE system is the delay for an update of a user's state (e. g. , his position in the virtual environment) to arrive at the workstations of those users who are affected by the update. This update delay often has a stringent requirement (e. g. , less than 100 ms) in order to ensure interactivity among users. <br /><br /> In designing a DVE system, an important issue is how well the system scales as the number of users increases. In terms of scalability, a promising system architecture is a two-level hierarchical architecture. At the lower level, multiple service facilities (or basic systems) are deployed; each basic system interacts with its assigned users. At the higher level, the various basic systems ensure that their copies of the virtual environment are as consistent as possible. Although this architecture is believed to have good properties with respect to scalability, not much is known about its performance characteristics. <br /><br /> This thesis is concerned with the performance characteristics of the two-level hierarchical architecture. We first investigate the issue of scalability. We obtain analytic results on the workload experienced by the various basic systems as a function of the number of users. Our results provide valuable insights into the scalability of the architecture. We also propose a novel technique to achieve weak consistency among copies of the virtual environment at the various basic systems. Simulation results on the consistency/scalability tradeoff are presented. <br /><br /> We next study the update delay in the two-level hierarchical architecture. The update delay has two main components, namely the delay at the basic system (or server delay) and the network delay. For the server delay, we use a network of queues model where each basic system may have one or more processors. We develop an approximation method to obtain results for the distribution of server delay. Comparisons with simulation show that our approximation method yields accurate results. We also measure the time to process an update on an existing online game server. Our approximate results are then used to characterize the 95th-percentile of the server delay, using the measurement data as input. <br /><br /> As to the network delay, we develop a general network model and obtain analytic results for the network delay distribution. Numerical examples are presented to show the conditions under which geographical distribution of basic systems will lead to an improvement in the network delay. We also develop an efficient heuristic algorithm that can be used to determine the best locations for the basic systems in a network.