Made available in DSpace on 2015-03-04T18:57:55Z (GMT). No. of bitstreams: 1 DissertacaoEllenCorrea.pdf: 9749144 bytes, checksum: 89fe6968016fd6c546c16b922bf067c5 (MD5) Previous issue date: 2012-08-09 / This work describes the fundamental elements of medical simulations. It also documents a medical training prototype that was developed. Such prototype is composed of 3d models of a surgery room, anatomical models of a patient as well as models of a medical crew. The prototype also allows one to perform actions linked to the medical training of the pleural puncture. Finally, the prototype contains an evaluation agent that verifies the performance of the users while performing the medical procedure. This work goes on to evaluate the usability of a head mounted display in a medical simulation when compared with a simpler solution that uses just a LCD display. / Este trabalho descreve os elementos fundamentais para a construção de um sistema para simulação médica e o desenvolvimento de um protótipo de um ambiente de realidade virtual para treinamento médico. O protótipo inclui modelos 3D de uma sala de cirurgia, da anatomia do paciente virtual e da equipe médica, bem como os módulos de software necessários para simular o treinamento médico no protocolo do procedimento de punção pleural. O sistema possui ainda um agente avaliador capaz de acompanhar o desempenho dos usuários nas etapas necessárias do procedimento médico. Além disso, avaliamos a usabilidade de um capacete de realidade virtual neste tipo de aplicação, através de testes comparativos com uma solução mais simples que utiliza apenas um monitor convencional como saída de vídeo.

So, King-yan Oldfield. / Thesis (M.Phil.)--Chinese University of Hong Kong, 1999. / Includes bibliographical references (leaves 68-70). / Abstracts in English and Chinese. / Abstract --- p.i / Acknowledgments --- p.ii / Chapter 1 --- Introduction --- p.1 / Chapter 1.1 --- Evolution of Communication Technologies --- p.1 / Chapter 1.2 --- The Internet --- p.2 / Chapter 1.3 --- The Distributed Virtual Environments --- p.2 / Chapter 1.3.1 --- Features of DVE --- p.3 / Chapter 1.3.2 --- Current and Potential Applications --- p.4 / Chapter 1.3.3 --- The Challenges --- p.5 / Chapter 1.4 --- Our Contributions --- p.6 / Chapter 2 --- System Architecture --- p.7 / Chapter 2.1 --- The SSDVE and MSDVE Architectures --- p.7 / Chapter 2.2 --- Issues in the MSDVE Architecture --- p.8 / Chapter 2.2.1 --- On the Server Side --- p.8 / Chapter 2.2.2 --- On the Client Side --- p.8 / Chapter 3 --- Balancing Work Load and Reducing Inter-server Communication --- p.10 / Chapter 3.1 --- Problem Formulation --- p.10 / Chapter 3.1.1 --- The Area of Interest --- p.11 / Chapter 3.1.2 --- The DVE Cells --- p.11 / Chapter 3.1.3 --- Expected Number of Avatars --- p.12 / Chapter 3.1.4 --- Cost Metrics in Different Types of Network --- p.13 / Chapter 3.1.5 --- Problem Definition --- p.14 / Chapter 3.1.6 --- Complexity --- p.18 / Chapter 3.2 --- Partitioning Algorithms --- p.19 / Chapter 3.2.1 --- A Simplified Case --- p.19 / Chapter 3.2.2 --- The General Case --- p.19 / Chapter 3.3 --- Experiments --- p.22 / Chapter 4 --- Communication Sub-graph --- p.31 / Chapter 4.1 --- Problem Formulation --- p.31 / Chapter 4.1.1 --- Optimization Metrics --- p.32 / Chapter 4.1.2 --- Design Considerations --- p.32 / Chapter 4.2 --- Communication Sub-graph Construction Algorithms --- p.34 / Chapter 4.2.1 --- The Minimum Diameter Sub-graph (MDS) --- p.34 / Chapter 4.2.2 --- The Core-based Tree (CBT) --- p.37 / Chapter 4.2.3 --- The Minimum Spanning Tree (MST) --- p.40 / Chapter 5 --- Synchronization --- p.42 / Chapter 5.1 --- Synchronization in a DVE System --- p.43 / Chapter 5.2 --- System Model --- p.46 / Chapter 5.2.1 --- Problem Definition --- p.47 / Chapter 5.2.2 --- The Markov Chain Model --- p.47 / Chapter 5.2.3 --- Deciding the Threshold Φ --- p.49 / Chapter 5.3 --- Optimal Synchronizing Interval --- p.50 / Chapter 5.3.1 --- "An ""on-average"" Guarantee" --- p.50 / Chapter 5.3.2 --- A Stochastic Guarantee --- p.52 / Chapter 5.3.3 --- Finding p with T and Φ --- p.52 / Chapter 5.3.4 --- Searching for r*p --- p.54 / Chapter 5.4 --- Experiments --- p.55 / Chapter 5.4.1 --- Simulation Results --- p.55 / Chapter 5.4.2 --- Theoretical Results --- p.58 / Chapter 6 --- Related Work --- p.63 / Chapter 6.1 --- Load Balancing on DVE --- p.63 / Chapter 6.2 --- Object State Synchronization Techniques --- p.63 / Chapter 6.3 --- Group Communication and Multicasting --- p.64 / Chapter 6.4 --- DVE System Development Toolkits --- p.64 / Chapter 6.5 --- Example DVE Systems --- p.65 / Chapter 7 --- Conclusion --- p.66 / Chapter 7.1 --- A Vision to the Future --- p.66 / Chapter 7.2 --- Conclusion --- p.66 / Bibliography --- p.68

Chan Ming-fei. / Thesis (M.Phil.)--Chinese University of Hong Kong, 1999. / Includes bibliographical references (leaves 63-66). / Abstract --- p.i / Acknowledgments --- p.iii / Chapter 1 --- Introduction --- p.1 / Chapter 1.1 --- Challenging Issues --- p.2 / Chapter 1.2 --- Previous Work --- p.4 / Chapter 1.3 --- Organization of the Thesis --- p.5 / Chapter 2 --- Distributed Virtual Environment --- p.6 / Chapter 2.1 --- Possible Architectures --- p.6 / Chapter 2.2 --- Representations of Clients as Avatars --- p.7 / Chapter 2.3 --- Dynamic Membership --- p.9 / Chapter 3 --- Bandwidth and Computation Reduction Techniques --- p.11 / Chapter 3.1 --- Network Communication --- p.12 / Chapter 3.2 --- Dead Reckoning --- p.13 / Chapter 3.3 --- Message Aggregation --- p.15 / Chapter 3.3.1 --- Network-Based Aggregation --- p.15 / Chapter 3.3.2 --- Organization-Based Aggregations --- p.16 / Chapter 3.3.3 --- Grid-Based Aggregations --- p.16 / Chapter 3.4 --- Relevance Filtering --- p.17 / Chapter 3.4.1 --- Entity-Based Filtering --- p.17 / Chapter 3.4.2 --- Grid-Based Filtering --- p.19 / Chapter 3.5 --- Quiescent Entities --- p.20 / Chapter 3.6 --- Spatial Partitioning --- p.21 / Chapter 3.6.1 --- Necessity of Spatial Partitioning --- p.22 / Chapter 3.6.2 --- Binary Space Partitioning Tree --- p.23 / Chapter 3.6.3 --- BSP Tree Construction --- p.23 / Chapter 4 --- Partitioning Algorithm --- p.25 / Chapter 4.1 --- Problem Formulation --- p.25 / Chapter 4.2 --- Exhaustive Partition (EP) Algorithm --- p.28 / Chapter 4.3 --- Partitioning Algorithm --- p.29 / Chapter 4.3.1 --- Recursive Bisection Partition (RBP) Algorithm --- p.30 / Chapter 4.3.2 --- Layering Partitioning (LP) Algorithm --- p.32 / Chapter 4.3.3 --- Communication Refinement Partitioning (CRP) Algorithm --- p.38 / Chapter 4.4 --- Parallel Approach --- p.42 / Chapter 4.5 --- Further Observation --- p.43 / Chapter 5 --- Experiments --- p.44 / Chapter 5.1 --- Experiment 1: Small Virtual World --- p.45 / Chapter 5.2 --- Experiment 2: Large Virtual World --- p.46 / Chapter 5.3 --- Experiment 3: Moving of Avatars --- p.47 / Chapter 5.4 --- Experiment 4: Dynamic Joining and Leaving --- p.48 / Chapter 5.5 --- Experiment 5: Parallel Approach --- p.49 / Chapter 6 --- Implementation Considerations --- p.55 / Chapter 6.1 --- Different Environments --- p.55 / Chapter 6.2 --- Platform --- p.56 / Chapter 6.3 --- Lessons learned --- p.57 / Chapter 7 --- Conclusion --- p.59 / A Simplex Method --- p.60 / Bibliography --- p.63

Ka Keung Caramon Lee. / Thesis (M.Phil.)--Chinese University of Hong Kong, 2000. / Includes bibliographical references (leaves 100-105). / Abstracts in English and Chinese. / Contents --- p.iii / List of Figures --- p.vi / List of Tables --- p.ix / Chapter 1 --- Introduction --- p.1 / Chapter 1.1 --- Motivation --- p.1 / Chapter 1.2 --- Related Work --- p.3 / Chapter 1.2.1 --- Empirical Psychophysical Equations --- p.3 / Chapter 1.2.2 --- Industry Standards --- p.4 / Chapter 1.2.3 --- Fuzzy Logic --- p.4 / Chapter 1.2.4 --- Neural Networks --- p.5 / Chapter 1.3 --- Organization of Thesis --- p.7 / Chapter 2 --- Experimental Design --- p.9 / Chapter 2.1 --- Human Motion Sense --- p.9 / Chapter 2.2 --- Full-Body Motion Virtual Reality System --- p.12 / Chapter 2.3 --- Human Sensation Measure --- p.15 / Chapter 2.4 --- Trajectory Segmentation --- p.16 / Chapter 3 --- Learning and Validation of Human Sensation Models --- p.22 / Chapter 3.1 --- Cascade Neural Networks --- p.23 / Chapter 3.1.1 --- Dynamic Mapping --- p.26 / Chapter 3.2 --- Experimental Trajectory Data --- p.26 / Chapter 3.3 --- Effect of Trajectory Segmentation --- p.31 / Chapter 3.4 --- Model Validation --- p.32 / Chapter 3.5 --- Similarity Measure --- p.33 / Chapter 3.6 --- Similarity Measure Results --- p.38 / Chapter 4 --- Input Reduction for Human Sensation Modeling --- p.40 / Chapter 4.1 --- Introduction --- p.40 / Chapter 4.2 --- Input Reduction --- p.41 / Chapter 4.3 --- Feature Extraction and Input Selection --- p.42 / Chapter 4.4 --- Feature Extraction Using Principal Component Analysis --- p.44 / Chapter 4.5 --- Independent Component Analysis --- p.48 / Chapter 4.5.1 --- Measure of Gaussianity --- p.50 / Chapter 4.5.2 --- The Fixed Point ICA Algorithm --- p.51 / Chapter 4.6 --- Input Reduction Using Independent Component Analysis --- p.52 / Chapter 4.6.1 --- ICA Without Dimension Reduction --- p.52 / Chapter 4.6.2 --- Feature Extraction Using ICA --- p.55 / Chapter 4.6.3 --- Input Selection Using ICA --- p.57 / Chapter 4.6.4 --- Applying Input Selection by ICA on the Furnace Data --- p.58 / Chapter 4.6.5 --- Applying Input Selection by ICA to Sensation Modeling --- p.65 / Chapter 4.6.6 --- Cross Verification of Selected Inputs --- p.70 / Chapter 4.7 --- Summary on Input Reduction for Human Sensation Modeling --- p.72 / Chapter 5 --- Stimulus Modification Based on Human Sensation --- p.74 / Chapter 5.1 --- Need for Stimulus Modification --- p.74 / Chapter 5.2 --- Sensation Grades --- p.75 / Chapter 5.3 --- Trajectory Modification Scheme --- p.77 / Chapter 5.4 --- Experiments --- p.80 / Chapter 6 --- Conclusion --- p.86 / Chapter 6.1 --- Contributions --- p.86 / Chapter 6.2 --- Future Work --- p.87 / Chapter A --- Platform Model --- p.88 / Chapter A.1 --- Inverse Kinematics --- p.90 / Chapter A.2 --- Forward Kinematics --- p.93 / Chapter A.3 --- Platform Dynamics --- p.99 / Bibliography --- p.100

Ma Mun Chung. / Thesis (M.Phil.)--Chinese University of Hong Kong, 1998. / Includes bibliographical references (leaves 86-88). / Abstract also in Chinese. / Abstract --- p.i / Declaration --- p.ii / Acknowledgement --- p.iii / List of Figures --- p.iv / List of Tables --- p.ix / Table of Contents --- p.x / Chapter 1. --- Introduction --- p.1 / Chapter 1.1. --- Motivation --- p.1 / Chapter 1.2. --- Thesis Roadmap --- p.3 / Chapter 1.3. --- Contribution / Chapter 2. --- System Architecture --- p.6 / Chapter 2.1. --- Tracker system --- p.6 / Chapter 2.1.1. --- Spatial Information --- p.6 / Chapter 2.1.2. --- Transmitter (Xmtr) --- p.6 / Chapter 2.1.3. --- Receiver (Recvr) --- p.7 / Chapter 2.2. --- Glove System --- p.7 / Chapter 2.2.1. --- CyberGlove Interface Unit (CGIU) --- p.7 / Chapter 2.2.2. --- Bend Sensors --- p.7 / Chapter 2.3. --- Integrating the tracker and the glove system --- p.9 / Chapter 2.3.1. --- System Layout --- p.9 / Chapter 3. --- Deformable Model --- p.11 / Chapter 3.1. --- Elastic models in computer --- p.11 / Chapter 3.2. --- Virtual object model --- p.17 / Chapter 3.3. --- Force displacement relationship --- p.18 / Chapter 3.3.1. --- Stress-strain relationship --- p.19 / Chapter 3.3.2. --- Stiffness matrix formulation --- p.20 / Chapter 3.4. --- Solving the linear system --- p.24 / Chapter 3.5. --- Implementation --- p.26 / Chapter 3.5.1. --- Data structure --- p.26 / Chapter 3.5.2. --- Global stiffness matrix formulation --- p.27 / Chapter 3.5.3. --- Re-assemble of nodal displacement --- p.30 / Chapter 4. --- Collision Detection --- p.32 / Chapter 4.1. --- Related Work --- p.31 / Chapter 4.2. --- Spatial Subdivision --- p.37 / Chapter 4.3. --- Hierarchy construction --- p.38 / Chapter 4.3.1. --- Data structure --- p.39 / Chapter 4.3.2. --- Initialisation --- p.41 / Chapter 4.3.3. --- Expanding the hierarchy --- p.42 / Chapter 4.4. --- Collision detection --- p.45 / Chapter 4.4.1. --- Hand Approximation --- p.45 / Chapter 4.4.2. --- Interference tests --- p.47 / Chapter 4.4.3. --- Searching the hierarchy --- p.51 / Chapter 4.4.4. --- Exact interference test --- p.51 / Chapter 4.5. --- Grasping mode --- p.53 / Chapter 4.5.1. --- Conditions for Finite Element Analysis (FEA) --- p.53 / Chapter 4.5.2. --- Attaching conditions --- p.53 / Chapter 4.5.3. --- Collision avoidance --- p.54 / Chapter 4.6. --- Repeating deformation in different orientation --- p.56 / Chapter 5. --- Enhancing performance --- p.59 / Chapter 5.1. --- Data communication --- p.60 / Chapter 5.1.1. --- Client-server model --- p.60 / Chapter 5.1.2. --- Internet protocol suite --- p.61 / Chapter 5.1.3. --- Berkeley socket --- p.61 / Chapter 5.1.4. --- Checksum problem --- p.62 / Chapter 5.2. --- Use of parallel tool --- p.62 / Chapter 5.2.1. --- Parallel code generation --- p.63 / Chapter 5.2.2. --- Optimising parallel code --- p.64 / Chapter 6. --- Implementation and Results --- p.65 / Chapter 6.1. --- Supporting functions --- p.65 / Chapter 6.1.1. --- Read file --- p.66 / Chapter 6.1.2. --- Keep shape --- p.67 / Chapter 6.1.3. --- Save as --- p.67 / Chapter 6.1.4. --- Exit --- p.67 / Chapter 6.2. --- Visual results --- p.67 / Chapter 6.3. --- An operation example --- p.75 / Chapter 6.4. --- Performance of parallel algorithm --- p.78 / Chapter 7. --- Conclusion and Future Work --- p.84 / Chapter 7.1. --- Conclusion --- p.84 / Chapter 7.2. --- Future Work --- p.84 / Reference --- p.86 / Appendix A Matrix Inversion --- p.89 / Appendix B Derivation of Equation 6.1 --- p.92 / Appendix C Derivation of (6.2) --- p.93

by Yu-Hang Siu. / Thesis (M.Phil.)--Chinese University of Hong Kong, 1998. / Includes bibliographical references (leaves 74-80). / Abstract also in Chinese. / Abstract --- p.iii / Acknowledgements --- p.v / Chapter 1 --- Introduction --- p.1 / Chapter 1.1 --- Volume Visualization --- p.2 / Chapter 1.2 --- Virtual Environment --- p.11 / Chapter 1.3 --- Approach --- p.12 / Chapter 1.4 --- Thesis Overview --- p.13 / Chapter 2 --- Contour Extraction --- p.15 / Chapter 2.1 --- Concept of Intelligent Scissors --- p.16 / Chapter 2.2 --- Dijkstra's Algorithm --- p.18 / Chapter 2.3 --- Cost Function --- p.20 / Chapter 2.4 --- Summary --- p.23 / Chapter 3 --- Volume Cutting --- p.24 / Chapter 3.1 --- Basic idea of the algorithm --- p.25 / Chapter 3.2 --- Intelligent Scissors on Surface Mesh --- p.27 / Chapter 3.3 --- Internal Cutting Surface --- p.29 / Chapter 3.4 --- Summary --- p.34 / Chapter 4 --- Three-dimensional Intelligent Scissors --- p.35 / Chapter 4.1 --- 3D Graph Construction --- p.36 / Chapter 4.2 --- Cost Function --- p.40 / Chapter 4.3 --- Applications --- p.42 / Chapter 4.3.1 --- Surface Extraction --- p.42 / Chapter 4.3.2 --- Vessel Tracking --- p.47 / Chapter 4.4 --- Summary --- p.49 / Chapter 5 --- Implementations in a Virtual Environment --- p.52 / Chapter 5.1 --- Volume Cutting --- p.53 / Chapter 5.2 --- Surface Extraction --- p.56 / Chapter 5.3 --- Vessel Tracking --- p.59 / Chapter 5.4 --- Summary --- p.64 / Chapter 6 --- Conclusions --- p.68 / Chapter 6.1 --- Summary of Results --- p.68 / Chapter 6.2 --- Future Directions --- p.70 / Chapter A --- Performance of Dijkstra's Shortest Path Algorithm --- p.72 / Chapter B --- IsoRegion Construction --- p.73

by Tsang Kwok Hang Elton. / Thesis submitted in: December 1998. / Thesis (M.Phil.)--Chinese University of Hong Kong, 1999. / Includes bibliographical references (leaves 78-83). / Abstract also in Chinese. / Chapter 1 --- Introduction --- p.1 / Chapter 2 --- Virtual-hand Recognition --- p.5 / Chapter 2.1 --- Hand model --- p.6 / Chapter 2.1.1 --- Hand structure --- p.6 / Chapter 2.1.2 --- Motions of the hand joints --- p.8 / Chapter 2.2 --- Hand-tracking technologies --- p.9 / Chapter 2.2.1 --- Glove-based tracking --- p.10 / Chapter 2.2.2 --- Image-based tracking --- p.12 / Chapter 2.3 --- Problems in virtual-hand recognition --- p.13 / Chapter 2.3.1 --- Hand complexity --- p.13 / Chapter 2.3.2 --- Human variations --- p.13 / Chapter 2.3.3 --- Immature hand-tracking technologies --- p.14 / Chapter 2.3.4 --- Time-varying signal --- p.14 / Chapter 2.3.5 --- Efficiency --- p.14 / Chapter 3 --- Previous Work --- p.16 / Chapter 3.1 --- Posture and gesture recognition algorithms --- p.16 / Chapter 3.1.1 --- Template Matching --- p.17 / Chapter 3.1.2 --- Neural networks --- p.18 / Chapter 3.1.3 --- Statistical classification --- p.20 / Chapter 3.1.4 --- Discontinuity matching --- p.21 / Chapter 3.1.5 --- Model-based analysis --- p.23 / Chapter 3.1.6 --- Hidden Markov Models --- p.23 / Chapter 3.2 --- Hand-input systems --- p.24 / Chapter 3.2.1 --- Gesture languages --- p.25 / Chapter 3.2.2 --- 3D modeling --- p.25 / Chapter 3.2.3 --- Medical visualization --- p.26 / Chapter 4 --- Posture Recognition --- p.28 / Chapter 4.1 --- Fuzzy concepts --- p.28 / Chapter 4.1.1 --- Degree of membership --- p.29 / Chapter 4.1.2 --- Certainty factor --- p.30 / Chapter 4.1.3 --- Evidence combination --- p.30 / Chapter 4.2 --- Fuzzy posture recognition system --- p.31 / Chapter 4.2.1 --- Objectives --- p.32 / Chapter 4.2.2 --- System overview --- p.32 / Chapter 4.2.3 --- Input parameters --- p.33 / Chapter 4.2.4 --- Posture database --- p.36 / Chapter 4.2.5 --- Classifier --- p.37 / Chapter 4.2.6 --- Identifier --- p.40 / Chapter 5 --- Performance Evaluation --- p.42 / Chapter 5.1 --- Experiments --- p.42 / Chapter 5.1.1 --- Accuracy analysis --- p.43 / Chapter 5.1.2 --- Efficiency analysis --- p.46 / Chapter 5.2 --- Discussion --- p.48 / Chapter 5.2.1 --- Strengths and weaknesses --- p.48 / Chapter 5.2.2 --- Summary --- p.50 / Chapter 6 --- Posture Database Editor --- p.51 / Chapter 6.1 --- System architecture --- p.51 / Chapter 6.1.1 --- Hardware configuration --- p.51 / Chapter 6.1.2 --- Software tools --- p.53 / Chapter 6.2 --- User interface --- p.54 / Chapter 6.2.1 --- Menu bar --- p.55 / Chapter 6.2.2 --- Working frame and data frame --- p.56 / Chapter 6.2.3 --- Control panel --- p.56 / Chapter 7 --- An Application: 3D Virtual World Modeler --- p.59 / Chapter 7.1 --- System Design --- p.60 / Chapter 7.2 --- Common operations --- p.62 / Chapter 7.3 --- Virtual VRML Worlds --- p.65 / Chapter 8 --- Conclusion --- p.70 / Chapter 8.1 --- Summaries on previous work --- p.70 / Chapter 8.2 --- Contributions --- p.73 / Chapter 9 --- Future Work --- p.75 / Bibliography --- p.78

Through immersive stereoscopic displays and natural user interfaces, virtual reality (VR) is capable of offering the user a sense of presence in the virtual space, and has been long expected to revolutionize how people interact with virtual content in various application scenarios. However, with many technical challenges solved over the last three decades to bring low cost and high fidelity to VR experiences, we still do not see VR technology used frequently in many seemingly suitable applications. Part of this is due to the lack of expressiveness and efficiency of traditional “simple and reality-based� 3D user interfaces (3DUIs). The challenge is especially obvious when complex interaction tasks with diverse requirements are involved, such as editing virtual objects from multiple scales, angles, perspectives, reference frames, and dimensions. A common approach to overcome such problems is through hybrid user interface (HUI) systems that combine complementary interface elements to leverage their strengths. Based on this method, the first contribution of this dissertation is the proposal of Force Extension, an interaction technique that seamlessly integrates position-controlled touch and rate-controlled force input for efficient multi-touch interaction in virtual environments. Using carefully designed mapping functions, it is capable of offering fluid transitions between the two contexts, as well as simulating shear force input realistically for multi-touch gestures. The second contribution extends the HUI concept into immersive VR by introducing a Hybrid Virtual Environment (HVE) level editing system that combines a tablet and a Head-Mounted Display (HMD). The HVE system improves user performance and experience in complex high-level world editing tasks by using a “World-In-Miniature� and 2D GUI rendered on a multi-touch tablet device to compensate for the interaction limitations of a traditional HMD- and wand-based VR system. The concept of Interaction Context (IC) is introduced to explain the relationship between tablet interaction and the immersive interaction, and four coordination mechanisms are proposed to keep the perceptual, functional, and cognitive flow continuous during IC transitions. To offer intuitive and realistic interaction experiences, most immersive 3DUIs are centered on the user’s virtual avatar, and obey the same physics rules of the real world. However, this design paradigm also employs unnecessary limitations that hinders the performance of certain tasks, such as selecting objects in cluttered space, manipulating objects in six degrees of freedom, and inspecting remote spaces. The third contribution of this dissertation proposes the Object Impersonation technique, which breaks the common assumption that one can only immerse in the VE from a single avatar, and allows the user to impersonate objects in the VE and interact from their perspectives and reference frames. This hybrid solution of avatar- and object-based interaction blurs the line between travel and object selection, creating a unique cross-task interaction experience in the immersive environment. Many traditional 3DUIs in immersive VR use simple and intuitive interaction paradigms derived from real world metaphors. But they can be just as limiting and ineffective as in the real world. Using the coordinated HUI or HVE systems presented in this dissertation, one can benefit from the complementary advantages of multiple heterogeneous interfaces (Force Extension), VE representations (HVE Level Editor), and interaction techniques (Object Impersonation). This advances traditional 3D interaction into the more powerful hybrid space, and allows future VR systems to be applied in more application scenarios to provide not only presence, but also improved productivity in people’s everyday tasks.

During a laparoscopic surgery, an endoscopic camera is used to provide visual feedback of the surgery to the surgeon and is controlled by a skilled assisting surgeon or a nurse. However, in robot-assisted teleoperated systems such as the daVinci surgical system, the same control lies with the operating surgeons. This results in an added task of constantly changing view point of the endoscope which can be disruptive and also increase the cognitive load on the surgeons. The work presented in this thesis aims to provide an approach that results in an intelligent camera control for such systems using machine learning algorithms. A particular task of pick and place was selected to demonstrate this approach. To add a layer of intelligence to the endoscope, the task was classified into subtasks representing the intent of the user. Neural networks with long short term memory cells (LSTMs) were trained to classify the motion of the instruments in the subtasks and a policy was calculated for each subtask using inverse reinforcement learning (IRL). Since current surgical robots do not enable the movement of the camera and instruments simultaneously, an expert data set was unavailable that could be used to train the models. Hence, a user study was conducted in which the participants were asked to complete the task of picking and placing a ring on a peg in a 3-D immersive simulation environment created using CHAI libraries. A virtual reality headset, Oculus Rift, was used during the study to track the head movements of the users to obtain their view points while they performed the task. This was considered to be expert data and was used to train the algorithm to automate the endoscope motion. A 71.3% accuracy was obtained for the classification of the task into 4 subtasks and the inverse reinforcement learning resulted in an automated trajectory of the endoscope which was 94.7% similar to the human trajectories collected demonstrating that the approach provided in thesis can be used to automate endoscopic motion similar to a skilled assisting surgeon.

I denna studie undersöks sambandet mellan visuella och taktila stimuli i Virtual Reality med avstamp i frågeställningarna: Hur stor roll spelar sambandet mellan visuella stimuli och taktila stimuli i en Virtual Reality-installation? Hur förändras deltagarens inlevelsekänsla när detta samband ändras? Hur förändras deltagarens rumsuppfattning när detta samband ändras? För att besvara frågeställningarna skapades två olika versioner av en interaktiv VR-installation utefter metoden forskning genom design. En av versionerna hade korrekt samband mellan visuella stimuli och taktila stimuli, det vill säga att deltagaren fysiskt kunde röra vid det denne såg med hjälp av fysiska objekt som var placerade i samma rum som installationen upplevdes i. Den andra versionen hade felaktigt samband mellan visuella stimuli och taktila stimuli, vilket innebar att de fysiska objekten hade flyttats en aning i Virtual Reality-installationen så att de ej speglade objekten i verkligheten. Det vill säga att informationen deltagaren avläste vid beröring ej stämde överens med informationen deltagaren avläste med synen. Därefter fick fyra deltagare testa dessa installationer och svara på frågor i en öppen intervjuform. Resultatet kopplades tydligt till Csíkszentmihályis flow-teori; när sambandet mellan dessa stimuli ej var korrekt så förändrades deltagarnas rumsuppfattning vilket bidrog till att de kände en lägre inlevelsekänsla. Möjligheten att kunna vidröra de fysiska objekten i sig ansågs dock som mer inlevelserikt än ifall möjligheten ej funnits där. Det ses även som styrkan med denna studie; att resultatet tydligt speglade de valda teorierna. Svagheten med studien var att resultatet enbart grundas på data från intervjuer med fyra deltagare, då flera ej hanns med inom tidsramen för studien.