Real-time, virtual and mixed reality systems have diverse uses for real-world data visualization, representation, and remote collaboration in distant learning settings, especially in universities. Design of such systems involves challenges in mapping the real world data and physical world structure accurately to digital form of physical space, also called as virtual models. Researchers have created similar systems using multiple cameras, stereo cameras, accelerometers, and motion detectors. This report presents a platform to detect and track real-time locations of people present in buildings and map their location information into virtual models as avatars using omni-directional cameras installed in the physical space. These models were created as part of the Mirror Worlds project. The project infrastructure is funded by National Science Foundation. This infrastructure enables users to connect virtual and physical aspects of the environment through a coordinate-based data networking system to enable interaction with the rest of the system including environment objects and other users. This is an interdisciplinary project where students from various departments have worked on the development of virtual model of the Moss Art Center and Torgersen Hall in Unity / X3D. Some students from the Department of Computer Science have developed a coordinate-based data networking system. The prototype of a detection and tracking algorithm to extract the location information was developed using background subtraction in MATLAB.
The proposed approach was developed using the combination of background subtraction and neural networks along with heuristics based on spatial information about the physical space. The system was scaled to work across multiple buildings, extract the location information of people present in the physical space, and map location information into shared virtual space as an avatar. The concept of remote presence was extended to create a collaborative object manipulation application using Leap Motion controller. Effects of fidelity were evaluated to perform the collaborative object manipulation task in shared virtual space based on user study conducted for this application.
Since no annotated people video dataset is publicly available with overhead view from omni-directional cameras, three videos were annotated manually to test the performance of the approach. The current approach almost works at near real-time rates. All three video sequences were evaluated to compute frame based detection accuracy. Precision and recall obtained for the first video sequence of people detection is 93.85% and 95.06% respectively. / Master of Science / Globalization and the advancements in the technological world pose the need for people working across different geographical regions to communicate not just over video conferencing but having remote presence which provides a stimuli to users’ senses to give the feeling of being at other location. The common approach to create remote presence application is by creating a real-time, virtual and mixed reality system. These mix reality systems have diverse uses for real-world data visualization, representation, and remote collaboration in distant learning settings, especially in universities. Design of such systems involves challenges in mapping the real world data and physical world structure accurately to digital form of physical space, also called as virtual models. Researchers have created similar systems using multiple cameras, depth sensors, accelerometers, and motion detectors.
This work introduces a platform to detect and track real-time locations of people present in buildings and map their location information into virtual models as avatars using cameras installed in the physical space. The proposed solution uses combination of background subtraction and neural networks with some heuristics based on spatial information about the physical space. The system is scaled to work across multiple buildings. The concept of remote presence was extended to create a collaborative object manipulation application.
Three videos were annotated manually to test the performance of the proposed approach.
| Identifer | oai:union.ndltd.org:VTETD/oai:vtechworks.lib.vt.edu:10919/86661 |
| Date | 19 July 2017 |
| Creators | Bharambe, Sachin Vasant |
| Contributors | Electrical and Computer Engineering, Knapp, R. Benjamin, Abbott, A. Lynn, Polys, Nicholas F. |
| Publisher | Virginia Tech |
| Source Sets | Virginia Tech Theses and Dissertation |
| Detected Language | English |
| Type | Thesis |
| Format | ETD, application/pdf, application/pdf |
| Rights | In Copyright, http://rightsstatements.org/vocab/InC/1.0/ |
Page generated in 0.0033 seconds