Virtual Reality has been growing in popularity and demand as technology has been substantially improved and become more readily available to the general public in the recent years. Similarly, the Architecture, Engineering and Construction industries have benefited from these advances and extensive research has been performed to adopt and streamline its utilization. An example of this adoption has been the use of Immersive Virtual Environments (IVE) as a representation of the built environment for different purposes such as building design and occupant behavior studies in the post construction stage – i.e., Human Building Interaction. This research has investigated a workflow for different alternatives of reality-capturing-based technologies that have been tested to generate a more realistic representation of the built environment regarding HBI. One of these alternatives considered was 360-image based IVEs. This alternative in particular was tested and compared by the means of a preliminary user study in order to evaluate whether it is an adequate representation of the built environment regarding HBI, and how it is compared to commonly used benchmarked Graphical based IVEs. Ultimately, participants of this user study reported a strong feeling of immersion and presence in the 360-image based IVE and showed a better performance in cognitive tasks such as reading speed and comprehension. In contrast, participants showed a better performance in object identification and finding in the Graphical based IVE. The results of our preliminary user study indicate that 360-image based IVEs could potentially be an adequate representation in the study of Human Building Interaction based on these metrics. Further research with a larger sample size should be done in performed in order to generalize any findings. / Master of Science / Virtual Reality has been growing in popularity and demand as technology has substantially improved and become more available to the general public in the recent years. Similarly, the Architecture, Engineering and Construction industries have benefited from these advances and extensive research has been performed to utilize this technology. An example of this adoption has been the use of Immersive Virtual Environments (IVE) as a representation of a building for different purposes such as design and understanding of the way occupants interact with a building. IVEs rely on using special digital goggles (called head mounted displays or HMDs) that help users immerse in a virtual environment and experience it. For this reason, our research has sought to explore different alternatives to possibly generate a more realistic immersive virtual environment that relies on immersive image-based technologies to test how humans behave, respond, and interact with a building. One of these alternatives considered was 360-degree cameras and their associated images. We sought to study whether these technologies provide an improved experience for users compared to the environments that are created through computer graphics. This thesis explains the processes that were investigated to understand the creation of an IVE, and the different alternatives available in the market to generate a 360-degree image based IVE. Then, one of these alternatives was tested and compared to a classic IVE through an experiment in order to evaluate whether 360-degree image based IVEs can be an adequate representation for building occupant interaction studies.

abstract: Generating real-world content for VR is challenging in terms of capturing and processing at high resolution and high frame-rates. The content needs to represent a truly immersive experience, where the user can look around in 360-degree view and perceive the depth of the scene. The existing solutions only capture and offload the compute load to the server. But offloading large amounts of raw camera feeds takes longer latencies and poses difficulties for real-time applications. By capturing and computing on the edge, we can closely integrate the systems and optimize for low latency. However, moving the traditional stitching algorithms to battery constrained device needs at least three orders of magnitude reduction in power. We believe that close integration of capture and compute stages will lead to reduced overall system power. We approach the problem by building a hardware prototype and characterize the end-to-end system bottlenecks of power and performance. The prototype has 6 IMX274 cameras and uses Nvidia Jetson TX2 development board for capture and computation. We found that capturing is bottlenecked by sensor power and data-rates across interfaces, whereas compute is limited by the total number of computations per frame. Our characterization shows that redundant capture and redundant computations lead to high power, huge memory footprint, and high latency. The existing systems lack hardware-software co-design aspects, leading to excessive data transfers across the interfaces and expensive computations within the individual subsystems. Finally, we propose mechanisms to optimize the system for low power and low latency. We emphasize the importance of co-design of different subsystems to reduce and reuse the data. For example, reusing the motion vectors of the ISP stage reduces the memory footprint of the stereo correspondence stage. Our estimates show that pipelining and parallelization on custom FPGA can achieve real time stitching. / Dissertation/Thesis / Prototype / Masters Thesis Electrical Engineering 2018