Panoramic image Virtual Reality (VR) is a 360 degree image which has been interpreted as a kind of VR that allows users to navigate, view, hear and have remote access to a virtual environment. Panoramic Video VR builds on this, where filming is done in the real world to create a highly dynamic and immersive environment. This is proving to be a very attractive technology and has introduced many possible applications but still present a number of challenges, considered in this research. An initial literature survey identified limitations in panoramic video to date: these were the technology (e.g. filming and stitching) and the design of effective navigation methods. In particular, there is a tendency for users to become disoriented during way-finding. In addition, an effective interface design to embed contextual information is required. The research identified the need to have a controllable test environment in order to evaluate the production of the video and the optimal way of presenting and navigating within the scene. Computer Graphics (CG) simulation scenes were developed to establish a method of capturing, editing and stitching the video under controlled conditions. In addition, a novel navigation method, named the “image channel” was proposed and integrated within this environment. This replaced hotspots: the traditional navigational jumps between locations. Initial user testing indicated that the production was appropriate and did significantly improve user perception of position and orientation over jump-based navigation. The interface design combined with the environment view alone was sufficient for users to understand their location without the need to augment the view with an on screen map. After obtaining optimal methods in building and improving the technology, the research looked for a natural, complex, and dynamic real environment for testing. The web-based virtual zoo (World Association of Zoos and Aquariums) was selected as an ideal production: It had the purpose to allow people to get close to animals in their natural habitat and created particular interest to develop a system for knowledge delivery, raising protection concerns, and entertaining visitors: all key roles of a zoo. The design method established from CG was then used to develop a film rig and production unit for filming a real animal habitat: the Formosan rock monkey in Taiwan. A web-based panoramic video of this was built and tested though user experience testing and expert interviews. The results of this were essentially identical to the testing done in the prototype environment, and validated the production. Also was successfully attracting users to the site repeatedly. The research has contributed to new knowledge in improvement to the production process, improvement to presentation and navigating within panoramic videos through the proposed Image Channel method, and has demonstrated that web-based virtual zoo can be improved to help address considerable pressure on animal extinction and animal habitat degradation that affect humans by using this technology. Further studies were addressed. The research was sponsored by Taiwan’s Government and Twycross Zoo UK was a collaborator.

Sistemas de telepresença têm sido pesquisados e desenvolvidos para inúmeras aplicações que exigem a presença física de pessoas em ambientes inacessíveis; tais situações são diversas, desde aquelas relacionadas com educação a distância até aquelas que envolvem alta periculosidade. Neste trabalho, a pesquisa e desenvolvimento se concentram na concepção e implementação de sistemas de telepresença voltados para ambientes imersivos em 360º, capazes de realizar a aquisição, transmissão e projeção de imagens em movimento em sistemas de multiprojeção imersivos, como é o caso da CAVERNA Digital®, desenvolvida pelo Laboratório de Sistemas Integráveis da Escola Politécnica da USP. Assim, o presente trabalho apresenta contribuições para o desenvolvimento de sistemas de telepresença, dentre os quais destacam-se: o detalhamento da arquitetura geral do sistema, a implementação de métodos para a calibração, correção das imagens e montagem de panoramas de 360º. Construiu-se um protótipo composto de: um Módulo de Aquisição, que adquire as imagens de oito câmeras (montadas num anel de câmeras sobre um robô), efetua as correções e monta uma imagem panorâmica; um Módulo de Composição que costura as várias imagens panorâmicas em um panorama final; e um Módulo de Exibição que ajusta e projeta o panorama nas telas da CAVERNA Digital®. Finalmente, apresentam-se considerações sobre o presente trabalho e perspectivas futuras. / Research and development in telepresence systems have been done in several applications that require physical presence of people into non-accessible environments. These situations may vary, from those related to distance education to those related to very hazardous places for humans. In this work, the main research and development goal is the conception and implementation of a telepresence system for 360 degrees immersive environments, this system is able to perform acquisition, transmission and projection of moving images on immersive multi-projection environments, such as the CAVERNA Digital®, developed by the Laboratório de Sistemas Integráveis at the Escola Politécnica, Universidade de São Paulo. The main contributions to the development of telepresence systems are: a detailed specification of the general architecture of the system, the implementation of the calibration, the imagery correction and the 360 degrees panoramas composition methods. The prototype that was implemented includes: an Acquisition Module, that acquires image of eight cameras (mounted on a ring of cameras placed on top of a robot), executes corrections and prepares a panoramic image; a Composition Module, that stitches the images in a final panorama; and an Exhibition Module, that adjusts and projects the panorama into the screens of the CAVERNA Digital®. Final, remarks on this present work and future perspectives are presented at the end.

Sistemas de telepresença têm sido pesquisados e desenvolvidos para inúmeras aplicações que exigem a presença física de pessoas em ambientes inacessíveis; tais situações são diversas, desde aquelas relacionadas com educação a distância até aquelas que envolvem alta periculosidade. Neste trabalho, a pesquisa e desenvolvimento se concentram na concepção e implementação de sistemas de telepresença voltados para ambientes imersivos em 360º, capazes de realizar a aquisição, transmissão e projeção de imagens em movimento em sistemas de multiprojeção imersivos, como é o caso da CAVERNA Digital®, desenvolvida pelo Laboratório de Sistemas Integráveis da Escola Politécnica da USP. Assim, o presente trabalho apresenta contribuições para o desenvolvimento de sistemas de telepresença, dentre os quais destacam-se: o detalhamento da arquitetura geral do sistema, a implementação de métodos para a calibração, correção das imagens e montagem de panoramas de 360º. Construiu-se um protótipo composto de: um Módulo de Aquisição, que adquire as imagens de oito câmeras (montadas num anel de câmeras sobre um robô), efetua as correções e monta uma imagem panorâmica; um Módulo de Composição que costura as várias imagens panorâmicas em um panorama final; e um Módulo de Exibição que ajusta e projeta o panorama nas telas da CAVERNA Digital®. Finalmente, apresentam-se considerações sobre o presente trabalho e perspectivas futuras. / Research and development in telepresence systems have been done in several applications that require physical presence of people into non-accessible environments. These situations may vary, from those related to distance education to those related to very hazardous places for humans. In this work, the main research and development goal is the conception and implementation of a telepresence system for 360 degrees immersive environments, this system is able to perform acquisition, transmission and projection of moving images on immersive multi-projection environments, such as the CAVERNA Digital®, developed by the Laboratório de Sistemas Integráveis at the Escola Politécnica, Universidade de São Paulo. The main contributions to the development of telepresence systems are: a detailed specification of the general architecture of the system, the implementation of the calibration, the imagery correction and the 360 degrees panoramas composition methods. The prototype that was implemented includes: an Acquisition Module, that acquires image of eight cameras (mounted on a ring of cameras placed on top of a robot), executes corrections and prepares a panoramic image; a Composition Module, that stitches the images in a final panorama; and an Exhibition Module, that adjusts and projects the panorama into the screens of the CAVERNA Digital®. Final, remarks on this present work and future perspectives are presented at the end.

The master thesis maps the state of the art of visual object tracking in panoramic 360° video. The thesis aims to reveal the main problems related to visual object tracking and moreover focuses on their solution in panoramic videos. In the study of the existing approaches was found that very few solutions of visual object tracking in equirectangular projection of panoramic video have been implemented so far. This thesis therefore presents two improvements of object tracking methods that are based on the adaptation of equirectangular frames. In addition, this thesis brings the manually created dataset of panoramic videos with more than 9900 annotations. Finally the detailed evaluation of 12 well known and state of the art trackers has been performed for this new dataset.

As unmanned air vehicle (UAV) utilization increases in Wilderness Search and Rescue (WiSAR) efforts, onboard sensors yielding more information will be desired. UAVs can assist WiSAR efforts by accelerating the ground search process through returning quality aerial footage of the terrain. Additionally, tracking the progress of a search by populating a digital map with video resolution data increases confidence that a comprehensive search of the region has been made. This thesis presents methods for acquiring video from multiple video sensors and fusing them into a single rendered video stream as a Virtual Gimbal. The panoramic video stream is the first of its kind to be constructed from video transmissions from a small UAV, and the first known video panorama to be used to quickly survey a region within a WiSAR context. The Virtual Gimbal comprises two video transmissions from a three camera array mounted in a downward-looking configuration on a UAV. This video stream has been shown to decrease the amount of time required to thoroughly survey a region by more than 40 percent.