No description available.

Detta examensarbete innefattar en utforskning om mitt arbete med immersivt ljud och hur det har påverkat min konstnärliga praktik som musikproducent. Utforskningen görs genom en analys av tre delstudier bestående av en 360-video som spelades in i S:t Jakobskyrkan i Stockholm, en VR upplevelse i en virtuell kopia av Nathan Milsteinsalen på Kungliga Musikhögskolan, samt en komposition som producerades med Dolby Atmos. Slutsatsen som dras är att arbetet har påverkat min konstnärliga praktik genom att ge mig nya insikter på komponerande, inspelning, och mixning. Slutligen ges förslag på fortsatta studier om audiovisuella metoder för immersiv musikproduktion samt nya infallsvinklar på stereomixning som är informerade av immersivt ljud. / This thesis examines how working with immersive audio has affected my artistic practice as a music producer. It does so by examining the insights gained during the production of three case studies consisting of a 360-video recorded at Saint James’s Church in Stockholm, Sweden, a Virtual Reality (VR) experience in a 3D replica of Nathan Milstein Hall at the Royal College of Music in Stockholm, as well as a composition produced using Dolby Atmos. The paper concludes that working with the case studies has changed my artistic practice by leading to new insights in the areas of composing, recording, and mixing. The paper also suggests future studies into the use of audiovisual approaches to immersive music production as well as considerations for stereo mix techniques based on insights gained from immersive audio.

In recent years, technological developments have led to the emergence of 360°. Google Street Map has provided panoramic imagery for a number of years, where users can immerse themselves and pan around in scenarios and follow streets of their liking, familiarizing and navigating themselves with the location. However, this service supports neither video nor moving imagery. This paper explores the use of 360° panoramic videos as a navigation aid on two different platforms as well as consider more subjective perspectives from potential users. The study is conducted through a set of interviews and a focus group where the findings reveal a number of design challenges concerning the development of a navigational system based on 360° panoramic videos. Based on these findings, design challenges are presented and later summarized as three design recommendations; 1) Design for navigation in a panoramic scenery, 2) Support overview and traditional navigation, 3) Support custom tailored features and content. The studies also suggest that the use of a head-mounted display (HMD) increase the feeling of presence which is beneficial for navigational purposes however, the users in this study preferred the mobile device based on practicality.

In this thesis we study users' viewing motions when watching 360° videos in order to provide information that can be used to optimize future view-dependent streaming protocols. More specifically, we develop an application that plays a sequence of 360° videos on an Oculus Rift Head Mounted Display and records the orientation and rotation velocity of the headset during playback. The application is used during an extensive user study in order to collect more than 21 hours of viewing data which is then analysed to expose viewing patterns, useful for optimizing 360° streaming protocols.

In this thesis we study users' viewing motions when watching 360° videos in order to provide information that can be used to optimize future view-dependent streaming protocols. More specifically, we develop an application that plays a sequence of 360° videos on an Oculus Rift Head Mounted Display and records the orientation and rotation velocity of the headset during playback. The application is used during an extensive user study in order to collect more than 21 hours of viewing data which is then analysed to expose viewing patterns, useful for optimizing 360° streaming protocols.

This is a study on what factors are important for a user to feel present when watching a 360° video of a musical experience. This is of interest because experiences in VR are still under exploration. Some artists are trying to live stream their concerts in VR showing that music in VR is something that is of interest. The video used in this study was recorded in Adolf Fredriks Kyrka, a church in Stockholm. I used a GoPro Omni Rig that records 360 degrees simultaneously to shoot the video in the church. The video was then shown in a user study with 23 participants divided into two groups, one VR-group that watched the video with an Oculus Rift and one control group that watched the video on a regular computer screen. The participants got to answer questions regarding their experience both before and after having watched the video. The result shows that there are few differences in the experience between the two groups. For example the VR-group felt more present in the virtual world than the control group. The thing that most participants commented on was the image quality of the video. It was not high enough according to some and it was also a bit blurry. Even though the image quality was a bit poor, the VR-group did not feel that it was too distracting for them to feel present. Overall the participants felt that it was a good and fun experience to watch. The most important factor for the feeling of presence is the level of immersion in the system. A fully immersive system will most likely give a higher feeling of presence than a non-immersive system. Two other factors that are important are the image quality and the audio. These three factors combined with interesting content will most likely lead to an experience where the user feels present in the virtual world and forgets the real world for a while. / Det här arbetet fokuserar på de faktorer som är viktiga för att en användare ska känna sig närvarande när han eller hon tittar på en 360° video av en musikupplevelse. Detta är intressant då upplevelser i VR är något som fortfarande utforskas. Artister har börjat livesända sina konserter i VR vilket visar på att just musik i VR är av intresse. Videon som användes i denna studie spelades in i Adolf Fredriks kyrka i Stockholm. För att spela in videon använde jag mig av en GoPro Omni Rig som spelar in i 360 grader samtidigt. Videon visades sedan i en användarstudie med 23 deltagare fördelat på två grupper, den ena gruppen var en VR-grupp som tittade på videon i en Oculus Rift och den andra gruppen var en kontrollgrupp som fick titta på videon på en vanlig datorskärm. Användarna fick svara på frågor gällande deras upplevelse både före och efter dem hade tittat på videon. Resultatet visar att det finns skillnader i upplevelsen mellan de två grupperna. Till exempel så kände sig VR-gruppen mer närvarande i den virtuella miljön än vad kontrollgruppen gjorde. Det som de flesta av användarna kommenterade på var kvaliteten på bilden. Upplösningen var inte tillräckligt hög och bilden var också lite suddig. Även om bildkvaliteten var låg så kände inte VR-gruppen att det distraherade dem för mycket från att känna sig närvarande. Överlag så tyckte användarna att det var en rolig och bra upplevelse att titta på. Den viktigaste faktorn för känslan av närvaro är nivån av immersion i systemet. Ett fullt immersivt system kommer troligtvis generera en högre känsla av närvaro än ett icke-immersivt system. Två andra viktiga faktorer är bildkvaliteten och ljudet. Dessa tre faktorer kombinerat med intressant innehåll, kommer med största sannolikhet leda till en upplevelse där användaren känner sig närvarande i den virtuella miljön och glömmer bort den riktiga världen för en stund.

A research report submitted to the faculty of Engineering and the built environment, University of the Witwatersrand, Johannesburg, in partial fulfilment of the requirements for the degree of Master of Science in Engineering. Johannesburg, August 2016 / There has been much interest in using the Microsoft Xbox 360 Kinect cameras for visual servo control applications. It is a relatively cheap device with expected shortcomings. This work contributes to the practical considerations of using the Kinect for visual servo control applications. A comprehensive characterisation of the Kinect is synthesised from existing literature and results from a nonlinear calibration procedure. The Kinect reduces computational overhead on image processing stages, such as pose estimation or depth estimation. It is limited by its 0.8m to 3.5m practical depth range and quadratic depth resolution of 1.8mm to 35mm, respectively. Since the Kinect uses an infra-red (IR) projector, a class one laser, it should not be used outdoors, due to IR saturation, and objects belonging to classes of non- IR-friendly surfaces should be avoided, due to IR refraction, absorption, or specular reflection. Problems of task stability due to invalid depth measurements in Kinect depth maps and practical depth range limitations can be reduced by using depth map preprocessing and activating classical visual servoing techniques when Kinect-based approaches are near task failure. / MT2017

Les vidéos omnidirectionnelles, également appelées vidéos sphériques ou vidéos360°, sont des vidéos avec des pixels enregistrés dans toutes les directions de l’espace. Un utilisateur qui regarde un tel contenu avec un Casques de Réalité Virtuelle (CRV) peut sélectionner la partie de la vidéo à afficher, usuellement nommée viewport, en bougeant la tête. Pour se sentir totalement immergé à l’intérieur du contenu, l’utilisateur a besoin de voir au moins 90 viewports par seconde en 4K. Avec les technologies de streaming traditionnelles, fournir une telle qualité nécessiterait un débit de plus de100 Mbit s−1, ce qui est bien trop élevé. Dans cette thèse, je présente mes contributions pour rendre possible le streaming de vidéos omnidirectionnelles hautement immersives sur l’Internet. On peut distinguer six contributions : une proposition d’architecture de streaming viewport adaptatif réutilisant une partie des technologies existantes ; une extension de cette architecture pour des vidéos à six degrés de liberté ; deux études théoriques des vidéos à qualité spatiale non-homogène; un logiciel open source de manipulation des vidéos 360°; et un jeu d’enregistrements de déplacements d’utilisateurs regardant des vidéos 360°. / Omnidirectional videos, also denoted as spherical videos or 360° videos, are videos with pixels recorded from a given viewpoint in every direction of space. A user watching such an omnidirectional content with a Head Mounted Display (HMD) can select the portion of the videoto display, usually denoted as viewport, by moving her head. To feel high immersion inside the content a user needs to see viewport with 4K resolutionand 90 Hz frame rate. With traditional streaming technologies, providing such quality would require a data rate of more than 100 Mbit s−1, which is far too high compared to the median Internet access band width. In this dissertation, I present my contributions to enable the streaming of highly immersive omnidirectional videos on the Internet. We can distinguish six contributions : a viewport-adaptive streaming architecture proposal reusing a part of existing technologies ; an extension of this architecture for videos with six degrees of freedom ; two theoretical studies of videos with non homogeneous spatial quality ; an open-source software for handling 360° videos ; and a dataset of recorded users’ trajectories while watching 360° videos.

[pt] A popularidade de plataformas para o armazenamento e compartilhamento de vídeo tem criado um volume massivo de horas de vídeo. Dado um conjunto de atores presentes em um vídeo, a geração de metadados com a determinação temporal dos intervalos em que cada um desses atores está presente, bem como a localização no espaço 2D dos quadros em cada um desses intervalos pode facilitar a recuperação de vídeo e a recomendação. Neste trabalho, nós investigamos a Clusterização Facial em Vídeo para a localização espaço-temporal de atores. Primeiro descrevemos nosso método de Clusterização Facial em Vídeo em que utilizamos métodos de detecção facial, geração de embeddings e clusterização para agrupar faces dos atores em diferentes quadros e fornecer a localização espaço-temporal destes atores. Então, nós exploramos, propomos, e investigamos aplicações inovadoras dessa localização espaço-temporal em três diferentes tarefas: (i) Reconhecimento Facial em Vídeo, (ii) Recomendação de Vídeos Educacionais e (iii) Posicionamento de Legendas em Vídeos 360 graus. Para a tarefa (i), propomos um método baseado na similaridade de clústeres que é facilmente escalável e obteve um recall de 99.435 por cento e uma precisão de 99.131 por cento em um conjunto de vídeos. Para a tarefa (ii), propomos um método não supervisionado baseado na presença de professores em diferentes vídeos. Tal método não requer nenhuma informação adicional sobre os vídeo e obteve um valor mAP aproximadamente 99 por cento. Para a tarefa (iii), propomos o posicionamento dinâmico de legendas baseado na localização de atores em vídeo 360 graus. / [en] The popularity of platforms for the storage and transmission of video content has created a substantial volume of video data. Given a set of actors present in a video, generating metadata with the temporal determination of the interval in which each actor is present, and their spatial 2D localization in each frame in these intervals can facilitate video retrieval and recommendation. In this work, we investigate Video Face Clustering for this spatio-temporal localization of actors in videos. We first describe our method for Video Face Clustering in which we take advantage of face detection, embeddings, and clustering methods to group similar faces of actors in different frames and provide the spatio-temporal localization of them. Then, we explore, propose, and investigate innovative applications of this spatio-temporal localization in three different tasks: (i) Video Face Recognition, (ii) Educational Video Recommendation and (iii) Subtitles Positioning in 360-video. For (i), we propose a cluster-matching-based method that is easily scalable and achieved a recall of 99.435 percent and precision of 99.131 percent in a small video set. For (ii), we propose an unsupervised method based on them presence of lecturers in different videos that does not require any additional information from the videos and achieved a mAP approximately 99 percent. For (iii), we propose a dynamic placement of subtitles based on the automatic localization of actors in 360-video.

The master thesis maps the state of the art of visual object tracking in panoramic 360° video. The thesis aims to reveal the main problems related to visual object tracking and moreover focuses on their solution in panoramic videos. In the study of the existing approaches was found that very few solutions of visual object tracking in equirectangular projection of panoramic video have been implemented so far. This thesis therefore presents two improvements of object tracking methods that are based on the adaptation of equirectangular frames. In addition, this thesis brings the manually created dataset of panoramic videos with more than 9900 annotations. Finally the detailed evaluation of 12 well known and state of the art trackers has been performed for this new dataset.