No description available.

Detta examensarbete innefattar en utforskning om mitt arbete med immersivt ljud och hur det har påverkat min konstnärliga praktik som musikproducent. Utforskningen görs genom en analys av tre delstudier bestående av en 360-video som spelades in i S:t Jakobskyrkan i Stockholm, en VR upplevelse i en virtuell kopia av Nathan Milsteinsalen på Kungliga Musikhögskolan, samt en komposition som producerades med Dolby Atmos. Slutsatsen som dras är att arbetet har påverkat min konstnärliga praktik genom att ge mig nya insikter på komponerande, inspelning, och mixning. Slutligen ges förslag på fortsatta studier om audiovisuella metoder för immersiv musikproduktion samt nya infallsvinklar på stereomixning som är informerade av immersivt ljud. / This thesis examines how working with immersive audio has affected my artistic practice as a music producer. It does so by examining the insights gained during the production of three case studies consisting of a 360-video recorded at Saint James’s Church in Stockholm, Sweden, a Virtual Reality (VR) experience in a 3D replica of Nathan Milstein Hall at the Royal College of Music in Stockholm, as well as a composition produced using Dolby Atmos. The paper concludes that working with the case studies has changed my artistic practice by leading to new insights in the areas of composing, recording, and mixing. The paper also suggests future studies into the use of audiovisual approaches to immersive music production as well as considerations for stereo mix techniques based on insights gained from immersive audio.

In recent years, technological developments have led to the emergence of 360°. Google Street Map has provided panoramic imagery for a number of years, where users can immerse themselves and pan around in scenarios and follow streets of their liking, familiarizing and navigating themselves with the location. However, this service supports neither video nor moving imagery. This paper explores the use of 360° panoramic videos as a navigation aid on two different platforms as well as consider more subjective perspectives from potential users. The study is conducted through a set of interviews and a focus group where the findings reveal a number of design challenges concerning the development of a navigational system based on 360° panoramic videos. Based on these findings, design challenges are presented and later summarized as three design recommendations; 1) Design for navigation in a panoramic scenery, 2) Support overview and traditional navigation, 3) Support custom tailored features and content. The studies also suggest that the use of a head-mounted display (HMD) increase the feeling of presence which is beneficial for navigational purposes however, the users in this study preferred the mobile device based on practicality.

This is a study on what factors are important for a user to feel present when watching a 360° video of a musical experience. This is of interest because experiences in VR are still under exploration. Some artists are trying to live stream their concerts in VR showing that music in VR is something that is of interest. The video used in this study was recorded in Adolf Fredriks Kyrka, a church in Stockholm. I used a GoPro Omni Rig that records 360 degrees simultaneously to shoot the video in the church. The video was then shown in a user study with 23 participants divided into two groups, one VR-group that watched the video with an Oculus Rift and one control group that watched the video on a regular computer screen. The participants got to answer questions regarding their experience both before and after having watched the video. The result shows that there are few differences in the experience between the two groups. For example the VR-group felt more present in the virtual world than the control group. The thing that most participants commented on was the image quality of the video. It was not high enough according to some and it was also a bit blurry. Even though the image quality was a bit poor, the VR-group did not feel that it was too distracting for them to feel present. Overall the participants felt that it was a good and fun experience to watch. The most important factor for the feeling of presence is the level of immersion in the system. A fully immersive system will most likely give a higher feeling of presence than a non-immersive system. Two other factors that are important are the image quality and the audio. These three factors combined with interesting content will most likely lead to an experience where the user feels present in the virtual world and forgets the real world for a while. / Det här arbetet fokuserar på de faktorer som är viktiga för att en användare ska känna sig närvarande när han eller hon tittar på en 360° video av en musikupplevelse. Detta är intressant då upplevelser i VR är något som fortfarande utforskas. Artister har börjat livesända sina konserter i VR vilket visar på att just musik i VR är av intresse. Videon som användes i denna studie spelades in i Adolf Fredriks kyrka i Stockholm. För att spela in videon använde jag mig av en GoPro Omni Rig som spelar in i 360 grader samtidigt. Videon visades sedan i en användarstudie med 23 deltagare fördelat på två grupper, den ena gruppen var en VR-grupp som tittade på videon i en Oculus Rift och den andra gruppen var en kontrollgrupp som fick titta på videon på en vanlig datorskärm. Användarna fick svara på frågor gällande deras upplevelse både före och efter dem hade tittat på videon. Resultatet visar att det finns skillnader i upplevelsen mellan de två grupperna. Till exempel så kände sig VR-gruppen mer närvarande i den virtuella miljön än vad kontrollgruppen gjorde. Det som de flesta av användarna kommenterade på var kvaliteten på bilden. Upplösningen var inte tillräckligt hög och bilden var också lite suddig. Även om bildkvaliteten var låg så kände inte VR-gruppen att det distraherade dem för mycket från att känna sig närvarande. Överlag så tyckte användarna att det var en rolig och bra upplevelse att titta på. Den viktigaste faktorn för känslan av närvaro är nivån av immersion i systemet. Ett fullt immersivt system kommer troligtvis generera en högre känsla av närvaro än ett icke-immersivt system. Två andra viktiga faktorer är bildkvaliteten och ljudet. Dessa tre faktorer kombinerat med intressant innehåll, kommer med största sannolikhet leda till en upplevelse där användaren känner sig närvarande i den virtuella miljön och glömmer bort den riktiga världen för en stund.

A research report submitted to the faculty of Engineering and the built environment, University of the Witwatersrand, Johannesburg, in partial fulfilment of the requirements for the degree of Master of Science in Engineering. Johannesburg, August 2016 / There has been much interest in using the Microsoft Xbox 360 Kinect cameras for visual servo control applications. It is a relatively cheap device with expected shortcomings. This work contributes to the practical considerations of using the Kinect for visual servo control applications. A comprehensive characterisation of the Kinect is synthesised from existing literature and results from a nonlinear calibration procedure. The Kinect reduces computational overhead on image processing stages, such as pose estimation or depth estimation. It is limited by its 0.8m to 3.5m practical depth range and quadratic depth resolution of 1.8mm to 35mm, respectively. Since the Kinect uses an infra-red (IR) projector, a class one laser, it should not be used outdoors, due to IR saturation, and objects belonging to classes of non- IR-friendly surfaces should be avoided, due to IR refraction, absorption, or specular reflection. Problems of task stability due to invalid depth measurements in Kinect depth maps and practical depth range limitations can be reduced by using depth map preprocessing and activating classical visual servoing techniques when Kinect-based approaches are near task failure. / MT2017

Les vidéos omnidirectionnelles, également appelées vidéos sphériques ou vidéos360°, sont des vidéos avec des pixels enregistrés dans toutes les directions de l’espace. Un utilisateur qui regarde un tel contenu avec un Casques de Réalité Virtuelle (CRV) peut sélectionner la partie de la vidéo à afficher, usuellement nommée viewport, en bougeant la tête. Pour se sentir totalement immergé à l’intérieur du contenu, l’utilisateur a besoin de voir au moins 90 viewports par seconde en 4K. Avec les technologies de streaming traditionnelles, fournir une telle qualité nécessiterait un débit de plus de100 Mbit s−1, ce qui est bien trop élevé. Dans cette thèse, je présente mes contributions pour rendre possible le streaming de vidéos omnidirectionnelles hautement immersives sur l’Internet. On peut distinguer six contributions : une proposition d’architecture de streaming viewport adaptatif réutilisant une partie des technologies existantes ; une extension de cette architecture pour des vidéos à six degrés de liberté ; deux études théoriques des vidéos à qualité spatiale non-homogène; un logiciel open source de manipulation des vidéos 360°; et un jeu d’enregistrements de déplacements d’utilisateurs regardant des vidéos 360°. / Omnidirectional videos, also denoted as spherical videos or 360° videos, are videos with pixels recorded from a given viewpoint in every direction of space. A user watching such an omnidirectional content with a Head Mounted Display (HMD) can select the portion of the videoto display, usually denoted as viewport, by moving her head. To feel high immersion inside the content a user needs to see viewport with 4K resolutionand 90 Hz frame rate. With traditional streaming technologies, providing such quality would require a data rate of more than 100 Mbit s−1, which is far too high compared to the median Internet access band width. In this dissertation, I present my contributions to enable the streaming of highly immersive omnidirectional videos on the Internet. We can distinguish six contributions : a viewport-adaptive streaming architecture proposal reusing a part of existing technologies ; an extension of this architecture for videos with six degrees of freedom ; two theoretical studies of videos with non homogeneous spatial quality ; an open-source software for handling 360° videos ; and a dataset of recorded users’ trajectories while watching 360° videos.

[pt] A popularidade de plataformas para o armazenamento e compartilhamento de vídeo tem criado um volume massivo de horas de vídeo. Dado um conjunto de atores presentes em um vídeo, a geração de metadados com a determinação temporal dos intervalos em que cada um desses atores está presente, bem como a localização no espaço 2D dos quadros em cada um desses intervalos pode facilitar a recuperação de vídeo e a recomendação. Neste trabalho, nós investigamos a Clusterização Facial em Vídeo para a localização espaço-temporal de atores. Primeiro descrevemos nosso método de Clusterização Facial em Vídeo em que utilizamos métodos de detecção facial, geração de embeddings e clusterização para agrupar faces dos atores em diferentes quadros e fornecer a localização espaço-temporal destes atores. Então, nós exploramos, propomos, e investigamos aplicações inovadoras dessa localização espaço-temporal em três diferentes tarefas: (i) Reconhecimento Facial em Vídeo, (ii) Recomendação de Vídeos Educacionais e (iii) Posicionamento de Legendas em Vídeos 360 graus. Para a tarefa (i), propomos um método baseado na similaridade de clústeres que é facilmente escalável e obteve um recall de 99.435 por cento e uma precisão de 99.131 por cento em um conjunto de vídeos. Para a tarefa (ii), propomos um método não supervisionado baseado na presença de professores em diferentes vídeos. Tal método não requer nenhuma informação adicional sobre os vídeo e obteve um valor mAP aproximadamente 99 por cento. Para a tarefa (iii), propomos o posicionamento dinâmico de legendas baseado na localização de atores em vídeo 360 graus. / [en] The popularity of platforms for the storage and transmission of video content has created a substantial volume of video data. Given a set of actors present in a video, generating metadata with the temporal determination of the interval in which each actor is present, and their spatial 2D localization in each frame in these intervals can facilitate video retrieval and recommendation. In this work, we investigate Video Face Clustering for this spatio-temporal localization of actors in videos. We first describe our method for Video Face Clustering in which we take advantage of face detection, embeddings, and clustering methods to group similar faces of actors in different frames and provide the spatio-temporal localization of them. Then, we explore, propose, and investigate innovative applications of this spatio-temporal localization in three different tasks: (i) Video Face Recognition, (ii) Educational Video Recommendation and (iii) Subtitles Positioning in 360-video. For (i), we propose a cluster-matching-based method that is easily scalable and achieved a recall of 99.435 percent and precision of 99.131 percent in a small video set. For (ii), we propose an unsupervised method based on them presence of lecturers in different videos that does not require any additional information from the videos and achieved a mAP approximately 99 percent. For (iii), we propose a dynamic placement of subtitles based on the automatic localization of actors in 360-video.

The master thesis maps the state of the art of visual object tracking in panoramic 360° video. The thesis aims to reveal the main problems related to visual object tracking and moreover focuses on their solution in panoramic videos. In the study of the existing approaches was found that very few solutions of visual object tracking in equirectangular projection of panoramic video have been implemented so far. This thesis therefore presents two improvements of object tracking methods that are based on the adaptation of equirectangular frames. In addition, this thesis brings the manually created dataset of panoramic videos with more than 9900 annotations. Finally the detailed evaluation of 12 well known and state of the art trackers has been performed for this new dataset.

Students, especially international ones have a hard time seeking accommodation in big cities like Stockholm in Sweden at an adequate pricing. To facilitate such a search process and to create a sense of trust, tenants and landlords use video and/or voice calls for early communication with one another. However, the establishment of trust among tenants and landlords in this context is a challenging task. This study thus aims to investigate if innovative virtual reality (VR) approaches like 360° videos can be used to develop trust among tenants and landlords. This study undertakes an investigation from student perspective due to the limited timeframe. This study was carried out in collaboration with Ett tak (English: One roof) which is a startup working in the rental market in Stockholm. The tenants are the students and the landlords are senior people having spare space in their homes. A pre-study was conducted to identify the expectations of the students. A landlord was filmed for the 360° video which was developed into a Unity VR app. The results from the usability study indicate that most students felt a high degree of trust towards the landlord by using an immersive 360° VR tour. Conversely, the students showed hesitation to use the 360° VR by buying their own VR headsets unless VR becomes mainstream in the different areas of life. This study builds upon existing literature that validate the positive impact of attributes like profile photos, ratings and reviews on the trust level between the providers and the consumers in a sharing economy. / Studenter, särskilt internationella, har svårt att hitta boende i storstäder som Stockholm i Sverige till lämplig prissättning. För att underlätta en sådan sökprocess och skapa en känsla av förtroende använder hyresgäster och hyresvärdar video och / eller röstsamtal för tidig kommunikation med varandra. Upprättande av förtroende bland hyresgäster och hyresvärdar är dock en utmanande uppgift i detta sammanhang. Denna studie syftar således till att undersöka om virtuell verklighet (VR) som 360° video kan användas för att utveckla förtroende bland hyresgäster och hyresvärdar. Denna studie genomför en undersökning från ett studentperspektiv på grund av den begränsade tidsramen. Denna studie genomfördes i samverkan med Ett tak, som arbetar på hyresmarknaden i Stockholm. Hyresgästerna är studenterna och hyresvärdarna är äldre människor som har ledigt utrymme i sina hem. En förstudie genomfördes för att identifiera studenternas förväntningar. En hyresvärd filmades för en 360° video som utvecklades till en Unity VR app. Resultaten från användbarhetsstudien indikerar att de flesta studenter kände en hög grad av förtroende gentemot hyresvärden genom att använda en fördjupande 360 ° VR rundtur. Däremot visade eleverna att de inte skulle köpa egna VR-headset om VR inte blir vanligt i de olika områdena i livet. Denna studie bygger på befintlig litteratur och bekräftar den positiva effekten av attribut som profilbilder, betyg och recensioner på förtroendenivå mellan leverantörer och konsumenter i en delningsekonomi.

This Master Thesis concerns a telepresence implementation which utilizes state-of-the-art virtual reality combined with live 360 degree video. Navigation interfaces for telepresence with virtual reality headsets were developed and evaluated through a user study. An evaluation of telepresence as a communication media was performed, comparing it to video communication. The result showed that telepresence was a better communication media than video communication.