We are witnessing a proliferation of video in the Internet; YouTube is the most bandwidth intensive service of today’s Internet. It accounts for 20 - 35% of the Internet traffic with 35 hours of videos uploaded every minute and more than 700 billion playbacks in 2010. Netflix, a web service that streams premium contents such as TV series, shows, and movies, consumes 30% of the network bandwidth in North America at peak time. Recently, leveraging the content distribution networks (CDNs), a new paradigm for video streaming on the Internet has emerged, namely, Adaptive HTTP Streaming (AHS). AHS has become the industry standard for video streaming over the Internet adopted by broadcast networks as well as VoD services such as YouTube, Netflix, Hulu, etc. In the 90’s and early 2000’s, Internet-based video streaming for high-bitrate video was challenging due to hardware limitations. In that era, to cover the hardware limitations, every software component of a video server needed to be carefully optimized to support the real-time guarantees for jitter-free video delivery. However, most of the software solutions have become less important with the remarkable hardware improvements over the past two decades. There is 100× speedup in CPU speeds; RAM capacity has increased by 1,000×; hard disk drive (HDD) capacity has grown by 10,000×. Today, CPU is no longer a bottleneck for video streaming. On the other hand, storage bandwidth and network bandwidth are still serious bottlenecks for large scale on-demand video streaming. In this dissertation, we aim at a scalable video content distribution system that addresses both storage bottleneck and network bottleneck. The first part of the dissertation pertains to the storage system on the server side: A multi-tiered storage system that exploits a flash memory solid-state drive (SSD) can meet the bandwidth needs in a much more cost- effective way than a traditional two-tier storage system. We first identify the challenges in architecting such a system given the performance quirks of flash-based SSDs, and the lim- itations of state-of-the-art multi-tiered storage systems for video streaming. Armed with the knowledge of these challenges, we show how to construct such a storage system and implement a real web server with multi-tiered storage, evaluate the system with AHS work- loads, and demonstrate significant performance gains while reducing the TCO. The second part of the dissertation pertains to the network system on the client side: Integrating peer- to-peer (P2P) technology with the client-server paradigm results in a much more scalable video content distribution system. AHS is a paradigm for client-driven video streaming; its philosophy matches well with that of P2P video streaming. An adaptation mechanism is the most important component of AHS that determines overall video streaming quality and user experience. We show a throughput-smoothing-based adaptation mechanism that is designed for a client-server architecture does not work well for a P2P architecture. We pro- vide a buffer-based adaptation mechanism, evaluate our solution with OMNeT++/INET simulator, and demonstrate significant performance gains.

In recent years, adaptive HTTP streaming protocols have become the de facto standard in the industry for the distribution of live and video-on-demand content over the Internet. In this thesis, we solve the problem of distributing adaptive HTTP live video streams to a large number of viewers using peer-to-peer (P2P) overlays. We do so by assuming that our solution must deliver a level of quality of user experience which is the same as a CDN while trying to minimize the load on the content provider’s infrastructure. Besides that, in the design of our solution, we take into consideration the realities of the HTTP streaming protocols, such as the pull-based approach and adaptive bitrate switching. The result of this work is a system which we call SmoothCache that provides CDN-quality adaptive HTTP live streaming utilizing P2P algorithms. Our experiments on a real network of thousands of consumer machines show that, besides meeting the the CDN-quality constraints, SmoothCache is able to consistently deliver up to 96% savings towards the source of the stream in a single bitrate scenario and 94% in a multi-bitrate scenario. In addition, we have conducted a number of pilot deployments in the setting of large enterprises with the same system, albeit tailored to private networks. Results with thousands of real viewers show that our platform provides an average offloading of bottlenecks in the private network of 91.5%. These achievements were made possible by advancements in multiple research areas that are also presented in this thesis. Each one of the contributions is novel with respect to the state of the art and can be applied outside of the context of our application. However, in our system they serve the purposes described below. We built a component-based event-driven framework to facilitate the development of our live streaming application. The framework allows for running the same code both in simulation and in real deployment. In order to obtain scalability of simulations and accuracy, we designed a novel flow-based bandwidth emulation model. In order to deploy our application on real networks, we have developed a network library which has the novel feature of providing on-the-fly prioritization of transfers. The library is layered over the UDP protocol and supports NAT Traversal techniques. As part of this thesis, we have also improved on the state of the art of NAT Traversal techniques resulting in higher probability of direct connectivity between peers on the Internet. Because of the presence of NATs on the Internet, discovery of new peers and collection of statistics on the overlay through peer sampling is problematic. Therefore, we created a peer sampling service which is NAT-aware and provides one order of magnitude fresher samples than existing peer sampling protocols. Finally, we designed SmoothCache as a peer-assisted live streaming system based on a distributed caching abstraction. In SmoothCache, peers retrieve video fragments from the P2P overlay as quickly as possible or fall back to the source of the stream to keep the timeliness of the delivery. In order to produce savings, the caching system strives to fill up the local cache of the peers ahead of playback by prefetching content. Fragments are efficiently distributed by a self-organizing overlay network that takes into account many factors such as upload bandwidth capacity, connectivity constraints, performance history and the currently being watched bitrate. / <p>QC 20131122</p>

Submitted by Renata Lopes (renatasil82@gmail.com) on 2016-05-17T12:01:58Z No. of bitstreams: 1 wagnerdealmeidajunior.pdf: 2594589 bytes, checksum: 9b2a2375eb8725cfda2a8aabadd86f22 (MD5) / Approved for entry into archive by Adriana Oliveira (adriana.oliveira@ufjf.edu.br) on 2016-06-28T14:01:42Z (GMT) No. of bitstreams: 1 wagnerdealmeidajunior.pdf: 2594589 bytes, checksum: 9b2a2375eb8725cfda2a8aabadd86f22 (MD5) / Made available in DSpace on 2016-06-28T14:01:42Z (GMT). No. of bitstreams: 1 wagnerdealmeidajunior.pdf: 2594589 bytes, checksum: 9b2a2375eb8725cfda2a8aabadd86f22 (MD5) Previous issue date: 2015-09-04 / A transmissão de conteúdo de mídia pela Internet é uma tendência que vem crescendo atualmente. Especialmente, o cenário de transmissão de vídeo sob demanda (VoD) já é bastante consolidado. Um novo desafio que surge é a transmissão de vídeo ao vivo, especialmente de conteúdo capaz de atrair um grande número de espectadores como eventos de grande escala. Neste trabalho nós caracterizamos a transmissão ao vivo pela Internet de dois eventos esportivos de grande escala. Esses eventos foram transmitidos gratuitamente por um dos maiores provedores de conteúdo da América Latina através de um portal web acessado pelo protocolo HTTP. Nos baseamos em registros de acesso dos servidores web do provedor para extrair as informações usadas nas caracterizações. Nós quantificamos o impacto da transmissão de um evento de grande escala no provedor de conteúdo e a qualidade de serviço recebida pelos usuários finais. Mostramos que os servidores de vídeo recebem carga substancial e que a carga varia significativamente entre jogos, o que dificulta o provisionamento de capacidade. Além disso apresentamos um mapa da difusão de acesso por região do Brasil com base em observações sobre a mídia recebida pelos clientes. / Streaming media on internet is currently a growing trend. In particular, video-on-demand (VoD) scenario is already quite consolidated. Although, live streaming still facing challenges, specially during large-scale events, which may attract a large number of viewers. In this work, we characterize two large-scale Internet live streaming sport events. These events were streamed free of charge through a web portal accessed via HTTP protocol by one of the largest content provider in Latin America. Our characterization relies on access logs we have obtained from the content provider’s web server. We quantify the impact of a large-scale event transmission on content provider servers. We also characterize the quality of service end users receive. Our results show that video servers receive substantial network load during a large-scale event. Moreover, this network load can vary significantly among distinct matches, which difficult capacity provisioning. In addition we present a map of the Brazilian Internet access distribution based on observations about the media content received by customers.

Aim of the thesis is to design and implement unique service based on the newest technologies in web apps fi eld which allows screen recording followed by P2P distribution between participating users using web browser only. Thesis deals with the analysis of modern and coming technologies which allow practical implementation of such a service. It involves obtaining access to the screen source data, its recording, transforming and transmission over the network followed by playing on the other side. Recorded videos are adapted for part by part use in a player with seeking capability. Distributed architecture for data exchange between peers using peer-to-peer connection based on hybrid P2P VoD network provides scalability of the solution. The player allows obtaining the necessary parts of the videos from the current watchers with smooth video playback experience during ongoing transmissions whilst allowing arbitrary video shifting. In consideration of early stages of development or experimental support for some of the APIs needed for this work, research into browsers' support with discussion on realistic applicability nowadays is always performed.

Konsumtionen av video på Internet har ökat det senaste decenniet och trenden är förväntad att fortsätta stiga. Strömmande video är en ofta använd lösning för att se video på Internet. Tekniken tillåter användaren att se film medan den laddas ned. Adaptive streaming är en teknik som låter videospelaren ändra bit-raten på videon baserat på användarens tillgängliga bandbredd. En annan lösning för att göra användarens upplevelse mer skräddarsydd är ickelinjära videor. Den här typen av videor kan spelas ur sekvens. Som exempel kan en tittare välja slutet i en film eller välja vägen genom en utställning. Examensarbetet presenterar designen och implementeringen av en ny struktur för ickelinjär video. Denna struktur används av videospelaren för nedladdning och visning av adaptiv video avsedd för ickelinjära filmer, som sparas på en server. Mediaskapare kommer också att ha lättare att både visualisera och skapa filmer på ett enhetligt sätt. Det här examensarbetet presenterar i detalj de modifikationer på Adobes Open Source Media Framework och Strobe Media Playback som tillåter spelning av ickelinjära videor. Ändringarna innefattar ett användargränssnitt för ickelinjär media. Back-end implementeringarna innefattar bufferthantering för parallell nedladdning och tekniker för att hantera den nya strukturen. Slutligen presenterar det här examensarbetet proof-of-concept-tester för att styrka funktionaliteten av designen och implementeringen. Rapporten avslutas med en diskussion om framtida arbete inom fältet. / Video consumption on the Internet has been growing over the last decade and is expected to continue to increase. Video streaming is a widely used approach to viewing video on the Internet, which allows users to watch the video while it is being downloaded. Adaptive streaming is a video streaming technique that allows the player to change the downloading video’s bit rate depending on the user’s available bandwidth. Another approach to a more personal viewing experience is non-linear videos. These videos can be played in a non-sequential order. For example, a viewer can be given the choice of what ending to watch in a movie, or the path through an exhibition. This thesis will present the design and implementation of a novel structure for non-linear video. This structure is used by a video player for downloading and viewing an adaptive video intended for non-linear viewing, stored at some server. Media creators will also have an easier time to both visualize and create uniform video experiences. This thesis presents modifications to Adobe’s Open Source Media Framework and Strobe Media Playback which allow playing non-linear video. Presented in this thesis are the design and implementation details. Changes in the player include a user interface for non-linear media. The changes to the back-end include buffer management for parallel downloading and techniques to handle the new structure. Finally, this thesis presents proof-of-concept validation tests that show the functionality of the design and implementation. The thesis is concluded with a discussion of future work in this area.