A compressão de vídeo é essencial para aplicações que manipulam vídeos digitais, em função da enorme quantidade de informação necessária para representar um vídeo sem nenhum tipo de compressão. Esta tese apresenta o desenvolvimento de soluções arquiteturais dedicadas e de alto desempenho para a compressão de vídeos, com foco no padrão H.264/AVC. O padrão H.264/AVC é o mais novo padrão de compressão de vídeo da ITU-T e da ISO e atinge as mais elevadas taxas de compressão dentre todos os padrões de codificação de vídeo existentes. Este padrão também possui a maior complexidade computacional dentre os padrões atuais. Esta tese apresenta soluções arquiteturais para os módulos da estimação de movimento, da compensação de movimento, das transformadas diretas e inversas e da quantização direta e inversa. Inicialmente, são apresentados alguns conceitos básicos de compressão de vídeo e uma introdução ao padrão H.264/AVC, para embasar as explicações das soluções arquiteturais desenvolvidas. Então, as arquiteturas desenvolvidas para os módulos das transformadas diretas e inversas, da quantização direta e inversa, da estimação de movimento e da compensação de movimento são apresentadas. Todas as arquiteturas desenvolvidas foram descritas em VHDL e foram mapeadas para FPGAs Virtex-II Pro da Xilinx. Alguns dos módulos foram, também, sintetizados para standard-cells. Os resultados obtidos através da síntese destas arquiteturas são apresentados e discutidos. Para todos os casos, os resultados de síntese indicaram que as arquiteturas desenvolvidas estão aptas para atender as demandas de codecs H.264/AVC direcionados para vídeos de alta resolução. / Video coding is essential for applications based in digital videos, given the enormous amount of bits which are required to represent a video sequence without compression. This thesis presents the design of dedicated and high performance architectures for video compression, focusing in the H.264/AVC standard. The H.264/AVC standard is the latest ITU-T and ISO standard for video compression and it reaches the highest compression rates amongst all the current video coding standards. This standard has also the highest computational complexity among all of them. This thesis presents architectural solutions for the modules of motion estimation, motion compensation, forward and inverse transforms and forward and inverse quantization. Some concepts of video compression and an introduction to the H.264/AVC standard are presented and they serve as basis for the architectural developments. Then, the designed architectures for forward and inverse transforms, forward and inverse quantization, motion estimation and motion compensation are presented. All designed architectures were described in VHDL and they were mapped to Xilinx Virtex-II Pro FPGAs. Some modules were also synthesized into standard-cells. The synthesis results are presented and discussed. For all cases, the synthesis results indicated that the architectures developed in this work are able to meet the demands of H.264/AVC codecs targeting high resolution videos.

Um codificador de entropia é responsável pela representação simbólica de dados de forma a representá-los com um menor número de bits. O H.264/AVC possui três codificadores de entropia: o Exponencial Golomb, o CAVLC que é o codificador de menor complexidade porém com um throughput maior de dados e o CABAC, com maior complexidade e com uma maior capacidade de compressão. A complexidade do codificador de entropia e a dependência dos dados sequenciais no bitstream original são os principais desafios para atender os requisitos de desempenho para compressão em tempo real. Por isso o desenvolvimento destas arquiteturas em hardware dedicado se faz necessário. Neste contexto, esta dissertação descreve os algoritmos que fazem parte da entropia do padrão H.264/AVC e as arquiteturas para estes codificadores entrópicos (Exponential Golomb, CAVLC e CABAC), além de uma arquitetura de hardware dedicada que integra todos estes a um montador final que atende às especificações da norma H.264/AVC. As arquiteturas foram escritas em VHDL e sintetizadas para dispositivos integrados FPGA. Em um dispositivo Virtex-5, este codificador de entropia completo suporta codificação de vídeos no nível 4.2 do padrão H.264/AVC (Full HD a 60 quadros por segundo). Esta arquitetura é a que apresenta o melhor desempenho de processamento dentre os melhores trabalhos relacionados, além de ser um codificador com todas as alternativas de codificação de entropia requeridas pela norma implementadas em um mesmo módulo. / An entropy encoder is responsible for the symbolic representation of a data stream so that the final representation contains less bits than the original. The H.264/AVC has three entropy coding schemes: the Exponential Golomb, the CAVLC encoder, that is less complex but with a higher data throughput, and the CABAC that is more complex while allowing for higher compression capability. The complexity of the entropy encoding and data dependencies on the original bitstream are the main challenges to meet the performance requirements for real-time compression. The development of these architectures in dedicated hardware is therefore necessary for high performance encoders. In this context, this work describes the algorithms that are part of the entropy encoders of the H.264/AVC standard, and the corresponding entropy coding architectures (Exponential Golomb, CAVLC and CABAC), plus a dedicated hardware architecture that integrates all of these encoders to a final bitstream assembler that is compliant to the aforementioned standard. The architectures were written in VHDL and synthesized into FPGA devices. In a Virtex-5 device, this full entropy encoder supports video encoding at level 4.2 of the H.264/AVC standard (Full HD at 60 frames per second). The developed architecture performs best among the most recent related architectures published, and has the unique feature of an encoder that implements in the same module all the alternative entropy encoders present in this standard for video compression.

Codificadores HEVC impõem diversos desafios em aplicações embarcadas com restrições computacionais, especialmente quando há restrições de processamento em tempo real. Para tornar a codificação de vídeos HEVC factível nessas situações, é proposto neste trabalho um Sistema de Controle de Complexidade (SCC) que se adapta dinamicamente a capacidades computacionais varáveis. Considera-se que o codificador faz parte de um sistema maior, o qual informa suas restrições como disponibilidade da CPU e processamento alvo para o SCC. Para desenvolver um sistema eficiente, uma extensiva análise de complexidade dos principais parâmetros de codificação é realizada. Nessa análise, foi definida uma métrica livre de particularidades da plataforma de simulação, como hierarquia de memória e acesso concorrente à unidade de processamento. Essa métrica foi chamada de Complexidade Aritmética e pode ser facilmente adaptada para diversas plataformas. Os resultados mostram que o SCC proposto atinge ganhos médios de 40% em complexidade com penalidade mínima em eficiência de compressão e qualidade. As análises de adaptabilidade e controlabilidade mostraram que o SCC rapidamente se adapta a diferentes restrições, por exemplo, quando a disponibilidade de recursos computacionais varia dinamicamente enquanto um vídeo é codificado. Comparado com o estado da arte, o SCC atinge uma redução de 44% no tempo de codificação com penalidade de 2.9% na taxa de compressão e acréscimo de 6% em BD-bitrate. / HEVC encoders impose several challenges in resource-/computationally-constrained embedded applications, especially under real-time throughput constraints. To make HEVC encoding feasible in such scenarios, an adaptive Computation Management Scheme (CMS) that dynamically adapts to varying compute capabilities is proposed in this work. It is assumed that the encoder is part of a larger system, which informs to the CMS its restrictions and requirements, like CPU availability and target frame rate. To effectively develop and apply such a scheme, an extensive computational effort analysis of key encoding parameters of the HEVC is carried out. For this analysis, a platform-orthogonal metric called “Arithmetic Complexity” was developed, which can be widely adopted for various computing platforms. The achieved results illustrate that the proposed CMS provides 40% cycle savings on average at the cost of small RD penalties. The adaptability and controllability analyses show that the CMS quickly adapts to different constrained scenarios, e.g., when the executing HEVC encoder requires more or less computation from the underlying platform. Compared to state of the art, the CMS achieves 44% encoding time savings while incurring a minor 2.9% increase in the bitrate and 6% increase in BD-bitrate.

The demand for efficient video coding techniques has increased in the recent past, resulting in the evolution of various video compression techniques. SVC (Scalable video coding) is the recent amendment of H.264/AVC (Advanced Video Coding), which adds a new dimension by providing the possibility of encoding a video stream into a combination of different sub streams that are scalable in areas corresponding to spatial resolution, temporal resolution and quality. Introduction of the scalability aspect is an effective video coding technique in a network scenario where the client can decode the sub stream depending on the available bandwidth in the network. A graceful degradation in the video quality is expected when any of the spatial, temporal or the quality layer is removed. Still the amount of degradation in video quality has to be measured in terms of Quality of Experience (QoE) from the user’s perspective. To measure the degradation in video quality, video streams consisting of different spatial and temporal layers have been extracted and efforts have been put to remove each layer starting from a higher dependency layer or the Enhancement layer and ending up with the lowest dependency layer or the Base layer. Extraction of a temporally downsampled layer had challenges with frame interpolation and to overcome this, temporal interpolation was employed. Similarly, a spatial downsampled layer has been upsampled in the spatial domain in order to compare with the original stream. Later, an objective video quality assessment has been made by comparing the extracted substream containing fewer layers that are downsampled both spatially and temporally with the original stream containing all layers. The Mean Opinion Scores (MOS) were obtained from objective tool named Perceptual Evaluation of Video Quality (PEVQ). The experiment is carried out for each layers and also for different test videos. Subjective tests were also performed to evaluate the user experience. The results provide recommendations to SVC capable router about the video quality available for each layer and hence the network transcoder can transmit a specific layer depending on the network conditions and capabilities of the decoding device. / Efterfrågan på effektiva video kodningstekniker har ökat under de senaste åren, vilket resulterar i utvecklingen av olika tekniker videokomprimering. SVC (Scalable Video Coding) är den senaste ändringen av H.264/AVC (Advanced Video Coding), vilket ger en ny dimension genom att möjligheten att koda en videoström till en kombination av olika sub strömmar som är skalbara i områden som motsvarar rumslig upplösning, tidsupplösning och kvalitet. Introduktion av skalbarhet aspekten är en effektiv video kodningsteknik i ett nätverk scenario där kunden kan avkoda sub strömmen beroende på den tillgängliga bandbredden i nätverket. En elegant nedbrytning i videokvaliteten förväntas när någon av den rumsliga, tidsmässiga eller kvaliteten skiktet avlägsnas. Fortfarande mängden nedbrytning i videokvalitet måste mätas i termer "Quality of Experience" (QoE) från användarens perspektiv. För att mäta försämring i video-kvalitet, har videoströmmar består av olika rumsliga och tidsmässiga skikt hämtats och ansträngningar har lagts för att ta bort varje lager från ett högre beroende lager eller förbättrande lagret och slutar upp med den lägsta beroendet lagret eller basen skikt. Extraktion av ett tidsmässigt nedsamplas lager hade problem med ram interpolation och för att övervinna detta, var temporal interpolering används. På liknande sätt har en rumslig nedsamplas skikt har uppsamplas i rumsdomänen för att jämföra med den ursprungliga strömmen. Senare har en objektiv videokvalitet bedömning gjorts genom att jämföra den extraherade underströmmen med färre lager som nedsamplade både rumsligt och tidsmässigt med den ursprungliga strömmen innehållande alla lager. De genomsnittliga yttrande poäng (MOS) erhölls från objektivt verktyg som heter Perceptuell utvärdering av Videokvalitet (PEVQ). Experimentet utförs för varje skikt och även för olika test video. Subjektiva tester utfördes också för att utvärdera användarupplevelsen. Resultaten ger rekommendationer till SVC kapabel router om videokvaliteten för varje lager och därmed nätverket kodomvandlaren kan överföra ett visst lager beroende på nätverksförhållanden och kapacitet avkodnings anordningen. / Tilak Varisetty, 518, Gamlainfartsvägen, Annebo, Karlskrona -37141, Mobil: 0723060131

To provide universal multimedia experience, multimedia streaming services need to transparently handle the variation and heterogeneity in operating environment. From the standpoint of streaming application, video adaptation techniques are intended to cope with the environmental variations by utilizing manipulations of the video content itself. Scalable video coding (SVC) schemes, like that suggested by the standards H.264 and its SVC extension, is highly attractive for designing a self-adaptive video streaming system. When SVC is employed in streaming system, the produced video stream can be then easily truncated or tailored to form several sub-streams which can be decoded separately to obtain a range of preferable picture size, quality and frame rate. However, questions about how to perform the adaptation using SVC and how much adaptation SVC enables are still remaining research issues. We still lack a thorough understanding of how to automate the scaling procedure in order to achieve an optimal video Quality-of-Experience for end users. Video QoE, depends highly on human perception. In this thesis, we introduce several video QoE studies around the usability of H.264 SVC. Several factors that contribute significantly to the overall QoEs have been identified and evaluated in these studies. As an example of application usage related factor, playback smoothness and application response time are critical performance measures which can benefit from temporal scalability. Targeting on applications that requires frequent interactivity, we propose a transcoding scheme that fully utilizes the benefits of Switching P and Switching I frames specified in H.264 to enhance video stream's temporal scalability.  Focusing on visual quality related factors, a series of carefully designed subjective quality assessment tests have been performed on mobile devices to investigate the effects of multi-dimensional scalability on human quality perception. Our study reveals that QoE degrades non-monotonically with bitrate and that scaling order preferences are content-dependent. Another study find out that the flickering effect caused by frequent switching between layers in SVC compliant bit-streams is highly related to the switching period. When the period is above a certain threshold, the flickering effect will disappear and layer switching should not be considered as harmful. We have also examined user perceived video quality in 3D virtual worlds. Our results show that the avatars' distance to the virtual screen in 3D worlds contribute significant to the video QoE, i.e., for a wide extent of distortion, there exists always a feasible virtual distance from where the distortion is not detectable for most of people, which makes sense to perform video adaptation. The work presented in this thesis is supposed to help improving the design of self adaptive video streaming services that can deliver video content independently of network technology and end-device capability while seeking the best possible experience for video. / Ardendo småföretagsdoktorand

In video transmission at low bit rates the challenge is to compress the video with a minimal reduction of the percieved quality. The compression can be adapted to knowledge of which regions in the video sequence are of most interest to the viewer. Region of interest (ROI) video coding uses this information to control the allocation of bits to the background and the ROI. The aim is to increase the quality in the ROI at the expense of the quality in the background. In order for this to occur the typical content of an ROI for a particular application is firstly determined and the actual detection is performed based on this information. The allocation of bits can then be controlled based on the result of the detection. In this licenciate thesis existing methods to control bit allocation in ROI video coding are investigated. In particular pre-processing methods that are applied independently of the codec or standard. This makes it possible to apply the method directly to the video sequence without modifications to the codec. Three filters are proposed in this thesis based on previous approaches. The spatial filter that only modifies the background within a single frame and the temporal filter that uses information from the previous frame. These two filters are also combined into a spatio-temporal filter. The abilities of these filters to reduce the number of bits necessary to encode the background and to successfully re-allocate these to the ROI are investigated. In addition the computational compexities of the algorithms are analysed. The theoretical analysis is verified by quantitative tests. These include measuring the quality using both the PSNR of the ROI and the border of the background, as well as subjective tests with human test subjects and an analysis of motion vector statistics. The qualitative analysis shows that the spatio-temporal filter has a better coding efficiency than the other filters and it successfully re-allocates the bits from the foreground to the background. The spatio-temporal filter gives an improvement in average PSNR in the ROI of more than 1.32 dB or a reduction in bitrate of 31 % compared to the encoding of the original sequence. This result is similar to or slightly better than the spatial filter. However, the spatio-temporal filter has a better performance, since its computational complexity is lower than that of the spatial filter.

[pt] Codificação de objetos de forma arbitrária extraídos de imagens regulares é um dos assuntos de maior desafio nos mais recentes padrões de codificação de vídeo (MPEG-4). Neste estudo, é proposta uma nova estratégia de codificação de objetos envolvendo maior eficiência na decomposição e quantização de sub-bandas. A técnica de transformação proposta envolve Transformada Wavelet Discreta (DWT) e a de quantização é baseada no algoritmo de Partição de Conjuntos em Árvores Hierárquicas (SPIHT). / [en] Coding of arbitrarily shaped objects extracted from regular images is one of the most challenging issues on the latest video coding standards (MPEG-4). In this work, a new object coding strategy is proposed, involving greater efficiency on subband splitting and quantization. The proposed transform technique involves Discrete Wavelet Transform (DWT) and the proposed quantization technique is based on the Set Partitioning in Hierarchical Trees (SPIHT) algorithm. / [es] La codificación de objetos de forma arbitraria extraídos de imágenes regulares es uno de los asuntos de mayor desafío en los más recientes padrones de codificación de video (MPEG-4). En este estudio, se propone una nueva estrategia de codificación de objetos con mayor eficiencia en la descomposición y cuantización de subbandas. La técnica de transformación propuesta incluye Transformada Wavelet Discreta (DWT) y la de cuantización tiene como base el algoritmo de Partición de Conjuntos en Árboles Jerárquicas (SPIHT).

With the recent comeback of three-dimensional (3D) movies to the cinemas, there have been increasing efforts to spread the commercial success of 3D to new markets. The possibility of a 3D experience at home, such as three-dimensional television (3DTV), has generated a great deal of interest within the research and standardization community. A central issue for 3DTV is the creation and representation of 3D content. Scene depth information plays a crucial role in all parts of the distribution chain from content capture via transmission to the actual 3D display. This depth information is transmitted in the form of depth maps and is accompanied by corresponding video frames, i.e. for Depth Image Based Rendering (DIBR) view synthesis. Nonetheless, scenarios do exist for which the original spatial resolutions of depth maps and video frames do not match, e.g. sensor driven depth capture or asymmetric 3D video coding. This resolution discrepancy is a problem, since DIBR requires accordance between the video frame and depth map. A considerable amount of research has been conducted into ways to match low-resolution depth maps to high resolution video frames. Many proposed solutions utilize corresponding texture information in the upscaling process, however they mostly fail to review this information for validity. In the strive for better 3DTV quality, this thesis presents the Edge-Weighted Optimization Concept (EWOC), a novel texture-guided depth upscaling application that addresses the lack of information validation. EWOC uses edge information from video frames as guidance in the depth upscaling process and, additionally, confirms this information based on the original low resolution depth. Over the course of four publications, EWOC is applied in 3D content creation and distribution. Various guidance sources, such as different color spaces or texture pre-processing, are investigated. An alternative depth compression scheme, based on depth map upscaling, is proposed and extensions for increased visual quality and computational performance are presented in this thesis. EWOC was evaluated and compared with competing approaches, with the main focus was consistently on the visual quality of rendered 3D views. The results show an increase in both objective and subjective visual quality to state-of-the-art depth map upscaling methods. This quality gain motivates the choice of EWOC in applications affected by low resolution depth. In the end, EWOC can improve 3D content generation and distribution, enhancing the 3D experience to boost the commercial success of 3DTV.

No description available.

Fourth Generation (4G) cellular technology Third Generation Partnership Project (3GPP) Long Term Evolution (LTE) offers high data rate capabilities to mobile users; and, operators are trying to deliver a true mobile broadband experience over LTE networks. Mobile TV and Video on Demand (VoD) are expected to be the main revenue generators in the near future [36] and efficient video streaming over wireless is the key to enabling this. 3GPP recommends the use of H.264 baseline profiles for all video based services in Third Generation (3G) Universal Mobile Telecommunication System (UMTS) networks. However, LTE networks need to support mobile devices with different display resolution requirements like small resolution mobile phones and high resolution laptops. Scalable Video Coding (SVC) is required to achieve this goal. Feasibility study of SVC for LTE is one of the main agenda of 3GPP Release10. SVC enhances H.264 with a set of new profiles and encoding tools that may be used to produce scalable bit streams. Efficient adaptation methods for SVC video transmission over LTE networks are proposed in this thesis. Advantages of SVC over H.264 are analyzed using real time use cases of mobile video streaming. Further, we study the cross layer adaptation and scheduling schemes for delivering SVC video streams most efficiently to the users in LTE networks in unicast and multicast transmissions. We propose SVC based video streaming scheme for unicast and multicast transmissions in the downlink direction, with dynamic adaptations and a scheduling scheme based on channel quality information from users. Simulation results indicate improved video quality for more number of users in the coverage area and efficient spectrum usage with the proposed methods.