For the next generation of multimedia applications such as digital video broadcasting, multimedia message service and video conference, enormous amounts of video context will be transmitted and exchanged through the wireless channel. Due to the limited communication bandwidth, how to achieve more efficient, reliable, and robust video compression is a very important issue. H.264/AVC (Advanced Video Coding) is one of the latest video coding standards, which is anticipated to be adopted in many future application systems due to its excellent compression efficiency. In this thesis, the implementation issue of the H.264 decoding algorithm on the SOC (System-On-Chip) development platform is addressed. Several key modules of H.264 decoders including color space converter, inter-interpolation, transformation rescale modules are all realized by dedicated hardware architectures. A novel low-cost fast scalable deblocking filter based on single-port memory architecture is also proposed which can support fast real-time deblocking filtering process. The entire H.264 decoder system is prototyped on the Altera SOPC platform, and the decoding result is displayed directly on the monitor. All the hardware modules are hooked on the system Avalon bus, and interact with Altera NIOS-¢º processor. Through the hardware/software co-design approach, the decoding speed can be increase by a factor of 1.9.

While there has always been a tremendous demand for streaming video over TCP/IP networks, the nature of the application still presents some challenging issues. These applications that transmit multimedia data over best-effort networks like the Internet must cope with the changing network behavior; specifically, the source encoder rate should be controlled based on feedback from a channel estimator that probes the network periodically. First, one such Multimedia Streaming TCP-Friendly Protocol (MSTFP) is considered, which iteratively integrates forward estimation of network status with feedback control to closely track the varying network characteristics. Second, a network-adaptive embedded bit stream is generated using a r-domain rate controller. The conceptual elegance of this r-domain framework stems from the fact that the coding bit rate ) (R is approximately linear in the percentage of zeros among the quantized spatial transform coefficients ) ( r , as opposed to the more traditional, complex and highly nonlinear ) ( Q R characterization. Though the r-model has been successfully implemented on a few other video codecs, its application to the emerging video coding standard H.264 is considered. The extensive experimental results show thatrobust rate control, similar or improved Peak Signal to Noise Ratio (PSNR), and a faster implementation.

H.264/AVC intra prediction method is an efficient tool to reduce spatial redundancies by using multidirectional spatial prediction modes. In this paper, a novel intra prediction method is designed to improve coding efficiency. Firstly, we propose a direction-distance oriented prediction which considers the distance between the predict value and the reference samples according to the direction of the prediction modes. Secondly, we apply the concept of image restoration by using the projections onto convex sets (POCS) to intra prediction which uses adaptively filtering based on the surrounding reconstructed pixel to predict blocks. The experimental results show that the average bit-rate reduction of 0.75% and PSNR gain improved of 0.119dB are achieved.

Video coding standards such as H.264 AVC (Advanced Video Coding) rely on predictive coding to achieve high compression efficiency. Predictive coding consists of predicting each frame using preceding frames. However, predictive coding incurs a cost when transmitting over unreliable networks: frames are no longer independent and the loss of data in one frame may affect future frames. In this thesis, we study the effectiveness of Flexible Macroblock Ordering (FMO) in mitigating the effect of errors on the decoded video and propose solutions to improve the error concealment on H.264 decoders. After introducing the subject matter, we present the H.264 profiles and briefly determine their intended applications. Then we describe FMO and justify its usefulness for transmission over lossy networks. More precisely, we study the cost in terms of overheads and the improvements it offers in visual quality for damaged video frames. The unavailability of FMO in most H.264 profiles leads us to design a lossless FMO removal scheme, which allows the playback of FMO-encoded video on non FMO-compliant decoders. Then, we describe the process of removing the FMO structure but also underline some limitations that prevent the application of the scheme. Finally, we assess the induced overheads and propose a model to predict these overheads when FMO Type 1 is employed. Eventually, we develop a new error concealment method to enhance video quality without relying on channel feedback. This method is shown to be superior to existing methods, including those from the JM reference software and can be applied to compensate for the limitations of the scheme proposed FMO-removal scheme. After introducing our new method, we evaluate its performance and compare it to some classical algorithms.

In this thesis, we study the concept of scalable video coding as implemented in the extension to the H.264 video coding standard. Specifically, for the spatial and quality scalability scenarios, we propose an optimization algorithm based on the Soft Decision Quantization (SDQ) concept, which aims at jointly optimizing all layers being encoded. The performance of the algorithm was assessed in an SVC implementation. Experimental results show, that the proposed method significantly improves the coding efficiency when compared to an unmodified SVC encoder.

With the rapid growth of video services on smartphones such as video conferencing, video telephone and WebTV, implementation of video compression on mobile terminal becomes extremely important. However, the low computation capability of mobile devices becomes a bottleneck which calls for low complexity techniques for video coding. This work presents two set of algorithms for reducing the complexity of motion estimation. Binary motion estimation techniques using one-bit and two-bit transforms reduce the computational complexity of matching error criterion, however sometimes generate inaccurate motion vectors. The first set includes two neighborhood matching based algorithms which attempt to reduce computations to only a fraction of other methods. Simulation results demonstrate that full search local binary pattern (FS-LBP) algorithm reconstruct visually more accurate frames compared to full search algorithm (FSA). Its reduced complexity LBP (RC-LBP) version decreases computations significantly to only a fraction of the other methods while maintaining acceptable performance. The second set introduces edge detection approach for partial distortion elimination based on binary patterns. Spiral partial distortion elimination (SpiralPDE) has been proposed in literature which matches the pixel-to-pixel distortion in a predefined manner. Since, the contribution of all the pixels to the distortion function is different, therefore, it is important to analyze and extract these cardinal pixels. The proposed algorithms are called lossless fast full search partial distortion elimination ME based on local binary patterns (PLBP) and lossy edge-detection pixel decimation technique based on local binary patterns (ELBP). PLBP reduces the matching complexity by matching more contributable pixels early by identifying the most diverse pixels in a local neighborhood. ELBP captures the most representative pixels in a block in order of contribution to the distortion function by evaluating whether the individual pixels belong to the edge or background. Experimental results demonstrate substantial reduction in computational complexity of ELBP with only a marginal loss in prediction quality.

With the rapid growth of video services on smartphones such as video conferencing, video telephone and WebTV, implementation of video compression on mobile terminal becomes extremely important. However, the low computation capability of mobile devices becomes a bottleneck which calls for low complexity techniques for video coding. This work presents two set of algorithms for reducing the complexity of motion estimation. Binary motion estimation techniques using one-bit and two-bit transforms reduce the computational complexity of matching error criterion, however sometimes generate inaccurate motion vectors. The first set includes two neighborhood matching based algorithms which attempt to reduce computations to only a fraction of other methods. Simulation results demonstrate that full search local binary pattern (FS-LBP) algorithm reconstruct visually more accurate frames compared to full search algorithm (FSA). Its reduced complexity LBP (RC-LBP) version decreases computations significantly to only a fraction of the other methods while maintaining acceptable performance. The second set introduces edge detection approach for partial distortion elimination based on binary patterns. Spiral partial distortion elimination (SpiralPDE) has been proposed in literature which matches the pixel-to-pixel distortion in a predefined manner. Since, the contribution of all the pixels to the distortion function is different, therefore, it is important to analyze and extract these cardinal pixels. The proposed algorithms are called lossless fast full search partial distortion elimination ME based on local binary patterns (PLBP) and lossy edge-detection pixel decimation technique based on local binary patterns (ELBP). PLBP reduces the matching complexity by matching more contributable pixels early by identifying the most diverse pixels in a local neighborhood. ELBP captures the most representative pixels in a block in order of contribution to the distortion function by evaluating whether the individual pixels belong to the edge or background. Experimental results demonstrate substantial reduction in computational complexity of ELBP with only a marginal loss in prediction quality.

This thesis focuses on video watermarking robust against geometric attacks and video compressions. In addition to the requirements for an image watermarking algorithm, a digital video watermarking algorithm has to be robust against advanced video compressions, frame loss, frame swapping, aspect ratio change, frame rate change, intra- and inter-frame filtering, etc. Video compression, especially, the most efficient compression standard, H.264, and geometric attacks, such as rotation and cropping, frame aspect ratio change, and translation, are considered the most challenging attacks for video watermarking algorithms. In this thesis, we first review typical watermarking algorithms robust against geometric attacks and video compressions, and point out their advantages and disadvantages. Then, we propose our robust video watermarking algorithms against Rotation, Scaling and Translation (RST) attacks and MPEG-2 compression based on the logpolar mapping and the phase-only filtering method. Rotation or scaling transformation in the spatial domain results in vertical or horizontal shift in the log-polar mapping (LPM) of the magnitude of the Fourier spectrum of the target frame. Translation has no effect in this domain. This method is very robust to RST attacks and MPEG-2 compression. We also demonstrate that this method can be used as a RST parameters detector to work with other watermarking algorithms to improve their robustness to RST attacks. Furthermore, we propose a new video watermarking algorithm based on the 1D DFT (one-dimensional Discrete Fourier Transform) and 1D projection. This algorithm enhances the robustness to video compression and is able to resist the most advanced video compression, H.264. The 1D DFT for a video sequence along the temporal domain generates an ideal domain, in which the spatial information is still kept and the temporal information is obtained. With detailed analysis and calculation, we choose the frames with highest temporal frequencies to embed the fence-shaped watermark pattern in the Radon transform domain of the selected frames. The performance of the proposed algorithm is evaluated by video compression standards MPEG-2 and H.264; geometric attacks such as rotation, translation, and aspect-ratio changes; and other video processing. The most important advantages of this video watermarking algorithm are its simplicity, practicality and robustness.

Multiterminal (MT) video coding is a practical application of the MT source coding theory. For MT source coding theory, two problems associated with achievable rate regions are well investigated into in this thesis: a new sufficient condition for BT sum-rate tightness, and the sum-rate loss for quadratic Gaussian MT source coding. Practical code design for ideal Gaussian sources with quadratic distortion measure is also achieved for cases more than two sources with minor rate loss compared to theoretical limits. However, when the theory is applied to practical applications, the performance of MT video coding has been unsatisfactory due to the difficulty to explore the correlation between different camera views. In this dissertation, we present an MT video coding scheme under the H.264/AVC framework. In this scheme, depth camera information can be optionally sent to the decoder separately as another source sequence. With the help of depth information at the decoder end, inter-view correlation can be largely improved and thus so is the compression performance. With the depth information, joint estimation from decoded frames and side information at the decoder also becomes available to improve the quality of reconstructed video frames. Experimental result shows that compared to separate encoding, up to 9.53% of the bit rate can be saved by the proposed MT scheme using decoder depth information, while up to 5.65% can be saved by the scheme without depth camera information. Comparisons to joint video coding schemes are also provided.

Novas arquiteturas de hardware desenvolvidas para blocos chave do padrão de codificação de vídeo ISO/IEC 14496-10 são discutidas, propostas, implementadas e validades nesta tese. Também chamado de H.264, AVC (Advanced Video Coder) ou MPEG-4 parte 10, o padrão é o estado da arte em codificação de vídeo, apresentando as mais altas taxas de compressão possíveis por um compressor de vídeo padronizado por organismos internacionais (ISO/IEC e ITU-T). O H.264 já passou por três revisões importantes: na primeira foram incluídos novos perfis, voltados para a extensão da fidelidade e aplicações profissionais, na segunda veio o suporte a escalabilidade (SVC – Scalable Video Coder). Uma terceira revisão suporta fontes de vídeo com múltiplas vistas (MVC – Multi-view Video Coder). Nesta tese são apresentadas arquiteturas para dois módulos do codificador H.264: o CABAC e o Filtro de Deblocagem (Deblocking Filter). O CABAC (Context-Adaptive Binary Arithmetic Coder) possui desafios importantes devido às dependências de dados de natureza bit-a-bit. Uma revisão das alternativas arquiteturais e uma solução específica para a codificação CABAC é apresentada nesta tese. O filtro de deblocagem também apresenta diversos desafios importantes para seu desenvolvimento e foi alvo de uma proposta arquitetural apresentada neste trabalho. Finalmente a arquitetura de uma plataforma de validação genérica para validar módulos desenvolvidos para o codificador e decodificador H.264 também é apresentada. Os módulos escolhidos estão de acordo com os demais trabalhos realizados pelo grupo de pesquisa da UFRGS, que têm por objetivo desenvolver um decodificador e um codificador completos capazes de processar vídeo digital de alta definição no formato 1080p em tempo real. / New hardware architectures developed for key blocks of the ISO/IEC 14496-10 video coding standard are discussed, proposed, implemented, and validated in this thesis. The standard is also called H.264, AVC (Advanced Video Coder) or MPEG-4 part 10, and is the state-of-the-art in video coding, presenting the highest compression ratios achievable by an internationally standardized video coder (ISO/IEC and ITU-T). The H.264 has already been revised three times: the first included new profiles for fidelity extension and professional applications. The second brought the scalability support (SVC – Scalable Video Coder). The third revision supports video sources with multiple views (MVC – Multi-view Video Coder). The present work developed high performance architectures for CABAC (Context-Adaptive Binary Arithmetic Coder), which were challenging because of the bitwise data dependencies. A through revision of the alternative architectures and a specific architectural solution for CABAC encoding are presented in this thesis. A dedicated hardware architecture for a HIGH profile Deblocking Filter is also presented, developed, validated and synthesized for two different targets: FPGA and ASIC. The validation methodology is presented and applied to three different modules of the H.264 encoder. The H.264 blocks dealt with in this thesis work complement those developed by other works in the UFRGS research group and contribute to the development of complete encoders for real-time processing of high definition digital video at 1080p.