<p>3D television using multiple views rendering is receiving increasing interest. In this technology a number of video sequences are transmitted simultaneously and provides a larger view of the scene or stereoscopic viewing experience. With two views stereoscopic rendition is possible. Nowadays 3D displays are available that are capable of displaying several views simultaneously and the user is able to see different views by moving his head.</p><p>The thesis work aims at implementing a demonstration system with a number of simultaneous views. The system will include two cameras, computers at both the transmitting and receiving end and a multi-view display. Besides setting up the hardware, the main task is to implement software so that the transmission can be done over an IP-network.</p><p>This thesis report includes an overview and experiences of similar published systems, the implementation of real time video, its compression, encoding, and transmission over the internet with the help of socket programming and finally the multi-view display in 3D format.  This report also describes the design considerations more precisely regarding the video coding and network protocols.</p>

A drastic increase in traffic of mobile broadband is seen in the past few years, which is further accelerated by the increase in usage of smart phones and its applications. The availability of good smart phones and better data connectivity are encouraging mobile users to use video services. This huge increase in usage will pose a lot of challenges to the wireless networks. The wireless network has to become content aware in order to offer enhanced quality of video service through efficient utilization of the wireless spectrum. This thesis focuses on improving the Quality of Experience (QoE) for video transmission over Long Term Evolution (LTE) networks by imparting the content awareness to the system and providing unequal error protection for critical video packets. Two different schemes for the improvement of video quality delivery over LTE networks are presented in this thesis. Using content awareness, the retransmission count of Hybrid Automatic Repeat reQuest (HARQ) are changed dynamically such that the most important video frame gets more number of retransmission attempts, which increases its success for delivery in-turn increasing the received video quality. Since Radio Link Control (RLC) is the link layer for radio interface, the second approach focuses on optimizing this layer for efficient video transmission. As part of this scheme, a new operation mode called Hybrid Mode (HM) for RLC is defined. This mode performs retransmission only for the critical video frames, leaving other frames to unacknowledged transmission. The simulation results of both proposed schemes provide significant improvement in achieving good video quality without affecting the system performance.

H.264, also known as MPEG-4 Part 10: Advanced Video Coding, is the latest MPEG standard for video coding. It provides approximately 50% bit rate savings for equivalent perceptual quality compared to any previous standard. In the same fashion as previous MPEG standards, only the bitstream syntax and the decoder are specified. Hence, coding performance is not only determined by the standard itself but also by the implementation of the encoder. In this report we propose two methods for improving the coding performance while remaining fully compliant to the standard. After transformation and quantization, the transform coefficients are usually entropy coded and embedded in the bitstream. However, some of them might be beneficial to discard if the number of saved bits are sufficiently large. This is usually referred to as coefficient thresholding and is investigated in the scope of H.264 in this report. Lagrangian optimization for video compression has proven to yield substantial improvements in perceived quality and the H.264 Reference Software has been designed around this concept. When performing Lagrangian optimization, lambda is a crucial parameter that determines the tradeoff between rate and distortion. We propose a new method to select lambda and the quantization parameter for non-reference frames in H.264. The two methods are shown to achieve significant improvements. When combined, they reduce the bitrate around 12%, while preserving the video quality in terms of average PSNR. To aid development of H.264, a software tool has been created to visualize the coding process and present statistics. This tool is capable of displaying information such as bit distribution, motion vectors, predicted pictures and motion compensated block sizes.

ITU - T and the Moving Picture Expert Group (MPEG) have jointly, under the name of Joint Video Team (JVT), developed a new video coding standard. The standard is called H.264 and is also known as Advanced Video Coding (AVC) or MPEG-4 part 10. Comparisons shows that H.264 greatly outperforms MPEG-2, currently used in DVD and digital TV. H.264 halves the bit rate with equal image quality. The great rate - distortion performance means nevertheless a high computational complexity. Especially on the encoder side. Handling of audio and video, e.g. compressing and filtering, is quite complex and requires high performance hardware and software. A video encoder consists of a number of modules that find the best coding parameters. For each macroblock several $modes$ are evaluated in order to achieve optimal coding. The reference implementation of H.264 uses a brute force search for this mode selection which is extremely computational constraining. In order to perform video encoding with satisfactory speed there is an obvious need for reducing the amount of modes that are evaluated. This thesis proposes an algorithm which reduces the number of modes and reference frames that are evaluated. The algorithm can be regulated in order to fulfill the demand on quality versus speed. Six times faster encoding can be obtained without loosing perceptual image quality. By allowing some quality degradation the encoding becomes up to 20 times faster.

Le champ du traitement des images et des vidéos attire l'attention depuis les deux dernières décennies. Ce champ couvre maintenant un spectre énorme d'applications comme la TV 3D, la télé-surveillance, la vision par ordinateur, l'imagerie médicale, la compression, la transmission, etc. En ce début de vingt et unième siècle nous sommes témoins d'une révolution importante. Les largeurs de bande des réseaux, les capacités de mémoire et les capacités de calcul ont été fortement augmentés durant cette période. Un client peut avoir un débit de plus de 100~mbps tandis qu'un autre peut utiliser une ligne à 56~kbps. Simultanément, un client peut avoir un poste de travail puissant, tandis que d'autres peuvent avoir juste un téléphone mobile. Au milieu de ces extrêmes, il y a des milliers de clients avec des capacités et des besoins très variables. De plus, les préférences d'un client doivent s'adapter à sa capacité, par exemple un client handicapé par sa largeur de bande peut être plus intéressé par une visualisation en temps réel sans interruption que d'avoir une haute résolution. Pour y faire face, des architectures hiérarchiques de codeurs vidéo ont été intoduites afin de comprimer une seule fois, et de décomprimer de différentes manières. Comme la DCT n'a pas la fonctionnalité de multi-résolution, une architecture vidéo hiérarchique est conçue pour faire face aux défis des largeurs de bande et des puissances de traitement hétérogènes. Avec l'inondation des contenus numériques, qui peuvent être facilement copiés et modifiés, le besoin de la protection des contenus vidéo a pris plus d'importance. La protection de vidéos peut être réalisée avec l'aide de trois technologies : le tatouage de méta-données et l'insertion de droits d'auteur, le cryptage pour limiter l'accès aux personnes autorisées et la prise des empreintes digitales active pour le traçage de traître. L'idée principale dans notre travail est de développer des technologies de protection transparentes à l'utilisateur. Cela doit aboutir ainsi à un codeur vidéo modifié qui sera capable de coder et d'avoir un flux de données protégé. Puisque le contenu multimédia hiérarchique a déjà commencé à voir le jour, algorithmes pour la protection indépendante de couches d'amélioration sont également proposées.

Recentemente, o Brasil adotou o padrão SBTVD (Sistema Brasileiro de TV Digital) para transmissão de TV digital. Este utiliza o CODEC (codificador e decodificador) de vídeo H.264/AVC, que é considerado o estado-da-arte no contexto de compressão de vídeo digital. Esta transição para o SBTVD requer o desenvolvimento de tecnologia para transmissão, recepção e decodificação de sinais, assim, o projeto Rede H.264 SBTVD foi iniciado e tem como um dos objetivos a produção de componentes de hardware para construção de um set-top box SoC (System on Chip) compatível com o SBTVD. No sentido de produzir IPs (Intellectual Property) para codificação e decodificação de vídeo digital segundo o padrão H.264/AVC, várias arquiteturas de hardware vêm sendo desenvolvidas no âmbito do projeto. Assim, o objetivo deste trabalho consiste na realização da implementação física em ASIC (Application-Specific Integrated Circuit) de algumas destas arquiteturas de hardware para decodificação de vídeo H.264/AVC, entre elas as arquiteturas parser e decodificação de entropia, predição intra-quadro e, por fim, quantização e transformadas inversas, que juntas formam uma versão funcional de um decodificador de vídeo H.264 chamado de decodificador intra-only. Além destas, também foi fisicamente implementada uma arquitetura para o módulo filtro redutor de efeito de bloco e arquiteturas para os perfis Main e High de um compensador de movimentos. Nesta dissertação de mestrado, é apresentada a metodologia de implementação standard-cells (ASIC) utilizada, assim como uma descrição detalhada de cada passo executado para se chegar ao leiaute de cada uma das arquiteturas. Também são apresentados os resultados das implementações e realizadas algumas comparações com outras implementações de arquiteturas descritas na literatura. A implementação do filtro possui 43,9K portas lógicas (equivalent-gates), 42mW de potência e possui a menor quantidade de memória interna, 12,375KB SRAM, quando comparada com outras implementações para a mesma resolução de vídeo, 1920x1080@30fps. As implementações para os perfis Main e High do compensador de movimento apresentam a melhor relação entre a quantidade de ciclos de relógio necessária para interpolar um macrobloco (MB), 304 ciclos/MB, e a quantidade de equivalent-gates de cada implementação, 98K e 102K, respectivamente. Já a implementação do decodificador H.264 intra-only possui 5KB SRAM, 11,4mW de potência e apresenta a menor quantidade de equivalent-gates, 150K, comparado com outras implementações de decodificadores H.264 com características similares. / Recently Brazil has adopted the SBTVD (Brazilian Digital Television System) for digital TV transmission. It uses the H.264/AVC video CODEC (coder and decoder), which is considered the state of the art in the context of digital video compression. This transition to the SBTVD standard requires the development of technology for transmitting, receiving and decoding signals, so a project called Rede H.264 was initiated with the objective of producing cutting edge hardware components to build a set-top box SoC (System on Chip) compatible with the SBTVD. In order to produce IPs (Intellectual Property) for encoding and decoding digital video according to the H.264/AVC standard, many hardware architectures have been developed under the project. Therefore, the objective of this work is to carry out the physical implementation flow for ASIC (Application-Specific Integrated Circuit) in some of these hardware architectures for H.264/AVC video decoding, including the architectures parser and entropy decoding, intra-prediction and inverse quantization and transforms, which together compound a working version of an H.264 video decoder called intra-only. Besides these architectures, it is also physically implemented an architecture for a deblocking filter module and architectures for motion compensation according the Main and High profiles. This master thesis presents the standard-cells (ASIC) implementation as well as a detailed description of each step necessary to outcome the layouts of each of the architecture. It also presents the results of the implementations and comparisons with other works in the literature. The implementation of the filter has 43.9K gates (equivalent-gates), 42mW of power consumption and it demands the least amount of internal memory, 12.375KB SRAM, when compared with other implementations for the same video resolution, 1920x1080@30fps. The implementations for the Main and High profiles of the motion compensator have the best relationship between the amount of required clock cycles to interpolate a macroblock (MB), 304 cycles/MB, and the equivalent-gate count of each implementation, 98K and 102K, respectively. Also, the implementation of the H.264 intra-only decoder has 5KB SRAM, 11.4 mW of power consumption and it has the least equivalent-gate count, 150K, compared with other implementations of H.264 decoders which have similar features.

O codificador aritmético binário adaptativo ao contexto adotado (CABAC – Context-based Adaptive Binary Arithmetic Coding) pelo padrão H.264/AVC a partir de perfil Main é o estado-da-arte em termos de eficiência de taxa de bits. Entretanto, o CABAC ocupa 9.6% do tempo total de processamento e seu throughput é limitado pelas dependências de dados no nível de bit (LIN, 2010). Logo, atingir os requisitos de desempenho em tempo real nos níveis mais altos do padrão H.264/AVC se torna uma tarefa árdua em software, sendo necesário então, a aceleração do CABAC através de implementações em hardware. As arquiteturas de hardware encontradas na literatura para o CABAC focam no Codificador Aritmético Binário (BAE - Binary Arithmetic Encoder) enquanto que a Binarização e Modelagem de Contextos (BCM – Binarization and Context Modeling) fica em segundo plano ou nem é apresentada. O BCM e o BAE juntos constituem o CABAC. Esta dissertação descreve detalhadamente o conjunto de algoritmos que compõem o BCM do padrão H.264/AVC. Em seguida, o projeto de uma arquitetura de hardware específica para o BCM é apresentada. A solução proposta é descrita em VHDL e os resultados de síntese mostram que a arquitetura alcança desempenho suficiente, em FPGA e ASIC, para processar vídeos no nível 5 do padrão H.264/AVC. A arquitetura proposta é 13,3% mais rápida e igualmente eficiente em área que os melhores trabalhos relacionados nestes quesitos. / Context-based Adaptive Binary Arithmetic Coding (CABAC) adopted in the H.264/AVC main profile is the state-of-art in terms of bit-rate efficiency. However, CABAC takes 9.6% of the total encoding time and its throughput is limited by bit-level data dependency (LIN, 2010). Moreover, meeting real-time requirement for a pure software CABAC encoder is difficult at the highest levels of the H.264/AVC standard. Hence, speeding up the CABAC by hardware implementation is required. The CABAC hardware architectures found in the literature focus on the Binary Arithmetic Encoder (BAE), while the Binarization and Context Modeling (BCM) is a secondary issue or even absent in the literature. Integrated, the BCM and the BAE constitute the CABAC. This dissertation presents the set of algorithms that describe the BCM of the H.264/AVC standard. Then, a novel hardware architecture design for the BCM is presented. The proposed design is described in VHDL and the synthesis results show that the proposed architecture reaches sufficiently high performance in FPGA and ASIC to process videos in real-time at the level 5 of H.264/AVC standard. The proposed design is 13.3% faster than the best works in these items, while being equally efficient in area.

Recentemente, o Brasil adotou o padrão SBTVD (Sistema Brasileiro de TV Digital) para transmissão de TV digital. Este utiliza o CODEC (codificador e decodificador) de vídeo H.264/AVC, que é considerado o estado-da-arte no contexto de compressão de vídeo digital. Esta transição para o SBTVD requer o desenvolvimento de tecnologia para transmissão, recepção e decodificação de sinais, assim, o projeto Rede H.264 SBTVD foi iniciado e tem como um dos objetivos a produção de componentes de hardware para construção de um set-top box SoC (System on Chip) compatível com o SBTVD. No sentido de produzir IPs (Intellectual Property) para codificação e decodificação de vídeo digital segundo o padrão H.264/AVC, várias arquiteturas de hardware vêm sendo desenvolvidas no âmbito do projeto. Assim, o objetivo deste trabalho consiste na realização da implementação física em ASIC (Application-Specific Integrated Circuit) de algumas destas arquiteturas de hardware para decodificação de vídeo H.264/AVC, entre elas as arquiteturas parser e decodificação de entropia, predição intra-quadro e, por fim, quantização e transformadas inversas, que juntas formam uma versão funcional de um decodificador de vídeo H.264 chamado de decodificador intra-only. Além destas, também foi fisicamente implementada uma arquitetura para o módulo filtro redutor de efeito de bloco e arquiteturas para os perfis Main e High de um compensador de movimentos. Nesta dissertação de mestrado, é apresentada a metodologia de implementação standard-cells (ASIC) utilizada, assim como uma descrição detalhada de cada passo executado para se chegar ao leiaute de cada uma das arquiteturas. Também são apresentados os resultados das implementações e realizadas algumas comparações com outras implementações de arquiteturas descritas na literatura. A implementação do filtro possui 43,9K portas lógicas (equivalent-gates), 42mW de potência e possui a menor quantidade de memória interna, 12,375KB SRAM, quando comparada com outras implementações para a mesma resolução de vídeo, 1920x1080@30fps. As implementações para os perfis Main e High do compensador de movimento apresentam a melhor relação entre a quantidade de ciclos de relógio necessária para interpolar um macrobloco (MB), 304 ciclos/MB, e a quantidade de equivalent-gates de cada implementação, 98K e 102K, respectivamente. Já a implementação do decodificador H.264 intra-only possui 5KB SRAM, 11,4mW de potência e apresenta a menor quantidade de equivalent-gates, 150K, comparado com outras implementações de decodificadores H.264 com características similares. / Recently Brazil has adopted the SBTVD (Brazilian Digital Television System) for digital TV transmission. It uses the H.264/AVC video CODEC (coder and decoder), which is considered the state of the art in the context of digital video compression. This transition to the SBTVD standard requires the development of technology for transmitting, receiving and decoding signals, so a project called Rede H.264 was initiated with the objective of producing cutting edge hardware components to build a set-top box SoC (System on Chip) compatible with the SBTVD. In order to produce IPs (Intellectual Property) for encoding and decoding digital video according to the H.264/AVC standard, many hardware architectures have been developed under the project. Therefore, the objective of this work is to carry out the physical implementation flow for ASIC (Application-Specific Integrated Circuit) in some of these hardware architectures for H.264/AVC video decoding, including the architectures parser and entropy decoding, intra-prediction and inverse quantization and transforms, which together compound a working version of an H.264 video decoder called intra-only. Besides these architectures, it is also physically implemented an architecture for a deblocking filter module and architectures for motion compensation according the Main and High profiles. This master thesis presents the standard-cells (ASIC) implementation as well as a detailed description of each step necessary to outcome the layouts of each of the architecture. It also presents the results of the implementations and comparisons with other works in the literature. The implementation of the filter has 43.9K gates (equivalent-gates), 42mW of power consumption and it demands the least amount of internal memory, 12.375KB SRAM, when compared with other implementations for the same video resolution, 1920x1080@30fps. The implementations for the Main and High profiles of the motion compensator have the best relationship between the amount of required clock cycles to interpolate a macroblock (MB), 304 cycles/MB, and the equivalent-gate count of each implementation, 98K and 102K, respectively. Also, the implementation of the H.264 intra-only decoder has 5KB SRAM, 11.4 mW of power consumption and it has the least equivalent-gate count, 150K, compared with other implementations of H.264 decoders which have similar features.

O codificador aritmético binário adaptativo ao contexto adotado (CABAC – Context-based Adaptive Binary Arithmetic Coding) pelo padrão H.264/AVC a partir de perfil Main é o estado-da-arte em termos de eficiência de taxa de bits. Entretanto, o CABAC ocupa 9.6% do tempo total de processamento e seu throughput é limitado pelas dependências de dados no nível de bit (LIN, 2010). Logo, atingir os requisitos de desempenho em tempo real nos níveis mais altos do padrão H.264/AVC se torna uma tarefa árdua em software, sendo necesário então, a aceleração do CABAC através de implementações em hardware. As arquiteturas de hardware encontradas na literatura para o CABAC focam no Codificador Aritmético Binário (BAE - Binary Arithmetic Encoder) enquanto que a Binarização e Modelagem de Contextos (BCM – Binarization and Context Modeling) fica em segundo plano ou nem é apresentada. O BCM e o BAE juntos constituem o CABAC. Esta dissertação descreve detalhadamente o conjunto de algoritmos que compõem o BCM do padrão H.264/AVC. Em seguida, o projeto de uma arquitetura de hardware específica para o BCM é apresentada. A solução proposta é descrita em VHDL e os resultados de síntese mostram que a arquitetura alcança desempenho suficiente, em FPGA e ASIC, para processar vídeos no nível 5 do padrão H.264/AVC. A arquitetura proposta é 13,3% mais rápida e igualmente eficiente em área que os melhores trabalhos relacionados nestes quesitos. / Context-based Adaptive Binary Arithmetic Coding (CABAC) adopted in the H.264/AVC main profile is the state-of-art in terms of bit-rate efficiency. However, CABAC takes 9.6% of the total encoding time and its throughput is limited by bit-level data dependency (LIN, 2010). Moreover, meeting real-time requirement for a pure software CABAC encoder is difficult at the highest levels of the H.264/AVC standard. Hence, speeding up the CABAC by hardware implementation is required. The CABAC hardware architectures found in the literature focus on the Binary Arithmetic Encoder (BAE), while the Binarization and Context Modeling (BCM) is a secondary issue or even absent in the literature. Integrated, the BCM and the BAE constitute the CABAC. This dissertation presents the set of algorithms that describe the BCM of the H.264/AVC standard. Then, a novel hardware architecture design for the BCM is presented. The proposed design is described in VHDL and the synthesis results show that the proposed architecture reaches sufficiently high performance in FPGA and ASIC to process videos in real-time at the level 5 of H.264/AVC standard. The proposed design is 13.3% faster than the best works in these items, while being equally efficient in area.

To achieve greater compression ratios new video and image CODECs like H.264 and JPEG 2000 take advantage of Context adaptive binary arithmetic coding. As it contains computationally heavy algorithms, fast implementations have to be made when they are performed on large amount of data such as compressing high resolution formats like HDTV. This document describes how entropy coding works in general with a focus on arithmetic coding and CABAC. Furthermore the document dicusses the demands of the different CABACs and propose different options to hardware and instruction level optimisation. Testing and benchmarking of these implementations are done to ease evaluation. The main contribution of the thesis is parallelising and unifying the CABACs which is discussed and partly implemented. The result of the ILA is improved program flow through a specialised branching operations. The result of the DHA is a two bit parallel accelerator with hardware sharing between JPEG 2000 and H.264 encoder with limited decoding support.