In July 2014, the second version of the HEVC/H.265 video coding standard was announced, and it included the Scalable High efficiency Video Coding (SHVC) extension. SHVC is used for coding a video stream with subset streams of the same video with lower quality, and it supports spatial, temporal and SNR scalability among others. This is used to enable easy adaption of a video stream, by dropping or adding packages, to devices with different screen sizes, computing power and bandwidth. In this project SHVC has been implemented in Ericsson's research encoder C65. Some cross-layer optimizations have also been implemented and evaluated. The main goal of these optimizations are to make better decisions when choosing the reference layer's motion parameters and QP, by doing multi-pass coding and using the coded enhancement layer information from the first pass.

La compression vidéo HEVC standardisée en 2013 offre des gains de compression dépassant les 50% par rapport au standard de compression précédent MPEG4-AVC/H.264. Ces gains de compression se paient par une augmentation très importante de la complexité de codage. Si on ajoute à cela l’augmentation de complexité générée par l’accroissement de résolution et de fréquence image du signal vidéo d’entrée pour passer de la Haute Définition (HD) à l’Ultra Haute Définition (UHD), on comprend vite l’intérêt des techniques de réduction de complexité pour le développement de codeurs économiquement viables. En premier lieu, un effort particulier a été réalisé pour réduire la complexité des images Intra. Nous proposons une méthode d'inférence des modes de codage à partir d'un pré-codage d'une version réduite en HD de la vidéo UHD. Ensuite, nous proposons une méthode de partitionnement rapide basée sur la pré-analyse du contenu. La première méthode offre une réduction de complexité d'un facteur 3 et la deuxième, d'un facteur 6, contre une perte de compression proche de 5%. En second lieu, nous avons traité le codage des images Inter. En mettant en œuvre une solution d'inférence des modes de codage UHD à partir d'un pré-codage au format HD, la complexité de codage est réduite d’un facteur 3 en considérant les 2 flux produits et d’un facteur 9.2 sur le seul flux UHD, pour une perte en compression proche de 3%. Appliqué à une configuration de codage proche d'un système réellement déployé, l'apport de notre algorithme reste intéressant puisqu'il réduit la complexité de codage du flux UHD d’un facteur proche de 2 pour une perte de compression limitée à 4%. Les stratégies de réduction de complexité mises en œuvre au cours de cette thèse pour le codage Intra et Inter offrent des perspectives intéressantes pour le développement de codeurs HEVC UHD plus économes en ressources de calculs. Elles sont particulièrement adaptées au domaine de la WebTV/OTT qui prend une part croissante dans la diffusion de la vidéo et pour lequel le signal vidéo est codé à des résolutions multiples pour adresser des réseaux et des terminaux de capacités variées. / The High Efficiency Video Coding (HEVC) standard was released in 2013 which reduced network bandwidth by a factor of 2 compared to the prior standard H.264/AVC. These gains are achieved by a very significant increase in the encoding complexity. Especially with the industrial demand to shift in format from High Definition (HD) to Ultra High Definition (UHD), one can understand the relevance of complexity reduction techniques to develop cost-effective encoders. In our first contribution, we attempted new strategies to reduce the encoding complexity of Intra-pictures. We proposed a method with inference rules on the coding modes from the modes obtained with pre-encoding of the UHD video down-sampled in HD. We, then, proposed a fast partitioning method based on a pre-analysis of the content. The first method reduced the complexity by a factor of 3x and the second one, by a factor of 6, with a loss of compression efficiency of 5%. As a second contribution, we adressed the Inter-pictures. By implementing inference rules in the UHD encoder, from a HD pre-encoding pass, the encoding complexity is reduced by a factor of 3x when both HD and UHD encodings are considered, and by 9.2x on just the UHD encoding, with a loss of compression efficiency of 3%. Combined with an encoding configuration imitating a real system, our approach reduces the complexity by a factor of close to 2x with 4% of loss. These strategies built during this thesis offer encouraging prospects for implementation of low complexity HEVC UHD encoders. They are fully adapted to the WebTV/OTT segment that is playing a growing part in the video delivery, in which the video signal is encoded with different resolution to reach heterogeneous devices and network capacities.

This thesis looks at hardware algorithms that help reduce dynamic power dissipation in video encoder applications. Computational complexity of motion estimation and the data traffic between external memory and the video processing engine are two main reasons for large power dissipation in video encoders. While motion estimation may consume 50% to 70% of total video encoder power, the power dissipated in external memory such as the DDR SDRAM can be of the order of 40% of the total system power. Reducing power dissipation in video encoders is important in order to improve battery life of mobile devices such as the smart phones and digital camcorders. We propose hardware algorithms which extract only the important features in the video data to reduce the complexity of computations, communications and storage, thereby reducing average power dissipation. We apply this concept to design hardware algorithms for optimizing motion estimation matching complexity, and reference frame storage and access from the external memory. In addition, we also develop techniques to reduce searching complexity of motion estimation. First, we explore a set of adaptive algorithms that reduce average power dissipated due to motion estimation. We propose that by taking into account the macro-block level features in the video data, the average matching complexity of motion estimation in terms of number of computations in real-time hardwired video encoders can be significantly reduced when compared against traditional hardwired implementations, that are designed to handle most demanding data sets. Current macro-block features such as pixel variance and Hadamard transform coefficients are analyzed, and are used to adapt the matching complexity. The macro-block is partitioned based on these features to obtain sub-block sums, which are used for matching operations. Thus, simple macro-blocks, without many features can be matched with much less computations compared to the macro-blocks with complex features, leading to reduction in average power dissipation. Apart from optimizing the matching operation, optimizing the search operation is a powerful way to reduce motion estimation complexity. We propose novel search optimization techniques including (1) a center-biased search order and (2) skipping unlikely search positions, both applied in the context of real time hardware implementation. The proposed search optimization techniques take into account and are compatible with the reference data access pattern from the memory as required by the hardware algorithm. We demonstrate that the matching and searching optimization techniques together achieve nearly 65% reduction in power dissipation due to motion estimation, without any significant degradation in motion estimation quality. A key to low power dissipation in video encoders is minimizing the data traffic between the external memory devices such as DDR SDRAM and the video processor. External memory power can be as high as 50% of the total power budget in a multimedia system. Other than the power dissipation in external memory, the amount of data traffic is an important parameter that has significant impact on the system cost. Large memory traffic necessitates high speed external memories, high speed on-chip interconnect, and more parallel I/Os to increase the memory throughput. This leads to higher system cost. We explore a lossy, scalar quantization based reference frame compression technique that can be used to reduce the amount of reference data traffic from external memory devices significantly. In this scheme, the quantization is adapted based on the pixel range within each block being compressed. We show that the error introduced by the scalar quantization is bounded and can be represented by smaller number of bits compared to the original pixel. The proposed reference frame compression scheme uses this property to minimize the motion compensation related traffic, thereby improving the compression scheme efficiency. The scheme maintains a fixed compression ratio, and the size of the quantization error is also kept constant. This enables easy storage and retrieval of reference data. The impact of using lossy reference on the motion estimation quality is negligible. As a result of reduction in DDR traffic, the DDR power is reduced significantly. The power dissipation due to additional hardware required for reference frame compression is very small compared to the reduction in DDR power. 24% reduction in peak DDR bandwidth and 23% net reduction in average DDR power is achieved. For video sequences with larger motion, the amount of bandwidth reduction is even higher (close to 40%) and reduction in power is close to 30%.

In recent years, 3D camera products and prototypes based on Integral imaging (II) technique have gradually emerged and gained broad attention. II is a method that spatially samples the natural light (light field) of a scene, usually using a microlens array or a camera array and records the light field using a high resolution 2D image sensor. The large amount of data generated by II and the redundancy it contains together lead to the need for an efficient compression scheme. During recent years, the compression of 3D integral images has been widely researched. Nevertheless, there have not been many approaches proposed regarding the compression of integral videos (IVs). The objective of the thesis is to investigate efficient coding methods for integral videos. The integral video frames used are captured by the first consumer used light field camera Lytro. One of the coding methods is to encode the video data directly by an H.265/HEVC encoder. In other coding schemes the integral video is first converted to an array of sub-videos with different view perspectives. The sub-videos are then encoded either independently or following a specific reference picture pattern which uses a MVHEVC encoder. In this way the redundancy between the multi-view videos is utilized instead of the original elemental images. Moreover, by varying the pattern of the subvideo input array and the number of inter-layer reference pictures, the coding performance can be further improved. Considering the intrinsic properties of the input video sequences, a QP-per-layer scheme is also proposed in this thesis. Though more studies would be required regarding time and complexity constraints for real-time applications as well as dramatic increase of number of views, the methods proposed inthis thesis prove to be an efficient compression for integral videos.

Block partitioning is a computationally heavy step in the video coding process. Previously, this stage has been done using a full-search-esque algorithm. Recently, Artificial Neural Networks (ANN) approaches to speed-up block partitioning in encoders compliant to the Versatile Video Coding (VVC) standard have shown to significantly decrease the time needed for block partitioning. In this degree project, a state of the art Convolutional Neural Network (CNN) was ported to VTM16. It was ablated into 7 new models which were trained and tested. The eects of the ablations were compared and discussed with respect to the number of Multiply-Accumulate operations (MAC) a model required, the speed-up in the encoding stage as well as the quality of the encoding. The results show that the number of MACs can be substantially decreased from that of the state of the art model while having low negative eects on the quality of the encoding. Furthermore, the results show that the two tested approaches of reducing the computational complexity of the model were eective. Those were: 1) reducing the image’s resolution earlier in the model. 2) reducing the number of features in the beginning layers. The results point towards the first approach being more eective. / Blockpartitionering är ett beräkningstungt steg i videokodningsprocessen. Tidigare har detta gjorts genom att använda en algoritm i fullsökningsstil. Nyligen har artificiella neurala nätverk (ANN) visats vara eektiva för att minska tidsåtgången för blockpartitioneringen i enkodare som följer Versatile Video Coding-standarden (VVC). I detta examensarbete har en framgångsrik Convolutional Neural Networkmodell (CNN) portats till VTM16. Stegvisa ändringar på denna modell har gjorts för att ta fram sju modeller som tränades och testades. Eekten av ändringarna på ursprungsmodellen jämfördes och diskuterades med hänsyn till antalet Multiply-Accumulate-operationer (MAC) som respektive modell krävde, deras påverkan på tidsåtgången samt deras påverkan på kvalitén av kodningen. Resultaten visar att antalet MACs kan minskas betydligt utan att betydelsefullt minska kvalitén på kodningen. Resultaten visar att de båda testade tillvägagångssätt för att minska beräkningskomplexiteten var eektiva. Tillvägagångssätten var 1) minska bildens upplösning i ett tidigare skede i modellen. 2) minska antalet kanaler i de tidigare lagren. Resultaten pekar mot att det första tillvägagångssättet är mer eektivt.

Current trends on video technologies and services are demanding higher bit rates, highervideo resolutions and better video qualities. This issue results in the need of a new generationof video coding techniques to increase the quality and compression rates of previous standards.Since the release of HEVC, ITU-T VCEG and ISO/IEC MPEG have been studying the potentialneed for standardization of future video coding technologies with a compression capability thatsignificantly exceeds the ones from current standards. These new e↵orts of standardization andcompression enhancements are being implemented and evaluated over a software test modelknown under the name of Joint Exploration Model (JEM). One of the blocks being explored inJEM is an Adaptive Loop Filter (ALF) at the end of each frame’s processing flow. ALF aimsto minimize the error between original pixels and decoded pixels using Wiener-based adaptivefilter coefficients, reporting, in its JEM’s implementation, improvements of around a 1% in theBD MS-SSIM rate. A lot of e↵orts have been devoted on improving this block over the pastyears. However, current ALF implementations do not consider the potential use of adaptive QPalgorithms at the encoder. Adaptive QP algorithms enable the use of di↵erent quality levels forthe coding of di↵erent parts of a frame to enhance its subjective quality.In this thesis, we explore potential improvements over di↵erent dimensions of JEM’s AdaptiveLoop Filter block considering the potential use of adaptive QP algorithms. In the document, weexplore a great gamut of modification over ALF processing stages, being the ones with betterresults (i) a QP-aware implementation of ALF were the filter coefficients estimation, the internalRD-optimization and the CU-level flag decision process are optimized for the use of adaptiveQP, (ii) the optimization of ALF’s standard block activity classification stage through the useof CU-level information given by the di↵erent QPs used in a frame, and (iii) the optimizationof ALF’s standard block activity classification stage in B-frames through the application of acorrection weight on coded, i.e. not predicted, blocks of B-frames. These ALF modificationscombined obtained improvements of a 0.419% on average for the BD MS-SSIM rate in the lumachannel, showing each modification individual improvements of a 0.252%, 0.085% and 0.082%,respectively. Thus, we concluded the importance of optimizing ALF for the potential use ofadaptive-QP algorithms in the encoder, and the benefits of considering CU-level and frame-levelmetrics in ALF’s block classification stage. / Utvecklingen inom video-teknologi och service kräver högre bithastighet, högre videoupplösningoch bättre kvalitet. Problemet kräver en ny generation av kodning och tekniker för att ökakvaliteten och komprimeringsgraden utöver vad tidigare teknik kunnat prestera. Sedan lanseringenav HEVC har ITU-T VCEG och ISO/IEC MPEG studerat ett eventuellt behov av standardiseringav framtida video-kodings tekniker med kompressions kapacitet som vida överstigerdagens system. Dessa försök till standardisering och kompressionsframsteg har implementeratsoch utvärderats inom ramen för en mjukvara testmodell som kallas Joint Exploration Model(JEM). Ett av områdena som undersöks inom ramen för JEM är adaptiva loopfilter (ALF) somläggs till i slutet av varje bilds processflöde. ALF har som mål att minimera felet mellan originalpixel och avkodad pixel genom Wiener-baserade adaptiva filter-koefficienter. Mycket kraft harlagts på att förbättra detta område under de senaste åren. Men, nuvarande ALF-appliceringbeaktar inte potentialen av att använda adaptiva QP algoritmer i videokodaren. Adaptiva QPalgoritmer tillåter användningen av olika kvalitet på kodning av olika delar av bilden för attförbättra den subjektiva kvaliteten.I föreliggande uppsats kommer vi undersöka den potentiella förbättringen av JEM:s adaptivaloopfilter som kan uppnås genom att använda adaptiva QP algoritmer. I uppsatsen kommervi undersöka ett stort antal modifikationer i ALF:s process-stadier, för att ta reda på vilkenmodifikationer som har bäst resultat: (i) en QP-medveten implementering av ALF där filterkoefficientensuppskattning av den interna RD-optimeringen och CU-nivåns flaggbeslutsprocessär optimerade för användnngen av adaptiv QP, (ii) optimeringen av ALF:s standard blockaktivitets klassificerings stadie genom användning av CU-nivå-information producerad av deolika QP:n som används i en bild, och (iii) optimeringen av ALF:s standard block aktivitetsklassificerings stadier i B-bilders genom applicering av en korrektursvikt i tidigare kod, d.v.sej förutsedda, block av B-bilder. När dessa ALF modifikationer kombinerades förbättradesi genomsnitt BD MS-SSIM hastigheten i luma kanalen med 0.419%, där varje modifikationförbättrade med 0.252%, 0.085% och 0.082% var. Därigenom drog vi slutstatsen att det är viktigtatt optimera ALF för det potentiella användandet av adaptiva QP-algoritmer i kodaren, ochfördelarna av att beakta CU-nivåmätningar och bild-nivåmätningar i ALF:s block klassificeringsstadie.

The purpose of this project is to investigate the differences between psychoacoustic principles and genetic algorithms (GA0). These will be discussed separately. The review will also compare the compression ratio and the quality of the decompressed files decoded by these two methods.

Indiana University-Purdue University Indianapolis (IUPUI) / HEVC has achieved significant coding efficiency improvement beyond existing video coding standard by employing many new coding tools. Deblocking Filter, Sample Adaptive Offset and Adaptive Loop Filter for in-loop filtering are currently introduced for the HEVC standardization. However these filters are implemented in spatial domain despite the fact of temporal correlation within video sequences. To reduce the artifacts and better align object boundaries in video , a new algorithm in in-loop filtering is proposed. The proposed algorithm is implemented in HM-11.0 software. This proposed algorithm allows an average bitrate reduction of about 0.7% and improves the PSNR of the decoded frame by 0.05%, 0.30% and 0.35% in luminance and chroma.

An experimental investigation of an M level (M = 16, 64 and 256) Quadrature Amplitude Modulation (QAM) transmission system suitable for video transmission is presented. The communication system is based on layered video coding and unequal error protection to make the video bitstream robust to channel errors. An implementation is described in which H.264 video is protected unequally by partitioning the compressed data into two layers of different visual importance. The partition scheme is based on a separation of the group of pictures (GoP) in the intra-coded frame (I-frame) and predictive coded frame (P frame). This partition scheme is then applied to split the H.264-coded video bitstream and is suitable for Constant Bit Rate (CBR) transmission. Unequal error protection is based on uniform and non-uniform M-QAM constellations in conjunction with different scenarios of splitting the transmitted symbol for protection of the more important information of the video data; different constellation arrangements are proposed and evaluated to increase the capacity of the high priority layer. The performance of the transmission system is evaluated under Additive White Gaussian Noise (AWGN) and Rayleigh fading conditions. Simulation results showed that in noisy channels the decoded video can be improved by assigning a larger portion of the video data to the enhancement layer in conjunction with non-uniform constellation arrangements; in better channel conditions the quality of the received video can be improved by assigning more bits in the high priority channel and using uniform constellations. The aforementioned varying conditions can make the video transmission more successful over error-prone channels. Further techniques were developed to combat various channel impairments by considering channel coding methods suitable for layered video coding applications. It is shown that a combination of non-uniform M-QAM and forward error correction (FEC) will yield a better performance. Additionally, antenna diversity techniques are examined and introduced to the transmission system that can offer a significant improvement in the quality of service of mobile video communication systems in environments that can be modelled by a Rayleigh fading channel.

Poussé par la demande croissante de services multimédia dans les réseaux Internet à haut débit, la technologie WIMAX a émergé comme une alternative compétitive à la solution filaire d’accès à haut débit. L’IEEE 802.16 constitue une solution qui offre des débits élevés en assurant une qualité de service (QoS) satisfaisante. En particulier, elle est adaptée aux applications multimédia qui ont des contraintes de QoS à satisfaire. Cependant, avec la présence d’utilisateurs hétérogènes qui ont des caractéristiques diverses en termes de bande passante, de conditions radio et de ressources disponibles, de nouveaux défis posés doivent être résolus. En effet, les applications multimédia doivent interagir avec leur environnement pour informer le réseau d’accès de leurs besoins en QoS et s’adapter dynamiquement aux variations des conditions du réseau.Dans ce contexte, nous proposons deux solutions pour la transmission des flux vidéo sur les réseaux 802.16 sur la base de l’approche Cross-layer. Nous nous intéressons à la fois à la transmission unicast et multicast sur le lien montant et descendant d’une ou plusieurs cellules WIMAX.Premièrement, nous proposons une architecture Cross-Layer qui permet l’adaptation et l’optimisation du streaming vidéo en fonction des ressources disponibles. Nous avons défini une entité CLO (Cross-Layer Optimizer) qui exploite des messages de gestion des flux de service, échangés entre BS et SS, au niveau MAC, pour déterminer l’adaptation nécessaire et optimale afin d’assurer le bon fonctionnement de l’application. Les adaptations se produisent en deux temps, lors de l'admission du flux et au cours de la session de streaming. L’analyse des performances, par simulations, de notre solution montre l’efficacité du CLO à adapter, d’une façon dynamique, le débit vidéo en fonction des conditions du réseau afin d’assurer une QoS optimale.Deuxièmement, nous proposons une solution de streaming multicast des flux vidéo dans les réseaux WIMAX. Cette solution permet de trouver un compromis entre la diversité des clients, en termes de conditions radio, de schémas de modulation et de ressources disponibles, ainsi que le format de codage vidéo hiérarchique SVC, pour offrir la meilleure qualité vidéo y compris pour les clients ayant de faibles conditions radio. En effet, cette solution permet à chaque utilisateur d’obtenir une qualité vidéo proportionnellement à ses conditions radio et à sa bande passante disponible. Pour atteindre cet objectif, plusieurs groupes multicast sont formés par couches vidéo SVC. Cette solution permet d’optimiser davantage les ressources radio et ainsi d’augmenter la capacité globale du système. / Driven by the increasing demand for multimedia services in broadband Internet networks, WIMAX technology has emerged as a competitive alternative to the wired broadband access solutions. The IEEE 802.16 is a solution that provides high throughput by ensuring a satisfactory QoS. In particular, it is suitable for multimedia applications that have strict QoS constraints. However, the users’ heterogeneity and diversity in terms of bandwidth, radio conditions and available resources, pose new deployment challenges. Indeed, multimedia applications need to interact with their environment to inform the access network about their QoS requirements and dynamically adapt to changing network conditions.In this context, we propose two solutions for video streaming over 802.16 networks based on Cross-Layer approach. We are interested in both unicast and multicast transmissions in uplink and downlink of one or more WIMAX cells.First, we proposed an architecture that enables Cross-Layer adaptation and optimization of video streaming based on available resources. We defined the entity CLO (Cross-Layer Optimizer) that takes benefits from service flow management messages, exchanged between BS and SS, at the MAC level, to determine the necessary adaptations / adjustment to ensure optimal delivery of the application. Adaptations occur at two epochs, during the admission of the video stream and during the streaming phase. The performance analysis, performed through simulations, shows the effectiveness of the CLO to adapt in a dynamic way, the video data rate depending on network conditions, and thus guarantee an optimal QoS.Second, we proposed a solution that enables IP multicast video delivery in WIMAX network. This solution allows finding the compromise between the diversity of end-user requirements, in terms of radio conditions, modulation schemes and available resources, along with the SVC hierarchy video format, to offer the best video quality even for users with low radio conditions. Indeed, we define a multicast architecture that allows each user to get a video quality proportionally to its radio conditions and its available bandwidth. Towards this end, several IP multicast groups are created depending on the SVC video layers. Subsequently, our solution allows optimizing the use of radio resources by exploiting the different modulations that can be selected by the end-users.