Le développement du multimédia, avec l'émergence des architectures parallèles, a ravivé l'intérêt de la programmation flux de données pour la conception de systèmes embarqués. En effet, la programmation flux de données offre une approche de développement suffisamment flexible pour créer des applications complexes tout en exprimant la concurrence et le parallélisme explicitement. Paradoxalement, la plupart des études portent sur des modèles flux de données statiques, même si un processus de développement pragmatique nécessite l'expressivité et la practicité d'un langage de programmation basé sur un modèle flux de données dynamiques, comme le langage de programmation utilisé dans le cadre de Reconfigurable Video Coding. Dans cette thèse, nous décrivons un environnement de développement pour la programmation flux de données qui facilite le développement multimédia pour des plates-formes multi-coeur embarquées. Cet environnement de développement repose sur une architecture logicielle modulaire qui bénéficie de techniques modernes de génie logiciel telles que la méta modélisation et la programmation orientée aspect. Ensuite, nous développons une implémentation logicielle optimisée des programmes flux de données ciblant aussi bien les ordinateurs de bureau que les plates-formes embarquées. Notre implémentation vise à combler le fossé entre la practicité du langage de programmation et l'efficacité de son exécution. Enfin, nous présentons un ensemble d'algorithmes de projection et d'ordonnancement d'acteurs qui permettent l'exécution de programmes flux de données dynamiques sur des plates-formes multi-coeur avec des performances extensibles. / The development of multimedia technology, along with the emergence of parallel architectures, has revived the interest on dataflow programming for designing embedded systems. Indeed, dataflow programming offers a flexible development approach in order to build complex applications while expressing concurrency and parallelism explicitly. Paradoxically, most of the studies focus on static dataflow models of computation, even if a pragmatic development process requires the expressiveness and the practicality of a programming language based on dynamic dataflow models, such as the language included in the Reconfigurable Video Coding framework. In this thesis, we describe a complete development environment for dataflow programming that eases multimedia development for embedded multi-core platforms. This development environment is built upon a modular software architecture that benefits from modern software engineering techniques such as meta modeling and aspect-oriented programming. Then, we develop an optimized software implementation of dataflow programs targeting desktop and embedded multi-core platforms. Our implementation aims to bridge the gap between the practicality of the programming language and the efficiency of the execution. Finally, we present a set of runtime actors mapping/scheduling algorithms that enable the execution of dynamic dataflow programs over multi-core platforms with scalable performance.

La compression vidéo est une étape cruciale pour une grande partie des applications de télécommunication. Depuis l'avènement de la norme H.261/MPEG-2, un nouveau standard de compression vidéo est produit tous les 10 ans environ, avec un gain en compression de 50% par rapport à la précédente. L'objectif de la thèse est d'obtenir des gains en compression par rapport à la dernière norme de codage vidéo HEVC. Dans cette thèse, nous proposons trois approches pour améliorer la compression vidéo en exploitant les corrélations du résidu de prédiction intra. Une première approche basée sur l'utilisation de résidus précédemment décodés montre que, si des gains sont théoriquement possibles, le surcoût de la signalisation les réduit pratiquement à néant. Une deuxième approche basée sur la quantification vectorielle mode-dépendent (MDVQ) du résidu préalablement à l'étape classique transformée-quantification scalaire, permet d'obtenir des gains substantiels. Nous montrons que cette approche est réaliste, car les dictionnaires sont indépendants du QP et de petite taille. Enfin, une troisième approche propose de rendre adaptatif les dictionnaires utilisés en MDVQ. Un gain substantiel est apporté par l'adaptivité, surtout lorsque le contenu vidéo est atypique, tandis que la complexité de décodage reste bien contenue. Au final on obtient un compromis gain-complexité compatible avec une soumission en normalisation. / Video compression has become a mandatory step in a wide range of digital video applications. Since the development of the block-based hybrid coding approach in the H.261/MPEG-2 standard, new coding standard was ratified every ten years and each new standard achieved approximately 50% bit rate reduction compared to its predecessor without sacrificing the picture quality. However, due to the ever-increasing bit rate required for the transmission of HD and Beyond-HD formats within a limited bandwidth, there is always a requirement to develop new video compression technologies which provide higher coding efficiency than the current HEVC video coding standard. In this thesis, we proposed three approaches to improve the intra coding efficiency of the HEVC standard by exploiting the correlation of intra prediction residue. A first approach based on the use of previously decoded residue shows that even though gains are theoretically possible, the extra cost of signaling could negate the benefit of residual prediction. A second approach based on Mode Dependent Vector Quantization (MDVQ) prior to the conventional transformed scalar quantization step provides significant coding gains. We show that this approach is realistic because the dictionaries are independent of QP and of a reasonable size. Finally, a third approach is developed to modify dictionaries gradually to adapt to the intra prediction residue. A substantial gain is provided by the adaptivity, especially when the video content is atypical, without increasing the decoding complexity. In the end we get a compromise of complexity and gain for a submission in standardization.

Context: The newest standard in video coding High Efficiency Video Coding (HEVC) should have an appropriate coder to fully use its potential. There are a lot of video quality assessment methods. These methods are necessary to establish the quality of the video. Objectives: This thesis is a comparison of video quality assessment methods. Objective is to find out which objective method is the most similar to the subjective method. Videos used in tests are encoded in the H.265/HEVC standard. Methods: For testing MSE, PSNR, SSIM methods there is special software created in MATLAB. For VQM method downloaded software was used for testing. Results and conclusions: For videos watched on mobile device: PSNR is the most similar to subjective metric. However for videos watched on television screen: VQM is the most similar to subjective metric. Keywords: Video Quality Assessment, Video Quality Prediction, Video Compression, Video Quality Metrics

En raison de la grande disponibilité des dispositifs de capture vidéo et des nouvelles pratiques liées aux réseaux sociaux, ainsi qu’à l’émergence des services en ligne, les images et les vidéos constituent aujourd’hui une partie importante de données transmises sur internet. Les applications de streaming vidéo représentent ainsi plus de 70% de la bande passante totale de l’internet. Des milliards d’images sont déjà stockées dans le cloud et des millions y sont téléchargés chaque jour. Les besoins toujours croissants en streaming et stockage nécessitent donc une amélioration constante des outils de compression d’image et de vidéo. Cette thèse vise à explorer des nouvelles approches pour améliorer les méthodes actuelles de prédiction inter-images. De telles méthodes tirent parti des redondances entre images similaires, et ont été développées à l’origine dans le contexte de la vidéo compression. Dans une première partie, de nouveaux outils de prédiction inter globaux et locaux sont associés pour améliorer l’efficacité des schémas de compression de bases de données d’image. En associant une compensation géométrique et photométrique globale avec une prédiction linéaire locale, des améliorations significatives peuvent être obtenues. Une seconde approche est ensuite proposée qui introduit un schéma de prédiction inter par régions. La méthode proposée est en mesure d’améliorer les performances de codage par rapport aux solutions existantes en estimant et en compensant les distorsions géométriques et photométriques à une échelle semi locale. Cette approche est ensuite adaptée et validée dans le cadre de la compression vidéo. Des améliorations en réduction de débit sont obtenues, en particulier pour les séquences présentant des mouvements complexes réels tels que des zooms et des rotations. La dernière partie de la thèse se concentre sur l’étude des méthodes d’apprentissage en profondeur dans le cadre de la prédiction inter. Ces dernières années, les réseaux de neurones profonds ont obtenu des résultats impressionnants pour un grand nombre de tâches de vision par ordinateur. Les méthodes basées sur l’apprentissage en profondeur proposées à l’origine pour de l’interpolation d’images sont étudiées ici dans le contexte de la compression vidéo. Des améliorations en terme de performances de codage sont obtenues par rapport aux méthodes d’estimation et de compensation de mouvements traditionnelles. Ces résultats mettent en évidence le fort potentiel de ces architectures profondes dans le domaine de la compression vidéo. / Due to the large availability of video cameras and new social media practices, as well as the emergence of cloud services, images and videos constitute today a significant amount of the total data that is transmitted over the internet. Video streaming applications account for more than 70% of the world internet bandwidth. Whereas billions of images are already stored in the cloud and millions are uploaded every day. The ever growing streaming and storage requirements of these media require the constant improvements of image and video coding tools. This thesis aims at exploring novel approaches for improving current inter-prediction methods. Such methods leverage redundancies between similar frames, and were originally developed in the context of video compression. In a first approach, novel global and local inter-prediction tools are associated to improve the efficiency of image sets compression schemes based on video codecs. By leveraging a global geometric and photometric compensation with a locally linear prediction, significant improvements can be obtained. A second approach is then proposed which introduces a region-based inter-prediction scheme. The proposed method is able to improve the coding performances compared to existing solutions by estimating and compensating geometric and photometric distortions on a semi-local level. This approach is then adapted and validated in the context of video compression. Bit-rate improvements are obtained, especially for sequences displaying complex real-world motions such as zooms and rotations. The last part of the thesis focuses on deep learning approaches for inter-prediction. Deep neural networks have shown striking results for a large number of computer vision tasks over the last years. Deep learning based methods proposed for frame interpolation applications are studied here in the context of video compression. Coding performance improvements over traditional motion estimation and compensation methods highlight the potential of these deep architectures.

This thesis deals with possible and realized decompression and playing of video on platforms, using FPGA unit. For implementation of this player is used platform FITKit, which has integrated VGA connector and large enough RAM memory. It uses a hard drive as memory medium with FAT32 file system.

Diploma thesis is focused on real-time 4K video encoding using NVENC technology. First chapter describes the most used video codecs H.264 and HEVC. There is an explanation of the principle of graphic cards and their programmable units. Analysis of the solution of open source Video Codec SDK is also part of the thesis. The main focus of the thesis is an implementation of an application which can handle 4K video encoding from multiple cameras in real time. Performance and qualitative tests were performed for application. Results of these tests were analyzed and discussed.

This diploma thesis focuses on current possibilities concerning the employment of wavelet transformation for video signal compression. One part of the work is devoted to the necessary execution of this task in practise. This deals with video signal and its features description, wavelet transformation and compression methods. The second part concentrates on description of selected compression method. It is the SPIHT (Set Partitioning In Hierarchical Trees) algorithm which is intended for compression of static image data. The algorithm is modified for usage with video signal compression which is specific for its time redundancy. The algorithm is called 3D SPIHT as it works in the spatial and time domain. The algorithm is implemented in the MATLAB programming environment which provides a sophisticated support for wavelet transformation (Wavelet Toolbox). To provide a simple and intuitive encoder control there has been developed an application delivering graphical user interface (GUI). On displayed image previews and measured graphs the user can change encoder parameters and monitor performed changes. There are four image test-sequential modes containing various scenes with different features. The final part of the work is focused on testing of the proposed encoding scheme, various image test-sequential modes and encoder settings. Measured values are graphically displayed and analyzed.

<div> <div> <div> <p>In this thesis, two aspects of video coding improvement are discussed, namely error resilience and coding efficiency. </p> <p>With the increasing amount of videos being created and consumed, better video compression tools are needed to provide reliable and fast transmission. Many popular video coding standards such as VPx, H.26x achieve video compression by using spa- tial and temporal dependencies in the source video signal. This makes the encoded bitstream vulnerable to errors during transmission. In this thesis, we investigate an error resilient video coding for the VP9 bitstreams using error resilience packets. An error resilient packet consists of encoded keyframe contents and the prediction sig- nals for each non-keyframe. Experimental results exhibit that our proposed method is effective under typical packet loss conditions. </p> <p>In the second part of the thesis, we first present an automatic stillness feature detection method for group of pictures. The encoder adaptively chooses the coding structure for each group of pictures based on its stillness feature to optimize the coding efficiency. </p> <p>Secondly, a content-based video coding method is proposed. Modern video codecs including the newly developed AOM/AV1 utilize hybrid coding techniques to remove spatial and temporal redundancy. However, the efficient exploitation of statistical dependencies measured by a mean squared error (MSE) does not always produce the best psychovisual result. One interesting approach is to only encode visually relevant information and use a different coding method for “perceptually insignificant” regions </p> </div> </div> <div> <div> <p>xiv </p> </div> </div> </div> <div> <div> <div> <p>in the frame. In this thesis, we introduce a texture analyzer before encoding the input sequences to identify detail irrelevant texture regions in the frame using convolutional neural networks. The texture region is then reconstructed based on one set of motion parameters. We show that for many standard test sets, the proposed method achieved significant data rate reductions. </p> </div> </div> </div>

Digital video is becoming extremely important nowadays and its importance has greatly increased in the last two decades. Due to the rapid development of information and communication technologies, the demand for Ultra-High Definition (UHD) video applications is becoming stronger. However, the most prevalent video compression standard H.264/AVC released in 2003 is inefficient when it comes to UHD videos. The increasing desire for superior compression efficiency to H.264/AVC leads to the standardization of High Efficiency Video Coding (HEVC). Compared with the H.264/AVC standard, HEVC offers a double compression ratio at the same level of video quality or substantial improvement of video quality at the same video bitrate. Yet, HE-VC/H.265 possesses superior compression efficiency, its complexity is several times more than H.264/AVC, impeding its high throughput implementation. Currently, most of the researchers have focused merely on algorithm level adaptations of HEVC/H.265 standard to reduce computational intensity without considering the hardware feasibility. What’s more, the exploration of efficient hardware architecture design is not exhaustive. Only a few research works have been conducted to explore efficient hardware architectures of HEVC/H.265 standard. In this dissertation, we investigate efficient algorithm adaptations and hardware architecture design of HEVC intra encoders. We also explore the deep learning approach in mode prediction. From the algorithm point of view, we propose three efficient hardware-oriented algorithm adaptations, including mode reduction, fast coding unit (CU) cost estimation, and group-based CABAC (context-adaptive binary arithmetic coding) rate estimation. Mode reduction aims to reduce mode candidates of each prediction unit (PU) in the rate-distortion optimization (RDO) process, which is both computation-intensive and time-consuming. Fast CU cost estimation is applied to reduce the complexity in rate-distortion (RD) calculation of each CU. Group-based CABAC rate estimation is proposed to parallelize syntax elements processing to greatly improve rate estimation throughput. From the hardware design perspective, a fully parallel hardware architecture of HEVC intra encoder is developed to sustain UHD video compression at 4K@30fps. The fully parallel architecture introduces four prediction engines (PE) and each PE performs the full cycle of mode prediction, transform, quantization, inverse quantization, inverse transform, reconstruction, rate-distortion estimation independently. PU blocks with different PU sizes will be processed by the different prediction engines (PE) simultaneously. Also, an efficient hardware implementation of a group-based CABAC rate estimator is incorporated into the proposed HEVC intra encoder for accurate and high-throughput rate estimation. To take advantage of the deep learning approach, we also propose a fully connected layer based neural network (FCLNN) mode preselection scheme to reduce the number of RDO modes of luma prediction blocks. All angular prediction modes are classified into 7 prediction groups. Each group contains 3-5 prediction modes that exhibit a similar prediction angle. A rough angle detection algorithm is designed to determine the prediction direction of the current block, then a small scale FCLNN is exploited to refine the mode prediction.

The compressed sensing (CS) model of signal processing, while offering many unique advantages in terms of low-cost sensor design, poses interesting challenges for both signal acquisition and recovery, especially for signals of large size. In this work, we investigate how CS might be applied practically and efficiently in the context of natural video. We make use of a CS video acquisition approach in line with the popular single-pixel camera framework of blind, nonaptive, random sampling while proposing new approaches for the subsequent recovery of the video signal which leverage interrame redundancy to minimize recovery error. We introduce a method of approximation, which we term multihypothesis (MH) frame prediction, to create accurate frame predictions by comparing hypotheses drawn from the spatial domain of chosen reference frames to the non-overlapping, block-by-block CS measurements of subsequent frames. We accomplish this frame prediction via a novel distance-weighted Tikhonov regularization technique. We verify through our experiments that MH frame prediction via distance-weighted regularization provides state-of-the-art performance for the recovery of natural video sequences from blind CS measurements. The distance-weighted regularization we propose need not be limited to just frame prediction for CS video recovery, but may also be used in a variety of contexts where approximations must be generated from a set of hypotheses or training data. To show this, we apply our technique to supervised hyperspectral image (HSI) classification via a novel classifier we term the nearest regularized subspace (NRS) classifier. We show that the distance-weighted regularization used in the NRS method provides greater classification accuracy than state-of-the-art classifiers for supervised HSI classification tasks. We also propose two modifications to the core NRS classifier to improve its robustness to variation of input parameters and and to further increase its classification accuracy.