When a demanding video decoding task requires more CPU resources then available, playback degrades ungracefully today: The decoder skips frames selected arbitrarily or by simple heuristics, which is noticed by the viewer as jerky motion in the good case or as images completely breaking up in the bad case. The latter can happen due to missing reference frames. This thesis provides a way to schedule individual decoding tasks based on a cost for performance trade. Therefore, I will present a way to preprocess a video, generating estimates for the cost in terms of execution time and the performance in terms of perceived visual quality. The granularity of the scheduling decision is a single slice, which leads to a much more fine-grained approach than dealing with entire frames. Together with an actual scheduler implementation that uses the generated estimates, this work allows for higher perceived quality video playback in case of CPU overload. / Wenn eine anspruchsvolle Video-Dekodierung mehr Prozessor-Ressourcen benötigt, als verfügbar sind, dann verschlechtert sich die Abspielqualität mit aktuellen Methoden drastisch: Willkürlich oder mit einfachen Heuristiken ausgewählten Bilder werden nicht dekodiert. Diese Auslassung nimmt der Betrachter im günstigsten Fall nur als ruckelnde Bewegung wahr, im ungünstigen Fall jedoch als komplettes Zusammenbrechen nachfolgender Bilder durch Folgefehler im Dekodierprozess. Meine Arbeit ermöglicht es, einzelne Teilaufgaben des Dekodierprozesses anhand einer Kosten-Nutzen-Analyse einzuplanen. Dafür ermittle ich die Kosten im Sinne von Rechenzeitbedarf und den Nutzen im Sinne von visueller Qualität für einzelne Slices eines H.264 Videos. Zusammen mit einer Implementierung eines Schedulers, der diese Werte nutzt, erlaubt meine Arbeit höhere vom Betrachter wahrgenommene Videoqualität bei knapper Prozessorzeit.

Cette thèse a pour ambition d’explorer d’un point de vue théorique et applicatif le paradigme de l’in-band enrichment. Emergence de la société des connaissances, le concept de média enrichi renvoie à toute association de métadonnée (textuelle, audiovisuelle, code exécutable) avec un média d’origine. Un tel principe peut être déployé dans une large variété d’applications comme la TVNi - Télévision Numérique interactive, les jeux ou la fouille des données. Le concept de l’inband enrichement conçu et développé par M. Mitrea et son équipe au Département ARTEMIS de Télécom SudParis, suppose que les données d’enrichissement sont insérées dans le contenu même à enrichir. Ainsi, un tel concept peut-il tirer parti de techniques de tatouage, dès lors que celles-ci démontrent qu’elles ont la capacité d’insérer la quantité d’information requise par ce nouveau type d’application : i.e. 10 à 1000 fois plus grande que celle nécessaire pour les enjeux d’authentification ou de protection de droit d’auteur. Si par tradition la marque est insérée dans le domaine non compressé, les contraintes relatives aux nombreuses applications émergentes (comme la VoD – Vidéo à la Demande ou la TVNi) font du tatouage en temps réel dans le domaine compressé un important défi théorique et applicatif. Cependant, le tatouage dans le domaine compressé est une alliance de mots contradictoires puisque la compression (élimination de la redondance) rend l’hôte plus sensible aux modifications et l’association hôte/marque, plus fragile / The present thesis, developed at Institut Télécom Télécom SudParis under the “Futur et Rupture” framework, takes the challenge of exploring from both theoretical and applicative points of views the in band enrichment paradigm. Emerged with the knowledge society, the enriched media refers to any type of association which may be established between some metadata (textual, audio, video, exe codes...) and a given original media. Such a concept is currently deployed in a large variety of applications like the iDTV (interactive Digital TV), games, data mining... The incremental notion of in band enrichment advanced at the ARTEMIS Department assumes that the enrichment data are directly inserted into the very original media to be enriched. In real life, in band enrichment can be supported by the watermarking technologies, assuming they afford a very large data payload, i.e. 10 to 1000 larger than the traditional copyright applications. The nowadays advent of the ubiquous media computing and storage applications imposes an additional constraint on the watermarking techniques: the enrichment data should be inserted into some compressed original media. A priori, such a requirement is a contradiction in terms, as compression eliminates the visual redundancy while the watermarking exploits the visual redundancy in order to imperceptibly insert the mark

Streaming of on-line gaming video is a challenging problem because of the enormous amounts of video data that need to be sent during game playing, especially within the limitations of uplink capabilities. The encoding complexity is also a challenge because of the time delay while on-line gamers are communicating. The main goal of this research study is to propose an enhanced on-line game video streaming system. First, the most common video coding techniques have been evaluated. The evaluation study considers objective and subjective metrics. Three widespread video coding techniques are selected and evaluated in the study; H.264, MPEG-4 Visual and VP- 8. Diverse types of video sequences were used with different frame rates and resolutions. The effects of changing frame rate and resolution on compression efficiency and viewers' satisfaction are also presented. Results showed that the compression process and perceptual satisfaction are severely affected by the nature of the compressed sequence. As a result, H.264 showed higher compression efficiency for synthetic sequences and outperformed other codecs in the subjective evaluation tests. Second, a fast inter prediction technique to speed up the encoding process of H.264 has been devised. The on-line game streaming service is a real time application, thus, compression complexity significantly affects the whole process of on-line streaming. H.264 has been recommended for synthetic video coding by our results gained in codecs comparative studies. However, it still suffers from high encoding complexity; thus a low complexity coding algorithm is presented as fast inter coding model with reference management technique. The proposed algorithm was compared to a state of the art method, the results showing better achievement in time and bit rate reduction with negligible loss of fidelity. Third, recommendations on tradeoff between frame rates and resolution within given uplink capabilities are provided for H.264 video coding. The recommended tradeoffs are offered as a result of extensive experiments using Double Stimulus Impairment Scale (DSIS) subjective evaluation metric. Experiments showed that viewers' satisfaction is profoundly affected by varying frame rates and resolutions. In addition, increasing frame rate or frame resolution does not always guarantee improved increments of perceptual quality. As a result, tradeoffs are recommended to compromise between frame rate and resolution within a given bit rate to guarantee the highest user satisfaction. For system completeness and to facilitate the implementation of the proposed techniques, an efficient game video streaming management system is proposed. Compared to existing on-line live video service systems for games, the proposed system provides improved coding efficiency, complexity reduction and better user satisfaction.

Today, eavesdropping is becoming a common issue in the rapidly growing digital network and has foreseen the need for secret communication channels embedded in digital media. In this paper, a novel steganography technique designed for Standard Definition Digital Television (SDTV) H.264/AVC encoded video sequences is presented. The algorithm introduced here makes use of the compression properties of the Context Adaptive Variable Length Coding (CAVLC) entropy encoder to achieve a low complexity and real-time inserting method. The chosen scheme hides the private message directly in the H.264/AVC bit stream by modifying the AC frequency quantized residual luminance coefficients of intrapredicted I-frames. In order to avoid error propagation in adjacent blocks, an interlaced embedding strategy is applied. Likewise, the steganography technique proposed allows self-detection of the hidden message at the target destination. The code source was implemented by mixing MATLAB 2010 b and Java development environments. Finally, experimental results have been assessed through objective and subjective quality measures and reveal that less visible artifacts are produced with the technique proposed by reaching PSNR values above 40.0 dB and an embedding bit rate average per secret communication channel of 425 bits/sec. This exemplifies that steganography is affordable in digital television.

"Packet loss can be detrimental to real-time interactive video over lossy networks because one lost video packet can propagate errors to many subsequent video frames due to the encoding dependency between frames. Feedback-based error control techniques use feedback information from the decoder to adjust coding parameters at the encoder or retransmit lost packets to reduce the error propagation due to data loss. Feedback-based error control techniques have been shown to be more effective than trying to conceal the error at the encoder or decoder alone since they allow the encoder and decoder to cooperate in the error control process. However, there has been no systematic exploration of the impact of video content and network conditions on the performance of feedback-based error control techniques. In particular, the impact of packet loss, round-trip delay, network capacity constraint, video motion and reference distance on the quality of videos using feedback-based error control techniques have not been systematically studied. This thesis presents analytical models for the major feedback-based error control techniques: Retransmission, Reference Picture Selection (both NACK and ACK modes) and Intra Update. These feedback-based error control techniques have been included in H.263/H.264 and MPEG4, the state of the art video in compression standards. Given a round-trip time, packet loss rate, network capacity constraint, our models can predict the quality for a streaming video with retransmission, Intra Update and RPS over a lossy network. In order to exploit our analytical models, a series of studies has been conducted to explore the effect of reference distance, capacity constraint and Intra coding on video quality. The accuracy of our analytical models in predicting the video quality under different network conditions is validated through simulations. These models are used to examine the behavior of feedback-based error control schemes under a variety of network conditions and video content through a series of analytic experiments. Analysis shows that the performance of feedback-based error control techniques is affected by a variety of factors including round-trip time, loss rate, video content and the Group of Pictures (GOP) length. In particular: 1) RPS NACK achieves the best performance when loss rate is low while RPS ACK outperforms other repair techniques when loss rate is high. However RPS ACK performs the worst when loss rate is low. Retransmission performs the worst when the loss rate is high; 2) for a given round-trip time, the loss rate where RPS NACK performs worse than RPS ACK is higher for low motion videos than it is for high motion videos; 3) Videos with RPS NACK always perform the same or better than videos without repair. However, when small GOP sizes are used, videos without repair perform better than videos with RPS ACK; 4) RPS NACK outperform Intra Update for low-motion videos. However, the performance gap between RPS NACK and Intra Update drops when the round-trip time or the intensity of video motion increases. 5) Although the above trends hold for both VQM and PSNR, when VQM is the video quality metric the performance results are much more sensitive to network loss. 6) Retransmission is effective only when the round-trip time is low. When the round-trip time is high, Partial Retransmission achieves almost the same performance as Full Retransmission. These insights derived from our models can help determine appropriate choices for feedback-based error control techniques under various network conditions and video content. "

"A mobile ultrasound system has been developed, which makes ultrasound examinations possible in harsh environments without reliable power sources, such as ambulances, helicopters, war zones, and disaster sites. The goal of this project was to analyze three different wireless communication technologies that could be integrated into the ultrasound system for possible utilization in remote data applications where medical information may be transmitted from the mobile unit to some centralized base station, such as an emergency room or field hospital. By incorporating wireless telecommunication technology into the design, on site medical personnel can be assisted in diagnostic decisions by remote medical experts. The wireless options that have been tested include the IEEE 802.11g standard, mobile broadband cards on a 3G cellular network, and a mobile satellite terminal. Each technology was tested in two phases. In the first phase, a client/server application was developed to measure and record general information about the quality of each link. Four different types of tests were developed to measure channel properties such as data rate, latency, inter-arrival jitter, and packet loss using various signal strengths, packet sizes, network protocols, and traffic loads. In the second phase of testing, the H.264 Scalable Video Codec (SVC) was used to transmit real-time ultrasound video streams over each of the wireless links to observe the image quality as well as the diagnostic value of the received video stream. The information gathered during both testing phases revealed the abilities and limitations of the different wireless technologies. The results from the performance testing will be valuable in the future for those trying to develop network applications for telemedicine procedures over these wireless telecommunication options. Additionally, the testing demonstrated that the system is currently capable of using H.264 SVC compression to transmit VGA quality ultrasound video at 30 frames per second (fps) over 802.11g while QVGA resolution at frame rates between 10 and 15 fps is possible over 3G and satellite networks."

All multimedia devices now incorporate video CODECs that comply with international video coding standards such as H.264 / MPEG4-AVC and the new High Efficiency Video Coding Standard (HEVC) otherwise known as H.265. Although the standard CODECs have been designed to include algorithms with optimal efficiency, large number of coding parameters can be used to fine tune their operation, within known constraints of for e.g., available computational power, bandwidth, consumer QoS requirements, etc. With large number of such parameters involved, determining which parameters will play a significant role in providing optimal quality of service within given constraints is a further challenge that needs to be met. Further how to select the values of the significant parameters so that the CODEC performs optimally under the given constraints is a further important question to be answered. This thesis proposes a framework that uses machine learning algorithms to model the performance of a video CODEC based on the significant coding parameters. Means of modelling both the Encoder and Decoder performance is proposed. We define objective functions that can be used to model the performance related properties of a CODEC, i.e., video quality, bit-rate and CPU time. We show that these objective functions can be practically utilised in video Encoder/Decoder designs, in particular in their performance optimisation within given operational and practical constraints. A Multi-objective Optimisation framework based on Genetic Algorithms is thus proposed to optimise the performance of a video codec. The framework is designed to jointly minimize the CPU Time, Bit-rate and to maximize the quality of the compressed video stream. The thesis presents the use of this framework in the performance modelling and multi-objective optimisation of the most widely used video coding standard in practice at present, H.264 and the latest video coding standard, H.265/HEVC. When a communication network is used to transmit video, performance related parameters of the communication channel will impact the end-to-end performance of the video CODEC. Network delays and packet loss will impact the quality of the video that is received at the decoder via the communication channel, i.e., even if a video CODEC is optimally configured network conditions will make the experience sub-optimal. Given the above the thesis proposes a design, integration and testing of a novel approach to simulating a wired network and the use of UDP protocol for the transmission of video data. This network is subsequently used to simulate the impact of packet loss and network delays on optimally coded video based on the framework previously proposed for the modelling and optimisation of video CODECs. The quality of received video under different levels of packet loss and network delay is simulated, concluding the impact on transmitted video based on their content and features.

Dans cet étude, nous nous concentrons sur l'analyse et l'indexation automatique des séquences vidéo. Cela permet une récupération et un accès efficace aux données. Les vidéos étant généralement stockées et transmises sous forme encodées, notre approche se base sur l'analyse dans le domaine compressé. Ce domaine présente un compromis raisonnable entre cout de calcul et qualité des résultats. Les informations les plus pertinentes encodées dans le flux H.264 sont les vecteurs de mouvement (VM). Ces vecteurs sont associés aux macroblocs dans les images prédites (de type B et P). Nous présentons diverses techniques non-supervisés pour déterminer le mouvement de la caméra, détecter et suivre les objets en mouvement dans les vidéos. Ces analyses sont basées sur les champs de VM. Nous démontrons le potentiel de l'analyse dans le domaine compressé avec plusieurs exemples d'applications, notamment la surveillance de trafic routier et la détection de copies. / The amount of generated audiovisual content continues to increase. In this work, we concentrate on the challenge of automatic video analysis and indexing, which builds the basis for efficient information access and retrieval. Taking advantage of the fact that video in most cases is stored and transmitted in encoded form, we pursue the approach of compressed domain processing, which represents a reasonable trade-off between computational complexity and quality of analysis results. The most valuable information encoded in H.264 streams is the motion vectors (MVs) that are associated with macroblocks in temporally predicted frames of type B and P. We present a number of completely unsupervised techniques to determine the camera motion and to detect and track moving objects from the extracted MV fields. We furthermore present the potential of compressed domain processing through several example applications, including traffic surveillance and video copy detection.

Cette thèse s'intéresse à améliorer les codeurs vidéo actuels tels que H.264 MPEG-4/AVC en utilisant avantageusement des informations structurelles contenues dans les images codées. Dans ce contexte, on observe que tous codeurs vidéo utilisent une étape de transformation permettant de décorréler mathématiquement les informations traitées afin d'en diminuer le coût de codage entropique. D'autre part, on remarque que toutes les images traitées qu'elles soient prédites ou non, possèdent des structures géométriques très marquées. Une étude des transformées existantes et possibles pour ces codeurs vidéo montre que peu d'entre elles (DCT, en ondelettes, à recouvrement, DCT sous forme lifting...) permettent de représenter efficacement ces structures géométriques des images. L'état de l'art de ces transformées exploitant les structures géométriques est porté historiquement par les ondelettes de seconde génération comme les contourlets, les bandelettes ou les directionlets. Mais, plusieurs études plus récentes utilisent des approches DCT, basées blocs, avec des orientations afin de mieux représenter ces structures géométriques. L'objectif de notre étude est d'améliorer le codage des images résiduelles H.264/AVC, issues de prédictions spatiales (Intra) ou temporelles (Inter), en utilisant leurs structures géométriques. Une première approche de ce travail de thèse nous a conduits à analyser et exploiter des méthodes connues de l'état de l'art. Pour cela, nous avons déni un schéma sous forme lifting réalisant les opérations de la DCT H.264/AVC. Ce schéma permet de voir cette DCT comme une transformée en ondelettes et donc de disposer d'une approche commune. On peut alors lui appliquer des outils de seconde génération an qu'elle représente au mieux les structures géométriques des images. Et, une version de transformée à recouvrement en pré- et post-traitements a été utilisée dans le codeur H.264/AVC. Une seconde approche de nos recherches a été de dénir une méthode d'orientation par pré- et post-traitements associée à un parcours adapté des coefficients quantifiés produits. Le pré-traitement de cette méthode d'orientation réalise des pseudo-rotations permettant de redresser les blocs des images vers l'horizontale ou la verticale. Cette opération est réalisée par cisaillements, soit par permutations circulaires des pixels, améliorant la décorrélation de la DCT qui suit sans présenter les défauts inhérents aux approches de l'état de l'art. Cette méthode, insérée dans un codeur H.264/AVC, présente de bonnes performances de codage. Cependant, le coût des informations d'orientation, sélectionnées selon un critère débit-distorsion, est élevé dégradant ces performances dans les bas débits, la méthode restant plus efficace que H.264/AVC dans les hauts débits (QP < 30). Les coefficients quantifiés issus de la méthode d'orientation précédente sont ensuite parcourus à la verticale, à l'horizontale ou en zigzag suivant les redressements appliqués ou le type de partitions. Cette adaptation de parcours permet de légèrement conserver du débit améliorant ainsi notre méthode globale qui devient plus efficaces que H.264/AVC dans les moyens débits (QP < 35).

<p>H.264/AVC is a new video compression standard designed for future broadband network. Compared with former video coding standards such as MPEG-2 and MPEG-4 part 2, it saves up to 40% in bit rate and provides important characteristics such as error resilience, stream switching etc. However, the improvement in performance also introduces increase in computational complexity, which requires more powerful hardware. At the same time, there are several image and video coding standards currently used such as JPEG and MPEG-4. Although ASIC design meets the performance requirement, it lacks flexibility for heterogeneous standards. Hence reconfigurable DSP processor is more suitable for media processing since it provides both real-time performance and flexibility. </p><p>Currently there are several single scalar DSP processors in the market. Compare to media processor, which is generally SIMD or VLIW, single scalar DSP is cheaper and has smaller area while its performance for video processing is limited. In this paper, a method to promote the performance of single scalar DSP by attaching hardware accelerators is proposed. And the bottleneck for performance promotion is investigated and the upper limit of acceleration of a certain single scalar DSP for H.264/AVC decoding is presented. </p><p>Behavioral model of H.264/AVC decoder is realized in pure software during the first step. Although real-time performance cannot be achieved with pure software implementation, computational complexity of different parts is investigated and the critical path in decoding was exposed by analyzing the first design of this software solution. Then both functional acceleration and addressing acceleration were investigated and designed to achieve the performance for real-time decoding using available clock frequency within 200MHz.</p>