All multimedia devices now incorporate video CODECs that comply with international video coding standards such as H.264 / MPEG4-AVC and the new High Efficiency Video Coding Standard (HEVC) otherwise known as H.265. Although the standard CODECs have been designed to include algorithms with optimal efficiency, large number of coding parameters can be used to fine tune their operation, within known constraints of for e.g., available computational power, bandwidth, consumer QoS requirements, etc. With large number of such parameters involved, determining which parameters will play a significant role in providing optimal quality of service within given constraints is a further challenge that needs to be met. Further how to select the values of the significant parameters so that the CODEC performs optimally under the given constraints is a further important question to be answered. This thesis proposes a framework that uses machine learning algorithms to model the performance of a video CODEC based on the significant coding parameters. Means of modelling both the Encoder and Decoder performance is proposed. We define objective functions that can be used to model the performance related properties of a CODEC, i.e., video quality, bit-rate and CPU time. We show that these objective functions can be practically utilised in video Encoder/Decoder designs, in particular in their performance optimisation within given operational and practical constraints. A Multi-objective Optimisation framework based on Genetic Algorithms is thus proposed to optimise the performance of a video codec. The framework is designed to jointly minimize the CPU Time, Bit-rate and to maximize the quality of the compressed video stream. The thesis presents the use of this framework in the performance modelling and multi-objective optimisation of the most widely used video coding standard in practice at present, H.264 and the latest video coding standard, H.265/HEVC. When a communication network is used to transmit video, performance related parameters of the communication channel will impact the end-to-end performance of the video CODEC. Network delays and packet loss will impact the quality of the video that is received at the decoder via the communication channel, i.e., even if a video CODEC is optimally configured network conditions will make the experience sub-optimal. Given the above the thesis proposes a design, integration and testing of a novel approach to simulating a wired network and the use of UDP protocol for the transmission of video data. This network is subsequently used to simulate the impact of packet loss and network delays on optimally coded video based on the framework previously proposed for the modelling and optimisation of video CODECs. The quality of received video under different levels of packet loss and network delay is simulated, concluding the impact on transmitted video based on their content and features.

Em dezembro de 2007, a partir da cidade de São Paulo, as transmissões de televisão digital terrestre e aberta tiveram início no Brasil. Um avanço significativo do Sistema Brasileiro de TV Digital (SBTVD) foi a adoção do padrão H.264/AVC e o formato de vídeo 1080i para a codificação de vídeo em alta definição. A adoção em larga escala de tecnologia de alta definição tem sido um processo observado em vários mercados do mundo, e novos formatos superiores ao 1080i já estão sendo discutidos e propostos. Tendo em vista o que será a próxima geração da televisão, centros de pesquisa, como o centro japonês da emissora NHK, investigam os fatores humanos determinantes para caracterizar o sistema que deverá ser o último passo em tecnologia de televisão 2D. Já nomeado de UHDTV, este sistema deve contemplar resolução de 7680 pontos horizontais por 4320 pontos verticais, além de outras características ainda em estudo. Ao mesmo tempo, o trabalho aqui apresentado discute as ferramentas de suporte da escalabilidade na codificação multimídia como forma de evolução gradual dos formatos de vídeo na radiodifusão. Especificamente este trabalho sistematiza as ferramentas de escalabilidade do padrão H.264/AVC tendo em vista a sua aplicação ao SBTVD. Neste sentido, são discutidas as possibilidades de evolução do sistema frente à escalabilidade e são apresentados levantamentos experimentais da atual ocupação do espectro na cidade de São Paulo, evidenciando a disponibilidade de taxa para expansões. São apresentados também resultados iniciais relativos à codificação SVC, que apontam objetivamente as vantagens da escalabilidade sobre o simulcast, evidenciando que esta técnica pode ser utilizada no SBTVD para prover novos formatos de vídeo, tendo como premissa a compatibilidade com os atuais receptores que suportam o formato 1080i. O trabalho apresenta contribuições teóricas e experimentais na direção de adoção da escalabilidade no SBTVD, apontando também possíveis trabalhos futuros que, se realizados, poderão confirmar a transmissão de formatos superiores de vídeo nos próximos anos no SBTVD. / On December 2007, starting from São Paulo city, the open digital terrestrial transmissions were launched in Brazil. A significant improvement of the Brazilian Digital TV System (SBTVD) was the adoption of the H.264/AVC standard supporting the 1080i video format for the high definition video coding. Wide adoption of high definition technology has been a process that can be observed in lots of countries, and new video formats, beyond 1080i have already been discussed and proposed. With both eyes in the next generation of TV, research centers like Japanese broadcaster NHK investigate human factors that should drive the system specifications of this one that may be the last step in terms of 2D television technology. Named UHDTV, the system may support 7680 horizontal dots per 4320 vertical dots in terms of resolution among other features. At the same time, the work exposed here discusses tools that support multimedia coding scalability as a way of gradually improving video formats for broadcast. This work specially deals with the H.264/AVC standard scalability tools, aiming their use within SBTVD. Therefore, the evolution of the system is discussed based on scalability and experimental results related to the digital TV spectral occupation in São Paulo city are analyzed, showing that there is enough exceeding bit rate available for future expansion. Initial results related to SVC coding are also shown, objectively indicating that video scalability is more advantageous than simulcast and that this technique can be used in SBTVD to provide new video formats, keeping compatibility with current receivers that only support 1080i format. This work presents theoretical and experimental contributions towards the adoption of SVC in the SBTVD, pointing out some future works that, if executed, could confirm the transmission of new video format in SBTVD in the next years.

<p>To provide universal multimedia experience, multimedia streaming services need to transparently handle the variation and heterogeneity in operating environment. From the standpoint of streaming application, video adaptation techniques are intended to cope with the environmental variations by utilizing manipulations of the video content itself. Scalable video coding (SVC) schemes, like that suggested by the standards H.264 and its SVC extension, is highly attractive for designing a self-adaptive video streaming system. When SVC is employed in streaming system, the produced video stream can be then easily truncated or tailored to form several sub-streams which can be decoded separately to obtain a range of preferable picture size, quality and frame rate. However, questions about how to perform the adaptation using SVC and how much adaptation SVC enables are still remaining research issues. We still lack a thorough understanding of how to automate the scaling procedure in order to achieve an optimal video Quality-of-Experience for end users.</p><p>Video QoE, depends highly on human perception. In this thesis, we introduce several video QoE studies around the usability of H.264 SVC. Several factors that contribute significantly to the overall QoEs have been identified and evaluated in these studies. As an example of application usage related factor, playback smoothness and application response time are critical performance measures which can benefit from temporal scalability. Targeting on applications that requires frequent interactivity, we propose a transcoding scheme that fully utilizes the benefits of Switching P and Switching I frames specified in H.264 to enhance video stream's temporal scalability.  Focusing on visual quality related factors, a series of carefully designed subjective quality assessment tests have been performed on mobile devices to investigate the effects of multi-dimensional scalability on human quality perception. Our study reveals that QoE degrades non-monotonically with bitrate and that scaling order preferences are content-dependent. Another study find out that the flickering effect caused by frequent switching between layers in SVC compliant bit-streams is highly related to the switching period. When the period is above a certain threshold, the flickering effect will disappear and layer switching should not be considered as harmful. We have also examined user perceived video quality in 3D virtual worlds. Our results show that the avatars' distance to the virtual screen in 3D worlds contribute significant to the video QoE, i.e., for a wide extent of distortion, there exists always a feasible virtual distance from where the distortion is not detectable for most of people, which makes sense to perform video adaptation.</p><p>The work presented in this thesis is supposed to help improving the design of self adaptive video streaming services that can deliver video content independently of network technology and end-device capability while seeking the best possible experience for video.</p> / Ardendo småföretagsdoktorand

<p>In video transmission at low bit rates the challenge is to compress the video with a minimal reduction of the percieved quality. The compression can be adapted to knowledge of which regions in the video sequence are of most interest to the viewer. Region of interest (ROI) video coding uses this information to control the allocation of bits to the background and the ROI. The aim is to increase the quality in the ROI at the expense of the quality in the background. In order for this to occur the typical content of an ROI for a particular application is firstly determined and the actual detection is performed based on this information. The allocation of bits can then be controlled based on the result of the detection.</p><p>In this licenciate thesis existing methods to control bit allocation in ROI video coding are investigated. In particular pre-processing methods that are applied independently of the codec or standard. This makes it possible to apply the method directly to the video sequence without modifications to the codec. Three filters are proposed in this thesis based on previous approaches. The spatial filter that only modifies the background within a single frame and the temporal filter that uses information from the previous frame. These two filters are also combined into a spatio-temporal filter. The abilities of these filters to reduce the number of bits necessary to encode the background and to successfully re-allocate these to the ROI are investigated. In addition the computational compexities of the algorithms are analysed.</p><p>The theoretical analysis is verified by quantitative tests. These include measuring the quality using both the PSNR of the ROI and the border of the background, as well as subjective tests with human test subjects and an analysis of motion vector statistics.</p><p>The qualitative analysis shows that the spatio-temporal filter has a better coding efficiency than the other filters and it successfully re-allocates the bits from the foreground to the background. The spatio-temporal filter gives an improvement in average PSNR in the ROI of more than 1.32 dB or a reduction in bitrate of 31 % compared to the encoding of the original sequence. This result is similar to or slightly better than the spatial filter. However, the spatio-temporal filter has a better performance, since its computational complexity is lower than that of the spatial filter.</p>

<p>Recent algorithms for video coding achieve a high-quality transmission at moderate bit rates. On the other hand, those coders are very sensitive to transmission errors. Many research projects focus on methods to conceal such errors in the decoded video sequence. </p><p>Motion compensated prediction is commonly used in video coding to achieve a high compression ratio. This thesis proposes an algorithm which uses the motion compensated prediction of a given video coder to predict a sequence of several complete frames, based on the last correctly decoded images, during a transmission interruption. The proposed algorithm is evaluated on a video coder which uses a dense motion field for motion compensation. </p><p>A drawback of predicting lost fields is the perceived discontinuity when the decoder switches back from the prediction to a normal mode of operation. Various approaches to reduce this discontinuity are investigated.</p>

<p>In this Master Thesis, a model of a video coding system that uses the transportation plan taken from the calculation of the Kantorovich distance is developed. The coder uses the transportation plan instead of the differential image and sends it through blocks of transformation, quantization and coding. </p><p>The Kantorovich distance is a rather unknown distance metric that is used in optimization theory but is also applicable on images. It can be defined as the cheapest way to transport the mass of one image into another and the cost is determined by the distance function chosen to measure distance between pixels. The transportation plan is a set of finitely many five-dimensional vectors that show exactly how the mass should be moved from the transmitting pixel to the receiving pixel in order to achieve the Kantorovich distance between the images. A vector in the transportation plan is called an arc. </p><p>The original transportation plan was transformed into a new set of four-dimensional vectors called the modified difference plan. This set replaces the transmitting pixel and the receiving pixel with the distance from the transmitting pixel of the last arc and the relative distance between the receiving pixel and the transmitting pixel. The arcs where the receiving pixels are the same as the transmitting pixels are redundant and were removed. The coder completed an eleven frame sequence of size 128x128 pixels in eight to ten hours.</p>

Fourth Generation (4G) cellular technology Third Generation Partnership Project (3GPP) Long Term Evolution (LTE) offers high data rate capabilities to mobile users; and, operators are trying to deliver a true mobile broadband experience over LTE networks. Mobile TV and Video on Demand (VoD) are expected to be the main revenue generators in the near future [36] and efficient video streaming over wireless is the key to enabling this. 3GPP recommends the use of H.264 baseline profiles for all video based services in Third Generation (3G) Universal Mobile Telecommunication System (UMTS) networks. However, LTE networks need to support mobile devices with different display resolution requirements like small resolution mobile phones and high resolution laptops. Scalable Video Coding (SVC) is required to achieve this goal. Feasibility study of SVC for LTE is one of the main agenda of 3GPP Release10. SVC enhances H.264 with a set of new profiles and encoding tools that may be used to produce scalable bit streams. Efficient adaptation methods for SVC video transmission over LTE networks are proposed in this thesis. Advantages of SVC over H.264 are analyzed using real time use cases of mobile video streaming. Further, we study the cross layer adaptation and scheduling schemes for delivering SVC video streams most efficiently to the users in LTE networks in unicast and multicast transmissions. We propose SVC based video streaming scheme for unicast and multicast transmissions in the downlink direction, with dynamic adaptations and a scheduling scheme based on channel quality information from users. Simulation results indicate improved video quality for more number of users in the coverage area and efficient spectrum usage with the proposed methods.

The protection of digital images and video from unauthorized access is important for a number of applications, including privacy protection in video surveillance and digital rights management for consumer applications. However, traditional cryptographic methods are not well suited to digital visual content. Applying standard encryption approaches to the entire content can require significant computational resources due to the large size of the data. Furthermore, digital images and video often need to be manipulated,such as by resizing or transcoding, which traditional encryption would hinder. A number of image and video-specific encryption approaches have been proposed in the literature, but many of the them have significant negative impact on the ability to compress the data, which is a necessary requirement of most imaging systems. In this work, a secure image coder, called Secure Set Partitioning in Hierarchical Trees (SecSPIHT), is proposed. It combines wavelet-based image coding (compression) with efficient encryption. The encryption is applied to a small number of selected bits in the code domain, to achieve complete confidentiality of all the content while having no negative impact on compression performance. The output of the system is a secure code that cannot be decrypted and decoded without the provision of a secret key. It has superior rate-distortion performance compared to JPEG and JPEG2000, and the bit-rate can be easily scaled via a simple truncation operation. The computational overhead of the encryption operation is very low, typically requiring less than 1% of the coded image data to be encrypted. A related secure object-based coding approach is also presented. Called Secure Shape and Texture Set Partitioning in Hierarchical Trees (SecST-SPIHT), it codes and encrypts arbitrarily-shaped visual objects. A privacy protection system for video surveillance is proposed, using SecST-SPIHT to protect private data, such as face and body images appearing in surveillance footage. During normal operation of the system, the private data objects are protected via SecST-SPIHT. If an incident occurs that requires access to the data (e.g., for investigation), a designated authority must release the key. This is superior to other methods of privacy protection which irreversibly blur or mask the private data.

Multimedia applications over the Internet and Internet Protocol Television (IPTV) gain a lot of attention. IPTV has a number of service requirements such as / high bandwidth, scalability, minimum delay, jitter and channel switch time. IP multicast, IMS (IP Multimedia System) Protocol and peer-to-peer approaches are proposed for implementing IPTV. However, IP multicast requires all the routers in the core network to possess multicast capability, IMS does not easily scale and P2P cannot eciently utilize the network resources because of its completely distributed nature. To this end, we propose new application layer multicast protocol Cluster Based Application Layer Multicast IPTV (CALMTV) which combines application layer multicast, scalable video coding and probing techniques to meet IPTV requirements. We present the components and their relevant algorithms and evaluate the performance of CALMTV with ns2 simulations. Our results compared with the published results of other IPTV architectures show that CALMTV has better performance in end-to-end delay and zapping time.

The protection of digital images and video from unauthorized access is important for a number of applications, including privacy protection in video surveillance and digital rights management for consumer applications. However, traditional cryptographic methods are not well suited to digital visual content. Applying standard encryption approaches to the entire content can require significant computational resources due to the large size of the data. Furthermore, digital images and video often need to be manipulated,such as by resizing or transcoding, which traditional encryption would hinder. A number of image and video-specific encryption approaches have been proposed in the literature, but many of the them have significant negative impact on the ability to compress the data, which is a necessary requirement of most imaging systems. In this work, a secure image coder, called Secure Set Partitioning in Hierarchical Trees (SecSPIHT), is proposed. It combines wavelet-based image coding (compression) with efficient encryption. The encryption is applied to a small number of selected bits in the code domain, to achieve complete confidentiality of all the content while having no negative impact on compression performance. The output of the system is a secure code that cannot be decrypted and decoded without the provision of a secret key. It has superior rate-distortion performance compared to JPEG and JPEG2000, and the bit-rate can be easily scaled via a simple truncation operation. The computational overhead of the encryption operation is very low, typically requiring less than 1% of the coded image data to be encrypted. A related secure object-based coding approach is also presented. Called Secure Shape and Texture Set Partitioning in Hierarchical Trees (SecST-SPIHT), it codes and encrypts arbitrarily-shaped visual objects. A privacy protection system for video surveillance is proposed, using SecST-SPIHT to protect private data, such as face and body images appearing in surveillance footage. During normal operation of the system, the private data objects are protected via SecST-SPIHT. If an incident occurs that requires access to the data (e.g., for investigation), a designated authority must release the key. This is superior to other methods of privacy protection which irreversibly blur or mask the private data.