Video applications have become more and more common in the past few decades, in the meantime, optimizing video coding has received more attention. Existing video codecs usually focus on the encoder itself, and try to do everything possible to compress video with spatial (intraframe) compression and temporal (interframe) compression with the premise of reasonable distortion rate and video performance. In this work, we proposed a practical approach to improve video coding efficiency at a lower bitrate, which is to combine traditional Video Codec with interpolation neural network. A new concept called ``virtual frames'' was proposed and applied to the video coding process. We use raw frames as Ground Truth and virtual frames to train the interpolation neural network GDCN (Generalized Deformable Convolution Network), then encode the video synthesized with virtual frames via traditional AV1 video codec. With the pre-trained network, we could simply reconstruct the frames. This method can significantly improve the video compression effect compared with traditional video codec technology. / Thesis / Master of Applied Science (MASc)

No description available.

No description available.

No description available.

Consumption and production of video signals drastically changed in recent years. Due to the advances in digital consumer technology and the growing availability of fast and reliable internet connections, an increasing amount of digital video sequences are being produced, stored and shared every day in different parts of the world. Video signals are inherently larger in size than other types of multimedia signals. For this reason in order to allow transmission and storage of such data, more efficient compression technology is needed. In this thesis novel methods for enhancing the efficiency of current and next generation video codecs are investigated. Several aspects of interest to video coding technology are taken into account, from computational complexity and compliance to standardisation efforts, to compression efficiency and quality of the decoded signals. Compression can be achieved exploiting redundancies by computing a prediction of a part of the signal using previously encoded portions of the signal. Novel prediction methods are proposed in this thesis based on analytical or statistical models with the aim of providing a solid theoretical basis to support the algorithmic implementation. It is shown in the thesis that appropriately defined synthetic content can be introduced in the signal to compensate for the lack of certain characteristics in the original content. Some of the methods proposed in this thesis aim to target a broader set of use cases than those typically addressed by conventional video coding methods, such as ultra high definition content or coding under high quality conditions.

Advances in digital camera technology have led to the development of image sensors that are capable of capturing High Dynamic Range (HDR) images. Although this has enabled the capture of greater depths of colour and illumination, there remain problems with regards to transmitting and displaying the HDR image data. Current consumer level displays are designed to only show images with a depth of 8-bits per pixel per channel. Typical HDR images can be 10-bits per pixel per channel and upwards, leading to the first problem, how to display HDR images on Standard Dynamic Range (SDR) displays. This is linked to a further problem, that of transmitting the HDR data to the SDR devices, due to the fact that most state-of-the-art image and video coding standards deal with only SDR data. Further, as with most technologies of this kind, current HDR displays are extremely expensive. Furthermore, media broadcast organisations have invested significant sums of money into their current architecture and are unwilling to completely change their systems at further cost.

No description available.

Video coding systems based on 3D wavelet transforms offer several advantages over traditional hybrid video coders. This thesis proposes two 3D wavelet-based video-coding approaches. In the first approach, motion compensation with redundant-wavelet multihypothesis, in which multiple predictions that are diverse in transform phase contribute to a single motion estimate, is deployed into the fully scalable MC-EZBC video coder. The bidirectional motion-compensated temporaliltering process of MC-EZBC is adapted to the redundant-wavelet domain, wherein transform redundancy is exploited to generate a phase-diverse multihypothesis prediction of the true temporal filtering. In the second approach, a video coder is proposed that does not perform motion compensation explicitly, instead relying on the motion-selective characteristics of the 3D dual-tree discrete wavelet transform to isolate moving features. The transform coefficients are coded with binary set-partitioning using k-d trees in an algorithm that exploits within-subband spatiotemporal coherency as well as cross-subband correlation to achieve efficient coding.

Multiple Description Coding (MDC) was fi rst formulated by A. Gersho and H. Witsenhausen as a way to improve the robustness of telephony links to outages. Lots of studies have been done in this area up to now. Another application of MDC is the transmission of an image in diff erent descriptions. If because of the link outage during transmission, any one of the descriptions fails, the image could still be reconstructed with some quality at the decoder side. In video coding, inter prediction is a way to reduce temporal redundancy. From an information theoretical point of view, one can model inter prediction with Causal Video Coding (CVC). If because of link outage, we lose any I-frame, how can we reconstruct the corresponding P- or B-frames at the decoder? In this thesis, we are interested in answering this question and we call this scenario as causal video coding with possible loss of the fi rst encoded frame and we denote it by CVC-PL as PL stands for possible loss. In this thesis for the fi rst time, CVC-PL is investigated. Although, due to lack of time, we mostly study two-frame CVC-PL, we extend the problem to M-frame CVC-PL as well. To provide more insight into two-frame CVC-PL, we derive an outer-bound to the achievable rate-distortion sets to show that CVC-PL is a subset of the region combining CVC and peer-to-peer coding. In addition, we propose and prove a new achievable region to highlight the fact that two-frame CVC-PL could be viewed as MDC followed by CVC. Afterwards, we present the main theorem of this thesis, which is the minimum total rate of CVC-PL with two jointly Gaussian distributed sources, i.e. X1 and X2 with normalized correlation coeffi cient r, for di fferent distortion pro files (D1,D2,D3). Defi ning Dr = r^2(D1 -1) + 1, we show that for small D3, i.e. D3 < Dr +D2 -1, CVC-PL could be treated as CVC with two jointly Gaussian distributed sources; for large D3, i.e. D3 > DrD2/(Dr+D2-DrD2), CVC-PL could be treated as two parallel peer-to-peer networks with distortion constraints D1 and D2; and for the other cases of D3, the minimum total rate is 0.5 log (1+ ??)(D3+??)/ (Dr+?? )(D2+?? ) + 0.5 log Dr/(D1D3) where ??=D3-DrD2+r[(1-D1)(1-D2)(D3-Dr)(D3-D2)]^0.5/[Dr+D2-(D3+1) ] We also determine the optimal coding scheme which achieves the minimum total rate. We conclude the thesis by comparing the scenario of CVC-PL with two frames with a coding scheme, in which both of the sources are available at the encoders, i.e. distributed source coding versus centralized source coding. We show that for small D2 or large D3, the distributed source coding can perform as good as the centralized source coding. Finally, we talk about future work and extend and formulate the problem for M sources.

The primary objective of this thesis is to provide a low-complexity rate-distortion optimal coding mode selection method in digital video encoding. To achieve optimal compression efficiency in the rate-distortion framework with low computational complexity, we first propose a rate-distortion model and then apply it to the coding mode selection problem. The computational complexity of the proposed method is very low compared to overall encoder complexity because the proposed method uses simple image properties such as variance that can be obtained easily. Also, the proposed method gives significant PSNR gains over the mode selection scheme used in TM5 for MPEG-2 because the rate-distortion model considers rate constraints of each mode as well as distortion. We extend the model-based mode selection approach to motion vector selection for further improvement of the coding efficiency. In addition to our theoretical work, we present practical solutions to real-time implementation of encoder modules including our proposed mode selection method on digital signal processors. First, we investigate the features provided by most of the recent digital signal processors, for example, hierarchical memory structure and efficient data transfer between on-chip and off-chip memory, and then present practical approaches for real-time implementation of a video encoder system with efficient use of the features.