<p>In this thesis, we study three different image interpolation applications in high definition (HD) video processing: video de-interlacing, frame rate up-conversion, and view interpolation. We propose novel methods for these applications which are based on the concept of Nonlocal-Means (NL-Means).</p> <p>In the first part of this thesis, we introduce a new de-interlacing method which uses NL-Means algorithm. In this method, every interpolated pixel is set to a weighted average of its neighboring pixels in the current, previous, and the next frames. Weights of the pixels used in this filtering are calculated according to the radiometric distance between the surrounding areas of the pixel being interpolated and the neighboring pixels. One of the main challenges of the NL-Means is finding a suitable size for the neighborhoods (similarity window) that we want to find radiometric distance for them. We address this problem by using a steering kernel in our distance function to adapt the effective size of similarity window to the local information of the image. In order to calculate the weights of the filter, we need to have an estimate of the progressive frames. Therefore, we introduce a low computational initial de-interlacing method. This method interpolates the missing pixel along a direction based on two criteria of having minimum variation and being used by the above or below pixels. This method preserves the edge structures and yields superior visual quality compared to the simple edge-based line-averaging and many other simple iv de-interlacing methods.</p> <p>The second part of this thesis is devoted to the frame rate up-conversion application. Our frame rate up-conversion method is based on two main steps: NL-Means and foreground /background segmentation. In this method, for every pixel being interpolated first we check whether it belongs to the background or foreground. If the pixel belongs to the background and the values of the next and previous frames’ pixels are the same, we simply set the pixel intensity to the intensity of its location in the previous or next frame. If the pixel belongs to the foreground, we use NL-Means based interpolation for it. We adjust the equations of the NL-means for frame rate up-conversion so that we do not need to have the neighborhoods of the intermediate for calculating the weights of the filter. The comparison of our method with other existing methods shows the better performance of our method.</p> <p>In the third part of this thesis, we introduce a novel view interpolation method without using disparity estimation. In this method, we let every pixel in the intermediate view be the output of the NL-means using the pixels in the reference views. The experimental results demonstrate the better quality of our results compared with other algorithms which use disparity estimation.</p> / Master of Applied Science (MASc)

Video frame rate up-conversion is an important issue for multimedia systems in achieving better video quality and motion portrayal. Motion-compensated methods offer better quality interpolated frames since the interpolation is performed along the motion trajectory. In addition, computational complexity, regularity, and memory bandwidth are important for a real-time implementation. Motion-compensated frame rate up-conversion (MC-FRC) is composed of two main parts: motion estimation (ME) and motion-compensated frame interpolation (MCFI). Since ME is an essential part of MC-FRC, a new fast motion estimation (FME) algorithm capable of producing sub-sample motion vectors at low computational-complexity has been developed. Unlike existing FME algorithms, the developed algorithm considers the low complexity sub-sample accuracy in designing the search pattern for FME. The developed FME algorithm is designed in such a way that the block distortion measure (BDM) is modeled as a parametric surface in the vicinity of the integer-sample motion vector; this modeling enables low computational-complexity sub-sample motion estimation without pixel interpolation. MC-FRC needs more accurate motion trajectories for better video quality; hence, a novel true-motion estimation (TME) algorithm targeting to track the projected object motion has been developed for video processing applications, such as motion-compensated frame interpolation (MCFI), deinterlacing, and denoising. Developed TME algorithm considers not only the computational complexity and regularity but also memory bandwidth. TME is obtained by imposing implicit and explicit smoothness constraints on block matching algorithm (BMA). In addition, it employs a novel adaptive clustering algorithm to keep the low-complexity at reasonable levels yet enable exploiting more spatiotemporal neighbors. To produce better quality interpolated frames, dense motion field at the interpolation instants are obtained for both forward and backward motion vectors (MVs); then, bidirectional motion compensation using forward and backward MVs is applied by mixing both elegantly.