In this paper we show how the theory of dynamical systems can be employed to solve problems in motion vision. In particular we develop algorithms for the recovery of dense depth maps and motion parameters using state space observers or filters. Four different dynamical models of the imaging situation are investigated and corresponding filters/ observers derived. The most powerful of these algorithms recovers depth and motion of general nature using a brightness change constraint assumption. No feature-matching preprocessor is required.

Digital watermarking of multimedia content has been proposed as a method for different applications such as copyright protection, content authentication, transaction tracking and data hiding. In this thesis, we propose a lossless watermarking approach based on Discrete Cosine Transform (DCT) for a new application of watermarking. A depth map obtained from a stereoscopic image pair is embedded into one of the two images using a reversible watermarking algorithm. Different from existing approaches which hide depth map in spatial domain, the depth information is hidden in the quantized DCT domain of the stereo image in our method. This modification makes the watermarking algorithm compatible with JPEG and MPEG-2 compression. After the investigation of the quantized DCT coefficients distribution of the compressed image and video, The bit-shift operation is utilized to embed the depth map into its associated 2D image reversibly for the purpose of achieving high compression efficiency of the watermarked image and/or video and high visual quality of stereo image and/or video after the depth map is extracted. We implement the proposed method to analyze its performance. The experimental results show that a very high payload of watermark (e.g. depth map) can be embedded into the JPEG compressed image and MPEG-2 video. The compression efficiency is only slightly reduced after the watermark embedding and the quality of the original image or video can be restored completely at the decoder side.

Digital watermarking of multimedia content has been proposed as a method for different applications such as copyright protection, content authentication, transaction tracking and data hiding. In this thesis, we propose a lossless watermarking approach based on Discrete Cosine Transform (DCT) for a new application of watermarking. A depth map obtained from a stereoscopic image pair is embedded into one of the two images using a reversible watermarking algorithm. Different from existing approaches which hide depth map in spatial domain, the depth information is hidden in the quantized DCT domain of the stereo image in our method. This modification makes the watermarking algorithm compatible with JPEG and MPEG-2 compression. After the investigation of the quantized DCT coefficients distribution of the compressed image and video, The bit-shift operation is utilized to embed the depth map into its associated 2D image reversibly for the purpose of achieving high compression efficiency of the watermarked image and/or video and high visual quality of stereo image and/or video after the depth map is extracted. We implement the proposed method to analyze its performance. The experimental results show that a very high payload of watermark (e.g. depth map) can be embedded into the JPEG compressed image and MPEG-2 video. The compression efficiency is only slightly reduced after the watermark embedding and the quality of the original image or video can be restored completely at the decoder side.

Digital watermarking of multimedia content has been proposed as a method for different applications such as copyright protection, content authentication, transaction tracking and data hiding. In this thesis, we propose a lossless watermarking approach based on Discrete Cosine Transform (DCT) for a new application of watermarking. A depth map obtained from a stereoscopic image pair is embedded into one of the two images using a reversible watermarking algorithm. Different from existing approaches which hide depth map in spatial domain, the depth information is hidden in the quantized DCT domain of the stereo image in our method. This modification makes the watermarking algorithm compatible with JPEG and MPEG-2 compression. After the investigation of the quantized DCT coefficients distribution of the compressed image and video, The bit-shift operation is utilized to embed the depth map into its associated 2D image reversibly for the purpose of achieving high compression efficiency of the watermarked image and/or video and high visual quality of stereo image and/or video after the depth map is extracted. We implement the proposed method to analyze its performance. The experimental results show that a very high payload of watermark (e.g. depth map) can be embedded into the JPEG compressed image and MPEG-2 video. The compression efficiency is only slightly reduced after the watermark embedding and the quality of the original image or video can be restored completely at the decoder side.

Digital watermarking of multimedia content has been proposed as a method for different applications such as copyright protection, content authentication, transaction tracking and data hiding. In this thesis, we propose a lossless watermarking approach based on Discrete Cosine Transform (DCT) for a new application of watermarking. A depth map obtained from a stereoscopic image pair is embedded into one of the two images using a reversible watermarking algorithm. Different from existing approaches which hide depth map in spatial domain, the depth information is hidden in the quantized DCT domain of the stereo image in our method. This modification makes the watermarking algorithm compatible with JPEG and MPEG-2 compression. After the investigation of the quantized DCT coefficients distribution of the compressed image and video, The bit-shift operation is utilized to embed the depth map into its associated 2D image reversibly for the purpose of achieving high compression efficiency of the watermarked image and/or video and high visual quality of stereo image and/or video after the depth map is extracted. We implement the proposed method to analyze its performance. The experimental results show that a very high payload of watermark (e.g. depth map) can be embedded into the JPEG compressed image and MPEG-2 video. The compression efficiency is only slightly reduced after the watermark embedding and the quality of the original image or video can be restored completely at the decoder side.

Le codage de source distribué est un paradigme qui consiste à encoder indépendamment deux sources corrélées et à les décoder conjointement. Wyner et Ziv ont montré que le codage de source distribué peut atteindre les mêmes performances débit-distorsion que le codage de source conjoint, pourvu que certaines contraintes soient satisfaites. Cette caractéristique rend le codage de source distribué très attractif pour des applications qui demandent un encodeur à faible complexité ou pour ne pas être obligé à avoir des communications entre les sources. Dans le cadre du codage vidéo distribué, les trames corrélées sont encodées séparément et décodées conjointement. Dans l'architecture ainsi dite de Stanford, le flux vidéo est séparée en trames clés et Wyner-Ziv. Les trames clés sont encodées INTRA. Les trames Wyner-Ziv sont données en entrée à un codeur de canal systématique ; seulement les bits de parité sont envoyés. Au décodeur, on produit une estimation de la trame Wyner-Ziv, appelée information adjacente, en interpolant les trames clés reçues. L'information adjacente, considérée comme une version bruitée de la trame Wyner-Ziv, est corrigée par les bits de parité. Dans cette thèse, nous proposons plusieurs algorithmes pour la génération de l'information adjacente et pour l'interpolation temporelle et inter-vue. On propose aussi un algorithme de fusion bayésienne des deux interpolations. Tous les algorithmes proposés donnent des résultats meilleurs par rapport à l'état de l'art en termes de performance débit-distorsion. Nous proposons aussi plusieurs algorithmes pour l'estimation de la trame Wyner-Ziv dans le cadre de la vidéo multi-vues plus profondeur. / Distributed source coding is a paradigm that consists in encoding two correlated sources independently, provided that they are decoded jointly.Wyner and Ziv proved that distributed source coding can attain the same rate distortion performance of joint coding, under some constraints.This feature makes distributed source coding very attractive for applications that require a low-complexity encoder or for avoiding communication between the sources. In distributed video coding, correlated frames are encoded separately but decoded jointly. In the Stanford Architecture, the video is split into Key Frames and Wyner-Ziv Frames. The Key Frames are INTRA coded. The Wyner-Ziv Frames are fed into a systematic channel coder and only the parity bits are sent to the decoder. At the decoder side, an estimation of the Wyner-Ziv Frame, called side information, is produced by interpolating the available frames. The side information, that can be considered as a noisy version of the real Wyner-Ziv Frame, is corrected by the parity bits sent by the encoder. In this thesis, we propose several algorithms for side information generation both for the temporal and inter-view interpolation. We also propose a Bayesian fusion of the two estimations. All our algorithms outperform the state-of-the-art in terms of rate distortion performance. We also propose several algorithms for Wyner-Ziv estimation in the context of multiview video plus depth.

Today’s 3D imaging techniques offer significant benefits over conventional 2D imaging techniques. The presence of natural depth information in the scene affords the observer an overall improved sense of reality and naturalness. A variety of systems attempting to reach this goal have been designed by many independent research groups, such as stereoscopic and auto-stereoscopic systems. Though the images displayed by such systems tend to cause eye strain, fatigue and headaches after prolonged viewing as users are required to focus on the screen plane/accommodation to converge their eyes to a point in space in a different plane/convergence. Holoscopy is a 3D technology that targets overcoming the above limitations of current 3D technology and was recently developed at Brunel University. This work is part W4.1 of the 3D VIVANT project that is funded by the EU under the ICT program and coordinated by Dr. Aman Aggoun at Brunel University, West London, UK. The objective of the work described in this thesis is to develop estimation and segmentation techniques that are capable of estimating precise 3D depth, and are applicable for holoscopic 3D imaging system. Particular emphasis is given to the task of automatic techniques i.e. favours algorithms with broad generalisation abilities, as no constraints are placed on the setting. Algorithms that provide invariance to most appearance based variation of objects in the scene (e.g. viewpoint changes, deformable objects, presence of noise and changes in lighting). Moreover, have the ability to estimate depth information from both types of holoscopic 3D images i.e. Unidirectional and Omni-directional which gives horizontal parallax and full parallax (vertical and horizontal), respectively. The main aim of this research is to develop 3D depth estimation and 3D image segmentation techniques with great precision. In particular, emphasis on automation of thresholding techniques and cues identifications for development of robust algorithms. A method for depth-through-disparity feature analysis has been built based on the existing correlation between the pixels at a one micro-lens pitch which has been exploited to extract the viewpoint images (VPIs). The corresponding displacement among the VPIs has been exploited to estimate the depth information map via setting and extracting reliable sets of local features. ii Feature-based-point and feature-based-edge are two novel automatic thresholding techniques for detecting and extracting features that have been used in this approach. These techniques offer a solution to the problem of setting and extracting reliable features automatically to improve the performance of the depth estimation related to the generalizations, speed and quality. Due to the resolution limitation of the extracted VPIs, obtaining an accurate 3D depth map is challenging. Therefore, sub-pixel shift and integration is a novel interpolation technique that has been used in this approach to generate super-resolution VPIs. By shift and integration of a set of up-sampled low resolution VPIs, the new information contained in each viewpoint is exploited to obtain a super resolution VPI. This produces a high resolution perspective VPI with wide Field Of View (FOV). This means that the holoscopic 3D image system can be converted into a multi-view 3D image pixel format. Both depth accuracy and a fast execution time have been achieved that improved the 3D depth map. For a 3D object to be recognized the related foreground regions and depth information map needs to be identified. Two novel unsupervised segmentation methods that generate interactive depth maps from single viewpoint segmentation were developed. Both techniques offer new improvements over the existing methods due to their simple use and being fully automatic; therefore, producing the 3D depth interactive map without human interaction. The final contribution is a performance evaluation, to provide an equitable measurement for the extent of the success of the proposed techniques for foreground object segmentation, 3D depth interactive map creation and the generation of 2D super-resolution viewpoint techniques. The no-reference image quality assessment metrics and their correlation with the human perception of quality are used with the help of human participants in a subjective manner.

Holoscopic 3D imaging also known as “Integral imaging” was first proposed by Lippmann in 1908. It facilitates a promising technique for creating full colour spatial image that exists in space. It promotes a single lens aperture for recording spatial images of a real scene, thus it offers omnidirectional motion parallax and true 3D depth, which is the fundamental feature for digital refocusing. While stereoscopic and multiview 3D imaging systems simulate human eye technique, holoscopic 3D imaging system mimics fly’s eye technique, in which viewpoints are orthographic projection. This system enables true 3D representation of a real scene in space, thus it offers richer spatial cues compared to stereoscopic 3D and multiview 3D systems. Focus has been the greatest challenge since the beginning of photography. It is becoming even more critical in film production where focus pullers are finding it difficult to get the right focus with camera resolution becoming increasingly higher. Holoscopic 3D imaging enables the user to carry out re/focusing in post-production. There have been three main types of digital refocusing methods namely Shift and Integration, full resolution, and full resolution with blind. However, these methods suffer from artifacts and unsatisfactory resolution in the final resulting image. For instance the artifacts are in the form of blocky and blurry pictures, due to unmatched boundaries. An upsampling method is proposed that improves the resolution of the resulting image of shift and integration approach. Sub-pixel adjustment of elemental images including “upsampling technique” with smart filters are proposed to reduce the artifacts, introduced by full resolution with blind method as well as to improve both image quality and resolution of the final rendered image. A novel 3D object extraction method is proposed that takes advantage of disparity, which is also applied to generate stereoscopic 3D images from holoscopic 3D image. Cross correlation matching algorithm is used to obtain the disparity map from the disparity information and the desirable object is then extracted. In addition, 3D image conversion algorithm is proposed for the generation of stereoscopic and multiview 3D images from both unidirectional and omnidirectional holoscopic 3D images, which facilitates 3D content reformation.

This paper describes two methods for constructing a depth map from images. Each method has two stages. First, one or more needle maps are determined using a pair of images. This process employs either the Marr-Poggio-Grimson stereo and shape-from-shading, or, instead, photometric stereo. Secondly, a depth map is constructed from the needle map or needle maps computed by the first stage. Both methods make use of an iterative relaxation method to obtain the final depth map.

With the recent comeback of three-dimensional (3D) movies to the cinemas, there have been increasing efforts to spread the commercial success of 3D to new markets. The possibility of a 3D experience at home, such as three-dimensional television (3DTV), has generated a great deal of interest within the research and standardization community. A central issue for 3DTV is the creation and representation of 3D content. Scene depth information plays a crucial role in all parts of the distribution chain from content capture via transmission to the actual 3D display. This depth information is transmitted in the form of depth maps and is accompanied by corresponding video frames, i.e. for Depth Image Based Rendering (DIBR) view synthesis. Nonetheless, scenarios do exist for which the original spatial resolutions of depth maps and video frames do not match, e.g. sensor driven depth capture or asymmetric 3D video coding. This resolution discrepancy is a problem, since DIBR requires accordance between the video frame and depth map. A considerable amount of research has been conducted into ways to match low-resolution depth maps to high resolution video frames. Many proposed solutions utilize corresponding texture information in the upscaling process, however they mostly fail to review this information for validity. In the strive for better 3DTV quality, this thesis presents the Edge-Weighted Optimization Concept (EWOC), a novel texture-guided depth upscaling application that addresses the lack of information validation. EWOC uses edge information from video frames as guidance in the depth upscaling process and, additionally, confirms this information based on the original low resolution depth. Over the course of four publications, EWOC is applied in 3D content creation and distribution. Various guidance sources, such as different color spaces or texture pre-processing, are investigated. An alternative depth compression scheme, based on depth map upscaling, is proposed and extensions for increased visual quality and computational performance are presented in this thesis. EWOC was evaluated and compared with competing approaches, with the main focus was consistently on the visual quality of rendered 3D views. The results show an increase in both objective and subjective visual quality to state-of-the-art depth map upscaling methods. This quality gain motivates the choice of EWOC in applications affected by low resolution depth. In the end, EWOC can improve 3D content generation and distribution, enhancing the 3D experience to boost the commercial success of 3DTV.