This paper presents a method for the estimation of scene structure and camera motion from a sequence of images. This approach is fundamentally new. No computation of optical flow or feature correspondences is required. The method processes image sequences of arbitrary length and exploits the redundancy for a significant reduction in error over time. No assumptions are made about camera motion or surface structure. Both quantities are fully recovered. Our method combines the "direct'' motion vision approach with the theory of recursive estimation. Each step is illustrated and evaluated with results from real images.

The packing and coding of digital video is a part of science where much innovation has taken place during the last few decades. The MPEG standards of video encoding are some of the most well-known and used video coding standards today. Since MPEG defines exact requirements for the decoder, but not the encoder, encoders can be made in many different ways and levels of complexity, as long as they produce legal MPEG streams that can be viewed on any MPEG-conformant decoder. This thesis will examine the possible performance of MPEG, in particular MPEG-1/MPEG-2 full TV resolution (720*576), for coding video at bitrates significantly lower than the 2-15 Mb/s MPEG-2 originally was designed for. For this purpose, encoding methods previously proposed by various researchers are presented. Furthermore a few new algorithms, which can be used for MPEG coding in general, but was constructed with a low-bitrate encoder in mind, were developed. Finally objective video quality benchmarks were conducted and the results evaluated.

In this thesis, a block-based motion estimation algorithm suitable for Super-Resolution (SR) image reconstruction is introduced. The motion estimation problem is formulated as an energy minimization problem that consists of both a data and regularization term. To handle cases when motion estimation fails, a block-based validity method is introduced, and is shown to outperform all other validity methods in the literature in terms of hybrid de-interlacing. By combining the validity metric into the energy minimization framework, it is shown that 1) the motion vector error is made less sensitive to block size, 2) a more uniform distribution of motion-compensated blocks results, and 3) the overall motion vector error is reduced. The final motion estimation algorithm is shown to outperform several state-of-the-art motion estimation algorithms in terms of both endpoint error and interpolation error, and is one of the fastest algorithms in the Middlebury benchmark. With the new motion estimation algorithm and validity metric, it is shown that artifacts are virtually eliminated from the POCS-based reconstruction of the high-resolution image.

Recent algorithms for video coding achieve a high-quality transmission at moderate bit rates. On the other hand, those coders are very sensitive to transmission errors. Many research projects focus on methods to conceal such errors in the decoded video sequence. Motion compensated prediction is commonly used in video coding to achieve a high compression ratio. This thesis proposes an algorithm which uses the motion compensated prediction of a given video coder to predict a sequence of several complete frames, based on the last correctly decoded images, during a transmission interruption. The proposed algorithm is evaluated on a video coder which uses a dense motion field for motion compensation. A drawback of predicting lost fields is the perceived discontinuity when the decoder switches back from the prediction to a normal mode of operation. Various approaches to reduce this discontinuity are investigated.

With low bit rate scenarios, a hybrid video coder (e.g. AVC/H.264) tends to allocate greater portion of bits for motion vectors, while saving bits on residual errors. According to this fact, a coding scheme with non-normative global motion models in combination with conventional local motion vectors is proposed, which describes the motion of a frame by the affine motion parameter sets drawn by motion segmentation of the luminance channel. The motion segmentation task is capable of adapting the number of motion objects to the video contents. 6-D affine model sets are driven by linear regression from the scalable block-based motion fields estimated by the existent MPEG encoder. In cases that the number of motion objects exceeds a certain threshold, the global affine models are disabled. Otherwise the 4 scaling factors of the affine models are compressed by a vector quantizer, designed with a unique cache memory for efficient searching and coding. The affine motion information is written in the slice header as a syntax. The global motion information is used for compensating those macroblocks whose Lagrange cost is minimized by the AFFINE mode. The rate-distortion cost is computed by a modified Lagrange equation, which takes into consideration the perceptual discrimination of human vision in different areas. Besides increasing the coding efficiency, the global affine model manifests the following two features that refine the compressed video quality. i) When the number of slices per frame is more than 1, the global affine motion model can enhance the error-resilience of the video stream, because the affine motion parameters are duplicated in the headers of different slices over the same frame. ii) The global motion model predicts a frame by warping the whole reference frame and this helps to decrease blocking artifacts in the compensation frame.

This dissertation deals with (a) fine granularity video compression technique and (b) its application to robust video transmission over wireless Internet. First, two wavelet-domain motion estimation algorithms, HMRME (Half-pixel Multi-Resolution Motion Estimation) and HSDD (Hierarchical Sum of Double Difference Metric), have been proposed to give wavelet-based FGS (Fine Granularity Scalability) video encoder with either low-complexity or high-performance features. Second, a VLSI-friendly high-performance embedded coder ABEC (Array-Based Embedded Coder) has been built to encode motion compensation residue as bitstream with fine granularity scalability. Third, the analysis of loss-rate prediction over Gilbert channel with loss-rate feedback, and several optimal FEC (Forward Error Correction) assignment schemes applicable for any real-time FGS video transmission system will be presented in this dissertation. In addition to those theoretical works mentioned above, for future study on embedded systems for wireless FGS video transmission, an initiative FPGA-based MPEG-4 video encoder has also been implemented in this work.

In this thesis, we proposed the series methods of moving object segmentation and object application. These methods are the moving object segmentation method in wavelet domain, double change detection method, global motion estimation method, and the moving object segmentation in the motion background. First, we proposed the Video Object Segmentation Method in Wavelet Domain. We use the Change Detection Method with the different thresholds in four wavelet sub-bands. The experiment results show that we obtain further object shape information and more accurately extracting the moving object. In the double change detection method, we proposed the method for moving object segmentation using three successive frames. We use change detection method twice in wavelet domain. After applying the Intersect Operation, we obtain the accurately moving object edge map and further object shape information. Besides, we proposed the global motion estimation method in motion scene. We propose a novel global motion estimation using cross point for the reconstruction of background scene in video sequences. Due to the robust character and limit number of cross points, we can get the Affine parameters of global motion in video sequences efficiency. At last, we proposed the object segmentation method in motion scene. We use the motion estimation method to estimate the global motion between the consecutive frames. We reconstruct a wide scene background without moving objects by the consecutive frames. At last, the moving objects will be segmented easily by comparing the object frame and the relative part in wide scene background. The Results of our proposed have good performance in the different type of video sequences. Hence, the methods of our thesis contribute to the video coding in Mpeg-4 and multimedia technology.

Αντικείμενο αυτής της διπλωματικής εργασίας είναι η εύρεση της θέσης ενός κινούμενου ρομπότ. Για το σκοπό αυτό, χρησιμοποιείται ένα στερεοσκοπικό σύστημα όμοιων καμερών (web-cameras) υπό γωνία μεταξύ τους. Κάθε μια εκ των 2 καμερών λαμβάνει μια ακολουθία φωτογραφιών σε κοινό χρόνο. Η εύρεση της θέσης γίνεται με τη μέθοδο της τριγωνοποίησης (triangulation), η οποία δίνει τις συντεταγμένες του κινούμενου αντικειμένου μέσω ομοίων τριγώνων που σχηματίζονται από τα 2 επίπεδα των φωτογραφιών (image planes), την ευθεία που ενώνει τα κέντρα των 2 καμερών καθώς και από το ίδιο το κινούμενο αντικείμενο. Ωστόσο, η γεωμετρική αυτή μέθοδος εφαρμόζεται σε φωτογραφίες που έχουν ληφθεί με παράλληλες κάμερες. Για τον λόγο αυτό, εφαρμόζεται η μέθοδος της διόρθωσης εικόνας (image rectification) η οποία μετασχηματίζει τις εικόνες έτσι ώστε να είναι σα να έχουν ληφθεί από παράλληλες κάμερες. Στη συνέχεια εντοπίζεται το κινούμενο αντικείμενο με μορφολογική επεξεργασία εικόνας, υπολογίζεται το κέντρο βάρους του και γίνεται η τριγωνοποίηση γεωμετρικά. Στην προαναφερθείσα διαδικασία, θεωρούνται γνωστά τα εσωτερικά στοιχεία των καμερών και η μεταξύ τους απόσταση. Επιπλέον, θεωρείται ότι όλος ο όγκος του ρομπότ αναπαρίσταται απο ένα σημείο P με συντεταγμένες (X,Y,Z) ως προς σύστημα συντεταγμένων με αρχή το κέντρο προβολής της αριστερής κάμερας. / The objective of this thesis is the position estimation of a moving robot. For this purpose, a stereoscopic system of two identical, non-parallel cameras has been used (web-cameras). Each of the two cameras takes a photo synchronized sequence. The position of the robot has been estimated with the method of triangulation, which computes the coordinates of the moving object via similar triangles formed by the two image planes, the straight line joining the centres of the cameras and by the moving object. However, triangulation is applicable to images acquired by parallel cameras. For this reason, the images have been rectified. Image rectification is a transformation process that is used to project two-or-more images onto a common image. Then, the moving object is detected with morphological image processing techniques, its centroid is calculated and finally triangulation has been applied. The intrinsic parameters of the cameras and the distance between them are known. Furthermore, we consider that the entire volume of the robot is represented by a point P with coordinates (X, Y, Z) with respect to the left camera coordinate system.

With the rapid growth of video services on smartphones such as video conferencing, video telephone and WebTV, implementation of video compression on mobile terminal becomes extremely important. However, the low computation capability of mobile devices becomes a bottleneck which calls for low complexity techniques for video coding. This work presents two set of algorithms for reducing the complexity of motion estimation. Binary motion estimation techniques using one-bit and two-bit transforms reduce the computational complexity of matching error criterion, however sometimes generate inaccurate motion vectors. The first set includes two neighborhood matching based algorithms which attempt to reduce computations to only a fraction of other methods. Simulation results demonstrate that full search local binary pattern (FS-LBP) algorithm reconstruct visually more accurate frames compared to full search algorithm (FSA). Its reduced complexity LBP (RC-LBP) version decreases computations significantly to only a fraction of the other methods while maintaining acceptable performance. The second set introduces edge detection approach for partial distortion elimination based on binary patterns. Spiral partial distortion elimination (SpiralPDE) has been proposed in literature which matches the pixel-to-pixel distortion in a predefined manner. Since, the contribution of all the pixels to the distortion function is different, therefore, it is important to analyze and extract these cardinal pixels. The proposed algorithms are called lossless fast full search partial distortion elimination ME based on local binary patterns (PLBP) and lossy edge-detection pixel decimation technique based on local binary patterns (ELBP). PLBP reduces the matching complexity by matching more contributable pixels early by identifying the most diverse pixels in a local neighborhood. ELBP captures the most representative pixels in a block in order of contribution to the distortion function by evaluating whether the individual pixels belong to the edge or background. Experimental results demonstrate substantial reduction in computational complexity of ELBP with only a marginal loss in prediction quality.

With the rapid growth of video services on smartphones such as video conferencing, video telephone and WebTV, implementation of video compression on mobile terminal becomes extremely important. However, the low computation capability of mobile devices becomes a bottleneck which calls for low complexity techniques for video coding. This work presents two set of algorithms for reducing the complexity of motion estimation. Binary motion estimation techniques using one-bit and two-bit transforms reduce the computational complexity of matching error criterion, however sometimes generate inaccurate motion vectors. The first set includes two neighborhood matching based algorithms which attempt to reduce computations to only a fraction of other methods. Simulation results demonstrate that full search local binary pattern (FS-LBP) algorithm reconstruct visually more accurate frames compared to full search algorithm (FSA). Its reduced complexity LBP (RC-LBP) version decreases computations significantly to only a fraction of the other methods while maintaining acceptable performance. The second set introduces edge detection approach for partial distortion elimination based on binary patterns. Spiral partial distortion elimination (SpiralPDE) has been proposed in literature which matches the pixel-to-pixel distortion in a predefined manner. Since, the contribution of all the pixels to the distortion function is different, therefore, it is important to analyze and extract these cardinal pixels. The proposed algorithms are called lossless fast full search partial distortion elimination ME based on local binary patterns (PLBP) and lossy edge-detection pixel decimation technique based on local binary patterns (ELBP). PLBP reduces the matching complexity by matching more contributable pixels early by identifying the most diverse pixels in a local neighborhood. ELBP captures the most representative pixels in a block in order of contribution to the distortion function by evaluating whether the individual pixels belong to the edge or background. Experimental results demonstrate substantial reduction in computational complexity of ELBP with only a marginal loss in prediction quality.