Video frame rate up-conversion is an important issue for multimedia systems in achieving better video quality and motion portrayal. Motion-compensated methods offer better quality interpolated frames since the interpolation is performed along the motion trajectory. In addition, computational complexity, regularity, and memory bandwidth are important for a real-time implementation. Motion-compensated frame rate up-conversion (MC-FRC) is composed of two main parts: motion estimation (ME) and motion-compensated frame interpolation (MCFI). Since ME is an essential part of MC-FRC, a new fast motion estimation (FME) algorithm capable of producing sub-sample motion vectors at low computational-complexity has been developed. Unlike existing FME algorithms, the developed algorithm considers the low complexity sub-sample accuracy in designing the search pattern for FME. The developed FME algorithm is designed in such a way that the block distortion measure (BDM) is modeled as a parametric surface in the vicinity of the integer-sample motion vector; this modeling enables low computational-complexity sub-sample motion estimation without pixel interpolation. MC-FRC needs more accurate motion trajectories for better video quality; hence, a novel true-motion estimation (TME) algorithm targeting to track the projected object motion has been developed for video processing applications, such as motion-compensated frame interpolation (MCFI), deinterlacing, and denoising. Developed TME algorithm considers not only the computational complexity and regularity but also memory bandwidth. TME is obtained by imposing implicit and explicit smoothness constraints on block matching algorithm (BMA). In addition, it employs a novel adaptive clustering algorithm to keep the low-complexity at reasonable levels yet enable exploiting more spatiotemporal neighbors. To produce better quality interpolated frames, dense motion field at the interpolation instants are obtained for both forward and backward motion vectors (MVs); then, bidirectional motion compensation using forward and backward MVs is applied by mixing both elegantly.

「動跡剪輯」是將多個不同內容的影片片段，根據影片中特定物體移動的關係，剪接成新的影片，使得產生的新影片能維持動作連貫及流暢的特性。本論文提出一套方法能夠自動找尋不同影片間相似的剪輯點，作為「動跡剪輯」的參考。此方法之重點在於建立影片的時空資訊，作為找尋剪輯點的依據。建立影片時空資訊的過程中，我們先將影片依偵測出的鏡頭轉換點分割成不同的影片片段，再將影片片段中前景物件的位置、大小與動作等資訊分離而成影片物件平面，並結合影片片段中的背景動作資訊與影片物件平面資訊，成為該影片片段之時空資訊，從而進行剪輯點之找尋與比對，擇其最佳點進行剪輯。 運用影片時空資訊於找尋影片間之剪輯點時，是以影片物件平面作為搜尋單位，此方式有助於提升結果的正確性，同時也提供了搜尋時的靈活度。 / With the rapid increasing of the multimedia applications in modern commercial and movie business, it becomes more desirable to have efficient video editing tools. However, conventional video editing requires too many manual interventions that reduce productivities as well as opportunities in better performance. In this thesis, we propose a MOtion-based Video Editing (MOVE) mechanism that can automatically select the most similar or suitable transition points from a given set of raw videos. A given video can be divided into a set of video clips using a shot detection algorithm. For each video clip, we provide an algorithm that can separate the global motions as well as the local motions using the principles of video object plane and accumulated difference. We introduce the concept of spatio-temporal information, a condensed information that associated with a video clip. We can use this information in finding a good video editing point. Since the spatio-temporal information is a concise representation of a video clip, searching in this domain will reduce the complexity of the problem and achieve better performance. We implemented our mechanism with successful experiments.

With the advance of the LCD technology, video quality is becoming increasingly important. In this thesis, we develop hardware-friendly low-complexity enhancement algorithms. Video quality enhancement methods can be classified into two main categories. Single frame methods are the first category. These methods have generally low computational complexity. Multi-frame methods combine information from more than one frame and require the motion information of objects in the scene to do so. We first concentrate on the contrast-enhancement problem by using both global (frame-wise) and local information derived from the image. We use the image histogram and present a regularization-based histogram modification method to avoid problems that are often created by histogram equalization. Next, we design a compression artifact reduction algorithm that reduces ringing artifacts, which is disturbing especially on large displays. Furthermore, to remove the blurriness in the original video we present a non-iterative diffusion-based sharpening algorithm, which enhances edges in a ringing-aware fashion. The diffusion-based technique works on gradient approximations in a neighborhood individually. This gives more freedom compared to modulating the high-pass filter output that is used to sharpen the edges. Motion estimation enables applications such as motion-compensated noise reduction, frame-rate conversion, de-interlacing, compression, and super-resolution. Motion estimation is an ill-posed problem and therefore requires the use of prior knowledge on motion of objects. Objects have inertia and are usually larger then pixels or a block of pixels in size, which creates spatio-temporal correlation. We design a method that uses temporal redundancy to improve motion-vector search by choosing bias vectors from the previous frame and adaptively penalizes deviations from the bias vectors. This increases the robustness of the motion-vector search. The spatial correlation is more reliable because temporal correlation is difficult to use when the objects move fast or accelerate in time, or have small sizes. Spatial smoothness is not valid across motion boundaries. We investigate using energy minimization for motion estimation and incorporate the spatial smoothness prior into the energy. By formulating the energy minimization iterations for each motion vector as the primal problem, we show that the dual problem is motion segmentation for that specific motion vector.

El treball desenvolupat en aquesta tesi aprofundeix i aporta solucions innovadores en el camp orientat a tractar el problema de la correspondència en imatges subaquàtiques. En aquests entorns, el que realment complica les tasques de processat és la falta de contorns ben definits per culpa d'imatges esborronades; un fet aquest que es deu fonamentalment a il·luminació deficient o a la manca d'uniformitat dels sistemes d'il·luminació artificials. Els objectius aconseguits en aquesta tesi es poden remarcar en dues grans direccions. Per millorar l'algorisme d'estimació de moviment es va proposar un nou mètode que introdueix paràmetres de textura per rebutjar falses correspondències entre parells d'imatges. Un seguit d'assaigs efectuats en imatges submarines reals han estat portats a terme per seleccionar les estratègies més adients. Amb la finalitat d'aconseguir resultats en temps real, es proposa una innovadora arquitectura VLSI per la implementació d'algunes parts de l'algorisme d'estimació de moviment amb alt cost computacional. / Underwater robotics was the motivation of this work, even though computer vision and parallel VLSI architectures played the most important role. Due to their low cost, high-rate and high-resolution, vision based systems represent a good option to provide information about a vehicle position. The apparent motion of a camera mounted on an underwater vehicle can be estimated by correlating two successive frames of an image sequence. Lack of well-defined contours, as well as non-uniform illumination makes underwater scenes much more difficult to be processed than normal images. Therefore, methods frequently used in standard image processing must be modified and adapted to these particular conditions. A method based on texture characterisation of points to reject outliers from the image correspondence problem is proposed. On the other hand, a parallel implementation was used to speed-up parts of the motion estimation algorithm which have a computationally high load. A new VLSI architecture is proposed with the aim of achieving frame-rate performance.

Esta dissertação de mestrado propõe uma hierarquia de memória para a Estimação de Movimento e de Disparidade (ME/DE) centrada nas referências da codificação, estratégia chamada de Reference-Centered Data Reuse (RCDR), com foco em redução de energia em codificadores de vídeo multivistas (MVC - Multiview Video Coding). Nos codificadores MVC, a ME/DE é responsável por praticamente 98% do consumo total de energia. Além disso, até 90% desta energia está relacionada com a memória do codificador: (a) acessos à memória externa para a busca das referências da ME/DE (45%) e (b) memória interna (cache) para manter armazenadas as amostras da área de busca e enviá-las para serem processadas pela ME/DE (45%). O principal objetivo deste trabalho é minimizar de maneira conjunta a energia consumida pelo módulo de ME/DE com relação às memórias externa e interna necessárias para a codificação MVC. A hierarquia de memória é composta por uma memória interna (a qual armazena a área de busca inteira), um controle dinâmico para a estratégia de power-gating da memória interna e um compressor de resultados parciais. Um controle de buscas foi proposto para explorar o comportamento da busca com o objetivo de atingir ainda mais reduções de energia. Além disso, este trabalho também agrega à hierarquia de memória um compressor de quadros de referência de baixa complexidade. A estratégia RCDR provê reduções de até 68% no consumo de energia quando comparada com estratégias estadoda- arte que são centradas no bloco atual da codificação. O compressor de resultados parciais é capaz de reduzir em 52% a comunicação com memória externa necessária para o armazenamento desses elementos. Quando comparada a técnicas de reuso de dados que não acessam toda área de busca, a estratégia RCDR também atinge os melhores resultados em consumo de energia, visto que acessos regulares a memórias externas DDR são energeticamente mais eficientes. O compressor de quadros de referência reduz ainda mais o número de acessos a memória externa (2,6 vezes menos acessos), aliando isso a perdas insignificantes na eficiência da codificação MVC. A memória interna requerida pela estratégia RCDR é até 74% menor do que estratégias centradas no bloco atual, como Level C. Além disso, o controle dinâmico para a técnica de power-gating provê reduções de até 82% na energia estática, o que é o melhor resultado entre os trabalho relacionados. A energia dinâmica é tratada pela técnica de união dos blocos candidatos, atingindo ganhos de mais de 65%. Considerando as reduções de consumo de energia atingidas pelas técnicas propostas neste trabalho, conclui-se que o sistema de hierarquia de memória proposto nesta dissertação atinge seu objetivo de atender às restrições impostas pela codificação MVC, no que se refere ao processamento do módulo de ME/DE. / This Master Thesis proposes a memory hierarchy for the Motion and Disparity Estimation (ME/DE) centered on the encoding references, called Reference-Centered Data Reuse (RCDR), focusing on energy reduction in the Multiview Video Coding (MVC). In the MVC encoders the ME/DE represents more than 98% of the overall energy consumption. Moreover, in the overall ME/DE energy, up to 90% is related to the memory issues, and only 10% is related to effective computation. The two items to be concerned with: (1) off-chip memory communication to fetch the reference samples (45%) and (2) on-chip memory to keep stored the search window samples and to send them to the ME/DE processing core (45%). The main goal of this work is to jointly minimize the on-chip and off-chip energy consumption in order to reduce the overall energy related to the ME/DE on MVC. The memory hierarchy is composed of an onchip video memory (which stores the entire search window), an on-chip memory gating control, and a partial results compressor. A search control unit is also proposed to exploit the search behavior to achieve further energy reduction. This work also aggregates to the memory hierarchy a low-complexity reference frame compressor. The experimental results proved that the proposed system accomplished the goal of the work of jointly minimizing the on-chip and off-chip energies. The RCDR provides off-chip energy savings of up to 68% when compared to state-of-the-art. the traditional MBcentered approach. The partial results compressor is able to reduce by 52% the off-chip memory communication to handle this RCDR penalty. When compared to techniques that do not access the entire search window, the proposed RCDR also achieve the best results in off-chip energy consumption due to the regular access pattern that allows lots of DDR burst reads (30% less off-chip energy consumption). Besides, the reference frame compressor is capable to improve by 2.6x the off-chip memory communication savings, along with negligible losses on MVC encoding performance. The on-chip video memory size required for the RCDR is up to 74% smaller than the MB-centered Level C approaches. On top of that, the power-gating control is capable to save 82% of leakage energy. The dynamic energy is treated due to the candidate merging technique, with savings of more than 65%. Due to the jointly off-chip communication and on-chip storage energy savings, the proposed memory hierarchy system is able to meet the MVC constraints for the ME/DE processing.

Esta dissertação de mestrado propõe uma hierarquia de memória para a Estimação de Movimento e de Disparidade (ME/DE) centrada nas referências da codificação, estratégia chamada de Reference-Centered Data Reuse (RCDR), com foco em redução de energia em codificadores de vídeo multivistas (MVC - Multiview Video Coding). Nos codificadores MVC, a ME/DE é responsável por praticamente 98% do consumo total de energia. Além disso, até 90% desta energia está relacionada com a memória do codificador: (a) acessos à memória externa para a busca das referências da ME/DE (45%) e (b) memória interna (cache) para manter armazenadas as amostras da área de busca e enviá-las para serem processadas pela ME/DE (45%). O principal objetivo deste trabalho é minimizar de maneira conjunta a energia consumida pelo módulo de ME/DE com relação às memórias externa e interna necessárias para a codificação MVC. A hierarquia de memória é composta por uma memória interna (a qual armazena a área de busca inteira), um controle dinâmico para a estratégia de power-gating da memória interna e um compressor de resultados parciais. Um controle de buscas foi proposto para explorar o comportamento da busca com o objetivo de atingir ainda mais reduções de energia. Além disso, este trabalho também agrega à hierarquia de memória um compressor de quadros de referência de baixa complexidade. A estratégia RCDR provê reduções de até 68% no consumo de energia quando comparada com estratégias estadoda- arte que são centradas no bloco atual da codificação. O compressor de resultados parciais é capaz de reduzir em 52% a comunicação com memória externa necessária para o armazenamento desses elementos. Quando comparada a técnicas de reuso de dados que não acessam toda área de busca, a estratégia RCDR também atinge os melhores resultados em consumo de energia, visto que acessos regulares a memórias externas DDR são energeticamente mais eficientes. O compressor de quadros de referência reduz ainda mais o número de acessos a memória externa (2,6 vezes menos acessos), aliando isso a perdas insignificantes na eficiência da codificação MVC. A memória interna requerida pela estratégia RCDR é até 74% menor do que estratégias centradas no bloco atual, como Level C. Além disso, o controle dinâmico para a técnica de power-gating provê reduções de até 82% na energia estática, o que é o melhor resultado entre os trabalho relacionados. A energia dinâmica é tratada pela técnica de união dos blocos candidatos, atingindo ganhos de mais de 65%. Considerando as reduções de consumo de energia atingidas pelas técnicas propostas neste trabalho, conclui-se que o sistema de hierarquia de memória proposto nesta dissertação atinge seu objetivo de atender às restrições impostas pela codificação MVC, no que se refere ao processamento do módulo de ME/DE. / This Master Thesis proposes a memory hierarchy for the Motion and Disparity Estimation (ME/DE) centered on the encoding references, called Reference-Centered Data Reuse (RCDR), focusing on energy reduction in the Multiview Video Coding (MVC). In the MVC encoders the ME/DE represents more than 98% of the overall energy consumption. Moreover, in the overall ME/DE energy, up to 90% is related to the memory issues, and only 10% is related to effective computation. The two items to be concerned with: (1) off-chip memory communication to fetch the reference samples (45%) and (2) on-chip memory to keep stored the search window samples and to send them to the ME/DE processing core (45%). The main goal of this work is to jointly minimize the on-chip and off-chip energy consumption in order to reduce the overall energy related to the ME/DE on MVC. The memory hierarchy is composed of an onchip video memory (which stores the entire search window), an on-chip memory gating control, and a partial results compressor. A search control unit is also proposed to exploit the search behavior to achieve further energy reduction. This work also aggregates to the memory hierarchy a low-complexity reference frame compressor. The experimental results proved that the proposed system accomplished the goal of the work of jointly minimizing the on-chip and off-chip energies. The RCDR provides off-chip energy savings of up to 68% when compared to state-of-the-art. the traditional MBcentered approach. The partial results compressor is able to reduce by 52% the off-chip memory communication to handle this RCDR penalty. When compared to techniques that do not access the entire search window, the proposed RCDR also achieve the best results in off-chip energy consumption due to the regular access pattern that allows lots of DDR burst reads (30% less off-chip energy consumption). Besides, the reference frame compressor is capable to improve by 2.6x the off-chip memory communication savings, along with negligible losses on MVC encoding performance. The on-chip video memory size required for the RCDR is up to 74% smaller than the MB-centered Level C approaches. On top of that, the power-gating control is capable to save 82% of leakage energy. The dynamic energy is treated due to the candidate merging technique, with savings of more than 65%. Due to the jointly off-chip communication and on-chip storage energy savings, the proposed memory hierarchy system is able to meet the MVC constraints for the ME/DE processing.

[pt] Os métodos de amortecimento exponencial, apesar de originalmente proposto nos anos 60, continuam em pleno uso nos dias de hoje. Neste trabalho apresentamos um método novo para previsão de séries temporais com ou sem sazonalidade utilizando as teorias de amortecimento exponencial e análise harmônica. Assume-se que a série seja composta por uma tendência secular (constante, linear ou quadrática) e seus parâmetros são atualizados seqüencialmente pelo procedimento de amortecimento direto. Já a parte sazonal é tratada separadamente através da técnica de análise harmônica, conforme sugerida por Harrison, 1964. Dessa forma, o método proposto se apresenta como uma alternativa ao método de Souza & Epprecht, (1983) ; tendo como principal vantagem a rotina de estimação inicial dos parâmetros que no método de Souza & Epprecht produz estimadores tendenciosos em alguns casos. / [en] The method of exponential smoothing, although originally propesed during the 60´s, still continues in use up to today. In this thesis we present a new forecasting method for time series / with and/or without seasonality, applying the theory of exponential smoothing and harmonic analysis. It is assume that the series is composed of secular trend (constant, linear or quadratic) and a seasonal part. The trend parameters are sequentially using direct smoothing procedure. The seasonal part of the process is treated / separately through the technic of harmonica analysis according to Harrison´s suggestion, (1964). In this way, the proposed method can be viewed as an alternative to that of Souza & Epprecht, (1983), which has, as the most important advantage, the routine of initial estimation of the parameters, which in Souza & Epprecht method produces, in some cases, biased estimators.

The recent emergence of time-of-flight cameras has opened up new possibilities in the world of computer vision. These compact sensors, capable of recording the depth of a scene in real-time, are very advantageous in many applications, such as scene or object reconstruction. This thesis first addresses the problem of fusing depth data with color images. A complete process to combine a time-of-flight camera with a color camera is described and its accuracy is evaluated. The results show that a satisfying precision is reached and that the step of calibration is very important. The second part of the work consists of applying super-resolution techniques to the time-of-flight camera in order to improve its low resolution. Different types of super-resolution algorithms exist but this thesis focuses on the combination of multiple shifted depth maps. The proposed framework is made of two steps: registration and reconstruction. Different methods for each step are tested and compared according to the improvements reached in term of level of details, sharpness and noise reduction. The results obtained show that Lucas-Kanade performs the best for the registration and that a non-uniform interpolation gives the best results in term of reconstruction. Finally, a few suggestions are made about future work and extensions for our solutions.

In this thesis we present a system to extract the eye motion from a video stream containing a human face and applying this eye motion into a virtual character. By the notation eye motion estimation, we mean the information which describes the location of the eyes in each frame of the video stream. Applying this eye motion estimation into a virtual character, we achieve that the virtual face moves the eyes in the same way than the human face, synthesizing eye motion into a virtual character. In this study, a system capable of face tracking, eye detection and extraction, and finally iris position extraction using video stream containing a human face has been developed. Once an image containing a human face is extracted from the current frame of the video stream, the detection and extraction of the eyes is applied. The detection and extraction of the eyes is based on edge detection. Then the iris center is determined applying different image preprocessing and region segmentation using edge features on the eye picture extracted. Once, we have extracted the eye motion, using MPEG-4 Facial Animation, this motion is translated into the Facial Animation arameters (FAPs). Thus we can improve the quality and quantity of Facial Animation expressions that we can synthesize into a virtual character.

Context: Video stabilization techniques have gained popularity for their permit to obtain high quality video footage even in non-optimal conditions. There have been significant works done on video stabilization by developing different algorithms. Most of the stabilization software displays the missing image areas in stabilized video. In the last few years hand-held video cameras have continued to grow in popularity, allowing everyone to easily produce personal video footage. Furthermore, with online video sharing resources being used by a rapidly increasing number of users, a large proportion of such video footage is shared with wide audiences. Sadly such videos often suffer from poor quality as frame vibration in video makes human perception not comfortable. In this research an attempt has been made to propose a robust video stabilization algorithm that stabilizes the videos effectively. Objectives: The main objective of our thesis work is to perform effective motion estimation using SIFT features to calculate the inter frame motion, allowing to find Global Motion Vectors and optimal motion compensation is to be achieved using adaptive fuzzy filter by removing the unwanted shakiness and preserve the panning leading to stabilized video. Methods: In this study three types of research questions are used- Experimentation and Literature review. To accomplish the goal of this thesis, experimentation is carried out for performing video stabilization. Motion estimation is done using feature based motion estimation using SIFT and GMVs are calculated. The intentional motion is filtered using Adaptive fuzzy filter to preserve panning and Motion compensation is performed to wrap the video to its stabilized position. MOS implies the mean scores of the subjective tests performed according to the recommendations of ITU-R BT.500-13 and ITU-T P.910 to analyze the results of our stabilized videos. Results: As a part of results from our work, we have successfully stabilized the videos of different resolutions from experimentation. Performance of our algorithm is found better using MOS. Conclusions: Video Stabilization can be achieved successfully by using SIFT features with pre conditions defined for feature matching and attempts are made to improve the video stabilization process.