<p>In this thesis we systematically reexamine the classical problem of image interpolation with an aim to better preserve the structural information, such as edges and textures, in the interpolated image. We take on the technical challenge of faithfully reconstructing high frequency components because this is critical to the perceptual quality. To achieve the above goal we develop three new adaptive image interpolation methods: 1) a classification-based method that is driven by contextual information of the low resolution image and the prior knowledge extracted from a training set of high resolution images; 2) An adaptive soft-decision block estimation method that learns and adapts to varying scene structures, guided by a two-dimension piecewise autoregressive model; 3) A model-based non-linear image restoration scheme in which the model parameters and high resolution pixels are jointly estimated through non-linear least squares estimation.</p> <p>The latter part of this thesis is devoted to the research of interpolation-based image compression, which is a relatively new topic. Our research is motivated by two important applications of visual communication: low bit-rate image coding and multiple description coding. We succeed in developing standard-compliant interpolation-based compression techniques for the above two applications. In their respective categories, these techniques exceed the best rate-distortion performance reported so far in the literature.</p> / Thesis / Doctor of Philosophy (PhD)

Este trabalho apresenta um estudo sobre métodos utilizados para realizar metamorfose entre imagens digitais. Segundo esses métodos é possível estabelecer uma imagem fonte, uma destino e, através de operações de mapeamento de texturas, realizar uma seqüência de imagens intermediárias. Essas imagens intermediárias apresentam-se como deformações sobre a imagem fonte ou como etapas de transição entre as imagens originais, efeito produzido pela inserção de um processo de fusão de imagens. O processo de deformação varia segundo o método empregado e influencia diretamente nos resultados por ele gerado. Assim, dois métodos com características diferentes para deformação de imagens foram escolhidos, dentre os encontrados, para um estudo mais aprofundado e são referenciados como o Método das Malhas e o Método dos Campos de Influência. Esses métodos são apresentados, seus funcionamentos discutidos e resultados gerados para se estabelecer uma análise entre os mesmos. A análise é efetuada segundo os aspectos de tempo de processamento, consumo de minoria e atribuição de correspondências entre as imagens, sendo que esta última característica leva em consideração o tempo total gasto para se estabelecer as correspondências necessárias entre a imagem fonte e a destino. Neste ponto também é estabelecido um quadro comparativo entre os métodos escolhidos, destacando-se vantagens e desvantagens. Os resultados da aplicação dos métodos analisados encontram-se distribuídos ao longo do texto sob a forma de imagens e tabelas, são utilizados para provar a eficiência desses métodos e foram gerados por um sistema de testes implementado. O sistema de testes desenvolvido possibilita a geração de imagens, segundo o método escolhido, informa os tempos utilizados no processamento e exibe essas imagens seqüencialmente sob a forma de uma animação. Por fim o trabalho apresenta conclusões sobre o estudo, salientando aspectos considerados relevantes quando do estudo e implementação dos métodos, e indica futuras extensões que podem ser efetuadas a fim de tornar o sistema de testes mais flexível e completo. / This work presents a study on methods used to make digital images metamorphosis. According to these methods it is possible to establish a source image, a destination image and also to generate a sequence of intermediate images through operations of texture mapping. These intermediate images appears as deformations of the source image or as transition steps between the original images which are produced by the insertion of a cross-dissolve process. The warping process varies according to the method which is used and it influences directly its results. Thus, two methods for image warping with different characteristics were chosen, among the available ones, for a more detailed study and are refered as the Mesh Method and as the Fields of Influence Method. These two methods are presented, their functioning is discussed and results are generated to allow comparative analyses. The analysis concerns aspects of processing time, memory allocation and correspondence establishment, which take into account the time waisted in order to establish the correspondences between the images. At this point it is also established a comparative table between the chosen methods highlighting advantages and disadvantages. The results of the application of the analysed methods are distributed throughout the text as images and tables, which are used to prove the efficiency of them and were generated by a test system implemented. The system developed provides the generation of images, according to the choosen method, informs the time waisted on processing and shows those images sequentially as an animation. Finally this work presents conclusions about the study and indicates future extensions that can be done in order to make the developed system more flexible and complete.

<p>The target of this master thesis is to evaluate the Image Warping technique and propose a possible design for an implementation in FPGA. The Image Warping is widely used in the image processing for image correction and rectification. A DSP is a usual choice for implantation of the image processing algorithms, but to decrease a cost of the target system it was proposed to use an FPGA for implementation.</p><p>In this work a different Image Warping methods was evaluated in terms of performance, produced image quality, complexity and design size. Also, considering that it is not only Image Warping algorithm which will be implemented on the target system, it was important to estimate a possible memory bandwidth used by the proposed design. The evaluation was done by implemented a C-model of the proposed design with a finite datapath to simulate hardware implementation as close as possible.</p>

Este trabalho apresenta um estudo sobre métodos utilizados para realizar metamorfose entre imagens digitais. Segundo esses métodos é possível estabelecer uma imagem fonte, uma destino e, através de operações de mapeamento de texturas, realizar uma seqüência de imagens intermediárias. Essas imagens intermediárias apresentam-se como deformações sobre a imagem fonte ou como etapas de transição entre as imagens originais, efeito produzido pela inserção de um processo de fusão de imagens. O processo de deformação varia segundo o método empregado e influencia diretamente nos resultados por ele gerado. Assim, dois métodos com características diferentes para deformação de imagens foram escolhidos, dentre os encontrados, para um estudo mais aprofundado e são referenciados como o Método das Malhas e o Método dos Campos de Influência. Esses métodos são apresentados, seus funcionamentos discutidos e resultados gerados para se estabelecer uma análise entre os mesmos. A análise é efetuada segundo os aspectos de tempo de processamento, consumo de minoria e atribuição de correspondências entre as imagens, sendo que esta última característica leva em consideração o tempo total gasto para se estabelecer as correspondências necessárias entre a imagem fonte e a destino. Neste ponto também é estabelecido um quadro comparativo entre os métodos escolhidos, destacando-se vantagens e desvantagens. Os resultados da aplicação dos métodos analisados encontram-se distribuídos ao longo do texto sob a forma de imagens e tabelas, são utilizados para provar a eficiência desses métodos e foram gerados por um sistema de testes implementado. O sistema de testes desenvolvido possibilita a geração de imagens, segundo o método escolhido, informa os tempos utilizados no processamento e exibe essas imagens seqüencialmente sob a forma de uma animação. Por fim o trabalho apresenta conclusões sobre o estudo, salientando aspectos considerados relevantes quando do estudo e implementação dos métodos, e indica futuras extensões que podem ser efetuadas a fim de tornar o sistema de testes mais flexível e completo. / This work presents a study on methods used to make digital images metamorphosis. According to these methods it is possible to establish a source image, a destination image and also to generate a sequence of intermediate images through operations of texture mapping. These intermediate images appears as deformations of the source image or as transition steps between the original images which are produced by the insertion of a cross-dissolve process. The warping process varies according to the method which is used and it influences directly its results. Thus, two methods for image warping with different characteristics were chosen, among the available ones, for a more detailed study and are refered as the Mesh Method and as the Fields of Influence Method. These two methods are presented, their functioning is discussed and results are generated to allow comparative analyses. The analysis concerns aspects of processing time, memory allocation and correspondence establishment, which take into account the time waisted in order to establish the correspondences between the images. At this point it is also established a comparative table between the chosen methods highlighting advantages and disadvantages. The results of the application of the analysed methods are distributed throughout the text as images and tables, which are used to prove the efficiency of them and were generated by a test system implemented. The system developed provides the generation of images, according to the choosen method, informs the time waisted on processing and shows those images sequentially as an animation. Finally this work presents conclusions about the study and indicates future extensions that can be done in order to make the developed system more flexible and complete.

Este trabalho apresenta um estudo sobre métodos utilizados para realizar metamorfose entre imagens digitais. Segundo esses métodos é possível estabelecer uma imagem fonte, uma destino e, através de operações de mapeamento de texturas, realizar uma seqüência de imagens intermediárias. Essas imagens intermediárias apresentam-se como deformações sobre a imagem fonte ou como etapas de transição entre as imagens originais, efeito produzido pela inserção de um processo de fusão de imagens. O processo de deformação varia segundo o método empregado e influencia diretamente nos resultados por ele gerado. Assim, dois métodos com características diferentes para deformação de imagens foram escolhidos, dentre os encontrados, para um estudo mais aprofundado e são referenciados como o Método das Malhas e o Método dos Campos de Influência. Esses métodos são apresentados, seus funcionamentos discutidos e resultados gerados para se estabelecer uma análise entre os mesmos. A análise é efetuada segundo os aspectos de tempo de processamento, consumo de minoria e atribuição de correspondências entre as imagens, sendo que esta última característica leva em consideração o tempo total gasto para se estabelecer as correspondências necessárias entre a imagem fonte e a destino. Neste ponto também é estabelecido um quadro comparativo entre os métodos escolhidos, destacando-se vantagens e desvantagens. Os resultados da aplicação dos métodos analisados encontram-se distribuídos ao longo do texto sob a forma de imagens e tabelas, são utilizados para provar a eficiência desses métodos e foram gerados por um sistema de testes implementado. O sistema de testes desenvolvido possibilita a geração de imagens, segundo o método escolhido, informa os tempos utilizados no processamento e exibe essas imagens seqüencialmente sob a forma de uma animação. Por fim o trabalho apresenta conclusões sobre o estudo, salientando aspectos considerados relevantes quando do estudo e implementação dos métodos, e indica futuras extensões que podem ser efetuadas a fim de tornar o sistema de testes mais flexível e completo. / This work presents a study on methods used to make digital images metamorphosis. According to these methods it is possible to establish a source image, a destination image and also to generate a sequence of intermediate images through operations of texture mapping. These intermediate images appears as deformations of the source image or as transition steps between the original images which are produced by the insertion of a cross-dissolve process. The warping process varies according to the method which is used and it influences directly its results. Thus, two methods for image warping with different characteristics were chosen, among the available ones, for a more detailed study and are refered as the Mesh Method and as the Fields of Influence Method. These two methods are presented, their functioning is discussed and results are generated to allow comparative analyses. The analysis concerns aspects of processing time, memory allocation and correspondence establishment, which take into account the time waisted in order to establish the correspondences between the images. At this point it is also established a comparative table between the chosen methods highlighting advantages and disadvantages. The results of the application of the analysed methods are distributed throughout the text as images and tables, which are used to prove the efficiency of them and were generated by a test system implemented. The system developed provides the generation of images, according to the choosen method, informs the time waisted on processing and shows those images sequentially as an animation. Finally this work presents conclusions about the study and indicates future extensions that can be done in order to make the developed system more flexible and complete.

The target of this master thesis is to evaluate the Image Warping technique and propose a possible design for an implementation in FPGA. The Image Warping is widely used in the image processing for image correction and rectification. A DSP is a usual choice for implantation of the image processing algorithms, but to decrease a cost of the target system it was proposed to use an FPGA for implementation. In this work a different Image Warping methods was evaluated in terms of performance, produced image quality, complexity and design size. Also, considering that it is not only Image Warping algorithm which will be implemented on the target system, it was important to estimate a possible memory bandwidth used by the proposed design. The evaluation was done by implemented a C-model of the proposed design with a finite datapath to simulate hardware implementation as close as possible.

Interpolation is essential in digital image processing, especially magnification. Many different approaches to interpolation specific to magnification have been developed in an effort to overcome the shortcomings of bilinear and bicubic interpolation. One of these approaches, Constraint-Based Interpolation, produces an image that is free of jaggies and has less blurring than bilinear or bicubic interpolation. Although Constraint-Based Interpolation produces a visually pleasing image, there are user-chosen parameters that make the algorithm difficult to use. In this thesis we propose a method for automatic selection of those parameters and an extension of Constraint-Based Interpolation to other forms of image manipulation, such as skew, rotation, warp, or any other invertable image transformation. By extending Constaint-Based Interpolation the same improvements observed in magnification could be observed in these other image transformations.

Digital cameras use a single image sensor array with a color filter array (CFA) to measure a color image. Instead of measuring a red, green, and blue value at every pixel, these cameras have a filter built onto each pixel so that only one portion of the visible spectrum is measured. To generate a full-color image, the camera must estimate the missing two values at every pixel. This process is known as color filter array interpolation. The Bayer CFA pattern samples the green image on half of the pixels of the imaging sensor on a quincunx grid. The other half of the pixels measure the red and blue images equally on interleaved rectangular sampling grids. This thesis analyzes this problem with sampling theory. The red and blue images are sampled at half the rate of the green image and therefore have a higher probability of aliasing in the output image. This is apparent when simple interpolation algorithms like bilinear interpolation are used for CFA interpolation. Two reference algorithms, a projections onto convex sets (POCS) algorithm and an edge-directed algorithm by Adams and Hamilton (AH), are studied. Both algorithms address aliasing in the green image. Because of the high correlation among the red, green, and blue images, information from the red and blue images can be used to better interpolate the green image. The reference algorithms are studied to learn how this information is used. This leads to two new interpolation algorithms for the green image. The red and blue interpolation algorithm of AH is also studied to determine how the inter-image correlation is used when interpolating these images. This study shows that because the green image is sampled at a higher rate, it retains much of the high-frequency information in the original image. This information is used to estimate aliasing in the red and blue images. We present a general algorithm based on the AH algorithm to interpolate the red and blue images. This algorithm is able to provide results that are on average, better than both reference algorithms, POCS and AH.

Scale is a widely used notion in medical image analysis that evolved in the form of scale-space theory where the key idea is to represent and analyze an image at various resolutions. Recently, a notion of local morphometric scale referred to as "tensor scale" was introduced using an ellipsoidal model that yields a unified representation of structure size, orientation and anisotropy. In the previous work, tensor scale was described using a 2-D algorithmic approach and a precise analytic definition was missing. Also, with previous framework, 3-D application is not practical due to computational complexity. The overall aim of the Ph.D. research is to establish an analytic definition of tensor scale in n-dimensional (n-D) images, to develop an efficient computational solution for 2- and 3-D images and to investigate its role in various medical imaging applications including image interpolation, filtering, and segmentation. Firstly, an analytic definition of tensor scale for n-D images consisting of objects formed by pseudo-Riemannian partitioning manifolds has been formulated. Tensor scale captures contextual structural information which is useful in local structure-adaptive anisotropic parameter control and local structure description for object/image matching. Therefore, it is helpful in a wide range of medical imaging algorithms and applications. Secondly, an efficient computational solution of tensor scale for 2- and 3-D images has been developed. The algorithm has combined Euclidean distance transform and several novel differential geometric approaches. The accuracy of the algorithm has been verified on both geometric phantoms and real images compared to the theoretical results generated using brute-force method. Also, a matrix representation has been derived facilitating several operations including tensor field smoothing to capture larger contextual knowledge. Thirdly, an inter-slice interpolation algorithm using 2-D tensor scale information of adjacent slices has been developed to determine the interpolation line at each image location in a gray level image. Experimental results have established the superiority of the tensor scale based interpolation method as compared to existing interpolation algorithms. Fourthly, an anisotropic diffusion filtering algorithm based on tensor scale has been developed. The method made use of tensor scale to design the conductance function for diffusion process so that along structure diffusion is encouraged and boundary sharpness is preserved. The performance has been tested on phantoms and medical images at various noise levels and the results were quantitatively compared with conventional gradient and structure tensor based algorithms. The experimental results formed are quite encouraging. Also, a tensor scale based n-linear interpolation method has been developed where the weights of neighbors were locally tuned based on local structure size and orientation. The method has been applied on several phantom and real images and the performance has been evaluated in comparison with standard linear interpolation and windowed Sinc interpolation methods. Experimental results have shown that the method helps to generate more precise structure boundaries without causing ringing artifacts. Finally, a new anisotropic constrained region growing method locally controlled by tensor scale has been developed for vessel segmentation that encourages axial region growing while arresting cross-structure leaking. The method has been successfully applied on several non-contrast pulmonary CT images. The accuracy of the new method has been evaluated using manually selection and the results found are very promising.

Image interpolation is the process of generating a new image utilizing a set of available images. The available images may be taken with a camera at different times, or with multiple cameras and from different viewpoints. Usually, the interpolation problem in the first scenario is called Frame Rate-Up Conversion (FRUC), and the second one view synthesis. This thesis focuses on image interpolation and addresses both FRUC and view synthesis problems. We propose a novel FRUC method using optical flow motion estimation and a patch-based reconstruction scheme. FRUC interpolates new frames between original frames of a video to increase the number of frames, and increases motion continuity. In our approach first, forward and backward motion vectors are obtained using an optical flow algorithm, and reconstructed versions of the current and previous frames are generated by our patch-based reconstruction scheme. Using the original and reconstructed versions of the current and previous frames, two mismatch masks are obtained. Then two versions of the middle frame are generated using a patch-based scheme, with estimated motion vectors and the current and previous frames. Finally, a middle mask, which identifies the mismatch areas of the two middle frames is reconstructed. Using these three masks, the best candidates for interpolation are selected and fused to obtain the final middle frame. Due to the patch-based nature of our interpolation scheme most of the holes and cracks will be filled. Although there is always a probability of having holes, the size and number of such holes are much smaller than those that would be generated using pixel-based mapping. The rare holes are filled using existing hole-filling algorithms. With fewer and smaller holes, simpler hole-filling algorithms can be applied to the image and the overall complexity of the required post processing decreases. View synthesis is the process of generating a new (virtual) view using available ones. Depending on the amount of available geometric information, view synthesis techniques can be divided into three categories: Image Based Rendering (IBR), Depth Image Based Rendering (DIBR), and Model Based Rendering (MBR). We introduce an adaptive version, patch-based scheme for IBR. This patch-based scheme reduces the size and number of holes during reconstruction. The size of patch is determined in response to edge information for better reconstruction, especially near the boundaries. In the first stage of the algorithm, disparity is obtained using optical flow estimation. Then, a reconstructed version of the left and right views are generated using our adaptive patch-based algorithm. The mismatches between each view and its reconstructed version are obtained in the mismatch detection steps. This stage results in two masks as outputs, which help with the refinement of disparities and the selection of the best patches for final synthesis. Finally, the remaining holes are filled using our simple hole filling scheme and the refined disparities. The adaptive version still benefits from the overlapping effect of the patches for hole reduction. However, compared with our fixed-size version, it results in better reconstruction near the edges, object boundaries, and inside the highly textured areas. We also propose an adaptive patch-based scheme for DIBR. The proposed method avoids unnecessary warping which is a computationally expensive step in DIBR. We divide nearby views into blocks, and only warp the center of each block. To have a better reconstruction near the edges and depth discontinuities, the block size is selected adaptively. In the blending step, an approach is introduced to calculate and refine the blending weights. Many of the existing DIBR schemes warp all pixels of nearby views during interpolation which is unnecessary. We show that using our adaptive patch-based scheme, it is possible to reduce the number of required warping without degrading the overall quality compared with existing schemes. / Thesis / Doctor of Philosophy (PhD)