Foreground extraction technology plays an important role in image and video processing tasks. It has been widely used in various industries. To better describe the overlap relationship between foreground and background, alpha channel is introduced. It reveals the opacity property of foreground objects. Thus, fully extracting a foreground object requires determining the alpha values for pixels, also known as extracting an alpha matte. In this thesis, we propose an improved sampling-based alpha matting algorithm, which is capable of generating high quality matting results. By analyzing the weakness of previous approaches, we optimize the sampling process and consider the cost of each sample pair to avoid missing any good samples. The good performance is demonstrated even for complex images. On the other hand, extracting foreground objects from video sequences is a more challenging task since it has higher demands on accuracy and efficiency. Previous approaches usually require a significant amount of user input and the results still suffer from inaccuracy. In this thesis, we successfully extend our algorithm to video sequences and let it run in an automatic fashion. Adaptive trimap, which is vital for matting, can be automatically generated and properly propagated in this system. Our method not only reduces the user interference but also guarantees the matting quality.

Foreground extraction technology plays an important role in image and video processing tasks. It has been widely used in various industries. To better describe the overlap relationship between foreground and background, alpha channel is introduced. It reveals the opacity property of foreground objects. Thus, fully extracting a foreground object requires determining the alpha values for pixels, also known as extracting an alpha matte. In this thesis, we propose an improved sampling-based alpha matting algorithm, which is capable of generating high quality matting results. By analyzing the weakness of previous approaches, we optimize the sampling process and consider the cost of each sample pair to avoid missing any good samples. The good performance is demonstrated even for complex images. On the other hand, extracting foreground objects from video sequences is a more challenging task since it has higher demands on accuracy and efficiency. Previous approaches usually require a significant amount of user input and the results still suffer from inaccuracy. In this thesis, we successfully extend our algorithm to video sequences and let it run in an automatic fashion. Adaptive trimap, which is vital for matting, can be automatically generated and properly propagated in this system. Our method not only reduces the user interference but also guarantees the matting quality.

Foreground extraction technology plays an important role in image and video processing tasks. It has been widely used in various industries. To better describe the overlap relationship between foreground and background, alpha channel is introduced. It reveals the opacity property of foreground objects. Thus, fully extracting a foreground object requires determining the alpha values for pixels, also known as extracting an alpha matte. In this thesis, we propose an improved sampling-based alpha matting algorithm, which is capable of generating high quality matting results. By analyzing the weakness of previous approaches, we optimize the sampling process and consider the cost of each sample pair to avoid missing any good samples. The good performance is demonstrated even for complex images. On the other hand, extracting foreground objects from video sequences is a more challenging task since it has higher demands on accuracy and efficiency. Previous approaches usually require a significant amount of user input and the results still suffer from inaccuracy. In this thesis, we successfully extend our algorithm to video sequences and let it run in an automatic fashion. Adaptive trimap, which is vital for matting, can be automatically generated and properly propagated in this system. Our method not only reduces the user interference but also guarantees the matting quality.

Alpha matting is an important topic in areas of computer vision. It has various applications, such as virtual reality, digital image and video editing, and image synthesis. The conventional approaches for alpha matting perform unsatisfactorily when they encounter complicated background and foreground. It is also difficult for them to extract alpha matte accurately when the foreground objects are transparent, semi-transparent, perforated or hairy. Fortunately, the rapid development of deep learning techniques brings new possibilities for solving alpha matting problems. In this thesis, we propose a residual convolutional grid network for alpha matting, which is based on the convolutional neural networks (CNNs) and can learn the alpha matte directly from the original image and its trimap. Our grid network consists of horizontal residual convolutional computation blocks and vertical upsampling/downsampling convolutional computation blocks. By choosing different paths to pass information by itself, our network can not only retain the rich details of the image but also extract high-level abstract semantic information of the image. The experimental results demonstrate that our method can solve the matting problems that plague conventional matting methods for decades and outperform all the other state-of-the-art matting methods in quality and visual evaluation. The only matting method performs a little better than ours is the current best matting method. However, that matting method requires three times amount of trainable parameters compared with ours. Hence, our matting method is the best considering the computation complexity, memory usage, and matting performance.

This thesis proposes a novel chroma keying method based on stereo images, which can be applied to post-process the alpha matte generated by any existing matting approach. Given a pair of stereo images, a new matting Laplacian matrix is constructed based on the affinities between matching pixels and their neighbors from two frames. Based on the new matting Laplacian matrix, a new cost function is also formulated to estimate alpha values of the reference image through the propagation between stereo images.

Matting, which aims to separate the foreground object from the background of an image, is an important problem in computer vision. Most existing methods rely on auxiliary information such as trimaps or scibbles to alleviate the difficulty arising from the underdetermined nature of the matting problem. However, such methods tend to be sensitive to the quality of auxiliary information, and are unsuitable for real-time deployment. In this paper, we propose a novel Attention-based Multi-scale Matting Network (AMMNet), which can estimate the alpha matte from a given RGB image without resorting to any auxiliary information. The proposed AMMNet consists of three (sub-)networks: 1) a multi-scale neural network designed to provide the semantic information of the foreground object, 2) a Unet-like network for attention mask generation, and 3) a Convolutional Neural Network (CNN) customized to integrate high- and low-level features extracted by the first two (sub-)networks. The AMMNet is generic in nature and can be trained end-to-end in a straightforward manner. The experimental results indicate that the performance of AMMNet is competitive against the state-of-the-art matting methods, which either require additional side information or are tailored to images with a specific type of content (e.g., portrait). / Thesis / Master of Applied Science (MASc)

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior / Video-based interactive storytelling has as its main goal the generation of interactive narratives, allowing users to control the course of the story and using cinematographic compositions to dramatize it. For this process to be possible, there is a need for large amounts of cinematographic content in the form of filmed scenes. This content, on the other hand, has to be properly pre-processed in order to be usable. This work proposes an approach for video segmentation aimed for video-based interactive storytelling, using alpha matting to extract color and transparency of video elements to be later recomposed for dramatization purposes. The developed solution uses background subtraction and energy minimization techniques to automatically generate trimaps. / O storytelling interativo baseado em vídeo tem como objetivo a geração de narrativas, permitindo o controle de um usuário sobre o rumo da estória e utilizando composições cinematográficas para dramatizá-la. Para que este processo seja possível, existe a necessidade de uma grande quantidade de conteúdo cinematográfico na forma de filmagens. Este conteúdo, por sua vez, precisa ser adequadamente pré-processado para permitir sua utilização adequada. Nesse trabalho, uma abordagem para segmentação de vídeos para storytelling interativo baseado em vídeo é proposta, utilizando alpha matting para extrair informações de cor e transparência de elementos de vídeos para serem reutilizados em processos de dramatização. A solução desenvolvida utiliza técnicas de subtração de fundo e minimização de energia para gerar trimaps de forma automática.

This thesis proposes a new chroma keying method that can automatically detect background, foreground, and unknown regions. For background color detection, we use K-means clustering in color space to calculate the limited number of clusters of background colors. We use spatial information to clean the background regions and minimize the unknown regions. Our method only needs minimum inputs from user. For unknown regions, we implement the alpha matte based on Wang's robust matting algorithm, which is considered one of the best algorithms in the literature, if not the best. Wang's algorithm is based on modified random walk. We proposed a better color selection method, which improves matting results in the experiments. In the thesis, a detailed implementation of robust matting is provided. The experimental results demonstrate that our proposed method can handle images with one background color, images with gridded background, and images with difficult regions such as complex hair stripes and semi-transparent clothes.

This thesis proposes a new chroma keying method that can automatically detect background, foreground, and unknown regions. For background color detection, we use K-means clustering in color space to calculate the limited number of clusters of background colors. We use spatial information to clean the background regions and minimize the unknown regions. Our method only needs minimum inputs from user. For unknown regions, we implement the alpha matte based on Wang's robust matting algorithm, which is considered one of the best algorithms in the literature, if not the best. Wang's algorithm is based on modified random walk. We proposed a better color selection method, which improves matting results in the experiments. In the thesis, a detailed implementation of robust matting is provided. The experimental results demonstrate that our proposed method can handle images with one background color, images with gridded background, and images with difficult regions such as complex hair stripes and semi-transparent clothes.

This thesis proposes a new chroma keying method that can automatically detect background, foreground, and unknown regions. For background color detection, we use K-means clustering in color space to calculate the limited number of clusters of background colors. We use spatial information to clean the background regions and minimize the unknown regions. Our method only needs minimum inputs from user. For unknown regions, we implement the alpha matte based on Wang's robust matting algorithm, which is considered one of the best algorithms in the literature, if not the best. Wang's algorithm is based on modified random walk. We proposed a better color selection method, which improves matting results in the experiments. In the thesis, a detailed implementation of robust matting is provided. The experimental results demonstrate that our proposed method can handle images with one background color, images with gridded background, and images with difficult regions such as complex hair stripes and semi-transparent clothes.