Image matting is a basic feature in most of image quality improvement applications and is considered as a fundamental problem in the computer vision field. In this paper, an End-to-End Image Matting Platform is proposed to segment the foreground and background by creating an alpha matte image. My end-to-end image matting algorithm is a learning based network which contains two stages. The first stage is to create a tri-map image from an RGB image using segmentation neural network where tri-map images are used to locate the expected foreground objects with rough outlines. The second stage is an image matting neural network, and it takes the outputs from the first stage as prior knowledge to predict precise alpha matte images. With the help of image matting formula and the outputs from the second stage, contents in an RGB image can be easily split into foreground and background. I applied both training and evaluating on Adobe matting benchmark and Car Media 2.0's car oriented image matting dataset, and the outcomes demonstrated the convenience and superior performance of our algorithm compared to existing state of the art methods. This paper put forward a web platform structure to integrate deep learning algorithms. I applied multiple strategies to enhance the performance of the platform. By using this platform, multiple users can work with different deep learning applications at the same time which dramatically increases the efficiency of the server usage. / Thesis / Master of Applied Science (MASc)

Cette thèse porte sur l’élaboration d’un système de manipulation de vidéo. De manière plus précise il s’agit d’extraction et de composition d’objets vidéo. Dans le domaine du traitement d’image fixe, les techniques d’extraction et de démélange (connus sous le nom de matting) et de composition ont vu une réelle amélioration au cours de la dernière décennie, surtout avec l’apparition de méthodes semi-automatiques profitant d’une interaction avec l’utilisateur pour surmonter le gap sémantique. Cela a permis d’aboutir à des algorithmes de plus en plus rapides et de plus en plus robustes. Dans le cadre du traitement de vidéo, cette problématique forme encore un très intéressant challenge, issu du caractère volumineux, en termes complexité de données et de nombre d’images dans la vidéo. Cet élément fait en sorte que la tâche accomplie par l’utilisateur pour marquer un objet d’intérêt peut être très fastidieuse ou souvent impossible. Les travaux que nous avons réalisés au cours de cette thèse se sont concentrés sur l’extension et l’adaptation de la transformée en distance et des courbes actives pour la propagation des marquages d’objets vidéo. Nous avons aussi proposé une amélioration d’une technique pouvant être utilisée avec ces marquages pour l’extraction d’objet vidéo.Dans le premier chapitre nous présentons le contexte et la problématique de nos travaux. Dans le deuxième chapitre nous faisons un tour d’horizon des approches, des outils d’édition de vidéo existant sur le marché, tout en les classant en deux familles : édition par morceaux ou par blocs et édition par objets vidéo. Ensuite, nous présentons un rapide état de l’art sur la segmentation que nous décomposons en trois parties : la segmentation classique, la segmentation interactive et l’image matting. Aussi nous détaillons l’extension de l’image matting au video matting en présentant les principales approches existantes. Le chapitre 3 présente notre première approche pour la propagation de marquage dans les vidéos. Cette approche est une approche volumique 2D+T tirant sa puissance de ce que nous avons bâti une CDT (transformée en distance couleur). Le chapitre 4, lui, présente notre évolution de perception vers un processus de propagation de marquages plus robuste et plus performant basé sur les courbes actives. Nous commençons par faire un état de l’art abrégé sur les courbes actives et nous présentons par la suite notre modélisation et son application. Nous détaillons, aussi le mécanisme de gestion dynamique des poids que nous avons mis en place. Dans le chapitre 5, nous allons discuter de l’application de notre système pour le matting vidéo et nous présentons les améliorations que nous avons apportés à l’approche Spectral Matting, dans ce but / Pas de résumé en anglais

No description available.

Foreground extraction technology plays an important role in image and video processing tasks. It has been widely used in various industries. To better describe the overlap relationship between foreground and background, alpha channel is introduced. It reveals the opacity property of foreground objects. Thus, fully extracting a foreground object requires determining the alpha values for pixels, also known as extracting an alpha matte. In this thesis, we propose an improved sampling-based alpha matting algorithm, which is capable of generating high quality matting results. By analyzing the weakness of previous approaches, we optimize the sampling process and consider the cost of each sample pair to avoid missing any good samples. The good performance is demonstrated even for complex images. On the other hand, extracting foreground objects from video sequences is a more challenging task since it has higher demands on accuracy and efficiency. Previous approaches usually require a significant amount of user input and the results still suffer from inaccuracy. In this thesis, we successfully extend our algorithm to video sequences and let it run in an automatic fashion. Adaptive trimap, which is vital for matting, can be automatically generated and properly propagated in this system. Our method not only reduces the user interference but also guarantees the matting quality.

Foreground extraction technology plays an important role in image and video processing tasks. It has been widely used in various industries. To better describe the overlap relationship between foreground and background, alpha channel is introduced. It reveals the opacity property of foreground objects. Thus, fully extracting a foreground object requires determining the alpha values for pixels, also known as extracting an alpha matte. In this thesis, we propose an improved sampling-based alpha matting algorithm, which is capable of generating high quality matting results. By analyzing the weakness of previous approaches, we optimize the sampling process and consider the cost of each sample pair to avoid missing any good samples. The good performance is demonstrated even for complex images. On the other hand, extracting foreground objects from video sequences is a more challenging task since it has higher demands on accuracy and efficiency. Previous approaches usually require a significant amount of user input and the results still suffer from inaccuracy. In this thesis, we successfully extend our algorithm to video sequences and let it run in an automatic fashion. Adaptive trimap, which is vital for matting, can be automatically generated and properly propagated in this system. Our method not only reduces the user interference but also guarantees the matting quality.

Foreground extraction technology plays an important role in image and video processing tasks. It has been widely used in various industries. To better describe the overlap relationship between foreground and background, alpha channel is introduced. It reveals the opacity property of foreground objects. Thus, fully extracting a foreground object requires determining the alpha values for pixels, also known as extracting an alpha matte. In this thesis, we propose an improved sampling-based alpha matting algorithm, which is capable of generating high quality matting results. By analyzing the weakness of previous approaches, we optimize the sampling process and consider the cost of each sample pair to avoid missing any good samples. The good performance is demonstrated even for complex images. On the other hand, extracting foreground objects from video sequences is a more challenging task since it has higher demands on accuracy and efficiency. Previous approaches usually require a significant amount of user input and the results still suffer from inaccuracy. In this thesis, we successfully extend our algorithm to video sequences and let it run in an automatic fashion. Adaptive trimap, which is vital for matting, can be automatically generated and properly propagated in this system. Our method not only reduces the user interference but also guarantees the matting quality.

The U.S. military requires the ability to rapidly deploy troops, equipment, and materials anywhere in the world. Recent operations have brought attention to the need to utilize austere, unsurfaced, and sometimes sub-standard airfields within a theater of interest. These airfields may require additional taxiways and aprons. One option for the rapid construction of such is airfield matting systems. The focus of the work for this thesis was commercially available airfield matting systems to support large military transport aircraft, such as the C 17. Several test sections with differing strength soils were built with chosen mats tested in an elimination method, using a load cart that simulates contingency loading of one main gear of the C 17. Matting systems were evaluated based on logistical and assembly requirements, and deformation and damage sustained during traffic. A modeling effort was performed to investigate the potential of a simple model to predict the response of these matting systems under full-scale testing.

In Mixed Reality scenarios, background replacement is a common way to immerse a user in a synthetic environment. Properly identifying the background pixels in an image or video is a dif- ficult problem known as matting. Proper alpha mattes usually come from human guidance, special hardware setups, or color dependent algorithms. This is a consequence of the under-constrained nature of the per pixel alpha blending equation. In constant color matting, research identifies and replaces a background that is a single color, known as the chroma key color. Unfortunately, the algorithms force a controlled physical environment and favor constant, uniform lighting. More generic approaches, such as natural image matting, have made progress finding alpha matte solutions in environments with naturally occurring backgrounds. However, even for the quicker algorithms, the generation of trimaps, indicating regions of known foreground and background pixels, normally requires human interaction or offline computation. This research addresses ways to automatically solve an alpha matte for an image in realtime, and by extension a video, using a consumer level GPU. It does so even in the context of noisy environments that result in less reliable constraints than found in controlled settings. To attack these challenges, we are particularly interested in automatically generating trimaps from depth buffers for dynamic scenes so that algorithms requiring more dense constraints may be used. The resulting computation is parallelizable so that it may run on a GPU and should work for natural images as well as chroma key backgrounds. Extra input may be required, but when this occurs, commodity hardware available in most Mixed Reality setups should be able to provide the input. This allows us to provide real-time alpha mattes for Mixed Reality scenarios that take place in relatively controlled environments. As a consequence, while monochromatic backdrops (such as green screens or retro-reflective material) aid the algorithm’s accuracy, they are not an explicit requirement. iii Finally we explore a sub-image based approach to parallelize an existing hierarchical approach on high resolution imagery. We show that locality can be exploited to significantly reduce the memory and compute requirements of previously necessary when computing alpha mattes of high resolution images. We achieve this using a parallelizable scheme that is both independent of the matting algorithm and image features. Combined, these research topics provide a basis for Mixed Reality scenarios using real-time natural image matting on high definition video sources.

Alpha matting is an important topic in areas of computer vision. It has various applications, such as virtual reality, digital image and video editing, and image synthesis. The conventional approaches for alpha matting perform unsatisfactorily when they encounter complicated background and foreground. It is also difficult for them to extract alpha matte accurately when the foreground objects are transparent, semi-transparent, perforated or hairy. Fortunately, the rapid development of deep learning techniques brings new possibilities for solving alpha matting problems. In this thesis, we propose a residual convolutional grid network for alpha matting, which is based on the convolutional neural networks (CNNs) and can learn the alpha matte directly from the original image and its trimap. Our grid network consists of horizontal residual convolutional computation blocks and vertical upsampling/downsampling convolutional computation blocks. By choosing different paths to pass information by itself, our network can not only retain the rich details of the image but also extract high-level abstract semantic information of the image. The experimental results demonstrate that our method can solve the matting problems that plague conventional matting methods for decades and outperform all the other state-of-the-art matting methods in quality and visual evaluation. The only matting method performs a little better than ours is the current best matting method. However, that matting method requires three times amount of trainable parameters compared with ours. Hence, our matting method is the best considering the computation complexity, memory usage, and matting performance.

Cette thèse porte sur l'élaboration d'un système de manipulation de vidéo. De manière plus précise il s'agit d'extraction et de composition d'objets vidéo. Dans le domaine du traitement d'image fixe, les techniques d'extraction et de démélange (connus sous le nom de matting) et de composition ont vu une réelle amélioration au cours de la dernière décennie, surtout avec l'apparition de méthodes semi-automatiques profitant d'une interaction avec l'utilisateur pour surmonter le gap sémantique. Cela a permis d'aboutir à des algorithmes de plus en plus rapides et de plus en plus robustes. Dans le cadre du traitement de vidéo, cette problématique forme encore un très intéressant challenge, issu du caractère volumineux, en termes complexité de données et de nombre d'images dans la vidéo. Cet élément fait en sorte que la tâche accomplie par l'utilisateur pour marquer un objet d'intérêt peut être très fastidieuse ou souvent impossible. Les travaux que nous avons réalisés au cours de cette thèse se sont concentrés sur l'extension et l'adaptation de la transformée en distance et des courbes actives pour la propagation des marquages d'objets vidéo. Nous avons aussi proposé une amélioration d'une technique pouvant être utilisée avec ces marquages pour l'extraction d'objet vidéo.Dans le premier chapitre nous présentons le contexte et la problématique de nos travaux. Dans le deuxième chapitre nous faisons un tour d'horizon des approches, des outils d'édition de vidéo existant sur le marché, tout en les classant en deux familles : édition par morceaux ou par blocs et édition par objets vidéo. Ensuite, nous présentons un rapide état de l'art sur la segmentation que nous décomposons en trois parties : la segmentation classique, la segmentation interactive et l'image matting. Aussi nous détaillons l'extension de l'image matting au video matting en présentant les principales approches existantes. Le chapitre 3 présente notre première approche pour la propagation de marquage dans les vidéos. Cette approche est une approche volumique 2D+T tirant sa puissance de ce que nous avons bâti une CDT (transformée en distance couleur). Le chapitre 4, lui, présente notre évolution de perception vers un processus de propagation de marquages plus robuste et plus performant basé sur les courbes actives. Nous commençons par faire un état de l'art abrégé sur les courbes actives et nous présentons par la suite notre modélisation et son application. Nous détaillons, aussi le mécanisme de gestion dynamique des poids que nous avons mis en place. Dans le chapitre 5, nous allons discuter de l'application de notre système pour le matting vidéo et nous présentons les améliorations que nous avons apportés à l'approche Spectral Matting, dans ce but