Coordenação de Aperfeiçoamento de Pessoal de Nível Superior / Video-based interactive storytelling has as its main goal the generation of interactive narratives, allowing users to control the course of the story and using cinematographic compositions to dramatize it. For this process to be possible, there is a need for large amounts of cinematographic content in the form of filmed scenes. This content, on the other hand, has to be properly pre-processed in order to be usable. This work proposes an approach for video segmentation aimed for video-based interactive storytelling, using alpha matting to extract color and transparency of video elements to be later recomposed for dramatization purposes. The developed solution uses background subtraction and energy minimization techniques to automatically generate trimaps. / O storytelling interativo baseado em vídeo tem como objetivo a geração de narrativas, permitindo o controle de um usuário sobre o rumo da estória e utilizando composições cinematográficas para dramatizá-la. Para que este processo seja possível, existe a necessidade de uma grande quantidade de conteúdo cinematográfico na forma de filmagens. Este conteúdo, por sua vez, precisa ser adequadamente pré-processado para permitir sua utilização adequada. Nesse trabalho, uma abordagem para segmentação de vídeos para storytelling interativo baseado em vídeo é proposta, utilizando alpha matting para extrair informações de cor e transparência de elementos de vídeos para serem reutilizados em processos de dramatização. A solução desenvolvida utiliza técnicas de subtração de fundo e minimização de energia para gerar trimaps de forma automática.

Cette thèse porte sur la segmentation du masque capillaire dans un visage. Nous présentons dans ce manuscrit une étude sur la localisation et la caractérisation d'une chevelure dans une image. L'objectif de ces travaux est de proposer une méthode permettant de définir automatiquement la zone capillaire avec la meilleure fiabilité possible afin de prendre en compte la grande variabilité de la représentation d'une chevelure. La segmentation est effectuée par un traitement "Matting" qui est une méthode de segmentation par approche "régions". Cet algorithme sépare une image en deux plans : un plan d'information qui représente la chevelure et un plan de fond. Il est initialisé par la définition de marqueurs qui vont être diffusés dans l'ensemble de l'image. L'obtention d'une bonne segmentation dépend directement de la précision dans le placement de ces marqueurs. Nous définissons leurs positions par une analyse conjointe de trois paramètres caractéristiques d'une chevelure : sa texture, sa teinte et sa position. Dans un premier temps, nous avons mis en place une analyse fréquentielle pour caractériser la texture. Nous utilisons un filtrage de l'image par un filtre passe-bande gaussien isotrope. Nous modélisons un masque de localisation des zones fréquentielles similaires à la chevelure. Dans un second temps, nous avons mis en place une analyse couleur pour caractériser la teinte. Nous utilisons un classifieur couleur qui représente la distribution du modèle couleur par une distribution gaussienne sur chaque composante de chrominance dans l'espace couleur YCbCr. Une première information commune de localisation d'une chevelure est définie et est combinée par une méthode de fusion de données basée sur le modèle des fonctions de croyances transférables. Cette approche permet notamment la prise en compte de différents degrés d'ignorance par la modélisation d'un état d'"incertitude". L'ajout de ce nouvel état est particulièrement adapté à notre algorithme de segmentation puisqu'il permet de contrôler la position des pixels dont l'état est estimé pendant le traitement "Matting". Cette approche est ensuite optimisée par l'ajout d'une fonction d'affaiblissement basée sur la caractérisation de la localisation d'une chevelure autour du visage qui est le troisième paramètre capillaire caractéristique. Cette fonction permet de pondérer la fiabilité de nos sources de données par rapport à l'éloignement du visage. En effet plus un pixel est loin d'un visage moins il a de probabilité d'appartenir au masque capillaire. La segmentation capillaire est évaluée grâce à une analyse quantitative de la définition d'un masque de chevelure en comparaison avec une base de référence obtenue par une segmentation semi-manuelle. Enfin le masque capillaire est caractérisé par trois descripteurs, un pour chaque paramètre capillaire. Cette classification permet la description d'une chevelure par une approche similaire à une description cognitive faite par un observateur humain.

This thesis proposes a new chroma keying method that can automatically detect background, foreground, and unknown regions. For background color detection, we use K-means clustering in color space to calculate the limited number of clusters of background colors. We use spatial information to clean the background regions and minimize the unknown regions. Our method only needs minimum inputs from user. For unknown regions, we implement the alpha matte based on Wang's robust matting algorithm, which is considered one of the best algorithms in the literature, if not the best. Wang's algorithm is based on modified random walk. We proposed a better color selection method, which improves matting results in the experiments. In the thesis, a detailed implementation of robust matting is provided. The experimental results demonstrate that our proposed method can handle images with one background color, images with gridded background, and images with difficult regions such as complex hair stripes and semi-transparent clothes.

This thesis proposes a new chroma keying method that can automatically detect background, foreground, and unknown regions. For background color detection, we use K-means clustering in color space to calculate the limited number of clusters of background colors. We use spatial information to clean the background regions and minimize the unknown regions. Our method only needs minimum inputs from user. For unknown regions, we implement the alpha matte based on Wang's robust matting algorithm, which is considered one of the best algorithms in the literature, if not the best. Wang's algorithm is based on modified random walk. We proposed a better color selection method, which improves matting results in the experiments. In the thesis, a detailed implementation of robust matting is provided. The experimental results demonstrate that our proposed method can handle images with one background color, images with gridded background, and images with difficult regions such as complex hair stripes and semi-transparent clothes.

This thesis proposes a new chroma keying method that can automatically detect background, foreground, and unknown regions. For background color detection, we use K-means clustering in color space to calculate the limited number of clusters of background colors. We use spatial information to clean the background regions and minimize the unknown regions. Our method only needs minimum inputs from user. For unknown regions, we implement the alpha matte based on Wang's robust matting algorithm, which is considered one of the best algorithms in the literature, if not the best. Wang's algorithm is based on modified random walk. We proposed a better color selection method, which improves matting results in the experiments. In the thesis, a detailed implementation of robust matting is provided. The experimental results demonstrate that our proposed method can handle images with one background color, images with gridded background, and images with difficult regions such as complex hair stripes and semi-transparent clothes.

This thesis proposes a new chroma keying method that can automatically detect background, foreground, and unknown regions. For background color detection, we use K-means clustering in color space to calculate the limited number of clusters of background colors. We use spatial information to clean the background regions and minimize the unknown regions. Our method only needs minimum inputs from user. For unknown regions, we implement the alpha matte based on Wang's robust matting algorithm, which is considered one of the best algorithms in the literature, if not the best. Wang's algorithm is based on modified random walk. We proposed a better color selection method, which improves matting results in the experiments. In the thesis, a detailed implementation of robust matting is provided. The experimental results demonstrate that our proposed method can handle images with one background color, images with gridded background, and images with difficult regions such as complex hair stripes and semi-transparent clothes.

Rutting failures are prominent in expedient airfields constructed with AM2 landing mats over soft existing subgrades. There are many issues that must be addressed when approaching this multiaceted problem. The load transfer mechanism occurring at interlocking mat joints and the mat-soil interface bonding condition affect near surface subgrade response. The repeated loading coupled with lateral aircraft wander causes significant principal stress rotation in the subgrade. This kneading action then causes variations in the excess pore-water pressure and a subsequent softening of the soil. The purpose of this study is to investigate the critical factors that lead to subgrade rutting failures in landing mats constructed over soft subgrades. A three dimensional finite element (3D FE) model of a landing mat system over soft subgrade is implemented under both static and pseudo-dynamic loading conditions with aircraft wander. To capture the complex stress histories induced by the simulated moving gear loads over the unique structural features of the AM2 mat system, an elastoplastic kinematic hardening constitutive model, the Multi-Mechanical Model, is developed, calibrated and used to represent the subgrade response. Under both static and pseudo-dynamic loading, the FE model results match very well with the stress and deformation results from full-scale instrumented testing of the AM2 mat over 6 CBR subgrade. Results show that incorporating the load transfer mechanism occurring at the mat joints and varying the mat-soil interface condition affect the near surface subgrade deformation and stress responses that contribute to rutting failures. Furthermore, rotation of the principal stress axes and changes in excess pore-water pressures occur in the subgrade because of the moving tire load. These phenomena contribute to extension of the field of deformation influence around the trafficked area in the subgrade and upheaval at the edges of the test section. Findings of this study show that although layered elastic analysis procedures are the basis of current airfield design methodologies, critical design features and the corresponding deformation responses can be better modeled using the FE approach. Furthermore, the proposed 3D modeling approach implementing aircraft wander can provide a reliable platform for accurately simulating the subgrade response under pseudo-dynamic loading conditions.

Textures are used widely in computer graphics to represent fine visual details and produce realistic looking images. Often it is necessary to remove some foreground object from the scene. Removal of the portion creates one or more holes in the texture image. These holes need to be filled to complete the image. Various methods like clone brush strokes and compositing processes are used to carry out this completion. User skill is required in such methods. Texture synthesis can also be used to complete regions where the texture is stationary or structured. Reconstructing methods can be used to fill in large-scale missing regions by interpolation. Inpainting is suitable for relatively small, smooth and non-textured regions. A number of other approaches focus on the edge and contour completion aspect of the problem. In this thesis we present a novel approach for addressing this image completion problem. Our approach focuses on image based completion, with no knowledge of the underlying scene. In natural images there is a strong horizontal orientation of texture/color distribution. We exploit this fact in our proposed algorithm to fill in missing regions from natural images. We follow the principle of figural familiarity and use the image as our training set to complete the image.

3D human models play an important role in computer graphics applications from a wide range of domains, including education, entertainment, medical care simulation and military training. In many situations, we want the 3D model to have a visual appearance that matches that of a specific living person and to be able to be controlled by that person in a natural manner. Among other uses, this approach supports the notion of human surrogacy, where the virtual counterpart provides a remote presence for the human who controls the virtual character's behavior. In this dissertation, a human modeling pipeline is proposed for the problem of creating a 3D digital model of a real person. Our solution involves reshaping a 3D human template with a 2D contour of the participant and then mapping the captured texture of that person to the generated mesh. Our method produces an initial contour of a participant by extracting the user image from a natural background. One particularly novel contribution in our approach is the manner in which we improve the initial vertex estimate. We do so through a variant of the ShortStraw corner-finding algorithm commonly used in sketch-based systems. Here, we develop improvements to ShortStraw, presenting an algorithm called IStraw, and then introduce adaptations of this improved version to create a corner-based contour segmentatiuon algorithm. This algorithm provides significant improvements on contour matching over previously developed systems, and does so with low computational complexity. The system presented here advances the state of the art in the following aspects. First, the human modeling process is triggered automatically by matching the participant's pose with an initial pose through a tracking device and software. In our case, the pose capture and skeletal model are provided by the Microsoft Kinect and its associated SDK. Second, color image, depth data, and human tracking information from the Kinect and its SDK are used to automatically extract the contour of the participant and then generate a 3D human model with skeleton. Third, using the pose and the skeletal model, we segment the contour into eight parts and then match the contour points on each segment to a corresponding anchor set associated with a 3D human template. Finally, we map the color image of the person to the 3D model as its corresponding texture map. The whole modeling process only take several seconds and the resulting human model looks like the real person. The geometry of the 3D model matches the contour of the real person, and the model has a photorealistic texture. Furthermore, the mesh of the human model is attached to the skeleton provided in the template, so the model can support programmed animations or be controlled by real people. This human control is commonly done through a literal mapping (motion capture) or a gesture-based puppetry system. Our ultimate goal is to create a mixed reality (MR) system, in which the participants can manipulate virtual objects, and in which these virtual objects can affect the participant, e.g., by restricting their mobility. This MR system prototype design motivated the work of this dissertation, since a realistic 3D human model of the participant is an essential part of implementing this vision.

The thesis deals with an image processing problem called image matting. The problem involves detection of a foreground and background in an image with minimal user interaction using trimaps. Foreground detection is used in image composition. The goal of the thesis is to apply already known algorithms, in this case A Global sampling matting, in an Android application. The most important result is an intuitive application that can be used for making creative viral photos. Agile methodology is applied throughout the whole application development cycle. From the very beginning, the application is publicly available as a minimum viable product on Google play. The work’s contribution is in optimization of the mentioned algorithm for use in mobile devices and parallelization on a GPU, together with a publicly available application.