Este trabalho propõe um algoritmo rápido para segmentação de partes móveis em vídeo, tendo como base a detecção de volumes fechados no espaço tridimensional. O vídeo de entrada é pré-processado com um algoritmo de detecção de bordas baseado em linhas de nível para produzir os objetos. Os objetos detectados são agrupados utilizando uma combinação dos métodos de mean shift clustering e meta-agrupamento. Para diminuir o tempo de computação, somente alguns objetos e quadros são utilizados no agrupamento. Uma vez que a forma de detecção garante que os objetos persistem com o mesmo rótulo em múltiplos quadros, a seleção de quadros impacta pouco no resultado final. Dependendo da aplicação desejada os grupos podem ser refinados em uma etapa de pós-processamento. / This work presents a very fast algorithm to segmentation of moving parts in a video, based on detection of surfaces of the scene with closed contours. The input video is preprocessed with an edge detection algorithm based on level lines to produce the objects. The detected objects are clustered using a combination of mean shift clustering and ensemble clustering. In order decrease even more the computation time required, two methods can be used combined: object filtering by size and selecting only a few frames of the video. Since the detected objects are coherent in time, frame skipping does not affect the final result. Depending on the application the detected clusters can be refined using post processing steps.

Este trabalho propõe um algoritmo rápido para segmentação de partes móveis em vídeo, tendo como base a detecção de volumes fechados no espaço tridimensional. O vídeo de entrada é pré-processado com um algoritmo de detecção de bordas baseado em linhas de nível para produzir os objetos. Os objetos detectados são agrupados utilizando uma combinação dos métodos de mean shift clustering e meta-agrupamento. Para diminuir o tempo de computação, somente alguns objetos e quadros são utilizados no agrupamento. Uma vez que a forma de detecção garante que os objetos persistem com o mesmo rótulo em múltiplos quadros, a seleção de quadros impacta pouco no resultado final. Dependendo da aplicação desejada os grupos podem ser refinados em uma etapa de pós-processamento. / This work presents a very fast algorithm to segmentation of moving parts in a video, based on detection of surfaces of the scene with closed contours. The input video is preprocessed with an edge detection algorithm based on level lines to produce the objects. The detected objects are clustered using a combination of mean shift clustering and ensemble clustering. In order decrease even more the computation time required, two methods can be used combined: object filtering by size and selecting only a few frames of the video. Since the detected objects are coherent in time, frame skipping does not affect the final result. Depending on the application the detected clusters can be refined using post processing steps.

A variedade de dispositivos eletrônicos capazes de gravar e reproduzir vídeos digitais vem crescendo rapidamente, aumentando com isso a disponibilidade deste tipo de informação nas mais diferentes plataformas. Com isso, se torna cada vez mais importante o desenvolvimento de formas eficientes de armazenamento, transmissão, e acesso a estes dados. Nesse contexto, a codificação de vídeos tem um papel fundamental ao compactar informação, otimizando o uso de recursos aplicados no armazenamento e na transmissão de vídeos digitais. Não obstante, tarefas que envolvem a análise de vídeos, manipulação e busca baseada em conteúdo também se tornam cada vez mais relevantes, formando uma base para diversas aplicações que exploram a riqueza da informação contida em vídeos digitais. Muitas vezes a solução destes problemas passa pela segmentação de vídeos, que consiste da divisão de um vídeo em regiões que apresentam homogeneidade segundo determinadas características, como por exemplo cor, textura, movimento ou algum aspecto semântico. Nesta tese é proposto um novo método para segmentação de vídeos em objetos constituintes com base na coerência de movimento de regiões. O método de segmentação proposto inicialmente identifica as correspondências entre pontos esparsamente amostrados ao longo de diferentes quadros do vídeo. Logo após, agrupa conjuntos de pontos que apresentam trajetórias semelhantes. Finalmente, uma classificação pixel a pixel é obtida a partir destes grupos de pontos amostrados. O método proposto não assume nenhum modelo de câmera ou de movimento global para a cena e/ou objetos, e possibilita que múltiplos objetos sejam identificados, sem que o número de objetos seja conhecido a priori. Para validar o método de segmentação proposto, foi desenvolvida uma abordagem para a codificação de vídeos baseada em objetos. Segundo esta abordagem, o movimento de um objeto é representado através de transformações afins, enquanto a textura e a forma dos objetos são codificadas simultaneamente, de modo progressivo. O método de codificação de vídeos desenvolvido fornece funcionalidades tais como a transmissão progressiva e a escalabilidade a nível de objeto. Resultados experimentais dos métodos de segmentação e codificação de vídeos desenvolvidos são apresentados, e comparados a outros métodos da literatura. Vídeos codificados segundo o método proposto são comparados em termos de PSNR a vídeos codificados pelo software de referência JM H.264/AVC, versão 16.0, mostrando a que distância o método proposto está do estado da arte em termos de eficiência de codificação, ao mesmo tempo que provê funcionalidades da codificação baseada em objetos. O método de segmentação proposto no presente trabalho resultou em duas publicações, uma nos anais do SIBGRAPI de 2007 e outra no períodico IEEE Transactions on Image Processing. / The variety of electronic devices for digital video recording and playback is growing rapidly, thus increasing the availability of such information in many different platforms. So, the development of efficient ways of storing, transmitting and accessing such data becomes increasingly important. In this context, video coding plays a key role in compressing data, optimizing resource usage for storing and transmitting digital video. Nevertheless, tasks involving video analysis, manipulation and content-based search also become increasingly relevant, forming a basis for several applications that exploit the abundance of information in digital video. Often the solution to these problems makes use of video segmentation, which consists of dividing a video into homogeneous regions according to certain characteristics such as color, texture, motion or some semantic aspect. In this thesis, a new method for segmentation of videos in their constituent objects based on motion coherence of regions is proposed. The proposed segmentation method initially identifies the correspondences of sparsely sampled points along different video frames. Then, it performs clustering of point sets that have similar trajectories. Finally, a pixelwise classification is obtained from these sampled point sets. The proposed method does not assume any camera model or global motion model to the scene and/or objects. Still, it allows the identification of multiple objects, without knowing the number of objects a priori. In order to validate the proposed segmentation method, an object-based video coding approach was developed. According to this approach, the motion of an object is represented by affine transformations, while object texture and shape are simultaneously coded, in a progressive way. The developed video coding method yields functionalities such as progressive transmission and object scalability. Experimental results obtained by the proposed segmentation and coding methods are presented, and compared to other methods from the literature. Videos coded by the proposed method are compared in terms of PSNR to videos coded by the reference software JM H.264/AVC, version 16.0, showing the distance of the proposed method from the sate of the art in terms of coding efficiency, while providing functionalities of object-based video coding. The segmentation method proposed in this work resulted in two publications, one in the proceedings of SIBGRAPI 2007 and another in the journal IEEE Transactions on Image Processing.

Este trabalho propõe um algoritmo rápido para segmentação de partes móveis em vídeo, tendo como base a detecção de volumes fechados no espaço tridimensional. O vídeo de entrada é pré-processado com um algoritmo de detecção de bordas baseado em linhas de nível para produzir os objetos. Os objetos detectados são agrupados utilizando uma combinação dos métodos de mean shift clustering e meta-agrupamento. Para diminuir o tempo de computação, somente alguns objetos e quadros são utilizados no agrupamento. Uma vez que a forma de detecção garante que os objetos persistem com o mesmo rótulo em múltiplos quadros, a seleção de quadros impacta pouco no resultado final. Dependendo da aplicação desejada os grupos podem ser refinados em uma etapa de pós-processamento. / This work presents a very fast algorithm to segmentation of moving parts in a video, based on detection of surfaces of the scene with closed contours. The input video is preprocessed with an edge detection algorithm based on level lines to produce the objects. The detected objects are clustered using a combination of mean shift clustering and ensemble clustering. In order decrease even more the computation time required, two methods can be used combined: object filtering by size and selecting only a few frames of the video. Since the detected objects are coherent in time, frame skipping does not affect the final result. Depending on the application the detected clusters can be refined using post processing steps.

De nos jours, l'augmentation rapide de les vidéos crée une demande massive d'applications d'édition de vidéos. Dans cette thèse, nous résolvons plusieurs problèmes relatifs au post-traitement vidéo. Nous nous concentrons sur l'application de suppression d'objets en vidéo. Pour mener à bien cette tâche, nous l'avons divisé en deux problèmes: (1) une étape de segmentation des objets vidéo pour sélectionner les objets à supprimer et (2) une étape d'inpainting vidéo pour remplir les zones endommagées. Pour le problème de la segmentation vidéo, nous concevons un système adapté aux applications de suppression d’objets avec différentes exigences en termes de précision et d’efficacité. Notre approche repose sur la combinaison de réseaux de neurones convolutifs (CNN) pour la segmentation et de la méthode classique de suivi des masks. Nous adoptons des réseaux de segmentation d’images et les appliquons à la casse vidéo en effectuant une segmentation image par image. En exploitant à la fois les formations en ligne et hors ligne avec uniquement une annotation de première image, les réseaux sont en mesure de produire une segmentation extrêmement précise des objets vidéo. En outre, nous proposons un module de suivi de masque pour assurer la continuité temporelle et un module de liaison de masque pour assurer la cohérence de l'identité entre les trames. De plus, nous présentons un moyen simple d’apprendre la couche de dilatation dans le masque, ce qui nous aide à créer des masques appropriés pour l’application de suppression d’objets vidéo.Pour le problème d’inpainting vidéo, nous divisons notre travail en deux catégories basées sur le type de fond. En particulier, nous présentons une méthode simple de propagation de pixels guidée par le mouvement pour traiter les cas d’arrière-plan statiques. Nous montrons que le problème de la suppression d'objets avec un arrière-plan statique peut être résolu efficacement en utilisant une technique simple basée sur le mouvement. Pour traiter le fond dynamique, nous introduisons la méthode d’inpainting vidéo en optimisant une fonction d’énergie globale basée sur des patchs. Pour augmenter la vitesse de l'algorithme, nous avons proposé une extension parallèle de l'algorithme 3D PatchMatch. Pour améliorer la précision, nous intégrons systématiquement le flux optique dans le processus global. Nous nous retrouvons avec une méthode d’inpainting vidéo capable de reconstruire des objets en mouvement ainsi que de reproduire des textures dynamiques tout en fonctionnant dans des délais raisonnables.Enfin, nous combinons les méthodes de segmentation des objets vidéo et d’inpainting vidéo dans un système unifié pour supprimer les objets non souhaités dans les vidéos. A notre connaissance, il s'agit du premier système de ce type. Dans notre système, l'utilisateur n'a qu'à délimiter approximativement dans le premier cadre les objets à modifier. Ce processus d'annotation est facilité par l'aide de superpixels. Ensuite, ces annotations sont affinées et propagées dans la vidéo par la méthode de segmentation des objets vidéo. Un ou plusieurs objets peuvent ensuite être supprimés automatiquement à l’aide de nos méthodes d’inpainting vidéo. Il en résulte un outil de montage vidéo informatique flexible, avec de nombreuses applications potentielles, allant de la suppression de la foule à la correction de scènes non physiques. / Nowadays, the rapid increase of video creates a massive demand for video-based editing applications. In this dissertation, we solve several problems relating to video post-processing and focus on objects removal application in video. To complete this task, we divided it into two problems: (1) A video objects segmentation step to select which objects to remove and (2) a video inpainting step to filling the damaged regions.For the video segmentation problem, we design a system which is suitable for object removal applications with different requirements in terms of accuracy and efficiency. Our approach relies on the combination of Convolutional Neural Networks (CNNs) for segmentation and the classical mask tracking method. In particular, we adopt the segmentation networks for image case and apply them to video case by performing frame-by-frame segmentation. By exploiting both offline and online training with first frame annotation only, the networks are able to produce highly accurate video object segmentation. Besides, we propose a mask tracking module to ensure temporal continuity and a mask linking module to ensure the identity coherence across frames. Moreover, we introduce a simple way to learn the dilation layer in the mask, which helps us create suitable masks for video objects removal application.For the video inpainting problem, we divide our work into two categories base on the type of background. In particular, we present a simple motion-guided pixel propagation method to deal with static background cases. We show that the problem of objects removal with a static background can be solved efficiently using a simple motion-based technique. To deal with dynamic background, we introduce video inpainting method by optimization a global patch-based energy function. To increase the speed of the algorithm, we proposed a parallel extension of the 3D PatchMatch algorithm. To improve accuracy, we systematically incorporate the optical flow in the overall process. We end up with a video inpainting method which is able to reconstruct moving objects as well as reproduce dynamic textures while running in a reasonable time.Finally, we combine the video objects segmentation and video inpainting methods into a unified system to removes undesired objects in videos. To the best of our knowledge, this is the first system of this kind. In our system, the user only needs to approximately delimit in the first frame the objects to be edited. These annotation process is facilitated by the help of superpixels. Then, these annotations are refined and propagated through the video by the video objects segmentation method. One or several objects can then be removed automatically using our video inpainting methods. This results in a flexible computational video editing tool, with numerous potential applications, ranging from crowd suppression to unphysical scenes correction.

A popularização de aplicativos e dispositivos capazes de produzir, exibir e editar conteúdos multimídia fez surgir a necessidade de se adaptar, modificar e customizar diferentes tipos de mídia a diferentes necessidades do usuário. Nesse contexto, a área de Personalização e Adaptação de Conteúdo busca desenvolver soluções que atendam a tais necessidades. Sistemas de personalização, em geral, necessitam conhecer os dados presentes na mídia, surgindo, assim, a necessidade de uma indexação do conteúdo presente na mídia. No caso de vídeo digital, os esforços para a indexação automática utilizam como passo inicial a segmentação de vídeos em unidades de informação menores, como tomadas e cenas. A segmentação em cenas, em especial, é um desafio para pesquisadores graças a enorme variedade entre os vídeos e a própria ausência de um consenso na definição de cena. Diversas técnicas diferentes para a segmentação em cenas são reportadas na literatura. Uma técnica, em particular, destaca-se pelo baixo custo computacional: a técnica baseada em coerências visual. Utilizando-se operações de histogramas, a técnica objetiva-se a comparar tomadas adjacentes em busca de similaridades que poderiam indicar a presença de uma cena. Para melhorar os resultados obtidos, autores de trabalhos com tal enfoque utilizam-se de outras características, capazes de medir a \"quantidade de movimento\" das cenas, como os vetores de movimento. Assim, este trabalho apresenta uma técnica de segmentação de vídeo digital em tomadas e em cenas através da coerência visual e do fluxo óptico. Apresenta-se, ainda, uma série de avaliações de eficácia e de desempenho da técnica ao segmentar em tomadas e em cenas uma base de vídeo do domínio filmes / The popularization of applications and devices capable of producing, displaying and editing multimedia content did increase the need to adapt, modify and customize different types of media for different user needs. In this context, the area of Personalization and Content Adaptation seeks to develop solutions that meet these needs. Personalization systems, in general, need to know the data present in the media, thus needing for a media indexing process. In the case of digital video, the efforts for automatic indexing usually involves, as an initial step, to segment videos into smaller information units, such as shots and scenes. The scene segmentation, in particular, is a challenge to researchers due to the huge variety among the videos and the very absence of a consensus on the scene definition. Several scenes segmentation techniques are reported in the literature. One technique in particular stands out for its low computational cost: those techniques based on visual coherence. By using histograms, the technique compares adjacent shots to find similar shots which may indicate the presence of a scene. To improve the results, some related works uses other features to evaluate the motion dynamics of the scenes using features such as motion vectors. In this sense, this work presents a digital video segmentation technique for shots and scenes, using visual coherence and optical flow as its features. It also presents a series of evaluation in terms of effectiveness and performance of the technique when segmenting scenes and shots of a custom video database of the film domain

In this study, an automatic video segmentation and classification system based on audio features has been presented. Video sequences are classified such as videos with &ldquo / speech&rdquo / , &ldquo / music&rdquo / , &ldquo / crowd&rdquo / and &ldquo / silence&rdquo / . The segments that do not belong to these regions are left as &ldquo / unclassified&rdquo / . For the silence segment detection, a simple threshold comparison method has been done on the short time energy feature of the embedded audio sequence. For the &ldquo / speech&rdquo / , &ldquo / music&rdquo / and &ldquo / crowd&rdquo / segment detection a multiclass classification scheme has been applied. For this purpose, three audio feature set have been formed, one of them is purely MPEG-7 audio features, other is the audio features that is used in [31] the last one is the combination of these two feature sets. For choosing the best feature a histogram comparison method has been used. Audio segmentation system was trained and tested with these feature sets. The evaluation results show that the Feature Set 3 that is the combination of other two feature sets gives better performance for the audio classification system. The output of the classification system is an XML file which contains MPEG-7 audio segment descriptors for the video sequence. An application scenario is given by combining the audio segmentation results with visual analysis results for getting audio-visual video segments.

Title from PDF of title page. / Document formatted into pages; contains 76 pages. / Thesis (M.S.E.E.)--University of South Florida, 2003. / Includes bibliographical references. / Text (Electronic thesis) in PDF format. / ABSTRACT: The use of digitized information is rapidly gaining acceptance in bio-medical applications. Video compression plays an important role in the archiving and transmission of different digital diagnostic modalities. The present scheme of video compression for low bit-rate networks is not suitable for medical video sequences. The instability is the result of block artifacts resulting from the block based DCT coefficient quantization. The possibility of applying deformable motion estimation techniques to make the video compression standard (H.263) more adaptable for bio-medial applications was studied in detail. The study on the network characteristics and the behavior of various congestion control mechanisms was used to analyze the complete characteristics of existing low bit rate video compression algorithms. The study was conducted in three phases. The first phase involved the implementation and study of the present H.263 compression standard and its limitations. / ABSTRACT: The second phase dealt with the analysis of an external force for active contours which was used to obtain estimates for deformable objects. The external force, which is termed Gradient Vector Flow (GVF), was computed as a diffusion of the gradient vectors associated with a gray-level or binary edge map derived from the image. The mathematical aspect of a multi-scale framework based on a medial representation for the segmentation and shape characterization of anatomical objects in medical imagery was derived in detail. The medial representations were based on a hierarchical representation of linked figural models such as protrusions, indentations, neighboring figures and included figures--which represented solid regions and their boundaries. The third phase dealt with the vital parameters for effective video streaming over the internet in the bottleneck bandwidth, which gives the upper limit for the speed of data delivery from one end point to the other in a network. / ABSTRACT: If a codec attempts to send data beyond this limit, all packets above the limit will be lost. On the other hand, sending under this limit will clearly result in suboptimal video quality. During this phase the packet-drop-rate (PDR) performance of TCP(1/2) was investigated in conjunction with a few representative TCP-friendly congestion control protocols (CCP). The CCPs were TCP(1/256), SQRT(1/256) and TFRC (256), with and without self clocking. The CCPs were studied when subjected to an abrupt reduction in the available bandwidth. Additionally, the investigation studied the effect on the drop rates of TCP-Compatible algorithms by changing the queuing scheme from Random Early Detection (RED) to DropTail. / System requirements: World Wide Web browser and PDF reader. / Mode of access: World Wide Web.

No description available.

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior / Video-based interactive storytelling has as its main goal the generation of interactive narratives, allowing users to control the course of the story and using cinematographic compositions to dramatize it. For this process to be possible, there is a need for large amounts of cinematographic content in the form of filmed scenes. This content, on the other hand, has to be properly pre-processed in order to be usable. This work proposes an approach for video segmentation aimed for video-based interactive storytelling, using alpha matting to extract color and transparency of video elements to be later recomposed for dramatization purposes. The developed solution uses background subtraction and energy minimization techniques to automatically generate trimaps. / O storytelling interativo baseado em vídeo tem como objetivo a geração de narrativas, permitindo o controle de um usuário sobre o rumo da estória e utilizando composições cinematográficas para dramatizá-la. Para que este processo seja possível, existe a necessidade de uma grande quantidade de conteúdo cinematográfico na forma de filmagens. Este conteúdo, por sua vez, precisa ser adequadamente pré-processado para permitir sua utilização adequada. Nesse trabalho, uma abordagem para segmentação de vídeos para storytelling interativo baseado em vídeo é proposta, utilizando alpha matting para extrair informações de cor e transparência de elementos de vídeos para serem reutilizados em processos de dramatização. A solução desenvolvida utiliza técnicas de subtração de fundo e minimização de energia para gerar trimaps de forma automática.