A área de recuperação de informação baseada em conteúdo visual vem ganhando importância graças ao volume de material visual existente (imagens e vídeo digitais), compartilhado e distribuído principalmente via Internet, e à capacidade de processamento alcançada pelos computadores pessoais na última década. Novas formas de consumo, manipulação e exploração de vídeo digital podem ser criadas através da organização e indexação apropriada desse material. A delimitação de tomadas fornece uma base para a abstração e estruturação de vídeo, agregando quadros contíguos em seqüências de mesmo contexto, isto é, trechos com unidade em termos de tempo e espaço. Nesta dissertação são apresentados os conceitos básicos de delimitação de tomadas e métodos tradicionais utilizados nesse tipo de segmentação, bem como vários resultados experimentais obtidos a partir de seqüências reais de TV. É analisada a distribuição das diferenças entre quadros sucessivos, calculada através de seus histogramas, na tentativa de caracterizar as transições entre tomadas e obter melhores parâmetros para a segmentação. Obtêm-se experimentalmente mais evidências que comprovam a superioridade da medida de intersecção de histogramas sobre outras medidas. A principal contribuição do trabalho consiste no desenvolvimento de um algoritmo baseado no método twin-comparison, que apresenta melhor desempenho que o método original na detecção dos limites de tomadas por utilizar análise local da variação visual entre os quadros do vídeo. / Visual content based information retrieval is an area of increasing importance due to the large volume of available material (digital images and videos), shared and distributed mainly by the internet, and the processing power achieved by personal computer in the last ten years. New ways to consume digital video and to manipulate and explore its visual information can be made by appropriately organizing and indexing this material. The shot boundary detection is a fundamental tool to video abstraction and structuring, combining near frames into sequences with similar context, segments with space and time unity. This work presents the basic concepts about shot boundary detection, traditional methods used and several experimental results obtained from a real TV data set. The distribution of differences of neighboring frames, calculated from histogram comparison, is used to define the transitions between frames and to obtain better parameters for segmentation. Our experimental results show the superiority of the histogram intersection method over other measures. Our main contribution is the development of a new algorithm based on the twin-comparison method, extended with local analysis of visual content variation between video frames. This algorithm was tested over hours of TV data, and performs better than the original method.

A área de recuperação de informação baseada em conteúdo visual vem ganhando importância graças ao volume de material visual existente (imagens e vídeo digitais), compartilhado e distribuído principalmente via Internet, e à capacidade de processamento alcançada pelos computadores pessoais na última década. Novas formas de consumo, manipulação e exploração de vídeo digital podem ser criadas através da organização e indexação apropriada desse material. A delimitação de tomadas fornece uma base para a abstração e estruturação de vídeo, agregando quadros contíguos em seqüências de mesmo contexto, isto é, trechos com unidade em termos de tempo e espaço. Nesta dissertação são apresentados os conceitos básicos de delimitação de tomadas e métodos tradicionais utilizados nesse tipo de segmentação, bem como vários resultados experimentais obtidos a partir de seqüências reais de TV. É analisada a distribuição das diferenças entre quadros sucessivos, calculada através de seus histogramas, na tentativa de caracterizar as transições entre tomadas e obter melhores parâmetros para a segmentação. Obtêm-se experimentalmente mais evidências que comprovam a superioridade da medida de intersecção de histogramas sobre outras medidas. A principal contribuição do trabalho consiste no desenvolvimento de um algoritmo baseado no método twin-comparison, que apresenta melhor desempenho que o método original na detecção dos limites de tomadas por utilizar análise local da variação visual entre os quadros do vídeo. / Visual content based information retrieval is an area of increasing importance due to the large volume of available material (digital images and videos), shared and distributed mainly by the internet, and the processing power achieved by personal computer in the last ten years. New ways to consume digital video and to manipulate and explore its visual information can be made by appropriately organizing and indexing this material. The shot boundary detection is a fundamental tool to video abstraction and structuring, combining near frames into sequences with similar context, segments with space and time unity. This work presents the basic concepts about shot boundary detection, traditional methods used and several experimental results obtained from a real TV data set. The distribution of differences of neighboring frames, calculated from histogram comparison, is used to define the transitions between frames and to obtain better parameters for segmentation. Our experimental results show the superiority of the histogram intersection method over other measures. Our main contribution is the development of a new algorithm based on the twin-comparison method, extended with local analysis of visual content variation between video frames. This algorithm was tested over hours of TV data, and performs better than the original method.

This thesis aims comparative analysis of the state of the art shot boundary detection algorithms. The major methods that have been used for shot boundary detection such as pixel intensity based, histogram-based, edge-based, and motion vectors based, are implemented and analyzed. A recent method which utilizes &ldquo / graph partition model&rdquo / together with the support vector machine classifier as a shot boundary detection algorithm is also implemented and analyzed. Moreover, a novel graph theoretic concept, &ldquo / dominant sets&rdquo / , is also successfully applied to the shot boundary detection problem as a contribution to the solution domain.

In this thesis, a mixed type video segmentation algorithm is implemented to find the scene cuts in MPEG compressed video data. The main aim is to have a computationally efficient algorithm for real time applications. Due to this reason partial decoding of the bitstream is used in segmentation. As a result of partial decoding, features such as bitrate, motion vector type, and DC images are implemented to find both continuous and discontinuous scene cuts on a MPEG-2 coded general TV broadcast data. The results are also compared with techniques found in literature.

Recent research approaches in semantics based video content analysis require shot boundary detection as the first step to divide video sequences into sections. Furthermore, with the advances in networking and computing capability, efficient retrieval of multimedia data has become an important issue. Content-based retrieval technologies have been widely implemented to protect intellectual property rights (IPR). In addition, automatic recognition of highlights from videos is a fundamental and challenging problem for content-based indexing and retrieval applications. In this thesis, a paradigm is proposed to segment, retrieve and interpret digital videos. Five algorithms are presented to solve the video segmentation task. Firstly, a simple shot cut detection algorithm is designed for real-time implementation. Secondly, a systematic method is proposed for shot detection using content-based rules and FSM (finite state machine). Thirdly, the shot detection is implemented using local and global indicators. Fourthly, a context awareness approach is proposed to detect shot boundaries. Fifthly, a fuzzy logic method is implemented for shot detection. Furthermore, a novel analysis approach is presented for the detection of video copies. It is robust to complicated distortions and capable of locating the copy of segments inside original videos. Then, iv objects and events are extracted from MPEG Sequences for Video Highlights Indexing and Retrieval. Finally, a human fighting detection algorithm is proposed for movie annotation.

Development of Internet and its massive spread resulted in increased volume of multimedia data. The increase in the amount of multimedia data raises the need for efficient similarity detection between multimedia files for the purpose of preventing and detecting violations of copyright licenses or for detection of similar or duplicate files. This thesis discusses the current options in the field of the content-based image and video comparison and focuses on the feature extraction techniques, distance metrics, design and implementation of the mediaDiff application module for the content-based comparison of video files.

This thesis focuses on four main research themes namely shot boundary detection, fast frame alignment, activity-driven video summarisation, and highlights based video annotation and retrieval. A number of novel algorithms have been proposed to address these issues, which can be highlighted as follows. Firstly, accurate and robust shot boundary detection is achieved through modelling of cuts into sub-categories and appearance based modelling of several gradual transitions, along with some novel features extracted from compressed video. Secondly, fast and robust frame alignment is achieved via the proposed subspace phase correlation (SPC) and an improved sub-pixel strategy. The SPC is proved to be insensitive to zero-mean-noise, and its gradient-based extension is even robust to non-zero-mean noise and can be used to deal with non-overlapped regions for robust image registration. Thirdly, hierarchical modelling of rush videos using formal language techniques is proposed, which can guide the modelling and removal of several kinds of junk frames as well as adaptive clustering of retakes. With an extracted activity level measurement, shot and sub-shot are detected for content-adaptive video summarisation. Fourthly, highlights based video annotation and retrieval is achieved, in which statistical modelling of skin pixel colours, knowledge-based shot detection, and improved determination of camera motion patterns are employed. Within these proposed techniques, one important principle is to integrate various kinds of feature evidence and to incorporate prior knowledge in modelling the given problems. High-level hierarchical representation is extracted from the original linear structure for effective management and content-based retrieval of video data. As most of the work is implemented in the compressed domain, one additional benefit is the achieved high efficiency, which will be useful for many online applications. / EU IST FP6 Project

Recent research approaches in semantics based video content analysis require shot boundary detection as the first step to divide video sequences into sections. Furthermore, with the advances in networking and computing capability, efficient retrieval of multimedia data has become an important issue. Content-based retrieval technologies have been widely implemented to protect intellectual property rights (IPR). In addition, automatic recognition of highlights from videos is a fundamental and challenging problem for content-based indexing and retrieval applications. In this thesis, a paradigm is proposed to segment, retrieve and interpret digital videos. Five algorithms are presented to solve the video segmentation task. Firstly, a simple shot cut detection algorithm is designed for real-time implementation. Secondly, a systematic method is proposed for shot detection using content-based rules and FSM (finite state machine). Thirdly, the shot detection is implemented using local and global indicators. Fourthly, a context awareness approach is proposed to detect shot boundaries. Fifthly, a fuzzy logic method is implemented for shot detection. Furthermore, a novel analysis approach is presented for the detection of video copies. It is robust to complicated distortions and capable of locating the copy of segments inside original videos. Then, iv objects and events are extracted from MPEG Sequences for Video Highlights Indexing and Retrieval. Finally, a human fighting detection algorithm is proposed for movie annotation.

This thesis focuses on four main research themes namely shot boundary detection, fast frame alignment, activity-driven video summarisation, and highlights based video annotation and retrieval. A number of novel algorithms have been proposed to address these issues, which can be highlighted as follows. Firstly, accurate and robust shot boundary detection is achieved through modelling of cuts into sub-categories and appearance based modelling of several gradual transitions, along with some novel features extracted from compressed video. Secondly, fast and robust frame alignment is achieved via the proposed subspace phase correlation (SPC) and an improved sub-pixel strategy. The SPC is proved to be insensitive to zero-mean-noise, and its gradient-based extension is even robust to non-zero-mean noise and can be used to deal with non-overlapped regions for robust image registration. Thirdly, hierarchical modelling of rush videos using formal language techniques is proposed, which can guide the modelling and removal of several kinds of junk frames as well as adaptive clustering of retakes. With an extracted activity level measurement, shot and sub-shot are detected for content-adaptive video summarisation. Fourthly, highlights based video annotation and retrieval is achieved, in which statistical modelling of skin pixel colours, knowledge-based shot detection, and improved determination of camera motion patterns are employed. Within these proposed techniques, one important principle is to integrate various kinds of feature evidence and to incorporate prior knowledge in modelling the given problems. High-level hierarchical representation is extracted from the original linear structure for effective management and content-based retrieval of video data. As most of the work is implemented in the compressed domain, one additional benefit is the achieved high efficiency, which will be useful for many online applications.