University of Macedonia Economic and Social Sciences
Αυτόματη σημασιολογική ταξινόμηση εικονοσειρών με τη χρήση χωρικών και χρονικών περιγραφέων: θεωρητική ανάλυση και εφαρμογή σε πραγματικά δεδομένα
Abstract
dc:descriptionDuring the last decades, digital multimedia recording devices have become increasingly cheap and widespread, leading to an abrupt worldwide accumulation of huge volumes of multimedia material. At the same time, the success of open publication web sites, as well as the significant increase in the number of specialized digital multimedia collections, such as TV archives or museum catalogues, has made the need for indexing such databases imperative. To this day, indexing methods fundamentally rely on the manual annotation of multimedia material. Research in Automatic Semantic Indexing aims at the creation of systems able to automatically detect the presence of high-level concepts within multimedia items, so as to reduce the cost of indexing and overcome the limitations of manual annotation. Our research focuses on low-level visual descriptors for video shots for the purpose Semantic Video Indexing. In chapter 1 we describe the aims and challenges of semantic indexing, briefly review past related work and contemporary research directions, and present the aims of this dissertation. Chapter 2 describes a number of fundamental concepts that are necessary for the reader to follow our work. In chapter 3, we exhaustively review and analyse previously proposed descriptors for color, texture, local features οn motion, while the dominant approaches in shape description are also briefly touched upon. For each descriptor, we present the underlying rationale, as well as its respective strengths and weaknesses, placing emphasis on the results of published experimental comparisons, wherever these are available. Following that analysis, in chapter 4 we propose three novel motion descriptors, each corresponding to a different aspect of motion information. Our first descriptor focuses on camera behaviour, emphasizing on the temporal relationships between different camera operations. These relationships concern both the possibility of coexistence of multiple camera operations at the same time, and the potential transitions between different consecutive camera operations. The second descriptor is derived from an attempt to improve a classic foreground motion descriptor, by replacing sparse motion fields with dense optical flow as its base information. As a further improvement, we shift the descriptors focus from the absolute local intensity of foreground motion, to its relative spatial distribution. Finally, our third descriptor aims at representing relationships between neighbouring motion vectors through Local Binary Patterns. As it constitutes an entirely novel descriptor for dynamic texture through local motion information, a series of possible variants are explored with respect to its structure. In chapter 5 we present a proposal for the application of the VACOR algorithm, from the field of Data Analysis, for the evaluation of descriptors and the analysis of each individual variables role in semantic content-based classification. In terms of experimental evaluations, we attempt data-driven Hierarchical Classification of a set of video shots from three semantic classes. Following an evaluation of the clustering of the three classes, an exhaustive analysis of the higher nodes of the classification tree is performed, where each descriptor variable is studied with respect to its role in the resulting classification. Besides being a first-level evaluation of our proposeddescriptors, this application serves as a broader example for descriptor evaluation using Data Analysis methods. In chapter 6, Semantic Classification is attempted using Support Vector Machines. Sixteen semantic classes are defined in a database of 1074 video shots, and a number of variants of our proposed descriptors are evaluated with respect to their relative performance in indexing unknown video shots. Consecutively, our descriptors are experimentally compared to their state-of-the-art counterparts, in order to measure the relative improvement achieved. Finally, we implement an integrated system for semantic video classification, based on a large number of state-of-the-art descriptors, selected from the analysis of chapter 3. Within the context of this system, we evaluate the contribution of our novel descriptors in improving the overall system performance, as well as the overall classification success of the final system incorporating both the classic and novel descriptors. Based on the experimental results, and the observed strengths and weaknesses of ourdescriptors, in chapter 7 we reach a series of conclusions concerning the degree of success of our project, the significance of our contributions, and the potential future directions of our research.
Degree
thesis:*- Grantor dc:publisher
- University of Macedonia Economic and Social Sciences
- Year dc:date
- 2011
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Zampoglou, Markos
- Ζάμπογλου, Μάρκος
Subjects
dc:subject × 12- Ταξινόμηση εικονοσειρών
- Οπτικοί περιγραφείς
- Σημασιολογική ταξινόμηση
- Semantic video indexing
- Visual descriptors
- Video concept detection
- Content-based video retrieval
- High-level feature extraction
- Φυσικές Επιστήμες
- Επιστήμη Ηλεκτρονικών Υπολογιστών και Πληροφορική
- Natural Sciences
- Computer and Information Sciences
Rights
- Language dc:language
- gre
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/24404