Aristotle University Of Thessaloniki (AUTH)
Σημασιολογική ανάλυση σε πολυμεσικά δεδομένα με μηχανική μάθηση
Abstract
dc:descriptionThis work studies the problem of semantic analysis of multimedia data. Given a segment of a multimedia document and a set of clearly defined concepts, our goal is to compute the probabilities that each concept exists in the input multimedia data. We follow the machine learning approach, where multiple low-level features are extracted from the content, which are subsequently fused, normalized and processed before the computation of probabilities by a classifier. These probabilities are used for the indexing and retrieval of multimedia data, even if the data have not been manually annotated. We first examine the use of classifiers that have been trained at foreign application domains for improving retrieval effectiveness. We propose a concept fusion scheme where a layer of base classifiers is used. These classifiers have been trained at a foreign application domain for each low-level feature separately. Using their outputs we train a final classifier that computes the output probabilities. We introduce the Maximum Information and Top-k criteria for selecting the base classifiers. The effectiveness of the proposed fusion scheme is experimentally evaluated on two large scale multimedia datasets (TRECVID-2005 and a collection of images from the Belga news agency), for a large number of concepts. We subsequently seek ways to replace the base classifiers with “property detectors”, that are simply probability distributions in the feature space. Our approach is to partition the feature space and assign one distribution to each partition subset, where the distribution is defined parametrically from the data. A set of experiments is performed where the proposed method increases the effectiveness in both classification and multimedia retrieval problems. Furthermore, we examine a set of problems that are related to the practical application of the detection system for a large number of concepts. We propose a protocol for concept selection and disambiguation, as well as the generation of the classifier training sets. This protocol was applied efficiently for more than 500 concepts. Furthermore, we evaluate a set of methods for the automatic construction of training sets using clickthrough data. A novel audio low-level feature is also proposed, that allows the effective detection of concepts that are related to the audio channel. Finally, a short report is included about two systems that achieved competitive results at the TRECVID large scale international video retrieval benchmark.
Degree
thesis:*- Grantor dc:publisher
- Aristotle University Of Thessaloniki (AUTH)
- Year dc:date
- 2010
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Diou, Christos
- Δίου, Χρήστος
Subjects
dc:subject × 16- Ανάκτηση πολυμέσων
- Μηχανική μάθηση
- Ανίχνευση εννοιών
- Μετασχηματισμός χαρακτηριστικών
- Παραγωγή συνόλων εκπαίδευσης
- Εξαγωγή χαρακτηριστικών ήχου
- Multimedia retrieval
- Machine learning
- Concept detection
- Feature space transformation
- Training set generation
- Audio feature extraction
- Επιστήμες Μηχανικού και Τεχνολογία
- Επιστήμη Ηλεκτρολόγου Μηχανικού, Ηλεκτρονικού Μηχανικού, Μηχανικού Η/Υ
- Engineering and Technology
- Electrical Engineering, Electronic Engineering, Information Engineering
Rights
- Language dc:language
- gre
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/23671