Back to search

National and Kapodistrian University of Athens

Πολυτροπική ολοκλήρωση υψηλού επιπέδου και εξαγωγή σημασιολογίας

Abstract

dc:description

This thesis on multimodal fusion and semantics extraction, focuses on automated detection and annotation of harmful content in video data. The aim is not only to reason out the existence of violence or not (i.e., the binary problem), but also to determine the type of violence (e.g., fight, explosion, murder). An introductory comparative study of several harmful content detection approaches is first presented and the lack of knowledge representation and reasoning approaches for the problem at hand is acknowledged. Towards filling this gap, we propose a semantic fusion approach that combines low to mid level modality specific semantics through ontological and rule reasoning. A major part of the proposed framework is the movie segmentation into meaningful and easy to handle units. Towards this direction, we investigate and evaluate a set of shot boundary detection approaches and their combination through a majority voting scheme. In the sequel, state of the art classification methods are employed to extract mid level semantics. The segmentation module and the modality specific analysis algorithms instantiate the corresponding video structure and modality specific ontologies developed in the context of the knowledge engineering framework. A set of consecutive and interleaved ontological and SWRL rule reasoning steps map sets and sequences of extracted semantics into higher level concepts represented in the harmful content ontology. We present the involved ontologies, the corresponding SWRL rule sets and the reasoning mechanism in detail. A comparative study of the most common ontological reasoners for the realization procedure of the semantic framework and the classification of the harmful content ontology is presented in detail. In the final part of the thesis we evaluate the proposed approach in a preanotated movie dataset and compare its results with the single modality approaches and a kNN late fusion classifier. We comment on the higher level semantics extraction ability and evaluate a set of extensions employed in the basic structure of the framework. The extensions concern the development of a scene detection module that combines Markov clustering with SQWRL queries and the incorporation of existing rating and movie genre metadata in the reasoning process.

Degree

thesis:*
Grantor dc:publisher
National and Kapodistrian University of Athens
Year dc:date
2011

Author and committee

dc:creator, dc:contributor.*
Authors dc:creator
  • Perperis, Thanasis
  • Περπερής, Αθανάσιος

Subjects

dc:subject × 14

Rights

Language dc:language
gre

Identifiers

dc:identifier.*
Identifier
10.12681/eadd/26721

Chain of custody

source
Harvested from
Greek National Archive of PhD Theses
Base URL
phdtheses.ekt.gr/eadd_oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Perperis, Thanasis; Περπερής, Αθανάσιος. Πολυτροπική ολοκλήρωση υψηλού επιπέδου και εξαγωγή σημασιολογίας. National and Kapodistrian University of Athens, 2011. http://hdl.handle.net/10442/hedi/26721