Back to search

Aristotle University Of Thessaloniki (AUTH)

Συστήματα επεξεργασίας οπτικοακουστικών πόρων

Abstract

dc:description

This work introduces and evaluates new methodologies for audiovisual content classification and retrieval, in media asset management systems. The proposed methodologies focus on the unimodal and multimodal semantic analysis of audiovisual content with emphasis on the employment of machine learning techniques. A new media asset management model, supporting the incorporation of semantic analysis algorithms on the basis of a hierarchical architecture, is introduced. Additionally, a collaborative annotation method based on the user social interaction is introduced, aiming at the development of large-scale, annotated ground truth datasets. In the context of the proposed hierarchical architecture, a number of novel audio-driven content analysis methodologies are presented and evaluated. A new method for audiovisual content segmentation based on speech / music discrimination is developed. The proposed algorithm relies on the hypothesis that short-time frame-level discrimination performance can be enhanced by identifying transition points between longer, semantically homogeneous segments of audio. As part of it, a two-step segmentation approach is employed to identify transition points between the homogeneous regions and subsequently classify the derived segments using a supervised binary classifier. The algorithm is evaluated against three publicly available datasets and experimental results are presented in comparison with existing state-of-the-art implementations. Furthermore, results from the MIREX 2015 competition, where the proposed system won the third place, are presented and discussed. Afterwards, a music structure segmentation algorithm, relying on the detection of transition points between repeated sections of content, is introduced. Vector-quantization is employed as a novel adaptive filtering mechanism for the extraction of the repeated time-series data. The method is evaluated against 124 pop songs from the INRIA Eurovision dataset and in comparison with an existing state-of-the-art music structure analysis algorithm. Finally, a multimodal speaker diarization approach, based on deep neural network embeddings, is proposed. A novel Long Short-term Memory, Siamese neural network is employed for the analysis of the audio mode, whereas a pre-trained convolutional neural network is deployed for the analysis of the visual modality. A multimodal fusion stage is introduced to enhance audio-driven speaker diarization performance by exploiting information derived from the visual input. The proposed method is assessed against two publicly available datasets and results are presented in comparison with an existing state-of-the-art algorithm.

Degree

thesis:*
Grantor dc:publisher
Aristotle University Of Thessaloniki (AUTH)
Year dc:date
2018

Author and committee

dc:creator, dc:contributor.*
Authors dc:creator
  • Tsipas, Nikolaos
  • Τσίπας, Νικόλαος

Subjects

dc:subject × 20

Rights

Language dc:language
gre

Identifiers

dc:identifier.*
Identifier
10.12681/eadd/44825

Chain of custody

source
Harvested from
Greek National Archive of PhD Theses
Base URL
phdtheses.ekt.gr/eadd_oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Tsipas, Nikolaos; Τσίπας, Νικόλαος. Συστήματα επεξεργασίας οπτικοακουστικών πόρων. Aristotle University Of Thessaloniki (AUTH), 2018. http://hdl.handle.net/10442/hedi/44825