Aristotle University Of Thessaloniki (AUTH)
Τεχνικές οπτικοακουστικής επεξεργασίας και ανάλυσης της διαμεσικής επικοινωνίας
Abstract
dc:descriptionSemantic web, smart mobile devices and cloud services are some of the concepts that define the modern technological landscape. In this context, new trends for infotainment and communication have been established and led to the rapid increase of capturing, producing and distributing multimedia content. This content is created both by professionals and everyday users and has affected well-established everyday processes or introduced new workflows for professionals. Meanwhile, real-time video communication has become a mainstream operation, while instant messaging services embrace rich media content, such as short audio and video clips. The continuous development of new communication infrastructures has led to the rapid increase of synchronous and asynchronous types of mediated communication, increasing the expectations of the underlying software infrastructure, including the field of semantic audio-visual analysis. The development of automated processes becomes a necessity, so as to ensure high-level user experience and effectiveness of managing this huge amount of data.From another point of view, accessing this mass-produced content enables new opportunities for advancing into the research field of semantic audiovisual analysis. The relevant algorithms are ubiquitous in contemporary multimedia applications and are applied to diverse machine learning tasks. Concerning semantic audio analysis, there is a variety of implementations that deal with speech recognition, emotion recognition, speaker diarisation or music information retrieval. Speech, music and noise content is dominant in the encountered classification taxonomies. On the other hand, visual analysis targets at human-related tasks, such as face recognition, speaker diarization, speaker localization or voice activity detection. There is no doubt that expectations, both in terms of pattern recognition accuracy and reduced computational requirements are high. Thus, algorithms should be under continuous review and revision. At the same time, the intense research activity in the field of artificial intelligence empowers all these algorithms.This thesis proposes an agile model for multimedia organization, incorporating various algorithmic units for supporting the automated processes of semantic content analysis. This approach intends to facilitate the construction, validation, and continuous update of ground-truth data, aiming at delivering consistent behavior under different circumstances. Specifically, a framework for generic video detection and classification, combining hierarchical bimodal segmentation with hybrid pattern classification is introduced as the algorithmic architecture of a mobile back-end system. An advancement of statistical feature integration is presented, attempting to fit data through the alpha-stable distribution. In addition to this, an enhanced temporal integration methodology is also introduced, establishing a solid and reduced-complexity feature-engine. Furthermore, the research focuses on optimizing lightweight configurations for convolutional network topologies that can be deployed on audio information retrieval tasks, such as voice activity detection, speaker diarization or speech emotion recognition. The outmost target of this effort is to set a protocol for developing deep convolutional topologies for audio classification tasks, minimizing complexity and computational needs. Along with the audio-related deep architectures, a convolutional long-short term memory neural topology is deployed for performing visual voice activity detection analyses. Finally, a multimodal workflow for localizing active speakers in video streams is developed, based on multimodal, audiovisual analysis and a fuzzy inference mechanism.
Degree
thesis:*- Grantor dc:publisher
- Aristotle University Of Thessaloniki (AUTH)
- Year dc:date
- 2019
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Lazaros, Vrysis
- Βρύσης, Λάζαρος
Subjects
dc:subject × 19- Οπτική επεξεργασία σήματος
- Στατιστική επεξεργασία σήματος
- Μηχανική μάθηση και Αναγνώριση προτύπων
- Βαθιά μάθηση
- Ηχητική κατάτμηση
- Αναγνώριση γενικευμένου ακουστικού σήματος
- Υπολογιστική νέφους
- Semantic visual content analysis
- Audio classification
- Short-term audio signal feature based analysis
- Audio segmentation
- Mobile cloud computing
- Deep learning
- Machine learning
- Semantic annotation of multimedia content
- Επιστήμες Μηχανικού και Τεχνολογία
- Επιστήμη Ηλεκτρολόγου Μηχανικού, Ηλεκτρονικού Μηχανικού, Μηχανικού Η/Υ
- Engineering and Technology
- Electrical Engineering, Electronic Engineering, Information Engineering
Rights
dc:rights- Statement dc:rights
-
- BY_NC_ND
- Language dc:language
- gre
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/46114