Aristotle University Of Thessaloniki (AUTH)
Αυτοματισμοί ανάλυσης και διαχείρισης οπτικοακουστικών ροών στα ψηφιακά μέσα και στη διαμεσική επικοινωνία
Abstract
dc:descriptionDigital media are going through a phase of intense developments and transformations. Advanced content production and processing capabilities combined with the connectivity and popularity of social media have led to an explosion in data production. The need for management of huge volumes of heterogeneous data, has set new challenges in terms of storage, processing and accessing data, leading to the development of the scientific field of big data. The abundance of available data and the evolution of tools and computing capabilities have created favorable conditions for the development of new strategies for machine and deep learning. Training models with a myriad of parameters allows for enhanced performance and robustness at levels that allow them to be used in real-world problems.This dissertation examines the integration of machine learning technologies into routine workflows in the production, collection, documentation, and distribution of audiovisual data, automating processes and augmenting content. An operating framework is specified that includes all stages of content production and distribution. Web radio and television production systems implementing the framework’s directions are presented. A radio production application is proposed, integrating automated annotation processes, as well as segmentation and transcription of unstructured audio streams. Users can search broadcasts through content-based queries and navigate the timeline through an interactive environment. For web TV, the active speaker is detected and used for camera shot selection, automating the directing process. Additional issues that have an important place in the production, collection and distribution of content are explored. One function with particular relevance in journalism is sentiment analysis. In audiovisual streams, speech emotion is analyzed. Credibility and authentication issues arise when collecting and reusing material. Original techniques for content authentication are designed for the needs of the dissertation.To support the proposed functionality, datasets and machine learning models have been developed. Models implementing Convolutional Neural Network (CNN) architecture, transfer learning techniques, and unsupervised clustering speaker recognition and diarization have been designed and evaluated. Multimodal speaker indexing, combining audio and visual information is proposed. Face detection models are used to automatically select camera shots, incorporating basic composition rules. For the emotional analysis of audio content, the expandable AESDD database has been created, to train machine learning models and a CNN for real-time speech emotion recognition. Transfer learning techniques using CNNs for the training of personalized models have been evaluated. Νovel authentication techniques based on detecting discontinuities in compression standards, recognizing environmental sound patters and calculating reverberation are designed and provided through a dedicated web service to journalists and the public
Degree
thesis:*- Grantor dc:publisher
- Aristotle University Of Thessaloniki (AUTH)
- Year dc:date
- 2020
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Vryzas, Nikolaos
- Βρύζας, Νικόλαος
Subjects
dc:subject × 20- Διαδραστικά και ψηφιακά μέσα
- Μηχανική μάθηση
- Μέλλον δημοσιογραφίας
- Σημασιολογική ανάλυση οπτικοακουστικού περιεχομένου
- Interactive and digital media
- Machine learning
- Future of journalism
- Semantic analysis of audiovisual content
- Επιστήμη Ηλεκτρολόγου Μηχανικού, Ηλεκτρονικού Μηχανικού, Μηχανικού Η/Υ
- Επιστήμες Μηχανικού και Τεχνολογία
- Τεχνολογία μέσων
- ΜΜΕ και Επικοινωνίες
- Κοινωνικές Επιστήμες
- Μέσα επικοινωνίας
- Electrical Engineering, Electronic Engineering, Information Engineering
- Engineering and Technology
- Media Technology
- Media and Communications
- Social Sciences
- Media
Rights
- Language dc:language
- gre
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/51612