This paper documents the planning, design, implementation, and preliminary evaluation of the Multimedia Story Builder and Multimedia Story Player – two applications that together are used to create, edit, manage, and deliver web-based multimedia presentations. These applications incorporate simple and intuitive interfaces, extensive functionality, multiple presentation methods, and support for multiple media types, all of which represent improvements over similar systems. The results of a preliminary usability test and suggestions for future testing and development are also described.

This thesis provides a novel architecture for the creation and management of virtual multimedia adaptation services offered by a multimedia-enabled cloud. The aim of the proposed scheme is to provide an optimal yet a transparent user access to adapted media contents while isolating them from the heterogeneity of the utilized devices, diversity of media formats, as well as the details of the adaptation services and performance variations of the underlying network. This goal is achieved through the development of service virtualization models that provide various levels of abstraction of the actual physical services and their performance parameters. Such virtual models offer adaptation functions by comprising adaptation services with accordance to their parameters. Additionally, parameters describing the functional specifics of the adaptation functions, as well as multimedia content features, are organized into a hierarchical structure that facilitates extraction of the virtual models capable of satisfying the conditions expressed by the user requests. At the same time the paramter/feature organization structure itself is flexible enough to allow users to specify media delivery requests at various levels of request details (e.g., summarize video vs. drop specific frames). As a result, in response to a user request for a multimedia content, an optimal virtual service adaptation path is calculated, describing the needed media adaptation operations as well as the appropriate mapping to the physical resources capable of executing such functions. The selection of the adaptation path is done with the use of a novel performance-history based selection mechanism that takes into account the performance variations and relations of the services in a dynamically changing environment of multimedia clouds. A number of experiments are conducted to demonstrate the potential of the proposed work in terms of the enhanced processing time and service quality.

Automated facial expression analysis for Emotion Recognition (ER) is an active research area towards creating socially intelligent systems. The eye region, often considered integral for ER by psychologists and neuroscientists, has received very little attention in engineering and computer sciences. Using eye region as an input signal presents several bene ts for low-cost, non-intrusive ER applications. This work proposes two frameworks towards ER from eye region images. The first framework uses Local Binary Patterns (LBP) as the feature extractor on grayscale eye region images. The results validate the eye region as a signi cant contributor towards communicating the emotion in the face by achieving high person-dependent accuracy. The system is also able to generalize well across di erent environment conditions. In the second proposed framework, a color-based approach to ER from the eye region is explored using Local Color Vector Binary Patterns (LCVBP). LCVBP extend the traditional LBP by incorporating color information extracting a rich and a highly discriminative feature set, thereby providing promising results.

Automated facial expression analysis for Emotion Recognition (ER) is an active research area towards creating socially intelligent systems. The eye region, often considered integral for ER by psychologists and neuroscientists, has received very little attention in engineering and computer sciences. Using eye region as an input signal presents several bene ts for low-cost, non-intrusive ER applications. This work proposes two frameworks towards ER from eye region images. The first framework uses Local Binary Patterns (LBP) as the feature extractor on grayscale eye region images. The results validate the eye region as a signi cant contributor towards communicating the emotion in the face by achieving high person-dependent accuracy. The system is also able to generalize well across di erent environment conditions. In the second proposed framework, a color-based approach to ER from the eye region is explored using Local Color Vector Binary Patterns (LCVBP). LCVBP extend the traditional LBP by incorporating color information extracting a rich and a highly discriminative feature set, thereby providing promising results.

(non disponible, voir en anglais)

This thesis provides a novel architecture for the creation and management of virtual multimedia adaptation services offered by a multimedia-enabled cloud. The aim of the proposed scheme is to provide an optimal yet a transparent user access to adapted media contents while isolating them from the heterogeneity of the utilized devices, diversity of media formats, as well as the details of the adaptation services and performance variations of the underlying network. This goal is achieved through the development of service virtualization models that provide various levels of abstraction of the actual physical services and their performance parameters. Such virtual models offer adaptation functions by comprising adaptation services with accordance to their parameters. Additionally, parameters describing the functional specifics of the adaptation functions, as well as multimedia content features, are organized into a hierarchical structure that facilitates extraction of the virtual models capable of satisfying the conditions expressed by the user requests. At the same time the paramter/feature organization structure itself is flexible enough to allow users to specify media delivery requests at various levels of request details (e.g., summarize video vs. drop specific frames). As a result, in response to a user request for a multimedia content, an optimal virtual service adaptation path is calculated, describing the needed media adaptation operations as well as the appropriate mapping to the physical resources capable of executing such functions. The selection of the adaptation path is done with the use of a novel performance-history based selection mechanism that takes into account the performance variations and relations of the services in a dynamically changing environment of multimedia clouds. A number of experiments are conducted to demonstrate the potential of the proposed work in terms of the enhanced processing time and service quality.

This thesis provides a novel architecture for the creation and management of virtual multimedia adaptation services offered by a multimedia-enabled cloud. The aim of the proposed scheme is to provide an optimal yet a transparent user access to adapted media contents while isolating them from the heterogeneity of the utilized devices, diversity of media formats, as well as the details of the adaptation services and performance variations of the underlying network. This goal is achieved through the development of service virtualization models that provide various levels of abstraction of the actual physical services and their performance parameters. Such virtual models offer adaptation functions by comprising adaptation services with accordance to their parameters. Additionally, parameters describing the functional specifics of the adaptation functions, as well as multimedia content features, are organized into a hierarchical structure that facilitates extraction of the virtual models capable of satisfying the conditions expressed by the user requests. At the same time the paramter/feature organization structure itself is flexible enough to allow users to specify media delivery requests at various levels of request details (e.g., summarize video vs. drop specific frames). As a result, in response to a user request for a multimedia content, an optimal virtual service adaptation path is calculated, describing the needed media adaptation operations as well as the appropriate mapping to the physical resources capable of executing such functions. The selection of the adaptation path is done with the use of a novel performance-history based selection mechanism that takes into account the performance variations and relations of the services in a dynamically changing environment of multimedia clouds. A number of experiments are conducted to demonstrate the potential of the proposed work in terms of the enhanced processing time and service quality.

The objectives of this project are to create a concept of system for multimedia documents management and long-term archiving and its realization. The opening chapters are devoted to theoretical analysis of the expected characteristics of this systém and their implementation in open source document management system Alfresco, which was used for implementation. Other chapters are devoted to modifications made to this system, which are enhancements to work with multimedia content and its metadata. Also possibilities of connecting the system to other systems are mentioned. Finally, this document also describes testing the changes and adjustments according to various criteria.

Zero-shot video generation involves generating videos of concepts (action classes) that are not seen in the training phase. Even though the research community has explored conditional video generation for long high-resolution videos, zero-shot video remains a fairly unexplored and challenging task. Most recent works can generate videos for action-object or motion-content pairs, where both the object (content) and action (motion) are observed separately during training, yet results often lack spatial consistency between foreground and background and cannot generalize to complex scenes with multiple objects or actions. In this work, we propose Concept2Vid that generates zero-shot videos for classes that are completely unseen during training. In contrast to prior work, our model is not limited to a predefined fixed set of class-level attributes, but rather utilizes semantic information from multiple videos of the same topic to generate samples from novel classes. We evaluate qualitatively and quantitatively on the Kinetics400 and UCF101 datasets, demonstrating the effectiveness of our proposed model. / Master of Science / Humans are able to generalize unseen scenarios without explicit feedback. They can be thought of as self-learning Artificial Intelligence agents that can collect data from various modalities (video, audio, text) found in surrounding environments, to develop new knowledge and acclimate to unseen situations without explicit feedback. Many recent studies have learned how to perform this process for images, but very few have been able to extend it to videos. Videos provide rich multi-modal data, such as text, audio, and images, and hence are composed of multifaceted knowledge that can introduce more complex temporal and spatial constraints. Leveraging videos in combination with text and audio data can assist intelligent systems to learn similar to how humans do. Zero-shot video generation (ZSVG) involves generating videos of concepts that are not seen in the training phase of a machine learning model. Generating a zero-shot video requires a multitude of temporal and spatial dependencies. In generating a video, not only does the model need temporal coherence but also the understanding of object properties. Current approaches for ZSVG are not well suited due to these challenges. We propose Concept2Vid which generates zero-shot videos for classes that are completely unseen during training. In contrast to prior work, our model is not limited to a predefined fixed set of class descriptions, but rather utilizes semantic information from multiple videos of the same topic to generate samples from novel classes. We evaluate qualitatively and quantitatively on the Kinetics400 and UCF101 datasets, demonstrating the effectiveness of our proposed model.

The search for information about entities, such as people or movies, plays an increasingly important role on the Web. This information is still scattered across many Web pages, making it more time consuming for a user to find all relevant information about an entity. This thesis describes techniques to extract entities and information about these entities from the Web, such as facts, opinions, questions and answers, interactive multimedia objects, and events. The findings of this thesis are that it is possible to create a large knowledge base automatically using a manually-crafted ontology. The precision of the extracted information was found to be between 75–90 % (facts and entities respectively) after using assessment algorithms. The algorithms from this thesis can be used to create such a knowledge base, which can be used in various research fields, such as question answering, named entity recognition, and information retrieval.