Continuous progress in multimedia research in recent years have led to proliferation of their applications in everyday life. The ever-growing demand in high performance multimedia applications creates the need for new and efficient storage and retrieval techniques. There exist numerous studies in the literature attempting to describe the content of these multimedia documents. Moving Picture Experts Group&rsquo / s XML based MPEG-7 is one of these studies that makes it possible to describe multimedia content in terms of both low and high level properties. MPEG-7 DDL allows defining new types using already defined types. Within the past ten years, it became a widely accepted standard in multimedia applications. In this thesis, an XML database application is developed to manage MPEG-7 descriptions, utilizing eXist XML DB as the database management system and a JAVA application as the frontend. MPEG-7 Description Schemes are extended by introducing fuzzy semantic types, such as FuzzyObject and FuzzyEvent, using the MPEG-7 DDL. From this point of view, the application of fuzzy XML methods in MPEG-7 standard is a novel approach.

There is an accelerating demand to access and work over the visual content of documents. Because of the insufficiency of text-based techniques for storing this data, content-based image retrieval (CBIR) systems have become a promising field. Due this fact, in this study a CBIR system is implemented that is Mpeg-7 compliant and ORDBMS based. The database contains images and their content summaries that are parsed from XML files. The summaries describe their dominant colors, color histograms, color spaces and labels, in order to be compliant with Mpeg-7. The query process requires only the summary not the image itself. Software implementation of the system is based on JSP and servlet technologies using Oracle database and Tomcat web server. It is shown that the usage of these tools in the proposed architecture brings security, portability, and speed.

A video data model that allows efficient and effective representation and querying of spatio-temporal properties of objects has been previously developed. The data model is focused on the semantic content of video streams. Objects, events, activities performed by objects are the main interests of the model. The model supports fuzzy spatial queries including querying spatial relationships between objects and querying the trajectories of objects. In this thesis, this work is used as a basis for the development of an XML-based video database system. This system is aimed to be compliant with the MPEG-7 Multimedia Description Schemes in order to obey a universal standard. The system is implemented using a native XML database management system. Query entrance facilities are enhanced via integrating an NLP interface.

Made available in DSpace on 2016-06-02T19:05:25Z (GMT). No. of bitstreams: 1 DissLDF.pdf: 1886677 bytes, checksum: dc37e62087bd48b0ae6005924f0c8b99 (MD5) Previous issue date: 2007-06-18 / Financiadora de Estudos e Projetos / This work presents a layer for the manipulation of media based on the multimedia content description interface, MPEG-7. This layer realizes some parts of the MPEG-7 MDS (Multimedia Description Schemes), and being built by components, makes a base for the implementation of any application that has media manipulation requisites that range from the simplest ones to a database multimedia system. In this last case, it can be a system that have many operations being executed in a database, like multimedia data mining and indexing by low level features, all supported by MPEG-7. This layer was built to unify the necessities on manipulating media of many applications, integrating them and acting as a mediator. It provides interfaces so that the insertion of media together with their semantic annotation is possible, along with queries based on these semantic metadata derived from these annotations. A simplified model for the semantic annotation and queries for the ease of use of these interfaces was proposed, being the annotations and queries made in a similar manner. Finally, this layer allows the exporting of the query metadata on the format used by MPEG-7 for interchange between systems. / Este trabalho apresenta uma camada de manipulação de mídias baseada no padrão de descrição de dados multimídia MPEG-7. Esta camada realiza algumas partes dos MDS (Multimedia Description Schemes) do padrão MPEG-7, e através de sua construção por componentes, forma uma base para a implementação de quaisquer aplicações que tenham requisitos de manipulação de mídias, desde as mais simples até um sistema de banco de dados multimídia. Nesse último caso, pode ser um sistema que execute as várias operações possíveis em um banco de dados, como mineração de dados multimídia e indexação por características de baixo-nível, tudo apoiado sobre o padrão. Esta camada foi feita com a intenção de unificar as necessidades de manipulação de mídias por várias aplicações, integrando-as e agindo dessa forma como mediadora. Ela provê interfaces para que sejam feitas inserções de mídias junto com as anotações semânticas, além de consultas sobre elas através desses metadados semânticos provindos das anotações. Foi proposto um modelo simplificado de anotação e consulta semântica para a fácil utilização dessas interfaces, sendo que tanto as anotações como as consultas são feitas de maneira semelhante. Finalmente, essa camada possibilita que seja feita a exportação dos metadados da consulta no formato usado pelo padrão MPEG-7, para intercâmbio de dados com outros sistemas.

La question de recherche des objets vidéo basés sur le contenu lui-même, est de plus en plus difficile et devient un élément obligatoire pour les moteurs de recherche vidéo. Cette thèse présente un cadre pour la recherche des objets vidéo définis par l'utilisateur et apporte deux grandes contributions. La première contribution, intitulée DOOR (Dynamic Object Oriented Retrieval), est un cadre méthodologique pour la recherche et récupération des instances d'objets vidéo sélectionnés par un utilisateur, tandis que la seconde contribution concerne le support offert pour la recherche des vidéos, à savoir la navigation dans les vidéo, le système de récupération de vidéos et l'interface avec son architecture sous-jacente.Dans le cadre DOOR, l’objet comporte une représentation hybride obtenues par une sur-segmentation des images, consolidé avec la construction des graphs d’adjacence et avec l’agrégation des points d'intérêt. L'identification des instances d'objets à travers plusieurs vidéos est formulée comme un problème d’optimisation de l'énergie qui peut approximer un tache NP-difficile. Les objets candidats sont des sous-graphes qui rendent une énergie optimale vers la requête définie par l'utilisateur. Quatre stratégies d'optimisation sont proposées: Greedy, Greedy relâché, recuit simulé et GraphCut. La représentation de l'objet est encore améliorée par l'agrégation des points d'intérêt dans la représentation hybride, où la mesure de similarité repose sur une technique spectrale intégrant plusieurs types des descripteurs. Le cadre DOOR est capable de s’adapter à des archives vidéo a grande échelle grâce à l'utilisation de représentation sac-de-mots, enrichi avec un algorithme de définition et d’expansion de la requête basée sur une approche multimodale, texte, image et vidéo. Les techniques proposées sont évaluées sur plusieurs corpora de test TRECVID et qui prouvent leur efficacité.La deuxième contribution, OVIDIUS (On-line VIDeo Indexing Universal System) est une plate-forme en ligne pour la navigation et récupération des vidéos, intégrant le cadre DOOR. Les contributions de cette plat-forme portent sur le support assuré aux utilisateurs pour la recherche vidéo - navigation et récupération des vidéos, interface graphique. La plate-forme OVIDIUS dispose des fonctionnalités de navigation hiérarchique qui exploite la norme MPEG-7 pour la description structurelle du contenu vidéo. L'avantage majeur de l'architecture propose c’est sa structure modulaire qui permet de déployer le système sur terminaux différents (fixes et mobiles), indépendamment des systèmes d'exploitation impliqués. Le choix des technologies employées pour chacun des modules composant de la plate-forme est argumentée par rapport aux d'autres options technologiques. / With the ever increasing amount of available video content on video repositories the issue of content-based video objects retrieval is growing in difficulty and becomes a mandatory feature for video search engines.The present thesis advances a user defined video object retrieval framework and brings two major contributions. The first contribution is a methodological framework for user selected video object instances retrieval, entitled DOOR (Dynamic Object Oriented Retrieval), while the second one concerns the support offered for video retrieval, namely the video navigation and retrieval system and interface and its underlying architecture.Under the DOOR framework, the user defined video object comports a hybrid representation obtained by over-segmenting the frames, constructing region adjacency graphs and aggregating interest points. The identification of object instances across multiple videos is formulated as an energy optimization problem approximating an NP-hard problem. Object candidates are sub-graphs that yield an optimum energy towards the user defined query. In order to obtain the optimum energy four optimization strategies are proposed: Greedy, Relaxed Greedy, Simulated Annealing and GraphCut. The region-based object representation is further improved by the aggregation of interest points into a hybrid object representation. The similarity between an object and a frame is achieved with the help of a spectral matching technique integrating both colorimetric and interest points descriptors.The DOOR framework is suitable to large scale video archives through the use of a Bag-of-Words representation enriched with a query definition and expansion mechanism based on a multi-modal, text-image-video principle.The performances of the proposed techniques are evaluated on multiple TRECVID video datasets prooving their effectiveness.The second contribution is related to the user support for video retrieval - video navigation, video retrieval, graphical interface - and consists in the OVIDIUS (On-line VIDeo Indexing Universal System) on-line video browsing and retrieval platform. The OVIDIUS platform features hierarchical video navigation functionalities that exploit the MPEG-7 approach for structural description of video content. The DOOR framework is integrated in the OVIDIUS platform, ensuring the search functionalities of the system. The major advantage of the proposed system concerns its modular architecture which makes it possible to deploy the system on various terminals (both fixed and mobile), independently of the exploitation systems involved. The choice of the technologies employed for each composing module of the platform is argumented in comparison with other technological options. Finally different scenarios and use cases for the OVIDIUS platform are presented.

The retrieval of similar images in databases is a wide and complex research field that shows a great demand for good performance applications. The increasing volume of information available in the Internet and the success of textual search engines motivate the development of tools that make possible image searches by content similarity. Many features can be applied in determining the similarity between images, such as size, color, shape, color variation, texture, objects and their spatial distribution, among others. Texture and color are the most important features which allow a preliminary analysis of image similarity. This dissertation presents many techniques introduced in the literature, which analyze texture and color. Some of them were implemented, their performances were compared and the results were presented. This comparison allows the determination of the best techniques, making possible the analysis of their applicability and can be used as a reference in future works. The quantitative performance analyses were done using the ANMRR metric, defined in the MPEG-7 standard, and the confusion matrices were presented for each of the tested techniques. Two groups of quantitative tests were realized: the first one was applied upon a gray scale texture database and the second one, upon a color image database. For the experiment with the gray scale texture images, the techniques PBLIRU16, MCNC and their combination presented the best performances. For the experiment with the color images, SCD, HDCIG and CSD techniques performed best. / A recuperação de imagens semelhantes em bancos de dados é um campo de pesquisa amplo, complexo e que apresenta grande demanda por aplicativos que apresentem bons resultados. O volume crescente de informações disponibilizadas ao público e o sucesso das ferramentas de busca textuais na Internet motivam a criação de utilitários que possibilitem a busca de imagens por semelhança de conteúdo. Podem-se utilizar várias características para a determinação da semelhança entre imagens digitais, tais como tamanho, cor, forma, variação de cores, textura, objetos e sua disposição espacial, entre outras. A textura e a cor são as duas características mais importantes que permitem uma análise preliminar da semelhança. Este trabalho apresenta várias técnicas constantes da literatura, que analisam textura e cor. Algumas dessas técnicas foram implementadas, seus desempenhos foram analisados e comparados e os resultados foram apresentados detalhadamente. Esse comparativo amplo permite determinar as melhores técnicas, possibilita a análise da aplicabilidade de cada uma delas e pode ser utilizada como referência em estudos futuros. As análises quantitativas de desempenho foram realizadas utilizando a métrica ANMRR, definida no padrão MPEG-7, e as matrizes de confusão, apresentadas para cada técnica testada. Dois grupos de testes quantitativos foram realizados: o primeiro utilizando um banco de imagens de texturas em tons de cinza e o segundo utilizando um banco de imagens coloridas. Os resultados dos testes com o banco de texturas em tons de cinza mostraram que as técnicas PBLIRU16, MCNC e sua combinação apresentaram os melhores desempenhos. Para o banco de imagens coloridas, os melhores desempenhos foram observados com a utilização das técnicas SCD, HDCIG e CSD. / Mestre em Ciências

The increase in the use of digital video/image has showed the need for modeling and querying the semantic content in them. Using manual annotation techniques for defining the semantic content is both costly in time and have limitations on querying capabilities. So, the need for content based information retrieval in multimedia domain is to extract the semantic content in an automatic way. The semantic content is usually defined with the objects in images/videos. In this thesis, a Genetic Algorithm based object extraction and classification mechanism is proposed for extracting the content of the videos and images. The object extraction is defined as a classification problem and a Genetic Algorithm based classifier is proposed for classification. Candidate objects are extracted from videos/images by using Normalized-cut segmentation and sent to the classifier for classification. Objects are defined with the Best Representative and Discriminative Feature (BRDF) model, where features are MPEG-7 descriptors. The decisions of the classifier are calculated by using these features and BRDF model. The classifier improves itself in time, with the genetic operations of GA. In addition to these, the system supports fuzziness by making multiple categorization and giving fuzzy decisions on the objects. Externally from the base model, a statistical feature importance determination method is proposed to generate BRDF model of the categories automatically. In the thesis, a platform independent application for the proposed system is also implemented.

Video databases and content based retrieval in these databases have become popular with the improvements in technology. Metadata extraction techniques are used for providing data to video content. One popular metadata extraction technique for mul- timedia is information extraction from text. For some domains, it is possible to &amp / #64257 / nd accompanying text with the video, such as soccer domain, movie domain and news domain. In this thesis, we present an approach of metadata extraction from match reports for soccer domain. The UEFA Cup and UEFA Champions League Match Reports are downloaded from the web site of UEFA by a web-crawler. These match reports are preprocessed by using regular expressions and then important events are extracted by using hand-written rules. In addition to hand-written rules, two di&amp / #64256 / erent machine learning techniques are applied on match corpus to learn event patterns and automatically extract match events. Extracted events are saved in an MPEG-7 &amp / #64257 / le. A user interface is implemented to query the events in the MPEG-7 match corpus and view the corresponding video segments.

In this thesis, a Content Based Image Retrieval (CBIR) system to query the objects in an image database is proposed. Images are represented as collections of regions after being segmented with Normalized Cuts algorithm. MPEG-7 content descriptors are used to encode regions in a 239-dimensional feature space. User of the proposed CBIR system decides which objects to query and labels exemplar regions to train the system using a graphical interface. Fuzzy ARTMAP algorithm is used to learn the mapping between feature vectors and binary coded class identification numbers. Preliminary recognition experiments prove the power of fuzzy ARTMAP as a region classifier. After training, features of all regions in the database are extracted and classified. Simple index files enabling fast access to all regions from a given class are prepared to be used in the querying phase. To retrieve images containing a particular object, user opens an image and selects a query region together with a label in the graphical interface of our system. Then the system ranks all regions in the indexed set of the query class with respect to their L2 (Euclidean) distance to the query region and displays resulting images. During retrieval experiments, comparable class precisions with respect to exhaustive searching of the database are maintained which demonstrates e ectiveness of the classifier in narrowing down the search space.

This study proposes a new method for obstacle detection and avoidance using low-level MPEG-7 visual descriptors. The method includes training a neural network with a subset of MPEG-7 visual descriptors extracted from outdoor scenes. The trained neural network is then used to estimate the obstacle presence in real outdoor videos and to perform obstacle avoidance. In our proposed method, obstacle avoidance solely depends on the estimated obstacle presence data. In this study, backpropagation algorithm on multi-layer perceptron neural network is utilized as a feature learning method. MPEG-7 visual descriptors are used to describe basic features of the given scene image and by further processing these features, input data for the neural network is obtained. The learning/training phase is carried out on specially constructed synthetic video sequence with known obstacles. Validation and tests of the algorithms are performed on actual outdoor videos. Tests on indoor videos are also performed to evaluate the performance of the proposed algorithms in indoor scenes. Throughout the study, OdBot 2 robot platform, which has been developed by the author, is used as reference platform. For final testing of the obstacle detection and avoidance algorithms, simulation environment is used. From the simulation results and tests performed on video sequences, it can be concluded that the proposed obstacle detection and avoidance methods are robust against visual changes in the environment that are common to most of the outdoor videos. Findings concerning the used methods are presented and discussed as an outcome of this study.