Recently, very large collections of images and videos have grown rapidly. In parallel with this growth, content-based retrieval and querying the indexed collections are required to access visual information. Three main components of the visual information are color, texture and shape. In this thesis, an XML based content-based image retrieval system is presented that combines three visual descriptors of MPEG-7 and measures similarity of images by applying a distance function. An XML database is used for storing these three descriptors. The system is also extended to support high dimensional indexing for efficient search and retrieval from its XML database. To do this, an index structure, called M-Tree, is implemented which uses weighted Euclidean distance function for similarity measure. Ordered Weighted Aggregation (OWA) operators are used to define the weights of the distance function and to combine three features&rsquo / distance functions into one. The system supports nearest neighbor queries and three types of fuzzy queries / feature-based, image-based and color-based queries. Also it is shown through experimental results and analysis of retrieval effectiveness of querying that the content-based retrieval system is effective in terms of retrieval and scalability.

Content-based retrieval of multimedia data has been still an active research area. The efficient retrieval of video data is proven a difficult task for content-based video retrieval systems. In this thesis study, a Content-Based Video Retrieval (CBVR) system that adapts two different index structures, namely Slim-Tree and BitMatrix, for efficiently retrieving videos based on low-level features such as color, texture, shape and motion is presented. The system represents low-level features of video data with MPEG-7 Descriptors extracted from video shots by using MPEG-7 reference software and stored in a native XML database. The low-level descriptors used in the study are Color Layout (CL), Dominant Color (DC), Edge Histogram (EH), Region Shape (RS) and Motion Activity (MA). Ordered Weighted Averaging (OWA) operator in Slim-Tree and BitMatrix aggregates these features to find final similarity between any two objects. The system supports three different types of queries: exact match queries, k-NN queries and range queries. The experiments included in this study are in terms of index construction, index update, query response time and retrieval efficiency using ANMRR performance metric and precision/recall scores. The experimental results show that using BitMatrix along with Ordered Weighted Averaging method is superior in content-based video retrieval systems.

A presente investigação pretende, de forma qualitativa, compreender se os vídeos indexados podem modificar o planejamento pedagógico do professor de escola municipal de Porto Alegre. Para atingir esse objetivo, buscamos embasamento em uma fundamentação teórica de fundo piagetiano, voltada para a construção do conhecimento e suas diversas possibilidades, além das teorias relativas às questões do vídeo e do computador no processo educativo, a partir do seu envolvimento como suporte pedagógico orientado na pedagogia de projetos. Inserido nesse contexto, propõe-se o SACCA (Sistema Automático de Catalogação de Conteúdo Audiovisual), que possibilita o processo de indexação de vídeos com a intermediação de um usuário que assiste ao vídeo e define as palavras-chaves que irão compor o índice. Posteriormente, poderá se navegar no interior desses vídeos permitindo ter uma visão rápida do que o vídeo trata. Foram acrescentados 49 vídeos da História do Brasil e disponibilizados para os professores de História do 3º Ciclo das Escolas da Rede Municipal de Porto Alegre participantes da pesquisa. Os dados foram coletados através de entrevistas. A partir das perguntas norteadoras da pesquisa e da investigação realizada, são definidas categorias de análise para as reflexões e as conclusões desta tese. Os resultados obtidos indicam que o uso dos vídeos indexados pode modificar o planejamento pedagógico do professor. / The present investigation intends to understand in a qualitative way if the indexed videos can modify the teacher's pedagogical planning of a public school in Porto Alegre. We searched for a theoretical Piagetian approach, in order to achieve this purpose, turned to the construction of knowledge and its several possibilities, besides the theories related to the points of the video and computer in the educational process, starting from its involvement as a pedagogical support guided in the pedagogy of projects. Inserted in this context, SACCA (Automatic System of Cataloguing of Audiovisual Content) is proposed, which makes the process of indexation of videos possible, with the user's intermediation who sees the video and defines the key-words, which will compose the index. Later, one can navigate in the interior of these videos allowing a fast vision of what the video deals with. 49 videos of the Brazilian History were added and available for the History teachers of the 3rd Cycle of the Public Schools in Porto Alegre, participants in the research. The data were collected through interviews. Starting from the directed questions of the research and of the accomplished investigation, analysis categories for the reflections and the conclusions of this theory are defined. The obtained results indicate that the use of the indexed videos can modify the teacher's pedagogical planning.

A presente investigação pretende, de forma qualitativa, compreender se os vídeos indexados podem modificar o planejamento pedagógico do professor de escola municipal de Porto Alegre. Para atingir esse objetivo, buscamos embasamento em uma fundamentação teórica de fundo piagetiano, voltada para a construção do conhecimento e suas diversas possibilidades, além das teorias relativas às questões do vídeo e do computador no processo educativo, a partir do seu envolvimento como suporte pedagógico orientado na pedagogia de projetos. Inserido nesse contexto, propõe-se o SACCA (Sistema Automático de Catalogação de Conteúdo Audiovisual), que possibilita o processo de indexação de vídeos com a intermediação de um usuário que assiste ao vídeo e define as palavras-chaves que irão compor o índice. Posteriormente, poderá se navegar no interior desses vídeos permitindo ter uma visão rápida do que o vídeo trata. Foram acrescentados 49 vídeos da História do Brasil e disponibilizados para os professores de História do 3º Ciclo das Escolas da Rede Municipal de Porto Alegre participantes da pesquisa. Os dados foram coletados através de entrevistas. A partir das perguntas norteadoras da pesquisa e da investigação realizada, são definidas categorias de análise para as reflexões e as conclusões desta tese. Os resultados obtidos indicam que o uso dos vídeos indexados pode modificar o planejamento pedagógico do professor. / The present investigation intends to understand in a qualitative way if the indexed videos can modify the teacher's pedagogical planning of a public school in Porto Alegre. We searched for a theoretical Piagetian approach, in order to achieve this purpose, turned to the construction of knowledge and its several possibilities, besides the theories related to the points of the video and computer in the educational process, starting from its involvement as a pedagogical support guided in the pedagogy of projects. Inserted in this context, SACCA (Automatic System of Cataloguing of Audiovisual Content) is proposed, which makes the process of indexation of videos possible, with the user's intermediation who sees the video and defines the key-words, which will compose the index. Later, one can navigate in the interior of these videos allowing a fast vision of what the video deals with. 49 videos of the Brazilian History were added and available for the History teachers of the 3rd Cycle of the Public Schools in Porto Alegre, participants in the research. The data were collected through interviews. Starting from the directed questions of the research and of the accomplished investigation, analysis categories for the reflections and the conclusions of this theory are defined. The obtained results indicate that the use of the indexed videos can modify the teacher's pedagogical planning.

A presente investigação pretende, de forma qualitativa, compreender se os vídeos indexados podem modificar o planejamento pedagógico do professor de escola municipal de Porto Alegre. Para atingir esse objetivo, buscamos embasamento em uma fundamentação teórica de fundo piagetiano, voltada para a construção do conhecimento e suas diversas possibilidades, além das teorias relativas às questões do vídeo e do computador no processo educativo, a partir do seu envolvimento como suporte pedagógico orientado na pedagogia de projetos. Inserido nesse contexto, propõe-se o SACCA (Sistema Automático de Catalogação de Conteúdo Audiovisual), que possibilita o processo de indexação de vídeos com a intermediação de um usuário que assiste ao vídeo e define as palavras-chaves que irão compor o índice. Posteriormente, poderá se navegar no interior desses vídeos permitindo ter uma visão rápida do que o vídeo trata. Foram acrescentados 49 vídeos da História do Brasil e disponibilizados para os professores de História do 3º Ciclo das Escolas da Rede Municipal de Porto Alegre participantes da pesquisa. Os dados foram coletados através de entrevistas. A partir das perguntas norteadoras da pesquisa e da investigação realizada, são definidas categorias de análise para as reflexões e as conclusões desta tese. Os resultados obtidos indicam que o uso dos vídeos indexados pode modificar o planejamento pedagógico do professor. / The present investigation intends to understand in a qualitative way if the indexed videos can modify the teacher's pedagogical planning of a public school in Porto Alegre. We searched for a theoretical Piagetian approach, in order to achieve this purpose, turned to the construction of knowledge and its several possibilities, besides the theories related to the points of the video and computer in the educational process, starting from its involvement as a pedagogical support guided in the pedagogy of projects. Inserted in this context, SACCA (Automatic System of Cataloguing of Audiovisual Content) is proposed, which makes the process of indexation of videos possible, with the user's intermediation who sees the video and defines the key-words, which will compose the index. Later, one can navigate in the interior of these videos allowing a fast vision of what the video deals with. 49 videos of the Brazilian History were added and available for the History teachers of the 3rd Cycle of the Public Schools in Porto Alegre, participants in the research. The data were collected through interviews. Starting from the directed questions of the research and of the accomplished investigation, analysis categories for the reflections and the conclusions of this theory are defined. The obtained results indicate that the use of the indexed videos can modify the teacher's pedagogical planning.

The field that deals with storing and effective searching of multimedia documents is called Information retrieval. This paper describes solution of effective searching in collections of shots. Multimedia documents are presented as vectors in high-dimensional space, because in such collection of documents it is easier to define semantics as well as the mechanisms of searching. The work aims at problems of similarity searching based on metric space, which uses distance functions, such as Euclidean, Chebyshev or Mahalanobis, for comparing global features and cosine or binary rating for comparing local features. Experiments on the TRECVid dataset compare implemented distance functions. Best distance function for global features appears to be Mahalanobis and for local features cosine rating.

La question de recherche des objets vidéo basés sur le contenu lui-même, est de plus en plus difficile et devient un élément obligatoire pour les moteurs de recherche vidéo. Cette thèse présente un cadre pour la recherche des objets vidéo définis par l'utilisateur et apporte deux grandes contributions. La première contribution, intitulée DOOR (Dynamic Object Oriented Retrieval), est un cadre méthodologique pour la recherche et récupération des instances d'objets vidéo sélectionnés par un utilisateur, tandis que la seconde contribution concerne le support offert pour la recherche des vidéos, à savoir la navigation dans les vidéo, le système de récupération de vidéos et l'interface avec son architecture sous-jacente.Dans le cadre DOOR, l'objet comporte une représentation hybride obtenues par une sur-segmentation des images, consolidé avec la construction des graphs d'adjacence et avec l'agrégation des points d'intérêt. L'identification des instances d'objets à travers plusieurs vidéos est formulée comme un problème d'optimisation de l'énergie qui peut approximer un tache NP-difficile. Les objets candidats sont des sous-graphes qui rendent une énergie optimale vers la requête définie par l'utilisateur. Quatre stratégies d'optimisation sont proposées: Greedy, Greedy relâché, recuit simulé et GraphCut. La représentation de l'objet est encore améliorée par l'agrégation des points d'intérêt dans la représentation hybride, où la mesure de similarité repose sur une technique spectrale intégrant plusieurs types des descripteurs. Le cadre DOOR est capable de s'adapter à des archives vidéo a grande échelle grâce à l'utilisation de représentation sac-de-mots, enrichi avec un algorithme de définition et d'expansion de la requête basée sur une approche multimodale, texte, image et vidéo. Les techniques proposées sont évaluées sur plusieurs corpora de test TRECVID et qui prouvent leur efficacité.La deuxième contribution, OVIDIUS (On-line VIDeo Indexing Universal System) est une plate-forme en ligne pour la navigation et récupération des vidéos, intégrant le cadre DOOR. Les contributions de cette plat-forme portent sur le support assuré aux utilisateurs pour la recherche vidéo - navigation et récupération des vidéos, interface graphique. La plate-forme OVIDIUS dispose des fonctionnalités de navigation hiérarchique qui exploite la norme MPEG-7 pour la description structurelle du contenu vidéo. L'avantage majeur de l'architecture propose c'est sa structure modulaire qui permet de déployer le système sur terminaux différents (fixes et mobiles), indépendamment des systèmes d'exploitation impliqués. Le choix des technologies employées pour chacun des modules composant de la plate-forme est argumentée par rapport aux d'autres options technologiques.

Audiovisuelle Medien, insbesondere Filme und Fernsehsendungen entwickelten sich innerhalb der letzten einhundert Jahre zu bedeutenden Massenmedien. Große Bestände audiovisueller Medien werden heute in Datenbanken und Mediatheken verwaltet und professionellen Nutzern ebenso wie den privaten Konsumenten zur Verfügung gestellt. Eine besondere Herausforderung liegt in der Indexierung, Durchsuchung und Beschreibung der multimedialen Datenbestände. Die Segmentierung audiovisueller Medien, als Teilgebiet der Videoanalyse, bildet die Grundlage für verschiedene Anwendungen im Bereich Multimedia-Information-Retrieval, Content-Browsing und Video-Summarization. Insbesondere die Segmentierung in semantische Handlungsanschnitte bei narrativen Medien gestaltet sich schwierig. Sie setzt ein besonderes Verständnis der filmischen Stilelemente vorraus, die im Rahmen des Schaffensprozesses genutzt wurden, um die Handlung und Narration zu unterstützten. Die Arbeit untersucht die bekannten filmischen Stilelemente und wie sie sich im Rahmen algorithmischer Verfahren für die Analyse nutzen lassen. Es kann gezeigt werden, dass unter Verwendung eines mehrstufigen Analyse-Prozesses semantische Zusammenhänge in narrativen audiovisuellen Medien gefunden werden können, die zu einer geeigneten Sequenz-Segmentierung führen. / Audiovisual media, especially movies and TV shows, developed within the last hundred years into major mass media. Today, large stocks of audiovisual media are managed in databases and media libraries. The content is provided to professional users as well as private consumers. A particular challenge lies in the indexing, searching and description of multimedia assets. The segmentation of audiovisual media as a branch of video analysis forms the basis for various applications in multimedia information retrieval, content browsing and video summarization. In particular, the segmentation into semantic meaningful scenes or sequences is difficult. It requires a special understanding of cinematic style elements that were used to support the narration during the creative process of film production. This work examines the cinematic style elements and how they can be used in the context of algorithmic methods for analysis. For this purpose, an analysis framework was developed as well as a method for sequence-segmentation of films and videos. It can be shown that semantic relationships can be found in narrative audiovisual media, which lead to an appropriate sequence segmentation, by using a multi-stage analysis process, based on visual MPEG-7 descriptors.

Audiovisuelle Medien, insbesondere Filme und Fernsehsendungen entwickelten sich innerhalb der letzten einhundert Jahre zu bedeutenden Massenmedien. Große Bestände audiovisueller Medien werden heute in Datenbanken und Mediatheken verwaltet und professionellen Nutzern ebenso wie den privaten Konsumenten zur Verfügung gestellt. Eine besondere Herausforderung liegt in der Indexierung, Durchsuchung und Beschreibung der multimedialen Datenbestände. Die Segmentierung audiovisueller Medien, als Teilgebiet der Videoanalyse, bildet die Grundlage für verschiedene Anwendungen im Bereich Multimedia-Information-Retrieval, Content-Browsing und Video-Summarization. Insbesondere die Segmentierung in semantische Handlungsanschnitte bei narrativen Medien gestaltet sich schwierig. Sie setzt ein besonderes Verständnis der filmischen Stilelemente vorraus, die im Rahmen des Schaffensprozesses genutzt wurden, um die Handlung und Narration zu unterstützten. Die Arbeit untersucht die bekannten filmischen Stilelemente und wie sie sich im Rahmen algorithmischer Verfahren für die Analyse nutzen lassen. Es kann gezeigt werden, dass unter Verwendung eines mehrstufigen Analyse-Prozesses semantische Zusammenhänge in narrativen audiovisuellen Medien gefunden werden können, die zu einer geeigneten Sequenz-Segmentierung führen. / Audiovisual media, especially movies and TV shows, developed within the last hundred years into major mass media. Today, large stocks of audiovisual media are managed in databases and media libraries. The content is provided to professional users as well as private consumers. A particular challenge lies in the indexing, searching and description of multimedia assets. The segmentation of audiovisual media as a branch of video analysis forms the basis for various applications in multimedia information retrieval, content browsing and video summarization. In particular, the segmentation into semantic meaningful scenes or sequences is difficult. It requires a special understanding of cinematic style elements that were used to support the narration during the creative process of film production. This work examines the cinematic style elements and how they can be used in the context of algorithmic methods for analysis. For this purpose, an analysis framework was developed as well as a method for sequence-segmentation of films and videos. It can be shown that semantic relationships can be found in narrative audiovisual media, which lead to an appropriate sequence segmentation, by using a multi-stage analysis process, based on visual MPEG-7 descriptors.

Avec l'explosion des données numériques disponibles (notamment via Internet), la question de l'accès aux documents reçoit depuis quelques années une attention accrue. En effet, l'indexation des documents, traditionnellement fondée sur la description textuelle, atteint rapidement ses limites en particulier lorsque le contenu concerné est musical. <br /><br />Cette thèse focalise sur la recherche de documents musicaux par chantonnement. Nous présentons un système qui permet de retrouver une musique à partir d'un extrait chanté par l'utilisateur. Sa réalisation a nécessité deux études préalables qui ont comblé quelques lacunes d'un domaine de recherche encore jeune. Nous nous sommes intéressés, d'une part, à la "justesse" des mélodies chantonnées (par l'étude de 500 requêtes), et d'autre part, à certains aspects de la similarité mélodique (par la réalisation de tests subjectifs).<br /><br />Grâce à ces études, nous proposons un système de recherche original et performant. Refusant une description tempérée de la requête (i.e. comportant une quantification des notes au demi-ton), le système proposé retrouve plus de 90% des documents musicaux attendus, pour une taille de requête moyenne (13 notes). La base de données consultée est constituée de 20.000 fichiers MIDI (40 millions de notes indexées). Le temps d'attente est acceptable puisqu'il ne faut que quelques secondes au système pour fournir sa réponse (i.e. la liste des documents les plus similaires à la requête, ceux-ci étant classés par ordre de similarité).<br /><br />Cette thèse apporte également une aide dans le processus d'évaluation de la qualité de tels systèmes. En effet, nous proposons une modélisation de l'imprécision des mélodies chantonnées. Celle-ci permet la génération de requêtes artificielles qui peuvent être substituées aux requêtes réelles lors du test de systèmes. Cette alternative permet d'alléger le processus de test tout en conservant une stimulation réaliste.