This thesis documents the research which has led to the design of a number of techniques aiming to improve the performance of content-based image retrieval (CBIR) systems in wavelets domain using texture analysis. Attention was drawn on CBIR in transform domain and in particular wavelets because of the excellent characteristics for compression and texture extraction applications and the wide adoption in both the research community and the industry. The issue of performance is addressed in terms of accuracy and speed. The rationale for this research builds upon the conclusion that CBIR has not yet reached a good performance balance of accuracy, efficiency and speed for wide adoption in practical applications. The issue of bridging the sensory gap, which is defined as "[the difference] between the object in the real world and the information in a (computational) description derived from a recording of that scene." has yet to be resolved. Furthermore, speed improvement remains an uncharted territory as is feature extraction directly from the bitstream of compressed images. To address the above requirements the first part of this work introduces three techniques designed to jointly address the issue of accuracy and processing cost of texture characterization in wavelets domain. The second part introduces a new model for mapping the wavelet coefficients of an orthogonal wavelet transformation to a circular locus. The model is applied in order to design a novel rotation-invariant texture descriptor. All of the aforementioned techniques are also designed to bridge the gap between texture-based image retrieval and image compression by using appropriate compatible design parameters. The final part introduces three techniques for improving the speed of a CBIR query through more efficient calculation of the Li-distance, when it is used as an image similarity metric. The contributions conclude with a novel technique which, in conjunction with a widely adopted wavelet-based compression algorithm, extracts texture information directly from the compressed bit-stream for speed and storage requirements savings. The experimental findings indicate that the proposed techniques form a solid groundwork which can be extended to practical applications.

The focus of this research is on a tolerance space-based approach to image analysis and correspondence. The problem considered in this thesis is one of extracting perceptually relevant information from groups of objects based on their descriptions. Object descriptions are represented by feature vectors containing probe function values in a manner similar to feature extraction in pattern classification theory. The motivation behind this work is the synthesizing of human perception of nearness for improvement of image processing systems. In these systems, the desired output is similar to the output of a human performing the same task. Thus, it is important to have systems that accurately model human perception. Near set theory provides a framework for measuring the similarity of objects based on features that describe them in much the same way that humans perceive the similarity of objects. In this thesis, near set theory is presented and advanced, and work is presented toward a near set approach to performing content-based image retrieval. Furthermore, results are given based on these new techniques and future work is presented. The contributions of this thesis are: the introduction of a nearness measure to determine the degree that near sets resemble each other; a systematic approach to finding tolerance classes, together with proofs demonstrating that the proposed approach will find all tolerance classes on a set of objects; an approach to applying near set theory to images; the application of near set theory to the problem of content-based image retrieval; demonstration that near set theory is well suited to solving problems in which the outcome is similar to that of human perception; two other near set measures, one based on Hausdorff distance, the other based on Hamming distance.

The focus of this research is on a tolerance space-based approach to image analysis and correspondence. The problem considered in this thesis is one of extracting perceptually relevant information from groups of objects based on their descriptions. Object descriptions are represented by feature vectors containing probe function values in a manner similar to feature extraction in pattern classification theory. The motivation behind this work is the synthesizing of human perception of nearness for improvement of image processing systems. In these systems, the desired output is similar to the output of a human performing the same task. Thus, it is important to have systems that accurately model human perception. Near set theory provides a framework for measuring the similarity of objects based on features that describe them in much the same way that humans perceive the similarity of objects. In this thesis, near set theory is presented and advanced, and work is presented toward a near set approach to performing content-based image retrieval. Furthermore, results are given based on these new techniques and future work is presented. The contributions of this thesis are: the introduction of a nearness measure to determine the degree that near sets resemble each other; a systematic approach to finding tolerance classes, together with proofs demonstrating that the proposed approach will find all tolerance classes on a set of objects; an approach to applying near set theory to images; the application of near set theory to the problem of content-based image retrieval; demonstration that near set theory is well suited to solving problems in which the outcome is similar to that of human perception; two other near set measures, one based on Hausdorff distance, the other based on Hamming distance.

Color has been widely used in content-based image retrieval (CBIR) applications. In such applications the color properties of an image are usually characterized by the probability distribution of the colors in the image. A distance measure is then used to measure the (dis-)similarity between images based on the descriptions of their color distributions in order to quickly find relevant images. The development and investigation of statistical methods for robust representations of such distributions, the construction of distance measures between them and their applications in efficient retrieval, browsing, and structuring of very large image databases are the main contributions of the thesis. In particular we have addressed the following problems in CBIR. Firstly, different non-parametric density estimators are used to describe color information for CBIR applications. Kernel-based methods using nonorthogonal bases together with a Gram-Schmidt procedure and the application of the Fourier transform are introduced and compared to previously used histogram-based methods. Our experiments show that efficient use of kernel density estimators improves the retrieval performance of CBIR. The practical problem of how to choose an optimal smoothing parameter for such density estimators as well as the selection of the histogram bin-width for CBIR applications are also discussed. Distance measures between color distributions are then described in a differential geometry-based framework. This allows the incorporation of geometrical features of the underlying color space into the distance measure between the probability distributions. The general framework is illustrated with two examples: Normal distributions and linear representations of distributions. The linear representation of color distributions is then used to derive new compact descriptors for color-based image retrieval. These descriptors are based on the combination of two ideas: Incorporating information from the structure of the color space with information from images and application of projection methods in the space of color distribution and the space of differences between neighboring color distributions. In our experiments we used several image databases containing more than 1,300,000 images. The experiments show that the method developed in this thesis is very fast and that the retrieval performance chievedcompares favorably with existing methods. A CBIR system has been developed and is currently available at http://www.media.itn.liu.se/cse. We also describe color invariant descriptors that can be used to retrieve images of objects independent of geometrical factors and the illumination conditions under which these images were taken. Both statistics- and physics-based methods are proposed and examined. We investigated the interaction between light and material using different physical models and applied the theory of transformation groups to derive geometry color invariants. Using the proposed framework, we are able to construct all independent invariants for a given physical model. The dichromatic reflection model and the Kubelka-Munk model are used as examples for the framework. The proposed color invariant descriptors are then applied to both CBIR, color image segmentation, and color correction applications. In the last chapter of the thesis we describe an industrial application where different color correction methods are used to optimize the layout of a newspaper page. / <p>A search engine based, on the methodes discribed in this thesis, can be found at http://pub.ep.liu.se/cse/db/?. Note that the question mark must be included in the address.</p>

Diese Diplomarbeit führt kurz in das Thema Information Retrieval mit den Schwerpunkten Evaluation und Evaluationskampagnen ein. Im Anschluss wird anhand der Nachteile eines vorhandenen Retrieval Systems ein neues Retrieval Framework zur experimentellen Evaluation von Ansätzen aus dem Information Retrieval entworfen und umgesetzt. Die Komponenten des Frameworks sind dabei so abstrakt angelegt, dass verschiedene, bestehende Retrieval Systeme, wie zum Beispiel Apache Lucene oder Terrier, integriert werden können. Anhand einer Referenzimplementierung für den ImageCLEF Photographic Retrieval Task des ImageCLEF Tracks des Cross Language Evaluation Forums wird die Funktionsfähigkeit des Frameworks überprüft und bestätigt.

Les documents visuels comprenant des images et des vidéos sont en croissance rapide sur Internet et dans nos collections personnelles. Cela nécessite une analyse automatique du contenu visuel qui fait appel à la conception de méthodes intelligentes pour correctement indexer, rechercher et récupérer des images et des vidéos. Cette thèse vise à améliorer la détection automatique des concepts dans les vidéos sur Internet. Nos contributions portent sur des différents niveaux dans le cadre de détection de concept et peuvent être divisés en trois parties principales. La première partie se focalise sur l’amélioration du modèle de représentation des vidéos « Bag-of-Words (BOW) » en proposant un nouveau mécanisme de construction qui utilise des étiquettes de concepts et une autre technique qui ajoute un raffinement à la signature BOW basée sur la distribution de ses éléments. Nous élaborons ensuite des méthodes pour intégrer des entités semblables et dissemblables pour construire des modèles de reconnaissance améliorés dans la deuxième partie. A ce stade-là, nous observons l’information potentielle que les concepts partagent et construisons des modèles pour les méta-concepts dont sont dérivés les résultats spécifiques de concepts. Cela améliore la reconnaissance des concepts qui ont peu d’exemples annotés. Enfin, nous concevons certaines méthodes d'apprentissage semi-supervisé pour bénéficier de la quantité importante de données non étiquetées. Nous proposons des techniques pour améliorer l'algorithme de cotraining avec une sélection optimale des classifieurs utilisés. / Visual material comprising images and videos is growing ever so rapidly over the internet and in our personal collections. This necessitates automatic understanding of the visual content which calls for the conception of intelligent methods to correctly index, search and retrieve images and videos. This thesis aims at improving the automatic detection of concepts in the internet videos by exploring all the available information and putting the most beneficial out of it to good use. Our contributions address various levels of the concept detection framework and can be divided into three main parts. The first part improves the Bag of Words (BOW) video representation model by proposing a novel BOW construction mechanism using concept labels and by including a refinement to the BOW signature based on the distribution of its elements. We then devise methods to incorporate knowledge from similar and dissimilar entities to build improved recognition models in the second part. Here we look at the potential information that the concepts share and build models for meta-concepts from which concept specific results are derived. This improves recognition for concepts lacking labeled examples. Lastly we contrive certain semi-supervised learning methods to get the best of the substantial amount of unlabeled data. We propose techniques to improve the semi-supervised cotraining algorithm with optimal view selection.

Este trabalho foi desenvolvido com o intuito de se estabelecer uma metodologia para a classificação das cores de imagens digitais em cores perceptuais para se gerar um vetor de características que permita recuperar imagens através de seu conteúdo em uma base de dados. Em trabalhos e estudos correlatos analisados, as metodologias de agrupamento das diversas cores possíveis de uma imagem não permitem uma associação entre a cor digitalizada e a cor percebida por seres humanos. Estudos mostram que a maioria das culturas humanas associam às cores apenas onze termos: vermelho, amarelo, violeta, azul, verde, rosa, marrom, preto, branco, laranja e cinza. Este trabalho propõe, portanto, uma metodologia baseada em regras da lógica fuzzy, que permite associar a todas as possíveis cores de imagens digitais uma das onze cores culturais definidas, criando assim um histograma perceptual de cores. Isso permitiu a geração de um vetor de características para a recuperação de imagens baseada em conteúdo em uma base de dados. / This work aims at establishing a digital image classification methodology based on perceptual colors, by generating a feature vector that allows retrieving images from a database by their content. In related works the methodologies of grouping the diverse possible colors of an image do not allow associate digitized colors and those colors perceived by human beings. Studies show that the majority of human being culture associates only eleven terms to all the possible colors: red, yellow, blue, green, pink, brown, black, white, purple, orange and gray. This work purpose a methodology based on fuzzy logic that allows to associate the eleven cultural color terms with all of digitized colors by a perceptual color histogram. The image color quantization generates a feature vector used for content-based image retrieval. The results show that it is possible to use the perceptual color histogram for CBIR and in the semantic gap reduction.

Este trabalho foi desenvolvido com o intuito de se estabelecer uma metodologia para a classificação das cores de imagens digitais em cores perceptuais para se gerar um vetor de características que permita recuperar imagens através de seu conteúdo em uma base de dados. Em trabalhos e estudos correlatos analisados, as metodologias de agrupamento das diversas cores possíveis de uma imagem não permitem uma associação entre a cor digitalizada e a cor percebida por seres humanos. Estudos mostram que a maioria das culturas humanas associam às cores apenas onze termos: vermelho, amarelo, violeta, azul, verde, rosa, marrom, preto, branco, laranja e cinza. Este trabalho propõe, portanto, uma metodologia baseada em regras da lógica fuzzy, que permite associar a todas as possíveis cores de imagens digitais uma das onze cores culturais definidas, criando assim um histograma perceptual de cores. Isso permitiu a geração de um vetor de características para a recuperação de imagens baseada em conteúdo em uma base de dados. / This work aims at establishing a digital image classification methodology based on perceptual colors, by generating a feature vector that allows retrieving images from a database by their content. In related works the methodologies of grouping the diverse possible colors of an image do not allow associate digitized colors and those colors perceived by human beings. Studies show that the majority of human being culture associates only eleven terms to all the possible colors: red, yellow, blue, green, pink, brown, black, white, purple, orange and gray. This work purpose a methodology based on fuzzy logic that allows to associate the eleven cultural color terms with all of digitized colors by a perceptual color histogram. The image color quantization generates a feature vector used for content-based image retrieval. The results show that it is possible to use the perceptual color histogram for CBIR and in the semantic gap reduction.

Diese Diplomarbeit führt kurz in das Thema Information Retrieval mit den Schwerpunkten Evaluation und Evaluationskampagnen ein. Im Anschluss wird anhand der Nachteile eines vorhandenen Retrieval Systems ein neues Retrieval Framework zur experimentellen Evaluation von Ansätzen aus dem Information Retrieval entworfen und umgesetzt. Die Komponenten des Frameworks sind dabei so abstrakt angelegt, dass verschiedene, bestehende Retrieval Systeme, wie zum Beispiel Apache Lucene oder Terrier, integriert werden können. Anhand einer Referenzimplementierung für den ImageCLEF Photographic Retrieval Task des ImageCLEF Tracks des Cross Language Evaluation Forums wird die Funktionsfähigkeit des Frameworks überprüft und bestätigt.