The focus of this research is on a tolerance space-based approach to image analysis and correspondence. The problem considered in this thesis is one of extracting perceptually relevant information from groups of objects based on their descriptions. Object descriptions are represented by feature vectors containing probe function values in a manner similar to feature extraction in pattern classification theory. The motivation behind this work is the synthesizing of human perception of nearness for improvement of image processing systems. In these systems, the desired output is similar to the output of a human performing the same task. Thus, it is important to have systems that accurately model human perception. Near set theory provides a framework for measuring the similarity of objects based on features that describe them in much the same way that humans perceive the similarity of objects. In this thesis, near set theory is presented and advanced, and work is presented toward a near set approach to performing content-based image retrieval. Furthermore, results are given based on these new techniques and future work is presented. The contributions of this thesis are: the introduction of a nearness measure to determine the degree that near sets resemble each other; a systematic approach to finding tolerance classes, together with proofs demonstrating that the proposed approach will find all tolerance classes on a set of objects; an approach to applying near set theory to images; the application of near set theory to the problem of content-based image retrieval; demonstration that near set theory is well suited to solving problems in which the outcome is similar to that of human perception; two other near set measures, one based on Hausdorff distance, the other based on Hamming distance.

The main contribution of this thesis, is to define similarity measures between two images with the main focus on content-based image retrieval (CBIR). Each image is considered as a set of visual elements that can be described with a set of visual descriptions (features). The similarity between images is then defined as the nearness between sets of elements based on a tolerance and a fuzzy tolerance relation. A tolerance relation is used to describe the approximate nature of the visual perception. A fuzzy tolerance relation is adopted to eliminate the need for a sharp threshold and hence model the gradual changes in perception of similarities. Three real valued similarity measures as well as a fuzzy valued similarity measure are proposed. All of the methods are then used in two CBIR experiments and the results are compared with classical measures of distance (namely, Kantorovich, Hausdorff and Mahalanobis). The results are compared with other published research papers. An important advantage of the proposed methods is shown to be their effectiveness in an unsupervised setting with no prior information. Eighteen different features (based on color, texture and edge) are used in all the experiments. A feature selection algorithm is also used to train the system in choosing a suboptimal set of visual features.

In this thesis, a Content Based Image Retrieval (CBIR) system to query the objects in an image database is proposed. Images are represented as collections of regions after being segmented with Normalized Cuts algorithm. MPEG-7 content descriptors are used to encode regions in a 239-dimensional feature space. User of the proposed CBIR system decides which objects to query and labels exemplar regions to train the system using a graphical interface. Fuzzy ARTMAP algorithm is used to learn the mapping between feature vectors and binary coded class identification numbers. Preliminary recognition experiments prove the power of fuzzy ARTMAP as a region classifier. After training, features of all regions in the database are extracted and classified. Simple index files enabling fast access to all regions from a given class are prepared to be used in the querying phase. To retrieve images containing a particular object, user opens an image and selects a query region together with a label in the graphical interface of our system. Then the system ranks all regions in the indexed set of the query class with respect to their L2 (Euclidean) distance to the query region and displays resulting images. During retrieval experiments, comparable class precisions with respect to exhaustive searching of the database are maintained which demonstrates e ectiveness of the classifier in narrowing down the search space.

Este trabalho apresenta o estudo e implementação de um banco de conhecimentos para auxiliar o diagnóstico de lesões da mama por inspeção visual, permitindo ao médico consultas através de características pictóricas da imagem e a comparação visual entre imagem investigada e imagens previamente classificadas e suas informações clínicas. As imagens encontram-se classificadas no banco de conhecimentos segundo o padrão \"Breast imaging reporting and data systems\" (BI-RADS) do Colégio Americano de Radiologia. A seleção das imagens, informações clínicas representativas, bem como sua classificação foram realizada em conjunto com médicos radiologistas do Centro de Ciências das Imagens e Física Médica (CCIFM) da Faculdade de Medicina de Ribeirão Preto (FMRP) da Universidade de São Paulo (USP). O processo de indexação e recuperação das imagens é baseado em atributos de textura extraídos de \"Regions of interest\" (ROIs) previamente estabelecidas em mamogramas digitalizados. Para simplificar este processo, foi utilizado a Análise de Componentes Principais (PCA), que visa a redução do número de atributos de textura e as informações redundantes existentes. Os melhores resultados obtidos foram para as ROIs 139 (Precisão = 0.80), 59 (Precisão = 0.86) e um valor de 100% de acerto para a ROI 40. / This work presents the survey and implementation of a database of knowledge to aid the diagnostic of breast lesions through visual inspection, allowing the physician a seach through the characteristics of the contents of the image and the visual comparison between the analysed image and the previously classified images and its clinical information. The images are classified into the database of knowledge according to the pattern Breast Imaging Reporting and Data Systems (BI-RADS) of the American College of Radiology. The selection of the images, the representative clinical information, as well as its classification have been performed in conjunction with practictioners radiologists of the Centro de Ciências das Imagens e Física Médica (CCIFM) from Faculdade de Medicina de Ribeirão Preto (FMRP) from Universidade de São Paulo (USP). The process of indexing and retrieving the images is based on characteristic of the texture extracted from the regions of interest (ROIs) previously established through scanned mammograms. To simplify this path, the Principal Components Analysis (PCA) was used it aims the reduction of the number of features of texture and the existing redundant information. The best results obtained were to the ROIs 139 (precision = 0.80), 59 (precision = 0.86) and a value of 100% of precision for ROI 40.

Os Sistemas de Gerenciamento de Bases de Dados devem atualmente ser capazes de gerenciar dados complexos, como dados multimídia, sequências genéticas, séries temporais, além dos dados tradicionais. Em consultas em grandes coleções de dados complexos, a similaridade entre os dados é o fator mais importante, e pode ser adequadamente expressada quando esses dados são representados em espaços métricos. Independentemente do domínio de um tipo de dados, existem aplicações que devem acompanhar a evolução temporal dos elementos de dados. Porém, os Métodos de Acesso Métrico existentes consideram que os dados são imutáveis com o decorrer do tempo. Visando o tratamento do tempo e dinamicidade em dados representados em espaços métricos, o trabalho apresentado nesta tese foi desenvolvido em duas frentes principais de atividades. A primeira frente tratou da inclusão das operações de remoção e atualização em métodos de acesso métrico, e visa atender às necessidades de domínios de aplicação em que dados em espaços métricos sofram atualização frequente, independentemente de necessitarem de tratamento temporal. Desta frente de atividades também resultou um novo método de otimização de àrvores métricas, baseado no algoritmo de remoção desenvolvido. A segunda frente de atividades aborda a inclusão do conceito de evolução temporal em dados representados em espaços métricos. Para isso foi proposto o Espaço Métrico-temporal, um modelo de representação de dados que permite a comparação de elementos métricos associado a informações temporais. O modelo conta com um método para identificar as contribuições relativas das componentes métrica e temporal no cálculo da similaridade. Também foram apresentadas estratégias para análise de trajetórias de dados métricos com o decorrer do tempo, através da imersão de espaços métrico-temporais em espaços dimensionais. Por fim, foi apresentado um novo método de balanceamento de múltiplos descritores para representação de imagens, fruto de modificações no método proposto para identificar as contribuições das componentes que podem formar um espaço métrico-temporal / Nowadays, the Database Management Systems (DBMS) must be able to manage complex data, such as multimedia data, genetic sequences, temporal series, besides the traditional data. For queries on large collections of complex data, the similarity among elements is the most relevant concept, and it can be adequately expressed when data are represented in metric spaces. Regardless of the data domain, there are applications that must tracking the evolution of data over time However, the existing Metric Access Methods assume that the data elements are immutable. Aiming at both treating time and allowing changes in metric data, the work presented in this thesis consisted of two main parts. The first part addresses the inclusion of the operations for element remotion and updating in metric access methods. These operations are meant to application domains that work with metric data that changes over time, regardless of the needed to manage temporal information. A new method for metric trees optimization was also developed in this part of the work. It was based on the proposed remotion algorithm. The second part of the thesis addresses including the temporal evolution concept in data represented in metric spaces. The Metric-Temporal Space was proposed, a representation model to allow comparing elements consisting of metric data with temporal information associated. The model includes a method to identify the relative contributions of the temporal and the metric components in the final similarity calculation. Strategies for trajectory analysis of metric data over time was also presented, through the immersion of metric-temporal spaced in dimensional spaces. Finally, a new method for weighting multiple image descriptors was presented. It was derived from changes in the proposed method to identify the contributions of the components of the metric-temporal space

Sistemas PACS (Picture Archieving and Communication Systems) têm sido desenvolvidos para armazenar de maneira integrada tanto os dados textuais e temporais dos pacientes quanto as imagens dos exames médicos a que eles se submetem para ampliar o uso das imagens no auxílio ao diagnóstico. Outra ferramenta valiosa para o auxílio ao diagnóstico médico são os sistemas CAD (Computer-Aided Diagnosis), para os quais pesquisas recentes mostram que o seu uso melhora significativamente a performance dos radiologistas em detectar corretamente anomalias. Dentro deste contexto, muitos trabalhos têm buscado métodos que possam reduzir o problema do \"gap semântico\", que refere-se ao que é perdido pela descrição sucinta da imagem e o que o usuário espera recuperar/reconhecer utilizando tal descrição. A grande maioria dos sistemas CBIR (do inglês Content-based image retrieval ) utiliza características primárias (baixo nível) para descrever elementos relevantes da imagem e proporcionar recuperação baseada em conteúdo. É necessário \"fundir\" múltiplos vetores com uma caracterí?stica em um vetor composto de características que possui baixa dimensionalidade e que ainda preserve, dentro do possível, as informações necessárias para a recuperação de imagens. O objetivo deste trabalho é propor novos extratores de características, baseados nos subespaços de imagens médicas gerados por transformadas wavelets. Estas características são armazenadas em vetores de características, os quais representam numericamente as imagens e permitindo assim sua busca por semelhança utilizando o conteúdo das próprias imagens. Esses vetores serão usados em um sistema de mineração de imagens em desenvolvimento no GBdI-ICMC-USP, o StARMiner, permitindo encontrar padrões pertencentes às imagens que as levem a ser classificadas em categorias / Picture Archiving and Communication Systems (PACS) aim at storing all the patients data, including their images, time series and textual description, allowing fast and effective transfer of information among devices and workstations. Therefore, PACS can be a powerful tool on improving the decision making during a diagnosing process. The CAD (Computer-Aided Diagnosis) systems have been recently employed to improve the diagnosis confidence, and recent research shows that they can effectively raise the radiologists performance on detecting anomalies on images. Content-based image retrieval (CBIR) techniques are essential to support CAD systems, and can significantly improve the PACS applicability. CBIR works on raw level features extracted from the images to describe the most meaningful characteristics of the images following a specific criterium. Usually, it is necessary to put together several features to compose a feature vector to describe an image more precisely. Therefore, the dimensionality of the feature vector is frequently large and many features can be correlated to each other. The objective of this Master Dissertation is to build new image features, based on wavelet-generated subspaces. The features form the feature vector, which succinctly represent the images and are used to process similarity queries. The feature vectors are analyzed by the StARMiner system, under development in the GbdI-ICMC-USP, in order to find the most meaningful features to represent the images as well as to find patterns in the images that allow them to be classified into categories. The project developed was evaluated with three different image sets and the results are promising

Two novel contributions to Content Based Image Retrieval are presented and discussed. The first is a search engine for font recognition. The intended usage is the search in very large font databases. The input to the search engine is an image of a text line, and the output is the name of the font used when printing the text. After pre-processing and segmentation of the input image, a local approach is used, where features are calculated for individual characters. The method is based on eigenimages calculated from edge filtered character images, which enables compact feature vectors that can be computed rapidly. A system for visualizing the entire font database is also proposed. Applying geometry preserving linear- and non-linear manifold learning methods, the structure of the high-dimensional feature space is mapped to a two-dimensional representation, which can be reorganized into a grid-based display. The performance of the search engine and the visualization tool is illustrated with a large database containing more than 2700 fonts. The second contribution is the inclusion of color-based emotion-related properties in image retrieval. The color emotion metric used is derived from psychophysical experiments and uses three scales: activity, weight and heat. It was originally designed for single-color combinations and later extended to include pairs of colors. A modified approach for statistical analysis of color emotions in images, involving transformations of ordinary RGB-histograms, is used for image classification and retrieval. The methods are very fast in feature extraction, and descriptor vectors are very short. This is essential in our application where the intended use is the search in huge image databases containing millions or billions of images. The proposed method is evaluated in psychophysical experiments, using both category scaling and interval scaling. The results show that people in general perceive color emotions for multi-colored images in similar ways, and that observer judgments correlate with derived values. Both the font search engine and the emotion based retrieval system are implemented in publicly available search engines. User statistics gathered during a period of 20 respectively 14 months are presented and discussed.

Social networking has encouraged users to find new ways to create, post, search, collaborate and share information of various forms. Unfortunately there is a lot of data in social networks that is not well-managed, which makes the experience within these networks less than optimal. Therefore people generally need more and more time as well as advanced tools that are used for seeking relevant information. A new search paradigm is emerging, where the user perspective is completely reversed: from finding to being found. The aim of present thesis research is to evaluate two approaches of identifying content of interest: user-initiated and system-initiated. The most suitable approaches will be implemented. Various recommendation systems for system-initiated content recommendations will also be investigated, and the best suited ones implemented. The analysis that was performed demonstrated that the users have used all of the implemented approaches and have provided positive and negative comments for all of them, which reinforces the belief that the methods for the implementation were selected correctly. The results of the user testing of the methods were evaluated based on the amount of time it took the users to find the desirable content and on the correspondence of the result compared to the user expectations.

Cette thèse a pour objectif de proposer un système de reconnaissance d’images utilisant des informations attentionnelles. Nous nous intéressons à la capacité d’une telle approche à améliorer la complexité en temps de calcul et en utilisation mémoire pour la reconnaissance d’objets. Dans un premier temps, nous avons proposé d’utiliser un système d’attention visuelle comme filtre pour réduire le nombre de points d’intérêt générés par les détecteurs traditionnels [Awad 12]. En utilisant l’architecture attentionnelle proposée par Perreira da Silva comme filtre [Awad 12] sur la base d’images de VOC 2005, nous avons montré qu’un filtrage de 60% des points d’intérêt (extraits par Harris-Laplace et Laplacien) ne fait diminuer que légèrement la performance d’un système de reconnaissance d’objets (différence moyenne de AUC ~ 1%) alors que le gain en complexité est important (40% de gain en vitesse de calcul et 60% en complexité). Par la suite, nous avons proposé un descripteur hybride perceptuel-texture [Awad 14] qui caractérise les informations fréquentielles de certaines caractéristiques considérées comme perceptuellement intéressantes dans le domaine de l’attention visuelle, comme la couleur, le contraste ou l’orientation. Notre descripteur a l’avantage de fournir des vecteurs de caractéristiques ayant une dimension deux fois moindre que celle des descripteurs proposés dans l’état de l’art. L’expérimentation de ce descripteur sur un système de reconnaissance d’objets (le détecteur restant SIFT), sur la base d’images de VOC 2007, a montré une légère baisse de performance (différence moyenne de précision ~5%) par rapport à l’algorithme original, basé sur SIFT mais gain de 50% en complexité. Pour aller encore plus loin, nous avons proposé une autre expérimentation permettant de tester l’efficacité globale de notre descripteur en utilisant cette fois le système d’attention visuelle comme détecteur des points d’intérêt sur la base d’images de VOC 2005. Là encore, le système n’a montré qu’une légère baisse de performance (différence moyenne de précision ~3%) alors que la complexité est réduite de manière drastique (environ 50% de gain en temps de calcul et 70% en complexité). / The main objective of this thesis is to propose a pipeline for an object recognition algorithm, near to human perception, and at the same time, address the problems of Content Based image retrieval (CBIR) algorithm complexity : query run time and memory allocation. In this context, we propose a filter based on visual attention system to select salient points according to human interests from the interest points extracted by a traditionnal interest points detectors. The test of our approach, using Perreira Da Silva’s system as filter, on VOC 2005 databases, demonstrated that we can maintain approximately the same performance of a object recognition system by selecting only 40% of interest points (extracted by Harris-Laplace and Laplacian), while having an important gain in complexity (40% gain in query-run time and 60% in complexity). Furthermore, we address the problem of high dimensionality of descriptor in object recognition system. We proposed a new hybrid texture descriptor, representing the spatial frequency of some perceptual features extracted by a visual attention system. This descriptor has the advantage of being lower dimension vs. traditional descriptors. Evaluating our descriptor with an object recognition system (interest points detectors are Harris-Laplace & Laplacian) on VOC 2007 databases showed a slightly decrease in the performance (with 5% loss in Average Precision) compared to the original system, based on SIFT descriptor (with 50% complexity gain). In addition, we evaluated our descriptor using a visual attention system as interest point detector, on VOC 2005 databases. The experiment showed a slightly decrease in performance (with 3% loss in performance), meanwhile we reduced drastically the complexity of the system (with 50% gain in run-query time and 70% in complexity).

Online shopping has become an important part of people’s daily life with the rapid development of e-commerce. In some domains such as books, electronics, and CD/DVDs, online shopping has surpassed or even replaced the traditional shopping method. Compared with traditional retailing, e-commerce is information intensive. One of the key factors to succeed in e-business is how to facilitate the consumers’ approaches to discover a product. Conventionally a product search engine based on a keyword search or category browser is provided to help users find the product information they need. The general goal of a product search system is to enable users to quickly locate information of interest and to minimize users’ efforts in search and navigation. In this process human factors play a significant role. Finding product information could be a tricky task and may require an intelligent use of search engines, and a non-trivial navigation of multilayer categories. Searching for useful product information can be frustrating for many users, especially those inexperienced users. This dissertation focuses on developing a new visual product search system that effectively extracts the properties of unstructured products, and presents the possible items of attraction to users so that the users can quickly locate the ones they would be most likely interested in. We designed and developed a feature extraction algorithm that retains product color and local pattern features, and the experimental evaluation on the benchmark dataset demonstrated that it is robust against common geometric and photometric visual distortions. Besides, instead of ignoring product text information, we investigated and developed a ranking model learned via a unified probabilistic hypergraph that is capable of capturing correlations among product visual content and textual content. Moreover, we proposed and designed a fuzzy hierarchical co-clustering algorithm for the collaborative filtering product recommendation. Via this method, users can be automatically grouped into different interest communities based on their behaviors. Then, a customized recommendation can be performed according to these implicitly detected relations. In summary, the developed search system performs much better in a visual unstructured product search when compared with state-of-art approaches. With the comprehensive ranking scheme and the collaborative filtering recommendation module, the user’s overhead in locating the information of value is reduced, and the user’s experience of seeking for useful product information is optimized.