Este trabalho apresenta o estudo e implementação de um banco de conhecimentos para auxiliar o diagnóstico de lesões da mama por inspeção visual, permitindo ao médico consultas através de características pictóricas da imagem e a comparação visual entre imagem investigada e imagens previamente classificadas e suas informações clínicas. As imagens encontram-se classificadas no banco de conhecimentos segundo o padrão \"Breast imaging reporting and data systems\" (BI-RADS) do Colégio Americano de Radiologia. A seleção das imagens, informações clínicas representativas, bem como sua classificação foram realizada em conjunto com médicos radiologistas do Centro de Ciências das Imagens e Física Médica (CCIFM) da Faculdade de Medicina de Ribeirão Preto (FMRP) da Universidade de São Paulo (USP). O processo de indexação e recuperação das imagens é baseado em atributos de textura extraídos de \"Regions of interest\" (ROIs) previamente estabelecidas em mamogramas digitalizados. Para simplificar este processo, foi utilizado a Análise de Componentes Principais (PCA), que visa a redução do número de atributos de textura e as informações redundantes existentes. Os melhores resultados obtidos foram para as ROIs 139 (Precisão = 0.80), 59 (Precisão = 0.86) e um valor de 100% de acerto para a ROI 40. / This work presents the survey and implementation of a database of knowledge to aid the diagnostic of breast lesions through visual inspection, allowing the physician a seach through the characteristics of the contents of the image and the visual comparison between the analysed image and the previously classified images and its clinical information. The images are classified into the database of knowledge according to the pattern Breast Imaging Reporting and Data Systems (BI-RADS) of the American College of Radiology. The selection of the images, the representative clinical information, as well as its classification have been performed in conjunction with practictioners radiologists of the Centro de Ciências das Imagens e Física Médica (CCIFM) from Faculdade de Medicina de Ribeirão Preto (FMRP) from Universidade de São Paulo (USP). The process of indexing and retrieving the images is based on characteristic of the texture extracted from the regions of interest (ROIs) previously established through scanned mammograms. To simplify this path, the Principal Components Analysis (PCA) was used it aims the reduction of the number of features of texture and the existing redundant information. The best results obtained were to the ROIs 139 (precision = 0.80), 59 (precision = 0.86) and a value of 100% of precision for ROI 40.

La recherche d’image est une thématique de recherche très active. Plusieurs approches permettant d'établir un lien entre les descripteurs de bas niveau et la sémantique ont été proposées. Parmi celles-là, nous citons la reconnaissance d'objets, les ontologies et le bouclage de pertinence. Cependant, leur limitation majeure est la haute dépendance d’une ressource externe et l'incapacité à combiner efficacement l'information visuelle et sémantique. Cette thèse propose un système basé sur un graphe de patrons, la sélection ciblée des descripteurs pour la phase en ligne et l'amélioration de la visualisation des résultats. L'idée est de (1) construire un graphe de patrons composé d'une ontologie modulaire et d'un modèle basé graphe pour l'organisation de l'information sémantique, (2) de construire un ensemble de collections de descripteurs pour guider la sélection des descripteurs à appliquer durant la recherche et (3) améliorer la visualisation des résultats en intégrant les relations sémantiques déduite du graphe de patrons.Durant la construction de graphe de patrons, les modules ontologiques associés à chaque domaine sont automatiquement construits. Le graphe de régions résume l'information visuelle en un format plus condensé et la classifie selon son domaine. Le graphe de patrons est déduit par composition de modules ontologiques. Notre système a été testé sur trois bases d’images. Les résultats obtenus montrent une amélioration au niveau du processus de recherche, une meilleure adaptation des descripteurs visuels utilisés aux domaines couverts et une meilleure visualisation des résultats qui diminue le niveau d’abstraction par rapport à leur logique de génération. / Image retrieval is a very active search area. Several image retrieval approaches that allow mapping between low-level features and high-level semantics have been proposed. Among these, one can cite object recognition, ontologies, and relevance feedback. However, their main limitation concern their high dependence on reliable external resources and lack of capacity to combine semantic and visual information.This thesis proposes a system based on a pattern graph combining semantic and visual features, relevant visual feature selection for image retrieval and improvement of results visualization. The idea is (1) build a pattern graph composed of a modular ontology and a graph-based model, (2) to build visual feature collections to guide feature selection during online retrieval phase and (3) improve the retrieval results visualization with the integration of semantic relations.During the pattern graph building, ontology modules associated to each domain are automatically built using textual corpuses and external resources. The region's graphs summarize the visual information in a condensed form and classify it given its semantics. The pattern graph is obtained using modules composition. In visual features collections building, association rules are used to deduce the best practices on visual features use for image retrieval. Finally, results visualization uses the rich information on images to improve the results presentation.Our system has been tested on three image databases. The results show an improvement in the research process, a better adaptation of the visual features to the domains and a richer visualization of the results.

Over the years, exemplar-based methods have yielded significant improvements over their model-based counterparts in image synthesis applications. Notably, texture synthesis algorithms using an exemplar-based approach have shown success where traditional stochastic methods failed. As an illustrative example, I will present an exemplar-based approach that yields substantial benefits for user-guided terrain synthesis using Digital Elevation Models (DEMs). This success is realized through exploitation of structural properties of natural terrain. In addition to their proliferation in the image synthesis domain, as annotated image datasets become increasingly available, exemplar-based methods are also gaining in popularity for image analysis applications. This thesis addresses the intersection between exemplar-based analysis and the problem of content-based image retrieval (CBIR). A basic problem in CBIR is the process by which the search criteria are refined by the user through the manipulation of returned exemplars. Exemplar-based analysis is particularly well-suited to query refinement due to its interpretability and the ease with which it can be incorporated into an interactive system. I investigate this connection in the domain of Computer-Assisted Diagnosis (CAD) of dermatological images. I demonstrate that exemplar-based approaches in CBIR can be effective for diagnosing pigmented skin lesions (PSLs). I will present an exemplar-based algorithm for segmenting PSLs in dermatoscopic images. In addition, I will present a generalized representation of dermoscopic features for detection and matching. This representation not only leads to an exemplar-based PSL diagnosis scheme, but it also enables us to realize interactive region-of-interest retrieval, which includes a relevance feedback mechanism to facilitate more flexible query-by-example analysis. Finally, I will assess the benefit of this CBIR-CAD approach through both quantitative evaluations and user studies.

Current Web image search engines, such as Google or Bing Images, adopt a hybrid search approach in which a text-based query (e.g. "apple") is used to retrieve a set of relevant images, which are then refined by the user (e.g. by re-ranking the retrieved images based on similarity to a selected example). This approach makes it possible to use both text information (e.g. the initial query) and image features (e.g. as part of the refinement stage) to identify images which are relevant to the user. One limitation of these current systems is that text and image features are treated as independent components and are often used in a decoupled manner. This work proposes to develop an integrated hybrid search method which leverages the synergies between text and image features. Recently, there has been tremendous progress in the computer vision community in learning models of visual concepts from collections of example images. While impressive performance has been achieved on standardized data sets, scaling these methods so that they are capable of working at web scale remains a significant challenge. This work will develop approaches to visual modeling that can be scaled to address the task of retrieving billions of images on the Web. Specifically, we propose to address two research issues related to integrated text- and image-based retrieval. First, we will explore whether models of visual concepts which are learned from collections of web images can be utilized to improve the image ranking associated with a text-based query. Second, we will investigate the hypothesis that the click-patterns associated with standard web image search engines can be utilized to learn query-specific image similarity measures that support improved query-refinement performance. We will evaluate our research by constructing a prototype integrated hybrid retrieval system based on the data from 300K real-world image queries. We will conduct user-studies to evaluate the effectiveness of our learned similarity measures and quantify the benefit of our method in real world search tasks such as target search.

The focus of this research is on a tolerance space-based approach to image analysis and correspondence. The problem considered in this thesis is one of extracting perceptually relevant information from groups of objects based on their descriptions. Object descriptions are represented by feature vectors containing probe function values in a manner similar to feature extraction in pattern classification theory. The motivation behind this work is the synthesizing of human perception of nearness for improvement of image processing systems. In these systems, the desired output is similar to the output of a human performing the same task. Thus, it is important to have systems that accurately model human perception. Near set theory provides a framework for measuring the similarity of objects based on features that describe them in much the same way that humans perceive the similarity of objects. In this thesis, near set theory is presented and advanced, and work is presented toward a near set approach to performing content-based image retrieval. Furthermore, results are given based on these new techniques and future work is presented. The contributions of this thesis are: the introduction of a nearness measure to determine the degree that near sets resemble each other; a systematic approach to finding tolerance classes, together with proofs demonstrating that the proposed approach will find all tolerance classes on a set of objects; an approach to applying near set theory to images; the application of near set theory to the problem of content-based image retrieval; demonstration that near set theory is well suited to solving problems in which the outcome is similar to that of human perception; two other near set measures, one based on Hausdorff distance, the other based on Hamming distance.

The main contribution of this thesis, is to define similarity measures between two images with the main focus on content-based image retrieval (CBIR). Each image is considered as a set of visual elements that can be described with a set of visual descriptions (features). The similarity between images is then defined as the nearness between sets of elements based on a tolerance and a fuzzy tolerance relation. A tolerance relation is used to describe the approximate nature of the visual perception. A fuzzy tolerance relation is adopted to eliminate the need for a sharp threshold and hence model the gradual changes in perception of similarities. Three real valued similarity measures as well as a fuzzy valued similarity measure are proposed. All of the methods are then used in two CBIR experiments and the results are compared with classical measures of distance (namely, Kantorovich, Hausdorff and Mahalanobis). The results are compared with other published research papers. An important advantage of the proposed methods is shown to be their effectiveness in an unsupervised setting with no prior information. Eighteen different features (based on color, texture and edge) are used in all the experiments. A feature selection algorithm is also used to train the system in choosing a suboptimal set of visual features.

The focus of this research is on a tolerance space-based approach to image analysis and correspondence. The problem considered in this thesis is one of extracting perceptually relevant information from groups of objects based on their descriptions. Object descriptions are represented by feature vectors containing probe function values in a manner similar to feature extraction in pattern classification theory. The motivation behind this work is the synthesizing of human perception of nearness for improvement of image processing systems. In these systems, the desired output is similar to the output of a human performing the same task. Thus, it is important to have systems that accurately model human perception. Near set theory provides a framework for measuring the similarity of objects based on features that describe them in much the same way that humans perceive the similarity of objects. In this thesis, near set theory is presented and advanced, and work is presented toward a near set approach to performing content-based image retrieval. Furthermore, results are given based on these new techniques and future work is presented. The contributions of this thesis are: the introduction of a nearness measure to determine the degree that near sets resemble each other; a systematic approach to finding tolerance classes, together with proofs demonstrating that the proposed approach will find all tolerance classes on a set of objects; an approach to applying near set theory to images; the application of near set theory to the problem of content-based image retrieval; demonstration that near set theory is well suited to solving problems in which the outcome is similar to that of human perception; two other near set measures, one based on Hausdorff distance, the other based on Hamming distance.

The main contribution of this thesis, is to define similarity measures between two images with the main focus on content-based image retrieval (CBIR). Each image is considered as a set of visual elements that can be described with a set of visual descriptions (features). The similarity between images is then defined as the nearness between sets of elements based on a tolerance and a fuzzy tolerance relation. A tolerance relation is used to describe the approximate nature of the visual perception. A fuzzy tolerance relation is adopted to eliminate the need for a sharp threshold and hence model the gradual changes in perception of similarities. Three real valued similarity measures as well as a fuzzy valued similarity measure are proposed. All of the methods are then used in two CBIR experiments and the results are compared with classical measures of distance (namely, Kantorovich, Hausdorff and Mahalanobis). The results are compared with other published research papers. An important advantage of the proposed methods is shown to be their effectiveness in an unsupervised setting with no prior information. Eighteen different features (based on color, texture and edge) are used in all the experiments. A feature selection algorithm is also used to train the system in choosing a suboptimal set of visual features.

In this thesis, a Content Based Image Retrieval (CBIR) system to query the objects in an image database is proposed. Images are represented as collections of regions after being segmented with Normalized Cuts algorithm. MPEG-7 content descriptors are used to encode regions in a 239-dimensional feature space. User of the proposed CBIR system decides which objects to query and labels exemplar regions to train the system using a graphical interface. Fuzzy ARTMAP algorithm is used to learn the mapping between feature vectors and binary coded class identification numbers. Preliminary recognition experiments prove the power of fuzzy ARTMAP as a region classifier. After training, features of all regions in the database are extracted and classified. Simple index files enabling fast access to all regions from a given class are prepared to be used in the querying phase. To retrieve images containing a particular object, user opens an image and selects a query region together with a label in the graphical interface of our system. Then the system ranks all regions in the indexed set of the query class with respect to their L2 (Euclidean) distance to the query region and displays resulting images. During retrieval experiments, comparable class precisions with respect to exhaustive searching of the database are maintained which demonstrates e ectiveness of the classifier in narrowing down the search space.

Los sistemas online para compartir fotos proporcionan una valiosa fuente de contenidos generado por el usuario (UGC). La mayor a de los sistemas de re- cuperaci on de im agenes Web utilizan las anotaciones textuales para rankear los resultados, sin embargo estas anotaciones no s olo ilustran el contenido visual de una imagen, sino que tambi en describen situaciones subjetivas, espaciales, temporales y sociales, que complican la tarea de b usqueda basada en palabras clave. La investigaci on en esta tesis se centra en c omo mejorar la recuperaci on de im agenes en sistemas de gran escala, es decir, la Web, combinando informaci on proporcionada por los usuarios m as el contenido visual de las im agenes. En el presente trabajo se exploran distintos tipos de UGC, tales como anotaciones de texto, anotaciones visuales, y datos de click-through, as como diversas t ecnicas para combinar esta informaci on con el objetivo de mejorar la recuperaci on de im agenes usando informaci on visual. En conclusi on, la investigaci on realizada en esta tesis se centra en la impor- tancia de incluir la informaci on visual en distintas etapas de la recuperaci on de contenido. Combinando informaci on visual con otras formas de UGC, es posible mejorar signi cativamente el rendimiento de un sistema de recuperaci on de im agenes y cambiar la experiencia del usuario en la b usqueda de contenidos multimedia en la Web. / Online photo sharing systems provide a valuable source of user generated content (UGC). Most Web image retrieval systems use textual annotations to rank the results, although these annotations do not only illustrate the visual content of an image, but also describe subjective, spatial, temporal, and social dimensions, complicating the task of keyword based search. The research in this thesis is focused on how to improve the retrieval of images in large scale context , i.e. the Web, using information provided by users combined with visual content from images. Di erent forms of UGC are explored, such as textual annotations, visual annotations, and click-through-data, as well as di erent techniques to combine these data to improve the retrieval of images using visual information. In conclusion, the research conducted in this thesis focuses on the impor- tance to include visual information into various steps of the retrieval of media content. Using visual information, in combination with various forms of UGC, can signi cantly improve the retrieval performance and alter the user experience when searching for multimedia content on the Web. 1