O objetivo deste projeto de Mestrado foi o estudo, a análise e o desenvolvimento de técnicas de Realimentação de Relevância (RR) para melhorar a respostas de consultas por similaridade que empregam técnicas de recuperação de imagens por conteúdo (do inglês content-based image retrieval - CBIR). A motivação para o desenvolvimento deste projeto veio do iRIS (internet Retrieval of Images System), que é um protótipo de servidor Web para o processamento de consultas por similaridade, em construção no GBdI (Grupo de Bases de Dados e Imagens) do ICMC-USP. O iRIS pode ser integrado a PACS (Picture and Archiving and Communication System) permitindo que estes possam recuperar imagens por semelhança. A principal restrição do uso de sistemas que incorporam CBIR é a descontinuidade semântica (semantic gap), que credita-se principalmente à utilização de características de baixo nível para descrever as imagens. As características mais utilizadas são baseadas em cor, textura e forma, e geralmente não conseguem mapear o que o usuário deseja/esperar recuperar, gerando um descontentamento do usuário em relação ao sistema. Entretanto, se sistema permitir a iteração do usuário na classificação do conjunto resposta e usar estas informações no processo de refinamento, as consultas podem ser re-processadas e os resultados tendem a atender a expectativa do usuário. Esse é o propósito das técnicas de realimentação de relevância. Este projeto desenvolveu duas técnicas de realimentação de relevância (RR): o RF Projection e o RF Multiple Point Projection. O ganho com a aplicação dessas técnicas foi expressivo, alcançando 29% a mais de precisão sobre a consulta original já na primeira iteração e 42% após 5 iterações. Os experimentos realizados com usuários mostraram que em média são executadas 3 iterações para chegar a um resultado satisfatório. Pelos resultados apresentados nos experimentos, podemos afirmar que RR é uma poderosa ferramenta para impulsionar o uso dos sistemas CBIR e aprimorar as consultas por similaridade. / This Master project aimed at studying, analyzing and developing relevance feedback (RF) techniques to enhance similarity queries that employ the content-based image retrieval (CBIR) approach. The motivation to develop this project came from the iRIS (internet Retrieval of Images System), which is a Web server prototype to process similarity queries. The iRIS can be integrated to a PACS (Picture and Archiving and Communication System) adding the functionality of retrieval images comparing their inherent alikeliness. The main reservation about using CBIR techniques is the semantic gap, because the general use of low level features to describe the images. The low level features, such as color, texture and shape, mostly cannot bridge the gap between what the users expect/want to what they get, generating disappointment and refusal of employing the system. However, if the user is allowed to interact with the system, classifying the query results and using such information on refinement steps, the queries can be reprocessed and the results tend to comply with the users’ expectation. This is just the core of the relevance feedback techniques. Looking at this scenario, this project developed two relevance feedback (RF) techniques: the RF Projection and the RF Multiple Point Projection. The improvements on the similarity queries were expressive going to up 29% with only one interaction, and to 42% on the fifth interaction, when compared to the original query. Experiments performed with users, have shown us that in average they run 3 iterations before get satisfactory results. By the results given by the experiment, one can claim that RF is a powerful approach to improve the use of CBIR systems and enhance similarity queries.

O objetivo deste projeto de Mestrado foi o estudo, a análise e o desenvolvimento de técnicas de Realimentação de Relevância (RR) para melhorar a respostas de consultas por similaridade que empregam técnicas de recuperação de imagens por conteúdo (do inglês content-based image retrieval - CBIR). A motivação para o desenvolvimento deste projeto veio do iRIS (internet Retrieval of Images System), que é um protótipo de servidor Web para o processamento de consultas por similaridade, em construção no GBdI (Grupo de Bases de Dados e Imagens) do ICMC-USP. O iRIS pode ser integrado a PACS (Picture and Archiving and Communication System) permitindo que estes possam recuperar imagens por semelhança. A principal restrição do uso de sistemas que incorporam CBIR é a descontinuidade semântica (semantic gap), que credita-se principalmente à utilização de características de baixo nível para descrever as imagens. As características mais utilizadas são baseadas em cor, textura e forma, e geralmente não conseguem mapear o que o usuário deseja/esperar recuperar, gerando um descontentamento do usuário em relação ao sistema. Entretanto, se sistema permitir a iteração do usuário na classificação do conjunto resposta e usar estas informações no processo de refinamento, as consultas podem ser re-processadas e os resultados tendem a atender a expectativa do usuário. Esse é o propósito das técnicas de realimentação de relevância. Este projeto desenvolveu duas técnicas de realimentação de relevância (RR): o RF Projection e o RF Multiple Point Projection. O ganho com a aplicação dessas técnicas foi expressivo, alcançando 29% a mais de precisão sobre a consulta original já na primeira iteração e 42% após 5 iterações. Os experimentos realizados com usuários mostraram que em média são executadas 3 iterações para chegar a um resultado satisfatório. Pelos resultados apresentados nos experimentos, podemos afirmar que RR é uma poderosa ferramenta para impulsionar o uso dos sistemas CBIR e aprimorar as consultas por similaridade. / This Master project aimed at studying, analyzing and developing relevance feedback (RF) techniques to enhance similarity queries that employ the content-based image retrieval (CBIR) approach. The motivation to develop this project came from the iRIS (internet Retrieval of Images System), which is a Web server prototype to process similarity queries. The iRIS can be integrated to a PACS (Picture and Archiving and Communication System) adding the functionality of retrieval images comparing their inherent alikeliness. The main reservation about using CBIR techniques is the semantic gap, because the general use of low level features to describe the images. The low level features, such as color, texture and shape, mostly cannot bridge the gap between what the users expect/want to what they get, generating disappointment and refusal of employing the system. However, if the user is allowed to interact with the system, classifying the query results and using such information on refinement steps, the queries can be reprocessed and the results tend to comply with the users’ expectation. This is just the core of the relevance feedback techniques. Looking at this scenario, this project developed two relevance feedback (RF) techniques: the RF Projection and the RF Multiple Point Projection. The improvements on the similarity queries were expressive going to up 29% with only one interaction, and to 42% on the fifth interaction, when compared to the original query. Experiments performed with users, have shown us that in average they run 3 iterations before get satisfactory results. By the results given by the experiment, one can claim that RF is a powerful approach to improve the use of CBIR systems and enhance similarity queries.

Hundreds of millions of users each day search the web and other repositories to meet their information needs. However, queries can fail to find documents due to a mismatch in terminology. Query expansion seeks to address this problem by automatically adding terms from highly ranked documents to the query. While query expansion has been shown to be effective at improving query performance, the gain in effectiveness comes at a cost: expansion is slow and resource-intensive. Current techniques for query expansion use fixed values for key parameters, determined by tuning on test collections. We show that these parameters may not be generally applicable, and, more significantly, that the assumption that the same parameter settings can be used for all queries is invalid. Using detailed experiments, we demonstrate that new methods for choosing parameters must be found. In conventional approaches to query expansion, the additional terms are selected from highly ranked documents returned from an initial retrieval run. We demonstrate a new method of obtaining expansion terms, based on past user queries that are associated with documents in the collection. The most effective query expansion methods rely on costly retrieval and processing of feedback documents. We explore alternative methods for reducing query-evaluation costs, and propose a new method based on keeping a brief summary of each document in memory. This method allows query expansion to proceed three times faster than previously, while approximating the effectiveness of standard expansion. We investigate the use of document expansion, in which documents are augmented with related terms extracted from the corpus during indexing, as an alternative to query expansion. The overheads at query time are small. We propose and explore a range of corpus-based document expansion techniques and compare them to corpus-based query expansion on TREC data. These experiments show that document expansion delivers at best limited benefits, while query expansion � including standard techniques and efficient approaches described in recent work � usually delivers good gains. We conclude that document expansion is unpromising, but it is likely that the efficiency of query expansion can be further improved.

Hundreds of millions of users each day search the web and other repositories to meet their information needs. However, queries can fail to find documents due to a mismatch in terminology. Query expansion seeks to address this problem by automatically adding terms from highly ranked documents to the query. While query expansion has been shown to be effective at improving query performance, the gain in effectiveness comes at a cost: expansion is slow and resource-intensive. Current techniques for query expansion use fixed values for key parameters, determined by tuning on test collections. We show that these parameters may not be generally applicable, and, more significantly, that the assumption that the same parameter settings can be used for all queries is invalid. Using detailed experiments, we demonstrate that new methods for choosing parameters must be found. In conventional approaches to query expansion, the additional terms are selected from highly ranked do cuments returned from an initial retrieval run. We demonstrate a new method of obtaining expansion terms, based on past user queries that are associated with documents in the collection. The most effective query expansion methods rely on costly retrieval and processing of feedback documents. We explore alternative methods for reducing query-evaluation costs, and propose a new method based on keeping a brief summary of each document in memory. This method allows query expansion to proceed three times faster than previously, while approximating the effectiveness of standard expansion. We investigate the use of document expansion, in which documents are augmented with related terms extracted from the corpus during indexing, as an alternative to query expansion. The overheads at query time are small. We propose and explore a range of corpus-based document expansion techniques and compare them to corpus-based query expansion on TREC data. These experiments show that document expansion delivers at best limited ben efits, while query expansion, including standard techniques and efficient approaches described in recent work, usually delivers good gains. We conclude that document expansion is unpromising, but it is likely that the efficiency of query expansion can be further improved.

Over the last decade, storage of non text-based data in databases has become an increasingly important trend in information management. Image in particular, has been gaining popularity as an alternative, and sometimes more viable, option for information storage. While this presents a wealth of information, it also creates a great problem in retrieving appropriate and relevant information during searching. This has resulted in an enormous growth of interest, and much active research, into the extraction of relevant information from non text-based databases. In particular,content-based image retrieval (CBIR) systems have been one of the most active areas of research. The retrieval principle of CBIR systems is based on visual features such as colour, texture, and shape or the semantic meaning of the images. To enhance the retrieval speed, most CBIR systems pre-process the images stored in the database. This is because feature extraction algorithms are often computationally expensive. If images are to be retrieved from the World-Wide-Web (WWW), the raw images have to be downloaded and processed in real time. In this case, the feature extraction speed becomes crucial. Ideally, systems should only use those feature extraction algorithms that are most suited for analysing the visual features that capture the common relationship between the images in hand. In this thesis, a statistical discriminant analysis based feature selection framework is proposed. Such a framework is able to select the most appropriate visual feature extraction algorithms by using relevance feedback only on the user labelled samples. The idea is that a smaller image sample group is used to analyse the appropriateness of each visual feature, and only the selected features will be used for image comparison and ranking. As the number of features is less, an improvement in the speed of retrieval is achieved. From experimental results, it is found that the retrieval accuracy for small sample data has also improved. Intelligent E-Business has been used as a case study in this thesis to demonstrate the potential of the framework in the application of image retrieval system. In addition, an inter-query framework has been proposed in this thesis. This framework is also based on the statistical discriminant analysis technique. A common approach in inter-query for a CBIR system is to apply the term-document approach. This is done by treating each image’s name or address as a term, and the query session as a document. However, scalability becomes an issue with this technique as the number of stored queries increases. Moreover, this approach is not appropriate for a dynamic image database environment. In this thesis, the proposed inter-query framework uses a cluster approach to capture the visual properties common to the previously stored queries. Thus, it is not necessary to “memorise” the name or address of the images. In order to manage the size of the user’s profile, the proposed framework also introduces a merging approach to combine clusters that are close-by and similar in their characteristics. Experiments have shown that the proposed framework has outperformed the short term learning approach. It also has the advantage that it eliminates the burden of the complex database maintenance strategies required in the term-document approach commonly needed by the interquery learning framework. Lastly, the proposed inter-query learning framework has been further extended by the incorporation of a new semantic structure. The semantic structure is used to connect the previous queries both visually and semantically. This structure provides the system with the ability to retrieve images that are semantically similar and yet visually different. To do this, an active learning strategy has been incorporated for exploring the structure. Experiments have again shown that the proposed new framework has outperformed the previous framework.

Un des premiers modèles d'indexation de documents qui utilise des termes comme descripteurs était une structure de treillis, cela une vingtaine d'années avant l'arrivée de l'analyse formelle de concepts (FCA pour "Formal Concept Analysis"), qui s'affirme maintenant comme un formalisme théorique important et solide pour l'analyse de données et la découverte de connaissances. Actuellement, la communauté en recherche d'information (RI) s'intéresse particulièrement à des techniques avancées pour la recherche des documents qui relèvent des probabilités et des statistiques. En parallèle, l'intérêt de la communauté FCA au développement de techniques qui font avancer l'état de l'art en RI tout en offrant des fonctionnalités sémantiques lui est toujours bien vivant. Dans cette thèse, nous présentons un ensemble de contributions sur ce que nous avons appelé les systèmes FCA de recherche d'information ("FCA-based IR systems''). Nous avons divisé nos contributions en deux parties, à savoir l'extraction et l'indexation. Pour la récupération, nous proposons une nouvelle technique qui exploite les relations sémantiques entre les descripteurs dans un corpus de documents. Pour l'indexation, nous proposons un nouveau modèle qui permet de mettre en oeuvre un modèle vectoriel d'indexation des documents s'appuyant sur un treillis de concepts (ou treillis de Galois). En outre, nous proposons un modèle perfectionné pour l'indexation hétérogène dans lequel nous combinons le modèle vectoriel et le modèle de recherche booléen. Finalement, nous présentons une technique de fouille de données inspiré de l'indexation des documents, à savoir un modèle d'énumération exhaustive des biclusters en utilisant la FCA. Le biclustering est une nouvelle technique d'analyse de données dans laquelle les objets sont liés via la similitude dans certains attributs de l'espace de description, et non pas par tous les attributs comme dans le "clustering'' standard. En traduisant ce problème en termes d'analyse formelle de concepts, nous pouvons exploiter l'algorithmique associée à la FCA pour développer une technique d'extraction de biclusters de valeurs similaires. Nous montrons le très bon comportement de notre technique, qui fonctionne mieux que les techniques actuelles de biclustering avec énumération exhaustive / One of the first models ever to be considered as an index for documents using terms as descriptors, was a lattice structure, a couple of decades before the arrival of Formal Concept Analysis (FCA) as a solid theory for data mining and knowledge discovery.While the Information Retrieval (IR) community has shifted to more advanced techniques for document retrieval, like probabilistic and statistic paradigms, the interest of the FCA community on developing techniques that would improve the state-of-the-art in IR while providing relevance feedback and semantic based features, never decayed. In this thesis we present a set of contributions on what we call FCA-based IR systems. We have divided our contributions in two sets, namely retrieval and indexing. For retrieval, we propose a novel technique that exploits semantic relations among descriptors in a document corpus and a new concept lattice navigation strategy (called cousin concepts), enabling us to support classification-based reasoning to provide better results compared with state-of-the-art retrieval techniques. The basic notion in our strategy is supporting query modification using "term replacements'' using the lattice structure and semantic similarity. For indexing, we propose a new model that allows supporting the vector space model of retrieval using concept lattices. One of the main limitations of current FCA-based IR systems is related to the binary nature of the input data required for FCA to generate a concept lattice. We propose the use of pattern structures, an extension of FCA to deal with complex object descriptions, in order to support more advanced retrieval paradigms like the vector space model. In addition, we propose an advanced model for heterogeneous indexing through which we can combine the vector space model and the Boolean retrieval model. The main advantage of this approach is the ability of supporting indexing of convex regions in an arbitrary vectorial space built from a document collection. Finally, we move forward to a mining model associated with document indexing, namely exhaustive bicluster enumeration using FCA. Biclustering is an emerging data analysis technique in which objects are related by similarity under certain attributes of the description space, instead of the whole description space like in standard clustering. By translating this problem to the framework of FCA, we are able to exploit the robust machinery associated with the computation of concept lattices to provide an algorithm for mining biclusters based on similar values. We show how our technique performs better than current exhaustive enumeration biclustering techniques.

An image retrieval system needs to be able to communicate with people using a common language, if it is to serve its user's information need. I propose techniques for interactive image search with the help of visual attributes, which are high-level semantic visual properties of objects (like "shiny" or "natural"), and are understandable by both people and machines. My thesis explores attributes as a novel form of user input for search. I show how to use attributes to provide relevance feedback for image search; how to optimally choose what to seek feedback on; how to ensure that the attribute models learned by a system align with the user's perception of these attributes; how to automatically discover the shades of meaning that users employ when applying an attribute term; and how attributes can help learn object category models. I use attributes to provide a channel on which the user of an image retrieval system can communicate her information need precisely and with as little effort as possible. One-shot retrieval is generally insufficient, so interactive retrieval systems seek feedback from the user on the currently retrieved results, and adapt their relevance ranking function accordingly. In traditional interactive search, users mark some images as "relevant" and others as "irrelevant", but this form of feedback is limited. I propose a novel mode of feedback where a user directly describes how high-level properties of retrieved images should be adjusted in order to more closely match her envisioned target images, using relative attribute feedback statements. For example, when conducting a query on a shopping website, the user might state: "I want shoes like these, but more formal." I demonstrate that relative attribute feedback is more powerful than traditional binary feedback. The images believed to be most relevant need not be most informative for reducing the system's uncertainty, so it might be beneficial to seek feedback on something other than the top-ranked images. I propose to guide the user through a coarse-to-fine search using a relative attribute image representation. At each iteration of feedback, the user provides a visual comparison between the attribute in her envisioned target and a "pivot" exemplar, where a pivot separates all database images into two balanced sets. The system actively determines along which of multiple such attributes the user's comparison should next be requested, based on the expected information gain that would result. The proposed attribute search trees allow us to limit the scan for candidate images on which to seek feedback to just one image per attribute, so it is efficient both for the system and the user. No matter what potentially powerful form of feedback the system offers the user, search efficiency will suffer if there is noise on the communication channel between the user and the system. Therefore, I also study ways to capture the user's true perception of the attribute vocabulary used in the search. In existing work, the underlying assumption is that an image has a single "true" label for each attribute that objective viewers could agree upon. However, multiple objective viewers frequently have slightly different internal models of a visual property. I pose user-specific attribute learning as an adaptation problem in which the system leverages any commonalities in perception to learn a generic prediction function. Then, it uses a small number of user-labeled examples to adapt that model into a user-specific prediction function. To further lighten the labeling load, I introduce two ways to extrapolate beyond the labels explicitly provided by a given user. While users differ in how they use the attribute vocabulary, there exist some commonalities and groupings of users around their attribute interpretations. Automatically discovering and exploiting these groupings can help the system learn more robust personalized models. I propose an approach to discover the latent factors behind how users label images with the presence or absence of a given attribute, from a sparse label matrix. I then show how to cluster users in this latent space to expose the underlying "shades of meaning" of the attribute, and subsequently learn personalized models for these user groups. Discovering the shades of meaning also serves to disambiguate attribute terms and expand a core attribute vocabulary with finer-grained attributes. Finally, I show how attributes can help learn object categories faster. I develop an active learning framework where the computer vision learning system actively solicits annotations from a pool of both object category labels and the objects' shared attributes, depending on which will most reduce total uncertainty for multi-class object predictions in the joint object-attribute model. Knowledge of an attribute's presence in an image can immediately influence many object models, since attributes are by definition shared across subsets of the object categories. The resulting object category models can be used when the user initiates a search via keywords such as "Show me images of cats" and then (optionally) refines that search with the attribute-based interactions I propose. My thesis exploits properties of visual attributes that allow search to be both effective and efficient, in terms of both user time and computation time. Further, I show how the search experience for each individual user can be improved, by modeling how she uses attributes to communicate with the retrieval system. I focus on the modes in which an image retrieval system communicates with its users by integrating the computer vision perspective and the information retrieval perspective to image search, so the techniques I propose are a promising step in closing the semantic gap. / text

A huge effort has been made in the development of image classification systemswith the objective of creating high-quality thematic maps and to establishprecise inventories about land cover use. The peculiarities of Remote SensingImages (RSIs) combined with the traditional image classification challengesmake RSI classification a hard task. Many of the problems are related to therepresentation scale of the data, and to both the size and therepresentativeness of used training set.In this work, we addressed four research issues in order to develop effectivesolutions for interactive classification of remote sensing images.The first research issue concerns the fact that image descriptorsproposed in the literature achieve good results in various applications, butmany of them have never been used in remote sensing classification tasks.We have tested twelve descriptors that encodespectral/color properties and seven texture descriptors. We have also proposeda methodology based on the K-Nearest Neighbor (KNN) classifier for evaluationof descriptors in classification context. Experiments demonstrate that JointAuto-Correlogram (JAC), Color Bitmap, Invariant Steerable Pyramid Decomposition(SID), and Quantized Compound Change Histogram (QCCH) yield the best results incoffee and pasture recognition tasks.The second research issue refers to the problem of selecting the scaleof segmentation for object-based remote sensing classification. Recentlyproposed methods exploit features extracted from segmented objects to improvehigh-resolution image classification. However, the definition of the scale ofsegmentation is a challenging task. We have proposedtwo multiscale classification approaches based on boosting of weak classifiers.The first approach, Multiscale Classifier (MSC), builds a strongclassifier that combines features extracted from multiple scales ofsegmentation. The other, Hierarchical Multiscale Classifier (HMSC), exploits thehierarchical topology of segmented regions to improve training efficiencywithout accuracy loss when compared to the MSC. Experiments show that it isbetter to use multiple scales than use only one segmentation scale result. Wehave also analyzed and discussed about the correlation among the useddescriptors and the scales of segmentation.The third research issue concerns the selection of training examples and therefinement of classification results through multiscale segmentation. We have proposed an approach forinteractive multiscale classification of remote sensing images.It is an active learning strategy that allows the classification resultrefinement by the user along iterations. Experimentalresults show that the combination of scales produces better results thanisolated scales in a relevance feedback process. Furthermore, the interactivemethod achieves good results with few user interactions. The proposed methodneeds only a small portion of the training set to build classifiers that are asstrong as the ones generated by a supervised method that uses the whole availabletraining set.The fourth research issue refers to the problem of extracting features of ahierarchy of regions for multiscale classification. We have proposed a strategythat exploits the existing relationships among regions in a hierarchy. Thisapproach, called BoW-Propagation, exploits the bag-of-visual-word model topropagate features along multiple scales. We also extend this idea topropagate histogram-based global descriptors, the H-Propagation method. The proposedmethods speed up the feature extraction process and yield good results when compared with globallow-level extraction approaches.

L'objectif de cette thèse est de développer des solutions efficaces pour laclassification interactive des images de télédétection. Cet objectif a étéréalisé en répondant à quatre questions de recherche.La première question porte sur le fait que les descripteursd'images proposées dans la littérature obtiennent de bons résultats dansdiverses applications, mais beaucoup d'entre eux n'ont jamais été utilisés pour la classification des images de télédétection. Nous avons testé douzedescripteurs qui codent les propriétés spectrales et la couleur, ainsi que septdescripteurs de texture. Nous avons également proposé une méthodologie baséesur le classificateur KNN (K plus proches voisins) pour l'évaluation desdescripteurs dans le contexte de la classification. Les descripteurs Joint Auto-Correlogram (JAC),Color Bitmap, Invariant Steerable Pyramid Decomposition (SID) etQuantized Compound Change Histogram (QCCH), ont obtenu les meilleursrésultats dans les expériences de reconnaissance des plantations de café et depâturages.La deuxième question se rapporte au choix del'échelle de segmentation pour la classification d'images baséesur objets.Certaines méthodes récemment proposées exploitent des caractéristiques extraitesdes objets segmentés pour améliorer classification des images hauterésolution. Toutefois, le choix d'une bonne échelle de segmentation est unetâche difficile.Ainsi, nous avons proposé deux approches pour la classification multi-échelles fondées sur le les principes du Boosting, qui permet de combiner desclassifieurs faibles pour former un classifieur fort.La première approche, Multiscale Classifier (MSC), construit unclassifieur fort qui combine des caractéristiques extraites de plusieurséchelles de segmentation. L'autre, Hierarchical Multiscale Classifier(HMSC), exploite la topologie hiérarchique de régions segmentées afind'améliorer l'efficacité des classifications sans perte de précision parrapport au MSC. Les expériences montrent qu'il est préférable d'utiliser des plusieurs échelles plutôt qu'une seul échelle de segmentation. Nous avons également analysé et discuté la corrélation entre lesdescripteurs et des échelles de segmentation.La troisième question concerne la sélection des exemplesd'apprentissage et l'amélioration des résultats de classification basés sur lasegmentation multiéchelle. Nous avons proposé une approche pour laclassification interactive multi-échelles des images de télédétection. Ils'agit d'une stratégie d'apprentissage actif qui permet le raffinement desrésultats de classification par l'utilisateur. Les résultats des expériencesmontrent que la combinaison des échelles produit de meilleurs résultats que leschaque échelle isolément dans un processus de retour de pertinence. Par ailleurs,la méthode interactive permet d'obtenir de bons résultats avec peud'interactions de l'utilisateur. Il n'a besoin que d'une faible partie del'ensemble d'apprentissage pour construire des classificateurs qui sont aussiforts que ceux générés par une méthode supervisée qui utilise l'ensembled'apprentissage complet.La quatrième question se réfère au problème de l'extraction descaractéristiques d'un hiérarchie des régions pour la classificationmulti-échelles. Nous avons proposé une stratégie qui exploite les relationsexistantes entre les régions dans une hiérarchie. Cette approche, appelée BoW-Propagation, exploite le modèle de bag-of-visual-word pour propagerles caractéristiques entre les échelles de la hiérarchie. Nous avons égalementétendu cette idée pour propager des descripteurs globaux basés sur leshistogrammes, l'approche H-Propagation. Ces approches accélèrent leprocessus d'extraction et donnent de bons résultats par rapport à l'extractionde descripteurs globaux. / A huge effort has been made in the development of image classification systemswith the objective of creating high-quality thematic maps and to establishprecise inventories about land cover use. The peculiarities of Remote SensingImages (RSIs) combined with the traditional image classification challengesmake RSI classification a hard task. Many of the problems are related to therepresentation scale of the data, and to both the size and therepresentativeness of used training set.In this work, we addressed four research issues in order to develop effectivesolutions for interactive classification of remote sensing images.The first research issue concerns the fact that image descriptorsproposed in the literature achieve good results in various applications, butmany of them have never been used in remote sensing classification tasks.We have tested twelve descriptors that encodespectral/color properties and seven texture descriptors. We have also proposeda methodology based on the K-Nearest Neighbor (KNN) classifier for evaluationof descriptors in classification context. Experiments demonstrate that JointAuto-Correlogram (JAC), Color Bitmap, Invariant Steerable Pyramid Decomposition(SID), and Quantized Compound Change Histogram (QCCH) yield the best results incoffee and pasture recognition tasks.The second research issue refers to the problem of selecting the scaleof segmentation for object-based remote sensing classification. Recentlyproposed methods exploit features extracted from segmented objects to improvehigh-resolution image classification. However, the definition of the scale ofsegmentation is a challenging task. We have proposedtwo multiscale classification approaches based on boosting of weak classifiers.The first approach, Multiscale Classifier (MSC), builds a strongclassifier that combines features extracted from multiple scales ofsegmentation. The other, Hierarchical Multiscale Classifier (HMSC), exploits thehierarchical topology of segmented regions to improve training efficiencywithout accuracy loss when compared to the MSC. Experiments show that it isbetter to use multiple scales than use only one segmentation scale result. Wehave also analyzed and discussed about the correlation among the useddescriptors and the scales of segmentation.The third research issue concerns the selection of training examples and therefinement of classification results through multiscale segmentation. We have proposed an approach forinteractive multiscale classification of remote sensing images.It is an active learning strategy that allows the classification resultrefinement by the user along iterations. Experimentalresults show that the combination of scales produces better results thanisolated scales in a relevance feedback process. Furthermore, the interactivemethod achieves good results with few user interactions. The proposed methodneeds only a small portion of the training set to build classifiers that are asstrong as the ones generated by a supervised method that uses the whole availabletraining set.The fourth research issue refers to the problem of extracting features of ahierarchy of regions for multiscale classification. We have proposed a strategythat exploits the existing relationships among regions in a hierarchy. Thisapproach, called BoW-Propagation, exploits the bag-of-visual-word model topropagate features along multiple scales. We also extend this idea topropagate histogram-based global descriptors, the H-Propagation method. The proposedmethods speed up the feature extraction process and yield good results when compared with globallow-level extraction approaches.

A mineração de grandes coleções de textos, imagens e outros tipos de documentos tem se mostrado uma forma efetiva para exploração e interação com grandes quantidades de informações disponíveis, principalmente na World Wide Web. Neste contexto, diversos trabalhos têm tratado de mineração tanto de coleções estáticas quanto de coleções dinâmicas de objetos. Adicionalmente, técnicas de visualização têm sido propostas para auxiliar o processo de entendimento e de exploração dessas coleções, permitindo que a interação do usuário melhore o processo de mineração (user in the loop). No caso específico de dados dinâmicos, foi desenvolvido por Roberto Pinho e colegas uma técnica incremental (IncBoard) com o objetivo de visualizar coleções dinâmicas de elementos. Tal técnica posiciona os elementos em um grid bidimensional baseado na similaridade de conteúdo entre os elementos. Procura-se manter elementos similares próximos no grid. A técnica foi avaliada em um processo que simulava a chegada de novos dados, apresentando iterativamente novos elementos a serem posicionados no mapa corrente. Observa-se, entretanto, que um aspecto importante de tal ferramenta seria a possibilidade de novos elementos - a serem exibidos no mapa, mantendo coerência com o mapa corrente - serem selecionados a partir do interesse demonstrado pelo usuário. Realimentação de relevância tem se mostrado muito efetiva na melhoria da acurácia do processo de recuperação. Entretanto, um problema ainda em aberto é como utilizar técnicas de realimentação de relevância em conjunto com exploração visual no processo de recuperação de informação. Neste trabalho, é investigado o desenvolvimento de técnicas de exploração visual utilizando realimentação de relevância para sistemas de recuperação de informação de domínio específico. O Amuzi, um sistema de busca de músicas, foi desenvolvido como uma prova de conceito para a abordagem investigada. Dados coletados da utilização do Amuzi, por usuários, sugerem que a combinação de tais técnicas oferece vantagens, quando utilizadas em determinados domínios. Nesta dissertação, a recuperação de informação com realimentação de relevância apoiada em visualização, bem como o sistema Amuzi são descritos. Também são analisados os registros de utilização dos usuários / The mining of large text collections, images and other types of digital objects has shown to be a very effective way to explore and interact with big data, specially on the World Wide Web. On that subject, many researchers have been done on data mining of static and dynamic collections. Moreover, data visualization techniques have been proposed to aid on the understanding and exploration of such data collections, also allowing users to interact with data, user in the loop. On the speciific subject of dynamic data, Roberto Pinho and colleagues have developed an incremental technique, called Inc-Board, which aims to visualize dynamic data collections. IncBoard displays the documents on a two dimensional grid in a way that similar elements tends to be close to each other. This technique was evaluated in a process that simulated the arrival of new data elements, iteratively inserting new elements on the grid. Nonetheless, it would be useful if the user could interact with such documents to point out which are relevant and which are not relevant to his/her search. Relevance Feedback has also shown to be effective on improving the accuracy of Information Retrieval techniques. An issue that still open is how to combine data visualization and Relevance Feedback to improve Information Retrieval. On this dissertation, the development of techniques with data visualization and Relevance Feedback are investigated to aid on the Information Retrieval task, for specific domains. Amuzi is an Information Retrieval system, built to be a proof of concept for the investigated approach. Data collected from the usage of the system suggests that combining such techniques may outperform traditional Information Retrieval systems when applied for specifc domains. This dissertation has the description the information retrieval process with feedback relevance supported by visualization and the Amuzi system. Usage log are processed and analyzed to evaluate the investigated approach