The National Library of Medicine (NLM), an institute in the National Institutes of Health (NIH), maintains a collection of 17,000 digitized spine X-ray images obtained from the second National Health and Nutrition Examination Survey (NHANES II). Research effort has been devoted to develop a web-accessible retrieval system that allows retrieval of images from the NHANES II database on relevant and frequently found pathologies. A comprehensive and successful image retrieval system requires effective image representation and matching methods, relevance feedback algorithms to incorporate user opinions, and efficient indexing schemes for fast access to image databases. This dissertation studies and develops approaches for all of the above areas within the context of content-Based Image Retrieval (CBIR) of spine X-ray images from the NHANES II collection. Shape is an important characteristic for describing pertinent pathologies in various types of medical images, including spine X-ray images. Retrieving images with shapes similar to a specific user query can be useful for finding pathologies exhibited in images in large survey collections. In this work, vertebral outlines are extracted for image retrieval using shape matching methods to detect the presence of anterior osteophytes. The Multiple Open Triangle (MOT) shape representation method is proposed for partial shape matching (PSM), and a Corner-Guided Dynamic Programming (DP) strategy is developed to search partial intervals for matching comparison based on a 9-point model marked by a board-certified radiologist. The MOT method demonstrates higher retrieval accuracy compared to other approaches and the retrieval speed is improved significantly through the use of Corner-Guided DP. Computer-calculated low-level image features fall short when imitating high-level human visual perception. Relevance Feedback (RF) attempts to bridge the gap between the two by analyzing and employing user feedback. The need for overcoming this gap is more evident in medical image retrieval. Existing RF approaches are analyzed and a weight-updating formula for RF is developed. A hybrid retrieval approach is proposed that utilizes both CBIR with RF and RF history. This hybrid approach uses short-term memory to store the feedback history, which contributes to the retrieval results and helps select images for user feedback. An approximate 20% average increase in retrieval recall percentage is achieved within two RF iterations. Efficient indexing methods are desired for fast database access. An agglomerative clustering algorithm is adopted to pre-index the database based on pre-calculated pair-wise distances between indexed parts. Retrieval with this pre-indexing procedure is shown to offer faster retrieval and maintain a comparable recall percentage.

An image mosaic is an artistic work that uses a number of smaller images creatively combined together to form another larger image. Each building block image, or tessera, has its own distinctive and meaningful content, but when viewed from a distance the tesserae come together to form an aesthetically pleasing montage. This work presents the design and implementation of MosaiX, a computer software system that generates these image mosaics automatically. To control the image mosaic creation process, several parameters are used within the system. Each parameter affects the overall mosaic quality, as well as required processing time, in its own unique way. A detailed analysis is performed to evaluate each parameter individually. Additionally, this work proposes two novel ways by which to evaluate the quality of an image mosaic in a quantitative way. One method focuses on the perceptual color accuracy of the mosaic reproduction, while the other concentrates on edge replication. Both measures include preprocessing to take into account the unique visual features present in an image mosaic. Doing so minimizes quality penalization due the inherent properties of an image mosaic that make them visually appealing.

Yes / The exponential growth in the volume of digital image databases is making it increasingly difficult to retrieve relevant information from them. Efficient retrieval systems require distinctive features extracted from visually rich contents, represented semantically in a human perception-oriented manner. This paper presents an efficient framework to model image contents as an undirected attributed relational graph, exploiting color, texture, layout, and saliency information. The proposed method encodes salient features into this rich representative model without requiring any segmentation or clustering procedures, reducing the computational complexity. In addition, an efficient graph-matching procedure implemented on specialized hardware makes it more suitable for real-time retrieval applications. The proposed framework has been tested on three publicly available datasets, and the results prove its superiority in terms of both effectiveness and efficiency in comparison with other state-of-the-art schemes. / Supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (2013R1A1A2012904).

In historical photo research, the presence of painted backdrops have the potential to help identify subjects, photographers, locations, and jl{events surrounding} certain photographs. Yet, research processes around these backdrops are poorly documented, with no known tools to aid in the task. We propose a four-step human-AI collaboration workflow to support the jl{discovery} and clustering of these backdrops. Focusing on the painted backdrops of the American Civil War (1861 -- 1865), we present Backdrop Explorer, a content-based image retrieval (CBIR) system incorporating computer vision and novel user interactions. We evaluated Backdrop Explorer on nine users of diverse experience levels and found that all were able to effectively utilize Backdrop Explorer to find photos with similar backdrops. We also document current practices and pain points in Civil War backdrop research through user interviews. Finally, we discuss how our findings and workflow can be applied to other topics and domains. / Master of Science / In historical photo research, the presence of painted backdrops have the potential to help identify subjects, photographers, locations, and events surrounding certain photographs. Yet, research processes around these backdrops are poorly documented, with no known tools to aid in the largely manual task. We present Backdrop Explorer, a reverse image search system that helps users discover and subsequently group photos with similar backdrops. We evaluated the system and found that it effectively supported the tasks. We also document current practices and pain points in Civil War backdrop research. Finally, we discuss how our findings and system can be applied to other domains.

Many applications lead to signals with nonnegative function values. Understanding the structure of the spaces of nonnegative signals is therefore of interest in many different areas. Hence, constructing effective representation spaces with suitable metrics and natural transformations is an important research topic. In this thesis, we present our investigations of the structure of spaces of nonnegative signals and illustrate the results with applications in the fields of multispectral color science and content-based image retrieval. The infinite-dimensional Hilbert space of nonnegative signals is conical and convex. These two properties are preserved under linear projections onto lower dimensional spaces. The conical nature of these coordinate vector spaces suggests the use of hyperbolic geometry. The special case of three-dimensional hyperbolic geometry leads to the application of the SU(1,1) or SO 2,1) groups. We introduce a new framework to investigate nonnegative signals. We use PCA-based coordinates and apply group theoretical tools to investigate sequences of signal coordinate vectors. We describe these sequences with oneparameter subgroups of SU(1,1) and show how to compute the one-parameter subgroup of SU(1,1) from a given set of nonnegative signals. In our experiments we investigate the following signal sequences: (i) blackbody radiation spectra; (ii) sequences of daylight/twilight spectra measured in Norrk¨oping, Sweden and in Granada, Spain; (iii) spectra generated by the SMARTS2 simulation program; and (iv) sequences of image histograms. The results show that important properties of these sequences can be modeled in this framework. We illustrate the usefulness with examples where we derive illumination invariants and introduce an efficient visualization implementation. Content-Based Image Retrieval (CBIR) is another topic of the thesis. In such retrieval systems, images are first characterized by descriptor vectors. Retrieval is then based on these content-based descriptors. Selection of contentbased descriptors and defining suitable metrics are the core of any CBIR system. We introduce new descriptors derived by using group theoretical tools. We exploit the symmetry structure of the space of image patches and use the group theoretical methods to derive low-level image filters in a very general framework. The derived filters are simple and can be used for multispectral images and images defined on different sampling grids. These group theoretical filters are then used to derive content-based descriptors, which will be used in a real implementation of a CBIR.

Multimedia documents are composed of multiple file format combinations, such as image and text, image and sound, or image, text and sound. The type of multimedia document determines the form of analysis for knowledge architecture design and retrieval methods. Over the last few decades, theories of text analysis have been proposed and applied effectively. In recent years, theories of image and sound analysis have been proposed to work with text retrieval systems and progressed quickly due in part to rapid progress in computer processing speed. Retrieval of multimedia documents formerly was divided into the categories of image and text, and image and sound. While standard retrieval process begins from text only, methods are developing that allow the retrieval process to be accomplished simultaneously using text and image. Although image processing for feature extraction and text processing for term extractions are well understood, there are no prior methods that can combine these two features into a single data structure. This dissertation will introduce a common representation format for multimedia documents (CRFMD) composed of both images and text. For image and text analysis, two techniques are used: the Lorenz Information Measurement and the Word Code. A new process named Jeong's Transform is demonstrated for extraction of text and image features, combining the two previous measurements to form a single data structure. Finally, this single data measurements to form a single data structure. Finally, this single data structure is analyzed by using multi-dimensional scaling. This allows multimedia objects to be represented on a two-dimensional graph as vectors. The distance between vectors represents the magnitude of the difference between multimedia documents. This study shows that image classification on a given test set is dramatically improved when text features are encoded together with image features. This effect appears to hold true even when the available text is diffused and is not uniform with the image features. This retrieval system works by representing a multimedia document as a single data structure. CRFMD is applicable to other areas of multimedia document retrieval and processing, such as medical image retrieval, World Wide Web searching, and museum collection retrieval.

The research investigated whether a Latent Semantic Analysis (LSA)-based approach to image retrieval can map pixel intensity into a smaller concept space with good accuracy and reasonable computational cost. From a large set of computed tomography (CT) images, a retrieval query found all images for a particular patient based on semantic similarity. The effectiveness of the LSA retrieval was evaluated based on precision, recall, and F-score. This work extended the application of LSA to high-resolution CT radiology images. The images were chosen for their unique characteristics and their importance in medicine. Because CT images are intensity-only, they carry less information than color images. They typically have greater noise, higher intensity, greater contrast, and fewer colors than a raw RGB image. The study targeted level of intensity for image features extraction. The focus of this work was a formal evaluation of the LSA method in the context of large number of high-resolution radiology images. The study reported on preprocessing and retrieval time and discussed how reduction of the feature set size affected the results. LSA is an information retrieval technique that is based on the vector-space model. It works by reducing the dimensionality of the vector space, bringing similar terms and documents closer together. Matlab software was used to report on retrieval and preprocessing time. In determining the minimum size of concept space, it was found that the best combination of precision, recall, and F-score was achieved with 250 concepts (k = 250). This research reported precision of 100% on 100% of the queries and recall close to 90% on 100% of the queries with k=250. Selecting a higher number of concepts did not improve recall and resulted in significantly increased computational cost.

O aumento da geração e do intercâmbio de imagens médicas digitais tem incentivado profissionais da computação a criarem ferramentas para manipulação, armazenamento e busca por similaridade dessas imagens. As ferramentas de recuperação de imagens por conteúdo, foco desse trabalho, têm a função de auxiliar na tomada de decisão e na prática da medicina baseada em estudo de casos semelhantes. Porém, seus principais obstáculos são conseguir uma rápida recuperação de imagens armazenadas em grandes bases e reduzir o gap semântico, caracterizado pela divergência entre o resultado obtido pelo computador e aquele esperado pelo médico. No presente trabalho, uma análise das funções de distância e dos descritores computacionais de características está sendo realizada com o objetivo de encontrar uma aproximação eficiente entre os métodos de extração de características de baixo nível e os parâmetros de percepção do médico (de alto nível) envolvidos na análise de imagens. O trabalho de integração desses três elementos (Extratores de Características, Função de Distância e Parâmetro Perceptual) resultou na criação de operadores de similaridade, que podem ser utilizados para aproximar o sistema computacional ao usuário final, visto que serão recuperadas imagens de acordo com a percepção de similaridade do médico, usuário final do sistema / The continuous growth of the medical images generation and their use in the day-to-day procedures in hospitals and medical centers has motivated the computer science researchers to develop algorithms, methods and tools to store, search and retrieve images by their content. Therefore, the content-based image retrieval (CBIR) field is also growing at a very fast pace. Algorithms and tools for CBIR, which are at the core of this work, can help on the decision making process when the specialist is composing the images analysis. This is based on the fact that the specialist can retrieve similar cases to the one under evaluation. However, the main reservation about the use of CBIR is to achieve a fast and effective retrieval, in the sense that the specialist gets what is expected for. That is, the problem is to bridge the semantic gap given by the divergence among the result automatically delivered by the system and what the user is expecting. In this work it is proposed the perceptual parameter, which adds to the relationship between the feature extraction algorithms and distance functions aimed at finding the best combination to deliver to the user what he/she expected from the query. Therefore, this research integrated the three main elements of similarity queries: the image features, the distance function and the perceptual parameter, what resulted in searching operators. The experiments performed show that these operators can narrow the distance between the system and the specialist, contributing to bridge the semantic gap

Este trabalho apresenta o estudo e implementação de um banco de conhecimentos para auxiliar o diagnóstico de lesões da mama por inspeção visual, permitindo ao médico consultas através de características pictóricas da imagem e a comparação visual entre imagem investigada e imagens previamente classificadas e suas informações clínicas. As imagens encontram-se classificadas no banco de conhecimentos segundo o padrão \"Breast imaging reporting and data systems\" (BI-RADS) do Colégio Americano de Radiologia. A seleção das imagens, informações clínicas representativas, bem como sua classificação foram realizada em conjunto com médicos radiologistas do Centro de Ciências das Imagens e Física Médica (CCIFM) da Faculdade de Medicina de Ribeirão Preto (FMRP) da Universidade de São Paulo (USP). O processo de indexação e recuperação das imagens é baseado em atributos de textura extraídos de \"Regions of interest\" (ROIs) previamente estabelecidas em mamogramas digitalizados. Para simplificar este processo, foi utilizado a Análise de Componentes Principais (PCA), que visa a redução do número de atributos de textura e as informações redundantes existentes. Os melhores resultados obtidos foram para as ROIs 139 (Precisão = 0.80), 59 (Precisão = 0.86) e um valor de 100% de acerto para a ROI 40. / This work presents the survey and implementation of a database of knowledge to aid the diagnostic of breast lesions through visual inspection, allowing the physician a seach through the characteristics of the contents of the image and the visual comparison between the analysed image and the previously classified images and its clinical information. The images are classified into the database of knowledge according to the pattern Breast Imaging Reporting and Data Systems (BI-RADS) of the American College of Radiology. The selection of the images, the representative clinical information, as well as its classification have been performed in conjunction with practictioners radiologists of the Centro de Ciências das Imagens e Física Médica (CCIFM) from Faculdade de Medicina de Ribeirão Preto (FMRP) from Universidade de São Paulo (USP). The process of indexing and retrieving the images is based on characteristic of the texture extracted from the regions of interest (ROIs) previously established through scanned mammograms. To simplify this path, the Principal Components Analysis (PCA) was used it aims the reduction of the number of features of texture and the existing redundant information. The best results obtained were to the ROIs 139 (precision = 0.80), 59 (precision = 0.86) and a value of 100% of precision for ROI 40.

Over the years, exemplar-based methods have yielded significant improvements over their model-based counterparts in image synthesis applications. Notably, texture synthesis algorithms using an exemplar-based approach have shown success where traditional stochastic methods failed. As an illustrative example, I will present an exemplar-based approach that yields substantial benefits for user-guided terrain synthesis using Digital Elevation Models (DEMs). This success is realized through exploitation of structural properties of natural terrain. In addition to their proliferation in the image synthesis domain, as annotated image datasets become increasingly available, exemplar-based methods are also gaining in popularity for image analysis applications. This thesis addresses the intersection between exemplar-based analysis and the problem of content-based image retrieval (CBIR). A basic problem in CBIR is the process by which the search criteria are refined by the user through the manipulation of returned exemplars. Exemplar-based analysis is particularly well-suited to query refinement due to its interpretability and the ease with which it can be incorporated into an interactive system. I investigate this connection in the domain of Computer-Assisted Diagnosis (CAD) of dermatological images. I demonstrate that exemplar-based approaches in CBIR can be effective for diagnosing pigmented skin lesions (PSLs). I will present an exemplar-based algorithm for segmenting PSLs in dermatoscopic images. In addition, I will present a generalized representation of dermoscopic features for detection and matching. This representation not only leads to an exemplar-based PSL diagnosis scheme, but it also enables us to realize interactive region-of-interest retrieval, which includes a relevance feedback mechanism to facilitate more flexible query-by-example analysis. Finally, I will assess the benefit of this CBIR-CAD approach through both quantitative evaluations and user studies.