Content Based Image Retrieval (CBIR) is an important research area in the field of multimedia information retrieval. The application of CBIR in the medical domain has been attempted before, however the use of CBIR in medical diagnostics is a daunting task. The goal of diagnostic medical image retrieval is to provide diagnostic support by displaying relevant past cases, along with proven pathologies as ground truths. Moreover, medical image retrieval can be extremely useful as a training tool for medical students and residents, follow-up studies, and for research purposes. Despite the presence of an impressive amount of research in the area of CBIR, its acceptance for mainstream and practical applications is quite limited. The research in CBIR has mostly been conducted as an academic pursuit, rather than for providing the solution to a need. For example, many researchers proposed CBIR systems where the image database consists of images belonging to a heterogeneous mixture of man-made objects and natural scenes while ignoring the practical uses of such systems. Furthermore, the intended use of CBIR systems is important in addressing the problem of "Semantic Gap". Indeed, the requirements for the semantics in an image retrieval system for pathological applications are quite different from those intended for training and education. Moreover, many researchers have underestimated the level of accuracy required for a useful and practical image retrieval system. The human eye is extremely dexterous and efficient in visual information processing; consequently, CBIR systems should be highly precise in image retrieval so as to be useful to human users. Unsurprisingly, due to these and other reasons, most of the proposed systems have not found useful real world applications. In this dissertation, an attempt is made to address the challenging problem of developing a retrieval system for medical diagnostics applications. More specifically, a system for semantic retrieval of Magnetic Resonance (MR) images in 3D brain volumes is proposed. The proposed retrieval system has a potential to be useful for clinical experts where the human eye may fail. Previously proposed systems used imprecise segmentation and feature extraction techniques, which are not suitable for precise matching requirements of the image retrieval in this application domain. This dissertation uses multiscale representation for image retrieval, which is robust against noise and MR inhomogeneity. In order to achieve a higher degree of accuracy in the presence of misalignments, an image registration based retrieval framework is developed. Additionally, to speed-up the retrieval system, a fast discrete wavelet based feature space is proposed. Further improvement in speed is achieved by semantically classifying of the human brain into various "Semantic Regions", using an SVM based machine learning approach. A novel and fast identification system is proposed for identifying a 3D volume given a 2D image slice. To this end, we used SVM output probabilities for ranking and identification of patient volumes. The proposed retrieval systems are tested not only for noise conditions but also for healthy and abnormal cases, resulting in promising retrieval performance with respect to multi-modality, accuracy, speed and robustness. This dissertation furnishes medical practitioners with a valuable set of tools for semantic retrieval of 2D images, where the human eye may fail. Specifically, the proposed retrieval algorithms provide medical practitioners with the ability to retrieve 2D MR brain images accurately and monitor the disease progression in various lobes of the human brain, with the capability to monitor the disease progression in multiple patients simultaneously. Additionally, the proposed semantic classification scheme can be extremely useful for semantic based categorization, clustering and annotation of images in MR brain databases. This research framework may evolve in a natural progression towards developing more powerful and robust retrieval systems. It also provides a foundation to researchers in semantic based retrieval systems on how to expand existing toolsets for solving retrieval problems.

Image matching is an important and necessary process in Content-Based Image Retrieval (CBIR). We propose three new methods for image matching: the first one is based on the Local Triplet Pattern (LTP) histograms; the second one is based on the Gaussian Mixture Models (GMMs) estimated by using the Extended Mass-constraint (EMass) algorithm; and the third one is called the DCT2KL algorithm. First, the LTP histograms are proposed to capture spatial relationships between color levels of neighboring pixels. An LTP level is extracted from each 3x3 pixel block, which is a unique number describing the color level relationship between a pixel and its neighboring pixels. Second, we consider how to represent and compare multi-dimensional color features using GMMs. GMMs are alternative methods to histograms for representing data distributions. GMMs address the high-dimensional problems from which histograms usually suffer inefficiency. In order to avoid local maxima problems in most GMM estimation algorithms, we apply the deterministic annealing method to estimate GMMs. Third, motivated by image compression algorithms, the DCT2KL method addresses the high dimensional data by using the Discrete Cosine Transform (DCT) coefficients in the YCbCr color space. The DCT coefficients are restored by partially decoding JPEG images. Assume that each DCT coefficient sequence is emitted from a memoryless source, and all these sources are independent of each other. For each target image we form a hypothesis that its DCT coefficient sequences are emitted from the same sources as the corresponding sequences in the query image. Testing these hypotheses by measuring the log-likelihoods leads to a simple yet efficient scheme that ranks each target image according to the Kullback-Leibler (KL) divergence between the empirical distribution of the DCT coefficient sequences in the query image and that in the target image. Finally we present a scheme to combine different features and methods to boost the performance of image retrieval. Experimental results on different image data sets show that these three methods proposed above outperform the related works in literature, and the combination scheme further improves the retrieval performance.

A novel approach is proposed for real-time retrieval of images from a large database of overlapping images of an indoor environment. The procedure extracts visual features from images using selected computer vision techniques, and processes the extracted features to create a reduced list of features annotated with the frame numbers they appear in. This method is named landmark indexing. Unlike some state-of-the-art approaches, the proposed method does not need to consider large image adjacency graphs because the overlap of the images in the map sufficiently increases information gain, and mapping of similar features to the same landmark reduces the search space to improve search efficiency. Empirical evidence from experiments on real datasets shows high (90-100%) accuracy in image retrieval, and improvement in search time from the order of 100-200 milliseconds to the order of 10-30 milliseconds. The image retrieval technique is also demonstrated by integrating it into a 3D real-time navigation system. This system is tested in several indoor environments and all experiments show accurate localization results in large indoor areas with errors in the order of 15-20 centimeters only. / Thesis (Master, Electrical & Computer Engineering) -- Queen's University, 2014-04-24 12:44:41.429

Over the last decade, storage of non text-based data in databases has become an increasingly important trend in information management. Image in particular, has been gaining popularity as an alternative, and sometimes more viable, option for information storage. While this presents a wealth of information, it also creates a great problem in retrieving appropriate and relevant information during searching. This has resulted in an enormous growth of interest, and much active research, into the extraction of relevant information from non text-based databases. In particular,content-based image retrieval (CBIR) systems have been one of the most active areas of research. The retrieval principle of CBIR systems is based on visual features such as colour, texture, and shape or the semantic meaning of the images. To enhance the retrieval speed, most CBIR systems pre-process the images stored in the database. This is because feature extraction algorithms are often computationally expensive. If images are to be retrieved from the World-Wide-Web (WWW), the raw images have to be downloaded and processed in real time. In this case, the feature extraction speed becomes crucial. Ideally, systems should only use those feature extraction algorithms that are most suited for analysing the visual features that capture the common relationship between the images in hand. In this thesis, a statistical discriminant analysis based feature selection framework is proposed. Such a framework is able to select the most appropriate visual feature extraction algorithms by using relevance feedback only on the user labelled samples. The idea is that a smaller image sample group is used to analyse the appropriateness of each visual feature, and only the selected features will be used for image comparison and ranking. As the number of features is less, an improvement in the speed of retrieval is achieved. From experimental results, it is found that the retrieval accuracy for small sample data has also improved. Intelligent E-Business has been used as a case study in this thesis to demonstrate the potential of the framework in the application of image retrieval system. In addition, an inter-query framework has been proposed in this thesis. This framework is also based on the statistical discriminant analysis technique. A common approach in inter-query for a CBIR system is to apply the term-document approach. This is done by treating each image’s name or address as a term, and the query session as a document. However, scalability becomes an issue with this technique as the number of stored queries increases. Moreover, this approach is not appropriate for a dynamic image database environment. In this thesis, the proposed inter-query framework uses a cluster approach to capture the visual properties common to the previously stored queries. Thus, it is not necessary to “memorise” the name or address of the images. In order to manage the size of the user’s profile, the proposed framework also introduces a merging approach to combine clusters that are close-by and similar in their characteristics. Experiments have shown that the proposed framework has outperformed the short term learning approach. It also has the advantage that it eliminates the burden of the complex database maintenance strategies required in the term-document approach commonly needed by the interquery learning framework. Lastly, the proposed inter-query learning framework has been further extended by the incorporation of a new semantic structure. The semantic structure is used to connect the previous queries both visually and semantically. This structure provides the system with the ability to retrieve images that are semantically similar and yet visually different. To do this, an active learning strategy has been incorporated for exploring the structure. Experiments have again shown that the proposed new framework has outperformed the previous framework.

Thesis (Ph. D.)--University of Washington, 1999. / Vita. Includes bibliographical references (p. [95]-100).

Weighted graph matching is a good way to align a pair of shapesrepresented by a set of descriptive local features; the set ofcorrespondences produced by the minimum cost of matching features fromone shape to the features of the other often reveals how similar thetwo shapes are. However, due to the complexity of computing the exactminimum cost matching, previous algorithms could only run efficientlywhen using a limited number of features per shape, and could not scaleto perform retrievals from large databases. We present a contourmatching algorithm that quickly computes the minimum weight matchingbetween sets of descriptive local features using a recently introducedlow-distortion embedding of the Earth Mover's Distance (EMD) into anormed space. Given a novel embedded contour, the nearest neighborsin a database of embedded contours are retrieved in sublinear time viaapproximate nearest neighbors search. We demonstrate our shapematching method on databases of 10,000 images of human figures and60,000 images of handwritten digits.

Sets of local features that are invariant to common image transformations are an effective representation to use when comparing images; current methods typically judge feature sets' similarity via a voting scheme (which ignores co-occurrence statistics) or by comparing histograms over a set of prototypes (which must be found by clustering). We present a method for efficiently comparing images based on their discrete distributions (bags) of distinctive local invariant features, without clustering descriptors. Similarity between images is measured with an approximation of the Earth Mover's Distance (EMD), which quickly computes the minimal-cost correspondence between two bags of features. Each image's feature distribution is mapped into a normed space with a low-distortion embedding of EMD. Examples most similar to a novel query image are retrieved in time sublinear in the number of examples via approximate nearest neighbor search in the embedded space. We also show how the feature representation may be extended to encode the distribution of geometric constraints between the invariant features appearing in each image.We evaluate our technique with scene recognition and texture classification tasks.

This thesis documents the research which has led to the design of a number of techniques aiming to improve the performance of content-based image retrieval (CBIR) systems in wavelets domain using texture analysis. Attention was drawn on CBIR in transform domain and in particular wavelets because of the excellent characteristics for compression and texture extraction applications and the wide adoption in both the research community and the industry. The issue of performance is addressed in terms of accuracy and speed. The rationale for this research builds upon the conclusion that CBIR has not yet reached a good performance balance of accuracy, efficiency and speed for wide adoption in practical applications. The issue of bridging the sensory gap, which is defined as "[the difference] between the object in the real world and the information in a (computational) description derived from a recording of that scene." has yet to be resolved. Furthermore, speed improvement remains an uncharted territory as is feature extraction directly from the bitstream of compressed images. To address the above requirements the first part of this work introduces three techniques designed to jointly address the issue of accuracy and processing cost of texture characterization in wavelets domain. The second part introduces a new model for mapping the wavelet coefficients of an orthogonal wavelet transformation to a circular locus. The model is applied in order to design a novel rotation-invariant texture descriptor. All of the aforementioned techniques are also designed to bridge the gap between texture-based image retrieval and image compression by using appropriate compatible design parameters. The final part introduces three techniques for improving the speed of a CBIR query through more efficient calculation of the Li-distance, when it is used as an image similarity metric. The contributions conclude with a novel technique which, in conjunction with a widely adopted wavelet-based compression algorithm, extracts texture information directly from the compressed bit-stream for speed and storage requirements savings. The experimental findings indicate that the proposed techniques form a solid groundwork which can be extended to practical applications.

This dissertation proposes a comprehensive working flow for image retrieval. It contains four components: denoising, restoration, color features extraction, and depth feature extraction. We propose a visual perceptual descriptor (VPD) to extract color features from an image. Gradient direction is calculated at each pixel, and the VPD is moved over the entire image to locate regions with similar gradient direction. Color features are extracted only at these pixels. Experiments demonstrate that VPD is an effective and reliable descriptor in image retrieval. We propose a novel depth feature for image retrieval. Regarding any 2D image as the convolution of a corresponding sharp image and a Gaussian kernel with unknown blur amount. Sparse depth map is computed as the absolute difference of the original image and its sharp version. Depth feature is extracted as the nuclear norm of the sparse depth map. Experiments validate the effectiveness of this approach on depth recovery and image retrieval. We present a model for image denoising. A gradient item is incorporated, and can be merged into the original model based on geometric measure theory. Experiments illustrate this model is effective for image denoising, and it can improve the retrieval performance by denoising a query image. A model is proposed for image restoration. It is an extension of the traditional singular value thresholding (SVT) algorithm, addressing the issue that SVT cannot recover a matrix with missing rows or columns. Proposed is a way to fill such rows and columns, and then apply SVT to restore the damaged image. The pre-filled entries are recomputed by averaging its neighboring pixels. Experiments demonstrate the effectiveness of this model on image restoration, and it can improve the retrieval performance by restoring a damaged query image. Finally, the capability of this working flow is tested. Experiments demonstrate its effectiveness in image retrieval.

CoordenaÃÃo de AperfeiÃoamento de Pessoal de NÃvel Superior / Sistemas de recuperaÃÃo de imagens baseada em conteÃdo (do termo em inglÃs, Content-Based Image Retrieval - CBIR) que operam em bases com grande volume de dados constituem um problema relevante e desafiador em diferentes Ãreas do conhecimento, a saber, medicina, biologia, computaÃÃo, catalogaÃÃo em geral, etc. A indexaÃÃo das imagens nestas bases pode ser realizada atravÃs de conteÃdo visual como cor, textura e forma, sendo esta Ãltima caracterÃstica a traduÃÃo visual dos objetos em uma cena. Tarefas automatizadas em inspeÃÃo industrial, registro de marca, biometria e descriÃÃo de imagens utilizam atributos da forma, como os cantos, na geraÃÃo de descritores para representaÃÃo, anÃlise e reconhecimento da mesma, possibilitando ainda que estes descritores se adequem ao uso em sistemas de recuperaÃÃo. Esta tese aborda o problema da extraÃÃo de caracterÃsticas de formas planares binÃrias a partir de cantos, na proposta de um detector multiescala de cantos e sua aplicaÃÃo em um sistema CBIR. O mÃtodo de detecÃÃo de cantos proposto combina uma funÃÃo de angulaÃÃo do contorno da forma, a sua decomposiÃÃo nÃo decimada por transformada wavelet ChapÃu Mexicano e a correlaÃÃo espacial entre as escalas do sinal de angulaÃÃo decomposto. A partir dos resultados de detecÃÃo de cantos, foi realizado um experimento com o sistema CBIR proposto, em que informaÃÃes locais e globais extraÃdas dos cantos detectados da forma foram combinadas à tÃcnica DeformaÃÃo Espacial DinÃmica (do termo em inglÃs, Dynamic Space Warping), para fins de anÃlise de similaridade formas com tamanhos distintos. Ainda com este experimento foi traÃada uma estratÃgia de busca e ajuste dos parÃmetros multiescala de detectores de cantos, segundo a maximizaÃÃo de uma funÃÃo de custo. Na avaliaÃÃo de desempenho da metodologia proposta, e outras tÃcnicas de detecÃÃo de cantos, foram empregadas as medidas PrecisÃo e RevocaÃÃo. Estas medidas atestaram o bom desempenho da metodologia proposta na detecÃÃo de cantos verdadeiros das formas, em uma base pÃblica de imagens cujas verdades terrestres estÃo disponÃveis. Para a avaliaÃÃo do experimento de recuperaÃÃo de imagens, utilizamos a taxa Bullâs eye em trÃs bases pÃblicas. Os valores alcanÃados desta taxa mostraram que o experimento proposto foi bem sucedido na descriÃÃo e recuperaÃÃo das formas, dentre os demais mÃtodos avaliados. / Content-based image retrieval (CBIR) applied to large scale datasets is a relevant and challenging problem present in medicine, biology, computer science, general cataloging etc. Image indexing can be done using visual information such as colors, textures and shapes (the visual translation of objects in a scene). Automated tasks in industrial inspection, trademark registration, biostatistics and image description use shape attributes, e.g. corners, to generate descriptors for representation, analysis and recognition; allowing those descriptors to be used in image retrieval systems. This thesis explores the problem of extracting information from binary planar shapes from corners, by proposing a multiscale corner detector and its use in a CBIR system. The proposed corner detection method combines an angulation function of the shape contour, its non-decimated decomposition using the Mexican hat wavelet and the spatial correlation among scales of the decomposed angulation signal. Using the information provided by our corner detection algorithm, we made experiments with the proposed CBIR. Local and global information extracted from the corners detected on shapes was used in a Dynamic Space Warping technique in order to analyze the similarity among shapes of different sizes. We also devised a strategy for searching and refining the multiscale parameters of the corner detector by maximizing an objective function. For performance evaluation of the proposed methodology and other techniques, we employed the Precision and Recall measures. These measures proved the good performance of our method in detecting true corners on shapes from a public image dataset with ground truth information. To assess the image retrieval experiments, we used the Bullâs eye score in three public databases. Our experiments showed our method performed well when compared to the existing approaches in the literature.