<p>In robot navigation, and image content searches reliable salient features are of pivotal importance. Also in biometric human recognition, salient features are increasingly used. </p><p>Regardless the application, image matching is one of the many problems in computer </p><p>vision, including object recognition. </p><p> </p><p>This report investigates some salient features to match sub-images of different images. </p><p>An underlying assumption is that sub-images, also called image objects, or objects, are </p><p>possible to recognize by the salient features that can be recognized independently. </p><p> </p><p>Since image objects are images of 3D objects, the salient features in 2D images must be </p><p>invariant to reasonably large viewing direction and distance (scale) changes. These </p><p>changes are typically due to 3D rotations and translations of the 3D object with respect to </p><p>the camera. Other changes that influence the matching of two 2D image objects is </p><p>illumination changes, and image acquisition noise. </p><p> </p><p>This thesis will discuss how to find the salient features and will compare them with </p><p>respect to their matching performance. Also it will explore how these features are </p><p>invariant to rotation and scaling.</p>

No description available.

No description available.

In robot navigation, and image content searches reliable salient features are of pivotal importance. Also in biometric human recognition, salient features are increasingly used. Regardless the application, image matching is one of the many problems in computer vision, including object recognition. This report investigates some salient features to match sub-images of different images. An underlying assumption is that sub-images, also called image objects, or objects, are possible to recognize by the salient features that can be recognized independently. Since image objects are images of 3D objects, the salient features in 2D images must be invariant to reasonably large viewing direction and distance (scale) changes. These changes are typically due to 3D rotations and translations of the 3D object with respect to the camera. Other changes that influence the matching of two 2D image objects is illumination changes, and image acquisition noise. This thesis will discuss how to find the salient features and will compare them with respect to their matching performance. Also it will explore how these features are invariant to rotation and scaling.

The image comparison operation ??sessing how well one image matches another ??rms a critical component of many image analysis systems and models of human visual processing. Two norms used commonly for this purpose are L1 and L2, which are specific instances of the Minkowski metric. However, there is often not a principled reason for selecting one norm over the other. One way to address this problem is by examining whether one metric better captures the perceptual notion of image similarity than the other. With this goal, we examined perceptual preferences for images retrieved on the basis of the L1 versus the L2 norm. These images were either small fragments without recognizable content, or larger patterns with recognizable content created via vector quantization. In both conditions the subjects showed a consistent preference for images matched using the L1 metric. These results suggest that, in the domain of natural images of the kind we have used, the L1 metric may better capture human notions of image similarity.

We present a new method to perform reliable matching between different images. This method exploits a projective invariant property between concentric circles and the corresponding projected ellipses to find complete region correspondences centered on interest points. The method matches interest points allowing for a full perspective transformation and exploiting all the available luminance information in the regions. Experiments have been conducted on many different data sets to compare our approach to SIFT local descriptors. The results show the new method offers increased robustness to partial visibility, object rotation in depth, and viewpoint angle change. / Singapore-MIT Alliance (SMA)

In this thesis, I explore region detection and consider its impact on image matching for exemplar-based object recognition. Detecting regions is important to provide semantically meaningful spatial cues in images. Matching establishes similarity between visual entities, which is crucial for recognition. My thesis starts by detecting regions in both local and object level. Then, I leverage geometric cues of the detected regions to improve image matching for the ultimate goal of object recognition. More specifically, my thesis considers four key questions: 1) how can we extract distinctively-shaped local regions that also ensure repeatability for robust matching? 2) how can object-level shape inform bottom-up image segmentation? 3) how should the spatial layout imposed by segmented regions influence image matching for exemplar-based recognition? and 4) how can we exploit regions to improve the accuracy and speed of dense image matching? I propose novel algorithms to tackle these issues, addressing region-based visual perception from low-level local region extraction, to mid-level object segmentation, to high-level region-based matching and recognition. First, I propose a Boundary Preserving Local Region (BPLR) detector to extract local shapes. My approach defines a novel spanning-tree based image representation whose structure reflects shape cues combined from multiple segmentations, which in turn provide multiple initial hypotheses of the object boundaries. Unlike traditional local region detectors that rely on local cues like color and texture, BPLRs explicitly exploit the segmentation that encodes global object shape. Thus, they respect object boundaries more robustly and reduce noisy regions that straddle object boundaries. The resulting detector yields a dense set of local regions that are both distinctive in shape as well as repeatable for robust matching. Second, building on the strength of the BPLR regions, I develop an approach for object-level segmentation. The key insight of the approach is that objects shapes are (at least partially) shared among different object categories--for example, among different animals, among different vehicles, or even among seemingly different objects. This shape sharing phenomenon allows us to use partial shape matching via BPLR-detected regions to predict global object shape of possibly unfamiliar objects in new images. Unlike existing top-down methods, my approach requires no category-specific knowledge on the object to be segmented. In addition, because it relies on exemplar-based matching to generate shape hypotheses, my approach overcomes the viewpoint sensitivity of existing methods by allowing shape exemplars to span arbitrary poses and classes. For the ultimate goal of region-based recognition, not only is it important to detect good regions, but we must also be able to match them reliably. A matching establishes similarity between visual entities (images, objects or scenes), which is fundamental for visual recognition. Thus, in the third major component of this thesis, I explore how to leverage geometric cues of the segmented regions for accurate image matching. To this end, I propose a segmentation-guided local feature matching strategy, in which segmentation suggests spatial layout among the matched local features within each region. To encode such spatial structures, I devise a string representation whose 1D nature enables efficient computation to enforce geometric constraints. The method is applied for exemplar-based object classification to demonstrate the impact of my segmentation-driven matching approach. Finally, building on the idea of regions for geometric regularization in image matching, I consider how a hierarchy of nested image regions can be used to constrain dense image feature matches at multiple scales simultaneously. Moving beyond individual regions, the last part of my thesis studies how to exploit regions' inherent hierarchical structure to improve the image matching. To this end, I propose a deformable spatial pyramid graphical model for image matching. The proposed model considers multiple spatial extents at once--from an entire image to grid cells to every single pixel. The proposed pyramid model strikes a balance between robust regularization by larger spatial supports on the one hand and accurate localization by finer regions on the other. Further, the pyramid model is suitable for fast coarse-to-fine hierarchical optimization. I apply the method to pixel label transfer tasks for semantic image segmentation, improving upon the state-of-the-art in both accuracy and speed. Throughout, I provide extensive evaluations on challenging benchmark datasets, validating the effectiveness of my approach. In contrast to traditional texture-based object recognition, my region-based approach enables to use strong geometric cues such as shape and spatial layout that advance the state-of-the-art of object recognition. Also, I show that regions' inherent hierarchical structure allows fast image matching for scalable recognition. The outcome realizes the promising potential of region-based visual perception. In addition, all my codes for local shape detector, object segmentation, and image matching are publicly available, which I hope will serve as useful new additions for vision researchers' toolbox. / text

Image matching is an important and necessary process in Content-Based Image Retrieval (CBIR). We propose three new methods for image matching: the first one is based on the Local Triplet Pattern (LTP) histograms; the second one is based on the Gaussian Mixture Models (GMMs) estimated by using the Extended Mass-constraint (EMass) algorithm; and the third one is called the DCT2KL algorithm. First, the LTP histograms are proposed to capture spatial relationships between color levels of neighboring pixels. An LTP level is extracted from each 3x3 pixel block, which is a unique number describing the color level relationship between a pixel and its neighboring pixels. Second, we consider how to represent and compare multi-dimensional color features using GMMs. GMMs are alternative methods to histograms for representing data distributions. GMMs address the high-dimensional problems from which histograms usually suffer inefficiency. In order to avoid local maxima problems in most GMM estimation algorithms, we apply the deterministic annealing method to estimate GMMs. Third, motivated by image compression algorithms, the DCT2KL method addresses the high dimensional data by using the Discrete Cosine Transform (DCT) coefficients in the YCbCr color space. The DCT coefficients are restored by partially decoding JPEG images. Assume that each DCT coefficient sequence is emitted from a memoryless source, and all these sources are independent of each other. For each target image we form a hypothesis that its DCT coefficient sequences are emitted from the same sources as the corresponding sequences in the query image. Testing these hypotheses by measuring the log-likelihoods leads to a simple yet efficient scheme that ranks each target image according to the Kullback-Leibler (KL) divergence between the empirical distribution of the DCT coefficient sequences in the query image and that in the target image. Finally we present a scheme to combine different features and methods to boost the performance of image retrieval. Experimental results on different image data sets show that these three methods proposed above outperform the related works in literature, and the combination scheme further improves the retrieval performance.

5th IFAC Symposium on Mechatronic Systems, Marriott Boston Cambridge, Cambridge, MA, USA, Sept 13-15, 2010

Sets of local features that are invariant to common image transformations are an effective representation to use when comparing images; current methods typically judge feature sets' similarity via a voting scheme (which ignores co-occurrence statistics) or by comparing histograms over a set of prototypes (which must be found by clustering). We present a method for efficiently comparing images based on their discrete distributions (bags) of distinctive local invariant features, without clustering descriptors. Similarity between images is measured with an approximation of the Earth Mover's Distance (EMD), which quickly computes the minimal-cost correspondence between two bags of features. Each image's feature distribution is mapped into a normed space with a low-distortion embedding of EMD. Examples most similar to a novel query image are retrieved in time sublinear in the number of examples via approximate nearest neighbor search in the embedded space. We also show how the feature representation may be extended to encode the distribution of geometric constraints between the invariant features appearing in each image.We evaluate our technique with scene recognition and texture classification tasks.