Artifacts made by humans, such as items of furniture and houses, exhibit an enormous amount of variability in shape. In this paper, we concentrate on models of the shapes of objects that are made up of fixed collections of sub-parts whose dimensions and spatial arrangement exhibit variation. Our goals are: to learn these models from data and to use them for recognition. Our emphasis is on learning and recognition from three-dimensional data, to test the basic shape-modeling methodology. In this paper we also demonstrate how to use models learned in three dimensions for recognition of two-dimensional sketches of objects. / Singapore-MIT Alliance (SMA)

A regular texture is formed from a regular congruent tiling of perceptually meaningful texture elements, also known as textons. If the tiling statistically deviates from regularity, either by texton structure, colour, or size, the texture is called near-regular. If we continue to perturb the tiling, the texture becomes stochastic. The set of possible textures that lie between regular and stochastic make up the texture spectrum: regular, near-regular, regular, near-stochastic, and stochastic. <p>In this thesis we provide a solution to the problem of creating, from a near-regular texture, a lattice which defines the placement of textons. We divide the problem into two distinct sub-areas: finding textons within an image, and lattice creation using both an ad-hoc method and a graph-theoretic method. <p>The problem of finding textons within an image is addressed using correlation. A texton selected by the user is correlated with the image and points of high correlation are extracted using non-maximal suppression. To extend this framework to irregular textures, we present early results on the use of feature space during correlation. We also present a method of correcting for a specific type of error in the texton finding result using frequency-space analysis. <p>Given texton locations, we provide two methods of creating a lattice. The ad-hoc method is able to create a lattice in spite of inconsistencies in the texton locating data. However, as texture becomes irregular the ad-hoc lattice construction method fails to correctly connect textons. To overcome this failure we adapt methods of creating proximity graphs, which join two textons whose neighbourhoods satisfy certain criteria, to our problem. The proximity graphs are parameterized for selection of the most appropriate graph choice for a given texture, solving the general lattice construction problem given correct texton locations. <p>In the output of the algorithm, centres of textons will be connected by edges in the lattice following the structure of texton placement within the input image. More precisely, for a texture T, we create a graph G = (V,E) dependent on T, where V is a set of texton centres, and E ={(v_i, v_j)} is a set of edges, where v_i, v_j are in V. Each edge e in E connects texton centre v in V to its most perceptually sensible neighbours.

A regular texture is formed from a regular congruent tiling of perceptually meaningful texture elements, also known as textons. If the tiling statistically deviates from regularity, either by texton structure, colour, or size, the texture is called near-regular. If we continue to perturb the tiling, the texture becomes stochastic. The set of possible textures that lie between regular and stochastic make up the texture spectrum: regular, near-regular, regular, near-stochastic, and stochastic. <p>In this thesis we provide a solution to the problem of creating, from a near-regular texture, a lattice which defines the placement of textons. We divide the problem into two distinct sub-areas: finding textons within an image, and lattice creation using both an ad-hoc method and a graph-theoretic method. <p>The problem of finding textons within an image is addressed using correlation. A texton selected by the user is correlated with the image and points of high correlation are extracted using non-maximal suppression. To extend this framework to irregular textures, we present early results on the use of feature space during correlation. We also present a method of correcting for a specific type of error in the texton finding result using frequency-space analysis. <p>Given texton locations, we provide two methods of creating a lattice. The ad-hoc method is able to create a lattice in spite of inconsistencies in the texton locating data. However, as texture becomes irregular the ad-hoc lattice construction method fails to correctly connect textons. To overcome this failure we adapt methods of creating proximity graphs, which join two textons whose neighbourhoods satisfy certain criteria, to our problem. The proximity graphs are parameterized for selection of the most appropriate graph choice for a given texture, solving the general lattice construction problem given correct texton locations. <p>In the output of the algorithm, centres of textons will be connected by edges in the lattice following the structure of texton placement within the input image. More precisely, for a texture T, we create a graph G = (V,E) dependent on T, where V is a set of texton centres, and E ={(v_i, v_j)} is a set of edges, where v_i, v_j are in V. Each edge e in E connects texton centre v in V to its most perceptually sensible neighbours.

Existing methods for image-based location estimation generally attempt to recognize every photo independently, and their resulting reliance on strong visual feature matches makes them most suited for distinctive landmark scenes. We observe that when touring a city, people tend to follow common travel patterns---for example, a stroll down Wall Street might be followed by a ferry ride, then a visit to the Statue of Liberty or Ellis Island museum. We propose an approach that learns these trends directly from online image data, and then leverages them within a Hidden Markov Model to robustly estimate locations for novel sequences of tourist photos. We further devise a set-to-set matching-based likelihood that treats each ``burst" of photos from the same camera as a single observation, thereby better accommodating images that may not contain particularly distinctive scenes. Our experiments with two large datasets of major tourist cities clearly demonstrate the approach's advantages over traditional methods that recognize each photo individually, as well as a naive HMM baseline that lacks the proposed burst-based observation model. / text

Current uses of tagged images typically exploit only the most explicit information: the link between the nouns named and the objects present somewhere in the image. We propose to leverage “unspoken” cues that rest within an ordered list of image tags so as to improve object localization. We define three novel implicit features from an image’s tags—the relative prominence of each object as signified by its order of mention, the scale constraints implied by unnamed objects, and the loose spatial links hinted by the proximity of names on the list. By learning a conditional density over the localization parameters (position and scale) given these cues, we show how to improve both accuracy and efficiency when detecting the tagged objects. We validate our approach with 25 object categories from the PASCAL VOC and LabelMe datasets, and demonstrate its effectiveness relative to both traditional sliding windows as well as a visual context baseline. / text

Perirhinal cortical-dependent behavior and single-unit neuron activity in this brain region were compared between normal aged and young rats. Three different variants of the spontaneous object recognition task were used in these experiments, and the results confirmed previous reports that aged animals are impaired at stimulus recognition. The novel contribution of the present experiments was the identification that the behavioral deficit in the aged rats was due to the old animals treating novel objects as familiar, rather than to forgetting the previously experienced stimuli. This pattern of results in the old animals mirror data obtained from rats with perirhinal cortical lesions and promotes a hypothesis that this area of the brain serves a perceptual-mnemonic function. Additionally, multiple single-unit recordings were obtained from perirhinal cortical cells while young and aged rats traversed a track that contained several objects. Perirhinal neurons exhibited selective increases in their firing rates at object locations. We have called these areas of higher perirhinal cortical cell activity `object fields'. While both young and old rats expressed object fields, a lower proportion of perirhinal neurons showed this type of activity in the aged compared to the young rats. Although familiar and novel objects were placed on the track as part of a systematic design, there was no effect of novelty on the overall firing rates of perirhinal cortical neurons or the proportion of cells expressing object fields under these experimental conditions. These data suggest that the physiological correlate of stimulus recognition is not decrements in perirhinal cortical activity when a stimulus goes from novel to familiar. A final important observation made during these studies in young rats was that place fields in the middle hippocampal CA1 subregion are affected by placing objects in the track. Because this same manipulation increases perirhinal cortical activity, it could indicate that age-related changes in the perirhinal cortex might alter the function of other closely associated structures.

Discriminative learning is challenging when examples are setsof local image features, and the sets vary in cardinality and lackany sort of meaningful ordering. Kernel-based classificationmethods can learn complex decision boundaries, but a kernelsimilarity measure for unordered set inputs must somehow solve forcorrespondences -- generally a computationally expensive task thatbecomes impractical for large set sizes. We present a new fastkernel function which maps unordered feature sets tomulti-resolution histograms and computes a weighted histogramintersection in this space. This ``pyramid match" computation islinear in the number of features, and it implicitly findscorrespondences based on the finest resolution histogram cell wherea matched pair first appears. Since the kernel does not penalize thepresence of extra features, it is robust to clutter. We show thekernel function is positive-definite, making it valid for use inlearning algorithms whose optimal solutions are guaranteed only forMercer kernels. We demonstrate our algorithm on object recognitiontasks and show it to be dramatically faster than currentapproaches.

Numerous psychophysical experiments have shown an important role for attentional modulations in vision. Behaviorally, allocation of attention can improve performance in object detection and recognition tasks. At the neural level, attention increases firing rates of neurons in visual cortex whose preferredstimulus is currently attended to. However, it is not yet known how these two phenomena are linked, i.e., how the visual system could be "tuned" in a task-dependent fashion to improve task performance. To answer this question, we performed simulations with the HMAX model of object recognition in cortex [45].We modulated firing rates of model neurons in accordance with experimental results about effects of feature-based attention on single neurons and measured changes in the model's performance in a variety of object recognition tasks. It turned out that recognition performance could only be improved under very limited circumstances and that attentional influences on the process of object recognition per se tend to display a lack of specificity or raise false alarm rates. These observations lead us to postulate a new role for the observed attention-related neural response modulations.

Local descriptors are increasingly used for the task of object recognition because of their perceived robustness with respect to occlusions and to global geometrical deformations. Such a descriptor--based on a set of oriented Gaussian derivative filters-- is used in our recognition system. We report here an evaluation of several techniques for orientation estimation to achieve rotation invariance of the descriptor. We also describe feature selection based on a single training image. Virtual images are generated by rotating and rescaling the image and robust features are selected. The results confirm robust performance in cluttered scenes, in the presence of partial occlusions, and when the object is embedded in different backgrounds.

Localized operators, like Gabor wavelets and difference-of-Gaussian filters, are considered to be useful tools for image representation. This is due to their ability to form a ‘sparse code’ that can serve as a basis set for high-fidelity reconstruction of natural images. However, for many visual tasks, the more appropriate criterion of representational efficacy is ‘recognition’, rather than ‘reconstruction’. It is unclear whether simple local features provide the stability necessary to subserve robust recognition of complex objects. In this paper, we search the space of two-lobed differential operators for those that constitute a good representational code under recognition/discrimination criteria. We find that a novel operator, which we call the ‘dissociated dipole’ displays useful properties in this regard. We describe simple computational experiments to assess the merits of such dipoles relative to the more traditional local operators. The results suggest that non-local operators constitute a vocabulary that is stable across a range of image transformations.