A área de reconhecimento de objetos tem assistido a um impressionante progresso na última década. O estudo de descritores, aliado à estratégias de amostragem usando quantizações espaciais e a combinação de classificadores têm permeado o estado da arte nos últimos anos. Neste trabalho é proposta uma nova quantização espacial com número arbitrário de níveis e subdivisões arbitrárias de regiões. Regiões adjacentes possuem sobreposição gerando redundância na representação destas regiões de fronteiras e, assim, evitando as quebras que acontecem nas pirâmides espaciais tradicionais que prejudicam a interpretação das formas. Apesar de melhorar o desempenho da abordagem do saco de palavras, as pirâmides espaciais não são robustas a variações na orientação dos objetos na imagem. Foi também proposto neste trabalho, uma divisão espacial utilizando regiões circulares concêntricas que aumentam a robustez a rotação dos objetos na imagem em aproximadamente 80% quando comparada às pirâmides espaciais. Além das novas divisões espaciais, é proposto neste trabalho um novo descritor baseado na aplicação de granulometria morfológica no mapa de bordas da imagem original. Este descritor foi utilizado na criação de modelos de classes em aplicações de categorização de objetos utilizando uma base de dados pública com resultados superiores aos do melhor descritor baseado em bordas reportado pela literatura. Todas estas novas técnicas propostas foram utilizadas em um problema desafiador de categorização de objetos de classes muito parecidas. Foi utilizado um subconjunto da base de pássaros Caltech-UCSD Birds-200 2011 com resultados comparáveis aos melhores resultados reportados pela literatura. A abordagem proposta cria uma classificação de dois níveis e utiliza modelos específicos por classe o que é intuitivo, pois cada espécie de pássaro possui características muito sutis que as diferenciam das demais espécies testadas. Vários descritores são utilizados na criação dos modelos de classes e uma combinação de classificadores gera a rotulação final para a amostra. O descritor proposto neste trabalho esteve presente no melhor modelo de 11 das 13 classes testadas e o resultado final obtido pela técnica de categorização proposta é o melhor resultado utilizando a abordagem do saco de palavras. / The object recognition area has experienced an impressive progress in the last decade. The study of descriptors, together with a sampling strategy using spatial quantization and the combination of classifiers have been presented in the state of art in recent years. This work proposes a new spatial quantizations with an arbitrary number of levels and divisions in each level. Adjacent regions have overlapping areas that generate redundant representation and avoid breakages in the structures that are in their border regions as it happens in the traditional spatial pyramids and impairs the correct interpretation of these structures. Despite spatial pyramids to improve the performance of the bag-of-words approach in object recognition, they are not robust to changes in object orientation in the image. It was also proposed, in this work, a spatial division using concentric circular regions that is almost 80% more robust to rotation of objects when compared to the spatial pyramids using rectangular divisions. In addition to the new spatial division of the image, it is proposed a new granulometric-based descriptor that it is applied to the map of edges of the original image. This descriptor was used in the building of categorys models for object categorization in a public database and showed a better performance than the most used edge-based descriptor reported in literature. All these new proposed techniques were used in a challenge problem of object categorization of very similar classes. It was used a subset of the public database Caltech-UCSD Birds-200 2011 and the method obtained results compared to the best results reported in the literature. The proposed approach uses a 2-level classification and builds class-specific models that are an intuitive way to model the species of birds as very subtle characteristics differ in each tested class of birds. Many descriptors are used in the building of models of species and a combination of classifiers generates the final label for a tested sample. The descriptor proposed here were presented in 11 of 13 best models of birds classes. The final result obtained by the proposed object categorization method is the best one using the bag-of-words approach.

A área de reconhecimento de objetos tem assistido a um impressionante progresso na última década. O estudo de descritores, aliado à estratégias de amostragem usando quantizações espaciais e a combinação de classificadores têm permeado o estado da arte nos últimos anos. Neste trabalho é proposta uma nova quantização espacial com número arbitrário de níveis e subdivisões arbitrárias de regiões. Regiões adjacentes possuem sobreposição gerando redundância na representação destas regiões de fronteiras e, assim, evitando as quebras que acontecem nas pirâmides espaciais tradicionais que prejudicam a interpretação das formas. Apesar de melhorar o desempenho da abordagem do saco de palavras, as pirâmides espaciais não são robustas a variações na orientação dos objetos na imagem. Foi também proposto neste trabalho, uma divisão espacial utilizando regiões circulares concêntricas que aumentam a robustez a rotação dos objetos na imagem em aproximadamente 80% quando comparada às pirâmides espaciais. Além das novas divisões espaciais, é proposto neste trabalho um novo descritor baseado na aplicação de granulometria morfológica no mapa de bordas da imagem original. Este descritor foi utilizado na criação de modelos de classes em aplicações de categorização de objetos utilizando uma base de dados pública com resultados superiores aos do melhor descritor baseado em bordas reportado pela literatura. Todas estas novas técnicas propostas foram utilizadas em um problema desafiador de categorização de objetos de classes muito parecidas. Foi utilizado um subconjunto da base de pássaros Caltech-UCSD Birds-200 2011 com resultados comparáveis aos melhores resultados reportados pela literatura. A abordagem proposta cria uma classificação de dois níveis e utiliza modelos específicos por classe o que é intuitivo, pois cada espécie de pássaro possui características muito sutis que as diferenciam das demais espécies testadas. Vários descritores são utilizados na criação dos modelos de classes e uma combinação de classificadores gera a rotulação final para a amostra. O descritor proposto neste trabalho esteve presente no melhor modelo de 11 das 13 classes testadas e o resultado final obtido pela técnica de categorização proposta é o melhor resultado utilizando a abordagem do saco de palavras. / The object recognition area has experienced an impressive progress in the last decade. The study of descriptors, together with a sampling strategy using spatial quantization and the combination of classifiers have been presented in the state of art in recent years. This work proposes a new spatial quantizations with an arbitrary number of levels and divisions in each level. Adjacent regions have overlapping areas that generate redundant representation and avoid breakages in the structures that are in their border regions as it happens in the traditional spatial pyramids and impairs the correct interpretation of these structures. Despite spatial pyramids to improve the performance of the bag-of-words approach in object recognition, they are not robust to changes in object orientation in the image. It was also proposed, in this work, a spatial division using concentric circular regions that is almost 80% more robust to rotation of objects when compared to the spatial pyramids using rectangular divisions. In addition to the new spatial division of the image, it is proposed a new granulometric-based descriptor that it is applied to the map of edges of the original image. This descriptor was used in the building of categorys models for object categorization in a public database and showed a better performance than the most used edge-based descriptor reported in literature. All these new proposed techniques were used in a challenge problem of object categorization of very similar classes. It was used a subset of the public database Caltech-UCSD Birds-200 2011 and the method obtained results compared to the best results reported in the literature. The proposed approach uses a 2-level classification and builds class-specific models that are an intuitive way to model the species of birds as very subtle characteristics differ in each tested class of birds. Many descriptors are used in the building of models of species and a combination of classifiers generates the final label for a tested sample. The descriptor proposed here were presented in 11 of 13 best models of birds classes. The final result obtained by the proposed object categorization method is the best one using the bag-of-words approach.

Researchers are still far from thoroughly understanding and building accurate computational models of the mechanisms in human mind that give rise to cognitive processes such as emergence of concepts and language acquisition. As a new attempt to give an insight into this issue, in this thesis, we are concerned about developing a computational model that leads to the emergence of concepts. Specically, we investigate how a robot can acquire verb and object concepts through learning affordances, a notion first proposed by J. J. Gibson in 1986. Using the affordance formalization framework of Sahin et al. in 2007, a humanoid robot acquires concepts through interactions in an embodied environment. For the acquisition of verb concepts, we take an alternative approach to the literature, which generally links verbs to specific behaviors of the robot, by linking them to specific effects that different behaviors may generate. We show how our robot can learn effect prototypes, represented in terms of feature changes in the perception vector of the robot, through demonstrations made by a human supervisor. As for the object concepts, we use the affordance relations of objects to create object concepts based on their functional relevance. Additionally, we show that the extracted eect prototypes corresponding to verb concepts can also be utilized to discover stable and variable properties of objects which can be associated to stable and variable affordances. Moreover, we show that the acquired concepts provide a suitable basis for communication with humans or other agents, for example to understand and imitate others&#039 / behaviors or for goal specication tasks. These capabilities are demonstrated in simple interaction games on the iCub humanoid robot platform.

This thesis is dedicated to the active research topic of generic Visual Object Categorization(VOC), which can be widely used in many applications such as videoindexation and retrieval, video monitoring, security access control, automobile drivingsupport etc. Due to many realistic difficulties, it is still considered to be one ofthe most challenging problems in computer vision and pattern recognition. In thiscontext, we have proposed in this thesis our contributions, especially concerning thetwo main components of the methods addressing VOC problems, namely featureselection and image representation.Firstly, an Embedded Sequential Forward feature Selection algorithm (ESFS)has been proposed for VOC. Its aim is to select the most discriminant features forobtaining a good performance for the categorization. It is mainly based on thecommonly used sub-optimal search method Sequential Forward Selection (SFS),which relies on the simple principle to add incrementally most relevant features.However, ESFS not only adds incrementally most relevant features in each stepbut also merges them in an embedded way thanks to the concept of combinedmass functions from the evidence theory which also offers the benefit of obtaining acomputational cost much lower than the one of original SFS.Secondly, we have proposed novel image representations to model the visualcontent of an image, namely Polynomial Modeling and Statistical Measures basedImage Representation, called PMIR and SMIR respectively. They allow to overcomethe main drawback of the popular "bag of features" method which is the difficultyto fix the optimal size of the visual vocabulary. They have been tested along withour proposed region based features and SIFT. Two different fusion strategies, earlyand late, have also been considered to merge information from different "channels"represented by the different types of features.Thirdly, we have proposed two approaches for VOC relying on sparse representation,including a reconstructive method (R_SROC) as well as a reconstructiveand discriminative one (RD_SROC). Indeed, sparse representation model has beenoriginally used in signal processing as a powerful tool for acquiring, representingand compressing the high-dimensional signals. Thus, we have proposed to adaptthese interesting principles to the VOC problem. R_SROC relies on the intuitiveassumption that an image can be represented by a linear combination of trainingimages from the same category. Therefore, the sparse representations of images arefirst computed through solving the ℓ1 norm minimization problem and then usedas new feature vectors for images to be classified by traditional classifiers such asSVM. To improve the discrimination ability of the sparse representation to betterfit the classification problem, we have also proposed RD_SROC which includes adiscrimination term, such as Fisher discrimination measure or the output of a SVMclassifier, to the standard sparse representation objective function in order to learna reconstructive and discriminative dictionary. Moreover, we have also proposedChapter 0. Abstractto combine the reconstructive and discriminative dictionary and the adapted purereconstructive dictionary for a given category so that the discrimination power canfurther be increased.The efficiency of all the methods proposed in this thesis has been evaluated onpopular image datasets including SIMPLIcity, Caltech101 and Pascal2007.

Perceptual categorization involves integrating bottom-up sensory information with top-down knowledge which is based on prior experience. Bottom-up information comes from the external world and visual saliency is a type of bottom-up information that is calculated on the differences between the visual characteristics of adjacent spatial locations. There is currently a related debate in municipal law enforcement communities about which are more ‘visible’: white police cars or black and white police cars. Municipalities do not want police cars to be hit by motorists and they also want police cars to be seen in order to promote a public presence. The present study used three behavioral experiments to investigate the effects of visual saliency on object detection and categorization. Importantly, the results indicated that so-called ‘object detection’ is not a valid construct. Rather than identifying objectness or objecthood prior to categorization, object categorization is an obligatory process, and object detection is a postcategorization decision with higher salience objects being categorized easier than lower salience objects. An additional experiment was conducted to examine the features that constitute a police car. Based on salience alone, black and white police cars were better categorized than white police cars and light bars were slightly more important police car defining components than markings.

In order for assistive robots to collaborate effectively with humans for completing everyday tasks, they must be endowed with the ability to effectively perceive scenes and more importantly, recognize human intentions. As a result, we present in this dissertation a novel scene-dependent human-robot collaborative system capable of recognizing and learning human intentions based on scene objects, the actions that can be performed on them, and human interaction history. The aim of this system is to reduce the amount of human interactions necessary for communicating tasks to a robot. Accordingly, the system is partitioned into scene understanding and intention recognition modules. For scene understanding, the system is responsible for segmenting objects from captured RGB-D data, determining their positions and orientations in space, and acquiring their category labels. This information is fed into our intention recognition component where the most likely object and action pair that the user desires is determined. Our contributions to the state of the art are manifold. We propose an intention recognition framework that is appropriate for persons with limited physical capabilities, whereby we do not observe human physical actions for inferring intentions as is commonplace, but rather we only observe the scene. At the core of this framework is our novel probabilistic graphical model formulation entitled Object-Action Intention Networks. These networks are undirected graphical models where the nodes are comprised of object, action, and object feature variables, and the links between them indicate some form of direct probabilistic interaction. This setup, in tandem with a recursive Bayesian learning paradigm, enables our system to adapt to a user's preferences. We also propose an algorithm for the rapid estimation of position and orientation values of scene objects from single-view 3D point cloud data using a multi-scale superquadric fitting approach. Additionally, we leverage recent advances in computer vision for an RGB-D object categorization procedure that balances discrimination and generalization as well as a depth segmentation procedure that acquires candidate objects from tabletops. We demonstrate the feasibility of the collaborative system presented herein by conducting evaluations on multiple scenes comprised of objects from 11 categories, along with 7 possible actions, and 36 possible intentions. We achieve approximately 81% reduction in interactions overall after learning despite changes to scene structure.

Cette thèse de doctorat est consacrée à un sujet de recherche très porteur : la Catégorisation générique d’objets Visuels (VOC). En effet, les applications possibles sont très nombreuses, incluant l’indexation d’images et de vidéos, la vidéo surveillance, le contrôle d’accès de sécurité, le soutien à la conduite automobile, etc. En raison de ses nombreux verrous scientifiques, ce sujet est encore considéré comme l’un des problèmes les plus difficiles en vision par ordinateur et en reconnaissance de formes. Dans ce contexte, nous avons proposé dans ce travail de thèse plusieurs contributions, en particulier concernant les deux principaux éléments des méthodes résolvant les problèmes de VOC, notamment la sélection des descripteurs et la représentation d’images. Premièrement, un algorithme nomme "Embedded Sequential Forward feature Selection"(ESFS) a été proposé pour VOC. Son but est de sélectionner les descripteurs les plus discriminants afin d’obtenir une bonne performance pour la catégorisation. Il est principalement basé sur la méthode de recherche sous-optimale couramment utilisée "Sequential Forward Selection" (SFS), qui repose sur le principe simple d’ajouter progressivement les descripteurs les plus pertinents. Cependant, ESFS non seulement ajoute progressivement les descripteurs les plus pertinents à chaque étape mais de plus les fusionne d’une manière intégrée grâce à la notion de fonctions de masses combinées empruntée à la théorie de l’évidence qui offre également l’avantage d’obtenir un coût de calcul beaucoup plus faible que celui de SFS original. Deuxièmement, nous avons proposé deux nouvelles représentations d’images pour modéliser le contenu visuel d’une image : la Représentation d’Image basée sur la Modélisation Polynomiale et les Mesures Statistiques, appelées respectivement PMIR et SMIR. Elles permettent de surmonter l’inconvénient principal de la méthode populaire "bag of features" qui est la difficulté de fixer la taille optimale du vocabulaire visuel. Elles ont été testées avec nos descripteurs bases région ainsi que les descripteurs SIFT. Deux stratégies différentes de fusion, précoce et tardive, ont également été considérées afin de fusionner les informations venant des "canaux «différents représentés par les différents types de descripteurs. Troisièmement, nous avons proposé deux approches pour VOC en s’appuyant sur la représentation sparse. La première méthode est reconstructive (R_SROC) alors que la deuxième est reconstructive et discriminative (RD_SROC). En effet, le modèle de représentation sparse a été utilisé originalement dans le domaine du traitement du signal comme un outil puissant pour acquérir, représenter et compresser des signaux de grande dimension. Ainsi, nous avons proposé une adaptation de ces principes intéressants au problème de VOC. R_SROC repose sur l’hypothèse intuitive que l’image peut être représentée par une combinaison linéaire des images d’apprentissage de la même catégorie. [...] / This thesis is dedicated to the active research topic of generic Visual Object Categorization(VOC), which can be widely used in many applications such as videoindexation and retrieval, video monitoring, security access control, automobile drivingsupport etc. Due to many realistic difficulties, it is still considered to be one ofthe most challenging problems in computer vision and pattern recognition. In thiscontext, we have proposed in this thesis our contributions, especially concerning thetwo main components of the methods addressing VOC problems, namely featureselection and image representation.Firstly, an Embedded Sequential Forward feature Selection algorithm (ESFS)has been proposed for VOC. Its aim is to select the most discriminant features forobtaining a good performance for the categorization. It is mainly based on thecommonly used sub-optimal search method Sequential Forward Selection (SFS),which relies on the simple principle to add incrementally most relevant features.However, ESFS not only adds incrementally most relevant features in each stepbut also merges them in an embedded way thanks to the concept of combinedmass functions from the evidence theory which also offers the benefit of obtaining acomputational cost much lower than the one of original SFS.Secondly, we have proposed novel image representations to model the visualcontent of an image, namely Polynomial Modeling and Statistical Measures basedImage Representation, called PMIR and SMIR respectively. They allow to overcomethe main drawback of the popular "bag of features" method which is the difficultyto fix the optimal size of the visual vocabulary. They have been tested along withour proposed region based features and SIFT. Two different fusion strategies, earlyand late, have also been considered to merge information from different "channels"represented by the different types of features.Thirdly, we have proposed two approaches for VOC relying on sparse representation,including a reconstructive method (R_SROC) as well as a reconstructiveand discriminative one (RD_SROC). Indeed, sparse representation model has beenoriginally used in signal processing as a powerful tool for acquiring, representingand compressing the high-dimensional signals. Thus, we have proposed to adaptthese interesting principles to the VOC problem. R_SROC relies on the intuitiveassumption that an image can be represented by a linear combination of trainingimages from the same category. Therefore, the sparse representations of images arefirst computed through solving the ℓ1 norm minimization problem and then usedas new feature vectors for images to be classified by traditional classifiers such asSVM. To improve the discrimination ability of the sparse representation to betterfit the classification problem, we have also proposed RD_SROC which includes adiscrimination term, such as Fisher discrimination measure or the output of a SVMclassifier, to the standard sparse representation objective function in order to learna reconstructive and discriminative dictionary. Moreover, we have also proposedChapter 0. Abstractto combine the reconstructive and discriminative dictionary and the adapted purereconstructive dictionary for a given category so that the discrimination power canfurther be increased.The efficiency of all the methods proposed in this thesis has been evaluated onpopular image datasets including SIMPLIcity, Caltech101 and Pascal2007.

No description available.

Object categorization is one of the fundamental topics in computer vision research. Most current work in object categorization aims to discriminate among generic object classes with gross differences. However, many applications require much finer distinctions. This thesis focuses on the design, evaluation and analysis of learning algorithms for fine- grained object classification. The contributions of the thesis are three-fold. First, we introduce two databases of high-resolution images of arthropod specimens we collected to promote the development of highly accurate fine-grained recognition methods. Second, we give a literature review on the development of Bag-of-words (BOW) approaches to image classification and present the stacked evidence tree approach we developed for the fine-grained classification task. We draw connections and analyze differences between those two genres of approaches, which leads to a better understanding about the design of image classification approaches. Third, benchmark results on our two datasets are pre- sented. We further analyze the influence of two important variables on the performance of fine-grained classification. The experiments corroborate our hypotheses that a) high resolution images and b) more aggressive information extraction, such as finer descriptor encoding with large dictionaries or classifiers based on raw descriptors, is required to achieve good fine-grained categorization accuracy. / Graduation date: 2013

Visual object category recognition is one of the most challenging problems in computer vision. Even assuming that we can obtain a near-perfect instance level representation with the advances in visual input devices and low-level vision techniques, object categorization still remains as a difficult problem because it requires drawing boundaries between instances in a continuous world, where the boundaries are solely defined by human conceptualization. Object categorization is essentially a perceptual process that takes place in a human-defined semantic space. In this semantic space, the categories reside not in isolation, but in relation to others. Some categories are similar, grouped, or co-occur, and some are not. However, despite this semantic nature of object categorization, most of the today's automatic visual category recognition systems rely only on the category labels for training discriminative recognition with statistical machine learning techniques. In many cases, this could result in the recognition model being misled into learning incorrect associations between visual features and the semantic labels, from essentially overfitting to training set biases. This limits the model's prediction power when new test instances are given. Using semantic knowledge has great potential to benefit object category recognition. First, semantic knowledge could guide the training model to learn a correct association between visual features and the categories. Second, semantics provide much richer information beyond the membership information given by the labels, in the form of inter-category and category-attribute distances, relations, and structures. Finally, the semantic knowledge scales well as the relations between categories become larger with an increasing number of categories. My goal in this thesis is to learn discriminative models for categorization that leverage semantic knowledge for object recognition, with a special focus on the semantic relationships among different categories and concepts. To this end, I explore three semantic sources, namely attributes, taxonomies, and analogies, and I show how to incorporate them into the original discriminative model as a form of structural regularization. In particular, for each form of semantic knowledge I present a feature learning approach that defines a semantic embedding to support the object categorization task. The regularization penalizes the models that deviate from the known structures according to the semantic knowledge provided. The first semantic source I explore is attributes, which are human-describable semantic characteristics of an instance. While the existing work treated them as mid-level features which did not introduce new information, I focus on their potential as a means to better guide the learning of object categories, by enforcing the object category classifiers to share features with attribute classifiers, in a multitask feature learning framework. This approach essentially discovers the common low-dimensional features that support predictions in both semantic spaces. Then, I move on to the semantic taxonomy, which is another valuable source of semantic knowledge. The merging and splitting criteria for the categories on a taxonomy are human-defined, and I aim to exploit this implicit semantic knowledge. Specifically, I propose a tree of metrics (ToM) that learns metrics that capture granularity-specific similarities at different nodes of a given semantic taxonomy, and uses a regularizer to isolate granularity-specific disjoint features. This approach captures the intuition that the features used for the discrimination of the parent class should be different from the features used for the children classes. Such learned metrics can be used for hierarchical classification. The use of a single taxonomy can be limited in that its structure is not optimal for hierarchical classification, and there may exist no single optimal semantic taxonomy that perfectly aligns with visual distributions. Thus, I next propose a way to overcome this limitation by leveraging multiple taxonomies as semantic sources to exploit, and combine the acquired complementary information across multiple semantic views and granularities. This allows us, for example, to synthesize semantics from both 'Biological', and 'Appearance'-based taxonomies when learning the visual features. Finally, as a further exploration of more complex semantic relations different from the previous two pairwise similarity-based models, I exploit analogies, which encode the relational similarities between two related pairs of categories. Specifically, I use analogies to regularize a discriminatively learned semantic embedding space for categorization, such that the displacements between the two category embeddings in both category pairs of the analogy are enforced to be the same. Such a constraint allows for a more confusing pair of categories to benefit from a clear separation in the matched pair of categories that share the same relation. All of these methods are evaluated on challenging public datasets, and are shown to effectively improve the recognition accuracy over purely discriminative models, while also guiding the recognition to be more semantic to human perception. Further, the applications of the proposed methods are not limited to visual object categorization in computer vision, but they can be applied to any classification problems where there exists some domain knowledge about the relationships or structures between the classes. Possible applications of my methods outside the visual recognition domain include document classification in natural language processing, and gene-based animal or protein classification in computational biology. / text