Dans le cadre du projet Européen X-Media, de nombreuses contributions ont été apportées aux problèmes de classification d'image et de recherche d'images par le contenu dans des contextes industriels hétérogènes. Ainsi, après avoir établi un état de l'art des descripteurs d'image les plus courant, nous nous sommes dans un premier temps intéressé a des méthodes globales, c'est à dire basée sur la description totale de l'image par des descripteurs. Puis, nous nous sommes attachés a une analyse plus fine du contenu des images afin d'en extraire des informations locales, sur la présence et la localisation d'objets d'intérêt. Enfin, nous avons proposé une méthode hybride de recherche d'image basée sur le contenu qui s'appuie sur la description locale des régions de l'image afin d'en tirer une signature pouvant être utilisée pour des requêtes globales et locales. / The explosive development of affordable, high quality image acquisition deviceshas made available a tremendous amount of digital content. Large industrial companies arein need of efficient methods to exploit this content and transform it into valuable knowledge.This PhD has been accomplished in the context of the X-MEDIA project, a large Europeanproject with two major industrial partners, FIAT for the automotive industry andRolls-Royce plc. for the aircraft industry. The project has been the trigger for research linkedwith strong industrial requirements. Although those user requirements can be very specific,they covered more generic research topics. Hence, we bring several contributions in thegeneral context of Content-Based Image Retrieval (CBIR), Indexing and Classification.In the first part of the manuscript we propose contributions based on the extraction ofglobal image descriptors. We rely on well known descriptors from the literature to proposemodels for the indexing of image databases, and the approximation of a user defined categorisation.Additionally, we propose a new descriptor for a CBIR system which has toprocess a very specific image modality, for which traditional descriptors are irrelevant. Inthe second part of the manuscript, we focus on the task of image classification. Industrialrequirements on this topic go beyond the task of global image classification. We developedtwo methods to localize and classify the local content of images, i.e. image regions, usingsupervised machine learning algorithms (Support Vector Machines). In the last part of themanuscript, we propose a model for Content-Based Image Retrieval based on the constructionof a visual dictionary of image regions. We extensively experiment the model in orderto identify the most influential parameters in the retrieval efficiency.

The objective of this work is to improve performance in fine-grained visual categorization (FGVC). In particular, we are interested in the large-scale classification between hundreds of different flower, bird, dog species. FGVC is challenging due to high intra-class variances caused by deformation, view angle, illumination and occlusion, and low inter-class variance since some categories only differ in detail that only experts notice. Applications include field guides, automatic image annotation, one-click shopping app and 3D reconstruction. At the start, we discuss the importance of foreground segmentation in FGVC, where we focus on the unsupervised segmentation of image training sets into fore- ground and background in order to improve image classification performance. To this end, we introduce a new scalable, alternation-based algorithm for co-segmentation, Bi-CoS, which is simpler than many of its predecessors, and yet has superior performance on standard benchmark image datasets. Next, we extend BiCos to a new model, Tri- CoS, that adds a class-discriminitiveness term directly into the segmentation objective. The new term aims at removing image regions that, although appearing as foreground, do not contribute to the discrimination between classes. We also propose a model that combines parts alignment and foreground segmentation into a unified convex framework. The model is called Symbiotic in that part discovery/localization is helped by segmentation and, conversely, the segmentation is helped by the detection (e.g. part layout). The joined system improves over what can be achieved with an analogous system that runs segmentation and part-localization independently. Finally, we built a new flower dataset consisting of 26,798 high quality images collected by ourselves and 187,559 images gathered from existing datasets. The construction of this dataset follows a strict biological taxonomy. We also evaluate the impact of using the Amazon Mechanical Turk (AMT) service for filtering fine-grained data.

Ces travaux de recherche ont pour ambition de contribuer à la problématique de la classification d’images de documents. Plus précisément, ces travaux tendent à répondre aux problèmes rencontrés par des sociétés de numérisation dont l’objectif est de mettre à disposition de leurs clients une version numérique des documents papiers accompagnés d’informations qui leurs sont relatives. Face à la diversité des documents à numériser, l’extraction d’informations peut s’avérer parfois complexe. C’est pourquoi la classification et l’indexation des documents sont très souvent réalisées manuellement. Ces travaux de recherche ont permis de fournir différentes solutions en fonction des connaissances relatives aux images que possède l’utilisateur ayant en charge l’annotation des documents.Le premier apport de cette thèse est la mise en place d’une méthode permettant, de manière interactive, à un utilisateur de classer des images de documents dont la nature est inconnue. Le second apport de ces travaux est la proposition d’une technique de recherche d’images de documents par l’exemple basée sur l’extraction et la mise en correspondance de points d’intérêts. Le dernier apport de cette thèse est l’élaboration d’une méthode de classification d’images de documents utilisant les techniques de sacs de mots visuels. / The aim of this research is to contribute to the document image classification problem. More specifically, these studies address digitizing company issues which objective is to provide the digital version of paper document with information relating to them. Given the diversity of documents, information extraction can be complex. This is why the classification and the indexing of documents are often performed manually. This research provides several solutions based on knowledge of the images that the user has. The first contribution of this thesis is a method for classifying interactively document images, where the content of documents and classes are unknown. The second contribution of this work is a new technique for document image retrieval by giving one example of researched document. This technique is based on the extraction and matching of interest points. The last contribution of this thesis is a method for classifying document images by using bags of visual words techniques.

Land cover classification using artificial neural networks Abstract This Diploma thesis deals with automatic classification of the satellite high spatial resolution image in the field of land cover. The first half of the work contains the theoretical information about remote sensing and classification methods. The biggest attention is given to the artificial neural networks. In practical part of Diploma thesis are these methods used for the classification of SPOT satellite image. Keywords: remote sensing, image classification, artificial neural networks, SPOT

Image classification is an active topic of computer vision research. This topic deals with the learning of patterns in order to allow efficient classification of visual information. However, most research efforts have focused on 2D image classification. In recent years, advances of 3D imaging enabled the development of applications and provided new research directions. In this thesis, we present methodologies and techniques for image classification using 3D image data. We conducted our research focusing on the attributes and limitations of depth information regarding possible uses. This research led us to the development of depth feature extraction methodologies that contribute to the representation of images thus enhancing the recognition efficiency. We proposed a new classification algorithm that adapts to the need of image representations by implementing a scale-based decision that exploits discriminant parts of representations. Learning from the design of image representation methods, we introduced our own which describes each image by its depicting content providing more discriminative image representation. We also propose a dictionary learning method that exploits the relation of training features by assessing the similarity of features originating from similar context regions. Finally, we present our research on deep learning algorithms combined with data and techniques used in 3D imaging. Our novel methods provide state-of-the-art results, thus contributing to the research of 3D image classification.

Doctor of Philosophy / Department of Computer Science / David A. Gustafson / In this paper we evaluate a general form of image classification algorithm based on dense SIFT sampling. This algorithm is present in some form in most state-of-the-art classification systems. However, in this algorithm, numerous parameters must be tuned, and current research provides little insight into effective parameter tuning. We explore the relationship between various parameters and classification performance. Many of our results suggest that there are basic modifications which would improve state-of-the-art algorithms. Additionally, we develop two novel concepts, sampling redundancy and semantic capacity, to explain our data. These concepts provide additional insight into the limitations and potential improvements of state-of-the-art algorithms.

It is estimated that less than ten percent of the world's species have been discovered and described. The main reason for the slow pace of new species description is that the science of taxonomy, as traditionally practiced, can be very laborious: taxonomists have to manually gather and analyze data from large numbers of specimens and identify the smallest subset of external body characters that uniquely diagnoses the new species as distinct from all its known relatives. The pace of data gathering and analysis can be greatly increased by the information technology. In this paper, we propose a content-based image retrieval system for taxonomic research. The system can identify representative body shape characters of known species based on digitized landmarks and provide statistical clues for assisting taxonomists to identify new species or subspecies. The experiments on a taxonomic problem involving species of suckers in the genera Carpiodes demonstrate promising results.

Segundo a Agência Nacional de Transportes Terrestres (ANTT) em seu Anuário Estatístico dos Transportes Terrestres AETT (2008), o Brasil em todo o seu território possui 211.678 quilômetros de rodovias pavimentadas. O valor de serventia do pavimento diminui com o passar do tempo por dois fatores principais: o tráfego e as intempéries (BERNUCCI et al., 2008). Monitorar a condição de uso de toda a extensão das rodovias brasileiras é tarefa dispendiosa e demorada. A investigação de novas técnicas que permitam o levantamento da condição dos pavimentos de forma ágil e automática é parte da pesquisa deste trabalho. Nos últimos anos, um número crescente de imagens de alta resolução espacial tem surgido no mercado mundial com o aparecimento dos novos satélites e sensores aeroembarcados de sensoriamento remoto. Da mesma forma, imagens multiespectrais e até mesmo hiperespectrais estão sendo disponibilizadas comercialmente e para pesquisa científica. Neste trabalho são utilizadas imagens hiperespectrais de sensor digital aeroembarcado. Uma metodologia para identificação automática dos pavimentos asfaltados e classificação das principais ocorrências dos defeitos do asfalto foi desenvolvida. A primeira etapa da metodologia é a identificação do asfalto na imagem, utilizando uma classificação híbrida baseada inicialmente em pixel e depois refinada por objetos foi possível a extração da informação de asfalto das imagens disponíveis. A segunda etapa da metodologia é a identificação e classificação das ocorrências dos principais defeitos nos pavimentos flexíveis que são observáveis nas imagens de alta resolução espacial. Esta etapa faz uso intensivo das novas técnicas de classificação de imagens baseadas em objetos. O resultado final é a geração de índices da condição do pavimento, a partir das imagens, que possam ser comparados com os indicadores da qualidade da superfície do pavimento já normatizados pelos órgãos competentes no país. / According to Statistical Survey of Land Transportation AETT (2008) of National Agency of Land Transportation (ANTT), Brazil has in its territory 211,678 kilometers of paved roads. The pavement Present Serviceability Ratio (PSR) value decreases over time by two main factors: traffic and weather (BERNUCCI et al., 2008). Monitor the condition of use of all Brazilian roads is expensive and time consuming task. The investigation of new techniques that allow a quick and automatic survey of pavement condition is part of this research. In recent years, an increasing number of images with high spatial resolution has emerged on the world market with the advent of new remote sensing satellites and airborne sensors. Similarly, multispectral and even hyperspectral imagery are become available commercially and for scientific research nowadays. Hyperspectral images from digital airborne sensor have been used in this work. A new methodology for automatic identification of asphalted pavement and also for classification of the main defects of the asphalt has been developed. The first step of the methodology is the identification of the asphalt in the image, using hybrid classification based on pixel initially and after improved by objects. Using this approach was feasible to extract asphalt information from the available images. The second step of the methodology is the identification and classification of the main defects of flexible pavement surface that are observable in high spatial resolution imagery. This step makes intensive use of new techniques for classification of images based on objects. The goal, is the generation of pavement surface condition index from the images that can be compared with quality index of pavement surface that are already regulated by the regulatory agency in the country.

In Image Classification, pictures of the same type of object can have very different pixel values. Traditional norm-based metrics therefore fail to identify objectsin the same category. Topology is a branch of mathematics that deals with homeomorphic spaces, by discarding length. With topology, we can discover patterns in the image that are invariant to rotation, translation and warping. Persistent Homology is a new approach in Applied Topology that studies the presence of continuous regions and holes in an image. It has been used successfully for image segmentation and classification [12]. However, current approaches in image classification require a grayscale image to generate the persistence modules. This means information encoded in colour channels is lost. This thesis investigates whether the information in the red, green and blue colour channels of an RGB image hold additional information that could help algorithms classify pictures. We apply two recent methods, one by Adams [2] and the other by Hofer [25], on the CUB-200-2011 birds dataset [40] andfind that Hofer’s method produces significant results. Additionally, a modified method based on Hofer that uses the RGB colour channels produces significantly better results than the baseline, with over 48 % of images correctly classified, compared to 44 % and with a more significant improvement at lower resolutions.This indicates that colour channels do provide significant new information and generating one persistence module per colour channel is a viable approach to RGB image classification.

Imagens multiespectrais estão presentes na grande maioria de dispositivos de imageamento atuais, desde câmeras pessoais até microscópios, telescópios e satélites. No entanto, grande parte dos trabalhos em análise de texturas e afins propõem abordagens monocromáticas, que muitas vezes consideram apenas níveis de cinza. Nesse contexto e considerando o aumento da capacidade dos computadores atuais, o uso da informação espectral deve ser considerada na construção de modelos melhores. Ultimamente redes neurais convolucionais profundas pré-treinadas tem sido usadas em imagens coloridas de 3 canais, porém são limitadas a apenas esse formato e computam muitas convoluções, o que demanda por hardware específico (GPU). Esses fatos motivaram esse trabalho, que propõem técnicas para a modelagem e caracterização de imagens multiespectrais baseadas em redes complexas, que tem se mostrado uma ferramenta eficiente em trabalhos anteriores e possui complexidade computacional similar à métodos tradicionais. São introduzidas duas abordagens para aplicação em imagens coloridas de três canais, denominadas Rede Multicamada (RM) e Rede Multicamada Direcionada (RMD). Esses métodos modelam todos os canais da imagem de forma conjunta, onde as redes possuem conexões intra e entre canais, de forma parecida ao processamento oponente de cor do sistema visual humano. Experimentos em cinco bases de textura colorida mostram a proposta RMD supera vários métodos da literatura no geral, incluindo redes convolucionais e métodos tradicionais integrativos. Além disso, as propostas demonstraram alta robustez a diferentes espaços de cor (RGB, LAB, HSV e I1I2I3), enquanto que outros métodos oscilam de base para base. Também é proposto um método para caracterizar imagens multiespectrais de muitos canais, denominado Rede Direcionada de Similaridade Angular (RDSA). Nessa proposta, cada pixel multiespectral é considerado como um vetor de dimensão equivalente à quantidade de canais da imagem e o peso das arestas representa sua similaridade do cosseno, apontando para o pixel de maior valor absoluto. Esse método é aplicado em um conjunto de imagens de microscopia por fluorescência de 32 canais, em um experimento para identificar variações na estrutura foliar do espécime Jacaranda Caroba submetidos à diferentes condições. O método RDSA obtém as maiores taxas de acerto de classificação nesse conjunto de dados, com 91, 9% usando o esquema de validação cruzada Leave-one-out e 90, 5(±1, 1)% com 10-pastas, contra 81, 8% e 84, 7(±2, 2) da rede convolucional VGG16. / Multispectral images are present in the vast majority of current imaging devices, from personal cameras to microscopes, telescopes and satellites. However, much of the work in texture analysis and the like proposes monochromatic approaches, which often consider only gray levels. In this context and considering the performance increase of current computers, the use of the spectral information must be considered in the construction of better models. Lately, pre-trained deep convolutional neural networks have been used in 3-channel color images, however they are limited to just this format and compute many convolutions, which demands specific hardware (GPU). These facts motivated this work, which propose techniques for the modeling and characterization of multispectral images based on complex networks, which has proved to be an efficient tool in previous works and has computational complexity similar to traditional methods. Two approaches are introduced for application in 3-channel color images, called Multilayer Network (RM) and Directed Multilayer Network (RMD). These methods model all channels of the image together, where the networks have intra- and inter-channel connections, similar to the opponent color processing of the human visual system. Experiments in five color texture datasets shows that the RMD proposal overcomes several methods of the literature in general, including convolutional networks and traditional integrative methods. In addition, the proposals have demonstrated high robustness to different color spaces (RGB, LAB, HSV and I1I2I3), while other methods oscillate from dataset to dataset. Moreover it is proposed a new method to characterize multispectral images of many channels, called Directed Network of Angular Similarity (RDSA). In this proposal, each multispectral pixel is considered as a vector of dimensions equivalent to the number of channels of the image and the weight of the edges represents its cosine similarity, pointing to the pixel of greatest absolute value. This method is applied to a set of fluorescence microscopy images of 32 channels in an experiment to identify variations in the leaf structure of the Jacaranda Caroba specimen under different conditions. The RDSA method obtains the highest classification rates in this dataset, with 91.9% with the Leave-one-out cross-validation scheme and 90.5(±1.1)% with 10-folds, against 81.8% and 84.7(±2.2) of the convolutional network VGG16.