A análise automática da similaridade entre imagens depende fortemente de descritores que consigam caracterizar o conteúdo das imagens em dados compactos e discriminativos. Esses dados extraídos e representados em um vetor-de-características tem o objetivo de representar as imagens nos processos de mineração e análise para classificação e/ou recuperação. Neste trabalho foi explorado o uso de dicionários visuais e contexto para representar e recuperar as características locais das imagens utilizando formalismos estendidos com alto poder descritivo. Esta tese apresenta em destaque três novas propostas que contribuem competitivamente com outros trabalhos da literatura no avanço do estado-da-arte, desenvolvendo novas metodologias para a caracterização de imagens e para o processamento de consultas por similaridade. A primeira proposta estende a modelagem Bag-of-Visual-Words, permitindo codificar a interação entre palavras-visuais e suas disposições espaciais na imagem. Para tal fim, três novas abordagem são apresentadas: (i) Weighted Histogram (WE); (ii) Bunch-of-2-grams e (iii) Global Spatial Arrangement (GSA). Cada uma dessas técnicas permitem extrair informações semanticamente complementares, que enriquecem a representação final das imagens descritas em palavras-visuais. A segunda proposta apresenta um novo descritor, chamado de Bag-of-Salience-Points (BoSP), que caracteriza e analisa a dissimilaridade de formas (silhuetas) de objetos explorando seus pontos de saliências. O descritor BoSP se apoia no uso de um dicionário de curvaturas e em histogramas espaciais para representar sucintamente as saliências de um objeto em um único vetor-de-características de tamanho fixo, permitindo recuperar formas usando funções de distâncias computacionalmente rápidas. Por fim, a terceira proposta apresenta um novo modelo de consulta por similaridade, denominada Similarity Based on Dominant Images (SimDIm), baseada no conceito de Imagens Dominantes, que é um conjunto que representa, de uma maneira mais diversificada e reduzida, toda a coleção de imagens da base de dados. Tal conceito permite dar mais eficiência quando se deseja analisar o contexto da coleção, que é o objetivo da proposta. Os experimentos realizados mostram que os métodos propostos contribuem de maneira efetiva para caracterizar e quantificar a similaridade entre imagens por meio de abordagens estendidas baseadas em dicionários visuais e análise contextual, reduzindo a lacuna semântica existente entre a percepção humana e a descrição computacional. / The automatic similarity analysis between images depends heavily on the use of descriptors that should be able to characterize the images\' content in compact and discriminative features. These extracted features are represented by a feature-vector employed to represent the images in the process of mining and analysis for classification and/or retrieval. This work investigated the use of visual dictionaries and context to represent and retrieve the local image features using extended formalism with high descriptive power. This thesis presents three new proposals that contribute in advancing the state-of-the-art by developing new methodologies for characterizing images and for processing similarity queries by content. The first proposal extends the Bag-of-Visual-Words model, by encoding the interaction between the visual words and their spatial arrangements in the image space. For this, three new techniques are presented: (i) Weighted Histogram (WE); (ii) Bunch-of--grams and (iii) Global Spatial Arrangement (GSA). These three techniques allow to extract additional semantically information that enrich the final image representation described in visual-words. The second proposal introduces a new descriptor, called Bag-of-Salience-Points (BoSP), which characterizes and analyzes the dissimilarity of shapes (silhouettes) exploring their salient point. The BoSP descriptor is based on using a dictionary of curvatures and spatial-histograms to represent succinctly the saliences of a shape into a single fixed-length feature-vector, allowing to retrieve shapes using distance functions computationally fast. Finally, the third proposal introduces a new similarity query model, called Similarity based on Dominant Images (SimDIm), based on the concept of dominant images, which is a set of images representing the entire collection of images of the database in a more diversified and reduced manner. This concept allows to efficiently analyze the context of the entire collection, which is the final goal. The experiments showed that the proposed methods effectively contributed to characterize and quantify the similarity between images using extended approaches based on visual dictionaries and contextual analysis, reducing the semantic gap between human perception and computational description.

This thesis deals with description of existing methods of image retrieval. It contains set of methods for image description, coding of global and local descriptor (SIFT, etc.) and describes method of effective searching in multidimensional space (LSH). It continues with proposal and testing of three global descriptors using color histograms, histogram of gradients and the combination of both. The last part deals with similar image retrieval using proposed descriptors and the indexing method LSH and compares the results with the existing method. Product of this work is an experimental application which demonstrates the proposed solution.

The Thesis consists of an introduction and four papers that contribute to the research of image moments and moment invariants. The first two papers focus on rectangular decomposition algorithms that rapidly speed up the moment calculations. The other two papers present a design of new moment invariants. We present a comparative study of cutting edge methods for the decomposition of 2D binary images, including original implementations of all the methods. For 3D binary images, finding the optimal decomposition is an NP-complete problem, hence a polynomial-time heuristic needs to be developed. We propose a sub-optimal algorithm that outperforms other state of the art approximations. Additionally, we propose a new form of blur invariants that are derived by means of projection operators in a Fourier domain, which improves mainly the discrimination power of the features. Furthermore, we propose new moment-based features that are tolerant to additive Gaussian image noise and we show by extensive image retrieval experiments that the proposed features are robust and outperform other commonly used methods.

In a known-item search task (KIS), the goal is to find a previously seen image in a multimedia collection. In this thesis, we discuss two different approaches based on the visual description of the image. In the first one, the user creates a collage of images (using images from an external search engine), based on which we provide the most similar results from the dataset. Our results show that preprocessing the images in the dataset by splitting them into several parts is a better way to work with the spatial information contained in the user input. We compared the approach to a baseline, which does not utilize this spatial information and an approach that alters a layer in a deep neural network. We also present an alternative approach to the KIS task, search by faces. In this approach, we work with the faces extracted from the images. We investigate face representation for the ability to sort the faces based on their similarity. Then we present a structure that allows easy exploration of the set of faces. We provide a demo, implementing all presented techniques.

Ongoing expansion of digital images requires new methods for sorting, browsing, and sear- ching through huge image databases. This is a domain of Content-Based Image Retrieval (CBIR) systems, which are database search engines for images. A user typically submit a query image or series of images and the CBIR system tries to find and to retrieve the most similar images from the database. Optimally, the retrieved images should not be sensitive to circumstances during their acquisition. Unfortunately, the appearance of natural objects and materials is highly illumination and viewpoint dependent. This work focuses on representation and retrieval of homogeneous images, called textu- res, under the circumstances with variable illumination and texture rotation. We propose a novel illumination invariant textural features based on Markovian modelling of spatial tex- ture relations. The texture is modelled by Causal Autoregressive Random field (CAR) or Gaussian Markov Random Field (GMRF) models, which allow a very efficient estimation of its parameters, without the demanding Monte Carlo minimisation. Subsequently, the estimated model parameters are transformed into the new illumination invariants, which represent the texture. We derived that our textural representation is invariant to changes of illumination intensity and...

The thesis is aimed on the digital analysis and processing of micro- scopic image data with a focus on cross-section samples from the artworks which fall into cultural heritage domain. It contributes to solution of two different problems of image processing - image seg- mentation and image retrieval. The performance evaluation of differ- ent image segmentation methods on a data set of cross-section images is carried out in order to study the behavior of individual approaches and to propose guidelines how to choose suitable method for segmen- tation of microscopic images. Moreover, the benefit of segmenta- tion combination approach is studied and several distinct combination schemes are proposed. The evaluation is backed up by a large number of experiments where image segmentation algorithms are assessed by several segmentation quality measures. Applicability of achieved re- sults is shown on image data of different origin. In the second part, content-based image retrieval of cross-section samples is addressed and functional solution is presented. Its implementation is included in Nephele system, an expert system for processing and archiving the material research reports with image processing features, designed and implemented for the cultural heritage application area. 1

Les documents visuels comprenant des images et des vidéos sont en croissance rapide sur Internet et dans nos collections personnelles. Cela nécessite une analyse automatique du contenu visuel qui fait appel à la conception de méthodes intelligentes pour correctement indexer, rechercher et récupérer des images et des vidéos. Cette thèse vise à améliorer la détection automatique des concepts dans les vidéos sur Internet. Nos contributions portent sur des différents niveaux dans le cadre de détection de concept et peuvent être divisés en trois parties principales. La première partie se focalise sur l’amélioration du modèle de représentation des vidéos « Bag-of-Words (BOW) » en proposant un nouveau mécanisme de construction qui utilise des étiquettes de concepts et une autre technique qui ajoute un raffinement à la signature BOW basée sur la distribution de ses éléments. Nous élaborons ensuite des méthodes pour intégrer des entités semblables et dissemblables pour construire des modèles de reconnaissance améliorés dans la deuxième partie. A ce stade-là, nous observons l’information potentielle que les concepts partagent et construisons des modèles pour les méta-concepts dont sont dérivés les résultats spécifiques de concepts. Cela améliore la reconnaissance des concepts qui ont peu d’exemples annotés. Enfin, nous concevons certaines méthodes d'apprentissage semi-supervisé pour bénéficier de la quantité importante de données non étiquetées. Nous proposons des techniques pour améliorer l'algorithme de cotraining avec une sélection optimale des classifieurs utilisés. / Visual material comprising images and videos is growing ever so rapidly over the internet and in our personal collections. This necessitates automatic understanding of the visual content which calls for the conception of intelligent methods to correctly index, search and retrieve images and videos. This thesis aims at improving the automatic detection of concepts in the internet videos by exploring all the available information and putting the most beneficial out of it to good use. Our contributions address various levels of the concept detection framework and can be divided into three main parts. The first part improves the Bag of Words (BOW) video representation model by proposing a novel BOW construction mechanism using concept labels and by including a refinement to the BOW signature based on the distribution of its elements. We then devise methods to incorporate knowledge from similar and dissimilar entities to build improved recognition models in the second part. Here we look at the potential information that the concepts share and build models for meta-concepts from which concept specific results are derived. This improves recognition for concepts lacking labeled examples. Lastly we contrive certain semi-supervised learning methods to get the best of the substantial amount of unlabeled data. We propose techniques to improve the semi-supervised cotraining algorithm with optimal view selection.

Today the process of sorting second-hand clothes and textiles is mostly manual. In this master’s thesis, methods for automating this process as well as improving the manual sorting process have been investigated. The methods explored include the automatic prediction of price and intended usage for second-hand clothes, as well as different types of image retrieval to aid manual sorting. Two models were examined: CLIP, a multi-modal model, and MAE, a self-supervised model. Quantitatively, the results favored CLIP, which outperformed MAE in both image retrieval and prediction. However, MAE may still be useful for some applications in terms of image retrieval as it returns items that look similar, even if they do not necessarily have the same attributes. In contrast, CLIP is better at accurately retrieving garments with as many matching attributes as possible. For price prediction, the best model was CLIP. When fine-tuned on the dataset used, CLIP achieved an F1-Score of 38.08 using three different price categories in the dataset. For predicting the intended usage (either reusing the garment or exporting it to another country) the best model managed to achieve an F1-Score of 59.04.

<p>An on-line quality assessment system in the industry is essential to prevent artifacts and guide manufacturing processes. Some well-developed systems can diagnose problems and help control the output qualities. However, some of the conventional methods are limited in time consumption and cost of expensive human labor. So, more efficient solutions are needed to guide future decisions and improve productivity. This thesis focuses on developing two image-based machine learning systems to accelerate the manufacturing process: one is to benefit nitrate sensor fabrication, and the other is to help image quality control for inkjet printers.</p> <p><br></p> <p>In the first work, we propose a system for predicting the nitrate sensor's performance based on non-contact images. Nitrate sensors are commonly used to reflect the nitrate levels of soil conditions in agriculture. In a roll-to-roll system, for manufacturing thin-film nitrate sensors, varying characteristics of the ion-selective membrane on screen-printed electrodes are inevitable and affect sensor performance. It is essential to monitor the sensor performance in real-time to guarantee the quality of the sensor. We also develop a system for predicting the sensor performance in on-line scenarios and making the neural networks efficiently adapt to the new data.</p> <p><br></p> <p>Streaks are the number one image quality problem in inkjet printers. In the second work, we focus on developing an efficient method to model and predict missing jets, which is the main contributor to streaks. In inkjet printing, the missing jets typically increase over printing time, and the print head needs to be purged frequently to recover missing jets and maintain print quality. We leverage machine learning techniques for developing spatio-temporal models to predict when and where the missing jets are likely to occur. The prediction system helps the inkjet printers make more intelligent decisions during customer jobs. In addition, we propose another system that will automatically identify missing jet patterns from a large-scale database that can be used in a diagnostic system to identify potential failures.</p>

While historically digital heritage libraries were first powered in image mode, they quickly took advantage of OCR technology to index printed collections and consequently improve the scope and performance of the information retrieval services offered to users. But the access to iconographic resources has not progressed in the same way, and the latter remain in the shadows: manual incomplete and heterogeneous indexation, data silos by iconographic genre. Today, however, it would be possible to make better use of these resources, especially by exploiting the enormous volumes of OCR produced during the last two decades, and thus valorize these engravings, drawings, photographs, maps, etc. for their own value but also as an attractive entry point into the collections, supporting discovery and serenpidity from document to document and collection to collection. This article presents an ETL (extract-transform-load) approach to this need, that aims to: Identify and extract iconography wherever it may be found, in image collections but also in printed materials (dailies, magazines, monographies); Transform, harmonize and enrich the image descriptive metadata (in particular with machine learning classification tools); Load it all into a web app dedicated to image retrieval. The approach is pragmatically dual, since it involves leveraging existing digital resources and (virtually) on-the-shelf technologies. / Si historiquement, les bibliothèques numériques patrimoniales furent d’abord alimentées par des images, elles profitèrent rapidement de la technologie OCR pour indexer les collections imprimées afin d’améliorer périmètre et performance du service de recherche d’information offert aux utilisateurs. Mais l’accès aux ressources iconographiques n’a pas connu les mêmes progrès et ces dernières demeurent dans l’ombre : indexation manuelle lacunaire, hétérogène et non viable à grande échelle ; silos documentaires par genre iconographique ; recherche par le contenu (CBIR, content-based image retrieval) encore peu opérationnelle sur les collections patrimoniales. Aujourd’hui, il serait pourtant possible de mieux valoriser ces ressources, en particulier en exploitant les énormes volumes d’OCR produits durant les deux dernières décennies (tant comme descripteur textuel que pour l’identification automatique des illustrations imprimées). Et ainsi mettre en valeur ces gravures, dessins, photographies, cartes, etc. pour leur valeur propre mais aussi comme point d’entrée dans les collections, en favorisant découverte et rebond de document en document, de collection à collection. Cet article décrit une approche ETL (extract-transform-load) appliquée aux images d’une bibliothèque numérique à vocation encyclopédique : identifier et extraire l’iconographie partout où elle se trouve (dans les collections image mais aussi dans les imprimés : presse, revue, monographie) ; transformer, harmoniser et enrichir ses métadonnées descriptives grâce à des techniques d’apprentissage machine – machine learning – pour la classification et l’indexation automatiques ; charger ces données dans une application web dédiée à la recherche iconographique (ou dans d’autres services de la bibliothèque). Approche qualifiée de pragmatique à double titre, puisqu’il s’agit de valoriser des ressources numériques existantes et de mettre à profit des technologies (quasiment) mâtures.