With the advent research of Document Image Analysis and Recognition (DIAR), an important line of research is explored on indexing and retrieval of graphics rich docu- ments. It aims at nding relevant documents relying on segmentation and recognition of text and graphics components underlying in non-standard layout where commercial OCRs can not be applied due to complexity. This thesis is focused towards text infor- mation extraction approaches in graphical documents and retrieval of such documents using text information. Automatic text recognition in graphical documents (map, engineering drawing, etc.) involves many challenges because text characters are usually printed in multi- oriented and multi-scale way along with di erent graphical objects. Text characters are used to annotate the graphical curve lines and hence, many times they follow curvi-linear paths too. For OCR of such documents, individual text lines and their corresponding words/characters need to be extracted. For recognition of multi-font, multi-scale and multi-oriented characters, we have proposed a feature descriptor for character shape using angular information from con- tour pixels to take care of the invariance nature. To improve the e ciency of OCR, an approach towards the segmentation of multi-oriented touching strings into individual characters is also discussed. Convex hull based background information is used to segment a touching string into possible primitive segments and later these primitive segments are merged to get optimum segmentation using dynamic programming. To overcome the touching/overlapping problem of text with graphical lines, a character spotting approach using SIFT and skeleton information is included. Afterwards, we propose a novel method to extract individual curvi-linear text lines using the fore- ground and background information of the characters of the text and a water reservoir concept is used to utilize the background information. We have also formulated the methodologies for graphical document retrieval ap- plications using query words and seals. The retrieval approaches are performed using recognition results of individual components in the document. Given a query text, the system extracts positional knowledge from the query word and uses the same to generate hypothetical locations in the document. Indexing of documents is also per- formed based on automatic detection of seals from documents containing cluttered background. A seal is characterized by scale and rotation invariant spatial feature descriptors computed from labelled text characters and a concept based on the Generalized Hough Transform is used to locate the seal in documents. Keywords: Document Image Analysis, Graphics Recognition, Dynamic Pro- gramming, Generalized Hough Transform, Character Recognition, Touching Charac- ter Segmentation, Text/Graphics Separation, Curve-Line Separation, Word Retrieval, Seal Detection and Recognition.

No description available.

Architectural floor plans are scale-accurate 2D drawings of one level of a building, seen from above, which convey structural and semantic information related to rooms, walls, symbols, textual data, etc. They consist of lines, curves, symbols, and textual markings, showing the relationships between rooms and all physical features, required for the proper construction or renovation of the building. First, this thesis provides a thorough study of state-of-the-art on symbol spotting methods for architectural drawings, an application domain providing the document image analysis and graphic recognition communities with an interesting set of challenges linked to the sheer complexity and density of embedded information, that have yet to be resolved. Second, we propose a hybrid method that capitalizes on strengths of both vector-based and pixel-based symbol spotting techniques. In the description phase, the salient geometric constituents of a symbol are extracted by a variety of vectorization techniques, including a proposed voting-based algorithm for finding partial ellipses. This enables us to better handle local shape irregularities and boundary discontinuities, as well as partial occlusion and overlap. In the matching phase, the spatial relationship between the geometric primitives is encoded via a primitive-aware proximity graph. A statistical approach is then used to rapidly yield a coarse localization of symbols within the plan. Localization is further refined with a pixel-based step implementing a modified cross-correlation function. Experimental results on the public SESYD synthetic dataset and real-world images demonstrate that our approach clearly outperforms other popular symbol spotting approaches. Traditional on-the-fly symbol spotting methods are unable to address the semantic challenge of graphical notation variability, i.e. low intra-class symbol similarity, an issue that is particularly important in architectural floor plan analysis. The presence of occlusion and clutter, characteristic of real-world plans, along with a varying graphical symbol complexity from almost trivial to highly complex, also pose challenges to existing spotting methods. Third, we address all the above issues by leveraging recent advances in deep learning-based neural networks and adapting an object detection framework based on the YOLO (You Only Look Once) architecture. We propose a training strategy based on tiles, avoiding many issues particular to deep learning-based object detection networks related to the relatively small size of symbols compared to entire floor plans, aspect ratios, and data augmentation. Experimental results demonstrate that our method successfully detects architectural symbols with low intra-class similarity and of variable graphical complexity, even in the presence of heavy occlusion and clutter. / Graduate

Dans l’état de l’art actuel, la reconnaissance de symboles signifie généralement la reconnaissance des symboles isolés. Cependant, ces méthodes de reconnaissance de symboles isolés ne sont pas toujours adaptés pour résoudre les problèmes du monde réel. Dans le cas des documents composites qui contiennent des éléments textuels et graphiques, on doit être capable d’extraire et de formaliser les liens qui existent entre les images et le texte environnant, afin d’exploiter les informations incorporées dans ces documents.Liés à ce contexte, nous avons d’abord introduit une méthode de reconnaissance graphique basée sur la programmation dynamique et la mise en correspondance de caractéristiques issues de la transformée de Radon. Cette méthode permet d’exploiter la propriété de cette transformée pour inclure à la fois le contour et la structure interne des formes sans utiliser de techniques de compression de la représentation du motif dans un seul vecteur et qui pourrait passer à côté d’informations importantes. La méthode surpasse en performances les descripteurs de forme de l’état de l’art, mais reste principalement adapté pour la reconnaissance de symboles isolés seulement. Nous l’avons donc intégrée dans une approche complètement nouvelle pour la reconnaissance de symboles basé sur la description spatio-structurelle d’un «vocabulaire» de primitives visuelles extraites. La méthode est basée sur les relations spatiales entre des paires de types étiquetés de ce vocabulaire (dont certains peuvent être caractérisés avec le descripteur mentionné précédemment), qui sont ensuite utilisées comme base pour construire un graphe relationnel attribué (ARG) qui décrit des symboles. Grâce à notre étiquetage des types d’attribut, nous évitons le problème classique NP-difficile d’appariement de graphes. Nous effectuons une comparaison exhaustive avec d’autres modèles de relations spatiales ainsi qu’avec l’état de l’art des approches pour la reconnaissance des graphismes afin de prouver que notre approche combine efficacement les descripteurs statistiques structurels et globaux et les surpasse de manière significative.Dans la dernière partie de cette thèse, nous présentons une approche de type sac de caractéristiques utilisant les relations spatiales, où chaque paire possible primitives visuelles est indexée par sa configuration topologique et les types visuels de ses composants. Ceci fournit un moyen de récupérer les symboles isolés ainsi que d’importantes parties connues de symboles en appliquant soit un symbole isolée comme une requête soit une collection de relations entre les primitives visuelles. Finalement, ceci ouvre des perspectives vers des processus de reconnaissance de symboles fondés sur le langage naturel / In the current state-of-the-art, symbol recognition usually means recognising isolated symbols. However, isolated symbol recognition methods are not always suitable for solving real-world problems. In case of composite documents that contain textual and graphical elements, one needs to be able to extract and formalise the links that exist between the images and the surrounding text, in order to exploit the information embedded in those documents.Related to this context, we first introduce a method for graphics recognition based on dynamic programming matching of the Radon features. This method allows to exploit the Radon Transform property to include both boundary and internal structure of shapes without compressing the pattern representation into a single vector that may miss information. The method outperforms all major set of state-of-the-art of shape descriptors but remains mainly suited for isolated symbol recognition only. We therefore integrate it in a completely new approach for symbol recognition based on the spatio-structural description of a ‘vocabulary’ of extracted visual primitives. The method is based on spatial relations between pairs of labelled vocabulary types (some of which can be characterised with the previously mentioned descriptor), which are further used as a basis for building an attributed relational graph (ARG) to describe symbols. Thanks to our labelling of attribute types, we avoid the general NP-hard graph matching problem. We provide a comprehensive comparison with other spatial relation models as well as state-of-the-art approaches for graphics recognition and prove that our approach effectively combines structural and statistical descriptors together and outperforms them significantly.In the final part of this thesis, we present a Bag-Of-Features (BOFs) approach using spatial relations where every possible pair of individual visual primitives is indexed by its topological configuration and the visual type of its components. This provides a way to retrieve isolated symbols as well as significant known parts of symbols by applying either an isolated symbol as a query or a collection of relations between the important visual primitives. Eventually, it opens perspectives towards natural language based symbol recognition process

Les travaux présentés dans ce mémoire de thèse abordent sous différents angles très intéressants, un sujet vaste et ambitieux : l’interprétation de plans cadastraux couleurs.Dans ce contexte, notre approche se trouve à la confluence de différentes thématiques de recherche telles que le traitement du signal et des images, la reconnaissance de formes, l’intelligence artificielle et l’ingénierie des connaissances. En effet, si ces domaines scientifiques diffèrent dans leurs fondements, ils sont complémentaires et leurs apports respectifs sont indispensables pour la conception d’un système d’interprétation. Le centre du travail est le traitement automatique de documents cadastraux du 19e siècle. La problématique est traitée dans le cadre d'un projet réunissant des historiens, des géomaticiens et des informaticiens. D'une part nous avons considéré le problème sous un angle systémique, s'intéressant à toutes les étapes de la chaîne de traitements mais aussi avec un souci évident de développer des méthodologies applicables dans d'autres contextes. Les documents cadastraux ont été l'objet de nombreuses études mais nous avons su faire preuve d'une originalité certaine, mettant l'accent sur l'interprétation des documents et basant notre étude sur des modèles à base de graphes. Des propositions de traitements appropriés et de méthodologies ont été formulées. Le souci de comblé le gap sémantique entre l’image et l’interprétation a reçu dans le cas des plans cadastraux étudiés une réponse. / This thesis tackles the problem of technical document interpretationapplied to ancient and colored cadastral maps. This subject is on the crossroadof different fields like signal or image processing, pattern recognition, artificial intelligence,man-machine interaction and knowledge engineering. Indeed, each of thesedifferent fields can contribute to build a reliable and efficient document interpretationdevice. This thesis points out the necessities and importance of dedicatedservices oriented to historical documents and a related project named ALPAGE.Subsequently, the main focus of this work: Content-Based Map Retrieval within anancient collection of color cadastral maps is introduced.