Nous proposons un système complet capable de retrouver des symboles dans des documents graphiques sans connaissance a priori. Nous nous basons sur une méthode de description structurelle qui permet de mettre en avant des régions pouvant contenir un symbole. A partir d'un découpage du document en chaînes de points, nous fusionnons successivement les régions entre elles en fonction d'un critère de densité et de convexité permettant la reconstruction de symboles potentiellement intéressant pour l'utilisateur. Un descripteur est ensuite calculé pour chacun de ses symboles, ce qui permet de faire une reconnaissance quand l'utilisateur soumet une requête. Afin de réduire le temps de réponse d'une requête nous avons développé une méthode d'indexation qui se base sur l'algorithme BIRCH en utilisant un descripteur robuste et discriminant. Puis nous montrons comment réduire davantage ce temps de réponse en combinant différentes règles de filtrage basées sur des descripteurs basiques. / A complete system able to find symbols in graphical document without a priori knowledge is proposed here. In a first place, this system is based on a structural method able to put in stress regions that may contain symbols. The document is represented by chain points that will be merged following a defined criteria. These merges allow potential symbols to be reconstructed. A descriptor is then calculated for each potential symbols, and the recognition can take place when the user submit a request. In order to speed up the retrieval, an indexing method based on BIRCH has been proposed by using a robust descriptor. Then we show that by combining filtering rules based on simple descriptors, we can rise the speed of the retrieval.

The aim of this work is to implement a conversion from the scanned math formula to the editable form as a TEX file as an extension of the working OCR system. In this work we closely analyze this problem, its division into several smaller parts, such as math symbol recognition and a recognition of structure of math formulas, and their solutions together with a description of various solutions. We test our implementations using our database of symbols and math formulas. An important part of the work is also a creation of a set of complex applications with a sophisticated graphical user interface, which allow easy accommodation of conversion to the user's needs. During the conversion we work with images, which may contain insignificant noise caused by a scanner of lower quality.

<p>Piping is an essential component in buildings, and its as-built information is critical to facility management tasks. Manually extracting piping information from legacy drawings that are in paper, PDF, or image format is mentally exerting, time-consuming, and error-prone. Symbol recognition and detection are core problems in the computer-based interpretation of piping drawings, and the main technical challenge is to determine robust features that are invariant to scaling, rotation, and translation. This thesis aims to use convolutional neural networks (CNNs) to automatically extract features from raw images, and consequently, to locate and recognize symbols in piping drawings.</p> <p>In this thesis, the Spatial Transformer Network (STN) is applied to improve the performance of a standard CNN model for recognizing piping symbols, and the Faster Region-based Convolutional Neural Network (Faster RCNN) is adopted to exploit its capacity in symbol detection. For experimentation, the synthetic data are generated as follows. Two datasets are generated for symbol recognition and detection, respectively. For recognition, eight types of symbols are synthesized based on the geometric constraints between the primitives. The drawing samples for detection are manually sketched using AutoCAD MEP software and its piping component library, and seven types of symbols are selected from the piping component library. Both sets of samples are augmented with various scales, rotations, and random noises.</p> <p>The experiment for symbol recognition is conducted and the accuracies of the recognition accuracy of the CNN + STN model and the standard CNN model are compared. It is observed that the spatial transformer layer improves the accuracy in classifying piping symbols from 95.39% to 98.26%. For the symbol detection task, the experiment is conducted using a public implementation of Faster RCNN. The mean Average Precision (mAP) is 82.8% when Intersection over Union (IoU) threshold equals to 0.5. Imbalanced data (i.e., imbalanced samples in each class) led to a decrease in the Average Precision in the minority class. Also, the symbol library, the small dataset, and the complex backbone network limit the generality of the model. Future work will focus on the collection of larger set of drawings and the improvement of the network’s geometric invariance.</p>

Cette thèse traite de la restructuration des documents déstructurés de type PDF contenant des éléments graphiques tels que les schémas, les plans et les dessins, dans l’objectif de les restructurer. En nous appuyant sur la méthode KDD (Knowledge Discovery in Database) pour la restructuration des données, nous introduisons la méthode (A)KDD (Antropocentric Knowledge Discovery in Database) que nous avons développé et qui est dérivée de la méthode KDD en ajoutant l’aspect incrémental et l’aspect centré sur l’utilisateur. Nous présentons, en particulier, une technique fondée sur le principe du tri par paquet pour extraire efficacement les symboles graphiques contenus dans un document PDF. Elle est comparée aux résultats de Puglissi sur les chaînes de caractères. Puis, nous formulons l’hypothèse selon laquelle la prise en compte de l’ordre chronologique présent dans les fichiers PDF dans le processus incrémental améliore la restructuration des documents. Nous montrons la validité de cette hypothèse sur un certain nombre d’exemples. Enfin, nous montrons l’efficacité du processus pour identifier les symboles en même temps que les équipotentielles. Le mémoire se conclut en montrant les avancées et les limites de la solution de la méthode (A)KDD et nous proposons des perspectives. / This thesis deals with the restructuring of unstructured PDF documents containing graphical elements such as schematics, plans and drawings, with the aim of restructuring them. Using the KDD (Knowledge Discovery in Database) method for data restructuring, we introduce the (A) KDD (Antropocentric Knowledge Discovery in Database) method that we developed which is derived from the KDD method by adding an incremental aspect and an user-centered approach. We present, in particular, a technique based on on the bucket sort algorithm pattern in order to extract with efficiency graphic symbols contained in a PDF file. It is compared to the results obtained by Puglissi on strings. Then, we formulate the hypothesis:”taking into account the chronological order present in the PDF files in the incremental process improves the restructuring of the documents”. We illustrate the validity of this hypothesis on several examples. Finally, we show the efficiency of the process in the identification of the symbols at the same time as the equipotentials. The thesis concludes by showing the advances and the limits of the solution of the (A) KDD method and we propose some perspectives.

[EN] Mathematical notation is well-known and used all over the world. Humankind has evolved from simple methods representing countings to current well-defined math notation able to account for complex problems. Furthermore, mathematical expressions constitute a universal language in scientific fields, and many information resources containing mathematics have been created during the last decades. However, in order to efficiently access all that information, scientific documents have to be digitized or produced directly in electronic formats. Although most people is able to understand and produce mathematical information, introducing math expressions into electronic devices requires learning specific notations or using editors. Automatic recognition of mathematical expressions aims at filling this gap between the knowledge of a person and the input accepted by computers. This way, printed documents containing math expressions could be automatically digitized, and handwriting could be used for direct input of math notation into electronic devices. This thesis is devoted to develop an approach for mathematical expression recognition. In this document we propose an approach for recognizing any type of mathematical expression (printed or handwritten) based on probabilistic grammars. In order to do so, we develop the formal statistical framework such that derives several probability distributions. Along the document, we deal with the definition and estimation of all these probabilistic sources of information. Finally, we define the parsing algorithm that globally computes the most probable mathematical expression for a given input according to the statistical framework. An important point in this study is to provide objective performance evaluation and report results using public data and standard metrics. We inspected the problems of automatic evaluation in this field and looked for the best solutions. We also report several experiments using public databases and we participated in several international competitions. Furthermore, we have released most of the software developed in this thesis as open source. We also explore some of the applications of mathematical expression recognition. In addition to the direct applications of transcription and digitization, we report two important proposals. First, we developed mucaptcha, a method to tell humans and computers apart by means of math handwriting input, which represents a novel application of math expression recognition. Second, we tackled the problem of layout analysis of structured documents using the statistical framework developed in this thesis, because both are two-dimensional problems that can be modeled with probabilistic grammars. The approach developed in this thesis for mathematical expression recognition has obtained good results at different levels. It has produced several scientific publications in international conferences and journals, and has been awarded in international competitions. / [ES] La notación matemática es bien conocida y se utiliza en todo el mundo. La humanidad ha evolucionado desde simples métodos para representar cuentas hasta la notación formal actual capaz de modelar problemas complejos. Además, las expresiones matemáticas constituyen un idioma universal en el mundo científico, y se han creado muchos recursos que contienen matemáticas durante las últimas décadas. Sin embargo, para acceder de forma eficiente a toda esa información, los documentos científicos han de ser digitalizados o producidos directamente en formatos electrónicos. Aunque la mayoría de personas es capaz de entender y producir información matemática, introducir expresiones matemáticas en dispositivos electrónicos requiere aprender notaciones especiales o usar editores. El reconocimiento automático de expresiones matemáticas tiene como objetivo llenar ese espacio existente entre el conocimiento de una persona y la entrada que aceptan los ordenadores. De este modo, documentos impresos que contienen fórmulas podrían digitalizarse automáticamente, y la escritura se podría utilizar para introducir directamente notación matemática en dispositivos electrónicos. Esta tesis está centrada en desarrollar un método para reconocer expresiones matemáticas. En este documento proponemos un método para reconocer cualquier tipo de fórmula (impresa o manuscrita) basado en gramáticas probabilísticas. Para ello, desarrollamos el marco estadístico formal que deriva varias distribuciones de probabilidad. A lo largo del documento, abordamos la definición y estimación de todas estas fuentes de información probabilística. Finalmente, definimos el algoritmo que, dada cierta entrada, calcula globalmente la expresión matemática más probable de acuerdo al marco estadístico. Un aspecto importante de este trabajo es proporcionar una evaluación objetiva de los resultados y presentarlos usando datos públicos y medidas estándar. Por ello, estudiamos los problemas de la evaluación automática en este campo y buscamos las mejores soluciones. Asimismo, presentamos diversos experimentos usando bases de datos públicas y hemos participado en varias competiciones internacionales. Además, hemos publicado como código abierto la mayoría del software desarrollado en esta tesis. También hemos explorado algunas de las aplicaciones del reconocimiento de expresiones matemáticas. Además de las aplicaciones directas de transcripción y digitalización, presentamos dos propuestas importantes. En primer lugar, desarrollamos mucaptcha, un método para discriminar entre humanos y ordenadores mediante la escritura de expresiones matemáticas, el cual representa una novedosa aplicación del reconocimiento de fórmulas. En segundo lugar, abordamos el problema de detectar y segmentar la estructura de documentos utilizando el marco estadístico formal desarrollado en esta tesis, dado que ambos son problemas bidimensionales que pueden modelarse con gramáticas probabilísticas. El método desarrollado en esta tesis para reconocer expresiones matemáticas ha obtenido buenos resultados a diferentes niveles. Este trabajo ha producido varias publicaciones en conferencias internacionales y revistas, y ha sido premiado en competiciones internacionales. / [CA] La notació matemàtica és ben coneguda i s'utilitza a tot el món. La humanitat ha evolucionat des de simples mètodes per representar comptes fins a la notació formal actual capaç de modelar problemes complexos. A més, les expressions matemàtiques constitueixen un idioma universal al món científic, i s'han creat molts recursos que contenen matemàtiques durant les últimes dècades. No obstant això, per accedir de forma eficient a tota aquesta informació, els documents científics han de ser digitalitzats o produïts directament en formats electrònics. Encara que la majoria de persones és capaç d'entendre i produir informació matemàtica, introduir expressions matemàtiques en dispositius electrònics requereix aprendre notacions especials o usar editors. El reconeixement automàtic d'expressions matemàtiques té per objectiu omplir aquest espai existent entre el coneixement d'una persona i l'entrada que accepten els ordinadors. D'aquesta manera, documents impresos que contenen fórmules podrien digitalitzar-se automàticament, i l'escriptura es podria utilitzar per introduir directament notació matemàtica en dispositius electrònics. Aquesta tesi està centrada en desenvolupar un mètode per reconèixer expressions matemàtiques. En aquest document proposem un mètode per reconèixer qualsevol tipus de fórmula (impresa o manuscrita) basat en gramàtiques probabilístiques. Amb aquesta finalitat, desenvolupem el marc estadístic formal que deriva diverses distribucions de probabilitat. Al llarg del document, abordem la definició i estimació de totes aquestes fonts d'informació probabilística. Finalment, definim l'algorisme que, donada certa entrada, calcula globalment l'expressió matemàtica més probable d'acord al marc estadístic. Un aspecte important d'aquest treball és proporcionar una avaluació objectiva dels resultats i presentar-los usant dades públiques i mesures estàndard. Per això, estudiem els problemes de l'avaluació automàtica en aquest camp i busquem les millors solucions. Així mateix, presentem diversos experiments usant bases de dades públiques i hem participat en diverses competicions internacionals. A més, hem publicat com a codi obert la majoria del software desenvolupat en aquesta tesi. També hem explorat algunes de les aplicacions del reconeixement d'expressions matemàtiques. A més de les aplicacions directes de transcripció i digitalització, presentem dues propostes importants. En primer lloc, desenvolupem mucaptcha, un mètode per discriminar entre humans i ordinadors mitjançant l'escriptura d'expressions matemàtiques, el qual representa una nova aplicació del reconeixement de fórmules. En segon lloc, abordem el problema de detectar i segmentar l'estructura de documents utilitzant el marc estadístic formal desenvolupat en aquesta tesi, donat que ambdós són problemes bidimensionals que poden modelar-se amb gramàtiques probabilístiques. El mètode desenvolupat en aquesta tesi per reconèixer expressions matemàtiques ha obtingut bons resultats a diferents nivells. Aquest treball ha produït diverses publicacions en conferències internacionals i revistes, i ha sigut premiat en competicions internacionals. / Álvaro Muñoz, F. (2015). Mathematical Expression Recognition based on Probabilistic Grammars [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/51665

A severe aphasia following a cerebral vascular accident is characterised by generalised deficits in most speech-language domains. The clinical dilemma remains focused on the extensive verbal speech impairment and in most cases little possibility of regaining verbal speech production. Many individuals living with severe aphasia use augmentative and alternative communication strategies to assist them in getting their communication needs met in their everyday lives. The Blissymbol system is one of the graphic symbol systems that can be used to supplement existing communication and speech strategies of the individual with little or no speech. Although the use of AAC strategies is gaining momentum in its application to severe aphasia, however, there still remain questions on how best to help these individuals learn and retain such strategies. Not only are individuals with severe aphasia faced with a memory task when learning AAC strategies such as Blissymbols, additional complexity to AAC interventions is derived from clinical presentation of severe aphasia. The presence of extensive damage to the neural centers responsible for linguistic processing and semantic retrieval makes learning of new AAC strategies all the more complicated. Research studies have looked at whether individuals with severe aphasia can learn to recognise and retain Blissymbols. Although these studies have successfully shown that individuals with severe aphasia can learn Blissymbols, there is little information available regarding how these symbols can best be taught and retained over time individuals with severe aphasia. Recently the research that has looked at the application of symbol learning with persons presenting with severe aphasia using computer technology and sophisticated application software has highlighted the importance of therapeutic methods that may enhance the learning of such software. This study looks at the application of the self-generation effect as a viable method for enhancing the recognition of Blissymbols in persons presenting with severe aphasia. The self-generation effect is the finding of superior retention and recall for stimuli constructed or generated by an individual. Memory for stimuli such as words, numbers and pictures were found to be enhanced by the extent to which the individual was involved in its construction. Using a 2X2X3 factorial design, this study compared the recognition levels for Blissymbols taught using two treatment approaches which was the self-generation condition and the non self-generation condition. During three experimental sessions which included two withdrawal periods participants were taught using both treatments to recognise a set of Blissymbols. Recognition levels were tested during recognition probes and retention probes. The results from these probes were compared in order to identify which treatment produced superior recognition levels. The data analysis conducted showed that although there was no recognition advantage for the self-generation effect seen during the three recognition probes some advantage for the self-generation effect was seen during the retention probes conducted. The self-generation effect began to emerge by the final retention probe following a withdrawal period of seven days. The self-generation treatment showed better retention of symbol recognition over time. Previous studies have shown that the self-generation effect failed to emerge with stimuli that were new or unfamiliar. This trend was also seen in this study. The results provide support for a semantic-association theory for the self-generation effect. / Thesis (PhD)--University of Pretoria, 2010. / Centre for Augmentative and Alternative Communication (CAAC) / unrestricted

Dans cette thèse, nous nous concentrons sur comment les représentations parcimonieuses peuvent aider à augmenter les performances pour réduire le bruit, extraire des régions de texte, reconnaissance des formes et localiser des symboles dans des documents graphiques. Pour ce faire, tout d'abord, nous donnons une synthèse des représentations parcimonieuses et ses applications en traitement d'images. Ensuite, nous présentons notre motivation pour l'utilisation de dictionnaires d'apprentissage avec des algorithmes efficaces pour les construire. Après avoir décrit l'idée générale des représentations parcimonieuses et du dictionnaire d'apprentissage, nous présentons nos contributions dans le domaine de la reconnaissance de symboles et du traitement des documents en les comparants aux travaux de l'état de l'art. Ces contributions s'emploient à répondre aux questions suivantes: La première question est comment nous pouvons supprimer le bruit des images où il n'existe aucune hypothèse sur le modèle de bruit sous-jacent à ces images ? La deuxième question est comment les représentations parcimonieuses sur le dictionnaire d'apprentissage peuvent être adaptées pour séparer le texte du graphique dans des documents? La troisième question est comment nous pouvons appliquer la représentation parcimonieuse à reconnaissance de symboles? Nous complétons cette thèse en proposant une approche de localisation de symboles dans les documents graphiques qui utilise les représentations parcimonieuses pour coder un vocabulaire visuel / In this thesis, we focus on how sparse representations can help to increase the performance of noise removal, text region extraction, pattern recognition and spotting symbols in graphical documents. To do that, first of all, we give a survey of sparse representations and its applications in image processing. Then, we present the motivation of building learning dictionary and efficient algorithms for constructing a learning dictionary. After describing the general idea of sparse representations and learned dictionary, we bring some contributions in the field of symbol recognition and document processing that achieve better performances compared to the state-of-the-art. These contributions begin by finding the answers to the following questions. The first question is how we can remove the noise of a document when we have no assumptions about the model of noise found in these images? The second question is how sparse representations over learned dictionary can separate the text/graphic parts in the graphical document? The third question is how we can apply the sparse representation for symbol recognition? We complete this thesis by proposing an approach of spotting symbols that use sparse representations for the coding of a visual vocabulary