Representações visuais têm sido adotadas na exploração de conjuntos de documentos, auxiliando a extração de conhecimento sem que seja necessária a análise individual de milhares de textos. Mapas de documentos, em particular, apresentam documentos individualmente representados espalhados em um espaço visual, refletindo suas relações de similaridade ou conexões. A construção destes mapas de documentos inclui, entre outras tarefas, o posicionamento dos textos e a identificação automática de áreas temáticas. Um desafio é a visualização de conjuntos dinâmicos de documentos. Na visualização de informação, é comum que alterações no conjunto de dados tenham um forte impacto na organização do espaço visual, dificultando a manutenção, por parte do usuário, de um mapa mental que o auxilie na interpretação dos dados apresentados e no acompanhamento das mudanças sofridas pelo conjunto de dados. Esta tese introduz um algoritmo para a construção dinâmica de mapas de documentos, capaz de manter uma disposição coerente à medida que elementos são adicionados ou removidos. O processo, inerentemente incremental e de baixa complexidade, utiliza um espaço bidimensional dividido em células, análogo a um tabuleiro de xadrez. Resultados consistentes foram alcançados em comparação com técnicas não incrementais de projeção de dados multidimensionais, tendo sido a técnica aplicada também em outros domínios, além de conjuntos de documentos. A visualização resultante não está sujeita a problemas de oclusão. A identificação de áreas temáticas é alcançada com técnicas de extração de regras de associação representativas para a identificação automática de tópicos. A combinação da extração de tópicos com a projeção incremental de dados em um processo integrado de mineração visual de textos compõe um espaço visual em que tópicos e áreas de interesse são destacados e atualizados à medida que o conjunto de dados é modificado / Visual representations are often adopted to explore document collections, assisting in knowledge extraction, and avoiding the thorough analysis of thousands of documents. Document maps present individual documents in visual spaces in such a way that their placement reflects similarity relations or connections between them. Building these maps requires, among other tasks, placing each document and identifying interesting areas or subsets. A current challenge is to visualize dynamic data sets. In Information Visualization, adding and removing data elements can strongly impact the underlying visual space. That can prevent a user from preserving a mental map that could assist her/him on understanding the content of a growing collection of documents or tracking changes on the underlying data set. This thesis presents a novel algorithm to create dynamic document maps, capable of maintaining a coherent disposition of elements, even for completely renewed sets. The process is inherently incremental, has low complexity and places elements on a 2D grid, analogous to a chess board. Consistent results were obtained as compared to (non-incremental) multidimensional scaling solutions, even when applied to visualizing domains other than document collections. Moreover, the corresponding visualization is not susceptible to occlusion. To assist users in indentifying interesting subsets, a topic extraction technique based on association rule mining was also developed. Together, they create a visual space where topics and interesting subsets are highlighted and constantly updated as the data set changes

In this thesis, the Twitter social media is data mined for information about the covid-19 outbreak during the month of March, starting from the 3’rd and ending on the 31’st. 100,000 tweets were collected from Harvard’s opensource data and recreated using Hydrate. This data is analyzed further using different Natural Language Processing (NLP) methodologies, such as termfrequency inverse document frequency (TF-IDF), lemmatizing, tokenizing, Latent Semantic Analysis (LSA) and Latent Dirichlet Allocation (LDA). Furthermore, the results of the LSA and LDA algorithms is reduced dimensional data that will be clustered using clustering algorithms HDBSCAN and K-Means for later comparison. Different methodologies are used to determine the optimal parameters for the algorithms. This is all done in the python programing language, as there are libraries for supporting this research, the most important being scikit-learn. The frequent words of each cluster will then be displayed and compared with factual data regarding the outbreak to discover if there are any correlations. The factual data is collected by World Health Organization (WHO) and is then visualized in graphs in ourworldindata.org. Correlations with the results are also looked for in news articles to find any significant moments to see if that affected the top words in the clustered data. The news articles with good timelines used for correlating incidents are that of NBC News and New York Times. The results show no direct correlations with the data reported by WHO, however looking into the timelines reported by news sources some correlation can be seen with the clustered data. Also, the combination of LDA and HDBSCAN yielded the most desireable results in comparison to the other combinations of the dimnension reductions and clustering. This was much due to the use of GridSearchCV on LDA to determine the ideal parameters for the LDA models on each dataset as well as how well HDBSCAN clusters its data in comparison to K-Means.

This thesis is focused on cluster analysis in the field of text mining and its application to real data. The aim of the thesis is to find suitable categories (clusters) in the transcribed calls recorded in the contact center of Česká pojišťovna a.s. by transferring these textual documents into the vector space using basic text mining methods and the implemented clustering algorithms. From the formal point of view, the thesis contains a description of preprocessing and representation of textual data, a description of several common clustering methods, cluster validation, and the application itself.

The main theme of this master's thesis is a description of text mining. This document is specialized to English texts and their automatic data preprocessing. The main part of this thesis analyses various stemming algorithms (Lovins, Porter and Paice/Husk). Stemming is a procedure for automatic conflating semantically related terms together via the use of rule sets. Next part of this thesis describes design of an application for various types of stemming algorithms. Application is based on the Java platform with using of graphic library Swing and MVC architecture. Next chapter contains description of implementation of the application and stemming algorithms. In the last part of this master's thesis experiments with stemming algorithms and comparing the algorithm from viewpoint to the results of classification the text are described.

This thesis pursues an automated extraction of keywords from documents. Its goal is to design and implement an application which will be able to extract an appropriate set of keywords related to the contents of the document. The major requirements for the application are speed and accuracy. That is why the first part of the thesis talks about already developed principles and a detailed classification based on various criteria. The second part is focused on choosing and a thorough functional describing of one of the methods which should have been used for extracting the keywords. The next parts contain a detailed draft of the application and its implementation. Finally, the last chapter is particularly important due to testing the application on a group of text documents and evaluating final results of the extraction process.

This master's thesis presents methods for mining important pieces of information from text. It analyses the problem of terms extraction from large document collection and describes the implementation using C# language and Microsoft SQL Server. The system uses stemming and a number of statistical methods for term extraction. This project also compares used methods and suggests the process of the dictionary derivation.

Process of text data clustering can be used to analysis, navigation and structure large sets of texts or hypertext documents. The basic idea is to group the documents into a set of clusters on the basis of their similarity. The well-known methods of text clustering, however, do not really solve the specific problems of text clustering like high dimensionality of the input data, very large size of the databases and understandability of the cluster description. This work deals with mentioned problems and describes the modern method of text data clustering based on the use of frequent term sets, which tries to solve deficiencies of other clustering methods.

The theme of the work is sentiment analysis, especially in terms of informatics (marginally from a linguistic point of view). The linguistic part discusses the term sentiment and language methods for its analysis, e.g. lemmatization, POS tagging, using the list of stopwords etc. More attention is paid to the structure of the sentiment analyzer which is based on some of the machine learning methods (support vector machines, Naive Bayes and maximum entropy classification). On the basis of the theoretical background, a functional analyzer is projected and implemented. The experiments are focused mainly on comparing the classification methods and on the benefits of using the individual preprocessing methods. The success rate of the constructed classifier reaches up to 84 % in the cross-validation.

Même si l’information est abondante dans le monde, l’information structurée, prête à être utilisée est rare. Ce travail propose l’Extraction d’Information (EI) comme une approche efficace pour la production de l’information structurée, utilisable sur la biologie, en présentant une tâche complète d’EI sur un organisme modèle, Arabidopsis thaliana. Un système d’EI se charge d’extraire les parties de texte les plus significatives et d’identifier leurs relations sémantiques. En collaboration avec des experts biologistes sur la plante A. Thaliana un modèle de connaissance a été conçu. Son objectif est de formaliser la connaissance nécessaire pour bien décrire le domaine du développement de la graine. Ce modèle contient toutes les entités et relations les connectant qui sont essentielles et peut être directement utilisé par des algorithmes. En parallèle ce modèle a été testé et appliqué sur un ensemble d’articles scientifiques du domaine, le corpus nécessaire pour l’entraînement de l’apprentissage automatique. Les experts ont annoté le texte en utilisant les entités et relations du modèle. Le modèle et le corpus annoté sont les premiers proposés pour le développement de la graine, et parmi les rares pour A. Thaliana, malgré son importance biologique. Ce modèle réconcilie les besoins d’avoir un modèle assez complexe pour bien décrirele domaine, et d’avoir assez de généralité pour pouvoir utiliser des méthodes d’apprentissage automatique. Une approche d’extraction de relations (AlvisRE) a également été élaborée et développée. Une fois les entités reconnues, l’extracteur de relations cherche à détecter les cas où le texte mentionne une relation entre elles, et identifier précisément de quel type de relation du modèle il s’agit. L’approche AlvisRE est basée sur la similarité textuelle et utilise à la fois des informations lexiques,syntactiques et sémantiques. Dans les expériences réalisées, AlvisRE donne des résultats qui sont équivalents et parfois supérieurs à l’état de l’art. En plus, AlvisRE a l’avantage de la modularité et adaptabilité en utilisant des informations sémantiques produites automatiquement. Ce dernier caractéristique permet d’attendre des performances équivalentes dans d’autres domaines. / While information is abundant in the world, structured, ready-to-use information is rare. Thiswork proposes Information Extraction (IE) as an efficient approach for producing structured,usable information on biology, by presenting a complete IE task on a model biological organism,Arabidopsis thaliana. Information Extraction is the process of extracting meaningful parts of text and identifying their semantic relations.In collaboration with experts on the plant A. Thaliana, a knowledge model was conceived. The goal of this model is providing a formal representation of the knowledge that is necessary to sufficiently describe the domain of grain development. This model contains all the entities and the relations between them which are essential and it can directly be used by algorithms. Inparallel, this model was tested and applied on a set of scientific articles of the domain. These documents constitute the corpus which is needed to train machine learning algorithms. Theexperts annotated the text using the entities and relations of the model. This corpus and this model are the first available for grain development and among very few on A. Thaliana, despite the latter’s importance in biology. This model manages to answer both needs of being complexenough to describe the domain well, and of having enough generalization for machine learning.A relation extraction approach (AlvisRE) was also elaborated and developed. After entityre cognition, the relation extractor tries to detect the cases where the text mentions that twoentities are in a relation, and identify precisely to which type of the model these relations belongto. AlvisRE’s approach is based on textual similarity and it uses all types of information available:lexical, syntactic and semantic. In the tests conducted, AlvisRE had results that are equivalentor sometimes better than the state of the art. Additionally, AlvisRE has the advantage of being modular and adaptive by using semantic information that was produced automatically. This last feature allows me to expect similar performance in other domains.

Der Großteil des geschäftsrelevanten Wissens liegt heute als unstrukturierte Information in Form von Textdaten auf Internetseiten, in Office-Dokumenten oder Foreneinträgen vor. Zur Extraktion und Verwertung dieser unstrukturierten Informationen wurde eine Vielzahl von Text-Mining-Lösungen entwickelt. Viele dieser Systeme wurden in der jüngeren Vergangenheit als Webdienste zugänglich gemacht, um die Verwertung und Integration zu vereinfachen. Die Kombination verschiedener solcher Text-Mining-Dienste zur Lösung konkreter Extraktionsaufgaben erscheint vielversprechend, da so bestehende Stärken ausgenutzt, Schwächen der Systeme minimiert werden können und die Nutzung von Text-Mining-Lösungen vereinfacht werden kann. Die vorliegende Arbeit adressiert die flexible Kombination von Text-Mining-Diensten in einem serviceorientierten System und erweitert den Stand der Technik um gezielte Methoden zur Auswahl der Text-Mining-Dienste, zur Aggregation der Ergebnisse und zur Abbildung der eingesetzten Klassifikationsschemata. Zunächst wird die derzeit existierende Dienstlandschaft analysiert und aufbauend darauf eine Ontologie zur funktionalen Beschreibung der Dienste bereitgestellt, so dass die funktionsgesteuerte Auswahl und Kombination der Text-Mining-Dienste ermöglicht wird. Des Weiteren werden am Beispiel entitätsextrahierender Dienste Algorithmen zur qualitätssteigernden Kombination von Extraktionsergebnissen erarbeitet und umfangreich evaluiert. Die Arbeit wird durch zusätzliche Abbildungs- und Integrationsprozesse ergänzt, die eine Anwendbarkeit auch in heterogenen Dienstlandschaften, bei denen unterschiedliche Klassifikationsschemata zum Einsatz kommen, gewährleisten. Zudem werden Möglichkeiten der Übertragbarkeit auf andere Text-Mining-Methoden erörtert.