National Technical University of Athens (NTUA)
Ευφυείς τεχνικές σημασιολογικής ανάλυσης και αναζήτησης κειμένου
Abstract
dc:descriptionThe rapid proliferation of digital text documents during the last years raises the need to create efficient organization and management systems. The textual content of digital documents is a huge source of information that grows every day, assisted by the global growth of the Internet, which contains large quantities of plain text. The size of available documents demands efficient ways to store and represent information using computers, as well as efficient methods for its organization, management, search and editing. The contribution of this PhD thesis rests in the fields of efficient and more thorough representation of documents and the efficient analysis of problems related with them (organization, classification etc.)The use of language is particularly complicated and creates various problems in the attempt to represent documents using computers. The meanings and senses that exist latently in a document because of the context, the relations between the words, the implied information (derived from natural or earlier knowledge), metaphors etc. are some of the problems that emerge, which, combined with the computational complexity that is introduced by the large size of documents, have not allowed for a stable and efficient way of representation to be found until today.In this PhD thesis we examine the basic units of document representation (syllables, words, phrases/sentences) and specifically the Vector Space Model (VSM) which is widely used for document representation based on words. Words are the main unit of document representation and, despite the drawbacks (large search space, breaking of multi-words etc.) introduced, they still are the base of most models. It is no random fact that World Wide Web search engines utilize keywords. Based on the idea that documents consist of words and therefore, every similarity or relation between documents can be reduced to the determination of the relation between the words that consist them and given the observation that searching with keywords is the main way of searching, this PhD thesis proposes a method of determining the semantic relation between words. The main goal is to find a scalar measure that quantifies any relation (synonym, antonym, hypernym etc.) between any two words. For this reason, the method utilizes the information provided by the hierarchical dictionary WordNet as well as the lexico-syntactic patterns extracted from the search results returned by a search engine using those words.Recognizing the weaknesses of a representation model based on words as well as the limitations that are bound by the scalar measurement of word relatedness, the next step in this PhD thesis is the introduction of a new representation model, which is not based on document words, but rather includes semantics in the representation, based on its named entities-concepts (which can contain more than one word). Wikipedia is utilized for this purpose, since its increasing size and rich structure (hierarchical organization, full well-written sentences, informative article headers etc.) provide many capabilities to enhance the document semantics with characteristics created by attributes derived from Wikipedia. The presented model leads to representations not only richer, but also more compressed ones, compared to the VSM model.The introduction of a better representation mode allows the PhD thesis to examine not only the relations between the words-senses of a document, but also the topics appearing in large document collections through the proposal of two document clustering techniques (according to their content). In both techniques the Wikipedia based representation model plays an important role utilizing the constructed features. The first technique is based on the most important concepts of the document collection examined and creates fast and efficiently a hierarchical tree structure of the groups that documents are clustered to, according to their content. The depth and width of the tree structure is fully controlled by the user. The second technique utilizes Self Organizing Maps (SOM) as a tool for document clustering. Self Organizing Map's initialization and initial neuron structure (grid) is modified to the original SOM algorithm (utilizing the Wikipedia based representation model) and training is carried out in three steps in accelerated time compared to the original process. At the end of training, user has the possibility to hierarchically organize similar neurons of the map.The actual contribution of this PhD thesis is summarized in the possibility of using intelligent techniques and utilizing various source of knowledge in order to improve aspects or solve problems related to the efficient representation and analysis of the growing in size documents. In order to derive useful conclusions, at every stage of the research and for each proposed method the results of appropriately designed and performed experiments and comparisons are provided, which do not focus only on the overall evaluation of the methodologies, but, in parallel, intend to justify the particular choices and to prove their merits.
Degree
thesis:*- Grantor dc:publisher
- National Technical University of Athens (NTUA)
- Year dc:date
- 2012
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Spanakis, Gerasimos
- Σπανάκης, Γεράσιμος
Subjects
dc:subject × 12Rights
- Language dc:language
- gre
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/28179