National Technical University of Athens (NTUA)
Διαχείριση πληροφορίας και κατηγοριοποίηση διεπαφών αναζήτησης στον παγκόσμιο ιστό με αλγόριθμους εμπνευσμένους από τη φύση και τεχνικές μηχανικής μάθησης για μεγάλα δεδομένα
Abstract
dc:descriptionA large portion of the content residing on the Web, though publicly available, is inaccessible by the traditional general purpose search engines, since it is only generated dynamically as a response to a query submission to a search interface. The identification of such interfaces is the first step towards the automated access to the content of the part of the Web known as Hidden Web. In this dissertation, after an introduction to this research field, a large scale analysis is conducted on a publicly available dataset containing web pages with complex interfaces. The aim of the analysis is to draw useful conclusions on the properties of the interfaces, and the construction of a training dataset for the induction of classification rules that will be used for the automated identification of search interfaces.Classification rule induction is one of the oldest machine learning techniques and it has been successfully applied in a plethora of problems. Its main advantage is the simplicity of the resulting classification model and the ability for a human to interpret it. One of the contributions of this dissertation, is the introduction of a novel distributed classification rule induction approach based on MapReduce. As a first step, the approach transforms any numeric attributes to discrete and then it exhaustively searches the space of possible rules to find the best one according to an evaluation criterion. The resulting rules are used for the functional classification of interfaces.Finally a novel nature inspired algorithm for searching for information on the Web is also presented. The proposed algorithm has the ability to locate relative information units by routing the search in the dynamic environment of the Web. The search is conducted stochastically, by combining techniques based on document similarity and techniques that emulate the communication of real world ants in their foraging process. The proposed algorithm when used in conjunction with the classification rules induced previously can locate, similar search interfaces on the Web in order to facilitate the access to Hidden Web content.
Degree
thesis:*- Grantor dc:publisher
- National Technical University of Athens (NTUA)
- Year dc:date
- 2015
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Kolias, Vasileios
- Κόλιας, Βασίλειος
Subjects
dc:subject × 14- Κρυμμένος παγκόσμιος ιστός
- Επαγωγή κανόνων
- Απεικόνιση - Μείωση
- Αναζήτηση στον παγκόσμιο ιστό
- Αποικία μυρμηγκιών
- Hidden web
- Rule induction
- MapReduce
- Web search
- Ant colony
- Επιστήμες Μηχανικού και Τεχνολογία
- Επιστήμη Ηλεκτρολόγου Μηχανικού, Ηλεκτρονικού Μηχανικού, Μηχανικού Η/Υ
- Engineering and Technology
- Electrical Engineering, Electronic Engineering, Information Engineering
Rights
- Language dc:language
- gre
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/37347