Back to search

University of Patras

Σχεδιασμός και υλοποίηση ενός συστήματος αποκομιδής ορισμένης πληροφορίας από τον παγκόσμιο ιστό, με τη χρήση σημασιολογικών δικτύων λημμάτων

Abstract

dc:description

This dissertation aims at the specification of an algorithmic methodology that will be applied towards the implementation of a web crawler, which will operate upon thematic criteria. Such crawlers are widely known as topical focused web crawlers. To realize our objective, the utilization of a web page thematic classification system (either existing or newly developed one) is imperative. In the course of our study, we designed and implemented a novel thematic classifier that makes extensive use of the semantic data encoded in WordNet semantic network and such decision was taken with the aspiration of tackling effectively sense ambiguity phenomena that degrade the performance of available classifiers. The suitability of WordNet towards resolving semantic ambiguity has been previously proven but never examined in a focused web crawling application and has never been exploited for the Greek language. Therefore, our thematic classifier and consequently our focused crawler that integrates it are innovative in the way in which they perform word sense disambiguation for achieving the effective detection of the web page topics (themes).In a broad sense, a web crawler is a program that based on a seed list of URLs it downloads the contents of the web pages it comes across and continues following their internal links with the utmost objective of fetching as much as web data as possible (depending on available resources, network capacity, etc.). Given that the web data grows at exponential rates, it is practically impossible to download all the web sources at any given time. One way to tackle such difficulty is to implement and employ topical focused crawlers that aim at downloading content of specific topics (potentially of interest to the user) rather than waste resources trying to download every single data source that is available on the web. Topically focused crawlers are extensively used for building topical focused indices, each of which can serve specialized user search requests, therefore dealing partially with the information overload problem.To carry out our work, we have extensively reviewed existing approaches with respect to topically focused crawling techniques upon which we relied for defining our own focused crawling methodology, which resulted into the implementation of a topical focused crawler that incorporates the following innovate features: it is tailored to operate on the Greek web, it disambiguates the web pages in order to uncover their topic and it incorporates numerous features, such as a named entities recognizer, a URL keyword extractor, personalization techniques, etc., in order to maximize its performance.To demonstrate the effectiveness of our method, we have applied our topical focused crawler on several datasets and experimentally evaluated the following issues: the efficiency of the sense resolution algorithm incorporated into our crawler, the performance of the topical classifier that the crawler consults prior to making a final decision as to whether a page should be downloaded as topically relevant to a subject of interest or not, the suitability of the URL keyword extractor module for judging the subject of a web page based entirely on the analysis of its URL, the usefulness of the named entities recognizer in organizing pages whose semantics are poorly represented within the contents of general-purpose semantic resources (including WordNet semantic network).Experimental results confirm the contribution of our topically focused crawler in downloading web content of specific interest and show that all the methods and techniques that we have successfully integrated into the crawler can interoperate with its other in a manner that improves the crawling performance while allowing for flexibility in the downloading process at the same time. Last but not least, experimental results showcase that our crawling methodology is equally effective for both English and Greek and we believe that it can be fruitfully applied to other natural languages provided that there the respective semantic resources and tools are available for analyzing their textual data.

Degree

thesis:*
Grantor dc:publisher
University of Patras
Year dc:date
2012

Author and committee

dc:creator, dc:contributor.*
Authors dc:creator
  • Kozanidis, Lefteris
  • Κοζανίδης, Ελευθέριος

Subjects

dc:subject × 17

Rights

Language dc:language
gre

Identifiers

dc:identifier.*
Identifier
10.12681/eadd/35267

Chain of custody

source
Harvested from
Greek National Archive of PhD Theses
Base URL
phdtheses.ekt.gr/eadd_oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Kozanidis, Lefteris; Κοζανίδης, Ελευθέριος. Σχεδιασμός και υλοποίηση ενός συστήματος αποκομιδής ορισμένης πληροφορίας από τον παγκόσμιο ιστό, με τη χρήση σημασιολογικών δικτύων λημμάτων. University of Patras, 2012. http://hdl.handle.net/10442/hedi/35267