Back to search

University of Thessaly (UTH)

Engineering search algorithms for web data

Abstract

dc:description

The massive growth of the information produced and disseminated through the Worldwide Web (WWW) has rendered Information Retrieval (IR) one of the most important and challenging research fields in modern computer science. As hundreds of Gigabytes are being published on the Web in a daily basis and billions of users require access to this huge amount of data, search engines have to constantly scale up in terms of both efficiency and effectiveness. In this dissertation we present novel engineering algorithms which contribute to the solution of key problems related to the current Web search engines. These algorithms lead to improvements in the query throughput of these systems (that is, the rate at which they serve the incoming queries), and the quality of the results they produce in response to these queries. In particular, we introduce PFBC, an efficient algorithm for organizing and compressing the positional data stored within an inverted index. In the sequel, we expand PFBC with the aim of supporting additional data within an inverted list posting such as the field (or zone) of a document where a word occurs. The new algorithm, namely TZP, exhibits a wide range of advantages against the current state-of-the-art generic integer compression methods. Based on TZP, we introduce BM25TOPF, a probabilistic ranking function which in contrast to the existing probabilistic functions of the Okapi family, takes into consideration the word ordering in the query, and combines term proximity with zone scoring. Furthermore, we examine the essential problems related to vertical searching, that is, searching for information by accounting only a specific portion of the Web. In particular, we study the problem of quantifying the influence flow in Blogosphere by taking into consideration the particular features which characterize Blogosphere such as the rapid blog post production and the temporal instability of this environment. We propose three such metrics: MEIBI, MEIBIX and the BP/BI-index. In the sequel, we examine how the proposed models for quantifying the bloggers’ influence can be employed by a vertical blog search engine to improve the quality of the generated results. Another vertical search system which gained our attention is academic search engines. In this dissertation we conducted a three-way research; The first part includes proposal of new scientometrics that is, metrics which measure the quality of the work of a scientist. We introduce the f-index, a novel metric which embodies coterminal citations and presents them as a generalization of self-citations and of co-citation. In addition, we introduce the topic-sensitive extensions, special versions of the most important scientometrics which attempt to evaluate the work of a scientist in only one particular research field. In the sequel, we discuss four strategies for computing these metrics in large-scale datasets by using a special-purpose algorithm parallelization framework (Hadoop/MapReduce). Finally, our last contribution regards a supervised machine-learning algorithm for classifying research papers. The results of all these three parts of our research can be utilized by all the current academic search engines and digital libraries to enhance their functionality. The final contribution of this dissertation concerns the problem of rank aggregation, or rank fusion. Here we present a family of algorithms which provide an effective manner for combining and re-ranking the results coming from multiple search engines. The new algorithms, QuadRank and the KE family take into consideration both statistical data (i.e. the individual rankings of each item and the number of its appearances) and document-related information (i.e. zone weighting, URL, etc.). All these algorithms have been implemented within QuadSearch, a prototype metasearch engine which we have developed as a testbed for evaluating new rank aggregation methods and generic solutions related to the wider problem of metasearching.

Degree

thesis:*
Grantor dc:publisher
University of Thessaly (UTH)
Year dc:date
2013

Author and committee

dc:creator, dc:contributor.*
Authors dc:creator
  • Akritidis, Leonidas
  • Ακριτίδης, Λεωνίδας

Subjects

dc:subject × 16

Rights

Language dc:language
eng

Identifiers

dc:identifier.*
Identifier
10.12681/eadd/29067

Chain of custody

source
Harvested from
Greek National Archive of PhD Theses
Base URL
phdtheses.ekt.gr/eadd_oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Akritidis, Leonidas; Ακριτίδης, Λεωνίδας. Engineering search algorithms for web data. University of Thessaly (UTH), 2013. http://hdl.handle.net/10442/hedi/29067