Back to search

University of North Texas

Keywords in the mist: Automated keyword extraction for very large documents and back of the book indexing.

Abstract

dc:description

This research addresses the problem of automatic keyphrase extraction from large documents and back of the book indexing. The potential benefits of automating this process are far reaching, from improving information retrieval in digital libraries, to saving countless man-hours by helping professional indexers creating back of the book indexes. The dissertation introduces a new methodology to evaluate automated systems, which allows for a detailed, comparative analysis of several techniques for keyphrase extraction. We introduce and evaluate both supervised and unsupervised techniques, designed to balance the resource requirements of an automated system and the best achievable performance. Additionally, a number of novel features are proposed, including a statistical informativeness measure based on chi statistics; an encyclopedic feature that taps into the vast knowledge base of Wikipedia to establish the likelihood of a phrase referring to an informative concept; and a linguistic feature based on sophisticated semantic analysis of the text using current theories of discourse comprehension. The resulting keyphrase extraction system is shown to outperform the current state of the art in supervised keyphrase extraction by a large margin. Moreover, a fully automated back of the book indexing system based on the keyphrase extraction system was shown to lead to back of the book indexes closely resembling those created by human experts.

Degree

thesis:*
Grantor dc:publisher
University of North Texas
Year dc:date
2008

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Csomai, Andras
Contributors dc:contributor
  • Mihalcea, Rada, 1974-
  • Chen, Jiangping
  • Tarau, Paul
  • Pasca, Marius

Subjects

dc:subject × 6

Rights

dc:rights
Statement dc:rights
  • Public
  • Copyright
  • Csomai, Andras
  • Copyright is held by the author, unless otherwise noted. All rights reserved.
Language dc:language
English

Identifiers

dc:identifier.*
Identifier
oclc: 262615405
https://digital.library.unt.edu/ark:/67531/metadc6118/
ark: ark:/67531/metadc6118

Chain of custody

source
Harvested from
University of North Texas
Base URL
digital.library.unt.edu/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Csomai, Andras. Keywords in the mist: Automated keyword extraction for very large documents and back of the book indexing.. University of North Texas, 2008. https://doi.org/10.12794/metadc6118