Return to search

Levenshtein distance for information extraction in databases and for natural language processing.

While performing information extraction or natural language processing tasks, one usually encounters problems when working with data or texts containing noise, typing mistakes or other different kinds of errors. In this thesis we investigate the use of modified Levenshtein edit distances to deal with these problems in two specific tasks. The first one is the record linkage in databases where distinct records can be representing the same entity. For this task we used and extended the WEKA API for Machine Learning and we were able to show that a modified Levenshtein distance provides good precision and recall results in the detection of records representing the same entities. The second task is the search and annotation of occurrences of specified words in texts written in natural language. Our main result in this task was the implementation of an approximate Gazetteer for GATE, the General Architecture for Text Engineering.

Identiferoai:union.ndltd.org:IBICT/oai:agregador.ibict.br.BDTD_ITA:oai:ita.br:529
Date21 December 2007
CreatorsBruno Woltzenlogel Paleo
ContributorsCarlos Henrique Costa Ribeiro
PublisherInstituto Tecnológico de Aeronáutica
Source SetsIBICT Brazilian ETDs
LanguageEnglish
Detected LanguageEnglish
Typeinfo:eu-repo/semantics/publishedVersion, info:eu-repo/semantics/masterThesis
Formatapplication/pdf
Sourcereponame:Biblioteca Digital de Teses e Dissertações do ITA, instname:Instituto Tecnológico de Aeronáutica, instacron:ITA
Rightsinfo:eu-repo/semantics/openAccess

Page generated in 0.0017 seconds