Back to search

Monterey, California. Naval Postgraduate School

Improving automated lexical and discourse analysis of online chat dialog

Abstract

dc:description.abstract

One of the goals of natural language processing (NLP) systems is determining the meaning of what is being transmitted. Although much work has been accomplished in traditional written and spoken language domains, little has been performed in the newer computer-mediated communication domain enabled by the Internet, to include text-based chat. This is due in part to the fact that there are no annotated chat corpora available to the broader research community. The purpose of our research is to build a chat corpus, initially tagged with lexical and discourse information. Such a corpus could be used to develop stochastic NLP applications that perform tasks such as conversation thread topic detection, author profiling, entity identification, and social network analysis. During the course of our research, we preserved 477,835 chat posts and associated user profiles in an XML format for future investigation. We privacy-masked 10,567 of those posts and part-of-speech tagged a total of 45,068 tokens. Using the Penn Treebank and annotated chat data, we achieved part-ofspeech tagging accuracy of 90.8%. We also annotated each of the privacy-masked corpus's 10,567 posts with a chat dialog act. Using a neural network with 23 input features, we achieved 83.2% dialog act classification accuracy.

Degree

thesis:*
Name thesis:degree_name
M.S.
Level thesis:degree_level
Masters
Discipline thesis:degree_discipline
Computer Science
Grantor dc:publisher
Monterey, California. Naval Postgraduate School
Year dc:date.issued
2007

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Forsyth, Eric N.
Advisor dc:contributor.advisor
  • Martell, Craig H.
Committee member dc:contributor.committeemember
  • squire, Kevin M.

Subjects

dc:subject × 1

Rights

dc:rights
Statement dc:rights
  • Approved for public release; distribution is unlimited.

Identifiers

dc:identifier.*
OCLC
176637337

Chain of custody

source
Harvested from
Naval Postgraduate School
Base URL
calhoun.nps.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Forsyth, Eric N.. Improving automated lexical and discourse analysis of online chat dialog. Masters thesis, Monterey, California. Naval Postgraduate School, 2007. https://hdl.handle.net/10945/3281