Back to search

Clemson University

Semantic Similarity Detection in Natural Language Documents

Abstract

dc:description.abstract

Data leak prevention (DLP) solutions monitor and control data flow. Current techniques find data that matches user defined syntactic patterns. Unfortunately, large classes of DLP relevant data are defined by information semantics, rather than data syntax. Syntax refers to data format, whereas semantics refers to data meaning. The class of social security numbers can be adequately expressed using data syntax, whereas a new industrial process can only be adequately described using information semantics. In this paper, we propose methods for extracting and identifying document semantics using training sets of limited size (tens of documents). The first method is based on singular value decomposition, which uses linear algebra to automatically extract semantic features from documents in the training set. The second method is to infer a hidden Markov model (HMM) expressing relations between the features extracted using the singular value method. This HMM can detect documents containing the intellectual property semantic information. A third method views the English language as a probabilistic context-free grammar (PCFG), and extracts semantic information from individual sentences in order to detect documents containing intellectual property. Test results on 5 document sets show the proposed methods give at least 84% true positive and below 22% false positive rates. Our methods are trained with only positive examples, and have lower false positive rates, compared to Latent Dirichlet Allocation (LDA) and Support Vector Machines (SVM).

Degree

thesis:*
Name thesis:degree_name
Master of Science (MS)
Level thesis:degree_level
Thesis
Discipline thesis:degree_discipline
Computer Engineering

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Zhao, Lianyu
Contributors dc:contributor
  • Brooks, Richard R
  • Wang , Kuang-Ching
  • Hoover , Adam

Subjects

dc:subject × 6

Identifiers

dc:identifier.*
Repository record dc:identifier
https://open.clemson.edu/all_theses/1526

Chain of custody

source
Harvested from
Clemson University
Base URL
open.clemson.edu/do/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Zhao, Lianyu. Semantic Similarity Detection in Natural Language Documents. Thesis thesis, https://open.clemson.edu/all_theses/1526