Back to search

Monterey, CA; Naval Postgraduate School

CLUSTER COMPUTING FOR AUTOMATED NETWORK ANALYSIS AT SCALE

Abstract

dc:description.abstract

Conventional single node packet analyzers are unable to monitor network traffic at scale. In this thesis, elements of the Apache Hadoop ecosystem, including HBase, Spark, and MapReduce, are employed to conduct network traffic analysis on a large collection of network traffic. Limited analysis is conducted directly on packet capture next generation (pcapng) files on the Hadoop Distributed File System (HDFS) using MapReduce. Next, to allow for repeated analysis on the same dataset without reading all source files in their entirety for every calculation, pcapng files are parsed and relevant meta-data is bulk loaded into HBase, a Not Only Structured Query Language (NoSQL) database employing the HDFS for parallelization. This NoSQL database is then accessed via Apache Spark where pertinent data is loaded into DataFrames and additional analysis on the network traffic takes place. This research demonstrates the viability of custom, modular, automated analytics, employing open-source software to enable parallelization, to conduct traffic analysis at scale.

Degree

thesis:*
Name thesis:degree_name
Master of Science in Electrical Engineering
Level thesis:degree_level
Masters
Discipline thesis:degree_discipline
Electrical Engineering
Department dc:contributor.department
Electrical and Computer Engineering (ECE)
Grantor dc:publisher
Monterey, CA; Naval Postgraduate School
Year dc:date.issued
2018

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Brida, Benjamin J.
Advisors dc:contributor.advisor
  • Kragh, Frank E.
  • Scrofani, James W.

Rights

dc:rights
Statement dc:rights
  • This publication is a work of the U.S. Government as defined in Title 17, United States Code, Section 101. Copyright protection is not available for this work in the United States.
  • Approved for public release; distribution is unlimited.

Identifiers

dc:identifier.*
Thesis Thesis Identifier
29276

Chain of custody

source
Harvested from
Naval Postgraduate School
Base URL
calhoun.nps.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Brida, Benjamin J.. CLUSTER COMPUTING FOR AUTOMATED NETWORK ANALYSIS AT SCALE. Masters thesis, Monterey, CA; Naval Postgraduate School, 2018. https://hdl.handle.net/10945/59618