Back to search

North Dakota State University

Analyzing Access Logs Data using Stream Based Architecture

Abstract

dc:description.abstract

Within the past decades, the enterprise-level IT infrastructure in many businesses have grown from a few to thousands of servers, increasing the digital footprints they produce. These digital footprints include access logs that contain information about different events such as activity related to usage patterns, networks and any hostile activity affecting the network. Apache Hadoop has been one of the most standardized frameworks and is used by many Information Technology (IT) companies for analyzing these log files in distributed batch mode using MapReduce programming model. As these access logs include important information related to security and usage patterns, companies are now looking for an architecture that allows analyzing these logs in real time. To overcome the limitations of the MapReduce based architecture of Hadoop, this paper proposes a new and more efficient data processing architecture using Apache Spark, Kafka and other technologies that can handle both real-time and batch-based data.

Degree

thesis:*
Grantor dc:publisher
North Dakota State University
Year dc:date.issued
2018

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Gautam, Nitendra

Subjects

dc:subject × 9

Rights

dc:rights
Statement dc:rights
  • NDSU Policy 190.6.2

Identifiers

dc:identifier.*
Handle dc:identifier.uri
https://hdl.handle.net/10365/28001

Chain of custody

source
Harvested from
North Dakota State University
Base URL
library.ndsu.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Gautam, Nitendra. Analyzing Access Logs Data using Stream Based Architecture. North Dakota State University, 2018. https://hdl.handle.net/10365/28001