<p>The
detection of anomalies in real-world networks is applicable in different
domains; the application includes, but is not limited to, credit card fraud
detection, malware identification and classification, cancer detection from
diagnostic reports, abnormal traffic detection, identification of fake media
posts, and the like. Many ongoing and current researches are providing tools
for analyzing labeled and unlabeled data; however, the challenges of finding
anomalies and patterns in large-scale datasets still exist because of rapid
changes in the threat landscape. </p><p>In this study, I implemented a
novel and robust solution that combines data science and cybersecurity to solve
complex network security problems. I used Long Short-Term Memory (LSTM) model, Louvain
algorithm, and PageRank algorithm to identify and group anomalies in large-scale
real-world networks. The network has billions of packets. The developed model
used different visualization techniques to provide further insight into how the
anomalies in the network are related. </p><p>Mean absolute error (MAE) and root mean square error (RMSE) was used to validate the anomaly detection models, the
results obtained for both are 5.1813e-04
and 1e-03 respectively. The low loss from the training
phase confirmed the low RMSE at loss: 5.1812e-04, mean absolute error:
5.1813e-04, validation loss: 3.9858e-04, validation mean absolute error:
3.9858e-04. The result from the community detection
shows an overall modularity value of 0.914 which is proof of the existence of
very strong communities among the anomalies. The largest sub-community of the
anomalies connects 10.42% of the total nodes of the anomalies. </p><p>The broader aim and impact of this study was to provide
sophisticated, AI-assisted countermeasures to cyber-threats in large-scale
networks. To close the existing gaps created by the shortage of skilled and
experienced cybersecurity specialists and analysts in the cybersecurity field,
solutions based on out-of-the-box thinking are inevitable; this research was aimed
at yielding one of such solutions. It was built to detect specific and
collaborating threat actors in large networks and to help speed up how the
activities of anomalies in any given large-scale network can be curtailed in
time.</p><div><div><div>
</div>
</div>
</div>
<br>
| Identifer | oai:union.ndltd.org:purdue.edu/oai:figshare.com:article/14512083 |
| Date | 29 April 2021 |
| Creators | Adefolarin Alaba Bolaji (10723926) |
| Source Sets | Purdue University |
| Detected Language | English |
| Type | Text, Thesis |
| Rights | CC BY 4.0 |
| Relation | https://figshare.com/articles/thesis/Community_Detection_of_Anomaly_in_Large-Scale_Network_Dissertation_-_Adefolarin_Bolaji_pdf/14512083 |
Page generated in 0.0026 seconds