Botnets are a global problem, and effective botnet detection requires cooperation of large Internet Service Providers, allowing near global visibility of traffic that can be exploited to detect them. The global visibility comes with huge challenges, especially in the amount of data that has to be analysed. To handle such large volumes of data, a robust and effective detection method is the need of the hour and it must rely primarily on a reduced or abstracted form of data such as a graph of hosts, with the presence of an edge between two hosts if there is any data communication between them. Such an abstraction would be easy to construct and store, as very little of the packet needs to be looked at. Structured P2P command and control have been shown to be robust against targeted and random node failures, thus are ideal mechanisms for botmasters to organize and command their botnets effectively. Thus this thesis develops a scalable, efficient and robust algorithm for the detection of structured P2P botnets in large traffic graphs. It draws from the advances in the state of the art in Community Detection, which aim to partition a graph into dense communities. Popular Community Detection Algorithms with low theoretical time complexities such as Label Propagation, Infomap and Louvain Method have been implemented and compared on large LFR benchmark graphs to study their efficiency. Louvain method is found to be capable of handling graphs of millions of vertices and billions of edges. This thesis analyses the performance of this method with two objective functions, Modularity and Stability and found that neither of them are robust and general. In order to overcome the limitations of these objective functions, a third objective function proposed in the literature is considered. This objective function has previously been used in the case of Protein Interaction Networks successfully, and used in this thesis to detect structured P2P botnets for the first time. Further, the differences in the topological properties - assortativity and density, of structured P2P botnet communities and benign communities are discussed. In order to exploit these differences, a novel measure based on mean regular degree is proposed, which captures both the assortativity and the density of a graph and its properties are studied. This thesis proposes a robust and efficient algorithm that combines the use of greedy community detection and community filtering using the proposed measure mean regular degree. The proposed algorithm is tested extensively on a large number of datasets and found to be comparable in performance in most cases to an existing botnet detection algorithm called BotGrep and found to be significantly faster.

Currently, networks and networked systems are essential media for us to communicate with other people, access resources, and share information. Reading (or sending) emails, navigating web sites, and uploading pictures to social medias are common behaviors using networks. Besides these, networks and networked systems are used to store or access sensitive or private information. In addition, major economic activities, such as buying food and selling used cars, can also be operated with networks. Likewise, we live with networks and networked systems. As network usages are increasing and popular, people face the problems of net- work attacks. Attackers on the networks can steal people’s private information, mislead people to pay money for fake products, and threaten people, who operate online commercial sites, by bothering their services. There are much more diverse types of network attacks that torture many people using networks, and the situation is still serious. The proposal in this dissertation starts from the following two research questions: (i) what kind of network attack is prevalent and how we can investigate it and (ii) how we can protect our networks and networked systems from these attacks. Therefore, this dissertation spans two main areas to provide answers for each question. First, we analyze the behaviors and characteristics of large-scale bot infected hosts, and it provides us new findings of network malware and new insights that are useful to detect (or defeat) recent network threats. To do this, we investigate the characteristics of victims infected by recent popular botnet - Conficker, MegaD, and Srizbi. In addition, we propose a method to detect these bots by correlating network and host features. Second, we suggest new frameworks to make our networks secure based on the new network technology of Software Defined Networking (SDN). Currently, SDN technology is considered as a future major network trend, and it can dynamically program networks as we want. Our suggested frameworks for SDN can be used to devise network security applications easily, and we also provide an approach to make SDN technology secure.

Botnets are networks formed with a number of machines infected by malware called bots. Detection of these malicious networks is a major concern as they pose a serious threat to network security. Most of the research on botnet detection is based on particular botnet characteristics which fail to detect other types of botnet. There exist several generic botnet detection methods that can detect varieties of botnets. But, these generic detection methods perform very poorly in real-life dataset as the methods are not developed based on a real-life botnet dataset. A crucial reason for those detection methods not being developed based on a real-life dataset is that there is a scarcity of large-scale real-life botnet dataset. Due to security and privacy concerns, organizations do not publish their real-life botnet dataset. Therefore, there is a dire need for a simulation methodology that generates a large-scale botnet dataset similar to the original real-life dataset while preserving the security and privacy of the network. In this dissertation, we develop a generic bot detection methodology that can detect a variety of bots and evaluate the methodology in a real-life, large, highly class-imbalanced dataset. Numerical results show that our methodology can detect bots more accurately than the existing methods. Realizing the need for real-life large-scale botnet dataset, we develop a simulation methodology to simulate a large-scale botnet dataset from a real-life botnet dataset. Our simulation methodology is based on Markov chain and role–mining process that can simulate the degree distributions along with triangles (community structures). To scale-up the original graph to large-scale graph, we also propose a scaling-up algorithm, Enterprise connection algorithm. We evaluate our simulated graph by comparing with the original graph as well as with the graph generated by Preferential attachment algorithm. Comparisons are done in the following three major categories: comparison of botnet subgraphs, comparison of overall graphs and comparison of scaled-up graphs. Result demonstrates that our methodology outperform Preferential attachment algorithm in simulating the triangle distributions and the botnet structure.

Yes / Technological advancements have been evolving for so long, particularly Internet of Things (IoT) technology that has seen an increase in the number of connected devices surpass non IoT connections. It has unlocked a lot of potential across different organisational settings from healthcare, transportation, smart cities etc. Unfortunately, these advancements also mean that cybercriminals are constantly seeking new ways of exploiting vulnerabilities for malicious and illegal activities. IoT is a technology that presents a golden opportunity for botnet attacks that take advantage of a large number of IoT devices and use them to launch more powerful and sophisticated attacks such as Distributed Denial of Service (DDoS) attacks. This calls for more research geared towards the detection and mitigation of botnet attacks in IoT systems. This paper proposes a feature selection approach that identifies and removes less influential features as part of botnet attack detection method. The feature selection is based on the frequency of occurrence of the value counts in each of the features with respect to total instances. The effectiveness of the proposed approach is tested and evaluated on a standard IoT dataset. The results reveal that the proposed feature selection approach has improved the performance of the botnet attack detection method, in terms of True Positive Rate (TPR) and False Positive Rate (FPR). The proposed methodology provides 100% TPR, 0% FPR and 99.9976% F-score.

HTTP botnets are currently the most popular form of botnets compared to IRC and P2P botnets. This is because, they are not only easier to implement, operate, and maintain, but they can easily evade detection. Likewise, HTTP botnets flows can easily be buried in the huge volume of legitimate HTTP traffic occurring in many organizations, which makes the detection harder. In this thesis, a new detection framework involving three detection models is proposed, which can run independently or in tandem. The first detector profiles the individual applications based on their interactions, and isolates accordingly the malicious ones. The second detector tracks the regularity in the timing of the bot DNS queries, and uses this as basis for detection. The third detector analyzes the characteristics of the domain names involved in the DNS, and identifies the algorithmically generated and fast flux domains, which are staples of typical HTTP botnets. Several machine learning classifiers are investigated for each of the detectors. Experimental evaluation using public datasets and datasets collected in our testbed yield very encouraging performance results. / Graduate

Network monitoring systems generate a high number of alerts reporting on anomalies and suspicious activity of IP addresses. From a huge number of alerts, only a small fraction is high priority and relevant from human evaluation. The rest is likely to be neglected. Assume that by analyzing large sums of these low priority alerts we can discover valuable information, namely, coordinated IP addresses and type of alerts likely to be correlated. This knowledge improves situational awareness in the field of network monitoring and reflects the requirement of security analysts. They need to have at their disposal proper tools for retrieving contextual information about events on the network, to make informed decisions. To validate the assumption new method is introduced to discover groups of coordinated IP addresses that exhibit temporal correlation in the arrival pattern of their events. The method is evaluated on real-world data from a sharing platform that accumulates 2.2 million alerts per day. The results show, that method indeed detected truly correlated groups of IP addresses.

No / In recent years, the Internet has enabled access to widespread remote services in the distributed computing environment; however, integrity of data transmission in the distributed computing platform is hindered by a number of security issues. For instance, the botnet phenomenon is a prominent threat to Internet security, including the threat of malicious codes. The botnet phenomenon supports a wide range of criminal activities, including distributed denial of service (DDoS) attacks, click fraud, phishing, malware distribution, spam emails, and building machines for illegitimate exchange of information/materials. Therefore, it is imperative to design and develop a robust mechanism for improving the botnet detection, analysis, and removal process. Currently, botnet detection techniques have been reviewed in different ways; however, such studies are limited in scope and lack discussions on the latest botnet detection techniques. This paper presents a comprehensive review of the latest state-of-the-art techniques for botnet detection and figures out the trends of previous and current research. It provides a thematic taxonomy for the classification of botnet detection techniques and highlights the implications and critical aspects by qualitatively analyzing such techniques. Related to our comprehensive review, we highlight future directions for improving the schemes that broadly span the entire botnet detection research field and identify the persistent and prominent research challenges that remain open. / University of Malaya, Malaysia (No. FP034-2012A)

With the advent of ubiquitous computing and Internet of Things (IoT), potentially billions of devices will create a broad range of data services and applications, which will require the communication networks to efficiently manage the increasing complexity. Cognitive network has been envisioned as a new paradigm to address this challenge, which has the capability of reasoning, planning and learning by incorporating cutting edge technologies including knowledge representation, context awareness, network optimization and machine learning. Cognitive network spans over the entire communication system including the core network and wireless links across the entire protocol stack. Cognitive Radio Network (CRN) is a part of cognitive network over wireless links, which endeavors to better utilize the spectrum resources. Core network provides a reliable backend infrastructure to the entire communication system. However, the CR communication and core network infrastructure have attracted various security threats, which become increasingly severe in pace with the growing complexity and adversity of the modern Internet. The focus of this dissertation is to exploit the security vulnerabilities of the state-of-the-art cognitive communication systems, and to provide detection, mitigation and protection mechanisms to allow security enhanced cognitive communications including wireless communications in CRNs and wired communications in core networks. In order to provide secure and reliable communications in CRNs: emph{first}, we incorporate security mechanisms into fundamental CRN functions, such as secure spectrum sensing techniques that will ensure trustworthy reporting of spectrum reading. emph{Second}, as no security mechanism can completely prevent all potential threats from entering CRNs, we design a systematic passive monitoring framework, emph{SpecMonitor}, based on unsupervised machine learning methods to strategically monitor the network traffic and operations in order to detect abnormal and malicious behaviors. emph{Third}, highly capable cognitive radios allow more sophisticated reactive jamming attack, which imposes a serious threat to CR communications. By exploiting MIMO interference cancellation techniques, we propose jamming resilient CR communication mechanisms to survive in the presence of reactive jammers. Finally, we focus on protecting the core network from botnet threats by applying cognitive technologies to detect network-wide Peer-to-Peer (P2P) botnets, which leads to the design of a data-driven botnet detection system, called emph{PeerClean}. In all the four research thrusts, we present thorough security analysis, extensive simulations and testbed evaluations based on real-world implementations. Our results demonstrate that the proposed defense mechanisms can effectively and efficiently counteract sophisticated yet powerful attacks. / Ph. D.

Cybersecurity has become an increasingly important concern for both military and civilian infrastructure globally. Because of the complexity that comes with wireless networks, adversaries have many means of infiltration and disruption of wireless networks. While there is much research done in defending these networks, understanding the robustness of these networks is tantamount for both designing new networks and examining possible security deficiencies in preexisting networks. This dissertation proposes to examine the robustness of wireless networks on three major fronts: the physical layer, the data-link layer, and the network layer. At the physical layer, denial-of-service jamming attacks are considered, and both additive interference and no interference are modeled in an optimal configuration and five common network topologies. At the data-link layer, data transmission efficacy and denial-of-sleep attacks are considered with the goal of maximizing throughput under a constrained lifetime. At the network layer, valid and anomalous communications are considered with the goal of classifying those anomalous communications apart from valid ones. This dissertation proposes that a thorough analysis of the aforementioned three layers provides valuable insights to robustness on general wireless networks.

The need for website administrators to efficiently and accurately detect the presence of web bots has shown to be a challenging problem. As the sophistication of modern web bots increases, specifically their ability to more closely mimic the behavior of humans, web bot detection schemes are more quickly becoming obsolete by failing to maintain effectiveness. Though machine learning-based detection schemes have been a successful approach to recent implementations, web bots are able to apply similar machine learning tactics to mimic human users, thus bypassing such detection schemes. This work seeks to address the issue of machine learning based bots bypassing machine learning-based detection schemes, by introducing a novel unsupervised learning approach to cluster users based on behavioral biometrics. The idea is that, by differentiating users based on their behavior, for example how they use the mouse or type on the keyboard, information can be provided for website administrators to make more informed decisions on declaring if a user is a human or a bot. This approach is similar to how modern websites require users to login before browsing their website; which in doing so, website administrators can make informed decisions on declaring if a user is a human or a bot. An added benefit of this approach is that it is a human observational proof (HOP); meaning that it will not inconvenience the user (user friction) with human interactive proofs (HIP) such as CAPTCHA, or with login requirements