Botnets represent one of the most serious threats against Internet security since they serve as platforms that are responsible for the vast majority of large-scale and coordinated cyber attacks, such as distributed denial of service, spamming, and information stolen. Detecting botnets is therefore of great importance and a number of network-based botnet detection systems have been proposed. However, as botnets perform attacks in an increasingly stealthy way and the volume of network traffic is rapidly growing, existing botnet detection systems are faced with significant challenges in terms of effectiveness and scalability. The objective of this dissertation is to build novel network-based solutions that can boost both the effectiveness of existing botnet detection systems by detecting botnets whose attacks are very hard to be observed in network traffic, and their scalability by adaptively sampling network packets that are likely to be generated by botnets. To be specific, this dissertation describes three unique contributions. First, we built a new system to detect drive-by download attacks, which represent one of the most significant and popular methods for botnet infection. The goal of our system is to boost the effectiveness of existing drive-by download detection systems by detecting a large number of drive-by download attacks that are missed by these existing detection efforts. Second, we built a new system to detect botnets with peer-to-peer (P2P) command&control (C&C) structures (i.e., P2P botnets), where P2P C&Cs represent currently the most robust C&C structures against disruption efforts. Our system aims to boost the effectiveness of existing P2P botnet detection by detecting P2P botnets in two challenging scenarios: i) botnets perform stealthy attacks that are extremely hard to be observed in the network traffic; ii) bot-infected hosts are also running legitimate P2P applications (e.g., Bittorrent and Skype). Finally, we built a novel traffic analysis framework to boost the scalability of existing botnet detection systems. Our framework can effectively and efficiently identify a small percentage of hosts that are likely to be bots, and then forward network traffic associated with these hosts to existing detection systems for fine-grained analysis, thereby boosting the scalability of existing detection systems. Our traffic analysis framework includes a novel botnet-aware and adaptive packet sampling algorithm, and a scalable flow-correlation technique.

<p>The progress in the field of computer networks and Internet is increasing with tremendous volume in recent years. This raises important issue with regards to security. Several solutions emerged in the past which provide security at host or network level. These traditional solutions like antivirus, firewall, spy-ware, and authentication mechanisms provide security to some extends, but they still face the challenge of inherent system flaws, OS bugs and social engineering attacks. Recently, some interesting solution emerged like Intrusion Detection and Prevention systems, but these too have some problems, like detecting and responding in real time, because they mostly require inputs from system administrator. Optimistically, we have succeeded in protecting the hosts to some extent by applying the reactive approach, such as antivirus, firewall and intrusion detection and response systems, But, if we critically analyze this approach, we will reach the conclusion that it has inherent flaws, since the number of penetrations, Internet crime cases, identity and financial data thefts, etc. are rising exponentially in recent years. The main reason is that we are using only reactive approach, i.e. protection system is activated only when some security breach occurs. Secondly, current techniques try to fix the overall huge problem of security using only small remedies (firewall, antivirus and intrusion detection and preventions system) – “point solutions”. Therefore, there is a need to develop a strategy using Mobile Agents in order to operate in reactive and proactive manners, what requires providing security on the principle of defense in depth. So, that ultimate goal of securing a system as a whole can be achieved. System is assumed to be secure if unauthorized access (penetrations) is not possible and system is safe against damages. This strategy will include three aspects: (a) autonomously detect vulnerabilities on different hosts (in a distributed network) before an attacker can exploit (b) protect hosts by detecting attempts of intrusions and responding to them in real time; and finally (c) perform tasks related to security management.</p>

<p>Ericsson’s Packet Radio Access Network (PRAN) is a network solution for packet transport in mobile networks, which utilizes the Internet Protocol (IP). The IP protocol offers benefits in responsiveness and performance adaptation to data bursts when compared to Asynchronous Transfer Mode (ATM), which is still often used. There are many manufacturers / operators providing IP services, which reduce costs. The IP’s use on the Internet brings greater end-user knowledge, wider user community and more programs designed for use in IP environments. Because of this, the spectrum of possible attacks against PRAN broadens. This thesis provides information on what protection an Intrusion Prevention System (IPS) can add to the current PRAN solution.</p><p>A risk analysis is performed to identify assets in and threats against PRAN, and to discover attacks that can be mitigated by the use of an IPS. Information regarding placement of an IPS in the PRAN network is given and tests of a candidate system are performed. IPS features in hardware currently used by Ericsson as well as missing features are pinpointed . Finally, requirements for an IPS intended for use in PRAN are concluded.</p>

ITC/USA 2012 Conference Proceedings / The Forty-Eighth Annual International Telemetering Conference and Technical Exhibition / October 22-25, 2012 / Town and Country Resort & Convention Center, San Diego, California / Telemetry systems are migrating from links to networks. Security solutions that simply encrypt radio links no longer protect the network of Test Articles or the networks that support them. The use of network telemetry is dramatically expanding and new risks and vulnerabilities are challenging issues for telemetry networks. Most of these vulnerabilities are silent in nature and cannot be detected with simple tools such as traffic monitoring. The Intrusion Detection System (IDS) is a security mechanism suited to telemetry networks that can help detect abnormal behavior in the network. Our previous research in Network Intrusion Detection Systems focused on "Password" attacks and "Syn" attacks. This paper presents a generalized method that can detect both "Password" attack and "Syn" attack. In this paper, a K-means Clustering algorithm is used for vector quantization of network traffic. This reduces the scope of the problem by reducing the entropy of the network data. In addition, a Hidden-Markov Model (HMM) is then employed to help to further characterize and analyze the behavior of the network into states that can be labeled as normal, attack, or anomaly. Our experiments show that IDS can discover and expose telemetry network vulnerabilities using Vector Quantization and the Hidden Markov Model providing a more secure telemetry environment. Our paper shows how these can be generalized into a Network Intrusion system that can be deployed on telemetry networks.

Intrusion Detection Systems (IDS) tools are deployed within networks to monitor data that is transmitted to particular destinations such as MySQL,Oracle databases or log files. The data is normally dumped to these destinations without a forensic standard structure. When digital evidence is needed, forensic specialists are required to analyse a very large volume of data. Even though forensic tools can be utilised, most of this process has to be done manually, consuming time and resources. In this research, we aim to address this issue by combining several existing tools to archive the original IDS data into a new container (Digital Evidence Bag) that has a structure based upon standard forensic processes. The aim is to develop a method to improve the current IDS database function in a forensic manner. This database will be optimised for future, forensic, analysis. Since evidence validity is always an issue, a secondary aim of this research is to develop a new monitoring scheme. This is to provide the necessary evidence to prove that an attacker had surveyed the network prior to the attack. To achieve this, we will set up a network that will be monitored by multiple IDSs. Open source tools will be used to carry input validation attacks into the network including SQL injection. We will design a new tool to obtain the original data in order to store it within the proposed DEB. This tool will collect the data from several databases of the different IDSs. We will assume that the IDS will not have been compromised.

Web applications have become increasingly popular in recent years. They are widely used in security-critical areas, such as financial, medical, and military systems. Meanwhile, the number and sophistication of attacks against web applications have increased rapidly. It is important for organizations and companies to add security functions to existing web application servers in order to maintain the confidentiality of critical information. One common approach to protect web systems is to build an Intrusion Detection and Prevention System (IDPS). In this thesis, we propose an IDPS framework to detect and prevent web attacks by employing Aspect-Oriented Programming (AOP) and Autonomic Computing (AC) technologies. This framework can also be used to discover whether a web application under protection has abilities to prevent certain web attacks itself. We developed a prototyping tool to implement the functionality of this framework partially. We evaluated this tool on two Java web applications to detect and prevent Cross Scripting Site (XSS) and Structured Query Language (SQL) Injection, which are two of the most common web attacks. The experimental results show that the prototyping tool based on AOP and AC technologies can be applied to detect and prevent the two common web attacks effectively.

In the last several years the number of computer network attacks has increased rapidly, while at the same time the attacks have become more and more complex and sophisticated. Intrusion detection systems (IDSs) have become essential security appliances for detecting and reporting these complex and sophisticated attacks. Security officers and analysts need to analyze intrusion alerts in order to extract the underlying attack scenarios and attack intelligence. These allow taking appropriate responses and designing adequate defensive or prevention strategies. Intrusion analysis is a resource intensive, complex and expensive process for any organization. The current generation of IDSs generate low level intrusion alerts that describe individual attack events. In addition, existing IDSs tend to generate massive amount of alerts with high rate of redundancies and false positives. Typical IDS sensors report attacks independently and are not designed to recognize attack plans or discover multistage attack scenarios. Moreover, not all the attacks executed against the target network will be detected by the IDS. False negatives, which correspond to the attacks missed by the IDS, will either make the reconstruction of the attack scenario impossible or lead to an incomplete attack scenario. Because of the above mentioned reasons, intrusion analysis is a challenging task that mainly relies on the analyst experience and requires manual investigation. In this dissertation, we address the above mentioned challenges by proposing a new framework that allows automatic intrusion analysis and attack intelligence extraction by analyzing the alerts and attacks semantics using both machine learning and knowledge-representation approaches. Particularly, we use ontological engineering, semantic correlation, and clustering methods to design a new automated intrusion analysis framework. The proposed alert analysis approach addresses many of the gaps observed in the existing intrusion analysis techniques, and introduces when needed new metrics to measure the quality of the alerts analysis process. We evaluated experimentally our framework using different benchmark intrusion detection datasets, yielding excellent performance results. / Graduate

In this study we address the problem of detecting new types of intrusions to computer systems which cannot be handled by widely implemented knowledge-based mechanisms. The solutions offered by behavior-based prototypes either suffer low accuracy and low completeness or require use data eplaining abnormal behavior which actually is not available. Our aim is to develop an algorithm which can produce a satisfactory model of the target system&rsquo / s behavior in the absence of negative data. First, we design and develop an intelligent and behavior-based detection mechanism using genetic-based machine learning techniques with subsidies in the Bucket Brigade Algorithm. It classifies the possible system states to be normal and abnormal and interprets the abnormal state observations as evidences for the presence of an intrusion. Next we provide another algorithm which focuses on capturing normal behavior of the target system to detect intrusions again by identifying anomalies. A compact and highly complete rule set is generated by continuously inserting observed states as rules into the rule set and combining similar rule pairs in each step. Experiments conducted using the KDD-99 data set have produced fairly good results for both of the algorihtms.

Mobile ad hoc networks (MANETs), due to their unique characteristics (e.g., unsecured wireless channel, dynamic mobility, absence of central supportive infrastructure and limited resources), are suffering from a wide range of security threats and attacks. Particularly, MANETs are susceptible to the Denial of Service (DoS) attack that aims to disrupt the network by consuming its resources. In MANETs, a special form of DoS attack has emerged recently as a potentially major threat: the flooding attack. This attack recruits multiple attack nodes to flood the MANET with overwhelming broadcast traffic. This flooding traffic is so large that all, or most of, MANET resources are exhausted. As a result, the MANET is not able to provide any services. This thesis aims to investigate the flooding attack and propose detective security measures to defend MANETs against such an attack.

Intrusion Detection systems are now an essential component in the overall network and data security arsenal. With the rapid advancement in the network technologies including higher bandwidths and ease of connectivity of wireless and mobile devices, the focus of intrusion detection has shifted from simple signature matching approaches to detecting attacks based on analyzing contextual information which may be specific to individual networks and applications. As a result, anomaly and hybrid intrusion detection approaches have gained significance. However, present anomaly and hybrid detection approaches suffer from three major setbacks; limited attack detection coverage, large number of false alarms and inefficiency in operation. / In this thesis, we address these three issues by introducing efficient intrusion detection frameworks and models which are effective in detecting a wide variety of attacks and which result in very few false alarms. Additionally, using our approach, attacks can not only be accurately detected but can also be identified which helps to initiate effective intrusion response mechanisms in real-time. Experimental results performed on the benchmark KDD 1999 data set and two additional data sets collected locally confirm that layered conditional random fields are particularly well suited to detect attacks at the network level and user session modeling using conditional random fields can effectively detect attacks at the application level. / We first introduce the layered framework with conditional random fields as the core intrusion detector. Layered conditional random field can be used to build scalable and efficient network intrusion detection systems which are highly accurate in attack detection. We show that our systems can operate either at the network level or at the application level and perform better than other well known approaches for intrusion detection. Experimental results further demonstrate that our system is robust to noise in training data and handles noise better than other systems such as the decision trees and the naive Bayes. We then introduce our unified logging framework for audit data collection and perform user session modeling using conditional random fields to build real-time application intrusion detection systems. We demonstrate that our system can effectively detect attacks even when they are disguised within normal events in a single user session. Using our user session modeling approach based on conditional random fields also results in early attack detection. This is desirable since intrusion response mechanisms can be initiated in real-time thereby minimizing the impact of an attack.