The thesis describes a design and implementation of Nemea system module purposed on generation of records about simulated network attacks. This thesis also contains brief description of Nemea system and several network attacks. Finally, part of this work is description of simulated attacks and methods of simulations.

It is widely accepted that modern computer networks (often presented as a heterogeneous collection of functioning organisations, applications, software, and hardware) contain vulnerabilities. This research proposes a new methodology to compute a dynamic severity cost for each state. Here a state refers to the behaviour of a system during an attack; an example of a state is where an attacker could influence the information on an application to alter the credentials. This is performed by utilising a modified variant of the Common Vulnerability Scoring System (CVSS), referred to as a Dynamic Vulnerability Scoring System (DVSS). This calculates scores of intrinsic, time-based, and ecological metrics by combining related sub-scores and modelling the problem’s parameters into a mathematical framework to develop a unique severity cost. The individual static nature of CVSS affects the scoring value, so the author has adapted a novel model to produce a DVSS metric that is more precise and efficient. In this approach, different parameters are used to compute the final scores determined from a number of parameters including network architecture, device setting, and the impact of vulnerability interactions. An attack graph (AG) is a security model representing the chains of vulnerability exploits in a network. A number of researchers have acknowledged the attack graph visual complexity and a lack of in-depth understanding. Current attack graph tools are constrained to only limited attributes or even rely on hand-generated input. The automatic formation of vulnerability information has been troublesome and vulnerability descriptions are frequently created by hand, or based on limited data. The network architectures and configurations along with the interactions between the individual vulnerabilities are considered in the method of computing the Cost using the DVSS and a dynamic cost-centric framework. A new methodology was built up to present an attack graph with a dynamic cost metric based on DVSS and also a novel methodology to estimate and represent the cost-centric approach for each host’ states was followed out. A framework is carried out on a test network, using the Nessus scanner to detect known vulnerabilities, implement these results and to build and represent the dynamic cost centric attack graph using ranking algorithms (in a standardised fashion to Mehta et al. 2006 and Kijsanayothin, 2010). However, instead of using vulnerabilities for each host, a CostRank Markov Model has developed utilising a novel cost-centric approach, thereby reducing the complexity in the attack graph and reducing the problem of visibility. An analogous parallel algorithm is developed to implement CostRank. The reason for developing a parallel CostRank Algorithm is to expedite the states ranking calculations for the increasing number of hosts and/or vulnerabilities. In the same way, the author intends to secure large scale networks that require fast and reliable computing to calculate the ranking of enormous graphs with thousands of vertices (states) and millions of arcs (representing an action to move from one state to another). In this proposed approach, the focus on a parallel CostRank computational architecture to appraise the enhancement in CostRank calculations and scalability of of the algorithm. In particular, a partitioning of input data, graph files and ranking vectors with a load balancing technique can enhance the performance and scalability of CostRank computations in parallel. A practical model of analogous CostRank parallel calculation is undertaken, resulting in a substantial decrease in calculations communication levels and in iteration time. The results are presented in an analytical approach in terms of scalability, efficiency, memory usage, speed up and input/output rates. Finally, a countermeasures model is developed to protect against network attacks by using a Dynamic Countermeasures Attack Tree (DCAT). The following scheme is used to build DCAT tree (i) using scalable parallel CostRank Algorithm to determine the critical asset, that system administrators need to protect; (ii) Track the Nessus scanner to determine the vulnerabilities associated with the asset using the dynamic cost centric framework and DVSS; (iii) Check out all published mitigations for all vulnerabilities. (iv) Assess how well the security solution mitigates those risks; (v) Assess DCAT algorithm in terms of effective security cost, probability and cost/benefit analysis to reduce the total impact of a specific vulnerability.

This thesis deals with the principles of network traffic obfuscation, in order to avoid its detection by the Intrusion Detection System installed in the network. At the beginning of the work, reader is familiarized with the fundamental principle of the basic types of IDS and introduced into the matter of obfuscation techniques, that serve as stepping stone in order to create our own library, whose design is described in the last part of the work. The outcome of the work is represented by a library, that provides all the implemented techniques for further use. The library can be well utilized in penetration testing of the new systems or used by the attacker.

This diploma thesis deals with the design and implementation of a tool for network attack detection from a captured network communication. It utilises the tshark packet analyser, the meaning of which is to convert the input file with the captured communications to the PDML format. The objective of this conversion being, increasing the flexibility of input data processing. When designing the tool, emphasis has been placed on the ability to expand it to detect new network attacks and on integrating these additions with ease. For this reason, the thesis also includes the design of a complex declarative descriptions for network attacks in the YAML serialization format. This allows us to specify the key properties of the network attacks and the conditions for their detection. The resulting tool acts as an interpreter of proposed declarative descriptions allowing it to be expanded with new types of attacks.

This thesis focuses on description of the current state of research in the detection of network attacks and subsequently on the improvement of detection capabilities of specific attacks by establishing a formal definition of network metrics. These metrics approximate the progress of network connection and create a signature, based on behavioral characteristics of the analyzed connection. The aim of this work is not the prevention of ongoing attacks, or the response to these attacks. The emphasis is on the analysis of connections to maximize information obtained and definition of the basis of detection system that can minimize the size of data collected from the network, leaving the most important information for subsequent analysis. The main goal of this work is to create the concept of the detection system by using defined metrics for reduction of the network traffic to signatures with an emphasis on the behavioral aspects of the communication. Another goal is to increase the autonomy of the detection system by developing an expert knowledge of honeypot system, with the condition of independence to the technological aspects of analyzed data (e.g. encryption, protocols used, technology and environment). Defining the concept of honeypot system's expert knowledge in the role of the teacher of classification algorithms creates autonomy of the~system for the detection of unknown attacks. This concept also provides the possibility of independent learning (with no human intervention) based on the knowledge collected from attacks on these systems. The thesis describes the process of creating laboratory environment and experiments with the defined network connection signature using collected data and downloaded test database. The results are compared with the state of the art of the network detection systems and the benefits of the proposed approximation methods are highlighted.

This thesis is focused on network behavior analysis (NBA) designed to detect network attacks. The goal of the thesis is to increase detection accuracy of obfuscated network attacks. Methods and techniques used to detect network attacks and network traffic classification were presented first. Intrusion detection systems (IDS) in terms of their functionality and possible attacks on them are described next. This work also describes principles of selected attacks against IDS. Further, obfuscation methods which can be used to overcome NBA are suggested. The tool for automatic exploitation, attack obfuscation and collection of this network communication was designed and implemented. This tool was used for execution of network attacks. A dataset for experiments was obtained from collected network communications. Finally, achieved results emphasized requirement of training NBA models by obfuscated malicious network traffic.