The detection and intrusion prevention systems could be realized as independent hardware or set in the software form on to the host. The primary purpose of these protective elements is the undesirable activity detection such as integrity intrusion of the files, invalid attempts while connecting to the remote service or acquisition of the local network data. The systems react to the event on the basis of the action that is defined by internal rules. We can include the caution sending or communication blocking among possible counteractions. The base principals of the detection and intrusion prevention systems are described in the dissertation. Various types of captured data analyses and processes of the inhere rules creation and further more caution formats are mentioned in the dissertation. There are also considered the alternatives of their location including advantages of selected situations. There is described the installation and setting up of particular elements of the realized network and security systems. In order to the verification of functionality and factor of the protection providing there was realized several selected types of attacks.

Context. The intrusion detection systems are being widely used for detecting the malicious traffic in many industries and they use a variety of technologies. Each IDs had different architecture and are deployed for detecting malicious activity. Intrusion detection system has a different set of rules which can defined based on requirement. Therefore, choosing intrusion detection system for and the appropriate environment is not an easy task. Objectives. The goal of this research is to evaluate three most used open source intrusion detection systems in terms of performance. And we give details about different types of attacks that can be detected using intrusion detection system. The tools that we select are Snort, Suricata, OSSEC. Methods. The experiment is conducted using TCP, SCAN, ICMP, FTP attack. Each experiment was run in different traffic rates under normal and malicious traffics all rule are active. All these tests are conducted in a virtual environment. Results. We can calculate the performance of IDS by using CPU usage, memory usage, packet loss and a number of alerts generated. These results are calculated for both normal and malicious traffic. Conclusions. We conclude that results vary in different IDS for different traffic rates. Specially snort showed better performance in alerts identification and OSSEC in the performance of IDS. These results indicated that alerts are low when the traffic rates high are which indicates this is due to the packet loss. Overall OSSEC provides better performance. And Snort provides better performance and accuracy for alert detection.

The 5G Radio Access Network (RAN) is a critical system that must be secured against potential attacks, particularly its Base-Band Unit (BBU), which is a common target for intrusions. Ericsson, which is a big provider of such systems, has placed significant emphasis on implementing Intrusion Detection Systems (IDS) to detect threats. However, the attention given to Intrusion Response Systems (IRS) in general is limited, with current challenges including false alarms, response cost, response time and reliability. Also, the hardware limitations of the BBU present difficulties in designing an effective IRS. To address these challenges, a semi-automated IRS was implemented with a dynamic and cost-based response selection approach. Open Source SECurity (OSSEC), which is a free, open-source endpoint detection and response tool, was employed to execute the selected responses. The effectiveness of the IRS was assessed based on Ericsson's requirements, reliability, response time, response cost and false alarms. The results obtained show that the proposed IRS is reliable as it can handle a huge number of intrusions and has negligible performance overhead in less extreme attack cases. These findings offer valuable insights into addressing intrusions within a system with constrained hardware resources.

This master's thesis deals with open source PBX security against security attacks. In the theoretical part is detailed description of problematic about attacks that could be used on VoIP systems with high focus on the Denial of Service attack. Furthermore are in theoretical part described methods of security of initialization protocol SIP. Individual chapter is devoted to intrusion detection and prevention of IDS and IPS systems, focusing on Snort and OSSEC. In the practical part of the work was created generator of attacks against various PBX systems, which was subsequently used for detailed testing. Special tests of PBX system are then used against DoS attacks, for which was created protection in form of active elements consisting of IDS Snort & OSSEC. These are capable to provide protection in real-time. The protection was tested on particular PBX systems and in matter of comparison were measured possibilities before and after of security implementation. The output of this work is attacks generator VoIPtester and creation of configuration rules for Snort and OSSEC.