Information Technology systems generate system log messages to allow for the monitoring of the system. In increasingly large and complex systems the volume of log data can overwhelm the analysts tasked with monitoring these systems. A system was developed that utilizes Principal Component Analysis to assist the analyst in the characterization of system health and events. Once trained, the system was able to accurately identify a state of heavy load on a device with a low false positive rate. The system was also able to accurately identify an error condition when trained on a single event. The method employed is able to assist in the real time monitoring of large complex systems, increasing the efficiency of trained analysts.

This master's thesis in the first part describes the AS/400 and its message system and concentrates especially on the following areas: predefinition of messages and their storing, types of messages and levels of their importance, work with variables included in message text and ways of sending messages. On the basis of AS/400 message system is designed and implemented message log system for the application loggin for Aegis. s.r.o. The analysis of the message log systems is also a part of the work. The syslog and syslog-ngused in UNIX systems are described, concerning types of messages, importance of messages and filtering and storing of messages. It further describes possibilities of application logging based on Java in the specific case of the Log4jutility. In the second part thesis describes own log message systems design and implementation.

System Administrators use many of the same tactics that are implemented by hackers to validate the security of their systems, such as port scanning and vulnerability scanning. Port scanning is slow, and can be highly inaccurate. After a scan is complete, the results of the scan must be cross checked with a vulnerability database to discover if any vulnerabilities are present. While these techniques are useful, they have severe limitations. System Administrators have full access to all of their machines. They should not have to rely exclusively on port scanning them from the outside of their machines to check for vulnerabilities when they have this level of access. This thesis introduces a novel concept for replacing port scanning with a Log File Inventory Management System. This system will be able to automatically build an accurate system inventory using existing log files. This system inventory will then be automatically cross checked with a database of known vulnerabilities in real-time resulting in faster and more accurate vulnerability reporting than is found in traditional port scanning methods.

Modern IT systems tend to become more and more complex, while the number of active systems in companies increases. Furthermore, the number of security-related incidents is at an all-time high. These new conditions impose new demands on organizations. For example, it is no longer possible to manually collect and examine the systems log messages. The purpose of this thesis has been to make a comprehensive study of solutions for automated collecting and managing of log messages, analyze the Swedish Armed Forces specification for solutions for central log collection and management, and evaluating exis- ting solutions. The work consisted primarily of literature studies and evaluations of two of the Swedish Armed Forces of selected products: NetIQ Security Manager and Splunk. The conclusion was that neither of the two products met the non-optional requirements posed by the specification. I personally think that the Swedish Armed Forces’ requirements specification for the central log management is far too strict and should hence be revised. A number of requirements in the current specification can be removed. Other requirements should be reformulated and/or re-evaluated. / Moderna IT-system tenderar att bli mer och mer komplexa, samtidigt som antalet ak- tiva system i ett fo ̈retag o ̈kar. Vidare a ̈r antalet sa ̈kerhetsrelaterade incidenter ho ̈gre a ̈n n ̊agonsin. Dessa nya omsta ̈ndigheter sta ̈ller nya krav p ̊a organisationer. Exempelvis a ̈r det inte la ̈ngre mo ̈jligt att manuellt samla in och granska systemens loggmeddelanden. Avsikten med den ha ̈r uppsatsen har varit att go ̈r en o ̈vergripande granskning av lo ̈sningar fo ̈r automatisk insamling och analys av loggmeddelanden, analysera de krav som Fo ̈rsvarsmakten sta ̈ller p ̊a lo ̈sningar fo ̈r central logghantering, samt utva ̈rdera befintliga lo ̈sningar. Arbetet bestod huvudsakligen av litteraturstudier samt utva ̈rderingar av tv ̊a av Fo ̈rsvarsmakten utvalda produkter: NetIQ Security Manager och Splunk. Slutsatsen blev att ingen av de tv ̊a produkterna uppfyller Fo ̈rsvarsmaktens samtliga krav fo ̈r central logghantering. Personligen anser jag att Fo ̈rsvarsmaktens kravspecifikation fo ̈r central logg- hantering a ̈r fo ̈r strikt och bo ̈r omarbetas. Ett antal krav i den nuvarande specifikationen kan med fo ̈rdel tas bort. Andra krav bo ̈r omformuleras och/eller omva ̈rderas.

Diploma thesis deals with monitoring critical infrastructure, critical information infrastructure and network monitoring in energetic industry. The goal is to create analytical environment for processing logs from the network, to map the most critical segments of the network and implementation of monitoring and network devices, that increase security and mitigate risks of security events or security incidents

Táto diplomová práca sa venuje problematike využitia strojového učenia na detekciu anomálií na základe analýzy systémových logov. Navrhnuté modely sú založené na algoritmoch strojového učenia s učiteľom, bez učiteľa a na hlbokom učení. Funkčnosť a správanie týchto algoritmov sú objasnené ako teoreticky, tak aj prakticky. Okrem toho boli využité metódy a postupy na predspracovanie dát predtým, než boli vložené do modelov strojového učenia. Navrhnuté modely sú na konci porovnané s využitím viacerých metrík a otestované na syslogoch, ktoré modely predtým nevideli. Najpresnejší výkon podali modely Klasifikátor rozhodovacích stromov, Jednotriedny podporný vektorový stroj a model Hierarchické zoskupovanie, ktoré správne označili 93,95%, 85,66% a 85,3% anomálií v uvedenom poradí.

Manually analysing logfiles is a very time consuming and error-prone effort. By developing a system to automatically analysing the logfiles it is possible to both increase the speed and accuracy of the analysis. This thesis presents a method for automatic anomaly detection in logfiles using statistical analysis and threshold based classification. The presented method uses five different threshold based approaches to identify anomalous entries within a logfile. Each of the five approaches was successful in identifying and reporting perceived anomalies within 805 logfiles provided by Sandvine, it was however not possible to do a formal evaluation of the results due to a lack of a ground truth.

There is a growing need for more efficient tools and services for log analysis. A need that comes from the ever-growing use of digital services and applications, each one generating thousands of lines of log event message for the sake of auditing and troubleshooting. This thesis was initiated on behalf of one of the departments of the IT consulting company TietoEvry in Karlstad. The purpose of this thesis project was to investigate whether the log analysis service Elastic Stack would be a suitable solution for TietoEvry’s need for a more efficient method of log event analysis. As part of this investigation, a small-scale deployment of Elastic Stack was created, used as proof of concept. The investigation showed that Elastic Stack would be a suitable tool for the monitoring and analysis needs of TietoEvry. The final version of deployment was, however, not able to fulfill all of the requirements that were initially set out by TietoEvry, however, this was mainly due to a lack of time and rather than limitations of Elastic Stack.

Monitoring and logging is fundamental in systems. Today, these techniques are widely used to detect functionality issues as well as malicious event. This thesis compares and evaluates two techniques commonly used for this purpose: the Simple Network Management Protocol (SNMP) and the Syslog protocol. The protocols are evaluated with regard to their vulnerability to traffic pattern analysis, impact on network load and performance of the system, and the attack surface of the technique. Each protocol’s applicability is investigated on a modelled system, containing a data diode, using experiments and analysis of their individual features. We observe only small differences in traffic load and performance of the two techniques. The findings suggest that each of the two techniques can successfully be applied on the modelled system, and the decision on which technique should be used at each instance can therefore be left to the customer of each specific system implementation instance.

<p>The company Västra Mälardalens Kommunalförbund, VMKF, wishes to revise and optimize their present data and IP-telephony network as of today consists of the three municipalities Köping, Arboga and Kungsör. As a municipal corporation, they seek consultation regarding internal as well as external review and investigation of the main structure of the network, its functionality and safety. By today’s increasing demands of Internet accessibility, availability of services and security far more extends the requirement of a complete network design. The foundation of networking rests on the balance between each of these necessities. Therefore, it is of grave importance to optimize a network design, use of hardware and to minimize the administrative overhead. In particular, when the municipality is short of resources and time means money. By letting an impartial investigation of the network act as a starting point it was established that several improvement could be applied. Among these a reconstructed and improved network topology that includes subjects as routing, switching, safety and security, quality of service and technical administrative overhead and the implementation of a real time monitoring of network bandwidth consumption.</p> / <p>Företaget Västra Mälardalens Kommunalförbund, VMKF, har önskemål om att granska och optimera deras befintliga data- och IP-telefoninätverk som i dagsläget spänner över de tre kommunerna Köping, Arboga och Kungsör. Som ett kommunalägt företag önskar de konsultation rörande intern såväl som extern granskning och optimering av huvuddelen av nätverkets funktionalitet samt säkerhet. I och med dagens ökade Internetanvändning och funktionalitetsbehov ställs allt högre krav på tillgänglighet, säkerhet och användarvänlighet. Nätverksteknik bygger mycket på balansen mellan dessa tre punkter. Därför gäller det att optimera nätverkets design, hårdvaruanvändning och att minimera administrativa laster. Detta i synnerhet då kommunens resurser är knappa och då tid i dagens samhälle innebär pengar. Genom att låta en granskning över nätverket som det ser ut i dag ligga till grund konstaterades att flertalet förbättringsmöjligheter kunde genomföras. Bland dessa återfinns en omstrukturerad nätverksdesign som innefattar routing, switching, säkerhet, QoS och teknisk administration samt implementeringen av en realtidsövervakning av bandbreddsanvändning.</p>