The goal of this master thesis is to make possible to perform system log analysis in more general way than well-known host-based instrusion detection systems (HIDS). The way how to achieve this goal is via proposed user-friendly regular expressions. This thesis deals with making regular expressions possible to use in the field of log analysis, and mainly by users unfamiliar with formal aspects of computer science.

This master's thesis in the first part describes the AS/400 and its message system and concentrates especially on the following areas: predefinition of messages and their storing, types of messages and levels of their importance, work with variables included in message text and ways of sending messages. On the basis of AS/400 message system is designed and implemented message log system for the application loggin for Aegis. s.r.o. The analysis of the message log systems is also a part of the work. The syslog and syslog-ngused in UNIX systems are described, concerning types of messages, importance of messages and filtering and storing of messages. It further describes possibilities of application logging based on Java in the specific case of the Log4jutility. In the second part thesis describes own log message systems design and implementation.

Modern semiconductor manufacturing is a complex process with a multitude of software applications. This application landscape has to be constantly monitored, since the communication and access patterns provide important insights. Because of the high event rates of the equipment log data stream in modern factories, big-data tools are required for scalable state and history analytics. The choice of suitable big-data solutions and their technical realization remains a challenging task. This thesis compares big-data architectures and discovers solutions for log-data ingest, enrichment, analytics and visualization. Based on the use cases and requirements of developers working in this field, a comparison of a custom assembled stack and a complete solution is made. Since the complete stack is a preferable solution, Datadog, Grafana Loki and the Elastic 8 Stack are selected for a more detailed study. These three systems are implemented and compared based on the requirements. All three systems are well suited for big-data logging and fulfill most of the requirements, but show different capabilities when implemented and used.:1 Introduction 1.1 Motivation 1.2 Structure 2 Fundamentals and Prerequisites 2.1 Logging 2.1.1 Log level 2.1.2 CSFW log 2.1.3 SECS log 2.2 Existing system and data 2.2.1 Production process 2.2.2 Log data in numbers 2.3 Requirements 2.3.1 Functional requirements 2.3.2 System requirements 2.3.3 Quality requirements 2.4 Use Cases 2.4.1 Finding specific communication sequence 2.4.2 Watching system changes 2.4.3 Comparison with expected production path 2.4.4 Enrichment with metadata 2.4.5 Decoupled log analysis 3 State of the Art and Potential Software Stacks 3.1 State of the art software stacks 3.1.1 IoT flow monitoring system 3.1.2 Big-Data IoT monitoring system 3.1.3 IoT Cloud Computing Stack 3.1.4 Big-Data Logging Architecture 3.1.5 IoT Energy Conservation System 3.1.6 Similarities of the architectures 3.2 Selection of software stack 3.2.1 Components for one layer 3.2.2 Software solutions for the stack 4 Analysis and Implementation 4.1 Full stack vs. a custom assembled stack 4.1.1 Drawbacks of a custom assembled stack 4.1.2 Advantages of a complete solution 4.1.3 Exclusion of a custom assembled stack 4.2 Selection of full stack solutions 4.2.1 Elastic vs. Amazon 4.2.2 Comparison of Cloud-Only-Solutions 4.2.3 Comparison of On-Premise-Solutions 4.3 Implementation of selected solutions 4.3.1 Datadog 4.3.2 Grafana Loki Stack 4.3.3 Elastic 8 Stack 5 Comparison 5.1 Comparison of components 5.1.1 Collection 5.1.2 Analysis 5.1.3 Visualization 5.2 Comparison of requirements 5.2.1 Functional requirements 5.2.2 System requirements 5.2.3 Quality requirements 5.3 Results 6 Conclusion and Future Work 6.1 Conclusion 6.2 Future Work / Die moderne Halbleiterfertigung ist ein komplexer Prozess mit einer Vielzahl von Softwareanwendungen. Diese Anwendungslandschaft muss ständig überwacht werden, da die Kommunikations- und Zugriffsmuster wichtige Erkenntnisse liefern. Aufgrund der hohen Ereignisraten des Logdatenstroms der Maschinen in modernen Fabriken werden Big-Data-Tools für skalierbare Zustands- und Verlaufsanalysen benötigt. Die Auswahl geeigneter Big-Data-Lösungen und deren technische Umsetzung ist eine anspruchsvolle Aufgabe. Diese Arbeit vergleicht Big-Data-Architekturen und untersucht Lösungen für das Sammeln, Anreicherung, Analyse und Visualisierung von Log-Daten. Basierend auf den Use Cases und den Anforderungen von Entwicklern, die in diesem Bereich arbeiten, wird ein Vergleich zwischen einem individuell zusammengestellten Stack und einer Komplettlösung vorgenommen. Da die Komplettlösung vorteilhafter ist, werden Datadog, Grafana Loki und der Elastic 8 Stack für eine genauere Untersuchung ausgewählt. Diese drei Systeme werden auf der Grundlage der Anforderungen implementiert und verglichen. Alle drei Systeme eignen sich gut für Big-Data-Logging und erfüllen die meisten Anforderungen, zeigen aber unterschiedliche Fähigkeiten bei der Implementierung und Nutzung.:1 Introduction 1.1 Motivation 1.2 Structure 2 Fundamentals and Prerequisites 2.1 Logging 2.1.1 Log level 2.1.2 CSFW log 2.1.3 SECS log 2.2 Existing system and data 2.2.1 Production process 2.2.2 Log data in numbers 2.3 Requirements 2.3.1 Functional requirements 2.3.2 System requirements 2.3.3 Quality requirements 2.4 Use Cases 2.4.1 Finding specific communication sequence 2.4.2 Watching system changes 2.4.3 Comparison with expected production path 2.4.4 Enrichment with metadata 2.4.5 Decoupled log analysis 3 State of the Art and Potential Software Stacks 3.1 State of the art software stacks 3.1.1 IoT flow monitoring system 3.1.2 Big-Data IoT monitoring system 3.1.3 IoT Cloud Computing Stack 3.1.4 Big-Data Logging Architecture 3.1.5 IoT Energy Conservation System 3.1.6 Similarities of the architectures 3.2 Selection of software stack 3.2.1 Components for one layer 3.2.2 Software solutions for the stack 4 Analysis and Implementation 4.1 Full stack vs. a custom assembled stack 4.1.1 Drawbacks of a custom assembled stack 4.1.2 Advantages of a complete solution 4.1.3 Exclusion of a custom assembled stack 4.2 Selection of full stack solutions 4.2.1 Elastic vs. Amazon 4.2.2 Comparison of Cloud-Only-Solutions 4.2.3 Comparison of On-Premise-Solutions 4.3 Implementation of selected solutions 4.3.1 Datadog 4.3.2 Grafana Loki Stack 4.3.3 Elastic 8 Stack 5 Comparison 5.1 Comparison of components 5.1.1 Collection 5.1.2 Analysis 5.1.3 Visualization 5.2 Comparison of requirements 5.2.1 Functional requirements 5.2.2 System requirements 5.2.3 Quality requirements 5.3 Results 6 Conclusion and Future Work 6.1 Conclusion 6.2 Future Work

The purpose of this study was to examine the relationship between keeping a daily leisure-time physical activity record and adolescent (a) feelings of competence toward leisure-time physical activity, (b) motivational profiles toward leisure-time physical activity, and (c) leisure-time physical activity behaviors. Participants were 124 junior high and high school physical education (PE) students. Students completed the Behavioural Regulation in Exercise Questionnaire–2, the Godin Leisure–Time Exercise Questionnaire, the Perceived Competence Scale, and were assigned to keep an online leisure-time physical activity record for three weeks as part of their regular PE class. A 2 (gender) x 4 (trials) repeated measures ANCOVA was used to examine the relationships between recording compliance and the variables of perceived competence, motivation, and physical activity. Results showed a significant interaction between recording compliance and leisure-time physical activity. As students kept the leisure-time physical activity record, boys' leisure-time physical activity levels significantly increased and girls' leisure-time physical activity levels significantly decreased. Also, a significant interaction between recording compliance and introjected regulation was found. The more students recorded the less motivated they were by guilt and obligation to exercise in their leisure time. Lastly a significant interaction was found between recording compliance and intrinsic regulation, showing that the more students recorded the more intrinsically motivated they were to exercise in their leisure time. Implications and suggestions are set forth for PE professionals.

[pt] Nos dias de hoje, temos acesso a dados de tamanho, dimensionalidade e complexidade sem precedentes. Para extrair informações desconhecidas e inesperadas desses dados complexos e dinâmicos, necessitamos de estratégias efetivas e eficientes. Uma dessas estratégias é usar aplicações de análise visual (visual analytics), que combinam técnicas de análise de dados e de visualização. Depois do processo de descoberta de conhecimento, um grande desafio é filtrar a informação essencial que levou à descoberta e comunicar os achados a outras pessoas. Nós propomos tirar proveito do traço deixado pela análise exploratória de dados, sob a forma do histórico da interação do usuário, para ajudar nesse processo. Com o traço, o usuário pode escolher os passos de interação desejados e criar uma narrativa, compartilhando o conhecimento adquirido com os leitores. Para atingir nosso objetivo, desenvolvemos o arcabouço BONNIE (Building Online Narratives from Noteworthy Interaction Events - Construindo Narrativas Online a partir de Eventos de Interação Relevantes). O arcabouço compreende um modelo de log para registrar os eventos de interação, código auxiliar para ajudar o(a) desenvolvedor(a) a instrumentar o seu próprio código, e um ambiente para visualizar o histórico de interação e construir narrativas. Esta tese apresenta nossa proposta para comunicar descobertas em aplicações de análise visual, o arcabouço BONNIE, e alguns estudos empíricos que realizamos para avaliar nossa solução. / [en] Nowadays, we have access to data of unprecedentedly large size, high dimensionality, and complexity. To extract unknown and unexpected information from such complex and dynamic data, we need effective and efficient strategies. One such strategy is to combine data analysis and visualization techniques, which is the essence of visual analytics applications. After the knowledge discovery process, a major challenge is to filter the essential information that led to a discovery and to communicate the findings to other people. We propose to take advantage of the trace left by the exploratory data analysis, in the form of ser interaction history, to aid in this process. With the trace, the user can choose the desired interaction steps and create a narrative, sharing the acquired knowledge with readers. To achieve our goal, we have developed the BONNIE (Building Online Narratives from Noteworthy Interaction Events) framework. The framework comprises a log model to register the interaction events, auxiliary code to help the developer instrument his or her own code, and an environment to view the user s own interaction history and build narratives. This thesis presents our proposal for communicating discoveries in visual analytics applications, the BONNIE framework, and a few empirical studies we conducted to evaluate our solution.

Process mining is the application of data science technologies on transactional business data to identify or monitor processes within an organization. The analyzed data often originates from process-unaware enterprise software, e.g. Enterprise Resource Planning (ERP) systems. The differences in data management between ERP and process mining systems result in a large fraction of ambiguous cases, affected by convergence and divergence. The consequence is a chasm between the process as interpreted by process mining, and the process as executed in the ERP system. In this thesis, a purchasing process of an SAP ERP system is used to demonstrate, how ERP data can be extracted and transformed into a process mining event log that expresses ambiguous cases as accurately as possible. As the content and structure of the event log already define the scope (i.e. which process) and granularity (i.e. activity types), the process mining results depend on the event log quality. The results of this thesis show how the consideration of case attributes, the notion of a case and the granularity of events can be used to manage the event log quality. The proposed solution supports continuous event extraction from the ERP system. / Process mining är användningen av datavetenskaplig teknik för transaktionsdata, för att identifiera eller övervaka processer inom en organisation. Analyserade data härstammar ofta från processomedvetna företagsprogramvaror, såsom SAP-system, vilka är centrerade kring affärsdokumentation. Skillnaderna i data management mellan Enterprise Resource Planning (ERP)och process mining-system resulterar i en stor andel tvetydiga fall, vilka påverkas av konvergens och divergens. Detta resulterar i ett gap mellan processen som tolkas av process mining och processen som exekveras i ERP-systemet. I denna uppsats används en inköpsprocess för ett SAP ERP-system för att visa hur ERP-data kan extraheras och omvandlas till en process mining-orienterad händelselogg som uttrycker tvetydiga fall så precist som möjligt. Eftersom innehållet och strukturen hos händelseloggen redan definierar omfattningen (vilken process) och granularitet (aktivitetstyperna), så beror resultatet av process mining på kvalitén av händelseloggen. Resultaten av denna uppsats visar hur definitioner av typfall och händelsens granularitet kan användas för att förbättra kvalitén. Den beskrivna lösningen stöder kontinuerlig händelseloggsextraktion från ERPsystemet.

No description available.

Tracking of tree logs from a harvesting site to its processing site is a legal requirement for timber-based industries. Wood log identification is an important task in the forestry industry and has traditionally relied on manual inspection by trained experts. However, with the increasing demand for wood products and the need for efficient and accurate identification, there has been a growing interest in developing automated wood log identification systems. In this study, we explored approaches to wood log identification with three objectives: automated log identification, damaged log identification (detection of damaged log), and pith detection (estimation of the pith location). We propose a novel approach for wood log identification using computer vision techniques. Our approach involves capturing images of wood logs using a digital camera, detection of pith location using machine learning model, then applying image processing algorithms to extract relevant features, such as the bark pattern, rings, and use information provided by radial line that extends from the bark to the pith to identify the wood log. In our experiments, we used different image processing techniques to build computer vision models for log identification, trained machine learning and deep learning models to classify the wood log into damaged or not, and deep learning models for estimation of the pith location. The findings of this study indicate that the proposed approach for wood log identification work best with Canny Edge Detection technique, and we can further extend this approach.

Since many log tracing application and diagnostic commands are now available on nodes at base station, event log can easily be collected, parsed and structured for network performance analysis. In order to improve In Service Performance of customer network, a sequential machine learning model can be trained, test, and deployed on each node to learn from the past events to predict future crashes or a failure. This thesis project focuses on the evaluation and analysis of the effectiveness of deep learning models in predicting log events. It explores the application of stacked long short-term memory(LSTM) based model in capturing temporal dependencies and patterns within log event data. In addition, it investigates the probability distribution of the next event from the logs and estimates event trigger time to predict the future node restart event. This thesis project aims to improve the node availability time in base station of Ericsson and contribute to further application in log event prediction using deep learning techniques. A framework with two main phases is utilized to analyze and predict the occurrence of restart events based on the sequence of events. In the first phase, we perform natural language processing(NLP) on the log content to obtain the log key, and then identify the sequence that will cause the restart event from the sequence node events. In the second phase, we analyze these sequence of events which resulted in restart, and predict how many minutes in the future the restart event will occur. Experiment results show that our framework achieves no less than 73% accuracy on restart prediction and more than 1.5 minutes lead time on restart. Moreover, our framework also performs well for non-restart events. / Eftersom många loggspårningsapplikationer och diagnostiska kommandon nu finns tillgängliga på noder vid basstationen, kan händelseloggar enkelt samlas in, analyseras och struktureras för analys av nätverksprestanda. För att förbättra kundnätverkets In Service Performance kan en sekventiell maskininlärningsmodell tränas, testas och distribueras på varje nod för att lära av tidigare händelser för att förutsäga framtida krascher eller ett fel. Detta examensarbete fokuserar på utvärdering och analys av effektiviteten hos modeller för djupinlärning för att förutsäga logghändelser. Den utforskar tillämpningen av staplade långtidsminne (LSTM)-baserad modell för att fånga tidsmässiga beroenden och mönster i logghändelsedata. Dessutom undersöker den sannolikhetsfördelningen för nästa händelse från loggarna och uppskattar händelseutlösningstiden för att förutsäga den framtida omstartshändelsen för noden. Detta examensarbete syftar till att förbättra nodtillgänglighetstiden i Ericssons basstation och bidra till ytterligare tillämpning inom logghändelseprediktion med hjälp av djupinlärningstekniker. Ett ramverk med två huvudfaser används för att analysera och förutsäga förekomsten av omstartshändelser baserat på händelseförloppet. I den första fasen utför vi naturlig språkbehandling (NLP) på logginnehållet för att erhålla loggnyckeln och identifierar sedan sekvensen som kommer att orsaka omstartshändelsen från sekvensnodhändelserna. I den andra fasen analyserar vi dessa händelseförlopp som resulterade i omstart och förutsäger hur många minuter i framtiden omstartshändelsen kommer att inträffa. Experimentresultat visar att vårt ramverk uppnår inte mindre än 73% noggrannhet vid omstartsförutsägelse och mer än 1,5 minuters ledtid vid omstart. Dessutom fungerar vårt ramverk bra för händelser som inte startar om.

Construction equipment manufacturers want to reduce the downtime of their equipment by moving from the typical reactive maintenance to a predictive maintenance approach. They would like to define a method to predict the failure of the construction equipment ahead of time by leveraging the real- world data that is being logged by their vehicles. This data is logged as general event data and specific sensor data belonging to different components of the vehicle. For the scope of this study, the focus is on articulated hauler vehicles with engine as the specific component under observation. In the study, extensive time and resources are spent on preparing both the real- world data sources and coming up with methods such that both data sources are ready for predictive maintenance and can also be merged together. The prepared data is used to build respective remaining useful life machine learning models which classify whether there will be a failure in the next x days. These models are built using data from two different approaches namely, lead data shift and resampling approach respectively. Three different experiments are carried out for both of these approaches using three different combinations of data namely event log only, engine sensor log only, event and sensor log combined. All these experiments have an increasing look ahead window size of how far into the future we would like to predict the failure. The results of these experiments are evaluated in relation to which is the best approach, data combination, and window size to foresee engine failures. The model performance is primarily distinguished by the F- Score and Area under Precision- Recall Curve. / Tillverkare av anläggningsutrustning vill minska stilleståndstiden för sin utrustning genom att övergå från det typiska reaktiva underhållet till ett förebyggande underhåll. De vill definiera en metod för att förutse fel på byggutrustningen i förväg genom att utnyttja de verkliga data som loggas av fordonen. Dessa data loggas som allmänna händelsedata och specifika sensordata som tillhör olika komponenter i fordonet. I den här studien ligger fokus på ledade dragfordon med motorn som den specifika komponent som observeras. I studien läggs mycket tid och resurser på att förbereda båda datakällorna i den verkliga världen och att ta fram metoder så att båda datakällorna är redo för förebyggande underhåll och kan slås samman. De förberedda uppgifterna används för att bygga maskininlärnings modeller för återstående livslängd som klassificerar om det kommer att ske ett fel inom de närmaste x dagarna. Modellerna byggs upp med hjälp av data från två olika metoder, nämligen lead data shift och resampling approach. Tre olika experiment utförs för båda dessa metoder med tre olika kombinationer av data, nämligen endast händelselogg, endast motorsensorlogg och kombinerad händelselogg och sensorlogg. Alla dessa experiment har en ökande fönsterstorlek för hur långt in i framtiden vi vill förutsäga felet. Resultaten av dessa experiment utvärderas med avseende på vilket tillvägagångssätt, vilken datakombination och vilken fönsterstorlek som är bäst för att förutse motorhaverier. Modellens prestanda bedöms i första hand med hjälp av F- poäng och arean under Precision- Recall- kurvan.