Information measures may be used to estimate the amount of information emitted by discrete information sources. Network streams are an example for such discrete information sources. This thesis investigates the use of information measures for the detection of events in network streams. Starting with the fundamental entropy and complexity measures proposed by Shannon and Kolmogorov, it reviews a range of candidate information measures for network event detection, including algorithms from the Lempel-Ziv family and a relative newcomer, the T-entropy. Using network trace data from the University of Auckland, the thesis demonstrates experimentally that these measures are in principle suitable for the detection of a wide range of network events. Several key parameters influence the detectability of network events with information measures. These include the amount of data considered in each traffic sample and the choice of observables. Among others, a study of the entropy behaviour of individual observables in event and non-event scenarios investigates the optimisation of these parameters. The thesis also examines the impact of some of the detected events on different information measures. This motivates a discussion on the sensitivity of various measures. A set of experiments demonstrating multi-dimensional network event classification with multiple observables and multiple information measures concludes the thesis.

Information measures may be used to estimate the amount of information emitted by discrete information sources. Network streams are an example for such discrete information sources. This thesis investigates the use of information measures for the detection of events in network streams. Starting with the fundamental entropy and complexity measures proposed by Shannon and Kolmogorov, it reviews a range of candidate information measures for network event detection, including algorithms from the Lempel-Ziv family and a relative newcomer, the T-entropy. Using network trace data from the University of Auckland, the thesis demonstrates experimentally that these measures are in principle suitable for the detection of a wide range of network events. Several key parameters influence the detectability of network events with information measures. These include the amount of data considered in each traffic sample and the choice of observables. Among others, a study of the entropy behaviour of individual observables in event and non-event scenarios investigates the optimisation of these parameters. The thesis also examines the impact of some of the detected events on different information measures. This motivates a discussion on the sensitivity of various measures. A set of experiments demonstrating multi-dimensional network event classification with multiple observables and multiple information measures concludes the thesis.

Thesis (M.S.)--Oregon State University, 2008. / Printout. Includes bibliographical references (leaves 53-57). Also available on the World Wide Web.

Thesis (Ph. D.)--University of Alabama at Birmingham, 2007. / Additional advisors: Elliot J. Lefkowitz, Kevin D. Reilly, Robert Thacker, Chengcui Zhang. Description based on contents viewed Feb. 7, 2008; title from title screen. Includes bibliographical references (p. 125-132).

Dissertation (Ph.D.)--Auburn University, 2006. / Abstract. Vita. Includes bibliographic references (p.149-156).

The Internet has become an indispensable resource for today's society. It is at the center of the today's business, entertainment, and social world. However, the core of our identities on the Internet, the IP addresses that are used to send and receive data throughout the Internet, are insecure. Attackers today are able to send data purporting to be from nearly any location (IP spoofing) and to reroute data destined for victims to the attackers themselves (IP prefix hijacking). Victims of these attacks may experience denial of service, misplaced blame, and theft of their traffic. These attacks are of the utmost importance since they affect the core layer of the Internet. Although the mechanisms of the attacks are different, they are essentially different sides of the same coin; spoofing attacks forge the identity of the sender, while hijacking attacks forge the identity of the receiver. They revolve around the same underlying lack of a secure identity on the Internet. This research reviews the existing state of the art IP spoofing and IP prefix hijacking research and proposes new defenses to close the missing gaps and provide a new level of security to our identities on the Internet. This material is based upon work supported by the National Science Foundation under Grants No. CNS-0520326 and CNS-1118101. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. This dissertation includes both previously published/unpublished and co-authored material.

Where the role of software-intensive systems has shifted from the traditional one of fulfilling isolated computational tasks, larger collaborative societies with interaction as primary resource, is gradually taking its place. This can be observed in anything from logistics to rescue operations and resource management, numerous services with key-roles in the modern infrastructure. In the light of this new collaborative order, it is imperative that the tools (compilers, debuggers, profilers) and methods (requirements, design, implementation, testing) that supported traditional software engineering values also adjust and extend towards those nurtured by the online instrumentation of software intensive systems. That is, to adjust and to help to avoid situations where limitations in technology and methodology would prevent us from ascertaining the well-being and security of systems that assists our very lives. Coupled with most perspectives on software development and maintenance is one well established member of, and complement to, the development process. Debugging; or the art of discovering, localising, and correcting undesirable behaviours in software-intensive systems, the need for which tend to far outlive development in itself. Debugging is currently performed based on a premise of the developer operating from a god-like perspective. A perspective that implies access and knowledge regarding source code, along with minute control over execution properties. However, the quality as well as accessibility of such information steadily decline with time as requirements, implementation, hardware components and their associated developers, all alike fall behind their continuously evolving surroundings. In this thesis, it is argued that the current practice of software debugging is insufficient, and as precursory action, introduce a technical platform suitable for experimenting with future methods regarding online debugging, maintenance and analysis. An initial implementation of this platform will then be used for experimenting with a simple method that is targeting online observation of software behaviour.

Modern automobiles have been proven vulnerable to hacking by security researchers. By exploiting vulnerabilities in the car's external interfaces, attackers can access a car's controller area network (CAN) bus and cause malicious effects. We seek to detect these attacks on the bus as a last line of defence against automotive cyber attacks. The CAN bus standard defines a low-level message structure, upon which manufacturers layer their own proprietary command protocols; attacks must similarly be tailored for their target. This variability makes intrusion detection methods difficult to apply to the automotive CAN bus. Nevertheless, the bus traffic is generated by machines; thus we hypothesize that it can be characterized with machine learning, and that attacks produce anomalous traffic. Our goals are to show that anomaly detection trained without understanding of the message contents can detect attacks, and to create a framework for understanding how the characteristics of a novel attack can be used to predict its detectability. We developed a model that describes attacks based on their effect on bus traffic, informed by a review of published material on car hacking in combination with analysis of CAN traffic from a 2012 Subaru Impreza. The model specifies three high-level categories of effects: attacks that insert foreign packets, attacks that affect packet timing, and attacks that only modify data within packets. Foreign packet attacks are trivially detectable. For timing-based anomalies, we developed features suitable for one-class classification methods. For packet stream data word anomalies, we adapted recurrent neural networks and multivariate Markov model methods to sequence anomaly detection and compared their performance. We conducted experiments to evaluate our detection methods with special attention to the trade-off between precision and recall, given that a practical system requires a very low false alarm rate. The methods were evaluated by synthesizing anomalies within each attack category, parameterized to adjust their covertness. We generalize from the results to enable prediction of detection rates for new attacks using these methods.

The ability to diagnose deviations and predict faults effectively is an important task in various industrial domains for minimizing costs and productivity loss and also conserving environmental resources. However, the majority of the efforts for diagnostics are still carried out by human experts in a time-consuming and expensive manner. Automated data-driven solutions are needed for continuous monitoring of complex systems over time. On the other hand, domain expertise plays a significant role in developing, evaluating, and improving diagnostics and monitoring functions. Therefore, automatically derived solutions must be able to interact with domain experts by taking advantage of available a priori knowledge and by incorporating their feedback into the learning process. This thesis and appended papers tackle the problem of generating a real-world self-monitoring system for continuous monitoring of machines and operations by developing algorithms that can learn data streams and their relations over time and detect anomalies using joint-human machine learning. Throughout this thesis, we have described a number of different approaches, each designed for the needs of a self-monitoring system, and have composed these methods into a coherent framework. More specifically, we presented a two-layer meta-framework, in which the first layer was concerned with learning appropriate data representations and detectinganomalies in an unsupervised fashion, and the second layer aimed at interactively exploiting available expert knowledge in a joint human-machine learning fashion. Furthermore, district heating has been the focus of this thesis as the application domain with the goal of automatically detecting faults and anomalies by comparing heat demands among different groups of customers. We applied and enriched different methods on this domain, which then contributed to the development and improvement of the meta-framework. The contributions that result from the studies included in this work can be summarized into four categories: (1) exploring different data representations that are suitable for the self-monitoring task based on data characteristics and domain knowledge, (2) discovering patterns and groups in data that describe normal behavior of the monitored system/systems, (3) implementing methods to successfully discriminate anomalies from the normal behavior, and (4) incorporating domain knowledge and expert feedback into self-monitoring.

District heating delivers more than 70% of the energy used for heating and domestichot water in Swedish buildings. To stay competitive, district heating needs toreduce its losses and increase capabilities to utilise low grade heat. Finding faultysubstations is one way to allow reductions in supply temperatures in district heatingnetworks, which in turn can help reduce the losses. In this work three suggestedsymptoms of faults: abnormal quantization, drifting and anomalous values, are investigatedwith the help of hourly meter data of: heat load, volume flow, supplyand return temperatures from district heating substations. To identify abnormalquantization, a method is proposed based on Shannon’s entropy, where lower entropysuggests higher risk of abnormal quantization. The majority of the substationsidentified as having abnormal quantization with the proposed method has a meterresolution lower than the majority of the substations in the investigated districtheating network. This lower resolution is likely responsible for identifying thesesubstation, suggesting the method is limited by the meter resolution of the availabledata. To improve result from the method higher resolution and sampling frequencyis likely needed.For identifying drift and anomalous values two methods are proposed, one for eachsymptom. Both methods utilize a software for predicting hourly heat load, volumeflow, supply and return temperatures in individual district heating substations.The method suggested for identifying drift uses the mean value of each predictedand measured quantity during the investigated period. The mean of the prediction iscompared to the mean of the measured values and a large difference would suggestrisk of drift. However this method has not been evaluated due to difficulties infinding a suitable validation method.The proposed method for detecting anomalous values is based on finding anomalousresiduals when comparing the prediction from the prediction software to themeasured values. To find the anomalous residuals the method uses an anomalydetection algorithm called IsolationForest. The method produces rankable lists inwhich substations with risk of anomalies are ranked higher in the lists. Four differentlists where evaluated by an experts. For the two best preforming lists approximatelyhalf of the top 15 substations where classified to contain anomalies by the expertgroup. The proposed method for detecting anomalous values shows promising resultespecially considering how easily the method could be added to a district heatingnetwork. Future work will focus on reducing the number of false positives. Suggestionsfor lowering the false positive rate include, alternations or checks on theprediction models used.