Abnormal behaviour may indicate important objects and events in a wide variety of domains. One such domain is intelligence and surveillance, where there is a clear trend towards more and more advanced sensor systems producing huge amounts of trajectory data from moving objects, such as people, vehicles, vessels and aircraft. In the maritime domain, for example, abnormal vessel behaviour, such as unexpected stops, deviations from standard routes, speeding, traffic direction violations etc., may indicate threats and dangers related to smuggling, sea drunkenness, collisions, grounding, hijacking, piracy etc. Timely detection of these relatively infrequent events, which is critical for enabling proactive measures, requires constant analysis of all trajectories; this is typically a great challenge to human analysts due to information overload, fatigue and inattention. In the Baltic Sea, for example, there are typically 3000–4000 commercial vessels present that are monitored by only a few human analysts. Thus, there is a need for automated detection of abnormal trajectory patterns. In this thesis, we investigate algorithms appropriate for automated detection of anomalous trajectories in surveillance applications. We identify and discuss some key theoretical properties of such algorithms, which have not been fully addressed in previous work: sequential anomaly detection in incomplete trajectories, continuous learning based on new data requiring no or limited human feedback, a minimum of parameters and a low and well-calibrated false alarm rate. A number of algorithms based on statistical methods and nearest neighbour methods are proposed that address some or all of these key properties. In particular, a novel algorithm known as the Similarity-based Nearest Neighbour Conformal Anomaly Detector (SNN-CAD) is proposed. This algorithm is based on the theory of Conformal prediction and is unique in the sense that it addresses all of the key properties above. The proposed algorithms are evaluated on real world trajectory data sets, including vessel traffic data, which have been complemented with simulated anomalous data. The experiments demonstrate the type of anomalous behaviour that can be detected at a low overall alarm rate. Quantitative results for learning and classification performance of the algorithms are compared. In particular, results from reproduced experiments on public data sets show that SNN-CAD, combined with Hausdorff distance  for measuring dissimilarity between trajectories, achieves excellent classification performance without any parameter tuning. It is concluded that SNN-CAD, due to its general and parameter-light design, is applicable in virtually any anomaly detection application. Directions for future work include investigating sensitivity to noisy data, and investigating long-term learning strategies, which address issues related to changing behaviour patterns and increasing size and complexity of training data.

Anomaly detection is a general theory of detecting unusual patterns or events in data. This master thesis investigates the subject of anomaly detection in two different applications. The first application is product inspection using a camera and the second application is surveillance using a 2D laser scanner. The first part of the thesis presents a system for automatic visual defect inspection. The system is based on aligning the images of the product to a common template and doing pixel-wise comparisons. The system is trained using only images of products that are defined as normal, i.e. non-defective products. The visual properties of the inspected products are modelled using three different methods. The performance of the system and the different methods have been evaluated on four different datasets. The second part of the thesis presents a surveillance system based on a single laser range scanner. The system is able to detect certain anomalous events based on the time, position and velocities of individual objects in the scene. The practical usefulness of the system is made plausible by a qualitative evaluation using unlabelled data.

Human operators of modern surveillance systems are confronted with an increasing amount of trajectory data from moving objects, such as people, vehicles, vessels, and aircraft. A large majority of these trajectories reflect routine traffic and are uninteresting. Nevertheless, some objects are engaged in dangerous, illegal or otherwise interesting activities, which may manifest themselves as unusual and abnormal trajectories. These anomalous trajectories can be difficult to detect by human operators due to cognitive limitations. In this thesis, we study algorithms for the automated detection of anomalous trajectories in surveillance applications. The main results and contributions of the thesis are two-fold. Firstly, we propose and discuss a novel approach for anomaly detection, called conformal anomaly detection, which is based on conformal prediction (Vovk et al.). In particular, we propose two general algorithms for anomaly detection: the conformal anomaly detector (CAD) and the computationally more efficient inductive conformal anomaly detector (ICAD). A key property of conformal anomaly detection, in contrast to previous methods, is that it provides a well-founded approach for the tuning of the anomaly threshold that can be directly related to the expected or desired alarm rate. Secondly, we propose and analyse two parameter-light algorithms for unsupervised online learning and sequential detection of anomalous trajectories based on CAD and ICAD: the sequential Hausdorff nearest neighbours conformal anomaly detector (SHNN-CAD) and the sequential sub-trajectory local outlier inductive conformal anomaly detector (SSTLO-ICAD), which is more sensitive to local anomalous sub-trajectories. We implement the proposed algorithms and investigate their classification performance on a number of real and synthetic datasets from the video and maritime surveillance domains. The results show that SHNN-CAD achieves competitive classification performance with minimum parameter tuning on video trajectories. Moreover, we demonstrate that SSTLO-ICAD is able to accurately discriminate realistic anomalous vessel trajectories from normal background traffic.

The goal of this thesis is to explore some variations of neural networks. The thesis is mainly split into two parts: a variation of the shaping functions in neural networks and a variation of learning rules in neural networks. In the first part, we mainly investigate polynomial perceptrons - a perceptron with a polynomial shaping function instead of a linear one. We prove the polynomial perceptron convergence theorem and illustrate the notion by showing that a higher order perceptron can learn the XOR function through empirical experiments with implementation. In the second part, we propose three models (SMLP, SA, SA2) for stream learning and anomaly detection in streams. The main technique allowing these models to perform at a level comparable to the state-of-the-art algorithms in stream learning is the learning rule used. We employ mini-batch gradient descent algorithm and stochastic gradient descent algorithm to speed up the models. In addition, the use of parallel processing with multi-threads makes the proposed methods highly efficient in dealing with streaming data. Our analysis shows that all models have linear runtime and constant memory requirement. We also demonstrate empirically that the proposed methods feature high detection rate, low false alarm rate, and fast response. The paper on the first two models (SMLP, SA) is published in the 29th Canadian AI Conference and won the best paper award. The invited journal paper on the third model (SA2) for Computational Intelligence is under peer review.

This paper discusses the use of diagnostics based on machine learning (ML) within a flight test context. The paper begins by discussing some of the problems associated with instrumenting a test aircraft and how they could be ameliorated using ML-based diagnostics. We then describe a number of types of supervised ML algorithms which can be used in this context. In addition, key practical aspects of applying these algorithms, such as feature engineering and parameter selection, are also discussed. The paper then outlines a real-world application developed by Curtiss-Wright, called Machine Learning for Advanced System Diagnostics (MLASD). This description includes key challenges that were encountered during the development process and how suitable input features were identified. Real-world results are also presented. Finally, we suggest some further applications of ML techniques, in addition to describing other areas of development.

In several applications, when anomalies are detected, human experts have to investigate or verify them one by one. As they investigate, they unwittingly produce a label - true positive (TP) or false positive (FP). In this thesis, we propose two methods (PAD and Clustering-based OMD/OJRank) that exploit this label feedback to minimize the FP rate and detect more relevant anomalies, while minimizing the expert effort required to investigate them. These two methods iteratively suggest the top-1 anomalous instance to a human expert and receive feedback. Before suggesting the next anomaly, the methods re-ranks instances so that the top anomalous instances are similar to the TP instances and dissimilar to the FP instances. This is achieved by learning to score anomalies differently in various regions of the feature space (OMD-Clustering) and by learning to score anomalies based on the distance to the real anomalies (PAD). An experimental evaluation on several real-world datasets is conducted. The results show that OMD-Clustering achieves statistically significant improvement in both detection precision and expert effort compared to state-of-the-art interactive anomaly detection methods. PAD reduces expert effort but there was no improvement in detection precision compared to state-of-the-art methods. We submitted a paper based on the work presented in this thesis, to the ECML/PKDD Workshop on "IoT Stream for Data Driven Predictive Maintenance".

Les réseaux sans fils et mobiles se sont énormément développés au cours de ces dernières années. Loin d'être réservés aux pays industrialisés, ces réseaux nécessitant une infrastructure fixe limitée se sont aussi imposés dans les pays émergents et les pays en voie de développement. En effet, avec un investissement structurel relativement très faible en comparaison de celui nécessaire à l'implantation d'un réseau filaire, ces réseaux permettent aux opérateurs d'offrir une couverture du territoire très large, avec un coût d'accès au réseau (prix du téléphone et des communications) tout à fait acceptable pour les utilisateurs. Aussi, il n'est pas surprenant qu'aujourd'hui, dans la majorité des pays, le nombre de téléphones sans fil soit largement supérieur à celui des téléphones fixes. Ce grand nombre de terminaux disséminé sur l'ensemble de la planète est un réservoir inestimable d'information dont une infime partie seulement est aujourd'hui exploitée. En effet, en combinant la position d'un mobile et sa vitesse de déplacement, il devient possible d'en déduire la qualité des routes ou du trafic routier. Dans un autre registre, en intégrant un thermomètre et/ou un hygromètre dans chaque terminal, ce qui à grande échelle impliquerait un coût unitaire dérisoire, ces terminaux pourraient servir de relai pour une météo locale plus fiable. Dans ce contexte, l'objectif de cette thèse consiste à étudier et analyser les opportunités offertes par l'utilisation des données issues des terminaux mobiles, de proposer des solutions originales pour le traitement de ces grands masses de données, en insistant sur les optimisations (fusion, agrégation, etc.) pouvant être réalisées de manière intermédiaire dans le cadre de leur transport vers les(s) centre(s) de stockage et de traitement, et éventuellement d'identifier les données non disponibles aujourd'hui sur ces terminaux mais qui pourraient avoir un impact fort dans les années à venir. Un prototype présentant un exemple typique d'utilisation permettra de valider les différentes approches. / Mobile and wireless networks have developed enormously over the recent years. Far from being restricted to industrialized countries, these networks which require a limited fixed infrastructure, have also imposed in emerging countries and developing countries. Indeed, with a relatively low structural investment as compared to that required for the implementation of a wired network, these networks enable operators to offer a wide coverage of the territory with a network access cost (price of devices and communications) quite acceptable to users. Also, it is not surprising that today, in most countries, the number of wireless phones is much higher than landlines. This large number of terminals scattered across the planet is an invaluable reservoir of information that only a tiny fraction is exploited today. Indeed, by combining the mobile position and movement speed, it becomes possible to infer the quality of roads or road traffic. On another level, incorporating a thermometer and / or hygrometer in each terminal, which would involve a ridiculous large-scale unit cost, these terminals could serve as a relay for more reliable local weather. In this context, the objective of this thesis is to study and analyze the opportunities offered by the use of data from mobile devices to offer original solutions for the treatment of these big data, emphasizing on optimizations (fusion, aggregation, etc.) that can be performed as an intermediate when transferred to center(s) for storage and processing, and possibly identify data which are not available now on these terminals but could have a strong impact in the coming years. A prototype including a typical sample application will validate the different approaches.

In South Africa, nearly 80 % of electricity is generated from coal fired power plants. Due to the complexity of the interconnected systems that make up a typical power plant, analysis of the root causes of load losses is not a straightforward process. This often leads to losses incorrectly being ascribed to the Induced Draught (ID) fan, where detection occurs, while the problem actually originates elsewhere in the plant. The focus of this study was to develop and demonstrate a modelling methodology to quantify the effects of major plant anomalies on the capacity of ID fans in coal fired power plants. The ensuing model calculates the operating point of the ID fan that is a result of anomalies experienced elsewhere in the plant. This model can be applied in conjunction with performance test data as part of a root cause analysis procedure. The model has three main sections that are integrated to determine the ID fan operating point. The first section is a water/steam cycle model that was pre-configured in VirtualPlantTM. The steam plant model was verified via energy balance calculations and validated against original heat balance diagrams. The second is a draught group model developed using FlownexSETM. This onedimensional network is a simplification of the flue gas side of the five main draught group components, from the furnace inlet to the chimney exit, characterising only the aggregate heat transfer and pressure loss in the system. The designated ID fan model is based on the original fan performance curves. The third section is a Boiler Mass and Energy Balance (BMEB) specifically created for this purpose to: (1) translate the VirtualPlant results for the steam cycle into applicable boundary conditions for the Flownex draught group model; and (2) to calculate the fluid properties applicable to the draught group based on the coal characteristics and combustion process. The integrated modelling methodology was applied to a 600 MW class coal fired power plant to investigate the impact of six major anomalies that are typically encountered. These are: changes in coal quality; increased boiler flue gas exit temperatures; air ingress into the boiler; air heater inleakage to the flue gas stream; feed water heaters out-of-service; and condenser backpressure degradation. It was inter alia found that a low calorific value (CV) coal of 14 MJ/kg compared to a typical 17 MJ/kg reduced the fan's capacity by 2.1 %. Also, having both HP FWH out of service decreased the fan's capacity by 16.2 %.

No description available.

This paper deals with a problem of detection of network traffic anomaly and classification of network flows. Based on existing methods, paper describes proposal and implementaion of a tool, which can automatically classify network flows. The tool uses CUDA platform for network data processing and computation of network flow metrics using graphics processing unit. Processed flows are subsequently classified by proposed methods for network anomaly detection.