Spelling suggestions: "subject:"anline anomaly detection"" "subject:"0nline anomaly detection""
1 |
Online Anomaly Detection for Time Series. Towards Incorporating Feature Extraction, Model Uncertainty and Concept Drift Adaptation for Improving Anomaly DetectionTambuwal, Ahmad I. January 2021 (has links)
Time series anomaly detection receives increasing research interest given
the growing number of data-rich application domains. Recent additions
to anomaly detection methods in research literature include deep learning
algorithms. The nature and performance of these algorithms in sequence
analysis enable them to learn hierarchical discriminating features
and time-series temporal nature. However, their performance is affected
by the speed at which the time series arrives, the use of a fixed threshold,
and the assumption of Gaussian distribution on the prediction error
to identify anomalous values. An exact parametric distribution is often
not directly relevant in many applications and it’s often difficult to select
an appropriate threshold that will differentiate anomalies with noise.
Thus, implementations need the Prediction Interval (PI) that quantifies the
level of uncertainty associated with the Deep Neural Network (DNN) point
forecasts, which helps in making a better-informed decision and mitigates
against false anomaly alerts. To achieve this, a new anomaly detection
method is proposed that computes the uncertainty in estimates using quantile
regression and used the quantile interval to identify anomalies. Similarly,
to handle the speed at which the data arrives, an online anomaly detection
method is proposed where a model is trained incrementally to adapt
to the concept drift that improves prediction. This is implemented using a
window-based strategy, in which a time series is broken into sliding windows
of sub-sequences as input to the model. To adapt to concept drift,
the model is updated when changes occur in the new arrival instances.
This is achieved by using anomaly likelihood which is computed using the
Q-function to define the abnormal degree of the current data point based
on the previous data points. Specifically, when concept drift occurs, the
proposed method will mark the current data point as anomalous. However,
when the abnormal behavior continues for a longer period of time,
the abnormal degree of the current data point will be low compared to the
previous data points using the likelihood. As such, the current data point is
added to the previous data to retrain the model which will allow the model
to learn the new characteristics of the data and hence adapt to the concept
changes thereby redefining the abnormal behavior. The proposed method
also incorporates feature extraction to capture structural patterns in the
time series. This is especially significant for multivariate time-series data,
for which there is a need to capture the complex temporal dependencies
that may exist between the variables. In summary, this thesis contributes
to the theory, design, and development of algorithms and models for the
detection of anomalies in both static and evolving time series data.
Several experiments were conducted, and the results obtained indicate the
significance of this research on offline and online anomaly detection in
both static and evolving time-series data. In chapter 3, the newly proposed
method (Deep Quantile Regression Anomaly Detection Method) is evaluated
and compared with six other prediction-based anomaly detection
methods that assume a normal distribution of prediction or reconstruction
error for the identification of anomalies. Results in the first part of
the experiment indicate that DQR-AD obtained relatively better precision
than all other methods which demonstrates the capability of the method
in detecting a higher number of anomalous points with low false positive
rates. Also, the results show that DQR-AD is approximately 2 – 3
times better than the DeepAnT which performs better than all the remaining
methods on all domains in the NAB dataset. In the second part of the
experiment, sMAP dataset is used with 4-dimensional features to demonstrate
the method on multivariate time-series data. Experimental result
shows DQR-AD have 10% better performance than AE on three datasets
(SMAP1, SMAP3, and SMAP5) and equal performance on the remaining
two datasets. In chapter 5, two levels of experiments were conducted
basis of false-positive rate and concept drift adaptation. In the first level
of the experiment, the result shows that online DQR-AD is 18% better
than both DQR-AD and VAE-LSTM on five NAB datasets. Similarly, results
in the second level of the experiment show that the online DQR-AD
method has better performance than five counterpart methods with a relatively
10% margin on six out of the seven NAB datasets. This result
demonstrates how concept drift adaptation strategies adopted in the proposed
online DQR-AD improve the performance of anomaly detection in
time series. / Petroleum Technology Development Fund (PTDF)
|
2 |
ANOMALIES IN SENSOR NETWORK DEPLOYMENTS: ANALYSIS, MODELING, AND DETECTIONAbuaitah, Giovani Rimon 20 August 2013 (has links)
No description available.
|
3 |
[en] A MOBILE AND ONLINE OUTLIER DETECTION OVER MULTIPLE DATA STREAMS: A COMPLEX EVENT PROCESSING APPROACH FOR DRIVING BEHAVIOR DETECTION / [pt] DETECÇÃO MÓVEL E ONLINE DE ANOMALIA EM MÚLTIPLOS FLUXOS DE DADOS: UMA ABORDAGEM BASEADA EM PROCESSAMENTO DE EVENTOS COMPLEXOS PARA DETECÇÃO DE COMPORTAMENTO DE CONDUÇÃOIGOR OLIVEIRA VASCONCELOS 24 July 2017 (has links)
[pt] Dirigir é uma tarefa diária que permite uma locomoção mais rápida e mais confortável, no entanto, mais da metade dos acidentes fatais estão relacionados à imprudência. Manobras imprudentes podem ser detectadas com boa precisão, analisando dados relativos à interação motorista-veículo, por exemplo, curvas, aceleração e desaceleração abruptas. Embora existam algoritmos para detecção online de anomalias, estes normalmente são projetados para serem executados em computadores com grande poder computacional. Além disso, geralmente visam escala através da computação paralela, computação em grid ou computação em nuvem. Esta tese apresenta uma abordagem baseada em complex event processing para a detecção online de anomalias e classificação do comportamento de condução. Além disso, objetivamos identificar se dispositivos móveis com poder computacional limitado, como os smartphones, podem ser usados para uma detecção online do comportamento de condução. Para isso, modelamos e avaliamos três algoritmos de detecção online de anomalia no paradigma de processamento de fluxos de dados, que recebem os dados dos sensores do smartphone e dos sensores à bordo do veículo como entrada. As vantagens que o processamento de fluxos de dados proporciona reside no fato de que este reduz a quantidade de dados transmitidos do dispositivo móvel para servidores/nuvem, bem como se reduz o consumo de energia/bateria devido à transmissão de dados dos sensores e possibilidade de operação mesmo se o dispositivo móvel estiver desconectado. Para classificar os motoristas, um mecanismo estatístico utilizado na mineração de documentos que avalia a importância de uma palavra em uma coleção de documentos, denominada frequência de documento inversa, foi adaptado para identificar a importância de uma anomalia em um fluxo de dados, e avaliar quantitativamente o grau de prudência ou imprudência das manobras dos motoristas. Finalmente, uma avaliação da abordagem (usando o algoritmo que obteve melhor resultado na primeira etapa) foi realizada através de um estudo de caso do comportamento de condução de 25 motoristas em cenário real. Os resultados mostram uma acurácia de classificação de 84 por cento e um tempo médio de processamento de 100 milissegundos. / [en] Driving is a daily task that allows individuals to travel faster and more comfortably, however, more than half of fatal crashes are related to recklessness driving behaviors. Reckless maneuvers can be detected with accuracy by analyzing data related to driver-vehicle interactions, abrupt turns, acceleration, and deceleration, for instance. Although there are algorithms for online anomaly detection, they are usually designed to run on computers with high computational power. In addition, they typically target scale through parallel computing, grid computing, or cloud computing. This thesis presents an online anomaly detection approach based on complex event processing to enable driving behavior classification. In addition, we investigate if mobile devices with limited computational power, such as smartphones, can be used for online detection of driving behavior. To do so, we first model and evaluate three online anomaly detection algorithms in the data stream processing paradigm, which receive data from the smartphone and the in-vehicle embedded sensors as input. The advantages that stream processing provides lies in the fact that reduce the amount of data transmitted from the mobile device to servers/the cloud, as well as reduce the energy/battery usage due to transmission of sensor data and possibility to operate even if the mobile device is disconnected. To classify the drivers, a statistical mechanism used in document mining that evaluates the importance of a word in a collection of documents, called inverse document frequency, has been adapted to identify the importance of an anomaly in a data stream, and then quantitatively evaluate how cautious or reckless drivers maneuvers are. Finally, an evaluation of the approach (using the algorithm that achieved better result in the first step) was carried out through a case study of the 25 drivers driving
behavior. The results show an accuracy of 84 percent and an average processing time of 100 milliseconds.
|
Page generated in 0.0939 seconds