Novel clustering methods are presented and applied to financial data. First, a scan-statistics method for detecting price point clusters in financial transaction data is considered. The method is applied to Electronic Business Transfer (EBT) transaction data of the Supplemental Nutrition Assistance Program (SNAP). For a given vendor, transaction amounts are fit via maximum likelihood estimation which are then converted to the unit interval via a natural copula transformation. Next, a new Markov type relation for order statistics on the unit interval is developed. The relation is used to characterize the distribution of the minimum exceedance of all copula transformed transaction amounts above an observed order statistic. Conditional on observed order statistics, independent and asymptotically identical indicator functions are constructed and the success probably as a function of the gaps in consecutive order statistics is specified. The success probabilities are shown to be a function of the hazard rate of the transformed transaction distribution. If gaps are smaller than expected, then the corresponding indicator functions are more likely to be one. A scan statistic is then applied to the sequence of indicator functions to detect locations where too many gaps are smaller than expected. These sets of gaps are then flagged as being anomalous price point clusters. It is noted that prominent price point clusters appearing in the data may be a historical vestige of previous versions of the SNAP program involving outdated paper "food stamps". The second part of the project develops a novel clustering method whereby the time series of daily total EBT transaction amounts are clustered by periodicity. The schemeworks by normalizing the time series of daily total transaction amounts for two distinct vendors and taking daily differences in those two series. The difference series is then examined for periodicity via a novel F statistic. We find one may cluster the monthly periodicities of vendors by type of store using the F statistic, a proxy for a distance metric. This may indicate that spending preferences for SNAP benefit recipients varies by day of the month, however, this opens further questions about potential forcing mechanisms and the apparent changing appetites for spending.

No description available.

No description available.

No description available.

No description available.

No description available.

This thesis aims to uncover anomalies in the data describing the performance behavior of a "robot controller" as measured by software metrics. The purpose of analyzing data is mainly to identify the changes that have resulted in different performance behaviors which we refer to as performance anomalies. To address this issue, two separate pre-processing approaches have been developed: one that adds the principal component to the data after cleaning steps and another that does not regard the principal component. Next, Isolation Forest is employed, which uses an ensemble of isolation trees for data points to segregate anomalies and generate scores that can be used to discover anomalies. Further, in order to detect anomalies, the highest distances matching cluster centroids are employed in the clustering procedure. These two data preparation methods, along with two anomaly detection algorithms, identified software builds that are very likely to be anomalies. According to an industrial evaluation conducted based on engineers’ domain knowledge, around 70% of the detected software builds as anomalous builds were successfully identified, indicating system variable deviations or software bugs.

Knowing when one does not know is crucial in decision making. By estimating uncertainties humans can recognize novelty both by intuition and reason, but most AI systems lack this self-reflective ability. In anomaly detection, a common approach is to train a model to learn the distinction between some notion of normal and some notion of anomalies. In contrast, we let the models build their own notion of normal by learning directly from the data in a self-supervised manner, and by introducing estimations of model uncertainty the models can recognize themselves when novel situations are encountered. In our work, the aim is to investigate the relationship between model uncertainty and anomalies in time series data. We develop a method based on a recurrent autoencoder approach, and we design an anomaly score function that aggregates model error with model uncertainty to indicate anomalies. Use the Monte Carlo Dropout as Bayesian approximation to derive model uncertainty. Asa proof of concept we evaluate our method qualitatively on real-world complex time series using stock market data. Results show that our method can identify extreme events in the stock market. We conclude that the relation between model uncertainty and anomalies can be utilized for anomaly detection in time series data.

This Master's Thesis focuses on the recent Cortical Learn-ing Algorithm (CLA), designed for temporal anomaly detection. It is here applied to the problem of anomaly detec-tion in user behavior of web services, which is getting moreand more important in a network security context. CLA is here compared to more traditional state-of-the-art algorithms of anomaly detection: Hidden Markov Models (HMMs) and t-stide (an N-gram-based anomaly detector), which are among the few algorithms compatible withthe online processing constraint of this problem. It is observed that on the synthetic dataset used forthis comparison, CLA performs signicantly better thanthe other two algorithms in terms of precision of the detection. The two other algorithms don't seem to be able tohandle this task at all. It appears that this anomaly de-tection problem (outlier detection in short sequences overa large alphabet) is considerably different from what hasbeen extensively studied up to now.

Mechanical devices in industriy are equipped with numerous sensors to capture thehealth state of the machines. The reliability of the machine’s health system depends on thequality of sensor data. In order to predict the health state of sensors, abnormal behaviourof sensors must be detected to avoid unnecessary cost.We proposed LSTM autoencoder in which the objective is to reconstruct input time seriesand predict the next time instance based on historical data, and we evaluate anomaliesin multivariate time series via reconstructed error. We also used exponential moving averageas a preprocessing step to smooth the trend of time series to remove high frequencynoise and low frequency deviation in multivariate time series data.Our experiment results, based on different datasets of multivariate time series of gasturbines, demonstrate that the proposed model works well for injected anomalies and realworld data to detect the anomaly. The accuracy of the model under 5 percent infectedanomalies is 98.45%.