The forestry and the sawmill industry have a lot of incoming and outgoing piles of wood. It's important to maintain quality and efficiency. This motivates an examination of whether machine learning- or more specifically, anomaly detection techniques can be implemented and used to detect faulty shipments. This thesis presents and evaluates some computer vision techniques and some deep learning techniques. Deep learning can be divided into groups; supervised, semi-supervised and unsupervised. In this thesis, all three groups were examined and it covers supervised methods such as Convolutional Neural Networks, semi-supervised methods such as a modified Convolutional Autoencoder (CAE) and lastly, an unsupervised technique such as Generative Adversarial Network (GAN) was being tested and evaluated.  A version of a GAN model proved to perform best for this thesis in terms of the accuracy of faulty detecting shipments with an accuracy rate of 68.2% and 79.8\% overall, which was satisfactory given the problems that were discovered during the progress of the thesis.

In this paper, we compare the performance of two methods to find the attempts at fraud from the data provided by Sinch (formerly CLX Communications, which is a telecommunications and cloud communications platform as a service (PaaS) company). We consider the problem as finding the anomaly in a time series signal, where we ignore the duration of a single call or other features and only care about the total volume of calls in a certain period.\\ We compare Seasonal and Trend decomposition using Loess(STL) and auto-encoder-decoder under the scenario to find the anomaly in a certain period. It comes out that additive models like STL can discriminate the trending anomaly. As for auto-encoder-decoder, the anomaly can easily be found using local information, which makes the method conveniently applied. It remains a problem that unsupervised learning methods usually require manual inspection. In practical applications, we need to iterate many times with experts to find the most suitable method for that scenario. / I det här dokumentet jämför vi resultatet av två metoder för att hitta bedrägeriförsöken från data som tillhandahålls av Sinch (tidigare CLX Communications, som är ett telekommunikations- och molnkommunikations-plattform som en tjänst (PaaS)-företag). Vi betraktar problemet som att hitta anomalien i en tidsseriesignal, där vi ignorerar varaktigheten av ett enstaka samtal eller andra funktioner och tar bara hänsyn av den totala volymen samtal under en viss period. Vi jämför säsongs- och trenduppdelning med Loess(STL) och auto-encoder-decoder under scenariot för att hitta anomalien under en viss period. Det visar sig att additivmodeller som STL kan diskriminera trendavvikelsen. När det gäller auto-encoder-decoder, kan avvikelsen lätt hittas med hjälp av lokal information, vilket gör metoden Lämplig att tillämpa. Det är fortfarande ett problem att oövervakade inlärningsmetoder vanligtvis kräver manuell inspektion. I praktiska tillämpningar måste vi iterera många gånger med experter för att hitta den mest lämpliga metoden för det scenariot.

Anomaly detection is the process of detecting samples in a dataset that are atypical or abnormal. Anomaly detection can for example be of great use in an industrial setting, where faults in the manufactured products need to be detected at an early stage. In this setting, the available image data might be from different non-natural domains, such as the depth domain. However, the amount of data available is often limited in these domains. This thesis aims to investigate if a convolutional neural network (CNN) can be trained to perform anomaly detection well on limited datasets in non-natural image domains. The attempted approach is to train the CNN as an autoencoder, in which the CNN is the encoder network. The encoder is then extracted and used as a feature extractor for the anomaly detection task, which is performed using Semantic Pyramid Anomaly Detection (SPADE). The results are then evaluated and analyzed. Two autoencoder models were used in this approach. As the encoder network, one of the models uses a MobileNetV3-Small network that had been pretrained on ImageNet, while the other uses a more basic network, which is a few layers deep and initialized with random weights. Both these networks were trained as regular convolutional autoencoders, as well as variational autoencoders. The results were compared to a MobileNetV3-Small network that had been pretrained on ImageNet, but had not been trained as an autoencoder. The models were tested on six different datasets, all of which contained images from the depth and intensity domains. Three of these datasets additionally contained images from the scatter domain, and for these datasets, the combination of all three domains was tested as well. The main focus was however on the performance in the depth domain. The results show that there is generally an improvement when training the more complex autoencoder on the depth domain. Furthermore, the basic network generally obtains an equivalent result to the more complex network, suggesting that complexity is not necessarily an advantage for this approach. Looking at the different domains, there is no apparent pattern to which domain yields the best performance. This rather seems to depend on the dataset. Lastly, it was found that training the networks as variational autoencoders did generally not improve the performance in the depth domain compared to the regular autoencoders. In summary, an improved anomaly detection was obtained in the depth domain, but for optimal anomaly detection with regard to domain and network, one must look at the individual datasets. / <p>Examensarbetet är utfört vid Institutionen för teknik och naturvetenskap (ITN) vid Tekniska fakulteten, Linköpings universitet</p>

With the rise of information technology and the dependence on these systems, it becomes increasingly more important to keep the systems secure. The possibility to detect an intrusion with intrusion detection systems (IDS) is one of multiple fundamental technologies that may increase the security of a system. One of the bigger challenges of an IDS, is to detect types of intrusions that have previously not been encountered, so called unknown intrusions. These types of intrusions are generally detected by using methods collectively called anomaly detection methods. In this thesis I evaluate the performance of the algorithm Tree Augmented Naive Bayes Classifier (TAN) as an intrusion detection classifier. More specifically, I created a TAN program from scratch in Python and tested the program on two data sets containing data traffic. The thesis aims to create a better understanding of how TAN works and evaluate if it is a reasonable algorithm for intrusion detection. The results show that TAN is able to perform at an acceptable level with a reasonably high accuracy. The results also highlights the importance of using the smoothing operator included in the standard version of TAN. / Med informationsteknikens utveckling och det ökade beroendet av dessa system, blir det alltmer viktigt att hålla systemen säkra. Intrångsdetektionssystem (IDS) är en av många fundamentala teknologier som kan öka säkerheten i ett system. En av de större utmaningarna inom IDS, är att upptäcka typer av intrång som tidigare inte stötts på, så kallade okända intrång. Dessa intrång upptäcks oftast med hjälp av metoder som kollektivt kallas för avvikelsedetektionsmetoder. I denna uppsats utvärderar jag algoritmen Tree Augmented Naive Bayes Classifiers (TAN) prestation som en intrångsdetektionsklassificerare. Jag programmerade ett TAN-program, i Python, och testade detta program på två dataset som innehöll datatrafik. Denna uppsats ämnar att skapa en bättre förståelse för hur TAN fungerar, samt utvärdera om det är en lämplig algoritm för detektion av intrång. Resultaten visar att TAN kan prestera på en acceptabel nivå, med rimligt hög noggrannhet. Resultaten markerar även betydelsen av "smoothing operator", som inkluderas i standardversionen av TAN.

For manufacturing enterprises, the potential of collecting large amounts of data from production processes has enabled the usage of machine learning for prediction-based monitoring and maintenance of machines. Yet common maintenance strategies still include reactive handling of machine failures or schedule-based maintenance conducted by experienced personnel. Both of which are time-consuming and costly for manufacturing enterprises. The incorporation of anomaly detection for production processes alleviates several problems connected to these resource-intensive maintenance strategies. Anomaly detection enables real-time maintenance alarms derived from the occurrence of anomalies and thereby a foundation for proactive maintenance during manufacturing. However, to realize this, one needs to investigate the correlation between machine failure and anomalies in the data. For the machine learning models, it is also of essence to handle the imbalance between failure and normal working condition data. In this work, we investigate the potential of anomaly detection to predict future tool failures of an active CNC-machine based on multivariate time series data collected through the standardized data collection protocol MTConnect. Two semi-supervised anomaly detection methods, DeepAnT and ROCKET OCSVM, were tested. Training and evaluation of the two models were conducted on three production part processes and the difference in anomaly distribution previous to failure and in the normal machine working condition was investigated. The results showed that both models, for all the investigated tool failures belonging to the three production part processes, found an abundance of anomalies preceding failure when compared to the normal working condition of the machines. For certain tool failures, the anomalies were found as far back as seven production cycles before failure, while other anomalies were mainly uncovered close to the failure. Furthermore, it was shown that both models performed optimally with 100 production cycles before tool failures excluded from training, indicating that more anomalies further back connected to failure or possible long-term degradation of machine tools could exist. Lastly, ROCKET OCSVM with RBF kernel showed greater reliability compared to the DeepAnT method in separating the normal working condition data of the CNC machine against the pre-failure data based on anomaly distribution. In conclusion, anomaly detection shows promising results in indicating future machine failure and could serve as a foundation for proactive maintenance strategies of machines. By incorporating proactive strategies, machine downtime, operator maintenance time, and resources and expenses resulting from machine failure could be reduced. / För produktionsföretag har potentialen att samla in stora mängder data från produktionsprocesser möjliggjort användningen av prediktionsbaserad övervakning och underhåll av maskiner genom maskininlärning. Ändå så utgörs fortfarande vanliga underhållsstrategier av reaktiv hantering av maskinfel eller schema baserat underhåll som utförs av erfaren personal. Båda dessa är tidskrävande och kostsamma för tillverkningsföretag. Införandet av anomali detektering för produktionsprocesser lindrar flera problem kopplade till dessa resursintensiva underhållsstrategier. Det möjliggör underhålls-larm i realtid härledda från förekomsten av anomalier, vilket skapar en grund för proaktivt underhåll under tillverkningen. Men för att möjliggöra detta måste man undersöka sambandet mellan maskinfel och anomalier i data utifrån definierade insamlingsmetod. Det är också viktigt att hantera obalansen mellan fel och normal arbetstillstånd data för maskininlärningsmodellerna. I det här arbetet undersöker vi potentialen för delvis övervakad anomali detektering för att förutsäga framtida verktyg fel hos en aktiv CNC-maskin baserat på multivariat tidsseriedata som samlats in genom det standardiserade datainsamling protokollet MT Connect. Två anomali detekterings metoder som endast tränats på normala arbetsförhållanden för maskiner testades, DeepAnT och ROCKET OCSVM. Träning och utvärdering av de två modellerna genomfördes på tre produktionsdelprocesser och skillnaden i anomali fördelning före fel och i det normala maskinens arbetstillstånd undersöktes. Resultaten visade att båda modellerna, för alla undersökta verktygsfel som hör till de tre produktionsdelprocesserna, fann ett överflöd av anomalier före fel i jämförelse med maskinernas normala arbetstillstånd. För vissa verktygsfel hittades anomalierna så långt tillbaka som sju produktionscykler före fel, medan andra anomalier huvudsakligen upptäcktes nära felet. Vidare visades det att båda modellerna presterar optimalt med 100 produktionscykler före verktygsfel uteslutna från träningen, vilket tyder på att fler anomalier tidigare än de åtta produktions cyklarna undersökta innan fel eller eventuell långvarig försämring av verktygsmaskiner kan förekomma. Slutligen visade ROCKET OCSVM med RBF som kärnfunktion större tillförlitlighet i jämförelse med DeepAnT metoden gällande att separera CNC-maskinens normala arbetstillstånd data från pre-failure-data baserat på anomali fördelning. Sammanfattningsvis visar avvikelse detektering lovande resultat för att indikera framtida maskinfel och kan fungera som en grund för proaktivt underhåll av maskiner. Genom att införskaffa proaktiva strategier kan maskinernas stilleståndstid, operatörens underhållstid samt resurser och kostnader till följd av maskinfel minskas.

Magnetic resonance imaging (MRI) is a commonly used non-invasive imaging technique that provides high resolution images of soft tissue. One problem with MRI is that it is sensitive to signal quality issues. The issues can arise for various reasons, for example by metal located either inside or outside of the body. Another common signal quality issue is caused by the patient being partly placed outside field of view of the MRI scanner.   This thesis aims to investigate the possibility to automatically detect these signal quality issues using deep neural networks. More specifically, two different 3D CNN network types were studied, a classification-based approach and a reconstruction-based approach. The datasets used consist of MRI volumes from UK Biobank which have been processed and manually annotated by operators at AMRA Medical. For the classification method four different network architectures were explored utilising supervised learning with multi-label classification. The classification method was evaluated using accuracy and label-based evaluation metrics, such as macro-precision, macro-recall and macro-F1. The reconstruction method was based on anomaly detection using an autoencoder which was trained to reconstruct volumes without any artefacts. A mean squared prediction error was calculated for the reconstructed volume and compared against a threshold in order to classify a volume with or without artefacts. The idea was that volumes containing artefacts should be more difficult to reconstruct and thus, result in a higher prediction error. The reconstruction method was evaluated using accuracy, precision, recall and F1-score.  The results show that the classification method has overall higher performance than the reconstruction method. The achieved accuracy for the classification method was 98.0% for metal artefacts and 97.5% for outside field of view artefacts. The best architecture for the classification method proved to be DenseNet201. The reconstruction method worked for metal artefacts with an achieved accuracy of 75.7%. Furthermore, it was concluded that reconstruction method did not work for detection of outside field of view artefacts.    The results from the classification method indicate that there is a possibility to automatically detect artefacts with deep neural networks. However, it is needed to further improve the method in order to completely replace a manual quality control step before using the volumes for calculation of biomarkers.

Detecting defects in industrially manufactured products is crucial to ensure their safety and quality. This process can be both expensive and error-prone if done manually, making automated solutions desirable. There is extensive research on industrial anomaly detection in images, but recent studies have shown that adding 3D information can increase the performance. This thesis aims to extend the 2D anomaly detection framework, PaDiM, to incorporate 3D information. The proposed methods combine RGB with depth maps or point clouds and the effects of using PointNet++ and vision transformers to extract features are investigated. The methods are evaluated on the MVTec 3D-AD public dataset using the metrics image AUROC, pixel AUROC and AUPRO, and on a small dataset collected with a Time-of-Flight sensor. This thesis concludes that the addition of 3D information improves the performance of PaDiM and vision transformers achieve the best results, scoring an average image AUROC of 86.2±0.2 on MVTec 3D-AD.

The internet has become an essential part of most people's daily lives in recent years, and as more devices connect to the internet, the risk of cyber threats increases dramatically. As malware becomes more sophisticated, traditional security prevention measures are becoming less effective at defending from cyber attacks. As a result, Signature Based Detection and Anomaly Detection are two of many advanced techniques that have become crucial to defend against cyber threats such as malware, but even these are sometimes not enough to stop modern cyberattacks. In this literature review the goal is to discuss how trustworthy each of the mentioned malware detection techniques are at detecting malware in wireless networks. The study will measure trustworthiness by looking further into scalability, adaptability and robustness and resource consumption. This study concludes that both anomaly and signature-based malware detection methods exhibit strengths and weaknesses in scalability, robustness, adaptability, and resource consumption. Furthermore, more research is needed and as malware becomes more sophisticated and an increased threat to the world it is an area that is highly relevant.

Anomaly detection in high-dimensional financial transaction data is challenging and resource-intensive, particularly when the dataset is unlabeled. Sometimes, one can alleviate the computational cost and improve the results by utilizing a pre-trained model, provided that the features learned from the pre-training are useful for learning the second task. Investigating this issue was the main purpose of this thesis. More specifically, it was to explore the potential gain of pre-training a detection model on one trader's transaction history and then retraining the model to detect anomalous trades in another trader's transaction history. In the context of transfer learning, the pre-trained and the retrained model are usually referred to as the source model and target model, respectively.  A deep LSTM autoencoder was proposed as the source model due to its advantages when dealing with sequential data, such as financial transaction data. Moreover, to test its anomaly detection ability despite the lack of labeled true anomalies, synthetic anomalies were generated and included in the test set. Various experiments confirmed that the source model learned to detect synthetic anomalies with highly distinctive features. Nevertheless, it is hard to draw any conclusions regarding its anomaly detection performance due to the lack of labeled true anomalies. While the same is true for the target model, it is still possible to achieve the thesis's primary goal by comparing a pre-trained model with an identical untrained model. All in all, the results suggest that transfer learning offers a significant advantage over traditional machine learning in this context.

In this thesis, the use of unsupervised and semi-supervised machine learning techniques was analyzed as potential tools for anomaly detection in the sensor network that the electrical system in a Scania truck is comprised of. The experimentation was designed to analyse the need for both point and contextual anomaly detection in this setting. For the point anomaly detection the method of Isolation Forest was experimented with and for contextual anomaly detection two different recurrent neural network architectures using Long Short Term Memory units was relied on. One model was simply a many to one regression model trained to predict a certain signal, while the other was an encoder-decoder network trained to reconstruct a sequence. Both models were trained in an semi-supervised manner, i.e. on data that only depicts normal behaviour, which theoretically should lead to a performance drop on abnormal sequences resulting in higher error terms. In both setting the parameters of a Gaussian distribution were estimated using these error terms which allowed for a convenient way of defining a threshold which would decide if the observation would be flagged as anomalous or not. Additional experimentation's using an exponential weighted moving average over a number of past observations to filter the signal was also conducted. The models performance on this particular task was very different but the regression model showed a lot of promise especially when combined with a filtering preprocessing step to reduce the noise in the data. However the model selection will always be governed by the nature the particular task at hand so the other methods might perform better in other settings. / I den här avhandlingen var användningen av oövervakad och halv-övervakad maskininlärning analyserad som ett möjligt verktyg för att upptäcka avvikelser av anomali i det sensornätverk som elektriska systemet en Scanialastbil består av. Experimentet var konstruerat för att analysera behovet av både punkt och kontextuella avvikelser av anomali i denna miljö. För punktavvikelse av anomali var metoden Isolation Forest experimenterad med och för kontextuella avvikelser av anomali användes två arkitekturer av återkommande neurala nätverk. En av modellerna var helt enkelt många-till-en regressionmodell tränad för att förutspå ett visst märke, medan den andre var ett kodare-avkodare nätverk tränat för att rekonstruera en sekvens.Båda modellerna blev tränade på ett halv-övervakat sätt, d.v.s. på data som endast visar normalt beteende, som teoretiskt skulle leda till minskad prestanda på onormala sekvenser som ger ökat antal feltermer. I båda fallen blev parametrarna av en Gaussisk distribution estimerade på grund av dessa feltermer som tillåter ett bekvämt sätt att definera en tröskel som skulle bestämma om iakttagelsen skulle bli flaggad som en anomali eller inte. Ytterligare experiment var genomförda med exponentiellt viktad glidande medelvärde över ett visst antal av tidigare iakttagelser för att filtera märket. Modellernas prestanda på denna uppgift var välidt olika men regressionmodellen lovade mycket, särskilt kombinerad med ett filterat förbehandlingssteg för att minska bruset it datan. Ändå kommer modelldelen alltid styras av uppgiftens natur så att andra metoder skulle kunna ge bättre prestanda i andra miljöer.