As the complexity of deep learning model increases, the transparency of the systems does the opposite. It may be hard to understand the predictions a deep learning model makes, but even harder to understand why these predictions are made. Using eXplainable AI (XAI), we can gain greater knowledge of how the model operates and how the input in which the model receives can change its predictions. In this thesis, we apply Integrated Gradients (IG), an XAI method primarily used on image data and on datasets containing tabular and time-series data. We also evaluate how the results of IG differ from various types of models and how the change of baseline can change the outcome. In these results, we observe that IG can be applied to both sequenced and nonsequenced data, with varying results. We can see that the gradient baseline does not affect the results of IG on models such as RNN, LSTM, and GRU, where the data contains time series, as much as it does for models like MLP with nonsequenced data. To confirm this, we also applied IG to SVM models, which gave the results that the choice of gradient baseline has a significant impact on the results of IG.

Side-channel attacks have become a realistic threat to the implementations of cryptographic algorithms. By analyzing the unintentional, side-channel leakage, the attacker is able to recover the secret of the target. Recently, a new type of side-channel leakage has been discovered, called far field EM emissions. Unlike attacks based on near field EM emissions or power consumption, the attack based on far field EM emissions is able to extract the secret key from the victim device of several meters distance. However, existing deep-learning attacks based far field EM commonly use a random or grid search method to optimize neural networks’ hyperparameters. Recently, an automated way for deep learning hyperparameter tuning based on Auto- Keras library, called AutoSCA framework, was applied to near-field EM attacks. In this work, we investigate if AutoSCA could help far field EM side-channel attacks. In our experiments, the target is a Bluetooth-5 supported Nordic Semiconductor nRF52832 development kit implementation of Advanced Encryption Standard (AES). Our experiments show that, by using a deep-learning model generated by the AutoSCA framework, we need 485 traces on average to recover a subkey from traces captured at 15 meters distance from the victim device without repeating each encryption. For the same conditions, the state-of-the-art method uses 510 traces. Furthermore, our model contains only 667,433 trainable parameters in total, implying that it requires roughly 9 times less training resources compared to the larger models in the previous work. / Angrepp på sidokanaler har blivit ett realistiskt hot mot implementeringen av kryptografiska algoritmer.Genom att analysera det oavsiktliga läckaget kan angriparen hitta hemligheten bakom målet.Nyligen har en ny typ av sidokanalläckage upptäckts, kallad fjärrfälts EM-utsläpp.Till skillnad från attacker baserade på nära fält EM- utsläpp eller energiförbrukning, kan attacken baserad på yttre fält EM-utsläpp extrahera den hemliga nyckeln från den skadade anordningen på flera meter avstånd.Men befintliga djupinlärningsattacker baserade på långt fält EM använder ofta en slumpmässig sökmetod för att optimera nervnätens hyperparametrar. Nyligen tillämpades ett automatiserat sätt för djupinlärning av hyperparametern baserad på Auto-Keras- bibliotek, AutoSCA- ramverket, vid EM-angrepp nära fältet.I det här arbetet undersöker vi om AutoSCA kan hjälpa till med EM-angrepp.I våra experiment är målet en Bluetooth-5-stödd nordisk semidirigent nR52832- utvecklingsutrustning för avancerad krypteringsstandard (AES).Våra experiment visar att genom att använda en djupinlärningsmodell skapad av AutoSCA-ramverket, behöver vi 485-spår i genomsnitt för att hämta en subnyckel från spår tagna på 15- meters avstånd från offrets apparat utan att upprepa varje kryptering.Under samma förhållanden använder den senaste metoden 510-spår.Dessutom innehåller vår modell bara 667,433-parametrar som totalt kan användas för utbildning, vilket innebär att det krävs ungefär nio gånger mindre utbildningsresurser jämfört med de större modellerna i det tidigare arbetet.

No description available.

Two types of artificial neural networks, Multilayer Perceptron (MLP) and Self-organizing Feature Map (SOM), were employed to detect mastitis for robotic milking stations using the preprocessed data relating to the electrical conductivity and milk yield. The SOM was developed to classify the health status into three categories: healthy, moderately ill and severely ill. The clustering results were successfully evaluated and validated by using statistical techniques such as K-means clustering, ANOVA and Least Significant Difference. The result shows that the SOM could be used in the robotic milking stations as a detection model for mastitis. For developing MLP models, a new mastitis definition based on higher EC and lower quarter yield was created and Principle Components Analysis technique was adopted for addressing the problem of multi-colinearity existed in the data. Four MLPs with four combined datasets were developed and the results manifested that the PCA-based MLP model is superior to other non-PCA-based models in many respects such as less complexity, higher predictive accuracy. The overall correct classification rate (CCR), sensitivity and specificity of the model was 90.74 %, 86.90 and 91.36, respectively. We conclude that the PCA-based model developed here can improve the accuracy of prediction of mastitis by robotic milking stations.

Modern methods of speech synthesis are not suitable for restoration of song signals due to lack of vitality and intonation in the resulted sounds. The aim of presented work is to synthesize melismas met in Lithuanian folk songs, by applying Artificial Neural Networks. An analytical survey of rather a widespread literature is presented. First classification and comprehensive discussion of melismas are given. The theory of dynamic systems which will make the basis for studying melismas is presented and finally the relationship for modeling a melisma with nonlinear and dynamic systems is outlined. Investigation of the most widely used Linear Prediction Coding method and possibilities of its improvement. The modification of original Linear Prediction method based on dynamic LPC frame positioning is proposed. On its basis, the new melisma synthesis technique is presented. Developed flexible generalized melisma model, based on two Artificial Neural Networks – a Multilayer Perceptron and Adaline – as well as on two network training algorithms – Levenberg- Marquardt and the Least Squares error minimization – is presented. Moreover, original mathematical models of Fortis, Gruppett, Mordent and Trill are created, fit for synthesizing melismas, and their minimal sizes are proposed. The last chapter concerns experimental investigation, using over 500 melisma records, and corroborates application of the new mathematical models to melisma synthesis of one performer.

Modern methods of speech synthesis are not suitable for restoration of song signals due to lack of vitality and intonation in the resulted sounds. The aim of presented work is to synthesize melismas met in Lithuanian folk songs, by applying Artificial Neural Networks. An analytical survey of rather a widespread literature is presented. First classification and comprehensive discussion of melismas are given. The theory of dynamic systems which will make the basis for studying melismas is presented and finally the relationship for modeling a melisma with nonlinear and dynamic systems is outlined. Investigation of the most widely used Linear Prediction Coding method and possibilities of its improvement. The modification of original Linear Prediction method based on dynamic LPC frame positioning is proposed. On its basis, the new melisma synthesis technique is presented. Developed flexible generalized melisma model, based on two Artificial Neural Networks – a Multilayer Perceptron and Adaline – as well as on two network training algorithms – Levenberg- Marquardt and the Least Squares error minimization – is presented. Moreover, original mathematical models of Fortis, Gruppett, Mordent and Trill are created, fit for synthesizing melismas, and their minimal sizes are proposed. The last chapter concerns experimental investigation, using over 500 melisma records, and corroborates application of the new mathematical models to melisma synthesis of one performer.

在這份研究中，我們提出一個類分散式神經網路框架，此框架為高頻交易系統研究下之子專案。在系統中，我們透過資料探勘程序發掘財務時間序列中的模式，其中所採用的資料探勘演算法之一即為類神經網路。我們實作一個在分散式平台上訓練類神經網路的框架。我們採用Apache Spark來建立底層的運算叢集，因為它提供高效能的記憶體內運算（in-memory computing）。我們分析一些分散式後向傳導演算法（特別是用來預測財務時間序列的），加以調整，並將其用於我們的框架。我們提供了許多細部的選項，讓使用者在進行類神經網路建模時有很高的彈性。 / In this research, we introduce a distributed framework of artificial neural network (ANN) as a subproject under the research of a high-frequency trading (HFT) system. In the system, ANNs are used in the data mining process for identifying patterns in financial time series. We implement a framework for training ANNs on a distributed computing platform. We adopt Apache Spark to build the base computing cluster because it is capable of high performance in-memory computing. We investigate a number of distributed backpropagation algorithms and techniques, especially ones for time series prediction, and incorporate them into our framework with some modifications. With various options for the details, we provide the user with flexibility in neural network modeling.

In this dissertation, generalized additive neural networks (GANNs) and multilayer perceptrons (MLPs) are studied and compared as prediction techniques. MLPs are the most widely used type of artificial neural network (ANN), but are considered black boxes with regard to interpretability. There is currently no simple a priori method to determine the number of hidden neurons in each of the hidden layers of ANNs. Guidelines exist that are either heuristic or based on simulations that are derived from limited experiments. A modified version of the neural network construction with cross–validation samples (N2C2S) algorithm is therefore implemented and utilized to construct good MLP models. This algorithm enables the comparison with GANN models. GANNs are a relatively new type of ANN, based on the generalized additive model. The architecture of a GANN is less complex compared to MLPs and results can be interpreted with a graphical method, called the partial residual plot. A GANN consists of an input layer where each of the input nodes has its own MLP with one hidden layer. Originally, GANNs were constructed by interpreting partial residual plots. This method is time consuming and subjective, which may lead to the creation of suboptimal models. Consequently, an automated construction algorithm for GANNs was created and implemented in the SAS R statistical language. This system was called AutoGANN and is used to create good GANN models. A number of experiments are conducted on five publicly available data sets to gain insight into the similarities and differences between GANN and MLP models. The data sets include regression and classification tasks. In–sample model selection with the SBC model selection criterion and out–of–sample model selection with the average validation error as model selection criterion are performed. The models created are compared in terms of predictive accuracy, model complexity, comprehensibility, ease of construction and utility. The results show that the choice of model is highly dependent on the problem, as no single model always outperforms the other in terms of predictive accuracy. GANNs may be suggested for problems where interpretability of the results is important. The time taken to construct good MLP models by the modified N2C2S algorithm may be shorter than the time to build good GANN models by the automated construction algorithm / Thesis (M.Sc. (Computer Science))--North-West University, Potchefstroom Campus, 2011.

In this dissertation, generalized additive neural networks (GANNs) and multilayer perceptrons (MLPs) are studied and compared as prediction techniques. MLPs are the most widely used type of artificial neural network (ANN), but are considered black boxes with regard to interpretability. There is currently no simple a priori method to determine the number of hidden neurons in each of the hidden layers of ANNs. Guidelines exist that are either heuristic or based on simulations that are derived from limited experiments. A modified version of the neural network construction with cross–validation samples (N2C2S) algorithm is therefore implemented and utilized to construct good MLP models. This algorithm enables the comparison with GANN models. GANNs are a relatively new type of ANN, based on the generalized additive model. The architecture of a GANN is less complex compared to MLPs and results can be interpreted with a graphical method, called the partial residual plot. A GANN consists of an input layer where each of the input nodes has its own MLP with one hidden layer. Originally, GANNs were constructed by interpreting partial residual plots. This method is time consuming and subjective, which may lead to the creation of suboptimal models. Consequently, an automated construction algorithm for GANNs was created and implemented in the SAS R statistical language. This system was called AutoGANN and is used to create good GANN models. A number of experiments are conducted on five publicly available data sets to gain insight into the similarities and differences between GANN and MLP models. The data sets include regression and classification tasks. In–sample model selection with the SBC model selection criterion and out–of–sample model selection with the average validation error as model selection criterion are performed. The models created are compared in terms of predictive accuracy, model complexity, comprehensibility, ease of construction and utility. The results show that the choice of model is highly dependent on the problem, as no single model always outperforms the other in terms of predictive accuracy. GANNs may be suggested for problems where interpretability of the results is important. The time taken to construct good MLP models by the modified N2C2S algorithm may be shorter than the time to build good GANN models by the automated construction algorithm / Thesis (M.Sc. (Computer Science))--North-West University, Potchefstroom Campus, 2011.

Made available in DSpace on 2016-12-12T20:22:52Z (GMT). No. of bitstreams: 1 RAFAEL ALCESTE BERRI.pdf: 28428368 bytes, checksum: 667b9facc9809bfd5e0847e15279b0e6 (MD5) Previous issue date: 2014-02-21 / Coordenação de Aperfeiçoamento de Pessoal de Nível Superior / In this work, three proposals of systems have been developed using a frontal camera to monitor the driver and enabling to identificate if a cell phone is being used while driving the vehicle. It is estimated that 80% of crashes and 65% of near collisions involved drivers who were inattentive in traffic for three seconds before the event. Five videos in real environment were generated to test the systems. The pattern recognition system (RP) uses adaptive skin segmentation, feature extraction, and machine learning to detect cell phone usage on each frame. The cell phone detection happens when, in periods of 3 seconds, 60% (threshold) of frames or more are identified as a cell phone use, individually. The average accuracy on videos achieved was 87.25% with Multilayer Perceptron (MLP), Gaussian activation function, and two neurons of the intermediate layer. The movement detection system (DM) uses optical flow, filtering the most relevant movements of the scene, and three successive frames for detecting the movements to take the phone to the ear and take it off. The DM proposal was not demonstrated as being an effective solution for detecting cell phone use, reaching an accuracy of 52.86%. The third solution is a hybrid system. It uses the RP system for classification and the DM for choosing the RP parameters. The parameters chosen for RP are the threshold and the classification system. The definition of these two parameters occurs at the end of each period, based on movement detected by the DM. Experimentally it was established that, when the movement induces to use cell phone, it is proper to use the threshold of 60%, and the classifier as MLP/Gaussian with seven neurons of the intermediate layer; otherwise, it is used threshold 85%, and MLP/Gaussian with two neurons of the intermediate layer for classification. The hybrid solution is the most robust system with average accuracy of 91.68% in real environment. / Neste trabalho, são desenvolvidas três propostas de sistemas que permitem identificar o uso de celular, durante o ato de dirigir um veículo, utilizando imagens capturadas de uma câmera posicionada em frente ao motorista. Estima-se que 80% das colisões e 65% das quase colisões envolveram motoristas que não estavam prestando a devida atenção ao trânsito por três segundos antes do evento. Cinco vídeos em ambiente real foram gerados com o intuito de testar os sistemas. A proposta de reconhecimento de padrões (RP) emprega segmentação de pele adaptativa, extração de características e aprendizado de máquina (classificador) na detecção do celular em cada quadro processado. A detecção do uso do celular ocorre quando, em períodos de 3 segundos, ao menos em 60% dos quadros (corte) são identificados com celular. A acurácia média nos vídeos alcançou 87, 25% ao utilizar Perceptron Multi-camadas (MLP) com função de ativação gaussiana e dois neurônios na camada intermediária como classificador. A proposta de detecção de movimento (DM) utiliza o fluxo ótico, filtragem dos movimentos mais relevantes da cena e três quadros consecutivos para detectar os momentos de levar o celular ao ouvido e o retirá-lo. A aplicação do DM, como solução para detectar o uso do celular, não se demostrou eficaz atingindo uma acurácia de 52, 86%. A terceira proposta, uma solução híbrida, utiliza o sistema RP como classificador e o de DM como seu parametrizador. Os parâmetros escolhidos para o sistema de RP são o corte e o sistema classificador. A definição desses dois parâmetros ocorre ao final de cada período, baseada na movimentação detectada pela DM. Com experimentações definiu-se que, caso a movimentação induza ao uso do celular, é adequado o uso do corte de 60% e o classificador MLP/Gaussiana com sete neurônios na camada intermediária, caso contrário, utiliza-se o corte de 85% e classificador MLP/Gaussiana com dois neurônios na mesma camada. A versão híbrida é a solução desenvolvida mais robusta, atingindo a melhor acurácia média de 91, 68% em ambiente real.