Cyber-Physical Systems (CPSs) are the core of modern critical infrastructure (e.g. power-grids) and securing them is of paramount importance. Anomaly detection in data is crucial for CPS security. While Artificial Neural Networks (ANNs) are strong candidates for the task, they are seldom deployed in safety-critical domains due to the perception that ANNs are black-boxes. Therefore, to leverage ANNs in CPSs, cracking open the black box through explanation is essential. The main objective of this dissertation is developing explainable ANN-based Anomaly Detection Systems for Cyber-Physical Systems (CP-ADS). The main objective was broken down into three sub-objectives: 1) Identifying key-requirements that an explainable CP-ADS should satisfy, 2) Developing supervised ANN-based explainable CP-ADSs, 3) Developing unsupervised ANN-based explainable CP-ADSs. In achieving those objectives, this dissertation provides the following contributions: 1) a set of key-requirements that an explainable CP-ADS should satisfy, 2) a methodology for deriving summaries of the knowledge of a trained supervised CP-ADS, 3) a methodology for validating derived summaries, 4) an unsupervised neural network methodology for learning cyber-physical (CP) behavior, 5) a methodology for visually and linguistically explaining the learned CP behavior. All the methods were implemented on real-world and benchmark datasets. The set of key-requirements presented in the first contribution was used to evaluate the performance of the presented methods. The successes and limitations of the presented methods were identified. Furthermore, steps that can be taken to overcome the limitations were proposed. Therefore, this dissertation takes several necessary steps toward developing explainable ANN-based CP-ADS and serves as a framework that can be expanded to develop trustworthy ANN-based CP-ADSs.

[pt] A deterioração do desempenho da turbina a gás é resultado de vários tipos de falhas, como acúmulo de sujeira, erosão e corrosão, que afetam os componentes no caminho do gás, sendo os principais o compressor, o combustor e a turbina. No presente trabalho é avaliado o desempenho de Redes Neurais Artificiais (RNA) no emprego de diagnóstico de falha de turbinas a gás. Todas as redes projetadas são do tipo MLP (multi-layer perceptron) com algoritmo de retropropagação (backpropagation). Para cada função de diagnóstico, várias arquiteturas foram testadas, modificando parâmetros de rede como o número de camadas escondidas e o número de neurônios em cada uma destas camadas. As RNAs para diagnóstico de falhas foram aplicadas ao modelo termodinâmico de uma turbina a gás industrial. Este modelo foi responsável pela criação de dados da usina saudável e também degradada, utilizados para o treinamento e validação das redes. Com os resultados obtidos do treinamento das redes é possível mostrar que as mesmas são capazes de detectar, isolar e quantificar falhas de componentes de turbinas a gás de forma satisfatória. / [en] The gas turbine performance deterioration is a result of several types of faults such as fouling, erosion and corrosion, which affects the components throughout the gas path. As the most significant of these components we can enumerate the compressor, the combustion chamber and the turbine itself. In this work the performance of different types of Artificial Neural Networks (ANN) are evaluated in the diagnosis of this kind of fault. Every neural network designed in this work is MLP (multi-layer perceptron) with back propagation algorithm. For each diagnosis function several architectures were tested, varying network parameters as the numbers of hidden layers and the number of neurons in each layer. The ANNs for fault diagnosis were applied in an industrial gas turbine thermodynamic model. This model was also used for healthy and degraded turbine data generation, which were used for ANNs training and validation. With the ANNs training results we can conclude that these networks are capable of detecting, isolating and quantifying gas turbine components faults in a satisfactory way.

Predictive Maintenance (PM) has been increasingly adopted in the Automotive industry, in the recent decades along with conventional approaches such as the Preventive Maintenance and Diagnostic/Corrective Maintenance, since it provides many advantages to estimate the failure before the actual occurrence proactively, and also being adaptive to the present status of the vehicle, in turn allowing flexible maintenance schedules for efficient repair or replacing of faulty components. PM necessitates the storage and analysis of large amounts of sensor data. This requirement can be a challenge in deploying this method on-board the vehicles due to the limited storage and computational power on the hardware of the vehicle. Hence, this thesis seeks to obtain low dimensional descriptive features from high dimensional data using Representation Learning. This low dimensional representation will be used for predicting vehicle faults, specifically Turbocharger related failures. Since the Logged Vehicle Data (LVD) was base on all the data utilized in this thesis, it allowed for the evaluation of large populations of trucks without requiring additional measuring devices and facilities. The gradual degradation methodology is considered for describing vehicle condition, which allows for modeling the malfunction/ failure as a continuous process rather than a discrete flip from healthy to an unhealthy state. This approach eliminates the challenge of data imbalance of healthy and unhealthy samples. Two important hypotheses are presented. Firstly, Parallel StackedClassical Autoencoders would produce better representations com-pared to individual Autoencoders. Secondly, employing Learned Em-beddings on Categorical Variables would improve the performance of the Dimensionality reduction. Based on these hypotheses, a model architecture is proposed and is developed on the LVD. The model is shown to achieve good performance, and in close standards to the previous state-of-the-art research. This thesis, finally, illustrates the potential to apply parallel stacked architectures with Learned Embeddings for the Categorical features, and a combination of feature selection and extraction for numerical features, to predict the Remaining Useful Life (RUL) of a vehicle, in the context of the Turbocharger. A performance improvement of 21.68% with respect to the Mean Absolute Error (MAE) loss with an 80.42% reduction in the size of data was observed.

The parallel nature of process execution on chip multiprocessors (CMPs) has considerably boosted levels of application performance in the past decade. Generally, a certain number of computing resources are shared among the several cores of a CMP, such as shared last-level caches, shared-buses, and shared-memory. This ensures architectural simplicity while also boosting performance for multi-threaded applications. However, a consequence of sharing computing resources is that concurrently executing applications may suffer performance degradation if their collective resource requirements exceed the total amount of resources available. If resource allocation is not carefully considered, the potential performance gain from having multiple cores may be outweighed by the losses due to contention among processes for shared resources. Furthermore, CMPs with inbuilt dynamic voltage-frequency scaling (DVFS) may try to compensate for the performance loss by scaling to a higher frequency. For performance degradation due to shared-resource contention, this does not necessarily improve performance but guarantees a significant penalty on power consumption due to the quadratic relation of electrical power and voltage (P ∝ V^{2}*f).

Several models for predicting future customer profitability early into customer life-cycles in the property and casualty business are constructed and studied. The objective is to model risk at a customer level with input data available early into a private consumer’s lifespan. Two retained models, one using Generalized Linear Model another using a multilayer perceptron, a special form of Artificial Neural Network are evaluated using actual data. Numerical results show that differentiation on estimated future risk is most effective for customers with highest claim frequencies.

This thesis examines how convolutional neural networks can applied to the problem of hand detection and hand pose estimation. Two families of convolutional neural networks are trained, aimed at performing the task of classification or regression. The networks are trained on specialized data generated from publicly available datasets. The algorithms used to generate the specialized data are also disclosed. The main focus has been to investigate the different structural properties of convolutional neural networks, not building optimized hand detection, or hand pose estimation, systems. Experiments revealed, that classifier networks featuring a relatively high number of convolutions offers the highest performance on external validation data. Additionally, shallow classifier networks featuring a relatively low number of convolutions, yields a high classification accuracy on training and testing data, but a very low accuracy on the validation set. This effect uncovers one of the fundamental difficulties in building a hand detection system: The asymmetric classification problem. In further investigation, it is also remarked, that relatively shallow classifier networks probably becomes color sensitive. Furthermore, regressor networks featuring multiscale inputs typically yielded the lowest error, when tasked with computing key-point locations directly from data. It is also revealed, that color data implicitly contain more information, making it easier to compute key-point locations, especially in the image space. However, to be able to derive the color invariant features, deeper regressor networks are required. / I detta examensarbete undersöks hur faltande neuronnät kan användas för detektering av, samt skattning av pose hos, händer. Två familjer av neuronnät tränas, med syftet att utföra klassificering eller regression. Neuronnäten tränas med specialiserad data genererad ur publikt tillgängliga dataset. Algoritmerna för att generera den specialiserade datan presenteras även i sin helhet. Huvudsyftet med arbetet, har varit att undersöka neuronnätens strukturella egenskaper, samt relatera dessa till prestanda, och inte bygga ett färdigt system för handdetektering eller skattning av handpose. Experimenten visade, att neuronnät för klassificering med ett relativt stor antal faltningar ger högst prestanda på valideringsdata. Vidare, så verkar neuronnät för klassificering med relativt litet antal faltningar ge en god prestanda på träning- och testdata, men mycket dålig prestand på valideringsdata. Detta sambandet avslöjar en fundamental svårighet med att träna ett neuronnät för klassificering av händer, nämligen det kraftigt asymmetriska klassificeringsproblemet. I vidare undersökningar visar det sig också, att neuronnät för klassificering med ett relativt litet antal faltningar troligtvis enbart blir färgkänsliga. Experimenten visade också, att neuronnät för regression som använde sig av data i flera skalor gav lägst fel när de skulle beräkna positioner av handmarkörer direkt ur data. Slutligen framkom det, att färgdata, i konstrast till djupdata, implicit innehåller mer information, vilket gör det relativt sett lättare att beräkna markörer, framför allt i det tvådimensionella bildrummet. Dock, för att kunna få fram den implicita informationen, så krävs relativt djupa neuronnät.

Machine vision systems combined with classification algorithms are being increasingly used for different applications in the age of automation. One such application would be the quality control of the painted automobile parts. The fundamental elements of the machine vision system include camera, illumination, image acquisition software and computer vision algorithms. Traditional way of thinking puts too much importance on camera systems and ignores other elements while designing a machine vision system. In this thesis work, it is shown that selecting an appropriate illumination for illuminating the surface being examined is equally important in case of machine vision system for examining specular surface. Knowledge about the nature of the surface, type and properties of the defect to be detected and classified are important factors while choosing the illumination system for the machine vision system. The main illumination system tested were bright field, dark field and structured illumination and out of the three, dark field and structured illumination gave best results. This thesis work proposes a dark field illumination based machine vision system for fault detection on specular painted surface. A single layer Artificial Neural Network model is employed for the classification of defects in intensity images of painted surface acquired with this machine vision system. The results of this research work proved that the quality of the images and size of data set used for training the Neural Network model play a vital role in the performance of the classifier algorithm.

Due to the rapidly increasing prominence of Artificial Intelligence in the last decade and the advancements in technology such as processing power and data storage, there has been increased interest in applying machine learning algorithms to time series prediction problems. There are many machine learning algorithms that can be used for time series prediction problems but selecting an algorithm can be challenging due to algorithms not being suitable to all types of datasets. This research investigates and evaluates machine learning algorithms that can be used for time series prediction. Experiments were carried out using the Artificial Neural Network (ANN), Support Vector Regressor (SVR) and Long Short-Term Memory (LSTM) algorithms on eight datasets. An empirical analysis was carried out by applying each machine learning algorithm to the selected datasets. A critical comparison of the algorithm performance was carried out using the Mean Absolute Error (MAE), the Mean Squared Error (MSE), the Root Mean Squared Error (RMSE) and the Mean Absolute Scaled Error (MASE). The second experiment focused on evaluating the stability and robustness of the optimal models identified in the first experiment. The key dataset characteristics identified; were the dataset size, stationarity, trend and seasonality. It was found that the LSTM performed the best for majority of the datasets, due to the algorithm's ability to deal with sequential dependency. The performance of the ANN and SVR were similar for datasets with trend and seasonality, while the LSTM overall proved superior to the aforementioned algorithms. The LSTM outperformed the ANN and SVR due to its ability to handle temporal dependency. However, due to its stochastic nature, the LSTM and ANN algorithms can have poor stability and robustness. In this regard, the LSTM was found to be a more robust algorithm than the ANN and SVR.

Star Vault AB is a video game development company that has developed the video game Mortal Online. The company has stated that they believe that players new to the game repeatedly find themselves being lost in the game. The objective of this study is to evaluate whether or not an Artificial Neural Network can be used to evaluate when a player is lost in the game Mortal Online. This is done using the free open source library Fast Artifical Neural Network Library. People are invited to a data collection event where they play a tweaked version of the game to facilitate data collection. Players specify whether they are lost or not and the data collected is flagged accordingly. The collected data is then prepared with different parameters to be used when training multiple Artificial Neural Networks. When creating an Artificial Neural Network there exists several parameters which have an impact on its performance. Performance is defined as the balance of high prediction accuracy against low false positive rate. These parameters vary depending on the purpose of the Artificial Neural Network. A quantitative approach is followed where these parameters are varied to investigate which values result in the Artificial Neural Network which best identifies when a player is lost. The parameters are grouped into stages where all combinations of parameter values within each stage are evaluated to reduce the amount of Artificial Neural Networks which have to be trained, with the best performing parameters of each stage being used in subsequent stages. The result is a set of values for the parameters that are considered as ideal as possible. These parameter values are then altered one at a time to verify that they are ideal. The results show that a set of parameters exist which can optimize the Artificial Neural Network model to identify when a player is lost, however not with the high performance that was hoped for. It is theorized that the ambiguity of the word "lost" and the complexity of the game are critical to the low performance.

Artificial neural networks are increasing in power, with attendant increases in demand for efficient processing. Performance is limited by clock speed and degree of parallelization available through multi-core processors and GPUs. With a design tailored to a specific network, a field-programmable gate array (FPGA) can be used to minimize latency without the need for geographically distributed computing. However, the task of programming an FPGA is outside the realm of most data scientists. There are tools to program FPGAs from a high level description of a network, but there is no unified interface for programmers across these tools. In this thesis, I present the design and implementation of NeuralSynth, a prototype Python framework which aims to bridge the gap between data scientists and FPGA programming for neural networks. My method relies on creating an extensible Python framework that is used to automate programming and interaction with an FPGA. The implementation includes a digital design for the FPGA that is completed by a Python framework. Programming and interacting with the FPGA does not require leaving the Python environment. The extensible approach allows multiple implementations, resulting in a similar workflow for each implementation. For evaluation, I compare the results of my implementation with a known neural network framework. / Includes bibliography. / Thesis (M.S.)--Florida Atlantic University, 2020. / FAU Electronic Theses and Dissertations Collection