<p dir="ltr">In the realm of deep learning and neural networks, there has been substantial advancement, but the persistent DNN vulnerability to adversarial attacks has prompted the search for more efficient defense strategies. Unfortunately, this becomes an arms race. Stronger attacks are being develops, while more sophisticated defense strategies are being proposed, which either require modifying the model's structure or incurring significant computational costs during training. The first part of the work makes a significant progress towards breaking this arms race. Let’s consider natural images, where all the feature values are discrete. Our proposed metrics are able to discover all the vulnerabilities surrounding a given natural image. Given sufficient computation resource, we are able to discover all the adversarial examples given one clean natural image, eliminating the need to develop new attacks. For remedial measures, our approach is to introduce a random factor into DNN classification process. Furthermore, our approach can be combined with existing defense strategy, such as adversarial training, to further improve performance.</p>

This research aims to identify a method for unsupervised online anomaly detection in multivariate time series in dynamic systems in general and on the case study of Devwards IoT-system in particular. A requirement of the solution is its explainability, online learning and low computational expense. A comprehensive literature review was conducted, leading to the experimentation and analysis of various anomaly detection approaches. Of the methods evaluated, a singular recurrent neural network autoencoder emerged as the most promising, emphasizing a simple model structure that encourages stable performance with consistent outputs, regardless of the average output. While other approaches such as Hierarchical Temporal Memory models and an ensemble strategy of adaptive model pooling yielded suboptimal results. A modified version of the Residual Explainer method for enhancing explainability in autoencoders for online scenarios showed promising outcomes. The use of Mahalanobis distance for anomaly detection was explored. Feature extraction and it's implications in the context of the proposed approach is explored. Conclusively, a single, streamlined recurrent neural network appears to be the superior approach for this application, though further investigation into online learning methods is warranted. The research contributes results into the field of unsupervised online anomaly detection in multivariate time series and contributes to the Residual Explainer method for online autoencoders. Additionally, it offers data on the ineffectiveness of the Mahalanobis distance in an online anomaly detection environment.

Survival analysis is an essential statistics and machine learning field in various critical applications like medical research and predictive maintenance. In these domains understanding models' predictions is paramount. While machine learning techniques are increasingly applied to enhance the predictive performance of survival models, they simultaneously sacrifice transparency and explainability.  Survival models, in contrast to regular machine learning models, predict functions rather than point estimates like regression and classification models. This creates a challenge regarding explaining such models using the known off-the-shelf machine learning explanation techniques, like Shapley Values, Counterfactual examples, and others.    Censoring is also a major issue in survival analysis where the target time variable is not fully observed for all subjects. Moreover, in predictive maintenance settings, recorded events do not always map to actual failures, where some components could be replaced because it is considered faulty or about to fail in the future based on an expert's opinion. Censoring and noisy labels create problems in terms of modeling and evaluation that require to be addressed during the development and evaluation of the survival models. Considering the challenges in survival modeling and the differences from regular machine learning models, this thesis aims to bridge this gap by facilitating the use of machine learning explanation methods to produce plausible and actionable explanations for survival models. It also aims to enhance survival modeling and evaluation revealing a better insight into the differences among the compared survival models. In this thesis, we propose two methods for explaining survival models which rely on discovering survival patterns in the model's predictions that group the studied subjects into significantly different survival groups. Each pattern reflects a specific survival behavior common to all the subjects in their respective group. We utilize these patterns to explain the predictions of the studied model in two ways. In the first, we employ a classification proxy model that can capture the relationship between the descriptive features of subjects and the learned survival patterns. Explaining such a proxy model using Shapley Values provides insights into the feature attribution of belonging to a specific survival pattern. In the second method, we addressed the "what if?" question by generating plausible and actionable counterfactual examples that would change the predicted pattern of the studied subject. Such counterfactual examples provide insights into actionable changes required to enhance the survivability of subjects. We also propose a variational-inference-based generative model for estimating the time-to-event distribution. The model relies on a regression-based loss function with the ability to handle censored cases. It also relies on sampling for estimating the conditional probability of event times. Moreover, we propose a decomposition of the C-index into a weighted harmonic average of two quantities, the concordance among the observed events and the concordance between observed and censored cases. These two quantities, weighted by a factor representing the balance between the two, can reveal differences between survival models previously unseen using only the total Concordance index. This can give insight into the performances of different models and their relation to the characteristics of the studied data. Finally, as part of enhancing survival modeling, we propose an algorithm that can correct erroneous event labels in predictive maintenance time-to-event data. we adopt an expectation-maximization-like approach utilizing a genetic algorithm to find better labels that would maximize the survival model's performance. Over iteration, the algorithm builds confidence about events' assignments which improves the search in the following iterations until convergence. We performed experiments on real and synthetic data showing that our proposed methods enhance the performance in survival modeling and can reveal the underlying factors contributing to the explainability of survival models' behavior and performance.

No description available.

Market manipulation has increased in line with the number of active players in the financialmarkets. The most common methods for monitoring financial markets are rule-based systems,which are limited to previous knowledge of market manipulation. This work was carried out incollaboration with the company Scila, which provides surveillance solutions for the financialmarkets.In this thesis, we will try to implement a complementary method to Scila's pre-existing rule-based systems to objectively detect outliers in all available data and present the result onsuspect transactions and customer behavior to an operator. Thus, the method needs to detectoutliers and show the operator why a particular market participant is considered an outlier. Theoutlier detection method needs to implement interpretability. This led us to the formulation of ourresearch question as: How can an outlier detection method be implemented as a tool for amarket surveillance operator to identify potential market manipulation outside Scila's rule-basedsystems?Two models, an outlier detection model Isolation Forest, and a feature importance model (MI-Local-DIFFI and its subset Path Length Indicator) were chosen to fulfill the purpose of the study.The study used three datasets, two synthetic datasets, one scattered and one clustered, andone dataset from Scila.The results show that Isolation Forest has an excellent ability to find outliers in the various datadistributions we investigated. We used a feature importance model to make Isolation Forest’sscoring of outliers interpretable. Our intention was that the feature importance model wouldspecify how important different features were in the process of an observation being defined asan outlier. Our results have a relatively high degree of interpretability for the scattered datasetbut worse for the clustered dataset. The Path Length Indicator achieved better performancethan MI-Local-DIFFI for both datasets. We noticed that the chosen feature importance model islimited by the process of how Isolation Forest isolates an outlier.

<p dir="ltr">Motion planning for Autonomous Vehicles (AVs) and Connected Autonomous Vehicles (CAVs) involves the crucial task of translating road environmental data obtained from sensors and connectivity devices into a sequence of executable vehicle actions. This task is critical for AVs and CAVs, because the efficacy of their driving decisions and overall performance depend on the quality of motion planning.</p><p dir="ltr">In the context of motion planning technologies, several fundamental questions and challenges remain despite the widespread adoption of advanced learning-based methods, including deep learning (DL) and deep reinforcement learning (DRL). In this regard, the following critical questions need to be answered: 1) How to design suitable DL architectures to comprehensively understand the driving scenario by integrating data from diverse sources including sensors and connectivity devices? 2) How to effectively use the fused information to make improved driving decisions, accounting for various optimality criteria? 3) How to leverage vehicle connectivity to generate cooperative decisions for multiple CAVs, in a manner that optimizes system-wide utility? 4) How to address the inherent interpretability limitations of DL-based methods to enhance user trust in AVs and CAVs? 5) Is it possible to extend learning-based approaches to operational-level decisions in a way that overcomes the inherent disadvantage of low explainability and lack of safety guarantee?</p><p dir="ltr">In an effort to address these questions and expand the existing knowledge in this domain, this dissertation introduces several learning-based motion planning frameworks tailored towards different driving scenarios of AV and CAV. Technically, these efforts target on developing trustworthy AI systems with a focus on the information fusion, “explainable AI” or XAI and safety critical AI. From a computational perspective, these frameworks introduce new learning-based models with state-of-the-art (SOTA) structures, including Convolutional Neural Network (CNN). Recurrent Neural Networks (RNN), Graph Neural Networks (GNN), Attention networks, and Transformers. They also incorporate reinforcement learning (RL) agents, such as Deep Q Networks (DQN) and Model-based RL. From an application standpoint, these developed frameworks can be deployed directly in AVs and CAVs at Level 3 and above. This can enhance the AV/CAV performance in terms of individual and system performance metrics, including safety, mobility, efficiency, and driving comfort.</p>

Deep-learning models have been shown to produce accurate predictions in various scientific and engineering applications, such as turbulence modelling, by efficiently learning complex nonlinear relations from data. However, deep networks are often black boxes and it is not clear from the model parameters which inputs are more important to a prediction. As a result, it is difficult to understand whether models are taking into account physically relevant information and little theoretical understanding of the phenomenon modelled by the deep network can be gained.  In this work, methods from the field of explainable AI, based on Shapley Value approximation, are applied to compute feature attributions in previously trained fully convolutional deep neural networks for predicting velocity fluctuations in an open channel turbulent flow using wall quantities as inputs. The results show that certain regions in the inputs to the model have a higher importance to a prediction, which is verified by computational experiments that confirm the models are more sensitive to those inputs as compared to randomly selected inputs, if the error in the prediction is considered. These regions correspond to certain strongly distinguishable features (visible structures) in the model inputs. The correlations between the regions with high importance and visible structures in the model inputs are investigated with a linear regression analysis. The results indicate that certain physical characteristics of these structures are highly correlated to the importance of individual input features within these structures. / Djupinlärningsmodeller har visat sig kunna producera korrekta förutsägelser i olika vetenskapliga och tekniska tillämpningar, såsom turbulensmodellering, genom att effektivt lära sig komplexa olinjära relationer från data. Djupa neurala nätverk är dock ofta svarta lådor och det framgår inte av modellparametrarna vilka delar av indata som är viktigast för en förutsägelse. Som ett resultat av detta är det svårt att förstå om modellerna tar hänsyn till fysiskt relevant information och de ger inte heller någon teoretisk förståelse för fenomenet som modelleras av det djupa nätverket. I detta arbete tillämpas metoder från området för förklarabar AI, baserade på approximation av så kallde Shapley värden, för att beräkna vilka delar av indata som är viktigst för de prediktioner som görs. Detta görs för djupa neurala faltningsnätverk som tränats för att förutsäga hastighetsfluktuationer i ett turbulent flöde i en öppen kanal med hjälp av väggkvantiteter som indata. Resultaten visar att vissa regioner i indata till modellen har större betydelse för en förutsägelse. Detta verifieras av beräkningsexperiment som bekräftar att modellerna är mer känsliga för dessa indata jämfört med slumpmässigt valda indata, baserat på det resulterande felet i förutsägelser som görs av det tränade nätverket. Dessa regioner motsvarar vissa starkt särskiljbara egenskaper (synliga strukturer) i indata till modellen. Korrelationerna mellan regionerna med hög betydelse och synliga strukturer i indata undersöks med linjär regressionsanalys. Resultaten indikerar att vissa fysiska egenskaper hos dessa strukturer är starkt korrelerade med de approximativa Shapley värden som beräknats för dessa delar av indata.

Human-AI interaction can improve overall performance, exceeding the performance that either humans or AI could achieve separately, thus producing a whole greater than the sum of the parts. Visual analytics enables collaboration between humans and AI through interactive visual interfaces. Semantic interaction is a design methodology to enhance visual analytics systems for sensemaking tasks. It is widely applied for sensemaking in high-stakes domains such as intelligence analysis and academic research. However, existing semantic interaction systems support collaboration between humans and traditional machine learning models only; they do not apply state-of-the-art deep learning techniques. The contribution of this work is the effective integration of deep neural networks into visual analytics systems with semantic interaction. More specifically, I explore how to redesign the semantic interaction pipeline to enable collaboration between human and deep learning models for sensemaking tasks. First, I validate that semantic interaction systems with pre-trained deep learning better support sensemaking than existing semantic interaction systems with traditional machine learning. Second, I integrate interactive deep learning into the semantic interaction pipeline to enhance inference ability in capturing analysts' precise intents, thereby promoting sensemaking. Third, I add semantic explanation into the pipeline to interpret the interactively steered deep learning model. With a clear understanding of DL, analysts can make better decisions. Finally, I present a neural design of the semantic interaction pipeline to further boost collaboration between humans and deep learning for sensemaking. / Doctor of Philosophy / Human AI interaction can harness the separate strengths of human and machine intelligence to accomplish tasks neither can solve alone. Analysts are good at making high-level hypotheses and reasoning from their domain knowledge. AI models are better at data computation based on low-level input features. Successful human-AI interactions can perform real-world, high-stakes tasks, such as issuing medical diagnoses, making credit assessments, and determining cases of discrimination. Semantic interaction is a visual methodology providing intuitive communications between analysts and traditional machine learning models. It is commonly utilized to enhance visual analytics systems for sensemaking tasks, such as intelligence analysis and scientific research. The contribution of this work is to explore how to use semantic interaction to achieve collaboration between humans and state-of-the-art deep learning models for complex sensemaking tasks. To do this, I first evaluate the straightforward solution of integrating the pretrained deep learning model into the traditional semantic interaction pipeline. Results show that the deep learning representation matches human cognition better than hand engineering features via semantic interaction. Next, I look at methods for supporting semantic interaction systems with interactive and interpretable deep learning. The new pipeline provides effective communication between human and deep learning models. Interactive deep learning enables the system to better capture users' intents. Interpretable deep learning lets users have a clear understanding of models. Finally, I improve the pipeline to better support collaboration using a neural design. I hope this work can contribute to future designs for the human-in-the-loop analysis with deep learning and visual analytics techniques.

Yes / The rapid rise of many e-commerce platforms for individual consumers has generated a large amount of text-based data, and thus researchers have begun to experiment with text mining techniques to extract information from the large amount of textual data to assist in sales forecasting. The existing literature focuses textual data on product reviews; however, consumer reviews are not something that companies can directly control, here we argue that textual product descriptions are also important determinants of consumer choice. We construct an artificial intelligence (AI) framework that combines text mining, WaveNet neural networks, multiple regression, and SHAP model to explain the impact of product descriptions on sales forecasting. Using data from nearly 200,000 sales records obtained from a cross-border e-commerce firm, an empirical study showed that the product description presented to customers can influence sales forecasting, and about 44% of the key phrases greatly affect sales forecasting results, the sales forecasting models that added key product description phrases had improved forecasting accuracy. This paper provides explainable results of sales forecasting, which can provide guidance for firms to design product descriptions with reference to the market demand reflected by these phrases, and adding these phrases to product descriptions can help win more customers. / The full-text of this article will be released for public view at the end of the publisher embargo on 24 Feb 2025.

The growing presence of algorithm-generated recommendations in AI-powered services highlights the importance of responsible systems that explain outputs in a human-understandable form, especially in an automotive context. Implementing explainability in recommendations of AI-powered eco-driving is important in ensuring that drivers understand the underlying reasoning behind the recommendations. Previous literature on explainable AI (XAI) has been primarily technological-centered, and only a few studies involve the end-user perspective. There is a lack of knowledge of drivers' needs and requirements for explainability in an AI-powered eco-driving context. This study addresses the attributes that make a “satisfactory” explanation, i,e., a satisfactory interface between humans and AI. This study uses scenario-based interviews to understand the explainability attributes that influence truck drivers' intention to use eco-driving recommendations. The study used thematic analysis to categorize seven attributes into context-dependent (Format, Completeness, Accuracy, Timeliness, Communication) and generic (Reliability, Feedback loop) categories. The study contributes context-dependent attributes along three design dimensions: Presentational, Content-related, and Temporal aspects of explainability. The findings of this study present an empirical foundation into end-users' explainability needs and provide valuable insights for UX and system designers in eliciting end-user requirements.