Estimating each marketing channel’s impact on conversion can help advertisers develop strategies and spend their marketing budgets optimally. This problem is often referred to as attribution modelling, and it is gaining increasing attention in both the industry and academia as access to online tracking data improves. Focusing on achieving higher predictive performance, the Long Short- Term Memory (LSTM) architecture is currently trending as a data-driven solution to attribution modelling. However, such deep neural networks have been criticised for being difficult to interpret. Interpretability is critical, since channel attributions are generally obtained by studying how a model makes a binary conversion prediction given a sequence of clicks or views of ads in different channels. Therefore, this degree project studies and compares the quality of LSTM attributions, calculated with SHapleyAdditive exPlanations (SHAP), attention and fractional scores to three baseline models. The fractional score is the mean difference in a model’s predicted conversion probability with and without a channel. Furthermore, a synthetic data generator based on a Poisson process is developed and validated against real data to measure attribution quality as the Mean Absolute Error (MAE) between calculated attributions and the true causal relationships between channel clicks and conversions. The experimental results demonstrate that the quality of attributions is not unambiguously reflected by the predictive performance of LSTMs. In general, it is not possible to assume a high attribution quality solely based on high predictive performance. For example, all models achieve ~82% accuracy on real data, whereas LSTM Fractional and SHAP produce the lowest attribution quality of 0:0566 and 0:0311 MAE respectively. This can be compared to an improved MAE of 0:0058, which is obtained with a Last-Touch Attribution (LTA) model. The attribution quality also varies significantly depending on which attribution calculation method is used for the LSTM. This suggests that the ongoing quest for improved accuracy may be questioned and that it is not always justified to use an LSTM when aiming for high quality attributions. / Genom att estimera påverkan varje marknadsföringskanal har på konverteringar, kan annonsörer utveckla strategier och spendera sina marknadsföringsbudgetar optimalt. Det här kallas ofta attributionsmodellering och det får alltmer uppmärksamhet i både näringslivet och akademin när tillgången till spårningsinformation ökar online. Med fokus på att uppnå högre prediktiv prestanda är Long Short-Term Memory (LSTM) för närvarande en populär datadriven lösning inom attributionsmodellering. Sådana djupa neurala nätverk har dock kritiserats för att vara svårtolkade. Tolkningsbarhet är viktigt, då kanalattributioner generellt fås genom att studera hur en modell gör en binär konverteringsprediktering givet en sekvens av klick eller visningar av annonser i olika kanaler. Det här examensarbetet studerar och jämför därför kvaliteten av en LSTMs attributioner, beräknade med SHapley Additive exPlanations (SHAP), attention och fractional scores mot tre grundmodeller. Fractional scores beräknas som medelvärdesdifferensen av en modells predikterade konverteringssannolikhet med och utan en viss kanal. Därutöver utvecklas en syntetisk datagenerator baserad på en Poissonprocess, vilken valideras mot verklig data. Generatorn används för att kunna mäta attributionskvalitet som Mean Absolute Error (MAE) mellan beräknade attributioner och de verkliga kausala sambanden mellan kanalklick och konverteringar. De experimentella resultaten visar att attributionskvaliteten inte entydigt avspeglas av en LSTMs prediktiva prestanda. Det är generellt inte möjligt att anta en hög attributionskvalitet enbart baserat på en hög prediktiv prestanda. Alla modeller uppnår exempelvis ~82% prediktiv träffsäkerhet på verklig data, medan LSTM Fractional och SHAP ger den lägsta attributionskvaliteten på 0:0566 respektive 0:0311 MAE. Det här kan jämföras mot en förbättrad MAE på 0:0058, som erhålls med en Last-touch-modell. Kvaliteten på attributioner varierar också signifikant beroende på vilket metod för attributionsberäkning som används för LSTM. Det här antyder att den pågående strävan efter högre prediktiv träffsäkerhet kan ifrågasättas och att det inte alltid är berättigat att använda en LSTM när attributioner av hög kvalitet eftersträvas.

В работе сравниваются алгоритмы машинного обучения для задачи оценки вероятности дефолта заёмщика и применяются методы интерпретации локальных предсказаний. / The paper compares machine learning algorithms for the problem of estimating the probability of borrower default, and applies methods for interpreting local predictions.

As the use of AI in society is developing, the requirement of explainable algorithms has increased. A challenge with many modern machine learning algorithms is that they, due to their often complex structures, lack the ability to produce human-interpretable explanations. Research within explainable AI has resulted in methods that can be applied on top of non- interpretable models to motivate their decision bases. The aim of this thesis is to compare an unexplained machine learning model used in combination with an explanatory method, and a model that is explainable through its inherent structure. Random forest was the unexplained model in question and the explanatory method was SHAP. The explainable model was logistic regression, which is explanatory through its feature weights. The comparison was conducted within the area of creditworthiness and was based on predictive performance and explainability. Furthermore, the thesis intends to use these models to investigate what characterizes loan applicants who are likely to default. The comparison showed that no model performed significantly better than the other in terms of predictive performance. Characteristics of bad loan applicants differed between the two algorithms. Three important aspects were the applicant’s age, where they lived and whether they had a residential phone. Regarding explainability, several advantages with SHAP were observed. With SHAP, explanations on both a local and a global level can be produced. Also, SHAP offers a way to take advantage of the high performance in many modern machine learning algorithms, and at the same time fulfil today’s increased requirement of transparency. / I takt med att AI används allt oftare för att fatta beslut i samhället, har kravet på förklarbarhet ökat. En utmaning med flera moderna maskininlärningsmodeller är att de, på grund av sina komplexa strukturer, sällan ger tillgång till mänskligt förståeliga motiveringar. Forskning inom förklarar AI har lett fram till metoder som kan appliceras ovanpå icke- förklarbara modeller för att tolka deras beslutsgrunder. Det här arbetet syftar till att jämföra en icke- förklarbar maskininlärningsmodell i kombination med en förklaringsmetod, och en modell som är förklarbar genom sin struktur. Den icke- förklarbara modellen var random forest och förklaringsmetoden som användes var SHAP. Den förklarbara modellen var logistisk regression, som är förklarande genom sina vikter. Jämförelsen utfördes inom området kreditvärdighet och grundades i prediktiv prestanda och förklarbarhet. Vidare användes dessa modeller för att undersöka vilka egenskaper som var kännetecknande för låntagare som inte förväntades kunna betala tillbaka sitt lån. Jämförelsen visade att ingen av de båda metoderna presterande signifikant mycket bättre än den andra sett till prediktiv prestanda. Kännetecknande särdrag för dåliga låntagare skiljde sig åt mellan metoderna. Tre viktiga aspekter var låntagarens °ålder, vart denna bodde och huruvida personen ägde en hemtelefon. Gällande förklarbarheten framträdde flera fördelar med SHAP, däribland möjligheten att kunna producera både lokala och globala förklaringar. Vidare konstaterades att SHAP gör det möjligt att dra fördel av den höga prestandan som många moderna maskininlärningsmetoder uppvisar och samtidigt uppfylla dagens ökade krav på transparens.

The field of wireless communications is arguably one of the most rapidly developing technological fields. Therefore, with each new advancement in this field, the complexity of wireless systems can grow significantly. This phenomenon is most visible in mobile communications, where the current 5G and 6G radio access networks (RANs) have reached unprecedented complexity levels to satisfy diverse increasing demands. In such increasingly complex environments, managing resources is becoming more and more challenging. Thus, experts employed performant artificial intelligence (AI) techniques to aid radio resource management (RRM) decisions. However, these AI techniques are often difficult to understand by humans, and may receive unimportant inputs which unnecessarily increase their complexity. In this work, we propose an explainability pipeline meant to be used for increasing humans’ understanding of AI models for RRM, as well as for reducing the complexity of these models, without loss of performance. To achieve this, the pipeline generates diverse feature importance explanations of the models with the help of three explainable artificial intelligence (XAI) methods: Kernel SHAP, CERTIFAI, and Anchors, and performs an importance-based feature selection using one of three different strategies. In the case of Anchors, we formulate and utilize a new way of computing feature importance scores, since no current publication in the XAI literature suggests a way to do this. Finally, we applied the proposed pipeline to a reinforcement learning (RL)- based RRM system. Our results show that we could reduce the complexity of the RL model between ∼ 27.5% and ∼ 62.5% according to different metrics, without loss of performance. Moreover, we showed that the explanations produced by our pipeline can be used to answer some of the most common XAI questions about our RL model, thus increasing its understandability. Lastly, we achieved an unprecedented result showing that our RL agent could be completely replaced with Anchors rules when taking RRM decisions, without a significant loss of performance, but with a considerable gain in understandability. / Området trådlös kommunikation är ett av de snabbast utvecklande tekniska områdena, och varje framsteg riskerar att medföra en signifikant ökning av komplexiteten för trådlösa nätverk. Det här fenomenet är som tydligast i mobil kommunikaiton, framför allt inom 5G och 6G radioaccessnätvärk (RANs) som har nåt nivåer av komplexitet som saknar motstycke. Detta för att uppfylla de ökande kraven som ställs på systemet. I dessa komplexa system blir resurshantering ett ökande problem, därför används nu artificiell intelligens (AI) allt mer för att ta beslut om hantering av radioresurser (RRM). Dessa AI tekniker är dock ofta svåra att förstå för människor, och kan således ges oviktig input vilket leder till att öka AI modellernas komplexitet. I detta arbete föreslås en förklarande pipeline vars mål är att användas för att öka människors förståelse av AI modeller för RRM. Målet är även att minska modellernas komplexitet, utan att förlora prestanda. För att åstadkomma detta genererar pipelinen förklaringar av betydande predikatorer för modellen med hjälp av tre metoder för förklarande artificiell intelligens (XAI). Dessa tre metoder är, Kernel SHAP, CERTIFAI och Anchors. Sedan görs ett predikatorurval baserat på predikatorbetydelse med en av dessa tre metoder. För metoden Anchors formuleras ett nytt sätt att beräkna betydelsen hos predikatorer, eftersom tidigare forskning inte föreslår någon metod för detta. Slutligen appliceras den föreslagna pipelinen på en förstärkt inlärnings- (RL) baserat RRM system. Resultaten visar att komplexiteten av RL modellen kunde reduceras med mellan ∼ 27, 5% och ∼ 62, 5% baserat på olika nyckeltal:er, utan att förlora någon prestanda. Utöver detta visades även att förklaringarna som producerats kan användas för att svara på de vanligaste XAI frågoran om RL modellen, och på det viset har även förståelsen för modellen ökat. Sistnämnt uppnåddes enastående resultat som visade att RL modellen helt kunde ersättas med regler producerade av Anchor-metoden för beslut inom RRM, utan någon störra förlust av prestanda, men med an stor vinst i förståelse.

В данной работе был разработан и реализован комплексный подход к анализу и интерпретации пользовательских данных, собранных в рамках системы веб-аналитики. Применяя методы машинного обучения и аналитики данных, были исследованы и выявлены ключевые события пользователей, влияющие на определенные бизнес-метрики. Начальные этапы проекта включали сбор и предварительную обработку данных, с последующей кластеризацией для выявления скрытых взаимосвязей и структур. Использовались или тестировались различные библиотеки для объяснимости работы моделей машинного обучении, такие как Eli5 и SHAP. Для решения задачи тестировались кластеризации, включая K-средних, DBSCAN, спектральную кластеризацию и OPTICS. В качестве алгоритмов применялась логистическая регрессия, случайны лес и CatBoost. Применялась нейронная сеть. Для определения значимости признаков использовались методы Permutation Importance, с применением моделей логистической регрессии, случайного леса и нейронной сети. Основным результатом стала разработка скрипта, осуществляющего автоматический сбор, обработку данных и определение наиболее значимых событий. Полученный инструментарий значительно облегчает задачу аналитиков, помогая определять ключевые аспекты поведения пользователей и строить более эффективные стратегии взаимодействия. Применение полученных результатов имеет высокий потенциал для улучшения бизнес–решений и оптимизации работы с пользовательской аудиторией. / In this work, an integrated approach to the analysis and interpretation of user data collected within the framework of a web analytics system was developed and implemented. Using machine learning and data analytics methods, key user events that impact certain business metrics were investigated and identified. The initial stages of the project included data collection and pre-processing, followed by clustering to identify hidden relationships and structures. Various libraries have been used or tested to make machine learning models explainable, such as Eli5 and SHAP. Clusterings including K-means, DBSCAN, spectral clustering, and OPTICS were tested to solve the problem. The algorithms used were logistic regression, random forest and CatBoost. A neural network was used. To determine the significance of features, Permutation Importance methods were used using logistic regression, random forest and neural network models. The main result was the development of a script that automatically collects, processes data and determines the most significant events. The resulting tools greatly facilitate the task of analysts, helping to identify key aspects of user behavior and build more effective interaction strategies. The application of the results obtained has high potential for improving business decisions and optimizing work with the user audience.

As the complexity of deep learning model increases, the transparency of the systems does the opposite. It may be hard to understand the predictions a deep learning model makes, but even harder to understand why these predictions are made. Using eXplainable AI (XAI), we can gain greater knowledge of how the model operates and how the input in which the model receives can change its predictions. In this thesis, we apply Integrated Gradients (IG), an XAI method primarily used on image data and on datasets containing tabular and time-series data. We also evaluate how the results of IG differ from various types of models and how the change of baseline can change the outcome. In these results, we observe that IG can be applied to both sequenced and nonsequenced data, with varying results. We can see that the gradient baseline does not affect the results of IG on models such as RNN, LSTM, and GRU, where the data contains time series, as much as it does for models like MLP with nonsequenced data. To confirm this, we also applied IG to SVM models, which gave the results that the choice of gradient baseline has a significant impact on the results of IG.

No description available.

Computer simulation is a commonly applied technique for studying electronic warfare duels. This thesis aims to apply machine learning techniques to convert simulation output data into knowledge and insights regarding defensive actions for a ship facing multiple hostile missiles. The analysis may support tactical decision-making, hence the interpretability aspect of predictions is necessary to allow for human evaluation and understanding of impacts from the explanatory variables. The final distance for the threats to the target and the probability of the threats hitting the target was modeled using a multi-layer perceptron model with a multi-task approach, including custom loss functions. The results generated in this study show that the selected methodology is more successful than a baseline using regression models. Modeling the outcome with artificial neural networks results in a black box for decision making. Therefore the concept of interpretable machine learning was applied using a post-hoc approach. Given the learned model, the features considered, and the multiple threats, the feature contributions to the model were interpreted using Kernel SHapley Additive exPlanations (SHAP). The method consists of local linear surrogate models for approximating Shapley values. The analysis primarily showed that an increased seeker activation distance was important, and the increased time for defensive actions improved the outcomes. Further, predicting the final distance to the ship at the beginning of a simulation is important and, in general, a guidance of the actual outcome. The action of firing chaff grenades in the tracking gate also had importance. More chaff grenades influenced the missiles' tracking and provided a preferable outcome from the defended ship's point of view.

<p dir="ltr">Anomaly detection is an active research field that widely applies to commercial applications to detect unusual patterns or outliers. Time series anomaly detection provides valuable insights into mission and safety-critical applications using ever-growing temporal data, including continuous streaming time series data from the Internet of Things (IoT), sensor networks, healthcare, stock prices, computer metrics, and application monitoring. While Generative Adversarial Networks (GANs) demonstrate promising results in time series anomaly detection, the opaque nature of generative deep learning models lacks explainability and hinders broader adoption. Understanding the rationale behind model predictions and providing human-interpretable explanations are vital for increasing confidence and trust in machine learning (ML) frameworks such as GANs. This study conducted a structured and comprehensive assessment of post-hoc local explainability in GAN-based time series anomaly detection using SHapley Additive exPlanations (SHAP). Using publicly available benchmarking datasets approved by Purdue’s Institutional Review Board (IRB), this study evaluated state-of-the-art GAN frameworks identifying their advantages and limitations for time series anomaly detection. This study demonstrated a systematic approach in quantifying the extent of GAN-based time series anomaly explainability, providing insights for businesses when considering adopting generative deep learning models. The presented results show that GANs capture complex time series temporal distribution and are applicable for anomaly detection. The analysis from this study shows SHAP can identify the significance of contributing features within time series data and derive post-hoc explanations to quantify GAN-detected time series anomalies.</p>

<p>A wide range of optoelectronic applications demand semiconductors optimized for purpose.</p> <p>My research focused on data-driven identification of ABX3 Halide perovskite compositions for optimum photovoltaic absorption in solar cells.</p> <p>I trained machine learning models on previously reported datasets of halide perovskite band gaps based on first principles computations performed at different fidelities.</p> <p>Using these, I identified mixtures of candidate constituents at the A, B or X sites of the perovskite supercell which leveraged how mixed perovskite band gaps deviate from the linear interpolations predicted by Vegard's law of mixing to obtain a selection of stable perovskites with band gaps in the ideal range of 1 to 2 eV for visible light spectrum absorption.</p> <p>These models predict the perovskite band gap using the composition and inherent elemental properties as descriptors.</p> <p>This enables accurate, high fidelity prediction and screening of the much larger chemical space from which the data samples were drawn.</p> <p><br></p> <p>I utilized a recently published density functional theory (DFT) dataset of more than 1300 perovskite band gaps from four different levels of theory, added to an experimental perovskite band gap dataset of \textasciitilde{}100 points, to train random forest regression (RFR), Gaussian process regression (GPR), and Sure Independence Screening and Sparsifying Operator (SISSO) regression models, with data fidelity added as one-hot encoded features.</p> <p>I found that RFR yields the best model with a band gap root mean square error of 0.12 eV on the total dataset and 0.15 eV on the experimental points.</p> <p>SISSO provided compound features and functions for direct prediction of band gap, but errors were larger than from RFR and GPR.</p> <p>Additional insights gained from Pearson correlation and Shapley additive explanation (SHAP) analysis of learned descriptors suggest the RFR models performed best because of (a) their focus on identifying and capturing relevant feature interactions and (b) their flexibility to represent nonlinear relationships between such interactions and the band gap.</p> <p>The best model was deployed for predicting experimental band gap of 37785 hypothetical compounds.</p> <p>Based on this, we identified 1251 stable compounds with band gap predicted to be between 1 and 2 eV at experimental accuracy, successfully narrowing the candidates to about 3% of the screened compositions.</p>