<div>Many real-world domains are relational, consisting of objects (e.g., users and pa- pers) linked to each other in various ways. Because class labels in graphs are often only available for a subset of the nodes, semi-supervised learning for graphs has been studied extensively to predict the unobserved class labels. For example, we can pre- dict political views in a partially labeled social graph dataset and get expected gross incomes of movies in an actor/movie graph with a few labels. Recently, advances in representation learning for graph data have made great strides for the semi-supervised node classification. However, most of the methods have mainly focused on learning node representations by considering simple relational properties (e.g., random walk) or aggregating nearby attributes, and it is still challenging to learn complex inter- action patterns in partially labeled graphs and provide explanations on the learned representations. </div><div><br></div><div>In this dissertation, multiple methods are proposed to alleviate both challenges for semi-supervised node classification. First, we propose a graph neural network architecture, REGNN, that leverages local inferences for unlabeled nodes. REGNN performs graph convolution to enable label propagation via high-order paths and predicts class labels for unlabeled nodes. In particular, our proposed attention layer of REGNN measures the role equivalence among nodes and effectively reduces the noise, which is generated during the aggregation of observed labels from distant neighbors at various distances. Second, we also propose a neural network archi- tecture that jointly captures both temporal and static interaction patterns, which we call Temporal-Static-Graph-Net (TSGNet). The architecture learns a latent rep- resentation of each node in order to encode complex interaction patterns. Our key insight is that leveraging both a static neighbor encoder, that learns aggregate neigh- bor patterns, and a graph neural network-based recurrent unit, that captures complex interaction patterns, improves the performance of node classification. Lastly, in spite of better performance of representation learning on node classification tasks, neural network-based representation learning models are still less interpretable than the pre- vious relational learning models due to the lack of explanation methods. To address the problem, we show that nodes with high bridgeness scores have larger impacts on node embeddings such as DeepWalk, LINE, Struc2Vec, and PTE under perturbation. However, it is computationally heavy to get bridgeness scores, and we propose a novel gradient-based explanation method, GRAPH-wGD, to find nodes with high bridgeness efficiently. In our evaluations, our proposed architectures (REGNN and TSGNet) for semi-supervised node classification consistently improve predictive performance on real-world datasets. Our GRAPH-wGD also identifies important nodes as global explanations, which significantly change both predicted probabilities on node classification tasks and k-nearest neighbors in the embedding space after perturbing the highly ranked nodes and re-learning low-dimensional node representations for DeepWalk and LINE embedding methods.</div>

Brain tumors present the second highest cause of death among pediatric cancers. About 60% are located in the posterior fossa region of the brain; among the most frequent types the ones considered for this project were astrocytomas, medulloblastomas, and ependymomas. Diagnosis can be done either through invasive histopathology exams or by non-invasive magnetic resonance (MR) scans. The tumors listed can be difficult to diagnose, even for trained radiologists, so machine learning methods, in particular deep learning, can be useful in helping to assess a diagnosis. Deep learning has been investigated only in a few other studies.The dataset used included 115 different subjects, some with multiple scan sessions, for which there were 142 T2-w, 119 T1Gd-w, and 89 volumes that presented both MR modalities. 2D slices have been manually extracted from the registered and skull-stripped volumes in the transversal, sagittal, and frontal anatomical plane and have been preprocessed by normalizing them and selecting the slices containing the tumor. The scans employed are T2-w, T1Gd-w, and a combination of the two referred to as multimodal images. The images were divided session-wise into training, validation, and testing, using stratified cross-validation and have also been augmented. The convolutional neural networks (CNN) investigated were ResNet50, VGG16, and MobileNetV2. The model performances were evaluated for two-class and three-class classification tasks by computing the confusion matrix, accuracy, receiver operating characteristic curve (ROC), the area under the curve (AUROC), and F1-score. Moreover,  explanations for the behavior of networks were investigated using GradCAMs and occlusion maps. Preliminary investigations showed that the best plane and modality were the transversal one and T2-w images. Overall the best model was VGG16, for the two-class tasks the best classification was between astrocytomas and medulloblastomas which reached an F1-score of 0.86 for both classes on multimodal images, followed by astrocytomas and ependymomas with an F1-score of 0.76 for astrocytomas and 0.74 for ependymomas on T2-w, and last F1-score of 0.30 for ependymomas and 0.65 for medulloblastomas on multimodal images. The three-class classification reached F1-score values of 0.59 for astrocytomas, 0.46 for ependymomas, and 0.64 for medulloblastomas on T2-w images. GradCAMs and occlusion maps showed that VGG16 was able to focus mostly on the tumor region but that there also seemed to be other information in the background of the images that contributed to the final classification.To conclude, the classification of infratentorial pediatric brain tumors can be achieved with acceptable results by means of deep learning and using a single MR modality, though one might have to account for the dataset size, number of classes and class imbalance. GradCAMs and occlusion maps offer important insights into the decision process of the networks

I en framtid där förutsägelser och beslut som tas av maskininlärningssystem överträffar människors förmåga behöver systemen att vara tolkbara för att vi skall kunna lita på och förstå dem. Vår studie utforskar världen av tolkbar maskininlärning genom att designa och undersöka artefakter. Vi genomför experiment för att utforska förklarbarhet, tolkbarhet samt tekniska utmaningar att skapa maskininlärningsmodeller för att identifiera liknande men unika objekt. Slutligen genomför vi ett användartest för att utvärdera toppmoderna förklaringsverktyg i ett direkt mänskligt sammanhang. Med insikter från dessa experiment diskuterar vi den potentiella framtiden för detta fält / In a future where predictions and decisions made by machine learning systems outperform humans we need the systems to be interpretable in order for us to trust and understand them. Our study explore the realm of interpretable machine learning through designing artifacts. We conduct experiments to explore explainability, interpretability as well as technical challenges of creating machine learning models to identify objects that appear similar to humans. Lastly, we conduct a user test to evaluate current state-of-the-art visual explanatory tools in a human setting. From these insights, we discuss the potential future of this field.

This thesis presents a novel approach to make Artificial Intelligence (AI) more explainable by using a Belief Rule Based Expert System (BRBES). A BRBES is a type of expert system that can handle both qualitative and quantitative information under uncertainty and incompleteness by using if-then rules with belief degrees. The BRBES can model the human inference process and provide transparent and interpretable reasoning for its decisions. However, designing a BRBES requires tuning several parameters, such as the rule weights, the belief degrees, and the inference parameters. To address this challenge, this thesis report proposes to use a Differential Evolution (DE) algorithm to optimize these parameters automatically. A DE algorithm such as BRB adaptive DE (BRBaDE) and Joint Optimization of BRB is a metaheuristic that optimizes a problem by iteratively creating new candidate solutions by combining existing ones according to some simple formulae. The DE algorithm does not require any prior knowledge of the problem or its gradient, and can handle complex optimization problems with multiple objectives and constraints. This model can provide explainability by using different model agnostic method including Local Interpretable Model-agnostic Explanations (LIME) and SHapley Additive exPlanations (SHAP). The proposed approach is applied to calculate Air Quality Index (AQI) using particle data. The results show that the proposed approach can improve the performance and explainability of AI systems compared to other existing methods. Moreover, the proposed model can ensure the balance between accuracy and explainablity in comparison to other models.

Explainability is a key component in building trust for computer calculated predictions when they are applied to areas with influence over individual people. This bachelor thesis project report focuses on the explanation regarding the decision making process of the machine learning method Logistic Regression when predicting mortality. The aim is to present theoretical information about the predictive model as well as an explainable interpretation when applied on the clinical MIMIC-III database. The project found that there was a significant difference between particular features considering the impact of each individual feature on the classification. The feature that showed the greatest impact was the Glasgow Coma Scale value, which could be proven through the fact that a good classifier could be constructed with only that and one other feature. An important conclusion from this study is that a great focus should be enforced early in the implementation process when the features are selected. In this specific case, when medical artificial intelligence is implemented, medical expertise is desired in order to make a good feature selection. / Förklarbarhet är en viktig komponent för att skapa förtroende för datorframtagna prognoser när de appliceras på områden som påverkar individuella personer. Denna kandidatexamensarbetesrapport fokuserar på förklarandet av beslutsprocessen hos maskininlärningsmetoden Logistic Regression när dödlighet ska förutsägas. Målet är att presentera information om den förutsägande modellen samt en förklarbar tolkning av resultaten när modellen appliceras på den kliniska databasen MIMIC-III. Projektet fann att det fanns signifikanta skillnader mellan särskilda egenskaper med hänsyn till den påverkan varje enskild egenskap har på klassificeringen. Den egenskapen som visade ha störst inverkan var Glascow Coma Scale värdet, vilket kunde visas via det faktum att en god klassificerare kunde konstrueras med endast den och en annan egenskap. En viktig slutsats av denna studie är att stort fokus bör läggas tidigt i implementationsprocessen då egenskaperna väljs. I detta specifika fall, då medicinsk artificiell intelligens implementeras, krävs medicinsk expertis för att göra ett gott egenskapsurval. / Kandidatexjobb i elektroteknik 2022, KTH, Stockholm

Artificial Intelligence (AI) has proven to be highly suitable for a wide range of problems in manufacturing environments, including the prediction of lead times. Most of these solutions are based on ”black-box” algorithms, which hinder practitioners to understand the prediction process. Explainable Artificial Intelligence (XAI) provides numerous tools and methods to counteract this problem. There is however a need to qualify the methods with human-centered studies in manufacturing environments, since explainabilityis context-specific. The purpose of this mixed-method case study is to examine the explainability of regression models for lead time prediction in quality control laboratories at a biopharmaceutical production site in Sweden. This entails the research questions of which methods can increase the explainability of lead time prediction, what type of explanation is required to enable explainability and what are the benefits of explaining regression models in this context. This is why relevant literature in the field of XAI and AI-based lead time prediction is reviewed. An explainable lead time prediction modelis developed and a Delphi study is carried out to gauge the importance of different explanation types and to identify explainability-related benefits. The results show a transparency-performance trade-off and highlight eight benefits that are mapped to the model’s life cycle. These findings provide new insights into the explainability requirements and benefits in quality control processes and support practitioners in steering their implementation efforts. / Artificiell Intelligens (AI) har visat sig vara mycket lämplig för ett stort antal problem i tillverkningsmiljöer, bland annat när det gäller att förutsäga ledtider. De flesta av dessa lösningar är baserade på algoritmer som är ”svarta lådor”, vilket gör det svårt för tillämparna att förstå förutsägelseprocessen. Explainable Artificial Intelligence (XAI) erbjuder många verktyg och metoder för att motverka detta problem. Det finns dock ett behov av att kvalificera metoderna med människocentrerade studier i tillverkningsmiljöer, eftersom förklarbarhet är kontextspecifikt. Syftet med denna fallstudie med blandad metod är att undersöka förklaringsbarheten hos regressionsmodeller för prediktion av ledtider i kvalitets kontrolllaboratorier vid en biopharmaceutisk produktionsanläggning i Sverige. Vilket syftar till forskningsfrågorna samt vilka metoder som kan öka förklaringsbarheten och av prognoser för ledtider, vilken typ av förklaring som krävs för att möjliggöra en förklarbarhet och vilka fördelar som finns med att förklara regressionsmodeller i detta sammanhang. Det är därför som relevant litteratur på området XAI och AI baserade prognostisering av ledtider granskas. En förklaringsbar modell för prognostisering av ledtider utvecklas och en Delphi-studie genomförs för att bedöma betydelsen av olika typer av förklaringar och för att identifiera förklaringsrelaterade fördelar.

The rapid advancement of cutting-edge techniques has propelled state-of-the-art (SOTA) language models to new heights. Despite their impressive capabilities across a variety of downstream tasks, large language models still face many challenges such as hallucination and bias. The thesis focuses on two key objectives: first, it measures the robustness of T0_3B and investigates feasible methodologies to enhance the model’s robustness. Second, it targets on the explainability of large language models, aiming to make the intrinsic working mechanism more transparent and, consequently enhance model’s steerability. Motivated by the importance of mitigating non-robust behavior in language models, the thesis initially measures model’s robustness on handling minor perturbation. After that, I proposed and verified an approach to enhance robustness by making input more contextualized, a method that does not require the step of fine-tuning. Moreover, to understand the complex working mechanism of large language models, I designed and introduced two novel visualization tools: ’Logit Lens’ and ’Hidden States Plot in Spherical Coordinate System’. These tools, combined with additional experimental analysis, revealed a noticeable differentiation of the predicted processes between the first predicted token and subsequent tokens. The contributions of the thesis are mainly in the two following aspects: it provides feasible methodologies to enhance the robustness of language models without the need of fine-tuning, and it contributes to the field of explainable AI through the development of two visualization tools that shed light on the understanding of the working mechanism. / Den snabba utvecklingen av banbrytande tekniker har drivit språkmodeller till nya höjder. Trots deras imponerande prestanda över diverse språkrelaterade uppgifter, trots detta har dessa modeller fortfarande problem som hallucinationer och bias. Avhandlingen är centrerad kring två huvudmål: för det första undersöker den robustheten hos T0_3B och undersöker framtida strategier för att förbättra dess robusthet. För det andra utforskar den språkmodellernas ”förklaringsbarhet” (dvs hur väl vi förstår deras beteende), i syfte att göra dem mer transparenta och följaktligen förbättra modellens styrbarhet. Det första vi gör är att visa experiment som vi har satt upp för att mäta modellens robusthet mot mindre störningar. Som svar föreslår och underbygger vi ett tillvägagångssätt för att öka robustheten genom att ge modellen mer kontext när en fråga ställs, en metod som inte kräver vidare träning av modellen. Dessutom, för att förstå den komplexiteten hos språkmodeller, introducerar jag två nya visualiseringsverktyg: Logit Lens och Hidden States Plot i sfäriskt koordinatsystem. Dessa verktyg, i kombination med ytterligare experimentell analys, avslöjar ett diskting mönstr för den första förutspådda ordet jämfört med efterföljande ord. Bidragen från avhandlingen är huvudsakligen i de två följande aspekterna: den ger praktiska åtgärder för att förbättra robustheten hos språkmodeller utan behov av vidare träning, och den bidrar till området för förklarabar AI genom utvecklingen av två visualiseringsverktyg som ökar våran förståelse för hur dessa modeller fungerar.

To produce smooth human-robot interactions, it is important for robots to be able to answer users’ questions accurately and provide a suitable explanation for why they arrive to the answer they provide. However, in the wild, the user may ask the robot questions relating to aspects of the scene that the robot is unfamiliar with and hence be unable to answer correctly all of the time. In order to gain trust in the robot and resolve failure cases where an incorrect answer is provided, we propose a method that uses Grad-CAM explainability on RGB-D data. Depth is a critical component in producing more intelligent robots that can respond correctly most of the time as some questions might rely on spatial relations within the scene, for which 2D RGB data alone would be insufficient. To our knowledge, this work is the first of its kind to leverage depth and an explainability module to produce an explainable Visual Question Answering (VQA) system. Furthermore, we introduce a new dataset for the task of VQA on RGB-D data, VQA-SUNRGBD. We evaluate our explainability method against Grad-CAM on RGB data and find that ours produces better visual explanations. When we compare our proposed model on RGB-D data against the baseline VQN network on RGB data alone, we show that ours outperforms, particularly in questions relating to depth such as asking about the proximity of objects and relative positions of objects to one another. / För att skapa smidiga interaktioner mellan människa och robot är det viktigt för robotar att kunna svara på användarnas frågor korrekt och ge en lämplig förklaring till varför de kommer fram till det svar de ger. Men i det vilda kan användaren ställa frågor till roboten som rör aspekter av miljön som roboten är obekant med och därmed inte kunna svara korrekt hela tiden. För att få förtroende för roboten och lösa de misslyckade fall där ett felaktigt svar ges, föreslår vi en metod som använder Grad-CAM-förklarbarhet på RGB-D-data. Djup är en kritisk komponent för att producera mer intelligenta robotar som kan svara korrekt för det mesta, eftersom vissa frågor kan förlita sig på rumsliga relationer inom scenen, för vilka enbart 2D RGB-data skulle vara otillräcklig. Såvitt vi vet är detta arbete det första i sitt slag som utnyttjar djup och en förklaringsmodul för att producera ett förklarabart Visual Question Answering (VQA)-system. Dessutom introducerar vi ett nytt dataset för uppdraget av VQA på RGB-D-data, VQA-SUNRGBD. Vi utvärderar vår förklaringsmetod mot Grad-CAM på RGB-data och finner att vår modell ger bättre visuella förklaringar. När vi jämför vår föreslagna modell för RGB-Ddata mot baslinje-VQN-nätverket på enbart RGB-data visar vi att vår modell överträffar, särskilt i frågor som rör djup, som att fråga om objekts närhet och relativa positioner för objekt jämntemot varandra.

Artificial Neural Networks are successfully solving a wide range of problems with impressive performance. Nevertheless, often very little or nothing is understood in the workings behind these black-box solutions as they are hard to interpret, let alone to explain. This thesis proposes a set of complementary interpretable visualization models of neural activity, developed through prototyping, to answer the research question ”How may neural activity of Recurrent Neural Networks for text sequence prediction be represented, transformed and visualized during the inference process to explain interpretable functionality with respect to the text domain of some individual hidden neurons, as well as automatically detect these?”. Specifically, a Vanilla and a Long Short-Term Memory architecture are utilized for character respectively word prediction as testbeds. The research method is experimental; causalities between text features triggering neurons and detected patterns of corresponding nerve impulses are investigated. The result reveals not only that there exist neurons with clear and consistent feature-specific patterns of activity, but also that the proposed models of visualization successfully may automatically detect and interpretably present some of these. / Artificiella Neurala Nätverk löser framgångsrikt ett brett spektrum av problem med imponerande prestanda. Ändå är det ofta mycket lite eller ingenting som går att förstå bakom dessa svart-låda-lösningar, eftersom de är svåra att tolka och desto svårare att förklara. Den här uppsatsen föreslår en uppsättning komplementerande tolkningsbara visualiseringsmodeller av neural aktivitet, utvecklad genom prototypering, för att besvara forskningsfrågan ”Hur kan användningsprocessen av Rekurrenta Neurala Nätverk för textgenerering visualiseras på ett sätt för att automatiskt detektera och förklara tolkningsbar funktionalitet hos några enskilda dolda neuroner?”. Specifikt används en standardoch en LSTM (långt korttidsminne)-arkitektur för teckenrespektive ordprediktering som testbäddar. Forskningsmetoden är experimentell; orsakssamband mellan specifika typer av tecken/ord i texten som triggar neuroner, och detekterade mönster av motsvarande nervimpulser undersöks. Resultatet avslöjar inte bara att neuroner med tydliga och konsekventa tecken/ord-specifika aktivitetsmönster existerar, men också att de utvecklade modellerna för visualisering framgångsrikt kan automatiskt upptäcka och tolkningsbart presentera några av dessa.

The rate of Emergency Department (ED) visits due to mental health and drug abuse among children and youth has been increasing for more than a decade and is projected to become the leading cause of ED visits. Identifying high-risk patients well before an ED visit will enable mental health care providers to better predict ED resource utilization, improve their service, and ultimately reduce the risk of a future ED visit. Many studies in the literature utilized medical history to predict future hospitalization. However, in mental health care, the medical history of new patients is not always available from the first visit and it is crucial to identify high risk patients from the beginning as the rate of drop-out is very high in mental health treatment. In this study, a new approach of creating a text representation of questionnaire data for deep learning analysis is proposed. Employing this new text representation has enabled us to use transfer learning and develop a deep Natural Language Processing (NLP) model that estimates the possibility of 6-month ED visit among children and youth using mental health patient reported outcome measures (PROM). The proposed method achieved an Area Under Receiver Operating Characteristic Curve of 0.75 for classification of 6-month ED visit. In addition, a novel method was proposed to identify the words that carry the highest amount of information related to the outcome of the deep NLP models. This measurement of word information using Entropy Gain increases the explainability of the model by providing insight to the model attention. Finally, the results of this method were analyzed to explain how the deep NLP model achieved a high classification performance. / Dissertation / Master of Applied Science (MASc) / In this document, an Artificial Intelligence (AI) approach for predicting 6-month Emergency Department (ED) visits is proposed. In this approach, the questionnaires gathered from children and youth admitted to an outpatient or inpatient clinic are converted to a text representation called Textionnaire. Next, AI is utilized to analyze the Textionnaire and predict the possibility of a future ED visit. This method was successful in about 75% of the time. In addition to the AI solution, an explainability component is introduced to explain how the natural language processing algorithm identifies the high risk patients.