Knowledge bases are employed in a variety of applications, from natural language processing to semantic web search; alas, in practice, their usefulness is hurt by their incompleteness. To address this issue, several techniques aim at performing knowledge base completion, of which embedding models are efficient, attain state-of-the-art accuracy, and eliminate the need for feature engineering. However, embedding models predictions are notoriously hard to interpret. In this work, we propose model-agnostic methods that allow one to interpret embedding models by extracting weighted Horn rules from them. More specifically, we show how the so-called \"pedagogical techniques\", from the literature on neural networks, can be adapted to take into account the large-scale relational aspects of knowledge bases, and show experimentally their strengths and weaknesses. / Bases de conhecimento apresentam diversas aplicações, desde processamento de linguagem natural a pesquisa semântica da web; contudo, na prática, sua utilidade é prejudicada por não serem totalmente completas. Para solucionar esse problema, diversas técnicas focam em completar bases de conhecimento, das quais modelos de embedding são eficientes, atingem estado da arte em acurácia, e eliminam a necessidade de fazer-se engenharia de características dos dados de entrada. Entretanto, as predições dos modelos de embedding são notoriamente difíceis de serem interpretadas. Neste trabalho, propomos métodos agnósticos a modelo que permitem interpretar modelos de embedding através da extração de regras Horn ponderadas por pesos dos mesmos. Mais espeficicamente, mostramos como os chamados \"métodos pedagógicos\", da literatura de redes neurais, podem ser adaptados para lidar com os aspectos relacionais e de larga escala de bases de conhecimento, e mostramos experimentalmente seus pontos fortes e fracos.

Machine Learning (ML) and Artificial Intelligence (AI) impact many aspects of human life, from recommending a significant other to assist the search for extraterrestrial life. The area develops rapidly and exiting unexplored design spaces are constantly laid bare. The focus in this work is one of these areas; ML systems where decisions concerning ML model training, usage and selection of target domain lay in the hands of domain experts.  This work is then on ML systems that function as a tool that augments and/or enhance human capabilities. The approach presented is denoted Human In Command ML (HIC-ML) systems. To enquire into this research domain design experiments of varying fidelity were used. Two of these experiments focus on augmenting human capabilities and targets the domains commuting and sorting batteries. One experiment focuses on enhancing human capabilities by identifying similar hand-painted plates. The experiments are used as illustrative examples to explore settings where domain experts potentially can: independently train an ML model and in an iterative fashion, interact with it and interpret and understand its decisions.  HIC-ML should be seen as a governance principle that focuses on adding value and meaning to users. In this work, concrete application areas are presented and discussed. To open up for designing ML-based products for the area an abstract model for HIC-ML is constructed and design guidelines are proposed. In addition, terminology and abstractions useful when designing for explicability are presented by imposing structure and rigidity derived from scientific explanations. Together, this opens up for a contextual shift in ML and makes new application areas probable, areas that naturally couples the usage of AI technology to human virtues and potentially, as a consequence, can result in a democratisation of the usage and knowledge concerning this powerful technology.

A robot is proficient if its performance for its task(s) satisfies a specific standard. While the design of autonomous robots often emphasizes such proficiency, another important attribute of autonomous robot systems is their ability to evaluate their own proficiency. A robot should be able to conduct proficiency self-assessment (PSA), i.e. assess how well it can perform a task before, during, and after it has attempted the task. We propose the assumption-alignment tracking (AAT) method, which provides time-indexed assessments of the veracity of robot generators' assumptions, for designing autonomous robots that can effectively evaluate their own performance. AAT can be considered as a general framework for using robot sensory data to extract useful features, which are then used to build data-driven PSA models. We develop various AAT-based data-driven approaches to PSA from different perspectives. First, we use AAT for estimating robot performance. AAT features encode how the robot's current running condition varies from the normal condition, which correlates with the deviation level between the robot's current performance and normal performance. We use the k-nearest neighbor algorithm to model that correlation. Second, AAT features are used for anomaly detection. We treat anomaly detection as a one-class classification problem where only data from the robot operating in normal conditions are used in training, decreasing the burden on acquiring data in various abnormal conditions. The cluster boundary of data points from normal conditions, which serves as the decision boundary between normal and abnormal conditions, can be identified by mainstream one-class classification algorithms. Third, we improve PSA models that predict robot success/failure by introducing meta-PSA models that assess the correctness of PSA models. The probability that a PSA model's prediction is correct is conditioned on four features: 1) the mean distance from a test sample to its nearest neighbors in the training set; 2) the predicted probability of success made by the PSA model; 3) the ratio between the robot's current performance and its performance standard; and 4) the percentage of the task the robot has already completed. Meta-PSA models trained on the four features using a Random Forest algorithm improve PSA models with respect to both discriminability and calibration. Finally, we explore how AAT can be used to generate a new type of explanation of robot behavior/policy from the perspective of a robot's proficiency. AAT provides three pieces of information for explanation generation: (1) veracity assessment of the assumptions on which the robot's generators rely; (2) proficiency assessment measured by the probability that the robot will successfully accomplish its task; and (3) counterfactual proficiency assessment computed with the veracity of some assumptions varied hypothetically. The information provided by AAT fits the situation awareness-based framework for explainable artificial intelligence. The efficacy of AAT is comprehensively evaluated using robot systems with a variety of robot types, generators, hardware, and tasks, including a simulated robot navigating in a maze-based (discrete time) Markov chain environment, a simulated robot navigating in a continuous environment, and both a simulated and a real-world robot arranging blocks of different shapes and colors in a specific order on a table.

This paper investigates the impact the LASSO, mRMR, SHAP, and Reinforcement Feature Selection techniques on random forest models for the breast cancer subtypes markers ER, HER2, PR, and TN as well as identifying a small subset of biomarkers that could potentially cause the disease and explain them using explainable AI techniques. This is important because in areas such as healthcare understanding why the model makes a specific decision is important it is a diagnostic of an individual which requires reliable AI. Another contribution is using feature selection methods to identify a small subset of biomarkers capable of predicting if a specific RNA sequence will have one of the cancer labels positive. The study begins by obtaining baseline accuracy metric using a random forest model on The Cancer Genome Atlas's breast cancer database to then explore the effects of feature selection, selecting different numbers of features, significantly influencing model accuracy, and selecting a small number of potential biomarkers that may produce a specific type of breast cancer. Once the biomarkers were selected, the explainable AI techniques SHAP and LIME were applied to the models and provided insight into influential biomarkers and their impact on predictions. The main results are that there are some shared biomarkers between some of the subsets that had high influence over the model prediction, LASSO and Reinforcement Feature selection sets scoring the highest accuracy of all sets and obtaining some insight into how the models used the features by using existing explainable AI methods SHAP and LIME to understand how these selected features are affecting the model's prediction.

The study examines how explainable AI (XAI), and transparency can increase trust and facilitate the adoption of AI technologies within small and medium-sized enterprises (SMEs). These businesses face significant challenges in integrating AI due to limited technical expertise and resources. The purpose of the study is to explore how XAI could bridge the gap between complex AI models and human understanding, thereby enhancing trust and operational efficiency.   The research methodology includes a case study with a literature review and expert interviews. The literature review provides background and context for the research question, while the expert interviews gather insights from employees in various roles and with different levels of experience within the participating SMEs. This approach offers a comprehensive understanding of the current state of AI adoption and the perceived importance of XAI and transparency.   The results indicate a significant knowledge gap among SME employees regarding AI technologies, with many expressing a lack of familiarity and trust. However, there is strong consensus on the importance of transparency and explainability in AI systems. Participants noted that XAI could significantly improve trust and acceptance of AI technologies by making AI decisions more understandable and transparent. Specific benefits identified include better decision support, increased operational efficiency, and enhanced customer confidence.   The study concludes that XAI and transparency are crucial for building trust and facilitating the adoption of AI technologies in SMEs. By making AI systems more comprehensible, XAI addresses the challenges posed by limited technical expertise and promotes broader acceptance of AI. The research emphasizes the need for continuous education and clear communication strategies to improve AI understanding among stakeholders within SMEs.   To enhance transparency and user trust in AI systems, SMEs should prioritize the integration of XAI frameworks. It is essential to develop user-centered tools that provide clear explanations of AI decisions and to invest in ongoing education and training programs. Additionally, a company culture that values transparency and ethical AI practices would further support the successful adoption of AI technologies. The study contributes to the ongoing discourse on AI adoption in SMEs by providing empirical evidence on the role of XAI in building trust and improving transparency. It offers practical recommendations for SMEs to effectively leverage AI technologies while ensuring ethical and transparent AI practices in line with regulatory requirements and societal expectations. / Studien undersöker hur förklarbar AI (XAI) och transparens kan öka förtroendet och underlätta införandet av AI-teknologier inom små och medelstora företag (SME). Dessa företag står inför betydande utmaningar vid integrationen av AI på grund av begränsad teknisk expertis och resurser. Syftet med studien är att undersöka hur XAI kan överbrygga klyftan mellan komplexa AI-modeller och mänsklig förståelse, vilket i sin tur främjar förtroende och operationell effektivitet.   Forskningsmetodiken inkluderar en fallstudie med en litteraturöversikt och expertintervjuer. Litteraturöversikten ger bakgrund och kontext till forskningsfrågan, medan expertintervjuerna samlar insikter från anställda i olika roller och med olika erfarenhetsnivåer i de deltagande SMEs. Detta tillvägagångssätt gav en omfattande förståelse av det nuvarande tillståndet för AI adoption och den upplevda vikten av XAI och transparens.   Resultaten visar på en betydande kunskapslucka bland SME-anställda när det gäller AI teknologier, med många som uttrycker en brist på bekantskap och förtroende. Det råder dock stark enighet om vikten av transparens och förklarbarhet i AI-system. Deltagarna angav att XAI avsevärt kunde förbättra förtroendet och acceptansen av AI-teknologier genom att göra AI beslut mer förståeliga och transparenta. Specifika fördelar som identifierades inkluderar bättre beslutsstöd, ökad operationell effektivitet och ökat kundförtroende.   Studien drar slutsatsen att XAI och transparens är avgörande för att skapa förtroende och underlätta införandet av AI-teknologier i SME. Genom att göra AI-system mer förståeliga adresserar XAI utmaningarna med begränsad teknisk expertis och främjar en bredare acceptans av AI. Forskningen understryker behovet av kontinuerlig utbildning och tydliga kommunikationsstrategier för att förbättra AI-förståelsen bland intressenter inom SME.   För att öka transparensen och användarförtroendet i AI-system bör SME prioritera integrationen av XAI-ramverk. Det är viktigt att utveckla användarcentrerade verktyg som ger tydliga förklaringar av AI-beslut och att investera i kontinuerliga utbildnings- och träningsprogram. Dessutom kommer en organisationskultur som värderar transparens och etiska AI-praktiker ytterligare stödja det framgångsrika införandet av AI-teknologier. Studien bidrar till den pågående diskursen om AI-adoption i SME genom att tillhandahålla empiriska bevis på rollenav XAI i att bygga förtroende och förbättra transparens. Den erbjuder praktiska rekommendationer för SME att effektivt utnyttja AI-teknologier, och säkerställa etiska och transparenta AI-praktiker som är i linje med regulatoriska krav och samhälleliga förväntningar.

This thesis delves into eye tracking, a technique for estimating an individual’s point of gaze and understanding human interactions with the environment. A blossoming area within eye tracking is appearance-based eye tracking, which leverages deep neural networks to predict gaze positions from eye images. Despite its efficacy, the decision-making processes inherent in deep neural networks remain as ’black boxes’ to humans. This lack of transparency challenges the trust human professionals place in the predictions of appearance-based eye tracking models. To address this issue, explainable AI is introduced, aiming to unveil the decision-making processes of deep neural networks and render them comprehensible to humans. This thesis employs various post-hoc explainable AI methods, including saliency maps, gradient-weighted class activation mapping, and guided backpropagation, to generate heat maps of eye images. These heat maps reveal discriminative areas pivotal to the model’s gaze predictions, and glints emerge as of paramount importance. To explore additional features in gaze estimation, a glint-free dataset is derived from the original glint-preserved dataset by employing blob detection to eliminate glints from each eye image. A corresponding glint-free model is trained on this dataset. Cross-evaluations of the two datasets and models discover that the glint-free model extracts complementary features (pupil, iris, and eyelids) to the glint-preserved model (glints), with both feature sets exhibiting comparable intensities in heat maps. To make use of all the features, an augmented dataset is constructed, incorporating selected samples from both glint-preserved and glint-free datasets. An augmented model is then trained on this dataset, demonstrating a superior performance compared to both glint-preserved and glint-free models. The augmented model excels due to its training process on a diverse set of glint-preserved and glint-free samples: it prioritizes glints when of high quality, and adjusts the focus to the entire eye in the presence of poor glint quality. This exploration enhances the understanding of the critical factors influencing gaze prediction and contributes to the development of more robust and interpretable appearance-based eye tracking models. / Denna avhandling handlar om ögonspårning, en teknik för att uppskatta en individs blickpunkt och förstå människors interaktioner med miljön. Ett viktigt område inom ögonspårning är bildbaserad ögonspårning, som utnyttjar djupa neuronnät för att förutsäga blickpositioner från ögonbilder. Trots dess effektivitet förblir beslutsprocesserna i djupa neuronnät som ”svarta lådor” för människor. Denna brist på transparens utmanar det förtroende som yrkesverksamma sätter i förutsägelserna från bildbaserade ögonspårningsmodeller. För att ta itu med detta problem introduceras förklarbar AI, med målet att avslöja beslutsprocesserna hos djupa neuronnät och göra dem begripliga för människor. Denna avhandling använder olika efterhandsmetoder för förklarbar AI, inklusive saliency maps, gradient-weighted class activation mapping och guidad backpropagation, för att generera värmekartor av ögonbilder. Dessa värmekartor avslöjar områden som är avgörande för modellens blickförutsägelser, och ögonblänk framstår som av yttersta vikt. För att utforska ytterligare funktioner i blickuppskattning, härleds ett dataset utan ögonblänk från det ursprungliga datasetet genom att använda blobdetektering för att eliminera blänk från varje ögonbild. En motsvarande blänkfri modell tränas på detta dataset. Korsutvärderingar av de två datamängderna och modellerna visar att den blänkfria modellen tar fasta på kompletterande särdrag (pupill, iris och ögonlock) jämfört med den blänkbevarade modellen, men båda modellerna visar jämförbara intensiteter i värmekartorna. För att utnyttja all information konstrueras ett förstärkt dataset, som inkorporerar utvalda exempel från både blänkbevarade och blänkfria dataset. En förstärkt modell tränas sedan på detta dataset, och visar överlägsen prestanda jämfört med de båda andra modellerna. Den förstärkta modellen utmärker sig på grund av sin träning på en mångfaldig uppsättning av exempel med och utan blänk: den prioriterar blänk när de är av hög kvalitet och justerar fokuset till hela ögat vid dålig blänkkvalitet. Detta arbete förbättrar förståelsen för de kritiska faktorerna som påverkar blickförutsägelse och bidrar till utvecklingen av mer robusta och tolkningsbara modeller för bildbaserad ögonspårning.

The rapid advancement of cutting-edge techniques has propelled state-of-the-art (SOTA) language models to new heights. Despite their impressive capabilities across a variety of downstream tasks, large language models still face many challenges such as hallucination and bias. The thesis focuses on two key objectives: first, it measures the robustness of T0_3B and investigates feasible methodologies to enhance the model’s robustness. Second, it targets on the explainability of large language models, aiming to make the intrinsic working mechanism more transparent and, consequently enhance model’s steerability. Motivated by the importance of mitigating non-robust behavior in language models, the thesis initially measures model’s robustness on handling minor perturbation. After that, I proposed and verified an approach to enhance robustness by making input more contextualized, a method that does not require the step of fine-tuning. Moreover, to understand the complex working mechanism of large language models, I designed and introduced two novel visualization tools: ’Logit Lens’ and ’Hidden States Plot in Spherical Coordinate System’. These tools, combined with additional experimental analysis, revealed a noticeable differentiation of the predicted processes between the first predicted token and subsequent tokens. The contributions of the thesis are mainly in the two following aspects: it provides feasible methodologies to enhance the robustness of language models without the need of fine-tuning, and it contributes to the field of explainable AI through the development of two visualization tools that shed light on the understanding of the working mechanism. / Den snabba utvecklingen av banbrytande tekniker har drivit språkmodeller till nya höjder. Trots deras imponerande prestanda över diverse språkrelaterade uppgifter, trots detta har dessa modeller fortfarande problem som hallucinationer och bias. Avhandlingen är centrerad kring två huvudmål: för det första undersöker den robustheten hos T0_3B och undersöker framtida strategier för att förbättra dess robusthet. För det andra utforskar den språkmodellernas ”förklaringsbarhet” (dvs hur väl vi förstår deras beteende), i syfte att göra dem mer transparenta och följaktligen förbättra modellens styrbarhet. Det första vi gör är att visa experiment som vi har satt upp för att mäta modellens robusthet mot mindre störningar. Som svar föreslår och underbygger vi ett tillvägagångssätt för att öka robustheten genom att ge modellen mer kontext när en fråga ställs, en metod som inte kräver vidare träning av modellen. Dessutom, för att förstå den komplexiteten hos språkmodeller, introducerar jag två nya visualiseringsverktyg: Logit Lens och Hidden States Plot i sfäriskt koordinatsystem. Dessa verktyg, i kombination med ytterligare experimentell analys, avslöjar ett diskting mönstr för den första förutspådda ordet jämfört med efterföljande ord. Bidragen från avhandlingen är huvudsakligen i de två följande aspekterna: den ger praktiska åtgärder för att förbättra robustheten hos språkmodeller utan behov av vidare träning, och den bidrar till området för förklarabar AI genom utvecklingen av två visualiseringsverktyg som ökar våran förståelse för hur dessa modeller fungerar.

Systematic review of research manuscripts is a common procedure in which research studies pertaining a particular field or domain are classified and structured in a methodological way. This process involves, between other steps, an extensive review and consolidation of scientific metrics and attributes of the manuscripts, such as citations, type or venue of publication. The extraction and mapping of relevant publication data, evidently, is a very laborious task if performed manually. Automation of such systematic mapping steps intend to reduce the human effort required and therefore can potentially reduce the time required for this process.The objective of this thesis is to automate the data extraction and mapping steps when systematically reviewing studies. The manual process is replaced by novel graph modelling techniques for effective knowledge representation, as well as novel machine learning techniques that aim to learn these representations. This eventually automates this process by characterising the publications on the basis of certain sub-properties and qualities that give the reviewer a quick high-level overview of each research study. The final model is a concept learner that predicts these sub-properties which in addition addresses the inherent concept-drift of novel manuscripts over time. Different models were developed and explored in this research study for the development of concept learner.Results show that: (1) Graph reasoning techniques which leverage the expressive power in modern graph databases are very effective in capturing the extracted knowledge in a so-called knowledge graph, which allows us to form concepts that can be learned using standard machine learning techniques like logistic regression, decision trees and neural networks etc. (2) Neural network models and ensemble models outperformed other standard machine learning techniques like logistic regression and decision trees based on the evaluation metrics. (3) The concept learner is able to detect and avoid concept drift by retraining the model. / Systematisk granskning av forskningsmanuskript är en vanlig procedur där forskningsstudier inom ett visst område klassificeras och struktureras på ett metodologiskt sätt. Denna process innefattar en omfattande granskning och sammanförande av vetenskapliga mätvärden och attribut för manuskriptet, såsom citat, typ av manuskript eller publiceringsplats. Framställning och kartläggning av relevant publikationsdata är uppenbarligen en mycket mödosam uppgift om den utförs manuellt. Avsikten med automatiseringen av processen för denna typ av systematisk kartläggning är att minska den mänskliga ansträngningen, och den tid som krävs kan på så sätt minskas. Syftet med denna avhandling är att automatisera datautvinning och stegen för kartläggning vid systematisk granskning av studier. Den manuella processen ersätts av avancerade grafmodelleringstekniker för effektiv kunskapsrepresentation, liksom avancerade maskininlärningstekniker som syftar till att lära maskinen dessa representationer. Detta automatiserar så småningom denna process genom att karakterisera publikationerna beserat på vissa subjektiva egenskaper och kvaliter som ger granskaren en snabb god översikt över varje forskningsstudie. Den slutliga modellen är ett inlärningskoncept som förutsäger dessa subjektiva egenskaper och dessutom behandlar den inneboende konceptuella driften i manuskriptet över tiden. Olika modeller utvecklades och undersöktes i denna forskningsstudie för utvecklingen av inlärningskonceptet. Resultaten visar att: (1) Diagrammatiskt resonerande som uttnytjar moderna grafdatabaser är mycket effektiva för att fånga den framställda kunskapen i en så kallad kunskapsgraf, och gör det möjligt att vidareutveckla koncept som kan läras med hjälp av standard tekniker för maskininlärning. (2) Neurala nätverksmodeller och ensemblemodeller överträffade andra standard maskininlärningstekniker baserat på utvärderingsvärdena. (3) Inlärningskonceptet kan detektera och undvika konceptuell drift baserat på F1-poäng och omlärning av algoritmen.

The influence of Artificial Intelligence (AI) on society is increasing, with applications in highly sensitive and complicated areas. Examples include using Deep Convolutional Neural Networks within healthcare for diagnosing cancer. However, the inner workings of such models are often unknown, limiting the much-needed trust in the models. To combat this, Explainable AI (XAI) methods aim to provide explanations of the models' decision-making. Two such methods, Spectral Relevance Analysis (SpRAy) and Testing with Concept Activation Methods (TCAV), were evaluated on a deep learning model classifying cat and dog images that contained introduced artificial noise. The task was to assess the methods' capabilities to explain the importance of the introduced noise for the learnt model. The task was constructed as an exploratory step, with the future aim of using the methods on models diagnosing oral cancer. In addition to using the TCAV method as introduced by its authors, this study also utilizes the CAV-sensitivity to introduce and perform a sensitivity magnitude analysis. Both methods proved useful in discerning between the model’s two decision-making strategies based on either the animal or the noise. However, greater insight into the intricacies of said strategies is desired. Additionally, the methods provided a deeper understanding of the model’s learning, as the model did not seem to properly distinguish between the noise and the animal conceptually. The methods thus accentuated the limitations of the model, thereby increasing our trust in its abilities. In conclusion, the methods show promise regarding the task of detecting visually distinctive noise in images, which could extend to other distinctive features present in more complex problems. Consequently, more research should be conducted on applying these methods on more complex areas with specialized models and tasks, e.g. oral cancer.

Requirements analysis is an essential part of the System Development Life Cycle (SDLC) in order to achieve success in a software development project. There are several methods, techniques and frameworks used when expressing, prioritizing and managing requirements in IT projects. It is widely established that it is difficult to determine requirements for traditional systems, so a question naturally arises on how the requirements analysis is executed as AI solutions (that even fewer individuals can grasp) are being developed. Little research has been made on how the vital requirements phase is executed during development of AI solutions. This research aims to investigate the requirements analysis phase during the development of AI solutions. To explore this topic, an extensive literature review was made, and in order to collect new information, a number of interviews were performed with five suitable organizations (i.e, organizations that develop AI solutions). The results from the research concludes that the requirements analysis does not differ between development of AI solutions in comparison to development of traditional systems. However, the research showed that there were some deviations that can be deemed to be particularly unique for the development of AI solutions that affects the requirements analysis. These are: (1) the need for an iterative and agile systems development process, with an associated iterative and agile requirements analysis, (2) the importance of having a large set of quality data, (3) the relative deprioritization of user involvement, and (4) the difficulty of establishing timeframe, results/feasibility and the behavior of the AI solution beforehand.