Credit scoring is critical for banks to evaluate new loan applications and monitor existing customers. Machine learning has been extensively researched for this case; however, the adoption of machine learning methods is minimal in financial risk management. The primary reason is that algorithms are viewed as ‘black box models’ and cannot satisfy regulatory requirements. While deep learning methods such as LSTM have been evaluated for behavioural credit scoring based on performance, research has not holistically evaluated these models on performance and explainability. To answer the research question, How can traditional machine learning and deep learning methods conform with regulatory guidelines for explainable artificial intelligence (XAI), and are they preferable to benchmark methods? this thesis used a public customer credit card dataset to compare the performance and explainability of machine learning and deep learning models against the benchmark statistical model linear regression. Model performance was evaluated using ROC-AUC, accuracy, Brier scores, F1 scores and the G-mean. The McNemar test evaluated whether, through pairwise comparison, the model performances were statistically different. The models were then evaluated on whether local and global explanations could be ascertained using feature/permutation importance and SHAP. The results found that neither the machine learning model, XGBoost, nor the deep learning model, LSTM, produced a statistically superior performance from the benchmark model. While there were performance improvements, only the machine learning model using post-hoc methods could produce local and global explanations. Given the strict regulatory environment, it is understandable that banks are hesitant to implement machine learning or deep learning models that lack the adequate levels of explainability regulators require.
| Identifer | oai:union.ndltd.org:UPSALLA1/oai:DiVA.org:su-219661 |
| Date | January 2023 |
| Creators | Salter, Robert Iain |
| Publisher | Stockholms universitet, Institutionen för data- och systemvetenskap |
| Source Sets | DiVA Archive at Upsalla University |
| Language | English |
| Detected Language | English |
| Type | Student thesis, info:eu-repo/semantics/bachelorThesis, text |
| Format | application/pdf |
| Rights | info:eu-repo/semantics/openAccess |
Page generated in 0.0025 seconds