Preventive maintenance plays a vital role in optimizing industrial operations. However, detecting equipment needing such maintenance using available data can be particularly challenging due to the class imbalance prevalent in real-world applications. The datasets gathered from equipment sensors primarily consist of records from well-functioning machines, making it difficult to identify those on the brink of failure, which is the main focus of preventive maintenance efforts. In this study, we employ neural network algorithms to address class imbalance and cost sensitivity issues in industrial scenarios for preventive maintenance. Our investigation centers on the "APS Failure in the Scania Trucks Data Set," a binary classification problem exhibiting significant class imbalance and cost sensitivity issues—a common occurrence across various fields. Inspired by image detection techniques, we introduce a novel loss function called Focal loss to traditional neural networks, combined with techniques like Cost-Sensitive Learning and Threshold Calculation to enhance classification accuracy. Our study's novelty is adapting image detection techniques to tackle the class imbalance problem within a binary classification task. Our proposed method demonstrates improvements in addressing the given optimization problem when confronted with these issues, matching or surpassing existing machine learning and deep learning techniques while maintaining computational efficiency. Our results indicate that class imbalance can be addressed without relying on conventional sampling techniques, which typically come at the cost of increased computational cost (oversampling) or loss of critical information (undersampling). In conclusion, our proposed method presents a promising approach for addressing class imbalance and cost sensitivity issues in industrial datasets heavily affected by these phenomena. It contributes to developing preventive maintenance solutions capable of enhancing the efficiency and productivity of industrial operations by detecting machines in need of attention: this discovery process we term predictive maintenance. The artifact produced in this study showcases the utilization of Focal Loss, Cost-Sensitive Learning, and Threshold Calculation to create reliable and effective predictive maintenance solutions for real-world applications. This thesis establishes a method that contributes to the body of knowledge in binary classification within machine learning, specifically addressing the challenges mentioned above. Our research findings have broader implications beyond industrial classification tasks, extending to other fields, such as medical or cybersecurity classification problems. The artifact (code) is at: https://shorturl.at/lsNSY

Artificial intelligence (AI) and machine learning (ML) present some application opportunities and challenges that can be framed as learning problems. The performance of machine learning models depends on algorithms and the data. Moreover, learning algorithms create a model of reality through learning and testing with data processes, and their performance shows an agreement degree of their assumed model with reality. ML algorithms have been successfully used in numerous classification problems. With the developing popularity of using ML models for many purposes in different domains, the validation of such predictive models is currently required more formally. Traditionally, there are many studies related to model evaluation, robustness, reliability, and the quality of the data and the data-driven models. However, those studies do not consider the concept of the applicability domain (AD) yet. The issue is that the AD is not often well defined, or it is not defined at all in many fields. This work investigates the robustness of ML classification models from the applicability domain perspective. A standard definition of applicability domain regards the spaces in which the model provides results with specific reliability. The main aim of this study is to investigate the connection between the applicability domain approach and the classification model performance. We are examining the usefulness of assessing the AD for the classification model, i.e. reliability, reuse, robustness of classifiers. The work is implemented using three approaches, and these approaches are conducted in three various attempts: firstly, assessing the applicability domain for the classification model; secondly, investigating the robustness of the classification model based on the applicability domain approach; thirdly, selecting an optimal model using Pareto optimality. The experiments in this work are illustrated by considering different machine learning algorithms for binary and multi-class classifications for healthcare datasets from public benchmark data repositories. In the first approach, the decision trees algorithm (DT) is used for the classification of data in the classification stage. The feature selection method is applied to choose features for classification. The obtained classifiers are used in the third approach for selection of models using Pareto optimality. The second approach is implemented using three steps; namely, building classification model; generating synthetic data; and evaluating the obtained results. The results obtained from the study provide an understanding of how the proposed approach can help to define the model’s robustness and the applicability domain, for providing reliable outputs. These approaches open opportunities for classification data and model management. The proposed algorithms are implemented through a set of experiments on classification accuracy of instances, which fall in the domain of the model. For the first approach, by considering all the features, the highest accuracy obtained is 0.98, with thresholds average of 0.34 for Breast cancer dataset. After applying recursive feature elimination (RFE) method, the accuracy is 0.96% with 0.27 thresholds average. For the robustness of the classification model based on the applicability domain approach, the minimum accuracy is 0.62% for Indian Liver Patient data at r=0.10, and the maximum accuracy is 0.99% for Thyroid dataset at r=0.10. For the selection of an optimal model using Pareto optimality, the optimally selected classifier gives the accuracy of 0.94% with 0.35 thresholds average. This research investigates critical aspects of the applicability domain as related to the robustness of classification ML algorithms. However, the performance of machine learning techniques depends on the degree of reliable predictions of the model. In the literature, the robustness of the ML model can be defined as the ability of the model to provide the testing error close to the training error. Moreover, the properties can describe the stability of the model performance when being tested on the new datasets. Concluding, this thesis introduced the concept of applicability domain for classifiers and tested the use of this concept with some case studies on health-related public benchmark datasets. / Ministry of Higher Education in Libya

Prostate cancer is the second most diagnosed cancer worldwide and its diagnosis is done through visual inspection of biopsy tissue by a pathologist, who assigns a score used by doctors to decide on the treatment. However, the scoring system, the Gleason score, is affected by a high inter and intra-observer variability, lack of standardization, and overestimation. Therefore, there is a need for new solutions that can reduce these issues and provide a more accurate diagnosis. Nowadays, high-resolution digital images of biopsy tissues can be obtained and stored. The availability of such images, called Whole Slide Images (WSI) allows the implementation of Machine and Deep learning models to assist pathologists in diagnosing prostate cancer. Multiple-Instance Learning (MIL) has been shown to reach very promising results in digital pathology and binary classification of prostate cancer slides. However, such models require large datasets to ensure good performances. This project wants to investigate the use of small sets of strongly annotated images to create new large datasets to train a MIL model. To evaluate the performance of this approach, the standard dataset is used to obtain baselines for both binary and multiclass classification tasks. For multiclassification, the International Society of Urological Pathology (ISUP) score is used, which is derived from the Gleason score. The dataset used is the publicly available PANDA. In this project, only the slides from RadboudUniversity Medical Center are used, which consists of 5160 images. The MIL model chosen is the Clustering-constrained Attention Multiple instance learning (CLAM) model, which is publicly available. The standard approach reaches a Cohen’s kappa (κ) of 0.78 and 0.59 for binary and multiclass classification respectively. To evaluate the new approach, large datasets are created starting from different set sizes. Using 500 images, the model reaches a κ of 0.72 and 0.38 respectively. While for the binary the results of the two approaches are comparable, the new approach is not beneficial for multiclass classification tasks.

<p dir="ltr">Recurrent stroke, infection, and seizure are some of the most common complications in stroke survivors. Recurrent stroke leads to death in 38.6% of survivors, and infections are the most common risk factor for seizures, with stroke survivors that experience an infection being at greater risk of experiencing a seizure. Two predictive models were generated, recurrent stroke and post-infection seizure, to determine stroke survivors at greatest risk to help providers focus on prevention in higher risk residents.</p><p dir="ltr">Predictive models were generated from a retrospective study of the Long-Term Care Minimum Data Set (MDS) 3.0 (2014-2018, n=262,301). Techniques included three data balancing methods (SMOTE for up sampling, ENN for down sampling, and SMOTEENN for up and down sampling) and three feature selection methods (LASSO, RFE, and PCA). The resulting datasets were then trained on four machine learning models (Logistic Regression, Random Forest, XGBoost, and Neural Network). Model performance was evaluated with AUC and accuracy, and interpretation used SHapley Addictive exPlanations.</p><p dir="ltr">Using data balancing methods improved the prediction performances of the machine learning models, but feature selection did not remove any features or affect performance. With all models having a high accuracy (78.6% to 99.9%), interpretation on all four models yielded the most holistic view. For recurrent stroke, SHAP values indicated that treatment combinations of occupational therapy, physical therapy, antidepressants, non-medical intervention for pain, therapeutic diet, anticoagulants, and diuretics contributed more to reducing recurrent stroke risk in the model when compared to individual treatments. For post-infection seizure, SHAP values indicated that therapy (speech, physical, occupational, and respiratory), independence (activities of daily living for walking, mobility, eating, dressing, and toilet use), and mood (severity score, anti-anxiety medications, antidepressants, and antipsychotics) features contributed the most. Meaning, stroke survivors who received fewer therapy hours, were less independent, and had a worse overall mood were at a greater risk of having a post-infection seizure.</p><p dir="ltr">The development of a tool to predict recurrent stroke and post-infection seizure in stroke survivors can be interpreted by providers to guide treatment and rehabilitation to prevent complications long-term. This promotes individualized plans that can increase the quality of resident care.</p>

This thesis aims to predict the probable undiscovered geothermal systems through investigation of spatial relation between geothermal occurrences and its surrounding geological phenomenon in Western Anatolia. In this context, four different public data, which are epicenter map, lineament map, Bouger gravity anomaly and magnetic anomaly maps, are utilized. In order to extract the necessary information for each map layer the raw public data is converted to a synthetic data which are directly used in the analysis. Synthetic data employed during the investigation process include Gutenberg-Richter b-value map, distance to lineaments map and distance to major grabens present in the area. Thus, these three layers including directly used magnetic anomaly maps are combined by means of Boolean logic model and Weights of Evidence method (WofE), which are multicriteria decision methods, in a Geographical Information System (GIS) environment. Boolean logic model is based on the simple logic of Boolean operators, while the WofE model depends on the Bayesian probability. Both of the methods use binary maps for their analysis. Thus, the binary map classification is the key point of the analysis. In this study three different binary map classification techniques are applied and thus three output maps were obtained for each of the method. The all resultant maps are evaluated within and among the methods by means of success indices. The findings reveal that the WofE method is better predictor than the Boolean logic model and that the third binarization approach, which is named as optimization procedure in this study, is the best estimator of binary classes due to obtained success indices. Finally, three output maps of each method are combined and the favorable areas in terms of geothermal potential are produced. According to the final maps the potential sites appear to be Aydin, Denizli and Manisa, of which first two have been greatly explored and exploited since today and thus not surprisingly found as potential in the output maps, while Manisa when compared to first two is nearly virgin.

We consider supervised learning problems, where one is given objects with labels, and the goal is to learn a model that can make accurate predictions on new objects. These problems abound in applications, ranging from medical diagnosis to information retrieval to computer vision. Examples include binary or multiclass classi cation, where the goal is to learn a model that can classify objects into two or more categories (e.g. categorizing emails into spam or non-spam); bipartite ranking, where the goal is to learn a model that can rank relevant objects above the irrelevant ones (e.g. ranking documents by relevance to a query); class probability estimation (CPE), where the goal is to predict the probability of an object belonging to different categories (e.g. probability of an internet ad being clicked by a user). In each case, the accuracy of a model is evaluated in terms of a specified `performance measure'. While there has been much work on designing and analyzing algorithms for different supervised learning tasks, we have complete understanding only for settings where the performance measure of interest is the standard 0-1 or a loss-based classification measure. These performance measures have a simple additive structure, and can be expressed as an expectation of errors on individual examples. However, in many real-world applications, the performance measure used to evaluate a model is often more complex, and does not decompose into a sum or expectation of point-wise errors. These include the binary or multiclass G-mean used in class-imbalanced classification problems; the F1-measure and its multiclass variants popular in text retrieval; and the (partial) area under the ROC curve (AUC) and precision@ employed in ranking applications. How does one design efficient learning algorithms for such complex performance measures, and can these algorithms be shown to be statistically consistent, i.e. shown to converge in the limit of infinite data to the optimal model for the given measure? How does one develop efficient learning algorithms for complex measures in online/streaming settings where the training examples need to be processed one at a time? These are questions that we seek to address in this thesis. Firstly, we consider the bipartite ranking problem with the AUC and partial AUC performance measures. We start by understanding how bipartite ranking with AUC is related to the standard 0-1 binary classification and CPE tasks. It is known that a good binary CPE model can be used to obtain both a good binary classification model and a good bipartite ranking model (formally, in terms of regret transfer bounds), and that a binary classification model does not necessarily yield a CPE model. However, not much is known about other directions. We show that in a weaker sense (where the mapping needed to transform a model from one problem to another depends on the underlying probability distribution), a good bipartite ranking model for AUC can indeed be used to construct a good binary classification model, and also a good binary CPE model. Next, motivated by the increasing number of applications (e.g. biometrics, medical diagnosis, etc.), where performance is measured, not in terms of the full AUC, but in terms of the partial AUC between two false positive rates (FPRs), we design batch algorithms for optimizing partial AUC in any given FPR range. Our algorithms optimize structural support vector machine based surrogates, which unlike for the full AUC; do not admit a straightforward decomposition into simpler terms. We develop polynomial time cutting plane solvers for solving the optimization, and provide experiments to demonstrate the efficacy of our methods. We also present an application of our approach to predicting chemotherapy outcomes for cancer patients, with the aim of improving treatment decisions. Secondly, we develop algorithms for optimizing (surrogates for) complex performance mea-sures in the presence of streaming data. A well-known method for solving this problem for standard point-wise surrogates such as the hinge surrogate, is the stochastic gradient descent (SGD) method, which performs point-wise updates using unbiased gradient estimates. How-ever, this method cannot be applied to complex objectives, as here one can no longer obtain unbiased gradient estimates from a single point. We develop a general stochastic method for optimizing complex measures that avoids point-wise updates, and instead performs gradient updates on mini-batches of incoming points. The method is shown to provably converge for any performance measure that satis es a uniform convergence requirement, such as the partial AUC, precision@ and F1-measure, and in experiments, is often several orders of magnitude faster than the state-of-the-art batch methods, while achieving similar or better accuracies. Moreover, for specific complex binary classification measures, which are concave functions of the true positive rate (TPR) and true negative rate (TNR), we are able to develop stochastic (primal-dual) methods that can indeed be implemented with point-wise updates, by using an adaptive linearization scheme. These methods admit convergence rates that match the rate of the SGD method, and are again several times faster than the state-of-the-art methods. Finally, we look at the design of consistent algorithms for complex binary and multiclass measures. For binary measures, we consider the practically popular plug-in algorithm that constructs a classifier by applying an empirical threshold to a suitable class probability estimate, and provide a general methodology for proving consistency of these methods. We apply this technique to show consistency for the F1-measure, and under a continuity assumption on the distribution, for any performance measure that is monotonic in the TPR and TNR. For the case of multiclass measures, a simple plug-in method is no longer tractable, as in the place of a single threshold parameter, one needs to tune at least as many parameters as the number of classes. Using an optimization viewpoint, we provide a framework for designing learning algorithms for multiclass measures that are general functions of the confusion matrix, and as an instantiation, provide an e cient and provably consistent algorithm based on the bisection method for multiclass measures that are ratio-of-linear functions of the confusion matrix (e.g. micro F1). The algorithm outperforms the state-of-the-art SVMPerf method in terms of both accuracy and running time. Overall, in this thesis, we have looked at various aspects of complex performance measures used in supervised learning problems, leading to several new algorithms that are often significantly better than the state-of-the-art, to improved theoretical understanding of the performance measures studied, and to novel real-life applications of the algorithms developed.

The thesis investigates a publicly available dataset which covers morethan three million events in football matches. The aim of the study isto train machine learning models capable of modeling the relationshipbetween a shot event and its outcome. That is, to predict if a footballshot will result in a goal or not. By representing the shot indifferent ways, the aim is to draw conclusion regarding what elementsof a shot allows for a good prediction of its outcome. The shotrepresentation was varied both by including different numbers of eventspreceding the shot and by varying the set of features describing eachevent.The study shows that the performance of the machine learning modelsbenefit from including events preceding the shot. The highestpredictive performance was achieved by a long short-term memory neuralnetwork trained on the shot event and six events preceding the shot.The features which were found to have the largest positive impact onthe shot events were the precision of the event, the position on thefield and how the player was in contact with the ball. The size of thedataset was also evaluated and the results suggest that it issufficiently large for the size of the networks evaluated.

Stressed Value-at-Risk (VaR) is a statistic used to measure an entity's exposure to market risk by evaluating possible extreme portfolio losses. Stressed VaR scenarios can be used as a metric to describe the state of the financial market and can be used to detect and counter procyclicality by allowing central clearing counterparities (CCP) to increase margin requirements. This thesis aims to implement and evaluate machine learning methods (e.g., neural networks) for selecting stressed VaR scenarios in price return stock datasets where one liquidity day is assumed. The models are implemented to counter the procyclical effects present in NASDAQ's dual lambda method such that the selection maximises the total margin metric. Three machine learning models are implemented together with a labelling algorithm, a supervised and unsupervised multilayer perceptron and a random forest model. The labelling algorithm employs a deviation metric to differentiate between stressed VaR and standard scenarios. The models are trained and tested using 5000 scenarios of price return values from historical stock datasets. The models are tested using visual results, confusion matrix, Cohen's kappa statistic, the adjusted rand index and the total margin metric. The total margin metric is computed using normalised profit and loss values from artificially generated portfolios. The implemented machine learning models and the labelling algorithm manage to counter the procyclical effects evident in the dual lambda method and selected stressed VaR scenarios such that the selection maximise the total margin metric. The random forest model shows the most promise in classifying stressed VaR scenarios, since it manages to maximise the total margin overall.

This thesis offers a thorough investigation into the application of machine learning algorithms for predicting the presence of vehicles in a traffic setting. The research primarily focuses on enhancing vehicle simulation by employing data-driven traffic prediction methods. The study approaches the problem as a binary classification task. Various supervised learning algorithms, including Random Forest (RF), Gradient Boosting (GB), Support Vector Machine (SVM), and Logistic Regression (LogReg) were evaluated and tested. The thesis encompasses six distinct implementations, each involving different combinations of algorithms, feature engineering, hyperparameter tuning, and data splitting. The performance of each model was assessed using metrics such as accuracy, precision, recall, and F1-score, and visualizations like ROC-AUC curves were used to gain insights into their discrimination capabilities. While the RF model achieved the highest accuracy at 97%, the AUC score of Combination 2 (RF+GB) suggests that this ensemble model could strike a better balance between high accuracy (86%) and effective class separation (99%). Ultimately, the study identifies an ensemble model as the preferred choice, leading to significant improvements in prediction accuracy. The research also explores working on the problem as a time-series prediction task, exploring the use of Long Short-Term Memory (LSTM) and Auto-Regressive Integrated Moving Average (Auto-ARIMA) models. However, we found that this approach was impractical due to the dataset’s discrete and non-sequential nature. This research contributes to the advancement of vehicle simulation and traffic forecasting, demonstrating the potential of machine learning in addressing complex real-world scenarios.

Online payment purchase flows are designed to be as effective and smooth as possible in regards to the user experience. The user is in the center of this process, who, to a certain degree decides whether the purchase eventually will be placed. What is left up to the payment provider is the process of enabling an effective purchase flow where information needs to be collected for various purposes. To design these purchase flows as efficiently as possible, this research investigates if and how consumer purchase behavior can be predicted. Which algorithms perform the best at modeling the outcome and what kind of underlying features can be used to model the outcome? The features are graded in regard to their feature importance to see how and how much they affect the best-performing model. To investigate consumer behavior, the task was set up as a supervised binary classification problem to model the outcome of user purchase sessions. Either the sessions result in a purchase or they do not. Several automatic machine learning (also referred to as automated machine learning) frameworks were considered before the choice of using H2O AutoML because of its historical performance on other supervised binary classification problems. The dataset contained information from user sessions relating to the consumer, the transaction, and the time when the purchase was initiated. These variables were either in a numerical or categorical format and were then evaluated using the SHAP importance metric as well as an aggregated SHAP summary plot, which describes how features are affecting the model. The results show that the Distributed Random Forest Algorithm performed the best, generating a 26 percentage points improvement in accuracy, predicting whether a session will be converted into a purchase from an undersampled baseline of 50%. Furthermore two of the most important features according to the model were categorical features related to the intersection of consumer and transaction information. Another time-based categorical variable also proved to be important in the model prediction. The research also shows that automatic machine learning has come a long way in the pre-processing of variables, enabling the developer of the models to more efficiently deploy these kinds of machine learning problems. The results echo some earlier findings confirming the possibility of predicting consumer purchase behavior and in particular, the outcome of a purchase flow consumer session. This implies that payment providers hypothetically could use these kinds of insights and predictions in the development of their flows, to individually cater to specific groups of consumers, enabling a more efficient and personalized payment flow. / Köpflöden för onlinebetalningar är utformade för att vara så effektiva och smidiga som möjligt med avseende på användarupplevelsen. I processen står användaren i centrum, som delvis avgör om köpet i slutändan konverteras eller ej. Det som är upp till betalningsleverantören är möjliggörandet av ett effektivt köpflöde där information behöver samlas in för olika ändamål. För att utforma dessa köpflöden så effektivt som möjligt undersöker detta arbete om och hur konsumenters köpbeteende kan förutsägas. Vilka algoritmer fungerar bäst på att modellera resultatet och vilken typ av underliggande attribut kan användas för att modellera resultatet? Dessa attribut graderas med avseende på deras relevans (feature importance) för att se hur och hur mycket de faktiskt påverkar den bäst presterande modellen. För att undersöka konsumentbeteendet sattes uppgiften upp som ett övervakat binärt klassificeringsproblem för att modellera resultatet av användarnas sessioner. Antingen resulterar sessionerna i ett köp eller så gör de det inte. Flera ramverk för automatisk maskininlärning övervägdes innan valet att använda H2O AutoML på grund av dess historiska prestanda på andra övervakade binära klassificeringsproblem. Dataunderlaget innehöll information från användarsessioner som rör konsumenten, transaktionen och tidpunkten då köpet påbörjades. Dessa variabler var antingen i ett numeriskt eller kategoriskt format och utvärderades sedan med hjälp av SHAP-viktighetsmåttet (SHAP Feature Importance) såväl som ett aggregerat SHAP-diagram, som beskriver hur de olika attributen påverkar modellen. Resultaten visar att Distributed Random Forest algoritmen presterade bäst, genererade en förbättring på 26 procentenheter i noggrannhet (accuracy), i prediktionen av ifall en session omvandlas till ett köp eller ej, baserat på ett undersamplat dataset med en baslinje på 50%. Dessutom var två av de viktigaste attributen enligt modellen kategoriska attribut relaterade till skärningspunkten mellan konsument- och transaktionsinformation. En annan tidsbaserad kategorisk variabel visade sig också vara viktig i prediktionen. Arbetet visar också att automatisk maskininlärning har kommit långt i förbearbetningen av variabler, vilket gör det möjligt för utvecklaren av modellerna att mer effektivt distribuera den här typen av maskininlärningsproblem. Resultaten återspeglar tidigare insikter som bekräftar möjligheten att förutsäga konsumenternas köpbeteende och i synnerhet resultatet av en konsumentsession i ett köpflöde. Detta innebär att betalningsleverantörer hypotetiskt skulle kunna använda denna typ av insikter och förutsägelser i utvecklingen av sina flöden, för att individuellt tillgodose specifika grupper av konsumenter, vilket möjliggör ett ännu mer effektivt och skräddarsytt betalningsflöde.