This thesis is divided into two parts. In the first part, a general problem of the robust template decomposition with restricted weights for cellular neural networks (CNNs) implementing an arbitrary Boolean function is investigated. In the second part, some primary study of the novel Wilcoxon learning machines is made. In the first part of the thesis for the robust CNN template design, the geometric margin of a linear classifier with respect to a training data set, a notion borrowed from the machine learning theory, is used to define the robustness of an uncoupled CNN implementing a linearly separable Boolean function. Consequently, the so-called maximal margin classifiers can be devised via support vector machines (SVMs) to provide the most robust template design for uncoupled CNNs implementing linearly separable Boolean functions. Some general properties of robust CNNs with or without restricted weights are discussed. Moreover, all robust CNNs with restricted weights are characterized. For an arbitrarily given Boolean function, we propose an algorithm, which is the generalized version of the well known CFC algorithm, to find a sequence of robust uncoupled CNNs implementing the given Boolean function. In the second part of the thesis, we investigate the novel Wilcoxon learning machines (WLMs). The invention of these learning machines was motivated by the Wilcoxon approach to linear regression problems in statistics. The resulting linear regressors are quits robust against outliers, as is well known in statistics. The Wilcoxon learning machines investigated in this thesis include Wilcoxon Neural Network (WNN), Wilcoxon Generalized Radial Basis Function Network (WGRBFN), Wilcoxon Fuzzy Neural Network (WFNN), and Kernel-based Wilcoxon Regressor (KWR).

Extreme Learning Machine (ELM) is a training algorithm for Single-Layer Feed-forward Neural Network (SLFN). The difference in theory of ELM from other training algorithms is in the existence of explicitly-given solution due to the immutability of initialed weights. In practice, ELMs achieve performance similar to that of other state-of-the-art training techniques, while taking much less time to train a model. Experiments show that the speedup of training ELM is up to the 5 orders of magnitude comparing to standard Error Back-propagation algorithm. ELM is a recently discovered technique that has proved its efficiency in classic regression and classification tasks, including multi-class cases. In this thesis, extensions of ELMs for non-typical for Artificial Neural Networks (ANNs) problems are presented. The first extension, described in the third chapter, allows to use ELMs to get probabilistic outputs for multi-class classification problems. The standard way of solving this type of problems is based 'majority vote' of classifier's raw outputs. This approach can rise issues if the penalty for misclassification is different for different classes. In this case, having probability outputs would be more useful. In the scope of this extension, two methods are proposed. Additionally, an alternative way of interpreting probabilistic outputs is proposed. ELM method prove useful for non-linear dimensionality reduction and visualization, based on repetitive re-training and re-evaluation of model. The forth chapter introduces adaptations of ELM-based visualization for classification and regression tasks. A set of experiments has been conducted to prove that these adaptations provide better visualization results that can then be used for perform classification or regression on previously unseen samples. Shape registration of 3D models with non-isometric distortion is an open problem in 3D Computer Graphics and Computational Geometry. The fifth chapter discusses a novel approach for solving this problem by introducing a similarity metric for spectral descriptors. Practically, this approach has been implemented in two methods. The first one utilizes Siamese Neural Network to embed original spectral descriptors into a lower dimensional metric space, for which the Euclidean distance provides a good measure of similarity. The second method uses Extreme Learning Machines to learn similarity metric directly for original spectral descriptors. Over a set of experiments, the consistency of the proposed approach for solving deformable registration problem has been proven.

This thesis addresses the problem of identifying nonlinear dynamic systems from a machine learning perspective. In this context, very little is assumed to be known about the system under investigation, and the only source of information comes from input/output measurements on the system. It corresponds to the black-box modeling approach. Numerous strategies and models have been proposed over the last decades in the machine learning field and applied to modeling tasks in a straightforward way. Despite of this variety, the methods can be roughly categorized into global and local modeling approaches. Global modeling consists in fitting a single regression model to the available data, using the whole set of input and output observations. On the other side of the spectrum stands the local modeling approach, in which the input space is segmented into several small partitions and a specialized regression model is fit to each partition. The first contribution of the thesis is a novel supervised global learning model, the Minimal Learning Machine (MLM). Learning in MLM consists in building a linear mapping between input and output distance matrices and then estimating the nonlinear response from the geometrical configuration of the output points. Given its general formulation, the Minimal Learning Machine is inherently capable of operating on nonlinear regression problems as well as on multidimensional response spaces. Naturally, its characteristics make the MLM able to tackle the system modeling problem. The second significant contribution of the thesis represents a different modeling paradigm, called Regional Modeling (RM), and it is motivated by the parsimonious principle. Regional models stand between the global and local modeling approaches. The proposal consists of a two-level clustering approach in which we first partition the input space using the Self-Organizing Map (SOM), and then perform clustering over the prototypes of the trained SOM. After that, regression models are built over the clusters of SOM prototypes, or regions in the input space. Even though the proposals of the thesis can be thought as quite general regression or supervised learning models, the performance assessment is carried out in the context of system identification. Comprehensive performance evaluation of the proposed models on synthetic and real-world datasets is carried out and the results compared to those achieved by standard global and local models. The experiments illustrate that the proposed methods achieve accuracies that are comparable to, and even better than, more traditional machine learning methods thus offering a valid alternative to such approaches. / This thesis addresses the problem of identifying nonlinear dynamic systems from a machine learning perspective. In this context, very little is assumed to be known about the system under investigation, and the only source of information comes from input/output measurements on the system. It corresponds to the black-box modeling approach. Numerous strategies and models have been proposed over the last decades in the machine learning field and applied to modeling tasks in a straightforward way. Despite of this variety, the methods can be roughly categorized into global and local modeling approaches. Global modeling consists in fitting a single regression model to the available data, using the whole set of input and output observations. On the other side of the spectrum stands the local modeling approach, in which the input space is segmented into several small partitions and a specialized regression model is fit to each partition. The first contribution of the thesis is a novel supervised global learning model, the Minimal Learning Machine (MLM). Learning in MLM consists in building a linear mapping between input and output distance matrices and then estimating the nonlinear response from the geometrical configuration of the output points. Given its general formulation, the Minimal Learning Machine is inherently capable of operating on nonlinear regression problems as well as on multidimensional response spaces. Naturally, its characteristics make the MLM able to tackle the system modeling problem. The second significant contribution of the thesis represents a different modeling paradigm, called Regional Modeling (RM), and it is motivated by the parsimonious principle. Regional models stand between the global and local modeling approaches. The proposal consists of a two-level clustering approach in which we first partition the input space using the Self-Organizing Map (SOM), and then perform clustering over the prototypes of the trained SOM. After that, regression models are built over the clusters of SOM prototypes, or regions in the input space. Even though the proposals of the thesis can be thought as quite general regression or supervised learning models, the performance assessment is carried out in the context of system identification. Comprehensive performance evaluation of the proposed models on synthetic and real-world datasets is carried out and the results compared to those achieved by standard global and local models. The experiments illustrate that the proposed methods achieve accuracies that are comparable to, and even better than, more traditional machine learning methods thus offering a valid alternative to such approaches.

Orientadores: Furio Damiani, Romis Ribeiro de Faissol Attux / Tese (doutorado) - Universidade Estadual de Campinas, Faculdade de Engenharia Elétrica e de Computação / Made available in DSpace on 2018-08-27T01:42:11Z (GMT). No. of bitstreams: 1 FernandesJunior_AntonioCarlosLopes_D.pdf: 3251957 bytes, checksum: 7a047b751489da833ab7c5efd9cd86ee (MD5) Previous issue date: 2015 / Resumo: A deteção de tempo em um sinal musical é uma tarefa muito importante em diversas aplicações. A presente tese apresenta os resultados da detecção de andamento usando uma nova abordagem baseada na extração de atributos de um conjunto de funções de detecção de periodicidade e aprendizado de máquina. Para isto a transformada wavelet foi utilizada para separar o sinal musical em diferentes resoluções e o domínio complexo retificado foi aplicado para a construção de funções de deteccão de onsets. Em seguida, as funções de deteccão de periodicidade para cada nível wavelet foram geradas por operações de autocorrelação. Descritores de áudio clássicos foram adaptados e extraídos de cada função de periodicidade e foram usados como entradas para a máquina de aprendizado que mapeia os descritores para o tempo da música. As máquinas utilizadas foram o perceptron de múltiplas camadas e a máquina de aprendizado extremo, com propostas diferenciadas de configuração. Um método para classificação e avaliação dos descritores foi proposto. Também, neste trabalho, um novo descritor foi proposto. Um método de seleção forward de atributos via Gram-Schmidt foi aplicado para a escolha do melhor subconjunto para o treinamento da máquina. Foi ainda aplicado um método de clustering via K-means para a partilha de observações entre os conjuntos de treinamento, teste e validação, e foi proposto um novo método de seleção de observações via análise de componentes principais denominado de seleção esférica de observações / Abstract: Tempo detection in a music signal is a very important task for many applications. This thesis presents results concerning this task using a new approach based on the extraction of features from a set of periodicity detection functions and on machine learning. The wavelet transform was utilized to separate the musical signal at different resolutions and the rectified complex domain was applied to the construction of onset detection functions. Then, periodicity detection functions for each resolution were generated by autocorrelation operations. Classic audio features were extracted from each periodicity function and were used as inputs to a neural network that maps descriptors to music tempo. The used machines were the multilayer perceptron and an extreme learning machine, with different configuration proposals. A method for classification and evaluation of features has been proposed. Also, in this work, a new descriptor has been proposed. A method of forward selection via Gram-Schmidt was applied to choosing the best subset for the machine training. A K-means clustering method was also applied for partitioning observations between the training sets and a new observation selection method via principal component analysis, called spherical selection of observations, was proposed / Doutorado / Eletrônica, Microeletrônica e Optoeletrônica / Doutor em Engenharia Elétrica

Sistemas de Recomendação (SR) vêm se apresentando como poderosas ferramentas para portais web tais como sítios de comércio eletrônico. Para fazer suas recomendações, os SR se utilizam de fontes de dados variadas, as quais capturam as características dos usuários, dos itens e suas transações, bem como de modelos de predição. Dada a grande quantidade de dados envolvidos, é improvável que todas as recomendações possam ser bem representadas por um único modelo global de predição. Um outro importante aspecto a ser observado é o problema conhecido por cold-start, que apesar dos avanços na área de SR, é ainda uma questão relevante que merece uma maior atenção. O problema está relacionado com a falta de informação prévia sobre novos usuários ou novos itens do sistema. Esta tese apresenta uma abordagem híbrida de recomendação capaz de lidar com situações extremas de cold-start. A abordagem foi desenvolvida com base no algoritmo SCOAL (Simultaneous Co-Clustering and Learning). Na sua versão original, baseada em múltiplos modelos lineares de predição, o algoritmo SCOAL mostrou-se eficiente e versátil, podendo ser utilizado numa ampla gama de problemas de classificação e/ou regressão. Para melhorar o algoritmo SCOAL no sentido de deixá-lo mais versátil por meio do uso de modelos não lineares, esta tese apresenta uma variante do algoritmo SCOAL que utiliza modelos de predição baseados em Máquinas de Aprendizado Extremo. Além da capacidade de predição, um outro fator que deve ser levado em consideração no desenvolvimento de SR é a escalabilidade do sistema. Neste sentido, foi desenvolvida uma versão paralela do algoritmo SCOAL baseada em OpenMP, que minimiza o tempo envolvido no cálculo dos modelos de predição. Experimentos computacionais controlados, por meio de bases de dados amplamente usadas na prática, comprovam que todos os desenvolvimentos propostos tornam o SCOAL ainda mais atraente para aplicações práticas variadas. / Recommender Systems (RS) are powerful and popular tools for e-commerce. To build its recommendations, RS make use of multiple data sources, capture the characteristics of items, users and their transactions, and take advantage of prediction models. Given the large amount of data involved in the predictions made by RS, is unlikely that all predictions can be well represented by a single global model. Another important aspect to note is the problem known as cold-start that, despite that recent advances in the RS area, it is still a relevant issue that deserves further attention. The problem arises due to the lack of prior information about new users and new items. This thesis presents a hybrid recommendation approach that addresses the (pure) cold start problem, where no collaborative information (ratings) is available for new users. The approach is based on an existing algorithm, named SCOAL (Simultaneous Co-Clustering and Learning). In its original version, based on multiple linear prediction models, the SCOAL algorithm has shown to be efficient and versatile. In addition, it can be used in a wide range of problems of classification and / or regression. The SCOAL algorithm showed impressive results with the use of linear prediction models, but there is still room for improvements with nonlinear models. From this perspective, this thesis presents a variant of the SCOAL based on Extreme Learning Machines. Besides improving the accuracy, another important issue related to the development of RS is system scalability. In this sense, a parallel version of the SCOAL, based on OpenMP, was developed, aimed at minimizing the computational cost involved as prediction models are learned. Experiments using real-world datasets has shown that all proposed developments make SCOAL algorithm even more attractive for a variety of practical applications.

Sistemas de Recomendação (SR) vêm se apresentando como poderosas ferramentas para portais web tais como sítios de comércio eletrônico. Para fazer suas recomendações, os SR se utilizam de fontes de dados variadas, as quais capturam as características dos usuários, dos itens e suas transações, bem como de modelos de predição. Dada a grande quantidade de dados envolvidos, é improvável que todas as recomendações possam ser bem representadas por um único modelo global de predição. Um outro importante aspecto a ser observado é o problema conhecido por cold-start, que apesar dos avanços na área de SR, é ainda uma questão relevante que merece uma maior atenção. O problema está relacionado com a falta de informação prévia sobre novos usuários ou novos itens do sistema. Esta tese apresenta uma abordagem híbrida de recomendação capaz de lidar com situações extremas de cold-start. A abordagem foi desenvolvida com base no algoritmo SCOAL (Simultaneous Co-Clustering and Learning). Na sua versão original, baseada em múltiplos modelos lineares de predição, o algoritmo SCOAL mostrou-se eficiente e versátil, podendo ser utilizado numa ampla gama de problemas de classificação e/ou regressão. Para melhorar o algoritmo SCOAL no sentido de deixá-lo mais versátil por meio do uso de modelos não lineares, esta tese apresenta uma variante do algoritmo SCOAL que utiliza modelos de predição baseados em Máquinas de Aprendizado Extremo. Além da capacidade de predição, um outro fator que deve ser levado em consideração no desenvolvimento de SR é a escalabilidade do sistema. Neste sentido, foi desenvolvida uma versão paralela do algoritmo SCOAL baseada em OpenMP, que minimiza o tempo envolvido no cálculo dos modelos de predição. Experimentos computacionais controlados, por meio de bases de dados amplamente usadas na prática, comprovam que todos os desenvolvimentos propostos tornam o SCOAL ainda mais atraente para aplicações práticas variadas. / Recommender Systems (RS) are powerful and popular tools for e-commerce. To build its recommendations, RS make use of multiple data sources, capture the characteristics of items, users and their transactions, and take advantage of prediction models. Given the large amount of data involved in the predictions made by RS, is unlikely that all predictions can be well represented by a single global model. Another important aspect to note is the problem known as cold-start that, despite that recent advances in the RS area, it is still a relevant issue that deserves further attention. The problem arises due to the lack of prior information about new users and new items. This thesis presents a hybrid recommendation approach that addresses the (pure) cold start problem, where no collaborative information (ratings) is available for new users. The approach is based on an existing algorithm, named SCOAL (Simultaneous Co-Clustering and Learning). In its original version, based on multiple linear prediction models, the SCOAL algorithm has shown to be efficient and versatile. In addition, it can be used in a wide range of problems of classification and / or regression. The SCOAL algorithm showed impressive results with the use of linear prediction models, but there is still room for improvements with nonlinear models. From this perspective, this thesis presents a variant of the SCOAL based on Extreme Learning Machines. Besides improving the accuracy, another important issue related to the development of RS is system scalability. In this sense, a parallel version of the SCOAL, based on OpenMP, was developed, aimed at minimizing the computational cost involved as prediction models are learned. Experiments using real-world datasets has shown that all proposed developments make SCOAL algorithm even more attractive for a variety of practical applications.

This thesis deals with fraud detection in a real-world environment with datasets coming from Svenska Handelsbanken. The goal was to investigate how well machine learning can classify fraudulent transactions and how new additional features affected classification. The models used were EFSVM, RUTSVM, CS-SVM, ELM, MLP, Decision Tree, Extra Trees, and Random Forests. To determine the best results the Mathew Correlation Coefficient was used as performance metric, which has been shown to have a medium bias for imbalanced datasets. Each model could deal with high imbalanced datasets which is common for fraud detection. Best results were achieved with Random Forest and Extra Trees. The best scores were around 0.4 for the real-world datasets, though the score itself says nothing as it is more a testimony to the dataset’s separability. These scores were obtained when using aggregated features and not the standard raw dataset. The performance measure recall’s scores were around 0.88-0.93 with an increase in precision by 34.4%-67%, resulting in a large decrease of False Positives. Evaluation results showed a great difference compared to test-runs, either substantial increase or decrease. Two theories as to why are discussed, a great distribution change in the evaluation set, and the sample size increase (100%) for evaluation could have lead to the tests not being well representing of the performance. Feature aggregation were a central topic of this thesis, with the main focus on behaviour features which can describe patterns and habits of customers. For these there were five categories: Sender’s fraud history, Sender’s transaction history, Sender’s time transaction history, Sender’shistory to receiver, and receiver’s history. Out of these, the best performance increase was from the first which gave the top score, the other datasets did not show as much potential, with mostn ot increasing the results. Further studies need to be done before discarding these features, to be certain they don’t improve performance. Together with the data aggregation, a tool (t-SNE) to visualize high dimension data was usedto great success. With it an early understanding of what to expect from newly added features would bring to classification. For the best dataset it could be seen that a new sub-cluster of transactions had been created, leading to the belief that classification scores could improve, whichthey did. Feature selection and PCA-reduction techniques were also studied and PCA showedgood results and increased performance. Feature selection had not conclusive improvements. Over- and under-sampling were used and neither improved the scores, though undersampling could maintain the results which is interesting when increasing the dataset. / Denna avhandling handlar om upptäcka bedrägerier i en real-world miljö med data från Svenska Handelsbanken. Målet var att undersöka hur bra maskininlärning är på att klassificera bedrägliga transaktioner, och hur nya attributer hjälper klassificeringen. Metoderna som användes var EFSVM, RUTSVM, CS-SVM, ELM, MLP, Decision Tree, Extra Trees och Random Forests. För evaluering av resultat används Mathew Correlation Coefficient, vilket har visat sig ha småttt beroende med hänsyn till obalanserade datamängder. Varje modell har inbygda värden för attklara av att bearbeta med obalanserade datamängder, vilket är viktigt för att upptäcka bedrägerier. Resultatmässigt visade det sig att Random Forest och Extra Trees var bäst, utan att göra p-test:s, detta på grund att dataseten var relativt sätt små, vilket gör att små skillnader i resultat ej är säkra. De högsta resultaten var cirka 0.4, det absoluta värdet säger ingenting mer än som en indikation om graden av separation mellan klasserna. De bäst resultaten ficks när nya aggregerade attributer användes och inte standard datasetet. Dessa resultat hade recall värden av 0,88-0,93 och för dessa kunde det synas precision ökade med 34,4% - 67%, vilket ger en stor minskning av False Positives. Evluation-resultaten hade stor skillnad mot test-resultaten, denna skillnad hade antingen en betydande ökning eller minskning. Två anledningar om varför diskuterades, förändring av evaluation-datan mot test-datan eller att storleksökning (100%) för evaluation har lett till att testerna inte var representativa. Attribute-aggregering var ett centralt ämne, med fokus på beteende-mönster för att beskriva kunders vanor. För dessa fanns det fem kategorier: Avsändarens bedrägerihistorik, Avsändarens transaktionshistorik, Avsändarens historik av tid för transaktion, Avsändarens historik till mottagaren och mottagarens historik. Av dessa var den största prestationsökningen från bedrägerihistorik, de andra attributerna hade inte lika positiva resultat, de flesta ökade inte resultaten.Ytterligare mer omfattande studier måste göras innan dessa attributer kan sägas vara givande eller ogivande. Tillsammans med data-aggregering användes t-SNE för att visualisera högdimensionsdata med framgång. Med t-SNE kan en tidig förståelse för vad man kan förvänta sig av tillagda attributer, inom klassificering. För det bästa dataset kan man se att ett nytt kluster som hade skapats, vilket kan tolkas som datan var mer beskrivande. Där förväntades också resultaten förbättras, vilket de gjorde. Val av attributer och PCA-dimensions reducering studerades och PCA-visadeförbättring av resultaten. Over- och under-sampling testades och kunde ej förbättrade resultaten, även om undersampling kunde bibehålla resultated vilket är intressant om datamängden ökar.

A crucial decision in the real-time management of today’s irrigation systems involves the coordination of diversions and delivery of water to croplands. Since most irrigation systems experience significant lags between when water is diverted and when it should be delivered, an important technical innovation in the next few years will involve improvements in short-term irrigation demand forecasting. The main objective of the researches presented was the development of these critically important models: (1) potential evapotranspiration forecasting; (2) hydraulic model error correction; and (3) estimation of aggregate water demands. These tools are based on statistical machine learning or data-driven modeling. These, of wide application in several areas of engineering analysis, can be used in irrigation and system management to provide improved and timely information to water managers. The development of such models is based on a Bayesian data-driven algorithm called the Relevance Vector Machine (RVM), and an extension of it, the Multivariate Relevance Vector Machine (MVRVM). The use of these types of learning machines has the advantage of avoidance of model overfitting, high robustness in the presence of unseen data, and uncertainty estimation for the results (error bars). The models were applied in an irrigation system located in the Lower Sevier River Basin near Delta, Utah. For the first model, the proposed method allows for estimation of future crop water demand values up to four days in advance. The model uses only daily air temperatures and the MVRVM as mapping algorithm. The second model minimizes the lumped error occurring in hydraulic simulation models. The RVM is applied as an error modeler, providing estimations of the occurring errors during the simulation runs. The third model provides estimation of future water releases for an entire agricultural area based on local data and satellite imagery up to two days in advance. The results obtained indicate the excellent adequacy in terms of accuracy, robustness, and stability, especially in the presence of unseen data. The comparison provided against another data-driven algorithm, of wide use in engineering, the Multilayer Perceptron, further validates the adequacy of use of the RVM and MVRVM for these types of processes.

Nos últimos 30 anos, a área de aprendizagem de máquina desenvolveu-se de forma comparável com a Física no início do século XX. Esse avanço tornou possível a resolução de problemas do mundo real que anteriormente não poderiam ser solucionados por máquinas, devido à dificuldade de modelos puramente estatísticos ajustarem-se de forma satisfatória aos dados de treinamento. Dentre tais avanços, pode-se citar a utilização de técnicas de aprendizagem de máquina na área de Química Medicinal, envolvendo métodos de análise, representação e predição de informação molecular por meio de recursos computacionais. Os dados utilizados no contexto biológico possuem algumas características particulares que podem influenciar no resultado de sua análise. Dentre estas, pode-se citar a complexidade das informações moleculares, o desbalanceamento das classes envolvidas e a existência de dados incompletos ou rotulados de forma incerta. Tais adversidades podem prejudicar o processo de identificação de compostos candidatos a novos fármacos, se não forem tratadas de forma adequada. Neste trabalho, foi abordada uma técnica de aprendizagem de máquina semi-supervisionada capaz de reduzir o impacto causado pelo problema da incerteza na rotulação dos dados, aplicando um método para estimar rótulos mais confiáveis para os compostos químicos existentes no conjunto de treinamento. Na tentativa de evitar os efeitos causados pelo desbalanceamento dos dados, foi incorporada ao processo de estimação de rótulos uma abordagem sensível ao custo, com o objetivo de evitar o viés em benefício da classe majoritária. Após o tratamento do problema da incerteza na rotulação, classificadores baseados em Máquinas de Aprendizado Extremo foram construídos, almejando boa capacidade de aproximação em um tempo de processamento reduzido em relação a outras abordagens de classificação comumente aplicadas. Por fim, o desempenho dos classificadores construídos foi avaliado por meio de análises dos resultados obtidos, confrontando o cenário com os dados originais e outros com as novas rotulações obtidas durante o processo de estimação semi-supervisionado / In the last 30 years, the area of machine learning has developed in a way comparable to Physics in the early twentieth century. This breakthrough has made it possible to solve real-world problems that previously could not be solved by machines because of the difficulty of purely statistical models to fit satisfactorily with training data. Among these advances, one can cite the use of machine learning techniques in the area of Medicinal Chemistry, involving methods for analysing, representing and predicting molecular information through computational resources. The data used in the biological context have some particular characteristics that can influence the result of its analysis. These include the complexity of molecular information, the imbalance of the classes involved, and the existence of incomplete or uncertainly labeled data. If they are not properly treated, such adversities may affect the process of identifying candidate compounds for new drugs. In this work, a semi-supervised machine learning technique was considered to reduce the impact caused by the problem of uncertainty in the data labeling, by applying a method to estimate more reliable labels for the chemical compounds in the training set. In an attempt to reduce the effects caused by data imbalance, a cost-sensitive approach was incorporated to the label estimation process, in order to avoid bias in favor of the majority class. After addressing the uncertainty problem in labeling, classifiers based on Extreme Learning Machines were constructed, aiming for good approximation ability in a reduced processing time in relation to other commonly applied classification approaches. Finally, the performance of the classifiers constructed was evaluated by analyzing the results obtained, comparing the scenario with the original data and others with the new labeling obtained by the semi-supervised estimation process

<div>This dissertation addresses two significant issues in the effective application of machine learning algorithms and models for the physical and engineering sciences. The first is the broad challenge of automated modeling of data across different processes in a physical system. The second is the dilemma of obtaining insightful interpretations on the relationships between the inputs and outcome of a system as inferred from complex, black box machine learning models.</div><div><br></div><div><b>Automated Geometric Shape Deviation Modeling for Additive Manufacturing Systems</b></div><div><b><br></b></div><div>Additive manufacturing systems possess an intrinsic capability for one-of-a-kind manufacturing of a vast variety of shapes across a wide spectrum of processes. One major issue in AM systems is geometric accuracy control for the inevitable shape deviations that arise in AM processes. Current effective approaches for shape deviation control in AM involve the specification of statistical or machine learning deviation models for additively manufactured products. However, this task is challenging due to the constraints on the number of test shapes that can be manufactured in practice, and limitations on user efforts that can be devoted for learning deviation models across different shape classes and processes in an AM system. We develop an automated, Bayesian neural network methodology for comprehensive shape deviation modeling in an AM system. A fundamental innovation in this machine learning method is our new and connectable neural network structures that facilitate the transfer of prior knowledge and models on deviations across different shape classes and AM processes. Several case studies on in-plane and out-of-plane deviations, regular and free-form shapes, and different settings of lurking variables serve to validate the power and broad scope of our methodology, and its potential to advance high-quality manufacturing in an AM system.</div><div><br></div><div><b>Interpretable Machine Learning</b></div><div><b><br></b></div><div>Machine learning algorithms and models constitute the dominant set of predictive methods for a wide range of complex, real-world processes. However, interpreting what such methods effectively infer from data is difficult in general. This is because their typical black box natures possess a limited ability to directly yield insights on the underlying relationships between inputs and the outcome for a process. We develop methodologies based on new predictive comparison estimands that effectively enable one to ``mine’’ machine learning models, in the sense of (a) interpreting their inferred associations between inputs and/or functional forms of inputs with the outcome, (b) identifying the inputs that they effectively consider relevant, and (c) interpreting the inferred conditional and two-way associations of the inputs with the outcome. We establish Fisher consistent estimators, and their corresponding standard errors, for our new estimands under a condition on the inputs' distributions. The significance of our predictive comparison methodology is demonstrated with a wide range of simulation and case studies that involve Bayesian additive regression trees, neural networks, and support vector machines. Our extended study of interpretable machine learning for AM systems demonstrates how our method can contribute to smarter advanced manufacturing systems, especially as current machine learning methods for AM are lacking in their ability to yield meaningful engineering knowledge on AM processes. <br></div>