In order for a robot to operate autonomously in an environment, it must be able to locate itself within it. A robot's position and orientation cannot be directly measured by physical sensors, so estimating it is a non-trivial problem. Some sensors provide this information, such as the Global Navigation Satellite System (GNSS) and Motion capture (Mo-cap). Nevertheless, these sensors are expensive to set up, or they are not useful in environments where autonomous vehicles are often deployed. Our proposal explores a new approach to sensing for relative motion and position estimation. It consists of one vision sensor and a marker that utilizes moiré phenomenon to estimate the position of the vision sensor by using Convolutional Neural Networks (CNN) trained to estimate the position from the pattern shown on the marker. We share the process of data collection and training of the network and share the hyperparameter search method used to optimize the structure of the network. We test the trained network in a setup to evaluate its ability in estimating position. The system achieved an average absolute error of 1 cm, showcasing a method that could be used to overcome the current limitations of vision approaches in pose estimation.

Prenatal hydronephrosis is a common condition that involves the accumulation of urine with consequent dilatation of the collecting system in fetal infants. There are several hydronephrosis classifications, however, all grading systems suffer from reliability issues as they contain subjective criteria. The severity of hydronephrosis impacts treatment and follow up times and can therefore directly influence a patient’s well-being and quality of care. Considering the importance of accurate diagnosis, it is concerning that no accurate, reliable or objective grading system exists. We believe that developing a convolutional neural network (CNN) based diagnostic aid for hydronephrosis will improve physicians’ objectivity, inter-rater reliability and accuracy. Developing CNN based diagnostic aid for ultrasound images has not been done before. Therefore, the current thesis conducted two studies using a database of 4670 renal ultrasound images to investigate two important methodological considerations: ultrasound image preprocessing and model architecture. We first investigated whether image segmentation and textural extraction are beneficial and improve performance when they are applied to CNN input images. Our results showed that neither preprocessing technique improved performance, and therefore might not be required when using CNN for ultrasound image classification. Our search for an optimal architecture resulted in a model with 49% 5-way classification accuracy. Further investigation revealed that images in our database had been mislabelled, and thus impacted model training and testing. Although our current best model is not ready for use as diagnostic aid, it can be used to verify the accuracy of our labels. Overall, these studies have provided insight into developing a diagnostic aid for hydronephrosis. Once our images and their respective labels have been verified, we can further optimize our model architecture by conducting an exhaustive search. We hypothesize that these two changes will significantly improve model performance and bring our diagnostic aid closer to clinical application. / Thesis / Master of Science (MSc) / Prenatal hydronephrosis is a serious condition that affects the kidneys of fetal infants and is graded using renal ultrasound. The severity of hydronephrosis impacts treatment and follow-up times. However, all grading systems suffer from reliability issues. Improving diagnostic reliability is important for patient well-being. We believe that developing a computer-based diagnostic aid is a promising option to do so. We conducted two studies to investigate how ultrasound images should be processed, and how the algorithm that produces the functionality of the aid should be designed. We found that two common recommendations for ultrasound processing did not improve model performance and therefore need not be applied. Our best performing algorithm had a classification accuracy of 49%. However, we found that several images in our database were mislabelled, which impacted accuracy metrics. Once our images and their labels have been verified, we can further optimize our algorithm’s design to improve its accuracy.

The electrochemical oxygen reduction reaction (ORR) is a very important catalytic process that is directly used in carbon-free energy systems like fuel cells. However, the lack of active, stable, and cost-effective ORR cathode materials has been a major impediment to the broad adoption of these technologies. So, the challenge for researchers in catalysis is to find catalysts that are electrochemically efficient to drive the reaction, made of earth-abundant elements to lower material costs and allow scalability, and stable to make them last longer. The majority of commercial catalysts that are now being used have been found through trial and error techniques that rely on the chemical intuition of experts. This method of empirical discovery is, however, very challenging, slow, and complicated because the performance of the catalyst depends on a myriad of factors. Researchers have recently turned to machine learning (ML) to find and design heterogeneous catalysts faster with emerging catalysis databases. Black-box models make up a lot of the ML models that are used in the field to predict the properties of catalysts that are important to their performance, such as their adsorption energies to reaction intermediates. However, as these black-box models are based on very complicated mathematical formulas, it is very hard to figure out how they work and the underlying physics of the desired catalyst properties remains hidden. As a way to open up these black boxes and make them easier to understand, more attention is being paid to interpretable and explainable ML. This work aims to speed up the process of screening and optimizing Pt monolayer alloys for ORR while gaining physical insights. We use a theory-infused machine learning framework in combination with a high-throughput active screening approach to effectively find promising ORR Pt monolayer catalysts. Furthermore, an explainability game-theory approach is employed to find electronic factors that control surface reactivity. The novel insights in this study can provide new design strategies that could shape the paradigm of catalyst discovery. / Doctor of Philosophy / The electrochemical oxygen reduction reaction (ORR) is a very important catalytic process that is used directly in carbon-free energy systems like fuel cells. But the lack of ORR cathode materials that are active, stable, and cheap has made it hard for these technologies to be widely used. Most of the commercially used catalysts have been found through trial-and-error methods that rely on the chemical intuition of experts. This method of finding out through experience is hard, slow, and complicated, though, because the performance of the catalyst depends on a variety of factors. Researchers are now using machine learning (ML) and new catalysis databases to find and design heterogeneous catalysts faster. But because black-box ML models are based on very complicated mathematical formulas, it is very hard to figure out how they work, and the physics behind the desired catalyst properties remains hidden. In recent years, more attention has been paid to ML that can be understood and explained as a way to decode these "black boxes" and make them easier to understand. The goal of this work is to speed up the screening and optimization of Pt monolayer alloys for ORR. We find promising ORR Pt monolayer catalysts by using a machine learning framework that is based on theory and a high-throughput active screening method. A game-theory approach is also used to find the electronic factors that control surface reactivity. The new ideas in this study can lead to new ways of designing that could alter how researchers find catalysts.

Computational chemistry and machine learning approaches are combined to understand the mechanisms, derive activity trends, and ultimately to search for active electrocatalysts for the electrochemical oxidation of ammonia (AOR) and nitrate reduction (NO3RR). Both re- actions play vital roles within the nitrogen cycle and have important applications within tackling current environmental issues. Mechanisms are studied through the use of density functional theory (DFT) for AOR and NO3RR, subsequently a descriptor based approach is used to understand activity trends on a wide range of electrocatalysts. For AOR inter- pretable machine learning is used in conjunction with active learning to screen for active and stable ternary electrocatalysts. We find Pt3RuCo, Pt3RuNi and Pt3RuFe show great activity, and are further validated via experimental results. By leveraging the advantages of the interpretible machine learning model we elucidate the underlying electronic factors for the stronger *N binding which leads to the observed improved activity. For NO3RR an interpretible machine learning model is used to understand ways to bypass the stringent limitations put on the electrocatalytic activity due to the *N vs *NO3 scaling relations. It is found that the *N binding energy can be tuned while leaving the *NO3 binding energy unaffected by ensuring that the subsurface atom interacts strongly with the *N. Based on this analysis we suggest the B2 CuPd as a potential active electrocatalyst for this reaction, which is further validated by experiments / Doctor of Philosophy / The chemical reactions that makeup the nitrogen cycle have played a pivotal role in human society, consider the fact that one of the most impactful achievements of the 20th century was the conversion of nitrogen (N2) to ammonia (NH3) via the Haber-Bosch process. The key class of materials to facilitate such transformations are called catalysts, which provide a reactive surface for the reaction to occur at reasonable reaction rates. Using quantum chemistry we can understand how various reactions proceed on the catalyst surface and how the catalyst can be designed to maximize the reaction rate. Specifically here we are interested in the electrochemical oxidation of ammonia (AOR) and reduction of nitrate (NO3RR), which have important energy and environmental applications. The atomistic insight provided by quantum chemistry helps us understand the reaction mechanism and key hurdles in developing new catalysts. Machine learning can then be leveraged in various ways to find novel catalysts. For AOR machine learning finds novel active catalysts from a diverse design space, which are then experimentally tested and verified. Through the use of our machine learning algorithm (TinNet) we also provide new insights into why the catalysts are more active, and suggest novel physics that can help design active catalysts. For NO3RR we use machine learning as a tool to help us understand the hurdles in catalyst design better which then guides our catalyst discovery. It is shown that CuPd could be a potential candidate and is also verified via experimental synthesis and performance testing.

High energy density experiments rely heavily on predictive physics simulations in the design process. Specifically in inertial confinement fusion (ICF), predictive physics simulations, such as in the radiation-hydrodynamics code xRAGE, are computationally expensive, limiting the design process and ability to find an optimal design. Machine learning provides a mechanism to leverage expensive simulation data and alleviate limitations on computational time and resources in the search for an optimal design. Machine learning efficiently identifies regions of design space with high predicted performance as well as regions with high uncertainty to focus simulations, which may lead to unexpected designs with great potential. This dissertation focuses on the application of Bayesian optimization to design optimization for ICF experiments conducted by the double shell campaign at Los Alamos National Lab (LANL). The double shell campaign is interested in implementing graded inner shell layers to their capsule geometry. Graded inner shell layers are expected to improve stability in the implosions with fewer sharp density jumps, but at the cost of lower yields, in comparison to the nominal bilayer inner shell targets. This work explores minimizing hydrodynamic instability and maximizing yield for the graded inner shell targets by building and coupling a multi-fidelity Bayesian optimization framework with multi-dimensional xRAGE simulations for an improved design process. / Doctor of Philosophy / Inertial confinement fusion (ICF) is an active field of research in which a fuel is compressed to extreme temperatures and densities to achieve thermonuclear ignition. Ignition is achieved when the fuel can continuously heat itself and sustain its reactions. These fusion reactions would produce large amounts of energy. Power plants using fusion could solve many of the world's energy concerns with far less pollution than current energy sources. Although ignition has not been achieved in the lab, ICF researchers are actively working towards this goal. At Los Alamos National Lab (LANL), ICF researchers are focused on studying ignition-relevant conditions for "double shell" targets through experiments at laser facilities, such at the National Ignition Facility (NIF). These experiments are extremely expensive to field, design, and analyze. To obtain the maximum information from each experiment, researchers rely on predictive physics simulations, which are computationally intensive, making it difficult to find optimal target designs. In this dissertation, better use of simulations is made by focusing on using machine learning along with simulation data to find optimal target designs. Machine learning allows for efficient use of limited computational time and resources on simulations, such that an optimal target design can be found in a reasonable amount of time before an ICF experiment. This dissertation specifically looks at using Bayesian optimization for design optimization of LANL's double shell capsules with graded material inner shells. Several Bayesian optimization frameworks are presented, along with a discussion of optimal designs and physics mechanisms that lead to high performing capsule designs. The work from this dissertation will create an improved design process for the LANL double shell (and other) campaigns, providing high fidelity optimization of ICF targets.

People interact with different critical applications in day-to-day life. Some examples of critical applications include computer programs, anonymous vehicles, digital healthcare, smart homes, etc. There are inherent risks in these critical applications if they fail to perform properly. In my dissertation, we mainly focus on developing methodologies for performance improvement for software security and healthcare prognosis. Cryptographic vulnerability tools are used to detect misuses of Java cryptographic APIs and thus classify secure and insecure parts of code. These detection tools are critical applications as misuse of cryptographic libraries and APIs causes devastating security and privacy implications. We develop two benchmarks that help developers to identify secure and insecure code usage as well as improve their tools. We also perform a comparative analysis of four static analysis tools. The developed benchmarks enable the first scientific comparison of the accuracy and scalability of cryptographic API misuse detection. Many published detection tools (CryptoGuard, CrySL, Oracle Parfait) have used our benchmarks to improve their performance in terms of the detection capability of insecure cases. We also examine the need for performance improvement for healthcare applications. Numerous prediction applications are developed to predict patients' health conditions. These are critical applications where misdiagnosis can cause serious harm to patients, even death. Due to the imbalanced nature of many clinical datasets, our work provides empirical evidence showing various prediction deficiencies in a typical machine learning model. We observe that missed death cases are 3.14 times higher than missed survival cases for mortality prediction. Also, existing sampling methods and other techniques are not well-equipped to achieve good performance. We design a double prioritized (DP) technique to mitigate representational bias or disparities across race and age groups. we show DP consistently boosts the minority class recall for underrepresented groups, by up to 38.0%. Our DP method also shows better performance than the existing methods in terms of reducing relative disparity by up to 88% in terms of minority class recall. Incorrect classification in these critical applications can have significant ramifications. Therefore, it is imperative to improve the performance of critical applications to alleviate risk and harm to people. / Doctor of Philosophy / We interact with many software using our devices in our everyday life. Examples of software usage include calling transport using Lyft or Uber, doing online shopping using eBay, using social media via Twitter, check payment status from credit card accounts or bank accounts. Many of these software use cryptography to secure our personal and financial information. However, the inappropriate or improper use of cryptography can let the malicious party gain sensitive information. To capture the inappropriate usage of cryptographic functions, there are several detection tools are developed. However, to compare the coverage of the tools, and the depth of detection of these tools, suitable benchmarks are needed. To bridge this gap, we aim to build two cryptographic benchmarks that are currently used by many tool developers to improve their performance and compare their tools with the existing tools. In another aspect, people see physicians and are admitted to hospitals if needed. Physicians also use different software that assists them in caring the patients. Among this software, many of them are built using machine learning algorithms to predict patients' conditions. The historical medical information or clinical dataset is taken as input to the prediction models. Clinical datasets contain information about patients of different races and ages. The number of samples in some groups of patients may be larger than in other groups. For example, many clinical datasets contain more white patients (i.e., majority group) than Black patients (i.e., minority group). Prediction models built on these imbalanced clinical data may provide inaccurate predictions for minority patients. Our work aims to improve the prediction accuracy for minority patients in important medical applications, such as estimating the likelihood of a patient dying in an emergency room visit or surviving cancer. We design a new technique that builds customized prediction models for different demographic groups. Our results reveal that subpopulation-specific models show better performance for minority groups. Our work contributes to improving the medical care of minority patients in the age of digital health. Overall, our aim is to improve the performance of critical applications to help people by decreasing risk. Our developed methods can be applicable to other critical application domains.

Understanding the relationship between gene expression and phenotype is important in many areas of biology and medicine. Current methods for measuring gene expression such as microarrays however are invasive, require biopsy, and expensive. These factors limit experiments to low rate temporal sampling of gene expression and prevent longitudinal studies within a single subject, reducing their statistical power. Thus methods for non-invasive measurements of gene expression are an important and current topic of research. An interesting approach (Segal et al, Nature Biotechnology 25 (6) 2007) to indirect measurements of gene expression has recently been reported that uses existing imaging techniques and machine learning to estimate a function mapping image features to gene expression patterns, providing an image-derived surrogate for gene expression. However, the design of machine learning methods for this purpose is hampered by the cost of training and validation. My thesis shows that populations of artificial organisms simulating genetic variation can be used for designing machine learning approaches to decoding gene expression patterns from images. If analysis of these images proves successful, then this can be applied to real biomedical images reducing the limitations of invasive imaging. The results showed that the box counting dimension was a suitable feature extraction method yielding a classification rate of at least 90% for mutation rates up to 40%. Also, the box-counting dimension was robust in dealing with distorted images. The performance of the classifiers using the fractal dimension as features, actually, seemed more vulnerable to the mutation rate as opposed to the applied distortion level. / Master of Science

Patellar tendinopathy (PT) is a knee injury resulting in pain localized to the patellar tendon. One main factor that causes PT is repetitive overloading of the tendon. Because of this mechanism, PT is commonly seen in "jumping sports" like basketball. This injury can severely impact a player's performance, and in order for a timely return to preinjury activity levels early diagnosis and treatment is important. The standard for the diagnosis of PT is a clinical examination, including a patient history and a physical assessment. Because PT has similar symptoms to injuries of other knee structures like the bursae, fat pad, and patellofemoral joint, imaging is regularly performed to aid in determining the correct diagnosis. One common imaging modality for the patellar tendon is gray-scale ultrasonography (GS-US). However, the accurate detection of PT in GS-US images is grader dependent and requires a high level of expertise. Machine learning (ML) models, which can accurately and objectively perform image classification tasks, could be used as a reliable automated tool to aid clinicians in assessing PT in GS-US images. ML models, like support vector machines (SVMs) and convolutional neural networks (CNNs), use features learned from labelled images, to predict the class of an unlabelled image. SVMs work by creating an optimal hyperplane between classes of labelled data points, and then classifies an unlabelled datapoint depending on which side of the hyperplane it falls. CNNs work by learning the set of features and recognizing what pattern of features describes each class. The objective of this study was to develop a SVM model and a CNN model to classify GS-US images of the patellar tendon as either normal or diseased (PT present), with an accuracy around 83%, the accuracy that experienced clinicians achieved when diagnosing PT in GS-US images that were already clinically diagnosed as either diseased or normal. We will also compare different test designs for each model to determine which achieved the highest accuracy. GS-US images of the patellar tendon were obtained from male and female Virginia Tech collegiate basketball athletes. Each image was labelled by an experienced clinician as either diseased or normal. These images were split into training and testing sets. The SVM and the CNN models were created using Python. For the SVM model, features were extracted from the training set using speeded up robust features (SURF). These features were then used to train the SVM model by calculating the optimal weights for the hyperplane. For the CNN model, the features were learned by layers within the CNN as were the optimal weights for classification. Both of these models were then used to predict the class of the images within the testing set, and the accuracy, sensitivity and precision of the models were calculated. For each model we looked at different test designs. The balanced designs had the same amount of diseased and normal images. The designs with Long images had only images taken in the longitudinal orientation, unlike Long+Trans, which had both longitudinal and transverse images. The designs with Full images contained the patellar tendon and surrounding tissue, whereas the ROI images removed the surrounding tissue. The best designs for the SVM model were the Unbalanced Long designs for both the Full and ROI images. Both designs had an accuracy of 77.5%. The best design for the CNN model was the Balanced Long+Trans Full design, with an accuracy of 80.3%. Both of the models had more difficulty classifying normal images than diseased images. This may be because the diseased images had a well defined feature pattern, while the normal images did not. Overall, the CNN features and classifier achieved a higher accuracy than the SURF features and SVM classifier. The CNN model is only slightly below 83%, the accuracy of an experienced clinician. These are promising results, and as the data set size increases and the models are fine tuned, the accuracy of the model will only continue to increase. / Master of Science / Patellar tendinopathy (PT) is a common knee injury. This injury is frequently seen in sports like basketball, where athletes are regularly jumping and landing, and ultimately applying a lot of force onto the patellar tendon. This injury can severely impact a player's performance, and in order for a timely return to preinjury activity levels early diagnosis and treatment is important. Currently, diagnosis of PT involves a patient history and a physical assessment, and is commonly supplemented by ultrasound imaging. However, clinicians need to have a high level of expertise in order to accurately assess these images for PT. In order to aid in this assessment, a tool like Machine learning (ML) models could be used. ML is becoming more and more prevalent in our every day lives. These models are everywhere, from the facial recognition tool on your phone to the list of recommended items on your Amazon account. ML models can use features learned from labelled images, to predict the class of an unlabeled image. The objective of this study was to develop ML models to classify ultrasound images of the patellar tendon as either normal or diseased (PT present).

Gas Chromatography (GC) is a technique for separating volatile compounds by relying on adherence differences in the chemical components of the compound. As conditions within the GC are changed, components of the mixture elute at different times. Sensors measure the elution and produce data which becomes chromatograms. By analyzing the chromatogram, the presence and quantity of the mixture's constituent components can be determined. Machine Learning (ML) is a field consisting of techniques by which machines can independently analyze data to derive their own procedures for processing it. Additionally, there are techniques for enhancing the performance of ML algorithms. Feature Selection is a technique for improving performance by using a specific subset of the data. Feature Engineering is a technique to transform the data to make processing more effective. Data Fusion is a technique which combines multiple sources of data so as to produce more useful data. This thesis applies machine learning algorithms to chromatograms. Five common machine learning algorithms are analyzed and compared, including K-Nearest Neighbour (KNN), Support Vector Machines (SVM), Convolutional Neural Network (CNN), Decision Tree, and Random Forest (RF). Feature Selection is tested by applying window sweeps with the KNN algorithm. Feature Engineering is applied via the Principal Component Analysis (PCA) algorithm. Data Fusion is also tested. It was found that KNN and RF performed best overall. Feature Selection was very beneficial overall. PCA was helpful for some algorithms, but less so for others. Data Fusion was moderately beneficial. / Master of Science / Gas Chromatography is a method for separating a mixture into its constituent components. A chromatogram is a time series showing the detection of gas in the gas chromatography machine over time. With a properly set up gas chromatographer, different mixtures will produce different chromatograms. These differences allow researchers to determine the components or differentiate compounds from each other. Machine Learning (ML) is a field encompassing a set of methods by which machines can independently analyze data to derive the exact algorithms for processing it. There are many different machine learning algorithms which can accomplish this. There are also techniques which can process the data to make it more effective for use with machine learning. Feature Engineering is one such technique which transforms the data. Feature Selection is another technique which reduces the data to a subset. Data Fusion is a technique which combines different sources of data. Each of these processing techniques have many different implementations. This thesis applies machine learning to gas chromatography. ML systems are developed to classify mixtures based on their chromatograms. Five common machine learning algorithms are developed and compared. Some common Feature Engineering, Feature Selection, and Data Fusion techniques are also evaluated. Two of the algorithms were found to be more effective overall than the other algorithms. Feature Selection was found to be very beneficial. Feature Engineering was beneficial for some algorithms but less so for others. Data Fusion was moderately beneficial.

This dissertation presents a study of machine learning methods with a focus on applications to statistical and condensed matter physics, in particular the problem of quantum state preparation, spin-glass and constraint satisfiability. We will start by introducing the core principles of machine learning such as overfitting, bias-variance tradeoff and the disciplines of supervised, unsupervised and reinforcement learning. This discussion will be set in the context of recent applications of machine learning to statistical physics and condensed matter physics. We then present the problem of quantum state preparation and show how reinforcement learning along with stochastic optimization methods can be applied to identify and define phases of quantum control. Reminiscent of condensed matter physics, the underlying phases of quantum control are identified via a set of order parameters and further detailed in terms of their universal implications for optimal quantum control. In particular, casting the optimal quantum control problem as an optimization problem, we show that it exhibits a generic glassy phase and establish a connection with the fields of spin-glass physics and constraint satisfiability problems. We then demonstrate how unsupervised learning methods can be used to obtain important information about the complexity of the phases described. We end by presenting a novel clustering framework, termed HAL for hierarchical agglomerative learning, which exploits out-of-sample accuracy estimates of machine learning classifiers to perform robust clustering of high-dimensional data. We show applications of HAL to various clustering problems.