<p dir="ltr">Humans exhibit remarkable ability in continual adaptation and learning new tasks throughout their lifetime while maintaining the knowledge gained from past experiences. In stark contrast, artificial neural networks (ANNs) under such continual learning (CL) paradigm forget the information learned in the past tasks upon learning new ones. This phenomenon is known as ‘Catastrophic Forgetting’ or ‘Catastrophic Interference’. The objective of this thesis is to enable efficient continual learning in deep neural networks while mitigating this forgetting phenomenon. Towards this, first, a continual learning algorithm (SPACE) is proposed where a subset of network filters or neurons is allocated for each task using Principal Component Analysis (PCA). Such task-specific network isolation not only ensures zero forgetting but also creates structured sparsity in the network which enables energy-efficient inference. Second, a fast and more efficient training algorithm for CL is proposed by introducing Gradient Projection Memory (GPM). Here, the most important gradient spaces (GPM) for each task are computed using Singular Value Decomposition (SVD) and the new tasks are learned in the orthogonal direction to GPM to minimize forgetting. Third, to improve new learning while minimizing forgetting, a Scaled Gradient Projection (SGP) method is proposed that, in addition to orthogonal gradient updates, allows scaled updates along the important gradient spaces of the past task. Next, for continual learning on an online stream of tasks a memory efficient experience replay method is proposed. This method utilizes saliency maps explaining network’s decision for selecting memories that are replayed during new tasks for preventing forgetting. Finally, a meta-learning based continual learner - Amphibian - is proposed that achieves fast online continual learning without any experience replay. All the algorithms are evaluated on short and long sequences of tasks from standard image-classification datasets. Overall, the methods proposed in this thesis address critical limitations of DNNs for continual learning and advance the state-of-the-art in this domain.</p>

<p dir="ltr">Machine learning (ML) algorithms are increasingly being used in a variety of applications and are heavily relied upon to make decisions that impact people’s lives. ML models are often praised for their precision, yet they can discriminate against certain groups due to biased data. These biases, rooted in historical inequities, pose significant challenges in developing fair and unbiased models. Central to addressing this issue is the mitigation of biases inherent in the training data, as their presence can yield unfair and unjust outcomes when models are deployed in real-world scenarios. This study investigates the efficacy of data acquisition, i.e., one of the stages of data preparation, akin to the pre-processing bias mitigation technique. Through experimental evaluation, we showcase the effectiveness of data acquisition, where the data is acquired using data valuation techniques to enhance the fairness of machine learning models.</p>

<p dir="ltr">In this work, first, unsupervised machine learning is proposed as a study for detecting and classifying jamming attacks targeting unmanned aerial vehicles (UAV) operating at a 2.4 GHz band. Three scenarios are developed with a dataset of samples extracted from meticulous experimental routines using various unsupervised learning algorithms, namely K-means, density-based spatial clustering of applications with noise (DBSCAN), agglomerative clustering (AGG) and Gaussian mixture model (GMM). These routines characterize attack scenarios entailing barrage (BA), single- tone (ST), successive-pulse (SP), and protocol-aware (PA) jamming in three different settings. In the first setting, all extracted features from the original dataset are used (i.e., nine in total). In the second setting, Spearman correlation is implemented to reduce the number of these features. In the third setting, principal component analysis (PCA) is utilized to reduce the dimensionality of the dataset to minimize complexity. The metrics used to compare the algorithms are homogeneity, completeness, v-measure, adjusted mutual information (AMI) and adjusted rank index (ARI). The optimum model scored 1.00, 0.949, 0.791, 0.722, and 0.791, respectively, allowing the detection and classification of these four jamming types with an acceptable degree of confidence.</p><p dir="ltr">Second, following a different study, supervised learning (i.e., random forest modeling) is developed to achieve a binary classification to ensure accurate clustering of samples into two distinct classes: clean and jamming. Following this supervised-based classification, two-class and three-class unsupervised learning is implemented considering three of the four jamming types: BA, ST, and SP. In this initial step, the four aforementioned algorithms are used. This newly developed study is intended to facilitate the visualization of the performance of each algorithm, for example, AGG performs a homogeneity of 1.0, a completeness of 0.950, a V-measure of 0.713, an ARI of 0.557 and an AMI of 0.713, and GMM generates 1, 0.771, 0.645, 0.536 and 0.644, respectively. Lastly, to improve the classification of this study, semi-supervised learning is adopted instead of unsupervised learning considering the same algorithms and dataset. In this case, GMM achieves results of 1, 0.688, 0.688, 0.786 and 0.688 whereas DBSCAN achieves 0, 0.036, 0.028, 0.018, 0.028 for homogeneity, completeness, V-measure, ARI and AMI respectively. Overall, this unsupervised learning is approached as a method for jamming classification, addressing the challenge of identifying newly introduced samples.</p>

<p dir="ltr">The goal of this thesis is to develop a method of evaluating long-term IAQ performance of an industrial facility and use machine-learning to model the relationship between critical air pollutants and the facility’s HVAC systems and processes. The facility under study for this thesis is an electroplating manufacturer. The air pollutants at this facility that were studied were particulate matter, total-volatile organic compounds, and carbon-dioxide. Upon sensor installation, seven “zones” were identified to isolate areas of the plant for measurement and analysis. A statistical review of the long-term data highlighted how this facility performed in terms of compliance. Their gaseous pollutants were well within regulation. Particulate matter, however, was found to be a pressing issue. PM10 was outside of compliance more than 15% of the time in five out of seven of the zones of study. Some zones were out of compliance up to 80% of the total collection period. The six pollutants that met these criteria were deemed critical and moved on to machine learning modeling. Our model of best fit for each pollutant used a gaussian process regression model, which fits best for non-linear rightly skewed datasets. The performance of each of our models was deemed significant. Every model had at least a regression coefficient of 0.935 and above for both validation and testing. The maximum average error was 12.64 ug.m^3, which is less than 10% of the average PM10 concentration. Through our modeling, we were able to study how HVAC and production played a role in particulate matter presence for each zone. Exhaust systems of the west side of the plant were found to be insufficient at removing particulates from their facility. Overall, the methods developed in this thesis project were able to meet the goal of analyzing IAQ compliance, modeling critical pollutants using machine learning, and identifying a relationship between these pollutants and an industrial facility’s HVAC and production systems.</p>

<p dir="ltr">Our current constitutive models of material behavior fall short of being able to describe the mechanics of soft tissues. This is because soft tissues like skin and rubber, unlike traditional engineering materials, exhibit extremely nonlinear mechanical behavior and usually undergo large deformations. Developing accurate constitutive models for such materials requires using flexible tools at the forefront of science, such as machine learning methods. However, our past experiences show that it is crucial to incorporate physical knowledge in models of physical phenomena. The past few years has witnessed the rise of physics-informed models where the goal is to impose governing physical laws by incorporating them in the loss function. However, we argue that such "soft" constraints are not enough. This "persuasion" method has no theoretical guarantees on the satisfaction of physics and result in overly complicated loss functions that make training of the models cumbersome. </p><p dir="ltr">We propose imposing the relevant physical laws as "hard" constraints. In this approach the physics of the problem are "baked in" into the structure of the model preventing it from ever violating them. We demonstrate the power of this paradigm on a number of constitutive models of soft tissue, including hyperelasticity, viscoelasticity and continuum damage models. </p><p dir="ltr">We also argue that new uncertainty quantification strategies have to be developed to address the rise in dimensionality and the inherent symmetries present in most machine learning models compared to traditional constitutive models. We demonstrate that diffusion models can be used to construct a generative framework for physics-constrained hyperelastic constitutive models.</p>

<p dir="ltr">This dissertation presents decentralized and agent-based solutions for organizing machine learning resources, such as datasets and learning models. It aims to democratize the analysis of these resources through a simple yet flexible query structure, automate common ML tasks such as training, testing, model selection, and hyperparameter tuning, and enable privacy-centric building of ML models over distributed datasets. Based on networked multi-agent systems, the proposed approach represents ML resources as autonomous and self-reliant entities. This representation makes the resources easily movable, scalable, and independent of geographical locations, alleviating the need for centralized control and management units. Additionally, as all machine learning and data mining tasks are conducted near their resources, providers can apply customized rules independently of other parts of the system. </p><p><br></p>

<p>Proteins are significant components of living cells. They perform a diverse range of biological functions such as cell shape and metabolism. The functions of proteins are determined by their three-dimensional structures. Cryogenic-electron microscopy (cryo-EM) is a technology known for determining the structure of large macromolecular structures including protein complexes. When individual atomic protein structures are available, a critical task in structure modeling is fitting the individual structures into the cryo-EM density map.</p> <p>In my research, I report a new computational method, MarkovFit, which is a machine learning-based method that performs simultaneous rigid fitting of the atomic structures of individual proteins into cryo-EM maps of medium to low resolution to model the three-dimensional structure of protein complexes. MarkovFit uses Markov random field (MRF), which allows probabilistic evaluation of fitted models. MarkovFit starts by searching the conformational space using FFT for potential poses of protein structures, computes scores which quantify the goodness-of-fit between each individual protein and the cryo-EM map, and the interactions between the proteins. Afterwards, proteins and their interactions are represented using a MRF graph. MRF nodes use a belief propagation algorithm to exchange information, and the best conformations are then extracted and refined using two structural refinement methods. </p> <p>The performance of MarkovFit was tested on three datasets; a dataset of simulated cryo-EM maps at resolution 10 Å, a dataset of high-resolution experimentally-determined cryo-EM maps, and a dataset of experimentally-determined cryo-EM maps of medium to low resolution. In addition to that, the performance of MarkovFit was compared to two state-of-the-art methods on their datasets. Lastly, MarkovFit modeled the protein complexes from the individual protein atomic models generated by AlphaFold, an AI-based model developed by DeepMind for predicting the 3D structure of proteins from their amino acid sequences.</p>

<p>The only datasets used in my thesis work are from TUDatasets, <a href="https://chrsmrrs.github.io/datasets/">TUDataset | TUD Benchmark datasets (chrsmrrs.github.io)</a>, a collection of public benchmark datasets for graph classification and regression.</p><p><br></p>

<p>  </p> <p>The proper classification and disposal of waste are crucial in reducing environmental impacts and promoting sustainability. Several solid waste classification systems have been developed over the years, ranging from manual sorting to mechanical and automated sorting. Manual sorting is the oldest and most commonly used method, but it is time-consuming and labor-intensive. Mechanical sorting is a more efficient and cost-effective method, but it is not always accurate, and it requires constant maintenance. Automated sorting systems use different types of sensors and algorithms to classify waste, making them more accurate and efficient than manual and mechanical sorting systems. In this thesis, we propose the development of an intelligent solid waste detection, classification and tracking system using artificial deep learning techniques. To address the limited samples in the TrashNetV2 dataset and enhance model performance, a data augmentation process was implemented. This process aimed to prevent overfitting and mitigate data scarcity issues while improving the model's robustness. Various augmentation techniques were employed, including random rotation within a range of -20° to 20° to account for different orientations of the recycled materials. A random blur effect of up to 1.5 pixels was used to simulate slight variations in image quality that can arise during image acquisition. Horizontal and vertical flipping of images were applied randomly to accommodate potential variations in the appearance of recycled materials based on their orientation within the image. Additionally, the images were randomly scaled to 416 by 416 pixels, maintaining a consistent image size while increasing the dataset's overall size. Further variability was introduced through random cropping, with a minimum zoom level of 0% and a maximum zoom level of 25%. Lastly, hue variations within a range of -20° to 20° were randomly introduced to replicate lighting condition variations that may occur during image acquisition. These augmentation techniques collectively aimed to improve the dataset's diversity and the model's performance. In this study, YOLOv8, EfficientNet-B0 and VGG16 architectures were evaluated, and stochastic gradient descent (SGD) and Adam were used as the optimizer. Although, SGD provided better test accuracies compared to Adam. </p> <p>Among the three models, YOLOv8 showed the best performance, with the highest average precision mAP of 96.5%. YOLOv8 emerges as the top performer, with ROC values varying from 92.70% (Metal) to 98.40% (Cardboard). Therefore, the YOLOv8 model outperforms both VGG16 and EfficientNet in terms of ROC values and mAP. The findings demonstrate that our novel classifier tracker system made of YOLOv8, and supervision algorithms surpass conventional deep learning methods in terms of precision, resilience, and generalization ability. Our contribution to waste management is in the development and implementation of an intelligent solid waste detection, classification, and tracking system using computer vision and deep learning techniques. By utilizing computer vision and deep learning algorithms, our system can accurately detect, classify, and localize various types of solid waste on a moving conveyor, including cardboard, glass, metal, paper, and plastic. This can significantly improve the efficiency and accuracy of waste sorting processes.</p> <p>This research provides a promising solution for detection, classification, localization, and tracking of solid waste materials in real time system, which can be further integrated into existing waste management systems. Through comprehensive experimentation and analysis, we demonstrate the superiority of our approach over traditional methods, with higher accuracy and faster processing times. Our findings provide a compelling case for the implementation of intelligent solid waste sorting.</p>

<p><br></p> <p>Police use of force has become a topic of significant concern, particularly given the disparate impact on communities of color. Research has shown that police officer involved shootings, misconduct and excessive use of force complaints exhibit network effects, where officers are at greater risk of being involved in these incidents when they socialize with officers who have a history of use of force and misconduct. Given that use of force and misconduct behavior appear to be transmissible across police networks, we are attempting to address if police networks can be altered to reduce use of force and misconduct events in a limited scope.</p> <p><br></p> <p>In this work, we analyze a novel dataset from the Indianapolis Metropolitan Police Department on officer field training, subsequent use of force, and the role of network effects from field training officers. We construct a network survival model for analyzing time-to-event of use of force incidents involving new police trainees. The model includes network effects of the diffusion of risk from field training officers (FTOs) to trainees. We then introduce a network rewiring algorithm to maximize the expected time to use of force events upon completion of field training. We study several versions of the algorithm, including constraints that encourage demographic diversity of FTOs. The results show that FTO use of force history is the best predictor of trainee's time to use of force in the survival model and rewiring the network can increase the expected time (in days) of a recruit's first use of force incident by 8%. </p> <p>We then discuss the potential benefits and challenges associated with implementing such an algorithm in practice.</p> <p><br></p>