<p dir="ltr">Small molecule structural elucidation of unknown compounds using tandem mass spectrometry (MS/MS) is a significant challenge due to the need for expert interpretation or the use of peak matching software. These methods are time-consuming or constrained by the available molecular entries in databases. To address these limitations, we introduce a machine learning (ML) framework designed to predict functional groups from MS/MS spectra without requiring database searches. The ML models were trained using data from the Mass Bank of North America (MONA) and a custom MS/MS database generated using a high-throughput desorption electrospray ionization (DESI) MS/MS platform. This approach showcases the transferability of model predictions across different instruments, independent of the acquisition method. Our models achieved an average molecular F1 score of 87% for MONA data and 76% for data generated with DESI, with corresponding molecular accuracies of 94% and 87%, respectively. These results demonstrate the robustness of the ML framework across diverse datasets and confirm its validity in predicting structural information.</p><p dir="ltr">To further validate the robustness of our machine learning framework, the models were rigorously tested on blind datasets from independent sources, specifically provided by Corteva Agriscience and Merck & Co., Inc. These datasets were entirely separate from those used in training and validation, ensuring no prior exposure during model development. The blind datasets included spectra generated using a different instrument and a different ionization method—specifically, electrospray ionization (ESI)—which contrasts with the desorption electrospray ionization (DESI) method used to generate the custom MS/MS database for model training one of the models. When tested on this new, unseen data, the model trained on the MONA database, which also contains ESI spectra, achieved an average molecular F1 score of 80% with an accuracy of 88%. The model trained on DESI-generated data performed comparably well, achieving an average molecular F1 score of 78% with an accuracy of 85%. These findings emphasize the practical applicability of the ML framework for mass spectrometry in real-world scenarios, where diverse data sources, instruments, and ionization methods are commonly encountered.</p><p dir="ltr">In an alternate example, lipidomics – the comprehensive study of lipids in a biological system – heavily relies on tandem MS experiments. In particular, multiple reaction monitoring (MRM) profiling has gained considerable attention in MS-based lipidomics, as it can be particularly advantageous for the rapid screening of lipids in biological samples. While effective, the large amounts of data generated from lipidomic MRM-profiling workflows can be daunting. Moreover, existing software tools for lipid annotation often lack comprehensive automated workflows and integration with statistical and bioinformatics platforms. To address these challenges, we developed the Comprehensive Lipidomic Automated Workflow (CLAW) platform, which includes end-to-end modules for worklist generation, lipid annotation, parsing, statistical analysis, and bioinformatic analysis. In particular, CLAW is designed specifically for MRM-profiling, including isomer-specific MRM profiling conducted utilizing ozone electrospray ionization (OzESI), affording lipid species annotation at the carbon-carbon double bond (C=C) level. Notably, CLAW represents a first of its kind platform with integrated AI agents. Importantly, by utilizing a developed integrated language user interface (LUI) with large language models (LLMs), users can interact with CLAW via a chatbot terminal. Specifically, AI agents use LLMs to interpret human language and interact with the various tools and functionalities provided by the CLAW framework, offering robust and context-aware assistance, marking the first end-to-end application of an AI agent in mass spectrometry lipidomics. This innovation represents a significant step forward in automating complex workflows and making advanced analytical tools more accessible to researchers, ultimately accelerating scientific discovery in the field. In short, this AI-driven interface allows users to perform complex statistical analyses more efficiently, positioning CLAW as a powerful tool for high-throughput lipidomics analysis. Successful applications of CLAW to a broad range of biological samples afforded unprecedented insights to disease pathogenesis, including neurodegenerative diseases like Alzheimer's disease.</p>

<p dir="ltr">The growth of electric vehicles (EVs) offers several benefits for air quality improvement and emissions reduction. Nonetheless, EVs also pose several challenges in the area of highway transportation. These barriers are related to the limitations of EV technology, particularly the charge duration and speed of battery recharging, which translate to vehicle range anxiety for EV users. A promising solution to these concerns is V2V DWC technology (Vehicle to Vehicle Dynamic Wireless Charging), particularly mobile energy disseminators (MEDs). The MED is mounted on a large vehicle or truck that charges all participating EVs within a specified locus from the MED. However, current research on MEDs offers solutions that are widely considered impractical for deployment, particularly in urban environments where range anxiety is common. Acknowledging such gap in the literature, this thesis proposes a comprehensive methodological framework for optimal MED deployment decisions. In the first component of the framework, a practical system, termed “ChargingEnv” is developed using reinforcement learning (RL). ChargingEnv simulates the highway environment, which consists of streams of EVs and an MED. The simulation accounts for a possible misalignment of the charging panel and incorporates a realistic EV battery model. The second component of the framework uses multiple deep RL benchmark models that are trained in “ChargingEnv” to maximize EV service quality within limited charging resource constraints. In this study, numerical experiments were conducted to demonstrate the MED deployment decision framework’s efficacy. The findings indicate that the framework’s trained model can substantially improve EV travel range and alleviate battery depletion concerns. This could serve as a vital tool that allows public-sector road agencies or private-sector commercial entities to efficiently orchestrate MED deployments to maximize service cost-effectiveness.</p>

<p dir="ltr">The concept of reinforcement learning has become increasingly relevant in learning- based applications, especially in the field of autonomous navigation, because of its fundamental nature to operate without the necessity of labeled data. However, the infeasibility of training reinforcement learning based autonomous navigation applications in a real-world setting has increased the popularity of researching and developing on autonomous navigation systems by creating simulated environments in game engine platforms. This thesis investigates the comparative performance of Unity and Unreal Engine 5 within the framework of a reinforcement learning system applied to autonomous race car navigation. A rudimentary simulated setting featuring a model car navigating a racetrack is developed, ensuring uniformity in environmental aspects across both Unity and Unreal Engine 5. The research employs reinforcement learning with genetic algorithms to instruct the model car in race track navigation; while the tools and programming methods for implementing reinforcement learning vary between the platforms, the fundamental concept of reinforcement learning via genetic algorithms remains consistent to facilitate meaningful comparisons. The implementation includes logging of key performance variables during run times on each platform. A comparative analysis of the performance data collected demonstrates Unreal Engine's superior performance across the collected variables. These findings contribute insights to the field of autonomous navigation systems development and reinforce the significance of choosing an optimal underlying simulation platform for reinforcement learning applications.</p>

<p dir="ltr">Autonomous vehicles require motion forecasting of their surrounding multiagents (pedestrians</p><p dir="ltr">and vehicles) to make optimal decisions for navigation. The existing methods focus on</p><p dir="ltr">techniques to utilize the positions and velocities of these agents and fail to capture semantic</p><p dir="ltr">information from the scene. Moreover, to mitigate the increase in computational complexity</p><p dir="ltr">associated with the number of agents in the scene, some works leverage Euclidean distance to</p><p dir="ltr">prune far-away agents. However, distance-based metric alone is insufficient to select relevant</p><p dir="ltr">agents and accurately perform their predictions. To resolve these issues, we propose the</p><p dir="ltr">Semantics-aware Interactive Multiagent Motion Forecasting (SIMMF) method to capture</p><p dir="ltr">semantics along with spatial information and optimally select relevant agents for motion</p><p dir="ltr">prediction. Specifically, we achieve this by implementing a semantic-aware selection of relevant</p><p dir="ltr">agents from the scene and passing them through an attention mechanism to extract</p><p dir="ltr">global encodings. These encodings along with agents’ local information, are passed through</p><p dir="ltr">an encoder to obtain time-dependent latent variables for a motion policy predicting the future</p><p dir="ltr">trajectories. Our results show that the proposed approach outperforms state-of-the-art</p><p dir="ltr">baselines and provides more accurate and scene-consistent predictions. </p>

<p dir="ltr">The original definition of Nash Equilibrium applied to normal form games, but the notion has now been extended to various other forms of games including leader-follower games (Stackelberg games), extensive form games, stochastic games, games of incomplete information, cooperative games, and so on. We focus on general-sum stochastic Stackelberg games in this work. An example where such games would be natural to consider is in security games where a defender wishes to protect some targets through deployment of limited resources and an attacker wishes to strategically attack the targets to benefit themselves. The hierarchical order of play arises naturally since the defender typically acts first and deploys a strategy, while the attacker observes the strategy ofthe defender before attacking. Another example where this framework fits is in testing during epidemics, where the leader (the government) sets testing policies and the follower (the citizens) decide at every time step whether to get tested. The government wishes to minimize the number of infected people in the population while the follower wishes to minimize the cost of getting sick and testing. This thesis presents a learning algorithm for players to converge to their stationary policies in a general sum stochastic sequential Stackelberg game. The algorithm is a two time scale implicit policy gradient algorithm that provably converges to stationary points of the optimization problems of the two players. Our analysis allows us to move beyond the assumptions of zero-sum or static Stackelberg games made in the existing literature for learning algorithms to converge.</p><p dir="ltr"><br></p>

<p dir="ltr">This dissertation presents decentralized and agent-based solutions for organizing machine learning resources, such as datasets and learning models. It aims to democratize the analysis of these resources through a simple yet flexible query structure, automate common ML tasks such as training, testing, model selection, and hyperparameter tuning, and enable privacy-centric building of ML models over distributed datasets. Based on networked multi-agent systems, the proposed approach represents ML resources as autonomous and self-reliant entities. This representation makes the resources easily movable, scalable, and independent of geographical locations, alleviating the need for centralized control and management units. Additionally, as all machine learning and data mining tasks are conducted near their resources, providers can apply customized rules independently of other parts of the system. </p><p><br></p>

<p dir="ltr">We present a framework for Trustworthy Artificial Intelligence (TAI) that dynamically assesses trust and scrutinizes past decision-making, aiming to identify both individual and community behavior. The modeling of behavior incorporates proposed concepts, namely trust pressure and trust sensitivity, laying the foundation for predicting future decision-making regarding community behavior, consensus level, and decision-making duration. Our framework involves the development and mathematical modeling of trust pressure and trust sensitivity, drawing on social validation theory within the context of environmental decision-making. To substantiate our approach, we conduct experiments encompassing (i) dynamic trust sensitivity to reveal the impact of learning actors between decision-making, (ii) multi-level trust measurements to capture disruptive ratings, and (iii) different distributions of trust sensitivity to emphasize the significance of individual progress as well as overall progress.</p><p dir="ltr">Additionally, we introduce TAI metrics, trustworthy acceptance, and trustworthy fairness, designed to evaluate the acceptance of decisions proposed by AI or humans and the fairness of such proposed decisions. The dynamic trust management within the framework allows these TAI metrics to discern support for decisions among individuals with varying levels of trust. We propose both the metrics and their measurement methodology as contributions to the standardization of trustworthy AI.</p><p dir="ltr">Furthermore, our trustability metric incorporates reliability, resilience, and trust to evaluate systems with multiple components. We illustrate experiments showcasing the effects of different trust declines on the overall trustability of the system. Notably, we depict the trade-off between trustability and cost, resulting in net utility, which facilitates decision-making in systems and cloud security. This represents a pivotal step toward an artificial control model involving multiple agents engaged in negotiation.</p><p dir="ltr">Lastly, the dynamic management of trust and trustworthy acceptance, particularly in varying criteria, serves as a foundation for causal AI by providing inference methods. We outline a mechanism and present an experiment on human-driven causal inference, where participant discussions act as interventions, enabling counterfactual evaluations once actor and community behavior are modeled.</p>

<p dir="ltr">Learning, planning, and representing knowledge at multiple levels of temporal abstractions provide an agent with the ability to predict consequences of different courses of actions, which is essential for improving the performance of sequential decision making. However, discovering effective temporal abstractions, which the agent can use as skills, and adopting the constructed temporal abstractions for efficient policy learning can be challenging. Despite significant advancements in single-agent settings, temporal abstractions in multi-agent systems remains underexplored. This thesis addresses this research gap by introducing novel algorithms for discovering and employing temporal abstractions in both cooperative and competitive multi-agent environments. We first develop an unsupervised spectral-analysis-based discovery algorithm, aiming at finding temporal abstractions that can enhance the joint exploration of agents in complex, unknown environments for goal-achieving tasks. Subsequently, we propose a variational method that is applicable for a broader range of collaborative multi-agent tasks. This method unifies dynamic grouping and automatic multi-agent temporal abstraction discovery, and can be seamlessly integrated into the commonly-used multi-agent reinforcement learning algorithms. Further, for competitive multi-agent zero-sum games, we develop an algorithm based on Counterfactual Regret Minimization, which enables agents to form and utilize strategic abstractions akin to routine moves in chess during strategy learning, supported by solid theoretical and empirical analyses. Collectively, these contributions not only advance the understanding of multi-agent temporal abstractions but also present practical algorithms for intricate multi-agent challenges, including control, planning, and decision-making in complex scenarios.</p>

<p>Autonomous systems are often deployed in dynamic environments and are challenged with unexpected changes (novelties) in the environments where they receive novel data that was not seen during training. Given the uncertainty, they should be able to operate without (or with limited) human intervention and they are expected to (1) Adapt to such changes while still being effective and efficient in performing their multiple tasks. The system should be able to provide continuous availability of its critical functionalities. (2) Make informed decisions independently from any central authority. (3) Be Cognitive: learns the new context, its possible actions, and be rich in knowledge discovery through mining and pattern recognition. (4) Be Reflexive: reacts to novel unknown data as well as to security threats without terminating on-going critical missions. These characteristics combine to create the workflow of autonomous decision-making process in multi-agent environments (i.e.,) any action taken by the system must go through these characteristic models to autonomously make an ideal decision based on the situation. </p> <p><br></p> <p>In this dissertation, we propose novel learning-based models to enhance the decision-making process in autonomous multi-agent systems where agents are able to detect novelties (i.e., unexpected changes in the environment), and adapt to it in a timely manner. For this purpose, we explore two complex and highly dynamic domains </p> <p>(1) Transportation Networks (e.g., Ridesharing application): where we develop AdaPool: a novel distributed diurnal-adaptive decision-making framework for multi-agent autonomous vehicles using model-free deep reinforcement learning and change point detection. (2) Multi-agent games (e.g., Monopoly): for which we propose a hybrid approach that combines deep reinforcement learning (for frequent but complex decisions) with a fixed-policy approach (for infrequent but straightforward decisions) to facilitate decision-making and it is also adaptive to novelties. (3) Further, we present a domain agnostic approach for decision making without prior knowledge in dynamic environments using Bootstrapped DQN. Finally, to enhance security of autonomous multi-agent systems, (4) we develop a machine learning based resilience testing of address randomization moving target defense. Additionally, to further  improve the decision-making process, we present (5) a novel framework for multi-agent deep covering option discovery that is designed to accelerate exploration (which is the first step of decision-making for autonomous agents), by identifying potential collaborative agents and encouraging visiting the under-represented states in their joint observation space. </p>

<p dir="ltr">Unmanned Aerial Vehicles (UAVs) are increasingly applied in agricultural tasks such as crop monitoring, especially with AI-driven enhancements significantly increasing their autonomy and ability to execute complex operations without human interventions. However, existing UAV systems lack efficiency, intuitive user interfaces using natural language processing for command input, and robust security which is essential for real-time operations in dynamic environments. In this paper, we propose a novel solution to create a secure, efficient, and user-friendly interface for UAV control by integrating Large Language Model (LLM) with the case study on agricultural environment. In particular, we designed a four-stage approach that allows only authorized user to issue voice commands to the UAV. The command is issued to the LLM controller processed by LLM using API and generates UAV control code. Additionally, we focus on optimizing UAV battery life and enhancing scene interpretation of the environment. We evaluate our approach using AirSim and an agricultural setting built in Unreal Engine, testing under various conditions, including variable weather and wind factors. Our experimental results confirm our method's effectiveness, demonstrating improved operational efficiency and adaptability in diverse agricultural scenarios.</p>