A Bayesian network approach to self-organization and learning is introduced for use with intelligent agents. Bayesian networks, with the help of influence diagrams, are employed to create a decision-theoretic intelligent agent. Influence diagrams combine both Bayesian networks and utility theory. In this research, an intelligent agent is modeled by its belief, preference, and capabilities attributes. Each agent is assumed to have its own belief about its environment. The belief aspect of the intelligent agent is accomplished by a Bayesian network. The goal of an intelligent agent is said to be the preference of the agent and is represented with a utility function in the decision theoretic intelligent agent. Capabilities are represented with a set of possible actions of the decision-theoretic intelligent agent. Influence diagrams have utility nodes and decision nodes to handle the preference and capabilities of the decision-theoretic intelligent agent, respectively. Learning is accomplished by Bayesian networks in the decision-theoretic intelligent agent. Bayesian network learning methods are discussed intensively in this paper. Because intelligent agents will explore and learn the environment, the learning algorithm should be implemented online. None of the existent Bayesian network learning algorithms has online learning. Thus, an online Bayesian network learning method is proposed to allow the intelligent agent learn during its exploration. Self-organization of the intelligent agents is accomplished because each agent models other agents by observing their behavior. Agents have belief, not only about environment, but also about other agents. Therefore, an agent takes its decisions according to the model of the environment and the model of the other agents. Even though each agent acts independently, they take the other agents behaviors into account to make a decision. This permits the agents to organize themselves for a common task. To test the proposed intelligent agent's learning and self-organizing abilities, Windows application software is written to simulate multi-agent systems. The software, IntelliAgent, lets the user design decision-theoretic intelligent agents both manually and automatically. The software can also be used for knowledge discovery by employing Bayesian network learning a database. Additionally, we have explored a well-known herding problem to obtain sound results for our intelligent agent design. In the problem, a dog tries to herd a sheep to a certain location, i.e. a pen. The sheep tries to avoid the dog by retreating from the dog. The herding problem is simulated using the IntelliAgent software. Simulations provided good results in terms of the dog's learning ability and its ability to organize its actions according to the sheep's (other agent) behavior. In summary, a decision-theoretic approach is applied to the self-organization and learning problems in intelligent agents. Software was written to simulate the learning and self-organization abilities of the proposed agent design. A user manual for the software and the simulation results are presented. This research is supported by the Office of Naval Research with the grant number N00014-98-1-0779. Their financial support is greatly appreciated. / Ph. D.

Modeling or simulating Complex Adaptive Systems (CASs) is both important and challenging. As the name suggests, CASs are systems consisting of large numbers of interacting adaptive compartments. They are studied across a wide range of disciplines and have unique properties. They model such systems as multicellular organisms, ecosystems, social networks, and many more. They are complex, in the sense that they are dynamical, nonlinear, and heterogeneous systems that cannot be simply scaled up/down. However, they are self-organized, in the sense that they can evolve into specific structures/patterns without guidance from outside sources. Modeling/Simulating CASs is challenging, not only because of the high complexity, but also because of the difficulty in explaining the underlying mechanism behind self-organization. The goal of this research is to provide a modeling framework as well as a simulation platform to advance the study of CASs. We argue that there are common principles behind self-organization processes of different systems across different domains. We explore, analyze, and perform experiments into these principles. We propose and implement modeling templates such as short-term and long-term adaptivity. We incorporate techniques from systems theory, employing computing paradigms, including multi-agent system and asynchronous message passing. We also consider an application from the biological domain to model and simulate under our framework, treating it as a CAS for validation purposes. / Ph. D.

The air transportation system has grown significantly during the past few decades. The demand for air travel has increased tremendously as compared to the increase in the supply. The air transportation system can be divided into four subsystems: airports, airlines, air traffic control, and passengers, each of them having different interests. These subsystems interact in a very complex way resulting in various phenomena. On the airport side, there is excessive flight demand during the peak hours that frequently exceeds the airport capacity resulting in serious flight delays. These delays incur costs to the airport, passengers, and airlines. The air traffic pattern is also affected by the characteristics of the air transportation network. The current network structure of most major airlines in United States is a hub-and-spoke network. The airports are interested in reducing congestion, especially during the peak time. The airlines act as direct demand to the airport and as the supplier to the passengers. They sometimes compete with other airlines on certain routes and sometimes they collaborate to maximize revenue. The flight schedule of airlines directly affects the travel demand. The flight schedule that minimizes the schedule delay of passengers in directed and connected flights will attract more passengers. The important factors affecting the airline revenue include ticket price, departure times, frequency, and aircraft type operated on each route. The revenue generated from airline depends also on the behavior of competing airlines, and their flight schedules. The passengers choose their flight based on preferred departure times, offered ticket prices, and willingness of airlines to minimize delay and cost. Hence, all subsystems of air transportation system are inter-connected to each other, meaning, strategy of each subsystem directly affects the performance of other subsystems. This interaction between the subsystems makes it more difficult to analyze the air transportation system. Traditionally, analytical top-down approach has been used to analyze the air transportation problem. In top-down approach, a set of objectives is defined and each subsystem is fixed in the overall scheme. On the other hand, in a bottom-up approach, many issues are addressed simultaneously and each individual system has greater autonomy to make decisions, communicate and to interact with one another to achieve their goals when considering complex air transportation system. Therefore, it seems more appropriate to approach the complex air traffic congestion and airline competition problems using a bottom-up approach. In this research, an agent-based model for the air transportation system has been developed. The developed model considers each subsystem as an independent type of agent that acts based on its local knowledge and its interaction with other agents. The focus of this research is to analyze air traffic congestion and airline competition in a hub-and-spoke network. The simulation model developed is based on evolutionary computation. It seems that the only way for analyzing emergent phenomenon (such as air traffic congestion) is through the development of simulation models that can simulate the behavior of each agent. In the agent-based model developed in this research, agents that represent airports can increase capacity or significantly change landing fee policy, while the agents that represent airlines learn all the time, change their markets, fare structure, flight frequencies, and flight schedules. Such a bottom-up approach facilitates a better understanding of the complex nature of congestion and gains more insights into the competition in air transportation, hence making it easier to understand, predict and control the overall performance of the complex air transportation system. / Ph. D.

As exploration in space increases through the use of larger telescopes, more sophisticated structures, and physical exploration, the use of autonomous robots will become instrumental to build and maintain the infrastructures required for this exploration. These systems must be autonomous to deal with the infeasibility of teleoperation due signal delay and task complexity. The reality of using robots in the real world without direct human input will require the autonomous systems to have the capability of responding to errors that occur in an assembly scenario on their own. As such, a system must be in place to allow for the sequencing and allocation of tasks to the robotic workforce autonomously, giving the ability to re-plan in real world stochastic environments. This work presents four contributions towards a system allowing for the autonomous sequencing and allocation of tasks for in-space assembly problems. The first contribution is the development of the Stochastic Assembly Problem Definition (SAPD) to articulate all of the features in an assembly problem that are applicable to the task sequencing and allocation. The second contribution is the formulation of a mixed integer program to solve for assembly schedules that are optimal or a quantifiable measurement from optimal. This contribution is expanded through the development of a genetic algorithm formulation to utilize the stochastic information present in the assembly problem. This formulation extends the state-of-the-art techniques in genetic algorithms to allow for the inclusion of new constraints required for the in-space assembly domain. The third contribution addresses how to estimate a robot's ability to complete a task if the robot must be assigned to a task it was previously not expected to work on. This is accomplished through the development of four metrics and analyzed through the use of screw theory kinematics. The final contribution focuses on a set of metrics to guide the selection of a good scheduling method for different assembly situations. The experiments in this work demonstrate how the developed theory can be utilized and shows the scheduling systems producing the best or close to the best schedules for assemblies. It also shows how the metrics used to quantify and estimate robot ability are applied. The theory developed in this work provides another step towards autonomous systems that are capable of assembling structures in-space without the need for human input. / Doctor of Philosophy / As space exploration continues to progress, autonomous robots are needed to allow for the necessary structures to be built in-space, on Mars, and on the Lunar surface. Since it is not possible to plan for every possible thing that could go wrong or break, the robots must be able to figure out how to build and repair structures without human input. The work presented here develops a framework that allows this in-space assembly problem to be framed in a way the robots can process. It then provides a method for generating assembly schedules that describe very good, if not the best way to complete the assembly quickly while still taking into account randomness that may be present. Additionally, this work develops a way to quantify and estimate how good robots will be at a task they have not attempted before. Finally, a set of considerations are proposed to aid in determining what scheduling method will work best for different assembly scenarios. The experiments in this work demonstrate how the developed theory can be used and shows the scheduling systems producing the best or close to the best schedules for assemblies. It also shows how the methods used to define robot ability are applied. The work developed here provides another step towards autonomous systems that are capable of assembling structures in-space without the need for human input.

We introduce a framework that would enable using autonomous aerial vehicles in search and rescue scenarios associated with missing person incidents to assist human searchers. We formulate a lost person behavior model and a human searcher model informed by data collected from past search missions. These models are used to generate a probabilistic heatmap of the lost person's position and anticipated searcher trajectories. We use Gaussian processes with a Gibbs' kernel for data fusion to accurately model a limited field-of-view sensor. Our algorithm thereby computes a set of trajectories for a team of aerial vehicles to autonomously navigate, so as to assist and complement human searchers' efforts. / Master of Science / Our goal is to assist human searchers using autonomous aerial vehicles in search and rescue scenarios associated with missing person incidents. We formulate a lost person behavior model and a human searcher model informed by data collected from past search missions. These models are used to generate a probabilistic heatmap of the lost person’s position and anticipated searcher trajectories. We use Gaussian processes for data fusion with Gibbs’ kernel to accurately model a limited field-of-view sensor. Our algorithm thereby computes a set of trajectories for a team of aerial vehicles to autonomously navigate, so as to assist and complement human searchers’ efforts.

The efficiency, security and resiliency are very important factors for the operation of a distribution power system. Taking into account customer demand and energy resource constraints, electric utilities not only need to provide reliable services but also need to operate a power grid as efficiently as possible. The objective of this dissertation is to design, develop and deploy the Multi-Agent Systems (MAS) - together with control algorithms - that enable demand response (DR) implementation at the customer level, focusing on both residential and commercial customers. For residential applications, the main objective is to propose an approach for a smart distribution transformer management. The DR objective at a distribution transformer is to ensure that the instantaneous power demand at a distribution transformer is kept below a certain demand limit while impacts of demand restrike are minimized. The DR objectives at residential homes are to secure critical loads, mitigate occupant comfort violation, and minimize appliance run-time after a DR event. For commercial applications, the goal is to propose a MAS architecture and platform that help facilitate the implementation of a Critical Peak Pricing (CPP) program. Main objectives of the proposed DR algorithm are to minimize power demand and energy consumption during a period that a CPP event is called out, to minimize occupant comfort violation, to minimize impacts of demand restrike after a CPP event, as well as to control the device operation to avoid restrikes. Overall, this study provides an insight into the design and implementation of MAS, together with associated control algorithms for DR implementation in smart buildings. The proposed approaches can serve as alternative solutions to the current practices of electric utilities to engage end-use customers to participate in DR programs where occupancy level, tenant comfort condition and preference, as well as controllable devices and sensors are taken into account in both simulated and real-world environments. Research findings show that the proposed DR algorithms can perform effectively and efficiently during a DR event in residential homes and during the CPP event in commercial buildings. / Ph. D.

This work addresses applications of search theory where a mobile search agent seeks to find an unknown number of stationary targets randomly distributed in a bounded search domain. We assume that the search mission is subject to a time or distance constraint, and that the local environmental conditions affect sensor performance. Because the environment varies by location, the effectiveness of the search sensor also varies by location. Our contribution to search theory includes new decision-theoretic approaches for generating optimal search plans in the presence of false alarms and uncertain environmental variability. We also formally define the value of environmental information for improving the effectiveness of a search mission, and we develop methods for optimal deployment of assets that can acquire environmental information in order to improve search effectiveness. Finally, we extend our research to the case of multiple cooperating search agents. For the case that inter-agent communication is severely bandwidth-limited, such as in subsea applications, we propose a method for assessing the expected value of inter-agent communication relative to joint search effectiveness. Our results lead to a method for determining when search agents should communicate. Our contributions to search theory address important applications that range from subsea mine-hunting to post-disaster search and rescue applications. / PHD / We address search applications where a mobile search agent seeks to find an unknown number of stationary targets randomly distributed in a bounded search domain. The search agent is equipped with a search sensor that detects the targets at a location. Sensor measurements are often imperfect due to possible missed detections and false alarms. We also consider that the local environmental conditions affect the quality of the data acquired from the search sensor. For instance, if we are searching for a target that has a rocky shape, we expect that it will be harder to find that target in a rocky environment. We consider that the search mission is subject to a time or distance constraint, and thus, search can be performed on only a subset of locations. Our goal in this study is to formally determine where to acquire the search measurements so that the search effectiveness can be maximized. We also formally define the value of acquiring environmental information for improving the effectiveness of a search mission, and we develop methods for optimal deployment of assets that can acquire environmental information in order to improve search effectiveness. Finally, we address the cases where multiple search assets collaboratively search the environment and they can communicate their local information with each other. We are particularly interested in determining when a vehicle should communicate with another vehicle so that the joint search effectiveness can be improved. Our contributions to search theory address important applications that range from subsea mine-hunting to post-disaster search and rescue applications.

This dissertation addresses a fundamental question of multi-agent coordination: what infor- mation should be sent to whom and when, with the limited resources available to each agent? Communication requirements for multi-agent systems can be rather high when an accurate picture of the environment and the state of other agents must be maintained. To reduce the impact of multi-agent coordination on networked systems, e.g., power and bandwidth, this dissertation introduces new concepts to enable Intelligent Knowledge Distribution (IKD), including Constrained-action POMDPs (CA-POMDP) and concurrent decentralized (CoDec) POMDPs for an agnostic plug-and-play capability for fully autonomous systems. Each agent runs a CoDec POMDP where all the decision making (motion planning, task allocation, asset monitoring, and communication) are separated into concurrent individual MDPs to reduce the combinatorial explosion of the action and state space while maintaining dependencies between the models. We also introduce the CA-POMDP with action-based constraints on partially observable Markov decision processes, rewards driven by the value of information, and probabilistic constraint satisfaction through discrete optimization and Markov chain Monte Carlo analysis. IKD is adapted real-time through machine learning of the actual environmental impacts on the behavior of the system, including collaboration strategies between autonomous agents, the true value of information between heterogeneous systems, observation probabilities and resource utilization. / Doctor of Philosophy / This dissertation addresses a fundamental question behind when multiple autonomous sys- tems, like drone swarms, in the field need to coordinate and share data: what information should be sent to whom and when, with the limited resources available to each agent? Intelligent Knowledge Distribution is a framework that answers these questions. Communication requirements for multi-agent systems can be rather high when an accurate picture of the environment and the state of other agents must be maintained. To reduce the impact of multi-agent coordination on networked systems, e.g., power and bandwidth, this dissertation introduces new concepts to enable Intelligent Knowledge Distribution (IKD), including Constrained-action POMDPs and concurrent decentralized (CoDec) POMDPs for an agnostic plug-and-play capability for fully autonomous systems. The IKD model was able to demonstrate its validity as a "plug-and-play" library that manages communications between agents that ensures the right information is being transmitted at the right time to the right agent to ensure mission success.

Optimizing connected automated vehicle movements through roadway intersections is a challenging problem. Traditional traffic control strategies, such as traffic signals are not optimal, especially for heavy traffic. Alternatively, centralized automated vehicle control strategies are costly and not scalable given that the ability of a central controller to track and schedule the movement of hundreds of vehicles in real-time is highly questionable. In this research, a series of fully distributed heuristic algorithms are proposed where vehicles in the vicinity of an intersection continuously cooperate with each other to develop a schedule that allows them to safely proceed through the intersection while incurring minimum delays. An algorithm is proposed for the case of an isolated intersection then a number of algorithms are proposed for a network of intersections where neighboring intersections communicate directly or indirectly to help the distributed control at each intersection makes a better estimation of traffic in the whole network. An algorithm based on the Godunov scheme outperformed optimized signalized control. The simulated experiments show significant reductions in the average delay. The base algorithm is successfully added to the INTEGRATION micro-simulation model and the results demonstrate improvements in delay, fuel consumption, and emissions when compared to roundabout, signalized, and stop sign controlled intersections. The study also shows the capability of the proposed technique to favor emergency vehicles, producing significant increases in mobility with minimum delays to the other vehicles in the network. / Ph. D.

Unfavorable environmental and (or) human displacement may engender the need for Search and Rescue (SAR). Challenges such as inaccessibility, large search areas, and heavy reliance on available responder count, limited equipment and training makes SAR a challenging problem. Additionally, SAR operations also pose significant risk to involved responders. This opens a remarkable opportunity for robotic systems to assist and augment human understanding of the harsh environments. A large body of work exists on the introduction of ground and aerial robots in visual and temporal inspection of search areas with varying levels of autonomy. Unfortunately, limited autonomy is the norm in such systems, due to the limitations presented by on-board UAV resources and networking capabilities. In this work we propose a new multi-agent approach to SAR and introduce a wearable compute cluster in the form factor of a backpack. The backpack allows offloading compute intensive tasks such as Lost Person Behavior Modelling, Path Planning and Deep Neural Network based computer vision applications away from the UAVs and offers significantly high performance computers to execute them. The backpack also provides for a strong networking backbone and task orchestrators which allow for enhanced coordination and resource sharing among all the agents in the system. On the basis of our benchmarking experiments, we observe that the backpack can significantly boost capabilities and success in modern SAR responses. / Master of Science / Unfavorable environmental and (or) human displacement may engender the need for Search and Rescue (SAR). Challenges such as inaccessibility, large search areas, and heavy reliance on available responder count, limited equipment and training makes SAR a challenging problem. Additionally, SAR operations also pose significant risk to involved responders. This opens a remarkable opportunity for robotic systems to assist and augment human understanding of the harsh environments. A large body of work exists on the introduction of ground and aerial robots in visual and temporal inspection of search areas with varying levels of autonomy. Unfortunately, limited autonomy is the norm in such systems, due to the limitations presented by on-board UAV resources and networking capabilities. In this work we propose a new multi-agent approach to SAR and introduce a wearable compute cluster in the form factor of a backpack. The backpack allows offloading compute intensive tasks such as Lost Person Behavior Modelling, Path Planning and Deep Neural Network based computer vision applications away from the UAVs and offers significantly high performance computers to execute them. The backpack also provides for a strong networking backbone and task orchestrators which allow for enhanced coordination and resource sharing among all the agents in the system. On the basis of our benchmarking experiments, we observe that the backpack can significantly boost capabilities and success in modern SAR responses.