Current research in multi-agent systems (MAS) has advanced to the development of open MAS, which are characterized by the heterogeneity of agents, free exit/entry and decentralized control. Conflicts of interest among agents are inevitable, and hence automated negotiation to resolve them is one of the promising solutions. This thesis studies three modifications on alternating-offer bargaining protocols for automated negotiation in open MAS. The long-term goal of this research is to design negotiation protocols which can be easily used by intelligent agents in accommodating their need in resolving their conflicts. In particular, we propose three modifications: allowing non-monotonic offers during the bargaining (non-monotonic-offers bargaining protocol), allowing strategic delay (delay-based bargaining protocol), and allowing strategic ignorance to augment argumentation when the bargaining comprises argumentation (ignorance-based argumentation-based negotiation protocol). <p>Utility theory and decision-theoretic approaches are used in the theoretical analysis part, with an aim to prove the benefit of these three modifications in negotiation among myopic agents under uncertainty. Empirical studies by means of computer simulation are conducted in analyzing the cost and benefit of these modifications. Social agents, who use common human bargaining strategies, are the subjects of the simulation. <p>In general, we assume that agents are bounded rational with various degrees of belief and trust toward their opponents. In particular in the study of the non-monotonic-offers bargaining protocol, we assume that our agents have diminishing surplus. We further assume that our agents have increasing surplus in the study of delay-based bargaining protocol. And in the study of ignorance-based argumentation-based negotiation protocol, we assume that agents may have different knowledge and use different ontologies and reasoning engines. <p>Through theoretical analysis under various settings, we show the benefit of allowing these modifications in terms of agents expected surplus. And through simulation, we show the benefit of allowing these modifications in terms of social welfare (total surplus). Several implementation issues are then discussed, and their potential solutions in terms of some additional policies are proposed. Finally, we also suggest some future work which can potentially improve the reliability of these modifications.

Reinforcement learning is a machine learning technique designed to mimic the way animals learn by receiving rewards and punishment. It is designed to train intelligent agents when very little is known about the agent’s environment, and consequently the agent’s designer is unable to hand-craft an appropriate policy. Using reinforcement learning, the agent’s designer can merely give reward to the agent when it does something right, and the algorithm will craft an appropriate policy automatically. In many situations it is desirable to use this technique to train systems of agents (for example, to train robots to play RoboCup soccer in a coordinated fashion). Unfortunately, several significant computational issues occur when using this technique to train systems of agents. This dissertation introduces a suite of techniques that overcome many of these difficulties in various common situations. First, we show how multi-agent reinforcement learning can be made more tractable by forming coalitions out of the agents, and training each coalition separately. Coalitions are formed by using information-theoretic techniques, and we find that by using a coalition-based approach, the computational complexity of reinforcement-learning can be made linear in the total system agent count. Next we look at ways to integrate domain knowledge into the reinforcement learning process, and how this can signifi-cantly improve the policy quality in multi-agent situations. Specifically, we find that integrating domain knowledge into a reinforcement learning process can overcome training data deficiencies and allow the learner to converge to acceptable solutions when lack of training data would have prevented such convergence without domain knowledge. We then show how to train policies over continuous action spaces, which can reduce problem complexity for domains that require continuous action spaces (analog controllers) by eliminating the need to finely discretize the action space. Finally, we look at ways to perform reinforcement learning on modern GPUs and show how by doing this we can tackle significantly larger problems. We find that by offloading some of the RL computation to the GPU, we can achieve almost a 4.5 speedup factor in the total training process.

Research on multi-agent systems performing cooperative tasks has received considerable attention in recent years. Because multiple agents perform cooperative tasks in close proximity, the coordination control of multiple agents to avoid collisions holds one of the critical keys to mission success. The potential function approach has been extensively employed for collision avoidance, but it has one inherent limitation of local minimum. This dissertation proposes a new avoidance strategy for the issue of local minimum. The primary objective of this research is to construct novel potential-function-based control schemes that drive agents from their initial to the goal configurations while avoiding collision with other agents and obstacles. The control schemes enable agents to avoid being trapped at a local minimum by forcing them to exit from the regions that may contain a local minimum. This dissertation consists of three studies, each of which has different technical assumptions. In the first study, all-to-all communication ability among agents is assumed. In addition, each agent is assumed to a priori know the location of all obstacles. In the second study, all-to-all communication ability is again assumed, but each agent is assumed to determine the location of obstacles using a sensor with a limited sensing range. In the third study, limited communication ability is assumed (i.e., each agent exchanges information only with agents within its limited communication range), and each agent is assumed to determine the location of the obstacles using its sensor with a limited sensing range. Relative to existing solutions, the new control schemes presented here have three distinct advantages. First, our avoidance strategy can provide cost-efficient solutions in applications because agents will never be trapped at a local minimum. Second, our control signals are continuous, which allows agents to change their speed in a realistic manner that is consistent with their natural motion traits. Finally, our control scheme allows for setting the upper bound of the velocity of each agent, which guarantees that the speed of agents will never exceed a maximum speed limit. / text

This thesis is to develop a novel consensus algorithm or protocol for multi-agent systems in the event of communication link failure over the network. The structure or topology of the system is modeled by an algebraic graph theory, and defined as a discrete time-invariant system with a second-order dynamics. The communication link failure is governed by a Bernoulli process. Lyapunov-based methodologies and Linear Matrix Inequality (LMI) techniques are then applied to find an appropriate controller gain by satisfying the sufficient conditions of the error dynamics. Therefore, the controller with the calculated gain is guaranteed to drive the system to reach a consensus. Finally, simulation and experiment studies are carried out by using two Mobile Robot Pioneer 3-DXs and one Pioneer 3-AT as a team to verify the proposed work.

Little attention has been paid, in depth, to the relationship between fitness evaluation in evolutionary algorithms and reputation mechanisms in multi-agent systems, but if these could be related it opens the way for implementation of distributed evolutionary systems via multi-agent architectures. Our investigation concentrates on the effectiveness with which social selection, in the form of reputation, can replace direct fitness observation as the selection bias in an evolutionary multi-agent system. We do this in two stages: In the first, we implement a peer-to-peer, adaptive Genetic Algorithm (GA), in which agents act as individual GAs that, in turn, evolve dynamically themselves in real-time, using the traditional evolutionary operators of fitness-based selection, crossover and mutation. In the second stage, we replace the fitness-based selection operator with a reputation-based one, in which agents choose their mates based on the collective past experiences of themselves and their peers. Our investigation shows that this simple model of distributed reputation can be successful as the evolutionary drive in such a system, exhibiting practically identical performance and scalability to direct fitness observation. Further, we discuss the effect of noise (in the form of “defective” agents) in both models. We show that the reputation-based model is significantly better at identifying the defective agents, thus showing an increased level of resistance to noise.

In this research, a multi-agent approach was applied to the design of a large axial flow compressor in order to optimise performance and to greatly enlarge the useful operating range of the machine. In this design a number of distributed software/hardware agents co-operate to control the internal geometry of the machine and thereby optimise the compressor characteristics in response to changes in flow conditions. The resulting machine is termed an ‘Intelligent Geometry Compressor’ (IGC). The design of a multi-agent system for the IGC was carried out in three main phases, each supported by computer simulation. In the first phase a steady-state model of the IGC was developed in which global control of the variable geometry is achieved by a single agent. This was used to help identify specific requirements for performance and the underlying parametric relationships. The subsequent phases incorporated additional agents into the machine design to meet these requirements. Initially, agents were deployed to optimise the settings of individual rows of stator vanes. In the final phase, the MAS was extended to incorporate agents into the machine design for the control of individual stator vanes. Simulation results were obtained which demonstrate the effectiveness of the intelligent geometry compressor in achieving delivery pressure regulation over a wide range of steady-state operating conditions whilst optimising overall machine efficiency and avoiding the occurrence of stall. Some of the implications for the physical design of an IGC arising from the MAS concept were briefly considered. The experience of the research supported by the specific results and observations from many simulation trials, led to the conclusion that multi-agent systems can provide an effective and novel alternative approach to the design of an intelligent geometry compressor. By implication, this conclusion may be extended to other intelligent machine applications where similar opportunity to apply a distributed control solution exists.

Natural disasters result in loss of lives, damage to built facilities, and interruption of businesses. The losses are not instantaneous rather they continue to occur until the community is restored to a functional socio-economic entity. Hence, it is essential that policy makers recognize this dynamic aspect of the incurring losses and make realistic plans to enhance the recovery. However, this cannot take place without understanding how homeowners react to recovery signals. These signals can come in different ways: from policy makers showing their strong commitment to restore the community by providing financial support and/or restoration of lifeline infrastructure; or from the neighbors showing their willingness to reconstruct. The goal of this research is to develop a model that can account for homeowners’ dynamic interactions in both organizational and spatial domains. Spatial domain of interactions focuses on how homeowners process signals from the environment such as neighbors reconstructing and local agencies restoring infrastructure, while organizational domain of interactions focuses on how agents process signals from other stakeholders that do not directly affect the environment like insurers. The hypothesis of this study is that these interactions significantly influence decisions to reconstruct and stay, or sell and leave. A multi-agent framework is used to capture emergent behavior such as spatial patterns and formation of clusters. The developed framework is illustrated and validated using experimental data sets.

Software agents and multi-agent systems (MAS) have grown into a very active area of research and commercial development activity. There are many current emerging real-world applications spanning multitude of diverse domains. In the context of agents, ontology has been widely recognised for their significant benefits to interoperability, reusability, and both development and operational aspects of agent systems and applications. Ontology-based multi-agent systems (OBMAS) exploit these advantages in providing intelligent and semantically aware applications. In addressing the lack of support for ontology in existing methodologies for multi-agent development, this thesis proposes a design methodology for the building of such intelligent multi-agent applications called MOMA. This alternative approach focuses on the development of ontology as the driving force of the development process. By allowing the domain and characteristics of utilisation and experimentation to be dictated through ontology, researchers and domain experts can specify the agent application without any knowledge of agent design and lower level programming. Through the use of a structured ontology model and the use of integrated tools, this approach contributes towards the building of semantically aware intelligent applications for use by researchers and domain experts. MOMA is evaluated through case studies in two different domains: financial services and e-Health.

We investigate the problem of developing a formal language for specifying and reasoning about real-time embedded distributed computer systems. In particular we investigate the problem of developing a theoretical framework for specifying and analyzing different aspects of real-time embedded distributed coordination. In addition to the theoretical framework we also consider the practical aspects of developing real-time embedded distributed systems. (For complete abstract open document)

The cooperative control of large-scale multi-agent systems has gained a significant interest in recent years from the robotics and control communities for multi-vehicle control. One motivator for the growing interest is the application of spatially and temporally distributed multiple unmanned aerial vehicle (UAV) systems for distributed sensing and collaborative operations. In this research, the multi-vehicle control problem is addressed using a decentralised control system. The work aims to provide a decentralised control framework that synthesises the self-organised and coordinated behaviour of natural swarming systems into cooperative UAV systems. The control system design framework is generalised for application into various other multi-agent systems including cellular robotics, ad-hoc communication networks, and modular smart-structures. The approach involves identifying suitable relationships that describe the behaviour of the UAVs within the swarm and the interactions of these behaviours to produce purposeful high-level actions for system operators. A major focus concerning the research involves the development of suitable analytical tools that decomposes the general swarm behaviours to the local vehicle level. The control problem is approached using two-levels of abstraction; the supervisory level, and the local vehicle level. Geometric control techniques based on differential geometry are used at the supervisory level to reduce the control problem to a small set of permutation and size invariant abstract descriptors. The abstract descriptors provide an open-loop optimal state and control trajectory for the collective swarm and are used to describe the intentions of the vehicles. Decentralised optimal control is implemented at the local vehicle level to synthesise self-organised and cooperative behaviour. A deliberative control scheme is implemented at the local vehicle level that demonstrates autonomous, cooperative and optimal behaviour whilst the preserv ing precision and reliability at the local vehicle level.