Different types of decentralized clustering problems have been studied so far for networks and multi-agent systems. In this paper we introduce a new type of a decentralized clustering problem for networks. The so called Decentralized Packet Clustering (DPC) problem is to find for packets that are sent around in a network a clustering. This clustering has to be done by the routers using only few computational power and only a small amount of memory. No direct information transfer between the routers is allowed. We investigate the behavior of new a type of decentralized k-means algorithm — called DPClust — for solving the DPC problem. DPClust has some similarities with ant based clustering algorithms. We investigate the behavior of DPClust for different clustering problems and for networks that consist of several subnetworks. The amount of packet exchange between these subnetworks is limited. Networks with different connection topologies for the subnetworks are considered. A dynamic situation where the packet exchange rates between the subnetworks varies over time is also investigated. The proposed DPC problem leads to interesting research problems for network clustering.

As more industrial processes, transportation and appliances have been automated or equipped with some level of artificial intelligence, the number and scale of interconnected systems has grown in the recent past. This is a development which can be expected to continue and therefore the research in performance of interconnected systems and networks is growing. Due to increased automation and sheer scale of networks, dynamically scaling networks is an increasing field and research into scalable performance measures is advancing. Recently, the notion gamma-robustness, a scalable network performance measure, was introduced as a measurement of interconnected systems robustness with respect to external disturbances. This thesis aims to investigate how the distribution of control effort and cost, within interconnected system, affects network performance, measured with gamma-robustness. Further, we introduce a notion of fairness and a measurement of unfairness in order to quantify the distribution of network properties and performance. With these in place, we also present distributed algorithms with which the distribution of control effort can be controlled in order to achieve a desired network performance. We close with some examples to show the strengths and weaknesses of the presented algorithms. / I och med att fler och fler system och enheter blir utrustade med olika grader av intelligens så växer både förekomsten och omfattningen av sammankopplade system, även kallat Multi-Agent Systems. Sådana system kan vi se exempel på i traffikledningssystem, styrning av elektriska nätverk och fordonståg, vi kan också hitta fler och fler exempel på så kallade sensornätverk i och med att Internet of Things och Industry 4.0 används och utvecklas mer och mer. Det som särskiljer sammankopplade system från mer traditionella system med flera olika styrsignaler och utsignaler är att dem sammankopplade systemen inte styrs från en central styrenhet. Istället styrs dem sammankopplade systemen på ett distribuerat sätt i och med att varje agent styr sig själv och kan även ha individuella mål som den försöker uppfylla. Det här gör att analysen av sammankopplade system försvåras, men tidigare forskning har hittat olika regler och förhållninssätt för agenterna och deras sammankoppling för att uppfylla olika krav, såsom stabilitet och robusthet. Men även om dem sammankopplade systemen är både robusta och stabila så kan dem ha egenskaper som vi vill kunna kontrollera ytterligare. Specifikt kan ett sådant prestandamått vara systemens motståndskraft mot påverkan av yttre störningar och i vanliga olänkade system finns det en inneboende avvägning mellan kostnad på styrsignaler och resiliens mot yttre störningar. Samma avvägning hittar vi i sammankopplade system, men i dessa system hittar vi också ytterligare en dimension på detta problem. I och med att ett visst mått av en nätverksprestanda inte nödvändigtvis betyder att varje agent i nätverket delar samma mått kan agenterna i ett nätverk ha olika utväxling mellan styrsignalskostnad och resiliens mot yttre störningar. Detta gör att vissa agenter kan ha onödigt höga styrsignalskonstander, i den mening att systemen skulle uppnå samma nätverksprestanda men med lägre styrsignalskostnad om flera av agenterna skulle vikta om sina kontrollinsatser. I det här examensarbetet har vi studerat hur olika val av kontrollinsats påverkar ett sammankopplat systems prestanda. Vi har gjort detta för att undersöka hur autonoma, men sammankopplade, agenter kan ändra sin kontrollinsats, men med bibehållen nätverksprestanda, och på det sättet minska sina kontrollkostnader. Detta har bland annat resulterat i en distruberad algoritm för att manipulera agenternas kontrollinsats så att skillnaderna mellan agenternas resiliens mot yttre störningar minskar och nätverksprestandan ökar. Vi avslutar rapporten med att visa ett par exempel på hur system anpassade med hjälp av den framtagna algoritmen får ökad prestanda. Avslutningsvis följer en diskussion kring hur vissa antaganden kring systemstruktur kan släppas upp, samt kring vilka områden framtida forskning skulle kunna fortsätta med.

The financial markets have been started to get decentralized and even distributed. Consumers can now purchase stocks from their home computers without the use of a traditional broker. The dynamism and unpredictability of this domain which is continuously growing in complexity and also the giant volume of information which can affect this market, makes it one of the best potential domains to take advantage of agents. This thesis considers the main concerns of securities e-trading area in order to highlight advantages and disadvantages of multi-agent negotiating systems for online trading of securities comparing to single-agent systems. And then presents a multi-agent system design named MASTNA which considers both decision making and negotiating. The design seeks to improve the main concerns of securities e-trading such as speed, accuracy and handling complexities. MASTNA works over a distributed market and engages different types of agents in order to perform different tasks. For handling the negotiations MASTNA takes advantage of mobile negotiator agents with the purpose of handling parallel negotiations over an unreliable network (Internet).

Deep Reinforcement Learning has achieved a plenty of breakthroughs in the past decade. Motivated by these successes, many publications extend the most prosperous algorithms to multi-agent systems. In this work, we firstly build solid theoretical foundations of Multi-Agent Reinforcement Learning (MARL), along with unified notations. Thereafter, we give a brief review of the most influential algorithms for Single-Agent and Multi-Agent RL. Our attention is focused mainly on Actor-Critic architectures with centralized training and decentralized execution. We propose a new model architec- ture called MATD3-FORK, which is a combination of MATD3 and TD3-FORK. Finally, we provide thorough comparative experiments of these algorithms on various tasks with unified implementation.

The multi-agent systems (MAS) has proven their suitability for implementation of complex software systems. In this work, we have analyzed and designed the data mining MAS by means of role-based organizational model. The organiza- tional model and the model of data mining methods have been formalized in the description logic. By matchmaking which is the main subject of our research, we understand the recommendation of computational agents, i.e. agents encap- sulating some computational method, according their capabilities and previous performances. The matchmaking thus consist of two parts: querying the ontol- ogy model and the meta-learning. Three meta-learning scenarios were tested: optimization in the parameter space, multi-objective optimization of data min- ing processes and method recommendation. A set of experiments in these areas have been performed. 1

No description available.

In various industries, such as manufacturing, logistics, agriculture, defense, search and rescue, and transportation, Multi-robot systems (MRSs) are increasingly gaining popularity. These systems involve multiple robots working together towards a shared objective, either autonomously or under human supervision. However, as MRSs operate in uncertain or even adversarial environments, and the sensors and actuators of each robot may be error-prone, they are susceptible to faults and security threats unique to MRSs. Classical techniques from distributed systems cannot detect or mitigate these threats. In this dissertation, novel techniques are proposed to enhance the security and fault-tolerance of MRSs through inter-robot observations and accusations. A fundamental security property is proposed for MRSs, which ensures that forbidden deviations from a desired multi-robot motion plan by the system supervisor are detected. Relying solely on self-reported motion information from the robots for monitoring deviations can leave the system vulnerable to attacks from a single compromised robot. The concept of co-observations is introduced, which are additional data reported to the supervisor to supplement the self-reported motion information. Co-observation-based detection is formalized as a method of identifying deviations from the expected motion plan based on discrepancies in the sequence of co-observations reported. An optimal deviation-detecting motion planning problem is formulated that achieves all the original application objectives while ensuring that all forbidden plan-deviation attacks trigger co-observation-based detection by the supervisor. A secure motion planner based on constraint solving is proposed as a proof-of-concept to implement the deviation-detecting security property. The security and resilience of MRSs against plan deviation attacks are further improved by limiting the information available to attackers. An efficient algorithm is proposed that verifies the inability of an attacker to stealthily perform forbidden plan deviation attacks with a given motion plan and announcement scheme. Such announcement schemes are referred to as horizon-limiting. An optimal horizon-limiting planning problem is formulated that maximizes planning lookahead while maintaining the announcement scheme as horizon-limiting. Co-observations and horizon-limiting announcements are shown to be efficient and scalable in protecting MRSs, including systems with hundreds of robots, as evidenced by a case study in a warehouse setting. Lastly, the Decentralized Blocklist Protocol (DBP), a method for designing Byzantine-resilient decentralized MRSs, is introduced. DBP is based on inter-robot accusations and allows cooperative robots to identify misbehavior through co-observations and share this information through the network. The method is adaptive to the number of faulty robots and is widely applicable to various decentralized MRS applications. It also permits fast information propagation, requires fewer cooperative observers of application-specific variables, and reduces the worst-case connectivity requirement, making it more scalable than existing methods. Empirical results demonstrate the scalability and effectiveness of DBP in cooperative target tracking, time synchronization, and localization case studies with hundreds of robots. The techniques proposed in this dissertation enhance the security and fault-tolerance of MRSs operating in uncertain and adversarial environments, aiding in the development of secure MRSs for emerging applications.

We investigate an approach to apply Genetic Programming for the evolution of optimal escaping strategies of a team of caribou agents in the wolf-caribou predator prey problem (WCPPP) where the WCPPP is comprised of a team of caribou agents attempting to escape from a single yet superior (in terms of sensory abilities, raw speed, and maximum energy) wolf agent in a simulated twodimensional infinite toroidal world. We empirically verify our hypothesis that the incorporation of empathy in caribou agents significantly improves both the evolution efficiency of the escaping behavior and the effectiveness of such a behavior. This finding may be viewed as a verification of the survival value of empathy and the resulting compassionate behavior of the escaping caribou agents. Moreover, considering the fact that a single caribou cannot escape from the superior wolf, the ability of a team of empathic caribou agents to escape may also be viewed as an illustration of the emergent nature of a successful escaping behavior – in that the team-level properties are more than the mere sum of the properties of the individual entities. Within this context, we also present empirical results that verify the complex (nonlinear) nature of the relationship between the size of team of caribou agents and the efficiency of their escaping behavior. / 博士(工学) / Doctor of Philosophy in Engineering / 同志社大学 / Doshisha University

Policy Hill Climbing (PHC) is a reinforcement learning algorithm that extends Q-learning to learn probabilistic policies for multi-agent games. WoLF-PHC extends PHC with the "win or learn fast" principle. A proof that PHC will diverge in self-play when playing Shapley's game is given, and WoLF-PHC is shown empirically to diverge as well. Various WoLF-PHC based modifications were created, evaluated, and compared in an attempt to obtain convergence to the single shot Nash equilibrium when playing Shapley's game in self-play without using more information than WoLF-PHC uses. Partial Commitment WoLF-PHC (PCWoLF-PHC), which performs best on Shapley's game, is tested on other matrix games and shown to produce satisfactory results.

Satisficing game theory is an alternative to traditional non-cooperative game theory which offers increased flexibility in modeling players' social interactions. However, satisficing players with conflicting attitudes may implement dysfunctional behaviors, leading to poor performance. In this thesis, we present two attempts to "bridge the gap" between satisficing and non-cooperative game theory. First, we present an evolutionary method by which players adapt their attitudes to increase raw payoff, allowing players to overcome dysfunction. We extend the Nash equilibrium concept to satisficing games, showing that the evolutionary method presented leads the players toward an equilibrium in their attitudes. Second, we introduce the conditional utility functions of satisficing theory into an otherwise traditional non-cooperative framework. While the conditional structure allows increased social flexibility in the players' behaviors, players maximize individual utility in the traditional sense, allowing us to apply the Nash equilibrium. We find that, by adjusting players' attitudes, we may alter the Nash equilibria that result.