<p dir="ltr">The original definition of Nash Equilibrium applied to normal form games, but the notion has now been extended to various other forms of games including leader-follower games (Stackelberg games), extensive form games, stochastic games, games of incomplete information, cooperative games, and so on. We focus on general-sum stochastic Stackelberg games in this work. An example where such games would be natural to consider is in security games where a defender wishes to protect some targets through deployment of limited resources and an attacker wishes to strategically attack the targets to benefit themselves. The hierarchical order of play arises naturally since the defender typically acts first and deploys a strategy, while the attacker observes the strategy ofthe defender before attacking. Another example where this framework fits is in testing during epidemics, where the leader (the government) sets testing policies and the follower (the citizens) decide at every time step whether to get tested. The government wishes to minimize the number of infected people in the population while the follower wishes to minimize the cost of getting sick and testing. This thesis presents a learning algorithm for players to converge to their stationary policies in a general sum stochastic sequential Stackelberg game. The algorithm is a two time scale implicit policy gradient algorithm that provably converges to stationary points of the optimization problems of the two players. Our analysis allows us to move beyond the assumptions of zero-sum or static Stackelberg games made in the existing literature for learning algorithms to converge.</p><p dir="ltr"><br></p>

The BDI agent architecture includes a plan library containing pre-de ned plans. The plan library is included in the agent architecture to reduce the need for expensive means-end reasoning, however can hinder the agent's e ectiveness when operating in a changing environment. Existing research on integrating di erent planning methods into the BDI agent to overcome this limitation include HTNs, state-space planning and Graphplan. Genetic Algorithms (GAs) have not yet been used for this purpose. This dissertation investigates the feasibility of using GAs as a plan modi cation mechanism for BDI agents. It covers the design of a plan structure that can be encoded into a binary string, which can be operated on by the genetic operators. The e ectiveness of the agent in a changing environment is compared to an agent without the GA plan modification mechanism. The dissertation shows that GAs are a feasible plan modification mechanism for BDI agents. / Information Science

Made available in DSpace on 2015-03-05T13:56:58Z (GMT). No. of bitstreams: 0 Previous issue date: 30 / Nenhuma / Este trabalho apresenta um modelo de percepção de dados não visíveis (por exemplo, olfativas e auditivas) para humanos virtuais baseado em informações puramente geométricas existentes em ambientes virtuais. Esta proposta procurou contextualizar vários tópicos de pesquisa no que diz respeito a agentes autônomos em ambientes virtuais e suas capacidades de percepção. Procurou-se, através da área da visão sintética, encontrar métodos que pudessem ser generalizados para que outros sentidos pudessem ser simulados. O objetivo principal deste modelo é possibilitar que o agente possa identificar diferentes ocorrências de eventos dentro de um ambiente virtual, no que diz respeito às informações não necessariamente visuais, como por exemplo, a presença de cheiros ou barulhos. Para o modelo, foram buscados métodos que proporcionem um bom desempenho computacional e que sejam facilmente generalizados e implementados de maneira a prover métodos de percepção a multidões de humanos virtuais / This work presents a perception data model which deals with not visible information (such as hearing and smelling). This model is based on geometric information that exists in the virtual environment. This proposal also identifies several research topics related to autonomous agents in virtual environments and their perception abilities. Investigations in synthetic vision area allows to find methods that could be generalized in order to be used for other sensors. Therefore, the main objective of this model is to allow to the agents the possibility of identifying different events, into a virtual environment, not necessarily visual data. For instance, the presence of smell and noise. In addition, we propose methods which aims to provide a good computational performance and which can be generalized and implemented in a way that they can provide the perception methods to virtual human beings.

<p>Autonomous agent systems are systems in which many simple entities, called “agents”, interact with each other. The behaviour resulting from such interactions can be much more complex than that of the individual agents. A group of interacting agents can for example accomplish tasks that no single agent could.</p> <p>Nature provides several examples of autonomous agent systems, such as flocks of birds and insects, schools of fish, and anthills. Progresses in robotics, electronics and telecommunications make it now also possible to design such systems in order to accomplish particular tasks, such as the surveillance or exploration of areas, or the maintenance of some environments.</p> <p>In this thesis, we analyze two issues related to autonomous agent systems, and more precisely, to the influence of the inter-agent communication network on the system behaviour. In a first part, we consider the problem of preserving the shape of a multi-agent formation by explicitly maintaining the distances between some agents constant. We study the case of distance constraints that are unilateral, that is, constraints for which the responsibility is given to a one of the two agents concerned. This leads to the notions of persistence and constraint consistence. The second part is devoted to the consensus problems: agents have a value which they update by averaging that of other agents. Eventually, all agents may obtain a common value, in which case we say that the system reaches a consensus. One major difficulty in the study of such system is the possible dependence of the interaction and communication topology on the values of the agents. We study two paradigmatic systems in which this dependence can be taken into account, and obtain results on their convergence and on the stability of their equlibria.</p>

The recent technological advances in the field of autonomous vehicles have resulted in a growing impetus for researchers to improve the current framework of mission planning and execution within both the military and civilian contexts. Many recent efforts towards this direction emphasize the importance of replacing the so-called monolithic paradigm, where a mission is planned, monitored, and controlled by a unique global decision maker, with a network centric paradigm, where the same mission related tasks are performed by networks of interacting decision makers (autonomous vehicles). The interest in applications involving teams of autonomous vehicles is expected to significantly grow in the near future as new paradigms for their use are constantly being proposed for a diverse spectrum of real world applications. One promising approach to extend available techniques for addressing problems involving a single autonomous vehicle to those involving teams of autonomous vehicles is to use the concept of Voronoi diagram as a means for reducing the complexity of the multi-vehicle problem. In particular, the Voronoi diagram provides a spatial partition of the environment the team of vehicles operate in, where each element of this partition is associated with a unique vehicle from the team. The partition induces, in turn, a graph abstraction of the operating space that is in a one-to-one correspondence with the network abstraction of the team of autonomous vehicles; a fact that can provide both conceptual and analytical advantages during mission planning and execution. In this dissertation, we propose the use of a new class of Voronoi-like partitioning schemes with respect to state-dependent proximity (pseudo-) metrics rather than the Euclidean distance or other generalized distance functions, which are typically used in the literature. An important nuance here is that, in contrast to the Euclidean distance, state-dependent metrics can succinctly capture system theoretic features of each vehicle from the team (e.g., vehicle kinematics), as well as the environment-vehicle interactions, which are induced, for example, by local winds/currents. We subsequently illustrate how the proposed concept of state-dependent Voronoi-like partition can induce local control schemes for problems involving networks of spatially distributed autonomous vehicles by examining different application scenarios.

<p>This master thesis investigates the principal capability of artificial evolution to produce tool use behavior in adaptive agents, excluding the application of life-time learning or adaptation mechanisms. Tool use is one aspect of complex behavior that is expected from autonomous agents acting in real-world environments. In order to achieve tool use behavior an agent needs to identify environmental objects as potential tools before it can use the tools in a problem-solving task. Up to now research in robotics has focused on life-time learning mechanisms in order to achieve this. However, these techniques impose great demands on resources, e.g. in terms of memory or computational power. All of them have shown limited results with respect to a general adaptivity. One might argue that even nature does not present any kind of omni-adaptive agent. While humans seem to be a good example of natural agents that master an impressive variety of life conditions and environments (at least from a human perspective, other examples are spectacular survivability observations of octopuses, scorpions or various viruses) even the most advanced engineering approaches can hardly compete with the simplest life-forms in terms of adaptation. This thesis tries to contribute to engineering approaches by promoting the application of artificial evolution as a complementing element with the presentation of successful pioneering experiments. The results of these experiments show that artificial evolution is indeed capable to render tool use behavior at different levels of complexity and shows that the application of artificial evolution might be a good complement to life-time approaches in order to create agents that are able to implicitly extract concepts and display tool use behavior. The author believes that off-loading at least parts of the concept retrieval process to artificial evolution will reduce resource efforts at life-time when creating autonomous agents with complex behavior such as tool use. This might be a first step towards the vision of a higher level of autonomy and adaptability. Moreover, it shows the demand for an experimental verification of commonly accepted limits between qualities of learned and evolved tool use capabilities.</p>

The BDI agent architecture includes a plan library containing pre-defined plans. The plan library is included in the agent architecture to reduce the need for expensive means-end reasoning, however can hinder the agent’s effectiveness when operating in a changing environment. Existing research on integrating different planning methods into the BDI agent to overcome this limitation include HTNs, state-space planning and Graphplan. Genetic Algorithms (GAs) have not yet been used for this purpose. This dissertation investigates the feasibility of using GAs as a plan modification mechanism for BDI agents. It covers the design of a plan structure that can be encoded into a binary string, which can be operated on by the genetic operators. The effectiveness of the agent in a changing environment is compared to an agent without the GA plan modification mechanism. The dissertation shows that GAs are a feasible plan modification mechanism for BDI agents. / Information Science

Este trabalho investiga a aplicação de rede de comportamentos estendidas ao domínio de jogos de computador. Redes de comportamentos estendidas (RCE) são uma classe de arquiteturas para seleção de ações capazes de selecionar bons conjuntos de ações para agentes complexos situados em ambientes contínuos e dinâmicos. Foram aplicadas com sucesso na Robocup, mas nunca foram aplicadas a jogos. PHISH-Nets, um modelo de redes de comportamentos capaz de selecionar apenas uma ação por vez, foi aplicado à modelagem de personagens, com bons resultados. Apesar de RCEs serem aplicáveis a um conjunto de domínios maior, nunca foram usadas para modelagem de personagens. Apresenta-se como projetar um agente controlado por uma rede de comportamentos para o domínio do Unreal Tournament e como integrar a rede de comportamentos a sensores nebulosos e comportamentos baseados em máquinas de estado-finito aumentadas. Investiga-se a qualidade da seleção de ações e a correção do mecanismo em uma série de experimentos. A performance é medida através da comparação das pontuações de um agente baseado em redes de comportamentos com outros dois agentes. Um dos agentes foi implementado por outro grupo e usava sensores, efetores e comportamentos diferentes. O outro agente era idêntico ao agente baseado em RCEs, exceto pelo mecanismo de controle empregado. A modelagem de personalidade é investigada através do projeto e análise de cinco estereótipos: Samurai, Veterano, Berserker, Novato e Covarde. Apresenta-se três maneiras de construir personalidades e situa-se este trabalho dentro de outras abordagems de projeto de personalidades. Conclui-se que a rede de comportamentos estendida é um bom mecanismo de seleção de ações para o domínio de jogos de computador e um mecanismo interessante para a construção de agentes com personalidades simples. / This work investigates the application of extended behavior networks to the computer game domain. We use as our test bed the game Unreal Tournament. Extended Behavior Networks (EBNs) are a class of action selection architectures capable of selecting a good set of actions for complex agents situated in continuous and dynamic environments. They have been successfully applied to the Robocup, but never before used in computer games. PHISH-Nets, a behavior network model capable of selecting just single actions, was applied to character modeling with promising results. Although extended behavior networks are applicable to a larger domain, they had not been used to character modeling before. We present how to design an agent with extended behavior networks, fuzzy sensors and finite-state machine based behaviors. We investigate the quality of the action selection mechanism and its correctness in a series of experiments. The performance is assessed comparing the scores of an agent using an extended behavior network against a plain reactive agent with identical sensory-motor apparatus and against a totally different agent built around finite-state machines. We investigate how EBNs fare on agent personality modeling via the design and analysis of five stereotypes in Unreal Tournament. We discuss three ways to build character personas and situate our work within other approaches. We conclude that extended behavior networks are a good action selection architecture for the computer game domain and an interesting mechanism to build agents with simple personalities.

Este trabalho investiga a aplicação de rede de comportamentos estendidas ao domínio de jogos de computador. Redes de comportamentos estendidas (RCE) são uma classe de arquiteturas para seleção de ações capazes de selecionar bons conjuntos de ações para agentes complexos situados em ambientes contínuos e dinâmicos. Foram aplicadas com sucesso na Robocup, mas nunca foram aplicadas a jogos. PHISH-Nets, um modelo de redes de comportamentos capaz de selecionar apenas uma ação por vez, foi aplicado à modelagem de personagens, com bons resultados. Apesar de RCEs serem aplicáveis a um conjunto de domínios maior, nunca foram usadas para modelagem de personagens. Apresenta-se como projetar um agente controlado por uma rede de comportamentos para o domínio do Unreal Tournament e como integrar a rede de comportamentos a sensores nebulosos e comportamentos baseados em máquinas de estado-finito aumentadas. Investiga-se a qualidade da seleção de ações e a correção do mecanismo em uma série de experimentos. A performance é medida através da comparação das pontuações de um agente baseado em redes de comportamentos com outros dois agentes. Um dos agentes foi implementado por outro grupo e usava sensores, efetores e comportamentos diferentes. O outro agente era idêntico ao agente baseado em RCEs, exceto pelo mecanismo de controle empregado. A modelagem de personalidade é investigada através do projeto e análise de cinco estereótipos: Samurai, Veterano, Berserker, Novato e Covarde. Apresenta-se três maneiras de construir personalidades e situa-se este trabalho dentro de outras abordagems de projeto de personalidades. Conclui-se que a rede de comportamentos estendida é um bom mecanismo de seleção de ações para o domínio de jogos de computador e um mecanismo interessante para a construção de agentes com personalidades simples. / This work investigates the application of extended behavior networks to the computer game domain. We use as our test bed the game Unreal Tournament. Extended Behavior Networks (EBNs) are a class of action selection architectures capable of selecting a good set of actions for complex agents situated in continuous and dynamic environments. They have been successfully applied to the Robocup, but never before used in computer games. PHISH-Nets, a behavior network model capable of selecting just single actions, was applied to character modeling with promising results. Although extended behavior networks are applicable to a larger domain, they had not been used to character modeling before. We present how to design an agent with extended behavior networks, fuzzy sensors and finite-state machine based behaviors. We investigate the quality of the action selection mechanism and its correctness in a series of experiments. The performance is assessed comparing the scores of an agent using an extended behavior network against a plain reactive agent with identical sensory-motor apparatus and against a totally different agent built around finite-state machines. We investigate how EBNs fare on agent personality modeling via the design and analysis of five stereotypes in Unreal Tournament. We discuss three ways to build character personas and situate our work within other approaches. We conclude that extended behavior networks are a good action selection architecture for the computer game domain and an interesting mechanism to build agents with simple personalities.

Este trabalho investiga a aplicação de rede de comportamentos estendidas ao domínio de jogos de computador. Redes de comportamentos estendidas (RCE) são uma classe de arquiteturas para seleção de ações capazes de selecionar bons conjuntos de ações para agentes complexos situados em ambientes contínuos e dinâmicos. Foram aplicadas com sucesso na Robocup, mas nunca foram aplicadas a jogos. PHISH-Nets, um modelo de redes de comportamentos capaz de selecionar apenas uma ação por vez, foi aplicado à modelagem de personagens, com bons resultados. Apesar de RCEs serem aplicáveis a um conjunto de domínios maior, nunca foram usadas para modelagem de personagens. Apresenta-se como projetar um agente controlado por uma rede de comportamentos para o domínio do Unreal Tournament e como integrar a rede de comportamentos a sensores nebulosos e comportamentos baseados em máquinas de estado-finito aumentadas. Investiga-se a qualidade da seleção de ações e a correção do mecanismo em uma série de experimentos. A performance é medida através da comparação das pontuações de um agente baseado em redes de comportamentos com outros dois agentes. Um dos agentes foi implementado por outro grupo e usava sensores, efetores e comportamentos diferentes. O outro agente era idêntico ao agente baseado em RCEs, exceto pelo mecanismo de controle empregado. A modelagem de personalidade é investigada através do projeto e análise de cinco estereótipos: Samurai, Veterano, Berserker, Novato e Covarde. Apresenta-se três maneiras de construir personalidades e situa-se este trabalho dentro de outras abordagems de projeto de personalidades. Conclui-se que a rede de comportamentos estendida é um bom mecanismo de seleção de ações para o domínio de jogos de computador e um mecanismo interessante para a construção de agentes com personalidades simples. / This work investigates the application of extended behavior networks to the computer game domain. We use as our test bed the game Unreal Tournament. Extended Behavior Networks (EBNs) are a class of action selection architectures capable of selecting a good set of actions for complex agents situated in continuous and dynamic environments. They have been successfully applied to the Robocup, but never before used in computer games. PHISH-Nets, a behavior network model capable of selecting just single actions, was applied to character modeling with promising results. Although extended behavior networks are applicable to a larger domain, they had not been used to character modeling before. We present how to design an agent with extended behavior networks, fuzzy sensors and finite-state machine based behaviors. We investigate the quality of the action selection mechanism and its correctness in a series of experiments. The performance is assessed comparing the scores of an agent using an extended behavior network against a plain reactive agent with identical sensory-motor apparatus and against a totally different agent built around finite-state machines. We investigate how EBNs fare on agent personality modeling via the design and analysis of five stereotypes in Unreal Tournament. We discuss three ways to build character personas and situate our work within other approaches. We conclude that extended behavior networks are a good action selection architecture for the computer game domain and an interesting mechanism to build agents with simple personalities.