<p dir="ltr">The concept of reinforcement learning has become increasingly relevant in learning- based applications, especially in the field of autonomous navigation, because of its fundamental nature to operate without the necessity of labeled data. However, the infeasibility of training reinforcement learning based autonomous navigation applications in a real-world setting has increased the popularity of researching and developing on autonomous navigation systems by creating simulated environments in game engine platforms. This thesis investigates the comparative performance of Unity and Unreal Engine 5 within the framework of a reinforcement learning system applied to autonomous race car navigation. A rudimentary simulated setting featuring a model car navigating a racetrack is developed, ensuring uniformity in environmental aspects across both Unity and Unreal Engine 5. The research employs reinforcement learning with genetic algorithms to instruct the model car in race track navigation; while the tools and programming methods for implementing reinforcement learning vary between the platforms, the fundamental concept of reinforcement learning via genetic algorithms remains consistent to facilitate meaningful comparisons. The implementation includes logging of key performance variables during run times on each platform. A comparative analysis of the performance data collected demonstrates Unreal Engine's superior performance across the collected variables. These findings contribute insights to the field of autonomous navigation systems development and reinforce the significance of choosing an optimal underlying simulation platform for reinforcement learning applications.</p>

<p dir="ltr">Autonomous vehicles require motion forecasting of their surrounding multiagents (pedestrians</p><p dir="ltr">and vehicles) to make optimal decisions for navigation. The existing methods focus on</p><p dir="ltr">techniques to utilize the positions and velocities of these agents and fail to capture semantic</p><p dir="ltr">information from the scene. Moreover, to mitigate the increase in computational complexity</p><p dir="ltr">associated with the number of agents in the scene, some works leverage Euclidean distance to</p><p dir="ltr">prune far-away agents. However, distance-based metric alone is insufficient to select relevant</p><p dir="ltr">agents and accurately perform their predictions. To resolve these issues, we propose the</p><p dir="ltr">Semantics-aware Interactive Multiagent Motion Forecasting (SIMMF) method to capture</p><p dir="ltr">semantics along with spatial information and optimally select relevant agents for motion</p><p dir="ltr">prediction. Specifically, we achieve this by implementing a semantic-aware selection of relevant</p><p dir="ltr">agents from the scene and passing them through an attention mechanism to extract</p><p dir="ltr">global encodings. These encodings along with agents’ local information, are passed through</p><p dir="ltr">an encoder to obtain time-dependent latent variables for a motion policy predicting the future</p><p dir="ltr">trajectories. Our results show that the proposed approach outperforms state-of-the-art</p><p dir="ltr">baselines and provides more accurate and scene-consistent predictions. </p>

<p dir="ltr">The original definition of Nash Equilibrium applied to normal form games, but the notion has now been extended to various other forms of games including leader-follower games (Stackelberg games), extensive form games, stochastic games, games of incomplete information, cooperative games, and so on. We focus on general-sum stochastic Stackelberg games in this work. An example where such games would be natural to consider is in security games where a defender wishes to protect some targets through deployment of limited resources and an attacker wishes to strategically attack the targets to benefit themselves. The hierarchical order of play arises naturally since the defender typically acts first and deploys a strategy, while the attacker observes the strategy ofthe defender before attacking. Another example where this framework fits is in testing during epidemics, where the leader (the government) sets testing policies and the follower (the citizens) decide at every time step whether to get tested. The government wishes to minimize the number of infected people in the population while the follower wishes to minimize the cost of getting sick and testing. This thesis presents a learning algorithm for players to converge to their stationary policies in a general sum stochastic sequential Stackelberg game. The algorithm is a two time scale implicit policy gradient algorithm that provably converges to stationary points of the optimization problems of the two players. Our analysis allows us to move beyond the assumptions of zero-sum or static Stackelberg games made in the existing literature for learning algorithms to converge.</p><p dir="ltr"><br></p>

Traditional software testing usually comes with manual definitions of test cases. This manual process can be time-consuming, tedious, and incomplete in covering important but elusive corner cases that are hardly identifiable. Automatic generation of random test cases emerges as a strategy to mitigate the challenges associated with the manual test case design. However, the effectiveness of random test cases in fault detection may be limited, leading to increased testing costs, particularly in systems where test execution demands substantial resources and time. Leveraging the domain knowledge of test experts can guide the automatic random generation of test cases to more effective zones. In this thesis, we target quality assurance of multiagent autonomous systems and aim to automate test generation for them by applying the domain knowledge of test experts. To formalize the specification of the domain expert's knowledge, we introduce a small Domain Specific Language (DSL) for formal specification of particular locality-based constraints for grid-based multiagent systems. We initially employ this DSL for filtering randomly generated test inputs. Then, we evaluate the effectiveness of the generated test cases through an experiment on a case study of autonomous agents. Applying statistical analysis on the experiment results demonstrates that utilizing the domain knowledge to specify test selection criteria for filtering randomly generated test cases significantly reduces the number of potentially costly test executions to identify the persisting faults.  Domain knowledge of experts can also be utilized to directly generate test inputs with constraint solvers. We conduct a comprehensive study to compare the performance of filtering random cases and constraint-solving approaches in generating selective test cases across various test scenario parameters. The examination of these parameters provides criteria for determining the suitability of random data filtering versus constraint solving, considering the varying size and complexity of the test input generation constraint. To conduct our experiments, we use QuickCheck tool for random test data generation with filtering, and we employ Z3 for constraint solving. The findings, supported by observations and statistical analysis, reveal that test scenario parameters impact the performance of filtering and constraint-solving approaches differently. Specifically, the results indicate complementary strengths between the two approaches: random generation and filtering approach excels for the systems with a large number of agents and long agent paths but shows degradation in larger grid sizes and stricter constraints. Conversely, constraint solving approach demonstrates robust performance for large grid sizes and strict constraints but experiences degradation with increased agent numbers and longer paths. Our initially proposed DSL is limited in its features and is only capable of specifying particular locality-based constraints. To be able to specify more elaborate test scenarios, we extend that DSL based on a more intricate model of autonomous agents and their environment. Using the extended DSL, we can specify test oracles and test scenarios for a dynamic grid environment and agents having several attributes. To assess the extended DSL's utility, we design a questionnaire to gather opinions from several experts and also run an experiment to compare the efficiency of the extended DSL with the initially proposed one. The questionnaire results indicate that the extended DSL was successful in specifying several scenarios that the experts found more useful than the scenarios specified by the initial DSL. Moreover, the experimental results demonstrate that testing with the extended DSL can significantly reduce the number of test executions to detect system faults, leading to a more efficient testing process. / Safety of Connected Intelligent Vehicles in Smart Cities – SafeSmart

The BDI agent architecture includes a plan library containing pre-de ned plans. The plan library is included in the agent architecture to reduce the need for expensive means-end reasoning, however can hinder the agent's e ectiveness when operating in a changing environment. Existing research on integrating di erent planning methods into the BDI agent to overcome this limitation include HTNs, state-space planning and Graphplan. Genetic Algorithms (GAs) have not yet been used for this purpose. This dissertation investigates the feasibility of using GAs as a plan modi cation mechanism for BDI agents. It covers the design of a plan structure that can be encoded into a binary string, which can be operated on by the genetic operators. The e ectiveness of the agent in a changing environment is compared to an agent without the GA plan modification mechanism. The dissertation shows that GAs are a feasible plan modification mechanism for BDI agents. / Information Science

Made available in DSpace on 2015-03-05T13:56:58Z (GMT). No. of bitstreams: 0 Previous issue date: 30 / Nenhuma / Este trabalho apresenta um modelo de percepção de dados não visíveis (por exemplo, olfativas e auditivas) para humanos virtuais baseado em informações puramente geométricas existentes em ambientes virtuais. Esta proposta procurou contextualizar vários tópicos de pesquisa no que diz respeito a agentes autônomos em ambientes virtuais e suas capacidades de percepção. Procurou-se, através da área da visão sintética, encontrar métodos que pudessem ser generalizados para que outros sentidos pudessem ser simulados. O objetivo principal deste modelo é possibilitar que o agente possa identificar diferentes ocorrências de eventos dentro de um ambiente virtual, no que diz respeito às informações não necessariamente visuais, como por exemplo, a presença de cheiros ou barulhos. Para o modelo, foram buscados métodos que proporcionem um bom desempenho computacional e que sejam facilmente generalizados e implementados de maneira a prover métodos de percepção a multidões de humanos virtuais / This work presents a perception data model which deals with not visible information (such as hearing and smelling). This model is based on geometric information that exists in the virtual environment. This proposal also identifies several research topics related to autonomous agents in virtual environments and their perception abilities. Investigations in synthetic vision area allows to find methods that could be generalized in order to be used for other sensors. Therefore, the main objective of this model is to allow to the agents the possibility of identifying different events, into a virtual environment, not necessarily visual data. For instance, the presence of smell and noise. In addition, we propose methods which aims to provide a good computational performance and which can be generalized and implemented in a way that they can provide the perception methods to virtual human beings.

<p>Autonomous agent systems are systems in which many simple entities, called “agents”, interact with each other. The behaviour resulting from such interactions can be much more complex than that of the individual agents. A group of interacting agents can for example accomplish tasks that no single agent could.</p> <p>Nature provides several examples of autonomous agent systems, such as flocks of birds and insects, schools of fish, and anthills. Progresses in robotics, electronics and telecommunications make it now also possible to design such systems in order to accomplish particular tasks, such as the surveillance or exploration of areas, or the maintenance of some environments.</p> <p>In this thesis, we analyze two issues related to autonomous agent systems, and more precisely, to the influence of the inter-agent communication network on the system behaviour. In a first part, we consider the problem of preserving the shape of a multi-agent formation by explicitly maintaining the distances between some agents constant. We study the case of distance constraints that are unilateral, that is, constraints for which the responsibility is given to a one of the two agents concerned. This leads to the notions of persistence and constraint consistence. The second part is devoted to the consensus problems: agents have a value which they update by averaging that of other agents. Eventually, all agents may obtain a common value, in which case we say that the system reaches a consensus. One major difficulty in the study of such system is the possible dependence of the interaction and communication topology on the values of the agents. We study two paradigmatic systems in which this dependence can be taken into account, and obtain results on their convergence and on the stability of their equlibria.</p>

The recent technological advances in the field of autonomous vehicles have resulted in a growing impetus for researchers to improve the current framework of mission planning and execution within both the military and civilian contexts. Many recent efforts towards this direction emphasize the importance of replacing the so-called monolithic paradigm, where a mission is planned, monitored, and controlled by a unique global decision maker, with a network centric paradigm, where the same mission related tasks are performed by networks of interacting decision makers (autonomous vehicles). The interest in applications involving teams of autonomous vehicles is expected to significantly grow in the near future as new paradigms for their use are constantly being proposed for a diverse spectrum of real world applications. One promising approach to extend available techniques for addressing problems involving a single autonomous vehicle to those involving teams of autonomous vehicles is to use the concept of Voronoi diagram as a means for reducing the complexity of the multi-vehicle problem. In particular, the Voronoi diagram provides a spatial partition of the environment the team of vehicles operate in, where each element of this partition is associated with a unique vehicle from the team. The partition induces, in turn, a graph abstraction of the operating space that is in a one-to-one correspondence with the network abstraction of the team of autonomous vehicles; a fact that can provide both conceptual and analytical advantages during mission planning and execution. In this dissertation, we propose the use of a new class of Voronoi-like partitioning schemes with respect to state-dependent proximity (pseudo-) metrics rather than the Euclidean distance or other generalized distance functions, which are typically used in the literature. An important nuance here is that, in contrast to the Euclidean distance, state-dependent metrics can succinctly capture system theoretic features of each vehicle from the team (e.g., vehicle kinematics), as well as the environment-vehicle interactions, which are induced, for example, by local winds/currents. We subsequently illustrate how the proposed concept of state-dependent Voronoi-like partition can induce local control schemes for problems involving networks of spatially distributed autonomous vehicles by examining different application scenarios.

<p>This master thesis investigates the principal capability of artificial evolution to produce tool use behavior in adaptive agents, excluding the application of life-time learning or adaptation mechanisms. Tool use is one aspect of complex behavior that is expected from autonomous agents acting in real-world environments. In order to achieve tool use behavior an agent needs to identify environmental objects as potential tools before it can use the tools in a problem-solving task. Up to now research in robotics has focused on life-time learning mechanisms in order to achieve this. However, these techniques impose great demands on resources, e.g. in terms of memory or computational power. All of them have shown limited results with respect to a general adaptivity. One might argue that even nature does not present any kind of omni-adaptive agent. While humans seem to be a good example of natural agents that master an impressive variety of life conditions and environments (at least from a human perspective, other examples are spectacular survivability observations of octopuses, scorpions or various viruses) even the most advanced engineering approaches can hardly compete with the simplest life-forms in terms of adaptation. This thesis tries to contribute to engineering approaches by promoting the application of artificial evolution as a complementing element with the presentation of successful pioneering experiments. The results of these experiments show that artificial evolution is indeed capable to render tool use behavior at different levels of complexity and shows that the application of artificial evolution might be a good complement to life-time approaches in order to create agents that are able to implicitly extract concepts and display tool use behavior. The author believes that off-loading at least parts of the concept retrieval process to artificial evolution will reduce resource efforts at life-time when creating autonomous agents with complex behavior such as tool use. This might be a first step towards the vision of a higher level of autonomy and adaptability. Moreover, it shows the demand for an experimental verification of commonly accepted limits between qualities of learned and evolved tool use capabilities.</p>

The BDI agent architecture includes a plan library containing pre-defined plans. The plan library is included in the agent architecture to reduce the need for expensive means-end reasoning, however can hinder the agent’s effectiveness when operating in a changing environment. Existing research on integrating different planning methods into the BDI agent to overcome this limitation include HTNs, state-space planning and Graphplan. Genetic Algorithms (GAs) have not yet been used for this purpose. This dissertation investigates the feasibility of using GAs as a plan modification mechanism for BDI agents. It covers the design of a plan structure that can be encoded into a binary string, which can be operated on by the genetic operators. The effectiveness of the agent in a changing environment is compared to an agent without the GA plan modification mechanism. The dissertation shows that GAs are a feasible plan modification mechanism for BDI agents. / Information Science