Made available in DSpace on 2015-05-14T12:36:51Z (GMT). No. of bitstreams: 1 arquivototal.pdf: 4482656 bytes, checksum: 11b85e413d691749edd8d5be0d8f56d4 (MD5) Previous issue date: 2014-03-28 / Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - CAPES / Electronic games and, in particular, real-time strategy (RTS) games, are increasingly seen as viable and important fields for artificial intelligence research because of commonly held characteristics, like the presence of complex environments, usually dynamic and with multiple agents. In commercial RTS games, the computer behavior is mostly designed with simple ad hoc, static techniques that require manual definition of actions and leave the agent unable to adapt to the various situations it may find. This approach, besides being lengthy and error-prone, makes the game relatively predictable after some time, allowing the human player to eventually discover the strategy used by the computer and develop an optimal way of countering it. Using machine learning techniques like reinforcement learning is a way of trying to avoid this predictability, allowing the computer to evaluate the situations that occur during the games, learning with these situations and improving its behavior over time, being able to choose autonomously and dynamically the best action when needed. This work proposes a modeling for the use of SARSA, a reinforcement learning technique, applied to combat situations in RTS games, with the goal of allowing the computer to better perform in this fundamental area for achieving victory in an RTS game. Several tests were made with various game situations and the agent applying the proposed modeling, facing the game's default AI opponent, was able to improve its performance in all of them, developing knowledge about the best actions to choose for the various possible game states and using this knowledge in an efficient way to obtain better results in later games / Jogos eletrônicos e, em especial, jogos de estratégia em tempo real (RTS), são cada vez mais vistos como campos viáveis e importantes para pesquisas de inteligência artificial por possuírem características interessantes para a área, como a presença de ambientes complexos, muitas vezes dinâmicos e com múltiplos agentes. Nos jogos RTS comerciais, o comportamento do computador é geralmente definido a partir de técnicas ad hoc simples e estáticas, com a necessidade de definição manual de ações e a incapacidade de adaptação às situações encontradas. Esta abordagem, além de demorada e propícia a erros, faz com que o jogo se torne relativamente previsível após algum tempo, permitindo ao jogador eventualmente descobrir a estratégia utilizada pelo computador e desenvolver uma forma ótima de enfrentá-lo. Uma maneira de tentar combater esta previsibilidade consiste na utilização de técnicas de aprendizagem de máquina, mais especificamente do aprendizado por reforço, para permitir ao computador avaliar as situações ocorridas durante as partidas, aprendendo com estas situações e aprimorando seu conhecimento ao longo do tempo, sendo capaz de escolher de maneira autônoma e dinâmica a melhor ação quando necessário. Este trabalho propõe uma modelagem para a utilização de SARSA, uma técnica do aprendizado por reforço, aplicada a situações de combate em jogos RTS, com o objetivo de fazer com o que o computador possa se portar de maneira mais adequada nessa área, uma das mais fundamentais para a busca da vitória em um jogo RTS. Nos testes realizados em diversas situações de jogo, o agente aplicando a modelagem proposta, enfrentando o oponente padrão controlado pela IA do jogo, foi sempre capaz de melhorar seus resultados ao longo do tempo, obtendo conhecimento acerca das melhores ações a serem tomadas a cada momento decisório e aproveitando esse conhecimento nas suas partidas futuras

Le travail présenté dans ce manuscrit porte sur le développement d’un simulateur de cible (RTS) pour radars automobiles 77 GHz. Afin de proposer des véhicules toujours plus sûrs, les constructeurs automobiles développent des systèmes ADAS de plus en plus performants. On assiste aujourd’hui à la démocratisation des radars automobiles d’alerte à la collision et de régulation de distance. La généralisation de tels systèmes sur des véhicules de série va nécessiter le recours accru à des moyens de tests, chez les constructeurs et dans les centres de contrôle technique. Pour pouvoir tester et calibrer les radars, il est nécessaire d’utiliser des RTS. Ces appareils permettent de simuler les scénarios rencontrés par le radar, ceux-ci devenant plus complexes avec le développement des voitures autonomes. Une cible est définie par trois paramètres : une vitesse, une distance et une SER. Afin de répondre à des exigences drastiques, Autocruise développe ses propres RTS pour des bancs de test de production et de R&D. Ils doivent s’adapter à tout radar fonctionnant sur la bande 76 – 81 GHz, avec différentes modulations et une bande de fréquence supérieure à 800 MHz. Le système doit être à bas coût, de faibles dimensions et flexible pour être intégré dans différentes applications. Le principal verrou technologique est la réalisation d’une ligne à retard variable, capable de simuler des distances comprises entre 1 m et 250 m, avec une résolution de 0,2 m et permettre le contrôle de la SER. Un compromis devra être trouvé afin de répondre aux spécifications. L’étude a montré l’impossibilité de couvrir l’ensemble de la plage de distances avec une seule technologie. Une architecture hybride est indispensable. Une ligne à retard hybride reconfigurable large bande est présentée. / The work presented in this thesis concerns the development of an automotive radar target simulator for 77 GHz radar sensors. In order to continue offering safer vehicles, manufacturers develop more and more performant ADAS systems. We are witnessing a democratization of automotive radar sensors for adaptive cruise control and collision warning. The generalization of such systems on standard cars will require an increased use of test devices both at the manufacturers and in technical control centers. To test and calibrate radars, it is necessary to use Radar Target Simulators (RTS). These devices enable to simulate situations encountered by the radar. Furthermore, these scenarios are becoming increasingly complex with the arrival of autonomous vehicles. A target is defined by three parameters: distance, velocity and radar cross-section (RCS). In order to meet drastic requirements, ZF TRW Autocruise develops its own RTS for production test benches and R&D. RTS must adapt to all radars within a 76 – 81 GHz frequency band, with different modulations and a frequency bandwidth higher than 800 MHz. The system must present the advantages of being a low-cost system, with small dimensions and flexible to be integrated in different applications. The major blocking point is the design of a reconfigurable delay line, able to simulate distances between 1 m and 250 m with a resolution of 0.2 m on a large frequency band and also allowing control of RCS. A compromise will have to be found in order to meet the different specifications. The study showed the impossibility to cover the entire range of distances with one single technology. A hybrid architecture is necessary. A hybrid, tunable, wideband delay line is at study.

Artificial intelligence (AI) is already a fundamental component of computer games. In this context is emotions a growing part in simulating real life. The proposed emotional state module, provides a way for the game agents to select an action in real-time virtual environments. The modules function has been tested with the open-source strategy game ORTS. This thesis proposes a new approach for the design of an interacting network, similar to a spreading activation system, of emotional states that keeps track of emotion intensities changing and interacting over time. The network of emotions can represent any number of persisting states, such as moods, emotions and drives. Any emotional signal can affect every state positively or negatively. The states' response to emotional signals are influenced by the other states represented in the network. The network is contained within an emotional state module. This interactions between emotions are not the focus of much research, neither is the representation model. The focus tend to be on the mechanisms eliciting emotions and on how to express the emotions.

This thesis is focused on design of framework for creation an articial opponents in strategy games. We will analyze different types of strategy games and artificial intelligence systems used in these types of games. Next we will describe problems, which can occur  in these systems and why agent-based systems makes better artificial opponents. Next we will use knowledge from this research to design and implement framework, which will act as support for creating an artificial intelligence in strategy games.

This thesis is focused on design of framework for creation an articial opponents in strategy games. We will analyze different types of strategy games and artificial intelligence systems used in these types of games. Next we will describe problems, which can occur  in these systems and why agent-based systems makes better artificial opponents. Next we will use knowledge from this research to design and implement framework, which will act as support for creating an artificial intelligence in strategy games.

Sometimes games that belong to the same genre achieve different levels of success among the playerbase of the genre. As an example, of two space RTS games Homeworld (released in 1999) and O.R.B. (releasedin 2002), which feature a similar setting and similar gameplay mechanics, Homeworld has beenmore successful. The purpose of this paper is to investigate the reasons for the different popularityof the two games, using game reviews as the primary source of data to learn about the strengths andthe weaknesses of the two games. Thematic analysis has been applied to a total of 27 game reviewsof space RTS games Homeworld and O.R.B. Codes describing various strong and weak aspects of bothgames have been extracted from the texts. The codes have been grouped into various categories traditionallydescribed in game reviews, such as the story line, the UI, the graphics, etc. Using these codesas a basis, a number of explanations have been suggested as to why the game Homeworld has beenmore successful than the game O.R.B. The main reasons discovered are that Homeworld was releasedseveral years before its competitor and that some of important game features of O.R.B. suffered frompoorer technical execution in comparison to equivalent features in Homeworld.

The diploma thesis is focused on the analysis of the implementation and valuation of the BIM model in a construction company. In the practical part, a specific construction company and its project is selected for the implementation and valuation using valuation plugins, which define the elements according to the design classification. For the purposes of this work, the constructions of the main building production are selected - piles, prefabricated load-bearing skeleton, wall perimeter cladding and roof folded cladding. Based on the model developed in this way, the BIM model is compiled and by exporting data from valuation plugins, an item budget is processed. The main result of this work is to perform an implementation analysis and show the advantages and disadvantages of the BIM approach to projects.

<p>There are several standards for wireless communication. People that are involved in computers and networking recognize names like Bluetooth, HiperLAN and IEEE 802.11. The last one was standardized in 1997 [2,6]and has begun to reach acceptance as a solid ground for wireless networking. A fundamental part of an IEEE 802.11 node is the Medium Access Controller, or MAC. It establishes and controls communication with other nodes, using a physical layer unit. </p><p>The work has been carried out as final project at Linkopings Universitet, it has been about the improvement of the functions of MAC layer. I have developed some of the required functions that PUM uses to interact with the MAC layer. Because of that, I have implemented the Reception functions of MAC layer, having the possibility of using short control frames RTS/CTS to minimize collision.</p>

There are several standards for wireless communication. People that are involved in computers and networking recognize names like Bluetooth, HiperLAN and IEEE 802.11. The last one was standardized in 1997 [2,6]and has begun to reach acceptance as a solid ground for wireless networking. A fundamental part of an IEEE 802.11 node is the Medium Access Controller, or MAC. It establishes and controls communication with other nodes, using a physical layer unit. The work has been carried out as final project at Linkopings Universitet, it has been about the improvement of the functions of MAC layer. I have developed some of the required functions that PUM uses to interact with the MAC layer. Because of that, I have implemented the Reception functions of MAC layer, having the possibility of using short control frames RTS/CTS to minimize collision.

The advances in computational and experimental modeling in the area of structural mechanics have stimulated research in a class of hybrid problems that require both of these modeling capabilities to be combined to achieve certain objectives. Real-time substructure (RTS) testing, structural system identification (SSI) and active control techniques fall in the category of hybrid problems that need efficient tools in both computational and experimental phases for their successful implementation. RTS is a hybrid testing method, which aims to overcome the scaling problems associated with the conventional dynamic testing methods (such as shake table test, effective force test and pseudo dynamic test) by testing the critical part of the structure experimentally with minimum compromise on spatio-temporal scaling, while modeling the remaining part numerically. The problem of SSI constitutes an important component within the broader framework of problems of structural health monitoring where, based on the in-situ measurements on the loading and a subset of critical responses of the structure, the system parameters are estimated with a view to detecting damage/degradation. Active control techniques are employed to maintain the functionality of important structures, especially under extreme dynamic loading. The work reported in the present thesis contributes to the areas of RTS, SSI and active control of nonlinear systems, the main focus being the computational aspects, i.e., in developing numerical strategies to address some of the unsolved issues, although limited efforts have also been made to undertake laboratory experimental investigations in the area of nonlinear SSI. The thesis is organized into seven chapters and five appendices. The first chapter contains an overview of the state of the art techniques in dynamic testing, SSI and structural control. The topics covered include effective force test, pseudo dynamic test, RTS test, time and frequency domain methods of nonlinear system identification, dynamic state estimation techniques with emphasis on particle filters, Rao-Blackwellization, structural control methods, control algorithms and active control of nonlinear systems. The review identifies a set of open problems that are subsequently addressed, to an extent, in the thesis. Chapter 2 focuses on the development of a time domain coupling technique, involving combined computational and experimental modeling, for vibration analysis of structures built-up of linear/nonlinear substructures. The numerical and experimental substructures are allowed to interact in real-time. The equation of motion of the numerical substructure is integrated using a step-by-step procedure that is formulated in the state space. For systems with nonlinear substructures, a multi-step transversal linearization method is used to integrate the equations of motion; and, a multi-step extrapolation scheme, based on the reproducing kernel particle method, is employed to handle the time delays that arise while accounting for the interaction between the substructures. Numerical illustrations on a few low dimensional vibrating structures are presented and these examples are fashioned after problems of seismic qualification testing of engineering structures using RTS testing techniques. The concept of substructuring is extended in Chapter 3 for implementing Rao-Blackwellization, a technique of combining particle filters with analytical computation through Kalman filters, for state and parameter estimations of a class of nonlinear dynamical systems with additive Gaussian process/observation noises. The strategy is based on decomposing the system to be estimated into mutually coupled linear and nonlinear substructures and then putting in place a rational framework to account for coupling between the substructures. While particle filters are applied to the nonlinear substructures, estimation of linear substructures proceeds using a bank of Kalman filters. Numerical illustrations for state/parameter estimations of a few linear and nonlinear oscillators with noise in both the process and measurements are provided to demonstrate the potential of the Rao-Blackwellized particle filter (RBPF) with substructuring. In Chapter 4, the concept of Rao-Blackwellization is extended to handle more general nonlinear systems, using two different schemes of linearization. A semi-analytical filter and a conditionally linearized filter, within the framework of Monte Carlo simulations, are proposed for state and parameter estimations of nonlinear dynamical systems with additively Gaussian process/observation noises. The first filter uses a local linearization of the nonlinear drift fields in the process/observation equations based on explicit Ito-Taylor expansions to transform the given nonlinear system into a family of locally linearized systems. Using the most recent observation, conditionally Gaussian posterior density functions of the linearized systems are analytically obtained through the Kalman filter. In the second filter, the marginalized posterior distribution of an appropriately chosen subset of the state vector is obtained using a particle filter. Samples of these marginalized states are then used to construct a family of conditionally linearized system of equations to obtain the posterior distribution of the states using a bank of Kalman filters. The potential of the proposed filters in state/parameter estimations is demonstrated through numerical illustrations on a few nonlinear oscillators. The problem of active control of nonlinear structural dynamical systems, in the presence of both process and measurement noises, is considered in Chapter 5. The focus of the study is on the exploitability of particle filters for state estimation in feedback control algorithms for nonlinear structures, when a limited number of noisy output measurements are available. The control design is done using the state dependent Riccati equation (SDRE) method. The Bayesian bootstrap filter and another based on sequential importance sampling are employed for state estimation. Numerical illustrations are provided for a few typically nonlinear oscillators of interest in structural engineering. The experimental validation of the RBPF using substructuring (developed in Chapter 3) and the conditionally linearized Monte Carlo filter (developed in Chapter 4), for parameter estimation, is reported in Chapter 6. Measured data available through laboratory experiments on simple building frame models subjected to harmonic base motions is processed using the proposed algorithms to identify the unknown parameters of the model. A brief summary of the contributions made in this thesis, together with a few suggestions for future research, are presented in Chapter 7. Appendix A provides an account of the multi-step transversal linearization method. The derivation of the reproducing kernel shape functions are presented in Appendix B. Appendix C provides the details of the stochastic Taylor expansion and derivation of the covariance structure of Gaussian MSI-s. The performance of a particle filtering algorithm (bootstrap filter) and Kalman filter in the state estimation of a linear system is provided in Appendix D and Appendix E contains the theoretical details of the Rao-Blackwellized particle filter.