From a system developer's perspective, designing a spoken dialogue system can be a time-consuming and difficult process. A developer may spend a lot of time anticipating how a potential user might interact with the system and then deciding on the most appropriate system response. These decisions are encoded in a dialogue strategy, essentially a mapping between anticipated user inputs and appropriate system outputs. To reduce the time and effort associated with developing a dialogue strategy, recent work has concentrated on modelling the development of a dialogue strategy as a sequential decision problem. Using this model, reinforcement learning algorithms have been employed to generate dialogue strategies automatically. These algorithms learn strategies by interacting with simulated users. Some progress has been made with this method but a number of important challenges remain. For instance, relatively little success has been achieved with the large state representations that are typical of real-life systems. Another crucial issue is the time and effort associated with the creation of simulated users. In this thesis, I propose an alternative to existing reinforcement learning methods of dialogue strategy development. More specifically, I explore how XCS, an evolutionary reinforcement learning algorithm, can be used to find dialogue strategies that cover large state spaces. Furthermore, I suggest that hand-coded simulated users are sufficient for the learning of useful dialogue strategies. I argue that the use of evolutionary reinforcement learning and hand-coded simulated users is an effective approach to the rapid development of spoken dialogue strategies. Finally, I substantiate this claim by evaluating a learned strategy with real users. Both the learned strategy and a state-of-the-art hand-coded strategy were integrated into an end-to-end spoken dialogue system. The dialogue system allowed real users to make flight enquiries using a live database for an Edinburgh-based airline. The performance of the learned and hand-coded strategies were compared. The evaluation results show that the learned strategy performs as well as the hand-coded one (81% and 77% task completion respectively) but takes much less time to design (two days instead of two weeks). Moreover, the learned strategy compares favourably with previous user evaluations of learned strategies.

A simple asset pricing model with two types of adaptively learning traders, fundamentalists and technical analysts, is studied. Fractions of these trader types, which are both boundedly rational, change over time according to evolutionary learning, with technical analysts conditioning their forecasting rule upon deviations from a benchmark fundamental. Volatility clustering arises endogenously in this model. Two mechanisms are proposed as an explanation. The first is coexistence of a stable steady state and a stable limit cycle, which arise as a consequence of a so-called Chenciner bifurcation of the system. The second is intermittency and associated bifurcation routes to strange attractors. Both phenomena are persistent and occur generically in nonlinear multi-agent evolutionary systems. (author's abstract) / Series: Working Papers SFB "Adaptive Information Systems and Modelling in Economics and Management Science"

I present a simple model of an evolutionary financial market with heterogeneous agents, based on the concept of adaptive belief systems introduced by Brock and Hommes (1997a). Agents choose between different forecast rules based on past performance, resulting in an evolutionary dynamics across predictor choice coupled to the equilibrium dynamics. The model generates endogenous price fluctuations with similar statistical properties as those observed in real return data, such as fat tails and volatility clustering. These similarities are demonstrated for data from the British, German, and Austrian stock market. (author's abstract) / Series: Working Papers SFB "Adaptive Information Systems and Modelling in Economics and Management Science"

The goal of Evolutionary Robotics (ER) is the development of automatic processes for the synthesis of robot control systems using evolutionary computation. The idea that it may be possible to synthesise robotic control systems using an automatic design process is appealing. However, ER is considerably more challenging and less automatic than its advocates would suggest. ER applies methods from the field of neuroevolution to evolve robot control systems. Neuroevolution is a machine learning algorithm that applies evolutionary computation to the design of Artificial Neural Networks (ANN). The aim of this thesis is to assay the practical characteristics of neuroevolution by performing bulk experiments on a set of Reinforcement Learning (RL) problems. This thesis was conducted with the view of applying neuroevolution to the design of neurocontrollers for small low-cost Autonomous Underwater Vehicles (AUV). A general approach to neuroevolution for RL problems is presented. The is selected to evolve ANN connection weights on the basis that it has shown competitive performance on continuous optimisation problems, is self-adaptive and can exploit dependencies between connection weights. Practical implementation issues are identified and discussed. A series of experiments are conducted on RL problems. These problems are representative of problems from the AUV domain, but manageable in terms of problem complexity and computational resources required. Results from these experiments are analysed to draw out practical characteristics of neuroevolution. Bulk experiments are conducted using the inverted pendulum problem. This popular control benchmark is inherently unstable, underactuated and non-linear: characteristics common to underwater vehicles. Two practical characteristics of neuroevolution are demonstrated: the importance of using randomly generated evaluation sets and the effect of evaluation noise on search performance. As part of these experiments, deficiencies in the benchmark are identified and modifications suggested. The problem of an underwater vehicle travelling to a goal in an obstacle free environment is studied. The vehicle is modelled as a Dubins car, which is a simplified model of the high-level kinematics of a torpedo class underwater vehicle. Two practical characteristics of neuroevolution are demonstrated: the importance of domain knowledge when formulating ANN inputs and how the fitness function defines the set of evolvable control policies. Paths generated by the evolved neurocontrollers are compared with known optimal solutions. A framework is presented to guide the practical application of neuroevolution to RL problems that covers a range of issues identified during the experiments conducted in this thesis. An assessment of neuroevolution concludes that it is far from automatic yet still has potential as a technique for solving reinforcement problems, although further research is required to better understand the process of evolutionary learning. The major contribution made by this thesis is a rigorous empirical study of the practical characteristics of neuroevolution as applied to RL problems. A critical, yet constructive, viewpoint is taken of neuroevolution. This viewpoint differs from much of the reseach undertaken in this field, which is often unjustifiably optimistic and tends to gloss over difficult practical issues.

Submitted by Luciana Ferreira (lucgeral@gmail.com) on 2016-04-27T13:37:50Z No. of bitstreams: 2 Dissertação - Francisco Calaça Xavier - 2012.pdf: 5019345 bytes, checksum: 0e64a53ebdeda990e6ef1175f1732c19 (MD5) license_rdf: 23148 bytes, checksum: 9da0b6dfac957114c6a7714714b86306 (MD5) / Approved for entry into archive by Luciana Ferreira (lucgeral@gmail.com) on 2016-04-27T13:40:15Z (GMT) No. of bitstreams: 2 Dissertação - Francisco Calaça Xavier - 2012.pdf: 5019345 bytes, checksum: 0e64a53ebdeda990e6ef1175f1732c19 (MD5) license_rdf: 23148 bytes, checksum: 9da0b6dfac957114c6a7714714b86306 (MD5) / Made available in DSpace on 2016-04-27T13:40:15Z (GMT). No. of bitstreams: 2 Dissertação - Francisco Calaça Xavier - 2012.pdf: 5019345 bytes, checksum: 0e64a53ebdeda990e6ef1175f1732c19 (MD5) license_rdf: 23148 bytes, checksum: 9da0b6dfac957114c6a7714714b86306 (MD5) Previous issue date: 2012-06-21 / The problem of decision making about the allocation of resources is present in many areas of society. The allocation of ambulances to the occurrence of accidents with victims and the allocation of teams to solve problems in the supply of electricity are examples of situations where it is necessary to make this decision. We can also mention the problems that occur in the allocation of hardware resources when a system is running in a distributed form. In this context, this paper presents the system COGNARE, which brings together techniques such as Genetic Algorithms, Fuzzy Logic and Multiagent Systems in order to allocate tasks to resources dynamically. The COGNARE was used in two different situations. At first, the problem was to allocate vehicles to a distributor of electricity to occurrences of failures in supply. In the second situation, the problem was to allocate hardware resources in a distributed system. In both cases, the COGNARE presented as a system of allocating resources efficiently. Keywords / O problema da tomada de decisão quanto a alocação de recursos está presente em diversas áreas da sociedade. A alocação de ambulâncias à ocorrências de acidentes com vítimas e a alocação de equipes para solução de problemas no fornecimento de energia elétrica são exemplos de situações onde são necessárias tomadas de decisão. Os problemas que ocorrem na alocação de recursos de hardware quando um sistema é executado de forma distribuída também requerem decisões. Neste contexto, este trabalho apresenta o sistema COGNARE, que reúne a utililização de técnicas como Algoritmos Genéticos, Lógica Fuzzy e Sistemas Multiagentes com o objetivo de alocar dinamicaminte tarefas a recursos. O COGNARE foi utilizado em duas situações distintas. Na primeira, o problema consistia em alocar dinamicamente viaturas de uma empresa de distribuição de energia elétrica a ocorrências de falhas no fornecimento. Na segunda situação, o problema consistia em alocar dinamicamente recursos de hardware em um sistema distribuído. Nestes dois casos, o COGNARE apresentou-se como um sistema de alocação de recursos eficiente.

<p dir="ltr">This dissertation investigates strategic decision-making under conditions of environmental inequality. The three chapters explore various forms of inequality across different decision contexts</p><p dir="ltr">The first chapter examines the impact of income inequality on individuals' participation in multiple public goods investments. Specifically, it analyzes how a global club good opportunity influences local public goods provision in indefinitely repeated interactions within a linear public goods game using a voluntary contribution mechanism. The study varies global club entry costs and local community endowment compositions to assess their effects on contributions and welfare. It finds that income inequality does not significantly alter contribution behaviors in single public good settings under indefinitely repeated interactions. With the introduction of a global club good, lower entry costs lead to higher participation rates among subjects, resulting in increased total welfare for both homogeneous and heterogeneous communities. Conversely, higher entry costs reduce participation and overall welfare. Heterogeneous communities discontinue club use sooner than homogeneous ones. Efficiency, measured as realized payoff relative to maximal social benefits, declines across all treatments following the introduction of a global club good. Additionally, counterfactual simulations using an individual evolutionary learning model demonstrate that the welfare benefits of a global club good opportunity hinge on its ability to yield substantial social benefits compared to local public goods.</p><p dir="ltr">The second chapter explores how power inequality influences cooperation in a dynamic game where competition and cooperation evolve over time. This research, conducted as part of a collaborative project with Yaroslav Rosokha, Denis Tverskoi, and Sergey Gavrilets, examines cooperation dynamics in scenarios where cooperation's benefits depend on political power derived from a contest. The study highlights that incumbency advantages in political contests precipitate a rapid breakdown of cooperation within social dilemmas. Furthermore, it investigates behavioral disparities between groups and individuals, leveraging simulations based on the Arifovic and Ledyard (2012) individual evolutionary learning model to shed light on the difference observed in the experiment.</p><p dir="ltr">The third chapter investigates the impact of unequal positions in a directed communication network on individuals' optimal stopping rules and social learning outcomes. The study involves subjects making predictions about uncertain states of the world using private information and social information obtained through a directed network. Theoretical predictions suggest that individuals should wait when the benefit of waiting exceeds the associated cost. Empirical results confirm that subjects indeed wait longer in more connected networks or when waiting costs are low. However, deviations from equilibrium predictions indicate influences of bounded rationality (supported by quantal response equilibrium) and heuristic decision-making, where some subjects consistently wait for a single turn regardless of positional advantage. Importantly, under-waiting at an information aggregator's position has negative externalities on group-wide information acquisition.</p>