The thesis examines Charles Taylor's theory of agency and the moral sources that he believes inform our modern notion of the self. Taylor's concept of the strong evaluator is outlined and brought to bear on post-structuralist and postmodernist literary-theoretical positions that attempt to reconcile amoral positions and nonagency with multicultural political demands and the demands of what Taylor calls a "culture of authenticity". In order to do full justice to a theory of art that would incorporate Taylor's concept of agency, however, it becomes necessary to critique the philosopher's account of art, which he derives from widely held doctrines of Romanticism and aesthetic autonomy found in the Western tradition. The concept of a pragmatist approach to art serves as the main argument against Taylor's views, which exclude certain agents and their social experiences. Those agents who do not subscribe to Romantic and high Modernist ideas about art's function can often be said to adopt a pragmatic critique, which takes into account the uses of art in defining modern identities, and exposes the social privilege that has typically accompanied the autonomy that art has been awarded.

In this work, we present a unified, general approach to variance reduction in agent evaluation using machine learning to minimize variance. Evaluating an agent's performance in a stochastic setting is necessary for agent development, scientific evaluation, and competitions. Traditionally, evaluation is done using Monte Carlo estimation (sample averages); the magnitude of the stochasticity in the domain or the high cost of sampling, however, can often prevent the approach from resulting in statistically significant conclusions. Recently, an advantage sum technique based on control variates has been proposed for constructing unbiased, low variance estimates of agent performance. The technique requires an expert to define a value function over states of the system, essentially a guess of the state's unknown value. In this work, we propose learning this value function from past interactions between agents in some target population. Our learned value functions have two key advantages: they can be applied in domains where no expert value function is available and they can result in tuned evaluation for a specific population of agents (e.g., novice versus advanced agents). This work has three main contributions. First, we consolidate previous work in using control variates for variance reduction into one unified, general framework and summarize the connections between this previous work. Second, our framework makes variance reduction practically possible in any sequential decision making task where designing the expert value function is time-consuming, difficult or essentially impossible. We prove the optimality of our approach and extend the theoretical understanding of advantage sum estimators. In addition, we significantly extend the applicability of advantage sum estimators and discuss practical methods for using our framework in real-world scenarios. Finally, we provide low-variance estimators for three poker domains previously without variance reduction and improve strategy selection in the expert-level University of Alberta poker bot.

Natural disasters result in loss of lives, damage to built facilities, and interruption of businesses. The losses are not instantaneous rather they continue to occur until the community is restored to a functional socio-economic entity. Hence, it is essential that policy makers recognize this dynamic aspect of the incurring losses and make realistic plans to enhance the recovery. However, this cannot take place without understanding how homeowners react to recovery signals. These signals can come in different ways: from policy makers showing their strong commitment to restore the community by providing financial support and/or restoration of lifeline infrastructure; or from the neighbors showing their willingness to reconstruct. The goal of this research is to develop a model that can account for homeowners’ dynamic interactions in both organizational and spatial domains. Spatial domain of interactions focuses on how homeowners process signals from the environment such as neighbors reconstructing and local agencies restoring infrastructure, while organizational domain of interactions focuses on how agents process signals from other stakeholders that do not directly affect the environment like insurers. The hypothesis of this study is that these interactions significantly influence decisions to reconstruct and stay, or sell and leave. A multi-agent framework is used to capture emergent behavior such as spatial patterns and formation of clusters. The developed framework is illustrated and validated using experimental data sets.

Software agents and multi-agent systems (MAS) have grown into a very active area of research and commercial development activity. There are many current emerging real-world applications spanning multitude of diverse domains. In the context of agents, ontology has been widely recognised for their significant benefits to interoperability, reusability, and both development and operational aspects of agent systems and applications. Ontology-based multi-agent systems (OBMAS) exploit these advantages in providing intelligent and semantically aware applications. In addressing the lack of support for ontology in existing methodologies for multi-agent development, this thesis proposes a design methodology for the building of such intelligent multi-agent applications called MOMA. This alternative approach focuses on the development of ontology as the driving force of the development process. By allowing the domain and characteristics of utilisation and experimentation to be dictated through ontology, researchers and domain experts can specify the agent application without any knowledge of agent design and lower level programming. Through the use of a structured ontology model and the use of integrated tools, this approach contributes towards the building of semantically aware intelligent applications for use by researchers and domain experts. MOMA is evaluated through case studies in two different domains: financial services and e-Health.

We investigate the problem of developing a formal language for specifying and reasoning about real-time embedded distributed computer systems. In particular we investigate the problem of developing a theoretical framework for specifying and analyzing different aspects of real-time embedded distributed coordination. In addition to the theoretical framework we also consider the practical aspects of developing real-time embedded distributed systems. (For complete abstract open document)

Shared ride systems match the travel demand of transport a client with the supply of vehicles, or hosts, so that the client find rides to their destinations. A peer-to-peer shared ride system allows a client to find rides in an ad-hoc manner, by negotiating directly with nearby hosts via radio-based communication. Such a peer-to-peer shared ride system has to deal with various types of hosts, such as private cars, taxicabs and mass transit vehicles. Agents, i.e. a client and hosts, have diverse behaviors in such systems. Their different behaviors affect the negotiation process, and consequently the travel choices. Preliminary research (Winter et al. 2005) has investigated peer-to-peer shared ride systems with homogeneous hosts and immobile client. This thesis extends their work to multiple types of agents. It focuses on what are typical agent behaviors in peer-to-peer shared ride systems, and how these behaviors affect negotiation processes in a dynamic transport environment. (For complete abstract open document)

The cooperative control of large-scale multi-agent systems has gained a significant interest in recent years from the robotics and control communities for multi-vehicle control. One motivator for the growing interest is the application of spatially and temporally distributed multiple unmanned aerial vehicle (UAV) systems for distributed sensing and collaborative operations. In this research, the multi-vehicle control problem is addressed using a decentralised control system. The work aims to provide a decentralised control framework that synthesises the self-organised and coordinated behaviour of natural swarming systems into cooperative UAV systems. The control system design framework is generalised for application into various other multi-agent systems including cellular robotics, ad-hoc communication networks, and modular smart-structures. The approach involves identifying suitable relationships that describe the behaviour of the UAVs within the swarm and the interactions of these behaviours to produce purposeful high-level actions for system operators. A major focus concerning the research involves the development of suitable analytical tools that decomposes the general swarm behaviours to the local vehicle level. The control problem is approached using two-levels of abstraction; the supervisory level, and the local vehicle level. Geometric control techniques based on differential geometry are used at the supervisory level to reduce the control problem to a small set of permutation and size invariant abstract descriptors. The abstract descriptors provide an open-loop optimal state and control trajectory for the collective swarm and are used to describe the intentions of the vehicles. Decentralised optimal control is implemented at the local vehicle level to synthesise self-organised and cooperative behaviour. A deliberative control scheme is implemented at the local vehicle level that demonstrates autonomous, cooperative and optimal behaviour whilst the preserv ing precision and reliability at the local vehicle level.

Research Doctorate - Doctor of Philosophy (PhD) / Mobile agents are programs that travel autonomously through a computer network in order to perform some computation or gather information on behalf of a human user or an application. In the last several years numerous applications of mobile agents have emerged, including e-commerce. However, mobile agent paradigm introduces a number of security threats both to the agents themselves and to the servers that they visit. This thesis gives an overview of the main security issues related to the mobile agent paradigm. The first part of the thesis focuses on security of mobile agent itself. In this part, we propose a new coupling technique based on trust as a social control to work together with existing traditional security mechanisms. It relies on the “reputation” of the hosts in the itinerary and ensures that the agent succeeds in accomplishing its task with a high probability. Due to the fact that the coupling technique requires an agent’s itinerary to be known in advance, we introduce two new concepts: a “Scout mobile agent”, whose primary purpose is to determine the itinerary required for accomplishing a given task, and a “Routed mobile agent”, which operates with an itinerary known in advance. This enables the Routed agent to incorporate various security mechanisms, including our new coupling technique. Our Routed agent technique is also applicable independently of the Scout agent, whenever the itinerary and the trust values of the platforms in the itinerary are known. We also proposed a Petrol Station as an entity that would provide a service to other entities, in the form of certifying mobile agents and equipping them with safe itinerary based on trust score and applying the Routed agent. In the second part of the thesis, we shed some light on the security of mobile agent platforms as it is considered more critical than the security of agents. In particular, we consider a scenario where a platform hosts a database containing confidential individual information and allows mobile agentstoquery the data base. This mobile agent maybe behave maliciously which is similar to an intruder in the Statistical Disclosure Control(SDC), where measuring disclosure risk is still considered as a difficult and only partly solved problem[111]. We introduce a scenario that is not adequately covered by any of the previous discloser risk measures. Shannon’s entropy can be considered a satisfactory measure for the disclosure risk that is related to the exact compromise. However, in the case of approximate compromise, we argue that Shannon’s entropy does not express precisely the intruder’s knowledge about a particular confidential value. We introduce a novel disclosure risk measure that is based on Shannon’s entropy but covers both exact and approximate compromise. The main advantage of our measure over previously proposed measures that it gives careful consideration to the attribute values in addition to the probabilities with which the values occur. We use a dynamic programming algorithm to calculate the disclosure risk for various levels of approximate compromise. Importantly, our proposed measure is independent of the applied SDC technique. Finally, we show how this measure can be used to evaluate the security mechanisms for protecting privacy in statistical databases and data mining. We conduct extensive experiments and apply our proposed security measure to three different data sets protected by three different SDC techniques, namely Sampling, Query Restriction, and Noise Addition.

Software agents and multi-agent systems (MAS) have grown into a very active area of research and commercial development activity. There are many current emerging real-world applications spanning multitude of diverse domains. In the context of agents, ontology has been widely recognised for their significant benefits to interoperability, reusability, and both development and operational aspects of agent systems and applications. Ontology-based multi-agent systems (OBMAS) exploit these advantages in providing intelligent and semantically aware applications. In addressing the lack of support for ontology in existing methodologies for multi-agent development, this thesis proposes a design methodology for the building of such intelligent multi-agent applications called MOMA. This alternative approach focuses on the development of ontology as the driving force of the development process. By allowing the domain and characteristics of utilisation and experimentation to be dictated through ontology, researchers and domain experts can specify the agent application without any knowledge of agent design and lower level programming. Through the use of a structured ontology model and the use of integrated tools, this approach contributes towards the building of semantically aware intelligent applications for use by researchers and domain experts. MOMA is evaluated through case studies in two different domains: financial services and e-Health.

Doctor of Philosophy / Computational models of motivation are software reasoning processes designed to direct, activate or organise the behaviour of artificial agents. Models of motivation inspired by psychological motivation theories permit the design of agents with a key reasoning characteristic of natural systems: experience-based attention focus. The ability to focus attention is critical for agent behaviour in complex or dynamic environments where only small amounts of available information is relevant at a particular time. Furthermore, experience-based attention focus enables adaptive behaviour that focuses on different tasks at different times in response to an agent’s experiences in its environment. This thesis is concerned with the synthesis of motivation and reinforcement learning in artificial agents. This extends reinforcement learning to adaptive, multi-task learning in complex, dynamic environments. Reinforcement learning algorithms are computational approaches to learning characterised by the use of reward or punishment to direct learning. The focus of much existing reinforcement learning research has been on the design of the learning component. In contrast, the focus of this thesis is on the design of computational models of motivation as approaches to the reinforcement component that generates reward or punishment. The primary aim of this thesis is to develop computational models of motivation that extend reinforcement learning with three key aspects of attention focus: rhythmic behavioural cycles, adaptive behaviour and multi-task learning in complex, dynamic environments. This is achieved by representing such environments using context-free grammars, modelling maintenance tasks as observations of these environments and modelling achievement tasks as events in these environments. Motivation is modelled by processes for task selection, the computation of experience-based reward signals for different tasks and arbitration between reward signals to produce a motivation signal. Two specific models of motivation based on the experience-oriented psychological concepts of interest and competence are designed within this framework. The first models motivation as a function of environmental experiences while the second models motivation as an introspective process. This thesis synthesises motivation and reinforcement learning as motivated reinforcement learning agents. Three models of motivated reinforcement learning are presented to explore the combination of motivation with three existing reinforcement learning components. The first model combines motivation with flat reinforcement learning for highly adaptive learning of behaviours for performing multiple tasks. The second model facilitates the recall of learned behaviours by combining motivation with multi-option reinforcement learning. In the third model, motivation is combined with an hierarchical reinforcement learning component to allow both the recall of learned behaviours and the reuse of these behaviours as abstract actions for future learning. Because motivated reinforcement learning agents have capabilities beyond those of existing reinforcement learning approaches, new techniques are required to measure their performance. The secondary aim of this thesis is to develop metrics for measuring the performance of different computational models of motivation with respect to the adaptive, multi-task learning they motivate. This is achieved by analysing the behaviour of motivated reinforcement learning agents incorporating different motivation functions with different learning components. Two new metrics are introduced that evaluate the behaviour learned by motivated reinforcement learning agents in terms of the variety of tasks learned and the complexity of those tasks. Persistent, multi-player computer game worlds are used as the primary example of complex, dynamic environments in this thesis. Motivated reinforcement learning agents are applied to control the non-player characters in games. Simulated game environments are used for evaluating and comparing motivated reinforcement learning agents using different motivation and learning components. The performance and scalability of these agents are analysed in a series of empirical studies in dynamic environments and environments of progressively increasing complexity. Game environments simulating two types of complexity increase are studied: environments with increasing numbers of potential learning tasks and environments with learning tasks that require behavioural cycles comprising more actions. A number of key conclusions can be drawn from the empirical studies, concerning both different computational models of motivation and their combination with different reinforcement learning components. Experimental results confirm that rhythmic behavioural cycles, adaptive behaviour and multi-task learning can be achieved using computational models of motivation as an experience-based reward signal for reinforcement learning. In dynamic environments, motivated reinforcement learning agents incorporating introspective competence motivation adapt more rapidly to change than agents motivated by interest alone. Agents incorporating competence motivation also scale to environments of greater complexity than agents motivated by interest alone. Motivated reinforcement learning agents combining motivation with flat reinforcement learning are the most adaptive in dynamic environments and exhibit scalable behavioural variety and complexity as the number of potential learning tasks is increased. However, when tasks require behavioural cycles comprising more actions, motivated reinforcement learning agents using a multi-option learning component exhibit greater scalability. Motivated multi-option reinforcement learning also provides a more scalable approach to recall than motivated hierarchical reinforcement learning. In summary, this thesis makes contributions in two key areas. Computational models of motivation and motivated reinforcement learning extend reinforcement learning to adaptive, multi-task learning in complex, dynamic environments. Motivated reinforcement learning agents allow the design of non-player characters for computer games that can progressively adapt their behaviour in response to changes in their environment.