The first goal of the present work focuses on the need for different rationing methods of the The Global Change and Financial Transition (GFT) work- ing group at the Potsdam Institute for Climate Impact Research (PIK): I provide a toolbox which contains a variety of rationing methods to be ap- plied to micro-economic disequilibrium models of the lagom model family. This toolbox consists of well known rationing methods, and of rationing methods provided specifically for lagom. To ensure an easy application the toolbox is constructed in modular fashion. The second goal of the present work is to present a micro-economic labour market where heterogenous labour suppliers experience consecu- tive job opportunities and need to decide whether to apply for employ- ment. The labour suppliers are heterogenous with respect to their qualifi- cations and their beliefs about the application behaviour of their competi- tors. They learn simultaneously – in Bayesian fashion – about their individ- ual perceived probability to obtain employment conditional on application (PPE) by observing each others’ application behaviour over a cycle of job opportunities. / In vorliegender Arbeit beschäftige ich mich mit zwei Dingen. Zum einen entwickle ich eine Modellierungstoolbox, die verschiedene Rationierungs- methoden enthält. Diese Rationierungsmethoden sind entweder aus der Literatur bekannt, oder wurden speziell für die lagom Modellfamilie ent- wickelt. Zum anderen zeige ich, dass man mit Hilfe von Rationierungsmetho- den aus der Modellierungstoolbox einen fiktiven Arbeitsmarkt modellie- ren kann. Auf diesem agieren arbeitssuchende Agenten, die heterogen im Bezug auf ihre Qualifikation und ihre Vorstellungen über das Bewerbungs- verhalten ihrer Konkurrenten sind. Sie erfahren aufeinanderfolgende Job- angebote und beobachten das Bewerbungsverhalten ihrer Konkurrenten, um in Bayesianischer Weise über ihre individuelle Wahrscheinlichkeit eine Stelle zu erhalten zu lernen.

Robots and other computational agents are increasingly becoming part of our daily lives. They will need to be able to learn to perform new tasks, adapt to novel situations, and understand what is wanted by their human users, most of whom will not have programming skills. To achieve these ends, agents must learn from humans using methods of communication that are naturally accessible to everyone. This thesis presents and formalizes interactive shaping, one such teaching method, where agents learn from real-valued reward signals that are generated by a human trainer. In interactive shaping, a human trainer observes an agent behaving in a task environment and delivers feedback signals. These signals are mapped to numeric values, which are used by the agent to specify correct behavior. A solution to the problem of interactive shaping maps human reward to some objective such that maximizing that objective generally leads to the behavior that the trainer desires. Interactive shaping addresses the aforementioned needs of real-world agents. This teaching method allows human users to quickly teach agents the specific behaviors that they desire. Further, humans can shape agents without needing programming skills or even detailed knowledge of how to perform the task themselves. In contrast, algorithms that learn autonomously from only a pre-programmed evaluative signal often learn slowly, which is unacceptable for some real-world tasks with real-world costs. These autonomous algorithms additionally have an inflexibly defined set of optimal behaviors, changeable only through additional programming. Through interactive shaping, human users can (1) specify and teach desired behavior and (2) share task knowledge when correct behavior is already indirectly specified by an objective function. Additionally, computational agents that can be taught interactively by humans provide a unique opportunity to study how humans teach in a highly controlled setting, in which the computer agent’s behavior is parametrized. This thesis answers the following question. How and to what extent can agents harness the information contained in human-generated signals of reward to learn sequential decision-making tasks? The contributions of this thesis begin with an operational definition of the problem of interactive shaping. Next, I introduce the tamer framework, one solution to the problem of interactive shaping, and describe and analyze algorithmic implementations of the framework within multiple domains. This thesis also proposes and empirically examines algorithms for learning from both human reward and a pre-programmed reward function within an MDP, demonstrating two techniques that consistently outperform learning from either feedback signal alone. Subsequently, the thesis shifts its focus from the agent to the trainer, describing two psychological studies in which the trainer is manipulated by either changing their perceived role or by having the agent intentionally misbehave at specific times; we examine the effect of these manipulations on trainer behavior and the agent’s learned task performance. Lastly, I return to the problem of interactive shaping, for which we examine a space of mappings from human reward to objective functions, where mappings differ by how much the agent discounts reward it expects to receive in the future. Through this investigation, a deep relationship is identified between discounting, the level of positivity in human reward, and training success. Specific constraints of human reward are identified (i.e., the “positive circuits” problem), as are strategies for overcoming these constraints, pointing towards interactive shaping methods that are more effective than the already successful tamer framework. / text

Agent-oriented software engineering is faced with challenges that impact on the adoption of agent technology by the wider software engineering community. This is generally due to lack of adequate comprehension of the concepts of agent technology. This thesis is based on the premise that the comprehension of the concepts of and the adoption of agent technology can be improved. Two approaches are explored: the first approach is the analysis and structuring of the interactions in multiagent systems; the second approach is sharing of experiences of what works and what does not in agent-oriented software engineering using software patterns. While analysis of interactions in multiagent systems improves the understanding of the behaviour of multiagent systems, sharing multiagent system development experience improves the understanding of the concepts of agent technology as well as the challenges that face the engineering of multiagent systems. It is therefore believed that interaction analysis and experience sharing can enhance the comprehension of agent technology and hence, the adoption of the technology by the wider community of software practitioners. This thesis addresses the challenges facing agent-oriented software engineering by presenting a dedicated approach for developing agent interaction protocols to guide the interactions in a multiagent system; and a comprehensive framework for classifying, analyzing and describing agent-oriented patterns for the purpose of sharing multiagent systems development experiences.

Conversational agents such as Siri, Google and Alexa are growing in popularity, and Artificial Intelligence in the form of natural language processing utilized by these agents is becoming more available and capable with time. Understanding how conversational agents are used today and what implications it has for our daily lives is important if this trend is going to continue. In this thesis I present how children interact with conversational agents today and the implications this has for families. Four families with children in the age of 6-9 were interviewed regarding how children interact with conversational agents today, what concerns parents have and how they view the agent. The results show that children regard the conversational agent as a tool, and that the primary interactions are entertainment and exploration. Parents were concerned what the agent might say when they are not there, and do not feel in control of the agent. In the beginning children have high expectations on the capabilities of the agent but quickly assess the capabilities through experimentation.

京都大学 / 0048 / 新制・課程博士 / 博士(情報学) / 甲第17925号 / 情博第507号 / 新制||情||90(附属図書館) / 30745 / 京都大学大学院情報学研究科知能情報学専攻 / (主査)教授 西田 豊明, 教授 河原 達也, 教授 黒橋 禎夫 / 学位規則第4条第1項該当 / Doctor of Informatics / Kyoto University / DGAM

京都大学 / 0048 / 新制・課程博士 / 博士(情報学) / 甲第19110号 / 情博第556号 / 新制||情||98(附属図書館) / 32061 / 京都大学大学院情報学研究科知能情報学専攻 / (主査)教授 西田 豊明, 教授 乾 敏郎, 教授 河原 達也 / 学位規則第4条第1項該当 / Doctor of Informatics / Kyoto University / DGAM

Past research on multi-agent simulation with cooperative reinforcement learning (RL) for homogeneous agents focuses on developing sharing strategies that are adopted and used by all agents in the environment. These sharing strategies are considered to be reciprocating because all participating agents have a predefined agreement regarding what type of information is shared, when it is shared, and how the participating agent's policies are subsequently updated. The sharing strategies are specifically designed around manipulating this shared information to improve learning performance. This thesis targets situations where the assumption of a single sharing strategy that is employed by all agents is not valid. This work seeks to address how agents with no predetermined sharing partners can exploit groups of cooperatively learning agents to improve learning performance when compared to Independent learning. Specifically, several intra-agent methods are proposed that do not assume a reciprocating sharing relationship and leverage the pre-existing agent interface associated with Q-Learning to expedite learning. The other agents' functions and their sharing strategies are unknown and inaccessible from the point of view of the agent(s) using the proposed methods. The proposed methods are evaluated on physically embodied agents in the multi-agent cooperative robotics field learning a navigation task via simulation. The experiments conducted focus on the effects of the following factors on the performance of the proposed non-reciprocating methods: scaling the number of agents in the environment, limiting the communication range of the agents, and scaling the size of the environment. / Master of Science

Flourishing economy, rapid industrialization and increasing trend of motorization have been shaping societies in the developing countries like India in an unprecedented manner.Infrastructure backlog amid such rapid growth in all imaginable directions has heavily exacerbated the urban transport crisis in these countries by alarming increase in vehicular travel demand, road fatalities, and environmental pollution. To address urban transport challenges, the necessary development and implementation of effective transport planning and policies have generally lagged in the developing countries compared to that seen in the developed countries due to several constraints including resource constraints, knowledge constraints, institutional constraints and so on. However, in the recent past, with the rapid development seen by several emerging economies and the explosive growth in transportation infrastructure investment, there is a growing interest in the development and implementati on of advanced travel demand modeling systems in developing countries. But lack of necessary research and exploration of travel behavior in a developing country context has left very limited knowledge for us to understand the extent of applicability of these advanced theories and methodologies in a different socio-cultural perspective. Assessing the practical relevance of the subject, this research adopts a comprehensive approach to explore the activity engagement pattern and time use behavior from a developing country standpoint. To accomplish this goal, a series of empirical and analytical studies are performed on a household travel survey data set available from Thane Metropolitan Area in India. The study also introduces new concepts and facilitates enhancements of existing modeling methodologies in the field of travel behavior and time use research. The study results provide very insightful findings and plausible interpretations consistent with a developing country perspective reco gnizing a wide spectrum of differences and similarities in activity patterns and time use behavior between a developed and a developing country. Specified model structures are meaningfully able to incorporate various socio-cultural and institutional constraints and reflected sensitivity to the behavioral variability between the contexts suggesting that advanced analytical techniques may be satisfactorily applied on the data set from developing countries which may contribute important ingredients in the development of advanced activity-based model system in the countries like India.

Networked telerobots are operated by humans through remote interactions and have found applications in unstructured environments, such as outer space, underwater, telesurgery, manufacturing etc. In precision agricultural robotics, target monitoring, recognition and detection is a complex task, requiring expertise, hence more efficiently performed by collaborative human-robot systems. A HUB is an online portal, a platform to create and share scientific and advanced computing tools. HUB-CI is a similar tool developed by PRISM center at Purdue University to enable cyber-augmented collaborative interactions over cyber-supported complex systems. Unlike previous HUBs, HUB-CI enables both physical and virtual collaboration between several groups of human users along with relevant cyber-physical agents. This research, sponsored in part by the Binational Agricultural Research and Development Fund (BARD), implements the HUB-CI model to improve the Collaborative Intelligence (CI) of an agricultural telerobotic system for early detection of anomalies in pepper plants grown in greenhouses. Specific CI tools developed for this purpose include: (1) Spectral image segmentation for detecting and mapping to anomalies in growing pepper plants; (2) Workflow/task administration protocols for managing/coordinating interactions between software, hardware, and human agents, engaged in the monitoring and detection, which would reliably lead to precise, responsive mitigation. These CI tools aim to minimize interactions’ conflicts and errors that may impede detection effectiveness, thus reducing crops quality. Simulated experiments performed show that planned and optimized collaborative interactions with HUB-CI (as opposed to ad-hoc interactions) yield significantly fewer errors and better detection by improving the system efficiency by between 210% to 255%. The anomaly detection method was tested on the spectral image data available in terms of number of anomalous pixels for healthy plants, and plants with stresses providing statistically significant results between the different classifications of plant health using ANOVA tests (P-value = 0). Hence, it improves system productivity by leveraging collaboration and learning based tools for precise monitoring for healthy growth of pepper plants in greenhouses.

L'essor des travaux en informatique affective voit la naissance de diverses questions de recherches pour étudier les interactions agents /humains. Parmi elles, se pose la question de l'impact des relations interpersonnelles sur les stratégies de communications. Les interactions entre un agent conversation et un utilisateur humain prennent généralement place dans des environnements collaboratifs où les interlocuteurs partagent des buts communs. La relation interpersonnelle que les individus créent durant leurs interactions affecte leurs stratégies de communications. Par ailleurs, des individus qui collaborent pour atteindre un but commun sont généralement amenés à négocier. Ce type de négociation permet aux négociateurs d'échanger des informations afin de mieux collaborer. L'objectif cette thèse est d'étudier l'impact de la relation interpersonnelle de dominance sur les stratégies de négociation collaborative entre un agent et un humain. Ce travail se base sur des études en psychologie sociale qui ont défini les comportements liés à la manifestation de la dominance dans une négociation. Nous proposons un modèle de négociation collaborative dont le modèle décisionnel est régi par la relation de dominance. En effet, en fonction de sa position dans le spectre de dominance, l'agent est capable d'exprimer une stratégie de négociation spécifique. En parallèle, l'agent simule une relation interpersonnelle de dominance avec son interlocuteur. Pour ce faire, nous avons doté l'agent d'un modèle de théorie de l'esprit qui permet à l'agent de raisonner sur les comportements de son interlocuteur afin de prédire sa position dans le spectre de dominance. Ensuite, il adapte sa stratégie de négociation vers une stratégie complémentaire à celle détectée chez son interlocuteur. Nos résultats ont montré que les comportements de dominance exprimés par notre agent sont correctement perçus. Par ailleurs, le modèle de la théorie de l'esprit est capable de faire de bonnes prédictions avec seulement une représentation partielle de l'état mental de l'interlocuteur. Enfin, la simulation de la relation interpersonnelle de dominance a un impact positif sur la négociation: les négociateurs atteignent de bon taux de gains communs. De plus, la relation de dominance augmente le sentiment d'appréciation entre les négociateurs et la négociation est perçue comme confortable. / The rise of work in affective computing sees the emergence of various research questions to study agent / human interactions. Among them raises the question of the impact of interpersonal relations on the strategies of communication. Human/agent interactions usually take place in collaborative environments in which the agent and the user share common goals. The interpersonal relations which individuals create during their interactions affects their communications strategies. Moreover, individuals who collaborate to achieve a common goal are usually brought to negotiate. This type of negotiation allows the negotiators to efficiently exchange information and their respective expertise in order to better collaborate. The objective of this thesis is to study the impact of the interpersonal relationship of dominance on collaborative negotiation strategies between an agent and a human. This work is based on studies from social psychology to define the behaviours related to the manifestation of dominance in a negotiation. We propose a collaborative negotiation model whose decision model is governed by the interpersonal relation of dominance. Depending on its position in the dominance spectrum, the agent is able to express a specific negotiation strategy. In parallel, the agent simulates an interpersonal relationship of dominance with his interlocutor. To this aim, we provided the agent with a model of theory of mind that allows him to reason about the behaviour of his interlocutor in order to predict his position in the dominance spectrum. Afterwards, the agent adapts his negotiation strategy to complement the negotiation strategy detected in the interlocutor. Our results showed that the dominance behaviours expressed by our agent are correctly perceived by human participants. Furthermore, our model of theory of mind is able de make accurate predictions of the interlocutor behaviours of dominance with only a partial representation of the other's mental state. Finally, the simulation of the interpersonal relation of dominance has a positive impact on the negotiation: the negotiators reach a good rate of common gains and the negotiation is perceived comfortable which increases the liking between the negotiators.