This paper analyzes the suitability of reinforcement learning (RL) for both programming and adapting situated agents. We discuss two RL algorithms: Q-learning and the Bucket Brigade. We introduce a special case of the Bucket Brigade, and analyze and compare its performance to Q in a number of experiments. Next we discuss the key problems of RL: time and space complexity, input generalization, sensitivity to parameter values, and selection of the reinforcement function. We address the tradeoffs between the built-in and learned knowledge and the number of training examples required by a learning algorithm. Finally, we suggest directions for future research.

This thesis looks at the practice of climate modelling at the urban scale in relation to projections of future climate. It responds to the question of how climate models perform in a policy context, and how these models are translated in order to have agency at the urban scale. It considers the means and circumstances through which models are constructed to selectively represent urban realities and potential realities in order to explore and reshape the built environment in response to a changing climate. This thesis is concerned with an interdisciplinary area of research and practice, while at the same time it is based on methodologies originating in science and technology studies which were later applied to architecture and planning, geography, and urban studies. Fieldwork consisted of participant-observation and interviews with three groups of practitioners: firstly, climate impacts modellers forming part of the Adaptation and Resilience in a Changing Climate (ARCC) programme; secondly, planners and adaptation policymakers in the cities of Manchester and London; and thirdly, boundary organisations such as the UK Climate Impacts Programme (UKCIP). Project and climate policy material pertinent to these projects and the case study cities were also analysed in tandem. Of particular interest was the common space shared to researchers and stakeholders where modelling results were explained, contextualised, and interrogated for policy-relevant results. This took the form of stakeholder meetings in which the limits of the models in relation to policy demands could be articulated and mediated. In considering the agency of models in relation to uncertainties, it was found that although generated in a context of applied science, models had a limited effect on policy. As such, the salience of urban climatic risk-based assessment for urban planning is restrained, because it presupposes a quantitative understanding of climate impacts that is only slowly forming due to societal and governmental pressures. This can be related both to the nature of models as sites of exploration and experimentation, and to the distribution of expertise in the climate adaptation community. Although both the research and policy communities operate partly in a common space, models and their associated tools operate at a level of sophistication that policy-makers have difficulty comprehending and integrating into planning policy beyond the level of simple guidance and messages. Adaptation in practice is constrained by a limited understanding of climate uncertainties and urban climatology, evident through the present emphasis on catch-all solutions like green infrastructure and win-win solutions rather than the empowerment of actors and a corresponding distribution of adequate resources. An analysis is provided on the means by which models and maps can shape climate adaptation at scales relevant for cities, based on considerations of how models gain agency through forms of encoded expertise like maps and the types of interaction between science and policy that they imply.