<p> Direct policy search (DPS) has been shown to be an efficient method for identifying optimal rules (i.e., policies) for adapting a system in response to changing conditions. This dissertation describes three major advances in the usage of DPS for long-range infrastructure planning, using a specific application domain of flood risk management. We first introduce a new adaptive way to incorporate learning into DPS. The standard approach identifies policies by optimizing their average performance over a large ensemble of future states of the world (SOW). Our approach exploits information gained over time, regarding what kind of SOW is being experienced, to further improve performance via adaptive meta-policies defining how control of the system should switch between policies identified by a standard DPS approach (but trained on different SOWs). We outline the general method and illustrate it using a case study of optimal dike heightening extending the work of Garner and Keller (2018). The meta-policies identified by the adaptive algorithm show Pareto-dominance in two objectives over the standard DPS, with an overall 68% improvement in hypervolume. We also see the improved performance over three grouped SOWs based on future extreme water levels, with the hypervolume improvements of 90%, 46%, and 35% for low, medium, and high water level SOWs respectively. Additionally, we evaluate the degree of improvement achieved by different ways of implementing the algorithm (i.e., different hyperparameter values). This provides guidance for decision makers with different degrees of risk aversion, and computational budgets. </p> <p>Due to simplifying assumptions and limitations of the adaptive DPS model used in the chapter, such as uniform levee design heights, the Surge and Waves Model for Protection Systems (SWaMPS) is presented as a more realistic application of the DPS framework. SWaMPS is a process-based model of surge-based flood risk. This chapter marks the first implementation of DPS using a realistic process-based risk model. The physical process of storm surge and rainfall is simulated independently over multiple reaches, and different frequencies are explored to manage the production system in SWaMPS. The performance of the DPS algorithm is evaluated versus a static intertemporal optimization.</p> <p>The computational burden of evaluating the large ensemble of SOWs to include possible future events in DPS motivates us to apply scenario reduction methods to select representative scenarios that more efficiently span an uncertain parameter space. This allows us to reduce the runtime of the optimization process. We explore a range of data-mining tools, including principal component analysis (PCA) and clustering to reduce the scenarios. We compare the computational efficiency and quality of policies to this optimization problem with reduced ensembles of SOWs.</p>