There has been an increased necessity to search for alternative transportation methods, mainly driven by limited fuel availability and the negative impacts of climate change and exhaust emissions. These factors have lead to increased regulations and a societal shift towards a cleaner and more e cient transportation system. Automotive and technology companies need to be looking for ways to reshape mobility, enhance safety, increase accessibility, and eliminate the ine ciencies of the current transportation system in order to address such a shift. Hybrid vehicles are a popular solution that address many of these goals. In order to fully realize the bene ts of hybrid vehicle technology, the power distribution decision needs to be optimized. In the past, global optimization techniques have been dismissed because they require knowledge of the journey of the vehicle in advance, and are generally computationally extensive. Recent advancements in technologies, like sensors, cameras, lidar, GPS, Internet of Things, and computing processors, have changed the basic assumptions that were made during the vehicle design process. In particular, it is becoming increasingly possible to know future driving conditions. In addition to this, autonomous vehicle technology is addressing many safety and e ciency concerns. This thesis considers and integrates recent technologies when de ning a new approach to hybrid vehicle supervisory controller design and optimization. The dynamic programming algorithm has been systematically applied to an autonomous vehicle with a power-split hybrid electric powertrain. First, a more realistic driving cycle, the Journey Mapping cycle, is introduced to test the performance of the proposed control strategy under more appropriate conditions. Techniques such as vectorization and partitioning are applied to improve the computational e ciency of the dynamic programming algorithm, as it is applied to the hybrid vehicle energy management problem. The dynamic programming control algorithm is benchmarked against rule-based algorithms to substantively measure its bene ts. It is proven that the DP solution improves vehicle performance by at least 9 to 17% when simulated over standard drive cycles. In addition, the dynamic programming solution improves vehicle performance by at least 32 to 39% when simulated over more realistic conditions. The results emphasize the bene ts of optimal hybrid supervisory control and the need to design and test vehicles over realistic driving conditions. Finally, the dynamic programming solution is applied to the process of adaptive control calibration. The particle swarm optimization algorithm is used to calibrate control variables to match an existing controller's operation to the dynamic programming solution. / Thesis / Master of Applied Science (MASc)

The identification of linear structural dynamic systems has been dealt with extensively in past studies. Identification methods for nonlinear structures have also been introduced in previous articles, including procedures based on the method of multiple scales, iterative and noniterative direct methods, and state space mappings. Here, a procedure is introduced for the identification of nonlinear structural dynamic systems which is readily applicable to simple as well as more complex multiple degree of freedom systems. The procedure is based on multiple step integration methods for the solution of differential equations. The multiple step integration procedure and the iterative direct method are applied to a number of nonlinear single degree of freedom examples, and are applied to a simple two degrees of freedom example as well. RMS based noise is added to a simulated measured response in order to monitor the effects of measurement errors on the procedures. The input data is filtered before final processing in the identification algorithms. The multistep algorithm is compared to the iterative direct method on the basis of criteria such as accuracy, ease of use, and numerical efficiency. / Master of Science

The parasitic weeds belonging to the genus Striga are among the world's most tenacious, prolific and destructive agricultural pests. Crop loss estimates due to Striga infestations can reach 100 percent. Furthermore, the weeds' affinity for low-fertility soils and low rainfall means that those farming the most marginal lands are most severely affected. Nonetheless, subsistence farmer have yet to adopt seemingly beneficial control practices to any appreciable degree. This paper develops a bio-economic model capable of identifying: (1) affordable, effective Striga control practices consistent with the resource constraints of subsistence farmers; and (2) barriers to the adoption of those practices. The model is comprised of two components: a biological component modeling Striga population dynamics, and an economic component representing the production opportunity set, resource constraints, and price parameters farmers face. The model is applied to two zones in Northwestern Mali, Sirakorola and Mourdiah, and solved using non-linear, dynamic programming. Data collected by the USAID IPM-CRSP/Mali project are used to specify the economic parameters of the model. A new technique for estimating the lower bound of a farmer's production planning horizon is also developed and employed in the application of the model to Sirakorola and Mourdiah. The results of several model scenarios indicate that the availability of information regarding the efficacy of Striga control practices is a primary barrier to their adoption by subsistence farmers. The movement of Striga seed between fields, however, is of limited importance. The "optimal control practices" identified by the model depend on the size and demographic composition of the production unit (UP), the zone in which the UP is located, and the cash budget available to the UP. At low budget levels, the model suggests planting millet without fertilizer at a high density in Sirakorola and a low density in Mourdiah. At high budget levels, the model suggests planting millet at a high density in both zones while applying urea. The benefits of adopting the optimal set of practices are presented in both nutritional and financial terms, and can reach as much as a ten-fold increase in the nutritional content of and financial returns to a harvest. / Ph. D.

In this dissertation we introduce a new class of pursuit-evasion games; the herding problem. Unlike regular pursuit evasion games where the pursuer aims to hunt the evader the objective of the pursuer in this game is to drive the evader to a certain location on the x-y grid. The dissertation deals with this problem using two different methodologies. In the first, the problem is introduced in the continuous-time, continuous-space domain. The continuous time model of the problem is proposed, analyzed and we came up with an optimal control law for the pursuer is obtained so that the evader is driven to the desired destination position in the x-y grid following the local shortest path in the Euler Lagrange sense. Then, a non-holonomic realization of the two agents is proposed. In this and we show that the optimal control policy is in the form of a feedback control law that enables the pursuer to achieve the same objective using the shortest path. The second methodology deals with the discrete model representation of the problem. In this formulation, the system is represented by a finite di-graph. In this di-graph, each state of the system is represented by a node in the graph. Applying dynamic programming technique and shortest path algorithms over the finite graph representing the system, we come up with the optimal control policy that the pursuer should follow to achieve the desired goal. To study the robustness, we formulate the problem in a stochastic setting also. We analyze the stochastic model and derive an optimal control law in this setting. Finally, the case with active evader is considered, the optimal control law for this case is obtained through the application of dynamic programming technique. / Ph. D.

In this dissertation, I study two topics in the context of revenue management. The First topic involves building a mathematical model to analyze the competition between many retailers who can change the price of their respective products in real time. I develop a game-theoretic model for the dynamic price competition where each retailer's objective is to maximize its own expected total revenue. I use the Nash equilibrium to predict market equilibrium and provide managerial insights into how each retailer should take into account its competitors' behavior when setting the price. The second topic involves working with Amtrak, the national railroad passenger corporation, to develop a revenue management model. The revenue management department of Amtrak provides the sales data of Auto Train, a service of Amtrak that allows passengers to bring their vehicles on the train. I analyze the demand structure from sales data and build a mathematical model to describe the sales process for Auto Train. I further develop an algorithm to calculate the optimal pricing strategy that yields the maximum revenue. Because of the distinctive service provided by Auto Train, my findings make important contribution to the revenue management literature. / Ph. D.

The increasing rate of deforestation, which began decades ago, has significantly impacted on ecosystem services. In this context, secondary forests have emerged as crucial elements in mitigating environmental degradation and restoration. This study is motivated by the need to understand the reforestation management in secondary forests to prevent irreversible ecosystem damage. We begin by setting the drift and volatility in stochastic primary forests. However, it is more manageable to take control of replantation. We employ a dynamic programing approach, integrating ecological and economic perspectives to assess ecosystem services. To simulate a real-world case, we investigate the model in the Brazil Amazon Basin. Special attention is given to the outcome at the turning point, tipping point, and transition point, considering a critical threshold beyond which recovery becomes implausible. Our findings suggest that reducing tenure costs has advantages, while substitution between primary and secondary forests is not necessarily effective in postponing ecosystem collapse. This research contributes to a broader goal of sustainable forest management and offers strategic guidance for future reforestation initiatives in the Amazon Basin and similar ecosystems worldwide. / Master of Science / Deforestation has been drawing attention from institutions since the 1940s, and this global issue has been discussed for its negative impacts and the ways to restore what has been lost. Reforestation initiatives introduced by global environmental organizations consider forest plantations essential in re-establishing trees and the natural ecosystem. This study aims to investigate how different techniques target the growth of secondary forests to mitigate the irreversible damage of ecosystem services. Our research begins by defining the uncertain primary forests. Primary forests and deforestation face long-term climate changes and immediate shocks like fires, droughts, and human activities, meanwhile, policymakers have difficulties predicting and fully controlling them. We integrate considerations of ecology and economy to the ecosystem functioning, introducing stochasticity in deforestation into our dynamic optimization problem. We apply our models to the Brazil Amazon Basin, a region known for its diverse tropical forests and vast cases of deforestation. We pay close attention to the timing of tipping point that leads to ecosystem collapse, the turning point where reforestation rate catches up with deforestation rate, and the moment of forest type transition. Through simulation and sensitivity analysis, we gain a better grasp on guiding the management of secondary forests under uncertain conditions. Our results indicate that reforestation approaches that lower tenure costs can be beneficial, but merely substituting primary forests cannot necessarily delay an ecosystem collapse. This paper provides practical insights for policymakers, local communities, and international organizations.

Background: Dynamic programming algorithms provide exact solutions to many problems in computational biology, such as sequence alignment, RNA folding, hidden Markov models (HMMs), and scoring of phylogenetic trees. Structurally analogous algorithms compute optimal solutions, evaluate score distributions, and perform stochastic sampling. This is explained in the theory of Algebraic Dynamic Programming (ADP) by a strict separation of state space traversal (usually represented by a context free grammar), scoring (encoded as an algebra), and choice rule. A key ingredient in this theory is the use of yield parsers that operate on the ordered input data structure, usually strings or ordered trees. The computation of ensemble properties, such as a posteriori probabilities of HMMs or partition functions in RNA folding, requires the combination of two distinct, but intimately related algorithms, known as the inside and the outside recursion. Only the inside recursions are covered by the classical ADP theory. Results: The ideas of ADP are generalized to a much wider scope of data structures by relaxing the concept of parsing. This allows us to formalize the conceptual complementarity of inside and outside variables in a natural way. We demonstrate that outside recursions are generically derivable from inside decomposition schemes. In addition to rephrasing the well-known algorithms for HMMs, pairwise sequence alignment, and RNA folding we show how the TSP and the shortest Hamiltonian path problem can be implemented efficiently in the extended ADP framework. As a showcase application we investigate the ancient evolution of HOX gene clusters in terms of shortest Hamiltonian paths. Conclusions: The generalized ADP framework presented here greatly facilitates the development and implementation of dynamic programming algorithms for a wide spectrum of applications.

Approved for public release; distribution in unlimited. / In this thesis, a new fictitious play (FP) procedure is presented to solve two-person zero-sum (TPZS) Blotto games. The FP solution procedure solves TPZS games by assuming that the two players take turns selecting optimal responses to the opponent's strategy observed so far. It is known that FP converges to an optimal solution, and it may be the only realistic approach to solve large games. The algorithm uses dynamic programming (DP) to solve FP subproblems. Efficiency is obtained by limiting the growth of the DP state space. Blotto games are frequently used to solve simple missile defense problems. While it may be unlikely that the models presented in this paper can be used directly to solve realistic offense and defense problems, it is hoped that they will provide insight into the basic structure of optimal and near-optimal solutions to these important, large games, and provide a foundation for solution of more realistic, and more complex, problems. / Captain, Republic of Korea Air Force

Applications of the existing steady-state plant-wide optimization and the single-scale fast-rate dynamic optimization strategies to an integrated plant with material recycle have been impeded by several factors. While the steady-state optimization formulation is very simple, the very long transient dynamics of an integrated plant have limited the optimizers execution rate to be extremely low, yielding a suboptimal performance. In contrast, performing dynamic plant-wide optimization at the same rate as local controllers requires exorbitant on-line computational load and may increase the sensitivity to high-frequency dynamics that are irrelevant to the plant-level interactions, which are slow-scale in nature. This thesis proposes a novel multi-scale dynamic optimization and control strategy suitable for an integrated plant. The dynamic plant-wide optimizer in this framework executes at a slow rate to track the slow-scale plant-wide interactions and economics, while leaving the local controllers to handle fast changes related to the local units. Moreover, this slow execution rate demands less computational and modeling requirement than the fast-rate optimizer. An important issue of this method is obtaining a suitable dynamic model when first-principles are unavailable. The difficulties in the system identification process are designing proper input signal to excite this ill-conditioned system and handling the lack of slow-scale dynamic data when the plant experiment cannot be conducted for a long time compared to the settling time. This work presents a grey-box modeling method to incorporate steady-state information to improve the model prediction accuracy. A case study of an integrated plant example is presented to address limitations of the nonlinear model predictive control (NMPC) in terms of the on-line computation and its inability to handle stochastic uncertainties. Then, the approximate dynamic programming (ADP) framework is investigated. This method computes an optimal operating policy under uncertainties off-line. Then, the on-line multi-stage optimization can be transformed into a single-stage problem, thus reducing the real-time computational effort drastically. However, the existing ADP framework is not suitable for an integrated plant with high dimensional state and action space. In this study, we combine several techniques with ADP to apply nonlinear optimal control to the integrated plant example and show its efficacy over NMPC.

The scientific domain of this thesis is optimization under uncertainty for discrete event stochastic systems. In particular, this thesis focuses on the practical implementation of the Dynamic Programming (DP) methodology to discrete event stochastic systems. Unfortunately DP in its crude form suffers from three severe computational obstacles that make its imple-mentation to such systems an impossible task. This thesis addresses these obstacles by developing and executing practical Approximate Dynamic Programming (ADP) techniques. Specifically, for the purposes of this thesis we developed the following ADP techniques. The first one is inspired from the Reinforcement Learning (RL) literature and is termed as Real Time Approximate Dynamic Programming (RTADP). The RTADP algorithm is meant for active learning while operating the stochastic system. The basic idea is that the agent while constantly interacts with the uncertain environment accumulates experience, which enables him to react more optimal in future similar situations. While the second one is an off-line ADP procedure These ADP techniques are demonstrated on a variety of discrete event stochastic systems such as: i) a three stage queuing manufacturing network with recycle, ii) a supply chain of the light aromatics of a typical refinery, iii) several stochastic shortest path instances with a single starting and terminal state and iv) a general project portfolio management problem. Moreover, this work addresses, in a systematic way, the issue of multistage risk within the DP framework by exploring the usage of intra-period and inter-period risk sensitive utility functions. In this thesis we propose a special structure for an intra-period utility and compare the derived policies in several multistage instances.