This dissertation consists of three essays related to revision games. The first essay proposes and analyzes a new model that we call “revision games,” which captures a situation where players in advance prepare their actions in a game. After the initial preparation, they have some opportunities to revise their actions, which arrive stochastically. Prepared actions are assumed to be mutually observable. We show that players can achieve a certain level of cooperation. The optimal behavior of players can be described by a simple differential equation. The second essay studies a version of revision games in which revision opportunities are asynchronous across players. In 2-player “common interest” games where there exists a best action profile for all players, this best action profile is the only equilibrium outcome of the revision game. In “opposing interest” games which are 2 x 2 games with Pareto-unranked strict Nash equilibria, the equilibrium outcome of the revision game is generically unique and corresponds to one of the stage-game Nash equilibria. Which equilibrium prevails depends on the payoff structure and on the relative frequency of the arrivals of revision opportunities for each of the players. The third essay studies a multi-agent search problem with a deadline: for instance, the situation that arises when a husband and a wife need to find an apartment by September 1. We provide an understanding of the factors that determine the positive search duration in reality. Specifically, we show that the expected search duration does not shrink to zero even in the limit as the search friction vanishes. Additionally, we find that the limit duration increases as more agents are involved, for two reasons: the ascending acceptability effect and the preference heterogeneity effect. The convergence speed is high, suggesting that the mere existence of some search friction is the main driving force of the positive duration in reality. Welfare implications and a number of discussions are provided. Results and proof techniques developed in the first two essays are useful in proving and understanding the results in the third essay. / Economics

This thesis consists of four essays in resource and behavioral economics. Resource Extraction, Capital Accumulation and Time Horizon The paper shows that relaxing the standard infinite horizon assumption can explain the patterns of exhaustible resource extraction and prices for the last century. An empirical test proposes a time horizon of roughly 28 years to be most likely. Model calibration yields an oil price which fits the falling price after WWII and suggests that the sharply increasing price after 1998 is due to scarcity. Optimal Forest Rotation under Climate Change    The scenario of forests growing faster over time, due to climate change, is analyzed. It is shown numerically that ignoring future changes is highly likely to be accurate in terms of harvesting and will cause insignificant profit losses. Tragedy of the Commons versus the Love of Variety    The opposing effects of overharvesting of renewable resources when property rights are missing and increased consumption variety, both due to trade, are analyzed. Trade increases welfare if the resource has strong regenerative power. If, instead, the resource regenerates slowly, then sufficient increases in the number of trade partners harms welfare and the stock may even collapse. Correcting policies may be very harsh and still improve upon laissez faire. The Distribution of Revealed Preferences under Social Pressure    Stated preferences, such as declared political opinions, are studied when individuals make the trade off between being true to their real opinions and conforming to a social norm. In orthodox societies, individuals will tend to either conform fully or ignore the social norm while individuals in liberal societies will tend to compromise between the two extremes. The model sheds light on phenomena such as polarization, alienation and hypocrisy. Furthermore, it suggests that orthodoxy cannot be maintained under pluralism.

Simulation, design, and control of dynamical systems play an important role in numerous scientific and industrial tasks. The need for detailed models leads to large-scale dynamical systems, posing tremendous computational difficulties when employed in numerical simulations. In order to overcome these challenges, we perform model reduction, replacing the large-scale dynamics with high-fidelity reduced representations. There exist a plethora of methods for reduced order modeling of linear systems, including the Iterative Rational Krylov Algorithm (IRKA), Balanced Truncation (BT), and Hankel Norm Approximation. However, these methods generally target stable systems and the approximation is performed over an infinite time horizon. If we are interested in a finite horizon reduced model, we utilize techniques such as Time-limited Balanced Truncation (TLBT) and Proper Orthogonal Decomposition (POD). In this dissertation we establish interpolation-based optimality conditions over a finite horizon and develop an algorithm, Finite Horizon IRKA (FHIRKA), that produces a locally optimal reduced model on a specified time-interval. Nonetheless, the quantities being interpolated and the interpolant are not the same as in the infinite horizon case. Numerical experiments comparing FHIRKA to other algorithms further support our theoretical results. Next, we discuss model reduction for nonlinear dynamical systems. For models with unstructured nonlinearities, POD is the method of choice. However, POD is input dependent and not optimal with respect to the output. Thus, we use operator splitting to integrate the best features of system theoretic approaches with trajectory based methods such as POD in order to mitigate the effect of the control inputs for the approximation of nonlinear dynamical systems. We reduce the linear terms with system theoretic methods and the nonlinear terms terms via POD. Evolving the linear and nonlinear terms separately yields the reduced operator splitting solution. We present an error analysis for this method, as well as numerical results that illustrate the effectiveness of our approach. While in this dissertation we only pursue the splitting of linear and nonlinear terms, this approach can be implemented with Quadratic Bilinear IRKA or Balanced Truncation for Quadratic Bilinear systems to further diminish the input dependence of the reduced order modeling. / Doctor of Philosophy / Simulation, design, and control of dynamical systems play an important role in numerous scientific and industrial tasks such as signal propagation in the nervous system, heat dissipation, electrical circuits and semiconductor devices, synthesis of interconnects, prediction of major weather events, spread of fires, fluid dynamics, machine learning, and many other applications. The need for detailed models leads to large-scale dynamical systems, posing tremendous computational difficulties when applied in numerical simulations. In order to overcome these challenges, we perform model reduction, replacing the large-scale dynamics with high-fidelity reduced representations. Reduced order modeling helps us to avoid the outstanding burden on computational resources. Numerous model reduction techniques exist for linear models over an infinite horizon. However, in practice we usually are interested in reducing a model over a specific time interval. In this dissertation, given a reduced order, we present a method that finds the best local approximation of a dynamical system over a finite horizon. We present both theoretical and numerical evidence that supports the proposed method. We also develop an algorithm that integrates operator splitting with model reduction to solve nonlinear models more efficiently while preserving a high level of accuracy.

Our aim is to examine the problem of optimal asset allocation for investors exhibiting a behaviour in the face of uncertainty which is not consistent with the usual axioms of Expected Utility Theory. This thesis is divided into two main parts. In the first one, comprising Chapter II, we consider an arbitrage-free discrete-time financial model and an investor whose risk preferences are represented by a possibly nonconcave utility function (defined on the non-negative half-line only). Under straightforward conditions, we establish the existence of an optimal portfolio. As for Chapter III, it consists of the study of the optimal investment problem within a continuous-time and (essentially) complete market framework, where asset prices are modelled by semi-martingales. We deal with an investor who behaves in accordance with Kahneman and Tversky's Cumulative Prospect Theory, and we begin by analysing the well-posedness of the optimisation problem. In the case where the investor's utility function is not bounded above, we derive necessary conditions for well-posedness, which are related only to the behaviour of the distortion functions near the origin and to that of the utility function as wealth becomes arbitrarily large (both positive and negative). Next, we focus on an investor whose utility is bounded above. The problem's wellposedness is trivial, and a necessary condition for the existence of an optimal trading strategy is obtained. This condition requires that the investor's probability distortion function on losses does not tend to zero faster than a given rate, which is determined by the utility function. Provided that certain additional assumptions are satisfied, we show that this condition is indeed the borderline for attainability, in the sense that, for slower convergence of the distortion function, there does exist an optimal portfolio. Finally, we turn to the case of an investor with a piecewise power-like utility function and with power-like distortion functions. Easily verifiable necessary conditions for wellposedness are found to be sufficient as well, and the existence of an optimal strategy is demonstrated.

This thesis covers a utility optimizing model designed and calibrated for agents of the Swedish economy. The main ingredient providing for this specific country is the modeling of the pension accumulation and pension benefits, which closely mimics the Swedish system. This characteristic is important since it measures one of the only two diversities between genders, that is, the income. The second characteristic is the survival probability. Except for these differences in national statistics, men and women are equal. The reminding model parameters are realistically set estimates from the surrounding economy. When using the model, firstly a baseline agent representing the entire labor force is under the microscope for evaluating the model itself. Next, one representative woman and one representative man from the private and public sectors respectively, composes a set of four samples for investigation of heterogeneity in optimality. The optimum level of consumption and risk-proportion of liquid wealth are solved by maximizing an Epstein-Zin utility function using the method of dynamic programming. The results suggests that both genders benefit from adapting the customized solutions to the problem.

The development and implementation challenges associated with a proposed load allocation paradigm for fault risk assessment and system health management based on uncertain fault diagnostic and failure prognostic information are investigated. Health management actions are formulated in terms of a value associated with improving system reliability, and a cost associated with inducing deviations from a system's nominal performance. Three simulated case study systems are considered to highlight some of the fundamental challenges of formulating and solving an optimization on the space of available supervisory control actions in the described health management architecture. Repeated simulation studies on the three case-study systems are used to illustrate an empirical approach for tuning the conservatism of health management policies by way of adjusting risk assessment metrics in the proposed health management paradigm. The implementation and testing of a real-world prognostic system is presented to illustrate model development challenges not directly addressed in the analysis of the simulated case study systems. Real-time battery charge depletion prediction for a small unmanned aerial vehicle is considered in the real-world case study. An architecture for offline testing of prognostics and decision making algorithms is explained to facilitate empirical tuning of risk assessment metrics and health management policies, as was demonstrated for the three simulated case study systems.

No description available.

We consider a single-product dynamic inventory problem where the demand distributions in each period are known and independent but with density. We assume the lead time and the fixed cost for ordering are zero and that there are no capacity constraints. There is a holding cost and a backorder cost for unfulfilled demand, which is backlogged until it is filled by another order. The problem may be nonstationary, and in fact our approximation of the optimal cost function using splines is most advantageous when demand falls suddenly. In this case the myopic policy, which is most often used in practice to calculate optimal inventory level, would be very costly. Our algorithm uses quadratic splines to approximate the optimal cost function for this dynamic inventory problem and calculates the optimal inventory level and optimal cost.

In Chapter 2, we propose several two-timescale simulation-based actor-critic algorithms for solution of infinite horizon Markov Decision Processes (MDPs) with finite state-space under the average cost criterion. On the slower timescale, all the algorithms perform a gradient search over corresponding policy spaces using two different Simultaneous Perturbation Stochastic Approximation (SPSA) gradient estimates. On the faster timescale, the differential cost function corresponding to a given stationary policy is updated and averaged for enhanced performance. A proof of convergence to a locally optimal policy is presented. Next, a memory efficient implementation using a feature-vector representation of the state-space and TD (0) learning along the faster timescale is discussed. A three-timescale simulation based algorithm for solution of infinite horizon discounted-cost MDPs via the Value Iteration approach is also proposed. An approximation of the Dynamic Programming operator T is applied to the value function iterates. A sketch of convergence explaining the dynamics of the algorithm using associated ODEs is presented. Numerical experiments on rate based flow control on a bottleneck node using a continuous-time queueing model are presented using the proposed algorithms. Next, in Chapter 3, we develop three simulation-based algorithms for finite-horizon MDPs (FHMDPs). The first algorithm is developed for finite state and compact action spaces while the other two are for finite state and finite action spaces. Convergence analysis is briefly sketched. We then concentrate on methods to mitigate the curse of dimensionality that affects FH-MDPs severely, as there is one probability transition matrix per stage. Two parametrized actor-critic algorithms for FHMDPs with compact action sets are proposed, the ‘critic’ in both algorithms learning the policy gradient. We show w.p1convergence to a set with the necessary condition for constrained optima. Further, a third algorithm for stochastic control of stopping time processes is presented. Numerical experiments with the proposed finite-horizon algorithms are shown for a problem of flow control in communication networks. Towards stochastic optimization, in Chapter 4, we propose five algorithms which are variants of SPSA. The original one measurement SPSA uses an estimate of the gradient of objective function L containing an additional bias term not seen in two-measurement SPSA. We propose a one-measurement algorithm that eliminates this bias, and has asymptotic convergence properties making for easier comparison with the two-measurement SPSA. The algorithm, under certain conditions, outperforms both forms of SPSA with the only overhead being the storage of a single measurement. We also propose a similar algorithm that uses perturbations obtained from normalized Hadamard matrices. The convergence w.p.1 of both algorithms is established. We extend measurement reuse to design three second-order SPSA algorithms, sketch the convergence analysis and present simulation results on an illustrative minimization problem. We then propose several stochastic approximation implementations for related algorithms in flow-control of communication networks, beginning with a discrete-time implementation of Kelly’s primal flow-control algorithm. Convergence with probability1 is shown, even in the presence of communication delays and stochastic effects seen in link congestion indications. Two relevant enhancements are then pursued :a) an implementation of the primal algorithm using second-order information, and b) an implementation where edge-routers rectify misbehaving flows. Also, discrete-time implementations of Kelly’s dual algorithm and primal-dual algorithm are proposed. Simulation results a) verifying the proposed algorithms and, b) comparing stability properties with an algorithm in the literature are presented.