The solution of optimal control problems through the Hamilton-Jacobi-Bellman (HJB) equation offers guaranteed satisfaction of both the necessary and sufficient conditions for optimality. However, finding an exact solution to the HJB equation is a near impossible task for many optimal control problems. This thesis presents an approximation method for solving finite-horizon optimal control problems involving nonlinear dynamical systems. The method uses finite-order approximations of the partial derivatives of the cost-to-go function, and successive higher-order differentiations of the HJB equation. Natural byproducts of the proposed method provide sensitivities of the controls to changes in the initial states, which can be used to approximate the solution to neighboring optimal control problems. For highly nonlinear problems, the method is modified to calculate control sensitivities about a nominal trajectory. In this framework, the method is shown to provide accurate control sensitivities at much lower orders of approximation. Several numerical examples are presented to illustrate both applications of the approximation method.

This research uses the optimal control theory to construct two kinds of dynamic economic systems, which are an economic system without externalities and with externalities. Within each economic system both the centralized economy model and the decentralized economy model are included. The centralized economy (a social planner) model representes a kind of ideal economy, and the goal what the social planner pursues is that the resource allocation of the society satisfies the Pareto Efficiency criteria. On the other hand, the decentralized economy model (consists of a representative producer and a representative consumer) demonstrates the real economy, where economic agents persue their own best interests. While constructing the models, goods market equilibrium, labors market equilibrium, the dynamic accumulation process of capital, and the dynamic accumulation course of pollution are under consideration. Then, I apply optimal control method to get the first order conditions, and compare these f.o.c¡¦.s to verify whether they are unanimous. This paper proves that when externalities of pollution does not exist in the dynamic economic system, the decentralized economy model can achieve the Pareto Efficiency. On the contrary, when externalities of pollution emerges in the dynamic economic system, the decentralized economy model cannot reach Pareto Efficiency. If the externalities of pollution is internalized by the dynamic decentralized economic system economy, it will coincide with Pareto Efficiency. Besides, Pigouvian tax is still an effective policy instrument. Finally, I discuss all dynamic models in this paper to find out whether there exists a long-term and stable steady state. I find stable steady state, saddle-point equilibria, do exist under certain restrictions.

This paper uses the optimal control theory to construct dynamic economic model. The primary purpose of this paper is to discuss how different policies alter the choice problem of the firm and influence the allocation of funds to existing and new research and development activities. I analyze how the fixed-cost regulatory standard and the marginal-cost standard let firm consider externality, in order to protect the consumer of asymmetric information and avoid the problem of adverse selection. The firm maximizes expected profit. At the same time I want to know how the consumer acceptance, mark and audit affect the farmer to purchase the quantity of seed and the allocation of funds . We want to discuss how different price influence the option input path, the option quantity path and the option path . I discuss the different between ultimatum and static model. Finally, I discuss dynamic models in this paper to find out whether there exists a long-term and stable steady state. Saddle-point stability exists under certain restrictions.

The object of this thesis is the study of some new financial models. The common feature is that they all involve optimal decisions. Some of the decisions take the form of a control and we enter the theory of stochastic optimal control and of Hamilton-Jacobi-Bellman (HJB) equations. Other decisions are "binary" and we deal with the theory of optimal stopping and free boundary problems. Throughout the thesis we will prefer a heuristic and intuitive approach to a too technical one which could hide the underlying ideas. In the first part we introduce the reader to option pricing, HJB equations and free boundary problems, and we review briefly the use of these mathematical tools in finance. The second part of the thesis deals with passport options. The pricing of these exotic options involves stochastic optimal control and free boundary problems. Finally, in the last part we study the end-of-the-year bonus for traders: how to optimally reward a trader?

Vehicle behaviour near the limit of adhesion is studied using linear optimal . control techniques and relatively simple vehicle models. Both time-invariant and time-varying approaches are used. Controllability is applied as a post-processing tool to analyse the resultant vehicle behaviour. First, a 4WS controller is developed using a linear time-invariant method, with a reference model control structure. Two handling objectives are defined, which are thought to provide predictable dynamics. Advantages of using a reference model control are clearly shown. With a developed control structure, it is shown that the prescribed target dynamics is achieved, provided tyre forces are available. It is also found that the controller is robust to small changes in the various vehicle parameter values. As a next step, time-varying modelling approach was used in order to better represent the vehicle operating conditions through the various dynamic range, including the limit of adhesion. An iterative vehicle path optimisation problem is formulated using a linear time-varying control approach. The validity of the optimisation method is studied against the steady-state simulation result at the limit of adhesion. It is shown that the method is capable of finding a trajectory in the vicinity of the friction limit, where the front tyres are used fully whilst retaining some margin at the rears. However, a couple of Issues are discovered. First, due to the quadratic nature of the road geometry cost function, the trajectory could get locked if the vehicle runs very close to the edge of the road. Hence, the . optimisation needs to be formulated such that the level of "optimality" on the trajectory remains consistent throughout the manoeuvre at each iteration. Secondly, it is found that inappropriate control demands are produced if the system matrix becomes poorly conditioned near the limit. This results in optimisation failure. In order to understand the mechanism of this failure, controllability of linear timevarying system was analysed and its properties were discussed in detail. First, the calculation methods of the controllability gramian matrix are investigated and some practical limitations are found. The gramian matrix is then used to define an open loop control sequence. It is found that the damping of the system has a significant influence on the control strategy. Subsequently, "the moving controllability window of a fixed time period" is found to provide the most relevant information of changing dynamics through the time. The study showed that the failure of the optimisation in the vicinity of the friction limit was indeed due to lack of controllability and the optimisation method itself was functioning correctly. The vehicle path optimisation problem is then extended to include longitudinal dynamics, enabling simulation of more general manoeuvres. The single corner simulation showed that the optimisation converges to an "out-in-out" path, with iterative solution improving continuously in a first order manner. Simulations with various controller settings showed that the strategy is reasonably robust provided that the changes in parameter settings are kept within a reasonable magnitude. It is also confirmed that the optimisation is able to drive a vehicle close to the limit under different types of operations required, i.e. braking, cornering and acceleration. The study was then performed with slightly more complex road geometry in order to investigate if the· optimisation is capable of prioritising certain· part of the manoeuvre in order to achieve better overall result. Unfortunately, this problem could not be solved successfully. The optimisation concentrated on the latter part of the manoeuvre as it had higher sensitivity to the final cost. This resulted in clearly sub-optimal overall performance. Finally, relatively simple study is conducted to investigate the correlation between various vehicle settings and optimisation results. Using the path optimisation problem formulation, iris found that the more oversteer vehicles are able to achieve better· result with more margin left in rear tyre force capacity. The handling objective functions used for the 4 WS controller is also calculated for the resultant trajectories. It is found that the neutral steer cost had a strong correlation, whereas the linearity cost showed no noticeable correlation. The controllability analysis was applied on the various vehicle settings using step steer simulation. It showed that more understeering vehicle retains higher controllability throughout the dynamics range. It is also found that higher inertia gives better controllability near the limit, however, it gives less controllability at more moderate operating conditions.

More strict control quality requirements are raised to the synthesis of modern algorithmic control systems which can not be satisfied by using classical methods of systems synthesis. Also, the usage of them sometimes becomes impossible, e.g. in cases when a mathematical model of the control object is described by means of complex differential equations or in cases when the model itself is unknown. By applying the suggested synthesis methods of quasi-optimal and variable structure systems as well as algorithms, it is possible to avoid disadvantages common with classical synthesis methods.

No description available.

The motor control of anthropomorphic robotic systems is a challenging computational task mainly because of the high levels of redundancies such systems exhibit. Optimality principles provide a general strategy to resolve such redundancies in a task driven fashion. In particular closed loop optimisation, i.e., optimal feedback control (OFC), has served as a successful motor control model as it unifies important concepts such as costs, noise, sensory feedback and internal models into a coherent mathematical framework. Realising OFC on realistic anthropomorphic systems however is non-trivial: Firstly, such systems have typically large dimensionality and nonlinear dynamics, in which case the optimisation problem becomes computationally intractable. Approximative methods, like the iterative linear quadratic gaussian (ILQG), have been proposed to avoid this, however the transfer of solutions from idealised simulations to real hardware systems has proved to be challenging. Secondly, OFC relies on an accurate description of the system dynamics, which for many realistic control systems may be unknown, difficult to estimate, or subject to frequent systematic changes. Thirdly, many (especially biologically inspired) systems suffer from significant state or control dependent sources of noise, which are difficult to model in a generally valid fashion. This thesis addresses these issues with the aim to realise efficient OFC for anthropomorphic manipulators. First we investigate the implementation of OFC laws on anthropomorphic hardware. Using ILQG we optimally control a high-dimensional anthropomorphic manipulator without having to specify an explicit inverse kinematics, inverse dynamics or feedback control law. We achieve this by introducing a novel cost function that accounts for the physical constraints of the robot and a dynamics formulation that resolves discontinuities in the dynamics. The experimental hardware results reveal the benefits of OFC over traditional (open loop) optimal controllers in terms of energy efficiency and compliance, properties that are crucial for the control of modern anthropomorphic manipulators. We then propose a new framework of OFC with learned dynamics (OFC-LD) that, unlike classic approaches, does not rely on analytic dynamics functions but rather updates the internal dynamics model continuously from sensorimotor plant feedback. We demonstrate how this approach can compensate for unknown dynamics and for complex dynamic perturbations in an online fashion. A specific advantage of a learned dynamics model is that it contains the stochastic information (i.e., noise) from the plant data, which corresponds to the uncertainty in the system. Consequently one can exploit this information within OFC-LD in order to produce control laws that minimise the uncertainty in the system. In the domain of antagonistically actuated systems this approach leads to improved motor performance, which is achieved by co-contracting antagonistic actuators in order to reduce the negative effects of the noise. Most importantly the shape and source of the noise is unknown a priory and is solely learned from plant data. The model is successfully tested on an antagonistic series elastic actuator (SEA) that we have built for this purpose. The proposed OFC-LD model is not only applicable to robotic systems but also proves to be very useful in the modelling of biological motor control phenomena and we show how our model can be used to predict a wide range of human impedance control patterns during both, stationary and adaptation tasks.

Recent developments in quantum information processing have presented new and interesting ways to perform advanced algorithms and improve signal to noise ratios. Examples of these include optimal control theory pulse generation algorithms and the usage of high Q-factor resonators. However, these developments are blocked by current spectrometer designs. This thesis details the design and testing of a new spectrometer with sufficient accuracy, bandwidth, and control to implement these advances. The proposed solution is to use an FPGA-based development board together with custom computer software. This gives access to high-speed analogue inputs and outputs, as well as digital output pins. The spectrometer is then used in two X-band electron spin resonance experiments, showing how the advantages of the system allow for superior results to that possible with the previous equipment. In addition, the setup is used in a Nitrogen Vacancy (NV) system where a rabi experiment is performed.

Quantum information is a growing field showing exciting possibilities for computational speed-up and communications. For the successful implementation of quantum computers, high-precision control is required to reach fault-tolerant thresholds. Control of quantum systems pertains to the manipulation of states and their evolution. In order to minimize the effects of the environment on the control operations of the qubits, control pulses should be made time-optimal. In addition, control pulses should be made robust to noise in the system, dispersion in energies and coupling elements, and uncertain parameters. In this thesis, we examine a robust time-optimal gradient ascent technique which is used to develop controls of the motional degrees of freedom for an ensemble of neutral atoms in a one-dimensional optical lattice in the high dispersion regime with shallow trapping potentials. As such, the system is analyzed in the delocalized basis. The system is treated as an ensemble of atoms with a range of possible quasimomenta across the first Brillouin zone. This gives the ensemble of Hamiltonians, indexed by the quasimomenta, a distinct spectra in their motional states and highly inhomogeneous control Hamiltonians. Thus, the optical lattice is seen as a model system for robust control. We find optimized control pulses designed using an ensemble modification of gradient-ascent pulse engineering robust to any range of quasimomentum. We show that it is possible to produce rotation controls with fidelities above 90\% for half of the first Brillouin zone with gate times in the order of several free oscillations. This is possible for a spectrum that shows upwards of 75\% dispersion in the energies of the band structure. We also show that NOT controls for qubit rotations on the entire Brillouin zone fidelities above 99\% were possible for 0.6\% dispersion in energies. The gate times were also in the order of several free oscillations. It is shown that these solutions are palindromic in time due to phase differences in some of the energy couplings when comparing one half of the Brillouin zone to another. We explore the limits of discretized sampling of a continuous ensemble for control.