While stochastic optimal control, together with associate formulations like Reinforcement Learning, provides a formal approach to, amongst other, motor control, it remains computationally challenging for most practical problems. This thesis is concerned with the study of relations between stochastic optimal control and probabilistic inference. Such dualities { exempli ed by the classical Kalman Duality between the Linear-Quadratic-Gaussian control problem and the filtering problem in Linear-Gaussian dynamical systems { make it possible to exploit advances made within the separate fields. In this context, the emphasis in this work lies with utilisation of approximate inference methods for the control problem. Rather then concentrating on special cases which yield analytical inference problems, we propose a novel interpretation of stochastic optimal control in the general case in terms of minimisation of certain Kullback-Leibler divergences. Although these minimisations remain analytically intractable, we show that natural relaxations of the exact dual lead to new practical approaches. We introduce two particular general iterative methods ψ-Learning, which has global convergence guarantees and provides a unifying perspective on several previously proposed algorithms, and Posterior Policy Iteration, which allows direct application of inference methods. From these, practical algorithms for Reinforcement Learning, based on a Monte Carlo approximation to ψ-Learning, and model based stochastic optimal control, using a variational approximation of posterior policy iteration, are derived. In order to overcome the inherent limitations of parametric variational approximations, we furthermore introduce a new approach for none parametric approximate stochastic optimal control based on a reproducing kernel Hilbert space embedding of the control problem. Finally, we address the general problem of temporal optimisation, i.e., joint optimisation of controls and temporal aspects, e.g., duration, of the task. Specifically, we introduce a formulation of temporal optimisation based on a generalised form of the finite horizon problem. Importantly, we show that the generalised problem has a dual finite horizon problem of the standard form, thus bringing temporal optimisation within the reach of most commonly used algorithms. Throughout, problems from the area of motor control of robotic systems are used to evaluate the proposed methods and demonstrate their practical utility.

Gun and gun turret systems are the primary units of the weapon systems of an aircraft. They are required to hit targets accurately during operations. That is why a complete, high precision control of weapon systems is required. This function is provided by accurate modeling of the system and the design of a suitable controller. This study presents the modeling of and controller design for the gun and turret system for an aircraft. For the controller design purpose, first the mathematical model of the system is constructed. Then the controller is designed to position the turret system as the target comes into sight. The reference input to the controller will either be obtained from a FLIR (Forward Looking Infrared) unit or from a HCU (Hand Control Unit). The basic specification for the controller is to hold theerror signal within the 5.5&deg / positioning envelope. This specification is satisfied by designing Linear Quadratic Gaussian and Internal Model Control type controllers. The performance of the overall system has been examined both by simulation studies and on the real physical system. Results have shown that the designed system is well over being sufficient.

One of the challenges that satellites face is the interaction between control movement and vibration of flexible appendages such as solar arrays and antennas that can negatively affect the performance of the spacecraft. The aim of this thesis is to develop a numerical model of a solar panel structure for KNATTE, a frictionless platform developed by the Onboard Space Systems group at Luleå University of Technology, and develop a control law that reduces the flexible vibration of the solar arrays when attitude control manoeuvres are performed. A set of solar panel structures have been designed and tested, the mathematical model of the multibody system, which consists of KNATTE and two flexible solar panels, has been developed in MATLAB by applying the finite element method. A finite element analysis has been performed in MATLAB to extract the natural frequencies of the system. The model has been numerically verified using a commercial software, and experimentally verified by performing testing on the frictionless vehicle, KNATTE, equipped with the solar panel structures and a number of piezoelectric sensors. Once the model has been verified, a Linear Quadratic Gaussian (LQG) controller has been developed using the results from the finite element model in order to reduce the amplitude of the vibrations of the flexible solar panel structure. The behaviour of the system has been simulated when the spacecraft performs an attitude manoeuvre. The finite element model provides the modal behaviour of the multibody system, obtaining its natural frequencies with low relative error. The LQG controller reduces the amplitude of the vibrations of the flexible solar panel structure.

This thesis presents the practical and novel work in the area of optimal placement of actuators and sensors for sound-structure interaction systems. The work has been done by the author during his PhD candidature. The research is concentrated in systems with non-ideal boundary conditions as in the case in practical engineering applications. An experimental acoustic cavity with five walls of timber and a thin aluminium sheet fixed tightly on the cavity mouth is chosen in this thesis as a good representation of general sound-structure interaction systems. The sheet is intentionally so fixed that it does not satisfy ideal boundary conditions. The existing methods for obtaining optimal sensor-actuator location using analytic models with ideal boundary conditions are of limited use for such problem with non-ideal boundary conditions. The method presented in this thesis for optimal placement of actuators and sensors is motivated by energy based approach and model uncertainty inclusion. The optimal placement of actuator and sensor for the experimental acoustic cavity is used to construct a robust feedback controller based on minimax LQG control design method. The controller is aimed to reduce acoustic potential energy in the cavity. This energy is due to the structure-borne sound inside the sound-structure interaction system. Practical aspects of the method for optimal placement of actuator and sensors are highlighted by experimental vibration and acoustic noise attenuation for arbitrary disturbance using feedback controllers with optimal placement of actuator and sensor. The disturbance is experimentally set to enter the system via a spatial location different from the controller input as would be in any practical applications of standard feedback disturbance rejections. Experimental demonstration of the novel methods presented in this thesis attenuate structural vibration up to 13 dB and acoustic noise up to 5 dB for broadband frequency range of interest. This attenuation is achieved without the explicit knowledge of the model of the disturbance.

We consider the solution of nonlinear optimal control problems subject to stochastic perturbations with incomplete observations. In particular, we generalize results obtained by Ito and Kunisch in [8] where they consider a receding horizon control (RHC) technique based on linearizing the problem on small intervals. The linear-quadratic optimal control problem for the resulting time-invariant (LTI) problem is then solved using the linear quadratic Gaussian (LQG) design. Here, we allow linearization about an instationary reference trajectory and thus obtain a linear time-varying (LTV) problem on each time horizon. Additionally, we apply a model predictive control (MPC) scheme which can be seen as a generalization of RHC and we allow covariance matrices of the noise processes not equal to the identity. We illustrate the MPC/LQG approach for a three dimensional reaction-diffusion system. In particular, we discuss the benefits of time-varying linearizations over time-invariant ones.

We consider the solution of nonlinear optimal control problems subject to stochastic perturbations with incomplete observations. In particular, we generalize results obtained by Ito and Kunisch in [8] where they consider a receding horizon control (RHC) technique based on linearizing the problem on small intervals. The linear-quadratic optimal control problem for the resulting time-invariant (LTI) problem is then solved using the linear quadratic Gaussian (LQG) design. Here, we allow linearization about an instationary reference trajectory and thus obtain a linear time-varying (LTV) problem on each time horizon. Additionally, we apply a model predictive control (MPC) scheme which can be seen as a generalization of RHC and we allow covariance matrices of the noise processes not equal to the identity. We illustrate the MPC/LQG approach for a three dimensional reaction-diffusion system. In particular, we discuss the benefits of time-varying linearizations over time-invariant ones.

Modern interconnected power systems are becoming highly complex and sophisticated, while increasing energy penetrations through congested inter-tie lines causing the operating point approaching stability margins. This as a result, exposes the overall system to potential low frequency power oscillation phenomena following disturbances. This in turn can lead to cascading events and blackouts. Recent approaches to counteract this phenomenon are based on utilization of wide area monitoring systems (WAMS) and power electronics based devices, such as flexible AC transmission systems (FACTS) and HVDC links for advanced power oscillation damping provision. The rise of hybrid AC-DC power systems is therefore sought as a viable solution in overcoming this challenge and securing wide-area stability. If multiple FACTS devices and HVDC links are integrated in a scheme with no supervising control actions considered amongst them, the overall system response might not be optimal. Each device might attempt to individually damp power oscillations ignoring the control status of the rest. This introduces an increasing chance of destabilizing interactions taking place between them, leading to under-utilized performance, increased costs and system wide-area stability deterioration. This research investigates the development of a novel supervisory control scheme that optimally coordinates a parallel operation of multiple FACTS devices and an HVDC link distributed across a power system. The control system is based on Linear Quadratic Gaussian (LQG) modern optimal control theory. The proposed new control scheme provides coordinating control signals to WAMS based FACTS devices and HVDC link, to optimally and coherently counteract inter-area modes of low frequency power oscillations inherent in the system. The thesis makes a thorough review of the existing and well-established improved stability practises a power system benefits from through the implementation of a single FACTS device or HVDC link, and compares the case –and hence raises the issue–when all active components are integrated simultaneously and uncoordinatedly. System identification approaches are also in the core of this research, serving as means of reaching a linear state space model representative of the non-linear power system, which is a pre-requisite for LQG control design methodology.

Smart structure control systems commonly use piezoceramic sensors or accelerometers as vibration measurement devices. These measurement devices often produce noisy and/or low-precision signals, which makes it difficult to measure small-amplitude vibrations. Laser interferometry devices pose as an alternative high-precision position measurement method, capable of nanometer-scale resolution. The aim of this research is to utilize a model-based design approach to develop and implement a real-time Linear Quadratic Gaussian (LQG) regulator for a piezoelectric actuated smart structure using a high-precision laser interferometry measurement system to suppress the excitation of vibratory modes. The analytical model of the smart structure is derived using the extended Hamilton Principle and Euler-Bernoulli beam theory, and the equations of motion for the system are constructed using the assumed-modes method. The analytical model is organized in state-space form, in which the effects of a low-pass filter and sampling of the digital control system are also accounted for. The analytical model is subsequently validated against a finite-element model in Abaqus, a lumped parameter model in Simscape Multibody, and experimental modal analysis using the physical system. A discrete-time proportional-derivative (PD) controller is designed in a heuristic fashion to serve as a baseline performance criterion for the LQG regulator. The Kalman Filter observer and Linear Quadratic Regulator (LQR) components of the LQG regulator are also derived from the state-space model. It is found that the behavior of the analytical model closely matches that of the physical system, and the performance of the LQG regulator exceeds that of the PD controller. The LQG regulator demonstrated quality estimation of the state variables of the system and further constitutes an exceptional closed-loop control system for active vibration control and disturbance rejection of the smart structure.