We present a new trajectory generation algorithm for autonomous guidance and control of unmanned vehicles from a given starting point to a given target location. We build and update incomplete a priori maps of the operating environment in real time using onboard sensors and compute level sets on the map reflecting the minimal cost of traversal from the current vehicle location to the goal. We convert the trajectory generation problem into a finite-time-horizon optimal control problem using the computed level sets as terminal costs in a receding horizon framework and transform it into a simpler nonlinear programming problem by discretization of the candidate control and state histories. We ensure feasibility of the generated trajectories by constraining the solution of the optimization problem using a simplified vehicle model. We provide strong performance guarantees by checking for stability of the algorithm through the test of matching conditions at the end of each iteration. The algorithm thus explicitly incorporates the vehicle dynamics and constraints and generates trajectories realizable by the vehicle in the field. Successful preliminary field demonstrations and complete simulation results for a marine unmanned surface vehicle demonstrate the efficacy of the proposed approach for fast operations in poorly characterized riverine environments. / Master of Science

A novel application is presented for a fault-tolerant adaptive model predictive control system for small-scale helicopters. The use of a joint Extended Kalman Filter, (EKF), for the estimation of the states and parameters of the UAV, provided the advantage of implementation simplicity and accuracy. A linear model of a small-scale helicopter was utilized for testing the proposed control system. The results, obtained through the simulation of different fault scenarios, demonstrated that the proposed scheme was able to handle different types of actuator and system faults effectively. The types of faults considered were represented in the parameters of the mathematical representation of the helicopter. Benefits provided by the proposed fault-tolerant adaptive model predictive control systems include: The use of the joint Kalman filter provided a straightforward approach to detect and handle different types of actuator and system faults, which were represented as changes of the system and input matrices. The built-in adaptability provided for the handling of slow time-varying faults, which are difficult to detect using the standard residual approach. The successful inclusion of fault tolerance yielded a significant increase in the reliability of the UAV under study. A byproduct of this research is an original comparison between the EKF and the Unscented Kalman Filter, (UKF). This comparison attempted to settle the conflicting claims found in the research literature concerning the performance improvements provided by the UKF. The results of the comparison indicated that the performance of the filters depends on the approximation used for the nonlinear model of the system. Noise sensitivity was found to be higher for the UKF, than the EKF. An advantage of the UKF appears to be a slightly faster convergence.

Covariance assignment theory, introduced in the late 1980s, provided the only means to directly control the steady-state error properties of a linear system subject to Gaussian white noise and parameter uncertainty. This theory, however, does not extend to control of the transient uncertainties and to date there exist no practical engineering solutions to the problem of directly and optimally controlling the uncertainty in a linear system from one Gaussian distribution to another. In this thesis I design a dual-mode Receding Horizon Controller (RHC) that takes a controllable, deterministic linear system from an arbitrary initial covariance to near a desired stationary covariance in finite time. The RHC solves a sequence of free-time Optimal Control Problems (OCP) that directly controls the fundamental solution matrices of the linear system; each problem is a right-invariant OCP on the matrix Lie group GLn of invertible matrices. A terminal constraint ensures that each OCP takes the system to the desired covariance. I show that, by reducing the Hamiltonian system of each OCP from T?GLn to gln? x GLn, the transversality condition corresponding to the terminal constraint simplifies the two-point Boundary Value Problem (BVP) to a single unknown in the initial or final value of the costate in gln?. These results are applied in the design of a dual-mode RHC. The first mode repeatedly solves the OCPs until the optimal time for the system to reach the de- sired covariance is less than the RHC update time. This triggers the second mode, which applies covariance assignment theory to stabilize the system near the desired covariance. The dual-mode controller is illustrated on a planar system. The BVPs are solved using an indirect shooting method that numerically integrates the fundamental solutions on R4 using an adaptive Runge-Kutta method. I contend that extension of the results of this thesis to higher-dimensional systems using either in- direct or direct methods will require numerical integrators that account for the Lie group structure. I conclude with some remarks on the possible extension of a classic result called Lie?s method of reduction to receding horizon control.

<p>This thesis presents a method for the design of a predictive controller with variable step sizes.Predictive methods such as receding horizon control (or model predictive control) use aa fixed sampling frequency when updating the inputs. In the proposed method, the switchingtimes are incorporated into an optimization problem, thus resulting in anadaptive step-size control process. The controller with variable timesteps is shown to require less tuning and to reduce the number of expensive model evaluations.An alternate solution approach had to be developed to accommodate the new problem formulation.The controller's stability is proven in a context that does not require terminal cost or constraints.The thesis presents examples that compare the performance of the variable switching time controllerwith the receding horizon method with a fixed step size. This research opens many roads for futureextension of the theoretical work and practical applications of the controller.</p> / Doctor of Science (PhD)

This dissertation addresses path planning for an autonomous vehicle navigating in a two dimensional environment for which an a priori map is inaccurate and for which the environment is sensed in real-time. For this class of application, planning decisions must be made in real-time. This work is motivated by the need for fast autonomous vehicles that require planning algorithms to operate as quickly as possible. In this dissertation, we first study the case in which there are only static obstacles in the environment. We propose a hybrid receding horizon control path planning algorithm that is based on level-set methods. The hybrid method uses global or local level sets in the formulation of the receding horizon control problem. The decision to select a new level set is made based on certain matching conditions that guarantee the optimality of the path. We rigorously prove sufficient conditions that guarantee that the vehicle will converge to the goal as long as a path to the goal exists. We then extend the proposed receding horizon formulation to the case when the environment possesses moving obstacles. Since all of the results in this dissertation are based on level-set methods, we rigorously investigate how level sets change in response to new information locally sensed by a vehicle. The result is a dynamic fast marching algorithm that usually requires significantly less computation that would otherwise be the case. We demonstrate the proposed dynamic fast marching method in a successful field trial for which an autonomous surface vehicle navigated four kilometers through a riverine environment. / Ph. D.

Waterflooding is a common type of oil recovery techniques where water is pumped into the reservoir for increased productivity. Reservoir states change with time, as such, different injection and production settings will be required to lead the process to optimal operation which is actually a dynamic optimization problem. This could be solved through optimal control techniques which traditionally can only provide an open-loop solution. However, this solution is not appropriate for reservoir production due to numerous uncertain properties involved. Models that are updated through the current industrial practice of ‘history matching’ may fail to predict reality correctly and therefore, solutions based on history-matched models may be suboptimal or non-optimal at all. Due to its ability in counteracting the effects uncertainties, direct feedback control has been proposed recently for optimal waterflooding operations. In this work, two feedback approaches were developed for waterflooding process optimization. The first approach is based on the principle of receding horizon control (RHC) while the second is a new dynamic optimization method developed from the technique of self-optimizing control (SOC). For the SOC methodology, appropriate controlled variables (CVs) as combinations of measurement histories and manipulated variables are first derived through regression based on simulation data obtained from a nominal model. Then the optimal feedback control law was represented as a linear function of measurement histories from the CVs obtained. Based on simulation studies, the RHC approach was found to be very sensitive to uncertainties when the nominal model differed significantly from the conceived real reservoir. The SOC methodology on the other hand, was shown to achieve an operational profit with only 2% worse than the true optimal control, but 30% better than the open-loop optimal control under the same uncertainties. The simplicity of the developed SOC approach coupled with its robustness to handle uncertainties proved its potentials to real industrial applications.

There is a lack of research in the area of hedging future contracts, especially in illiquid or very volatile market conditions. It is important to understand the volatility of the oil and currency markets because reduced fluctuations in these markets could lead to better hedging performance. This study compared different hedging methods by using a hedging error metric, supplementing the Receding Horizontal Control and Stochastic Programming (RHCSP) method by utilizing the London Interbank Offered Rate with the Levy process. The RHCSP hedging method was investigated to determine if improved hedging error was accomplished compared to the Black-Scholes, Leland, and Whalley and Wilmott methods when applied on simulated, oil, and currency futures markets. A modified RHCSP method was also investigated to determine if this method could significantly reduce hedging error under extreme market illiquidity conditions when applied on simulated, oil, and currency futures markets. This quantitative study used chaos theory and emergence for its theoretical foundation. An experimental research method was utilized for this study with a sample size of 506 hedging errors pertaining to historical and simulation data. The historical data were from January 1, 2005 through December 31, 2012. The modified RHCSP method was found to significantly reduce hedging error for the oil and currency market futures by the use of a 2-way ANOVA with a t test and post hoc Tukey test. This study promotes positive social change by identifying better risk controls for investment portfolios and illustrating how to benefit from high volatility in markets. Economists, professional investment managers, and independent investors could benefit from the findings of this study.

Waterflooding is a common type of oil recovery techniques where water is pumped into the reservoir for increased productivity. Reservoir states change with time, as such, different injection and production settings will be required to lead the process to optimal operation which is actually a dynamic optimization problem. This could be solved through optimal control techniques which traditionally can only provide an open-loop solution. However, this solution is not appropriate for reservoir production due to numerous uncertain properties involved. Models that are updated through the current industrial practice of ‘history matching’ may fail to predict reality correctly and therefore, solutions based on history-matched models may be suboptimal or non-optimal at all. Due to its ability in counteracting the effects uncertainties, direct feedback control has been proposed recently for optimal waterflooding operations. In this work, two feedback approaches were developed for waterflooding process optimization. The first approach is based on the principle of receding horizon control (RHC) while the second is a new dynamic optimization method developed from the technique of self-optimizing control (SOC). For the SOC methodology, appropriate controlled variables (CVs) as combinations of measurement histories and manipulated variables are first derived through regression based on simulation data obtained from a nominal model. Then the optimal feedback control law was represented as a linear function of measurement histories from the CVs obtained. Based on simulation studies, the RHC approach was found to be very sensitive to uncertainties when the nominal model differed significantly from the conceived real reservoir. The SOC methodology on the other hand, was shown to achieve an operational profit with only 2% worse than the true optimal control, but 30% better than the open-loop optimal control under the same uncertainties. The simplicity of the developed SOC approach coupled with its robustness to handle uncertainties proved its potentials to real industrial applications.

Air combat is a complex situation, training for it and analysis of possible tactics are time consuming and expensive. In order to circumvent those problems, mathematical models of air combat can be used. This thesis presents air combat as a one-on-one influence diagram game where the influence diagram allows the dynamics of the aircraft, the preferences of the pilots and the uncertainty of decision making in a structural and transparent way to be taken into account. To obtain the players’ game optimal control sequence with respect to their preferences, the influence diagram has to be solved. This is done by truncating the diagram with a moving horizon technique and determining and implementing the optimal controls for a dynamic game which only lasts a few time steps. The result is a working air combat model, where a player estimates the probability that it resides in any of four possible states. The pilot’s preferences are modeled by utility functions, one for each possible state. In each time step, the players are maximizing the cumulative sum of the utilities for each state which each possible action gives. These are weighted with the corresponding probabilities. The model is demonstrated and evaluated in a few interesting aspects. The presented model offers a way of analyzing air combat tactics and maneuvering as well as a way of making autonomous decisions in for example air combat simulators.

2025 / Over the last decade, the application of Wireless Sensor Networks (WSNs) has surged in fields such as environmental monitoring, human health, and smart cities. With this wealth of technologies comes the challenge of how to extract volumes of data collected by such sensor nodes distributed over large, often remote, geographical regions. Data harvesting is the problem of extracting measurements from the remote nodes of WSNs using mobile agents such as ground vehicles or drones. The use of mobile agents can significantly reduce the energy consumption of sensor nodes relative to other modes of extracting the data, extending the lifetime and capabilities of the WSN. Moreover, in remote areas where GPS may not be feasible due to limited power resources on the sensor nodes, the need for accurate sensor node localization and signal broadcasting model estimation becomes critical. Therefore, designing the trajectory of mobile agents is crucial for rapid data collection and information gathering while adhering to vehicle constraints such as dynamics and energy usage. In this thesis, we focus on the application of optimal control methods to design trajectories for mobile agents in data harvesting. This thesis makes contributions in three areas: the creation of a parameterized optimal control policy, the application of a Deep Reinforcement Learning (DRL) based control, and the use of Fisher Information (FI) as a cost matrix in a Receding Horizon Control (RHC) method. Parameterized Optimal Control Policy: Our contributions in this area begin by considering a data harvesting problem in 1-D space. We use a Hamiltonian analysis to show that the optimal control can be described using a parameterized policy and then develop a gradient descent scheme using Infinitesimal Perturbation Analysis (IPA) to calculate the gradients of the cost function with respect to the control parameters. We also consider this problem in a multi-agent setting. To avoid collisions between agents, we apply a Control Barrier Function (CBF) technique to ensure the agents closely track the desired optimal trajectory to complete their mission while avoiding any collisions. Finally, we extend the problem to a mobile sensor scenario. In this more complicated setting we demonstrate that the optimization problem for the control policy parameters can be effectively solved using a heuristic approach. Deep-Reinforcement-Learning based Control: The parametric optimal control approach cannot be easily extended from the 1-D setting to 2-D space. For this reason, we turn to DRL techniques. We utilize Hamiltonian analysis again to get the necessary conditions for optimal control and then translate the problem to a Markov Decision Process (MDP) in discrete time. We apply reinforcement learning techniques, including double deep Q-learning and Proximal Policy Optimization (PPO), to find high-performing solutions across different scenarios. We demonstrate the effectiveness of these methods in 2-D simulations. Fisher-Information-based Receding Horizon Control: For the data harvesting problem in large scale unknown environments, estimating the parameters defining the broadcast model and the location of all the nodes in the environment is critical for efficient extraction of the data. To address that, we start with a Received Signal Strength (RSS) model that relies on a Line-of-Sight (LoS) path-loss model with measurements that are corrupted by Gaussian distributed noise. We first consider a single agent tasked with estimating these unknown parameters in discrete time, and then develop a Fisher Information Matrix (FIM) Receding Horizon (RH) controller for agent motion planning in real time. We also design a Neural Network (NN)-based controller to approximate the optimal solution to the Hamilton-Jacobi-Bellman (HJB) problem, maximizing information gain along a continuous time trajectory. Additionally, a two-stage formation-based RH controller is designed for multi-agent scenarios. The experiments demonstrate that the optimal control policy contribute to the high performance of data collection and the FI-based RHC methods enhance the estimation accuracy in various simulation environments.