The goal of this thesis is to synthesize believable motions of a character interacting with its surroundings and manipulating objects through physical contacts and forces. Human-like autonomous avatars are in increasing demand in areas such as entertainment, education, and health care. Yet modeling the basic human motor skills of locomotion and manipulation remains a long-standing challenge in animation research. The seemingly simple tasks of navigating an uneven terrain or grasping cups of different shapes involve planning with complex kinematic and physical constraints as well as adaptation to unexpected perturbations. Moreover, natural movements exhibit unique personal characteristics that are complex to model. Although motion capture technologies allow virtual actors to use recorded human motions in many applications, the recorded motions are not directly applicable to tasks involving interactions for two reasons. First, the acquired data cannot be easily adapted to new environments or different tasks goals. Second, acquisition of accurate data is still a challenge for fine scale object manipulations. In this work, we utilize data to create natural looking animations, and mitigate data deficiency with physics-based simulations and numerical optimizations. We develop algorithms based on a single reference motion for three types of control problems. The first problem focuses on motions without contact constraints. We use joint torque patterns identified from the captured motion to simulate responses and recovery of the same style under unexpected pushes. The second problem focuses on locomotion with foot contacts. We use contact forces to control an abstract dynamic model of the center of mass, which sufficiently describes the locomotion task in the input motion. Simulation of the abstract model under unexpected pushes or anticipated changes of the environment results in responses consistent with both the laws of physics and the style of the input. The third problem focuses on fine scale object manipulation tasks, in which accurate finger motions and contact information are not available. We propose a sampling method to discover contact relations between the hand and the object from only the gross motion of the wrists and the object. We then use the abundant contact constraints to synthesize detailed finger motions. The algorithm creates finger motions of various styles for a diverse set of object shapes and tasks, including ones that are not present at capture time. The three algorithms together control an autonomous character with dexterous hands to interact naturally with a virtual world. Our methods are general and robust across character structures and motion contents when testing on a wide variety of motion capture sequences and environments. The work in this thesis brings closer the motor skills of a virtual character to its human counterpart. It provides computational tools for the analysis of human biomechanics, and can potentially inspire the design of novel control algorithms for humanoid robots.

In this paper we examine the stabilization of LTI discrete-time systems with control input constraints in the form of saturation nonlinearities. This kind of constraint is usually introduced to simulate the effect of actuator limitations. Since global controllability can not be assumed in the presence of constrained control, the controllable regions and their characterizations are analyzed first. We present an efficient algorithm for finding controllable regions in terms of their boundary hyperplanes (inequality constraints). A previously open question about the exact number of irredundant boundary hyperplanes is also resolved here. The main result of this research is a time-optimal nonlinear controller which stabilizes the system on its controllable region. We give analgorithm for on-line computation of control which is also implementable for high-order systems. Simulation results show superior response even in the presence of disturbances.

This report presents optimal control methods integrated with hierarchical control framework to realize real-time collision-free optimal trajectories for motion control in kinematic chain manipulator (KCM) robot systems under dynamic environments. Recently, they have been increasingly used in applications where manipulators are required to interact with random objects and humans. As a result, more complex trajectory planning schemes are required. The main objective of this research is to develop new motion control strategies that can enable such robots to operate efficiently and optimally in such unknown and dynamic environments. Two direct optimal control methods: The direct collocation method and discrete mechanics for optimal control methods are investigated for solving the related constrained optimal control problem and the results are compared. Using the receding horizon control structure, open-loop sub-optimal trajectories are generated as real-time input to the controller as opposed to the predefined trajectory over the entire time duration. This, in essence, captures the dynamic nature of the obstacles. The closed-loop position controller is then engaged to span the robot end-effector along this desired optimal path by computing appropriate torque commands for the joint actuators. Employing a two-degree of freedom technique, collision-free trajectories and robot environment information are transmitted in real-time by the aid of a bidirectional connectionless datagram transfer. A hierarchical network control platform is designed to condition triggering of precedent activities between a dedicated machine computing the optimal trajectory and the real-time computer running a low-level controller. Experimental results on a 2-link planar robot are presented to validate the main ideas. Real-time implementation of collision-free workspace trajectory control is achieved for cases where obstacles are arbitrarily changing in the robot workspace.

In this paper we examine the stabilization of LTI discrete-time systems with control input constraints in the form of saturation nonlinearities. This kind of constraint is usually introduced to simulate the effect of actuator limitations. Since global controllability can not be assumed in the presence of constrained control, the controllable regions and their characterizations are analyzed first. We present an efficient algorithm for finding controllable regions in terms of their boundary hyperplanes (inequality constraints). A previously open question about the exact number of irredundant boundary hyperplanes is also resolved here. The main result of this research is a time-optimal nonlinear controller which stabilizes the system on its controllable region. We give analgorithm for on-line computation of control which is also implementable for high-order systems. Simulation results show superior response even in the presence of disturbances.

In recent years, Unmanned Aerial Vehicles (UAVs) have been used extensively in military conflict situations to execute intelligence, surveillance and reconnaissance missions. However, most of the current UAV platforms have limited collaborative capabilities, and consequently they must be controlled individually by operators on the ground. The purpose of the research presented in this thesis is to derive algorithms that can enable multiple UAVs to reason about the movements of multiple ground targets and autonomously coordinate their efforts in real-time to ensure that the targets do not escape. By improving the autonomy of multivehicle systems, the workload placed on the command and control operators is reduced significantly. To derive effective adversarial control algorithms, the adversarial scenario is modeled as a multiplayer differential game. However, due to the inherent computational complexity of multiplayer differential games, three less computationally demanding differential pursuit-evasion game-based algorithms are presented. The purpose of the algorithms is to quickly derive interception strategies for a team of autonomous vehicles. The algorithms are applicable to scenarios with different base assumptions, that is, the three algorithms are meant to complement one another by addressing different types of adversarial problems.

Modern power systems operate much closer to their stability limits than before. With the introduction of highly sensitive industrial and residential loads, the loss of system stability becomes increasingly costly. Reinforcing the power grid by installing additional transmission lines, creating more complicated meshed networks and increasing the voltage level are among the effective, yet expensive solutions. An alternative approach is to improve the performance of the existing power system components by incorporating more intelligent control techniques. This can be achieved in two ways: introducing intelligent local controllers for the existing components in the power network in order to employ their utmost capabilities, and implementing global intelligent schemes for optimizing the performance of multiple local controllers based on an objective function associated with the overall performance of the power system. Both these aspects are investigated in this thesis. In the first section, artificial neural networks are adopted for designing an optimal nonlinear controller for a static compensator (STATCOM) connected to a multimachine power system. The neurocontroller implementation is based on the adaptive critic designs (ACD) technique and provides an optimal control policy over the infinite horizon time of the problem. The ACD based neurocontroller outperforms a conventional controller both in terms of improving the power system dynamic stability and reducing the control effort required. The second section investigates the further improvement of the power system behavior by introducing an ACD based neurocontroller for hierarchical control of a multimachine power system. The proposed wide area controller improves the power system dynamic stability by generating optimal control signals as auxiliary reference signals for the synchronous generators automatic voltage regulators and the STATCOM line voltage controller. This multilevel hierarchical control scheme forces the different controllers throughout the power system to optimally respond to any fault or disturbance by reducing a predefined cost function associated with the power system performance.

This dissertation addresses design and analysis aspects of stochastic dynamical systems using Fokker-Planck equation (FPE). A new numerical methodology based on the partition of unity meshless paradigm is developed to tackle the greatest hurdle in successful numerical solution of FPE, namely the curse of dimensionality. A local variational form of the Fokker-Planck operator is developed with provision for h- and p- refinement. The resulting high dimensional weak form integrals are evaluated using quasi Monte-Carlo techniques. Spectral analysis of the discretized Fokker- Planck operator, followed by spurious mode rejection is employed to construct a new semi-analytical algorithm to obtain near real-time approximations of transient FPE response of high dimensional nonlinear dynamical systems in terms of a reduced subset of admissible modes. Numerical evidence is provided showing that the curse of dimensionality associated with FPE is broken by the proposed technique, while providing problem size reduction of several orders of magnitude. In addition, a simple modification of norm in the variational formulation is shown to improve quality of approximation significantly while keeping the problem size fixed. Norm modification is also employed as part of a recursive methodology for tracking the optimal finite domain to solve FPE numerically. The basic tools developed to solve FPE are applied to solving problems in nonlinear stochastic optimal control and nonlinear filtering. A policy iteration algorithm for stochastic dynamical systems is implemented in which successive approximations of a forced backward Kolmogorov equation (BKE) is shown to converge to the solution of the corresponding Hamilton Jacobi Bellman (HJB) equation. Several examples, including a four-state missile autopilot design for pitch control, are considered. Application of the FPE solver to nonlinear filtering is considered with special emphasis on situations involving long durations of propagation in between measurement updates, which is implemented as a weak form of the Bayes rule. A nonlinear filter is formulated that provides complete probabilistic state information conditioned on measurements. Examples with long propagation times are considered to demonstrate benefits of using the FPE based approach to filtering.

In this thesis, a new guidance method for fighter aircrafts and a new guidance method for missiles are developed. Also, guidance and control systems of the aircraft and the missile used are designed to simulate the generic engagement scenarios between the missile and the aircraft. Suggested methods have been tested under excessive simulation studies. The aircraft guidance method developed here is a rule based missile evasion method. The main idea to develop this method stems from the maximization of the miss distance for an engagement scenario between a missile and an aircraft. To do this, an optimal control problem with state and input dependent inequality constraints is solved and the solution method is applied on different problems that represent generic scenarios. Then, the solutions of the optimal control problems are used to extract rules. Finally, a method that uses the interpolation of the extracted rules is given to guide the aircraft. The new guidance method developed for missiles is formulated by modifying the classical proportional navigation guidance method using the position estimates. The position estimation is obtained by utilization of a Kalman based filtering method, called interacting multiple models.

Distributed optimal control problems with transient convection dominated diffusion convection reaction equations are considered. The problem is discretized in space by using three types of discontinuous Galerkin (DG) method: symmetric interior penalty Galerkin (SIPG), nonsymmetric interior penalty Galerkin (NIPG), incomplete interior penalty Galerkin (IIPG). For time discretization, Crank-Nicolson and backward Euler methods are used. The discretize-then-optimize approach is used to obtain the finite dimensional problem. For one-dimensional unconstrained problem, Newton-Conjugate Gradient method with Armijo line-search. For two-dimensional control constrained problem, active-set method is applied. A priori error estimates are derived for full discretized optimal control problem. Numerical results for one and two-dimensional distributed optimal control problems for diffusion convection equations with boundary layers confirm the predicted orders derived by a priori error estimates.

Many real-life applications such as the shape optimization of technological devices, the identification of parameters in environmental processes and flow control problems lead to optimization problems governed by systems of convection diusion partial dierential equations (PDEs). When convection dominates diusion, the solutions of these PDEs typically exhibit layers on small regions where the solution has large gradients. Hence, it requires special numerical techniques, which take into account the structure of the convection. The integration of discretization and optimization is important for the overall eciency of the solution process. Discontinuous Galerkin (DG) methods became recently as an alternative to the finite dierence, finite volume and continuous finite element methods for solving wave dominated problems like convection diusion equations since they possess higher accuracy. This thesis will focus on analysis and application of DG methods for linear-quadratic convection dominated optimal control problems. Because of the inconsistencies of the standard stabilized methods such as streamline upwind Petrov Galerkin (SUPG) on convection diusion optimal control problems, the discretize-then-optimize and the optimize-then-discretize do not commute. However, the upwind symmetric interior penalty Galerkin (SIPG) method leads to the same discrete optimality systems. The other DG methods such as nonsymmetric interior penalty Galerkin (NIPG) and incomplete interior penalty Galerkin (IIPG) method also yield the same discrete optimality systems when penalization constant is taken large enough. We will study a posteriori error estimates of the upwind SIPG method for the distributed unconstrained and control constrained optimal control problems. In convection dominated optimal control problems with boundary and/or interior layers, the oscillations are propagated downwind and upwind direction in the interior domain, due the opposite sign of convection terms in state and adjoint equations. Hence, we will use residual based a posteriori error estimators to reduce these oscillations around the boundary and/or interior layers. Finally, theoretical analysis will be confirmed by several numerical examples with and without control constraints