Predicting human behavior from a small amount of training examples is a challenging machine learning problem. In this thesis, we introduce the principle of maximum causal entropy, a general technique for applying information theory to decision-theoretic, game-theoretic, and control settings where relevant information is sequentially revealed over time. This approach guarantees decision-theoretic performance by matching purposeful measures of behavior (Abbeel & Ng, 2004), and/or enforces game-theoretic rationality constraints (Aumann, 1974), while otherwise being as uncertain as possible, which minimizes worst-case predictive log-loss (Gr¨unwald & Dawid, 2003). We derive probabilistic models for decision, control, and multi-player game settings using this approach. We then develop corresponding algorithms for efficient inference that include relaxations of the Bellman equation (Bellman, 1957), and simple learning algorithms based on convex optimization. We apply the models and algorithms to a number of behavior prediction tasks. Specifically, we present empirical evaluations of the approach in the domains of vehicle route preference modeling using over 100,000 miles of collected taxi driving data, pedestrian motion modeling from weeks of indoor movement data, and robust prediction of game play in stochastic multi-player games.

There is a considerable scope for improving the temperature control of exothermic processes. In this thesis, a sub-optimal control strategy is developed through utilizing the dynamic, simulation tool. This scheme is built around easily obtained knowledge of the system and still retains flexibility. It can be applied to both exothermic batch and fed-batch processes. It consists of servo and regulatory modes, where a Generalized Predictive Controller (GPC) was used to provide self-tuning facilities. The methods outlined allow for limited thermal runaway whilst keeping some spare cooling capacity to ensure that operation at constraints are not violated. A special feature of the method proposed is that switching temperatures and temperature profiles can be readily found from plant trials whilst the addition rate profile Is capable of fairly straightforward computation. The work shows that It is unnecessary to demand stability for the whole of the exothermic reaction cycle, permitting a small runaway has resulted in a fast temperature response within the given safety margin. The Idea was employed for an exothermic single Irreversible reaction and also to a set of complex reactions. Both are carried out in a vessel with a heating/cooling coil. Two constraints are Imposed; (1) limited heat transfer area, and (11) a maximum allowable reaction temperature Tmax. The non-minimum phase problem can be considered as one of the difficulties in managing exothermic fed-batch process when cold reactant Is added to vessel at the maximum operating temperature. The control system coped with this within limits, a not unexpected result. In all cases, the new strategy out-performed the conventional controller and produced smoother variations in the manipulated variable. The simulation results showed that batch to batch variations and disturbances In cooling were successfully handled. GPC worked well but can be susceptible to measurement noise.

<p>This dissertation presents a novel distributed optimal control (DOC) problem formulation that is applicable to multiscale dynamical systems comprised of numerous interacting systems, or agents, that together give rise to coherent macroscopic behaviors, or coarse dynamics, that can be modeled by partial differential equations (PDEs) on larger spatial and time scales. The DOC methodology seeks to obtain optimal agent state and control trajectories by representing the system's performance as an integral cost function of the macroscopic state, which is optimized subject to the agents' dynamics. The macroscopic state is identified as a time-varying probability density function to which the states of the individual agents can be mapped via a restriction operator. Optimality conditions for the DOC problem are derived analytically, and the optimal trajectories of the macroscopic state and control are computed using direct and indirect optimization algorithms. Feedback microscopic control laws are then derived from the optimal macroscopic description using a potential function approach.</p><p>The DOC approach is demonstrated numerically through benchmark multi-agent trajectory optimization problems, where large systems of agents were given the objectives of traveling to goal state distributions, avoiding obstacles, maintaining formations, and minimizing energy consumption through control. Comparisons are provided between the direct and indirect optimization techniques, as well as existing methods from the literature, and a computational complexity analysis is presented. The methodology is also applied to a track coverage optimization problem for the control of distributed networks of mobile omnidirectional sensors, where the sensors move to maximize the probability of track detection of a known distribution of mobile targets traversing a region of interest (ROI). Through extensive simulations, DOC is shown to outperform several existing sensor deployment and control strategies. Furthermore, the computation required by the DOC algorithm is proven to be far reduced compared to that of classical, direct optimal control algorithms.</p> / Dissertation

This report presents optimal control methods integrated with hierarchical control framework to realize real-time collision-free optimal trajectories for motion control in kinematic chain manipulator (KCM) robot systems under dynamic environments. Recently, they have been increasingly used in applications where manipulators are required to interact with random objects and humans. As a result, more complex trajectory planning schemes are required. The main objective of this research is to develop new motion control strategies that can enable such robots to operate efficiently and optimally in such unknown and dynamic environments. Two direct optimal control methods: The direct collocation method and discrete mechanics for optimal control methods are investigated for solving the related constrained optimal control problem and the results are compared. Using the receding horizon control structure, open-loop sub-optimal trajectories are generated as real-time input to the controller as opposed to the predefined trajectory over the entire time duration. This, in essence, captures the dynamic nature of the obstacles. The closed-loop position controller is then engaged to span the robot end-effector along this desired optimal path by computing appropriate torque commands for the joint actuators. Employing a two-degree of freedom technique, collision-free trajectories and robot environment information are transmitted in real-time by the aid of a bidirectional connectionless datagram transfer. A hierarchical network control platform is designed to condition triggering of precedent activities between a dedicated machine computing the optimal trajectory and the real-time computer running a low-level controller. Experimental results on a 2-link planar robot are presented to validate the main ideas. Real-time implementation of collision-free workspace trajectory control is achieved for cases where obstacles are arbitrarily changing in the robot workspace.

Management of knowledge involved in the new product development (NPD) projects is critical to the success of firms competing in environments that require rapid innovation. Unfortunately, many firms lack an understanding of how to develop knowledge management (KM) strategies that drive successful outcomes. In this thesis I develop a rich and multifaceted understanding of how KM strategies drive successful NPD outcomes. I examine KM strategies for NPD at two different decision making levels. First, I consider the how the manager of a single NPD project should pursue knowledge acquisition for its product and process design teams and knowledge transfer between the teams over time throughout the development project. The ability to develop and integrate knowledge drives the net revenue earned at the product release time. I show that two different dynamic KM strategies arise: a delay strategy and a front-loading strategy. I characterize drivers of each strategy and the drivers of the market entry time strategy. In contrast to the deterministic approach above, I introduce a stochastic model. The manager of a single NPD project maximizes expected net revenue which reflects the effectiveness of product and process development. I consider the effect of rework that occurs as a result of the KM activities. Although manager's strategies for knowledge creation satisfy either the delay or front-loading strategy the drivers of each strategy in this model are substantially different from those in the first model reflecting the stochastic nature of the project and the effect of rework. In a third model, I consider the strategic level question of how a firm engages in relationships with its competitor regarding the sharing or transfer of knowledge resources for NPD. I consider two cooperative mechanisms: knowledge transfer when both firms ultimately enter the market separately as competitors versus knowledge sharing when both firms enter the market together following the joint development of a new product. In this thesis, I develop the KM strategies followed by the firms for each cooperation mechanism. In addition, I analyze the impact of firm and market characteristics on firms decision to whether to cooperate or not, and other KM decisions.

Fundação de Amparo à Pesquisa do Estado do Rio de Janeiro / Neste trabalho, será considerado um problema de controle ótimo quadrático para a equação do calor em domínios retangulares com condição de fronteira do tipo Dirichlet é nos quais, a função de controle (dependente apenas no tempo) constitui um termo de fonte. Uma caracterização da solução ótima é obtida na forma de uma equação linear em um espaço de funções reais definidas no intervalo de tempo considerado. Em seguida, utiliza-se uma sequência de projeções em subespaços de dimensão finita para obter aproximações para o controle ótimo, o cada uma das quais pode ser gerada por um sistema linear de dimensão finita. A sequência de soluções aproximadas assim obtidas converge para a solução ótima do problema original. Finalmente, são apresentados resultados numéricos para domínios espaciais de dimensão 1. / In this work, a quadratic optimal control problem will be considered for the heat equation in rectangular domains with Dirichlet type boundary conditions in which the control function (depending only on time) constitutes a source term. A characterization of the solution is obtained in the form of a linear equation in a real function space defined in a considered time interval. Then, a sequence of projections in finite dimensional subspaces is used to obtain approximations for the optimal control, each of them can be generated by a finite dimension linear system. The sequence of approximate solutions obtained in this way converges to an optimal solution of the original problem. Finally, numerical results are presented for spatial domains of 1 dimension.

Made available in DSpace on 2014-06-11T19:27:08Z (GMT). No. of bitstreams: 0 Previous issue date: 2006-09-12Bitstream added on 2014-06-13T19:47:37Z : No. of bitstreams: 1 izelli_rc_me_sjrp.pdf: 916240 bytes, checksum: 24bbf9996f6955ca38766b92b37822c8 (MD5) / Estamos interessados em estudar uma generalização do Princípio do Máximo de Pontryagin para problema de controle ótimo com restrições mistas envolvendo funções nãodiferenciáveis, pois este princípio não se aplica para todos os tipos de problemas. O principal objetivo deste trabalho é apresentar as condições necessárias de otimalidade na forma do princípio do máximo que serão aplicadas para o problema de controle ótimo com restrições mistas envolvendo funções não-diferenciáveis. Para alcançar este objetivo apresentamos estudos sobre cones normais e cones tangentes os quais são utilizados no desenvolvimento da teoria de subdiferenciais. Após esse embasamento formulamos o problema de controle ótimo envolvendo funções não-diferenciáveis, e apresentamos as condições necessárias de otimalidade. / We are interested in study a generalization of the Pontryagin Maximum Principle for optimal control problems with mixed constraints involving nondi erentiable functions, because this principle can not be applied for all the types of problems. The main objective of this work is to present the necessary conditions of optimality in the form of the maximum principle that will be applied for the optimal control problem with mixed constraints involving nondi erentiable functions. To achieve this objective we present studies above normal cones and tangent cones which are used in the development of the theory of subdi erentials. After this foundation we formulate the optimal control problem involving nondi erentiable functions, and we present the necessary conditions of optimality.

In this thesis, we study homogenization of optimal control problems in various oscillatory domains. Specifically, we consider four types of domains given in Figure 1 below. Figure 1: Oscillating Domains The thesis is organized into six chapters. Chapter 1 provides an introduction to our work and the rest of the thesis. The main contributions of the thesis are contained in Chapters 2-5. Chapter 6 presents the conclusions of the thesis and possible further directions. A brief description of our work (Chapters 2-5) follows: Chapter 2: Asymptotic behaviour of a fourth order boundary optimal control problem with Dirichlet boundary data posed on an oscillating domain as in Figure 1(A) is analyzed. We use the unfolding operator to study the asymptotic behavior of this problem. Chapter 3: Homogenization of a time dependent interior optimal control problem on a branched structure domain as in Figure 1(B) is studied. Here we pose control on the oscillating interior part of the domain. The analysis is carried out by appropriately defined unfolding operators suitable for this domain. The optimal control is characterized using various unfolding operators defined at each branch of every level. Chapter 4: A new unfolding operator is developed for a general oscillating domain as in Figure 1(C). Homogenization of a non-linear elliptic problem is studied using this new un-folding operator. Using this idea, homogenization of an optimal control problem on a circular oscillating domain as in Figure 1(D) is analyzed. Chapter 5: Homogenization of a non-linear optimal control problem posed on a smooth oscillating domain as in Figure 1(C) is studied using the unfolding operator.

Cette thèse porte sur l'analyse et le contrôle optimal d'un digesteur anaérobie. L'objectif est de proposer une stratégie de commande optimale pour maximiser la quantité de biogaz produit dans un bioréacteur anaérobie sur une période de temps donnée. Plus particulièrement, à partir d'un modèle simple de bioprocédé et en considérant une classe importante de cinétiques de croissance, nous résolvons un problème de maximisation de biogaz produit par le système pendant un temps fixé, en utilisant le taux de dilution D(.) comme variable de contrôle. Dépendant des conditions initiales du système, l'analyse du problème de contrôle optimal fait apparaître des degrés de difficulté très divers. Dans une première partie, nous résolvons le problème dans le cas où le taux de dilution permettant de maximiser le débit de gaz à l'équilibre est à l'intérieur des bornes minimales et maximales du débit d'alimentation pouvant être appliqué au système : il s'agit du cas WDAC (Well Dimensioned Actuator Case). La synthèse du contrôle optimal est obtenue par une approche de comparaison de trajectoires d'un système dynamique. Une étude comparative des solutions exactes avec celle obtenues avec une approche numérique directe en utilisant le logiciel "BOCOP" est faite. Une comparaison des performances du contrôleur optimal avec celles obtenues en appliquant une loi heuristique est discutée. On montre en particulier que les deux lois de commande amènent le système vers le même point optimal. Dans une deuxième partie, dans le cas où l'actionneur est sous- (ou sur-) dimensionné, c'est-à-dire si la valeur du taux de dilution à appliquer pour obtenir le maximum de biogaz à l'équilibre est en dehors de la valeur minimale ou maximale de l'actionneur, alors nous définissons les cas UDAC (Uder dimensioned Actuator Case) et ODAC (Over Dimensioned Actuator Case) que nous résolvons en appliquant le principe du maximum de Pontryagin. / This thesis focuses on the optimal control of an anaerobic digestor for maximizing its biogas production. In particular, using a simple model of the anaerobic digestion process, we derive a control law to maximize the biogas production over a period of time using the dilution rate D(.) as the control variable. Depending on initial conditions and constraints on the actuator, the search for a solution to the optimal control problem reveals very different levels of difficulty. In the first part, we consider that there are no severe constraints on the actuator. In particular, the interval in which the input flow rate lives includes the value which allows the biogas to be maximized at equilibrium. For this case, named WDAC (Well Dimensioned Actuator Case) we solve the optimal control problem using classical tools of differential equations analysis.Numerical simulations illustrate the robustness of the control law with respect to several parameters, notably with respect to initial conditions. We use these results to show that an heuristic control law proposed in the literature is optimal in a certain sense. The optimal trajectories are then compared with those given by a purely numerical optimal control solver (i.e. the "BOCOP" toolkit) which is an open-source toolbox for solving optimal control problems. When the exact analytical solution to the optimal control problem cannot be found, we suggest that such numerical tool can be used to intuiter optimal solutions.In the second part, the problem of maximizing the biogas production is treated when the actuator is under (-over) dimensioned. These are the cases UDAC (Under Dimensioned Actuator Cases) and ODAC (Over Dimensioned Actuator Cases). Then we solve these optimal problems using the Maximum Principle of Pontryagin.

This thesis is concerned with drawing out high-level insight from otherwise complex mathematical models of physical processes. This is achieved through detailed analysis of model behaviour as constituent parameters are varied. A particular focus is the well-posedness of parameter estimation from noisy data, and its relationship to the parametric sensitivity properties of the model. Other topics investigated include the verification of model performance properties over large ranges of parameters, and the simplification of models based upon their response to parameter perturbation. Several methodologies are proposed, which account for various model classes. However, shared features of the models considered include nonlinearity, parameters with considerable scope for variability, and experimental data corrupted by significant measurement uncertainty. We begin by considering models described by systems of nonlinear ordinary differen- tial equations with parameter dependence. Model output, in this case, can only be obtained by numerical integration of the relevant equations. Therefore, assessment of model behaviour over tracts of parameter space is usually carried out by repeated model simulation over a grid of parameter values. We instead reformulate this as- sessment as an algebraic problem, using polynomial programming techniques. The result is an algorithm that produces parameter-dependent algebraic functions that are guaranteed to bound user-defined aspects of model behaviour over parameter space. We then consider more general classes of parameter-dependent model. A theoretical framework is constructed through which we can explore the duality between model sensitivity to non-local parameter perturbations, and the well-posedness of parameter estimation from significantly noisy data. This results in an algorithm that can uncover functional relations on parameter space over which model output is insensitive and parameters cannot be estimated. The methodology used derives from techniques of nonlinear optimal control. We use this algorithm to simplify benchmark models from the systems biology literature. Specifically, we uncover features such as fast-timescale subsystems and redundant model interactions, together with the sets of parameter values over which the features are valid. We finally consider parameter estimation in models that are acknowledged to im- perfectly describe the modelled process. We show that this invalidates standard statistical theory associated with uncertainty quantification of parameter estimates. Alternative theory that accounts for this situation is then developed, resulting in a computationally tractable approximation of the covariance of a parameter estimate with respect to noise-induced fluctuation of experimental data.