abstract: The grand transition of electric grids from conventional fossil fuel resources to intermittent bulk renewable resources and distributed energy resources (DERs) has initiated a paradigm shift in power system operation. Distributed energy resources (i.e. rooftop solar photovoltaic, battery storage, electric vehicles, and demand response), communication infrastructures, and smart measurement devices provide the opportunity for electric utility customers to play an active role in power system operation and even benefit financially from this opportunity. However, new operational challenges have been introduced due to the intrinsic characteristics of DERs such as intermittency of renewable resources, distributed nature of these resources, variety of DERs technologies and human-in-the-loop effect. Demand response (DR) is one of DERs and is highly influenced by human-in-the-loop effect. A data-driven based analysis is implemented to analyze and reveal the customers price responsiveness, and human-in-the-loop effect. The results confirm the critical impact of demographic characteristics of customers on their interaction with smart grid and their quality of service (QoS). The proposed framework is also applicable to other types of DERs. A chance-constraint based second-order-cone programming AC optimal power flow (SOCP-ACOPF) is utilized to dispatch DERs in distribution grid with knowing customers price responsiveness and energy output distribution. The simulation shows that the reliability of distribution gird can be improved by using chance-constraint. / Dissertation/Thesis / Masters Thesis Electrical Engineering 2019

A incerteza é um elemento fundamental da realidade. Então, torna-se natural a busca por métodos que nos permitam representar o desconhecido em termos matemáticos. Esses problemas originam uma grande classe de programas probabilísticos reconhecidos como modelos de programação estocástica. Eles são mais realísticos que os modelos determinísticos, e tem por objetivo incorporar a incerteza em suas definições. Essa tese aborda os problemas probabilísticos da classe de problemas de multi-estágio com incerteza e com restrições probabilísticas e com restrições probabilísticas conjuntas. Inicialmente, nós propomos um modelo de administração de ativos e passivos multi-estágio estocástico para a indústria de fundos de pensão brasileira. Nosso modelo é formalizado em conformidade com a leis e políticas brasileiras. A seguir, dada a relevância dos dados de entrada para esses modelos de otimização, tornamos nossa atenção às diferentes técnicas de amostragem. Elas compõem o processo de discretização desses modelos estocásticos Nós verificamos como as diferentes metodologias de amostragem impactam a solução final e a alocação do portfólio, destacando boas opções para modelos de administração de ativos e passivos. Finalmente, nós propomos um “framework” para a geração de árvores de cenário e otimização de modelos com incerteza multi-estágio. Baseados na tranformação de Knuth, nós geramos a árvore de cenários considerando a representação filho-esqueda, irmão-direita o que torna a simulação mais eficiente em termos de tempo e de número de cenários. Nós também formalizamos uma reformulação do modelo de administração de ativos e passivos baseada na abordagem extensiva implícita para o modelo de otimização. Essa técnica é projetada pela definição de um processo de filtragem com “bundles”; e codifciada com o auxílio de uma linguagem de modelagem algébrica. A eficiência dessa metodologia é testada em um modelo de administração de ativos e passivos com incerteza com restrições probabilísticas conjuntas. Nosso framework torna possível encontrar a solução ótima para árvores com um número razoável de cenários. / Uncertainty is a key element of reality. Thus, it becomes natural that the search for methods allows us to represent the unknown in mathematical terms. These problems originate a large class of probabilistic programs recognized as stochastic programming models. They are more realistic than deterministic ones, and their aim is to incorporate uncertainty into their definitions. This dissertation approaches the probabilistic problem class of multistage stochastic problems with chance constraints and joint-chance constraints. Initially, we propose a multistage stochastic asset liability management (ALM) model for a Brazilian pension fund industry. Our model is formalized in compliance with the Brazilian laws and policies. Next, given the relevance of the input parameters for these optimization models, we turn our attention to different sampling models, which compose the discretization process of these stochastic models. We check how these different sampling methodologies impact on the final solution and the portfolio allocation, outlining good options for ALM models. Finally, we propose a framework for the scenario-tree generation and optimization of multistage stochastic programming problems. Relying on the Knuth transform, we generate the scenario trees, taking advantage of the left-child, right-sibling representation, which makes the simulation more efficient in terms of time and the number of scenarios. We also formalize an ALM model reformulation based on implicit extensive form for the optimization model. This technique is designed by the definition of a filtration process with bundles, and coded with the support of an algebraic modeling language. The efficiency of this methodology is tested in a multistage stochastic ALM model with joint-chance constraints. Our framework makes it possible to reach the optimal solution for trees with a reasonable number of scenarios.

Quality of municipal water is sustained by addition of disinfectant, generally chlorine, to the water distribution network. Because of health problems, chlorine concentration in the network is limited between maximum and minimum limits. Cancerogenic disinfectant by-products start to occur at high concentrations so it is desired to have minimum amount of chlorine without violating the limit. In addition to the health issues, minimum injection amount is favorable concerning cost. Hence, an optimization model is necessary which covers all of these considerations. However, there are uncertain factors as chlorine is reactive and decays both over time and space. Thus, probabilistic approach is necessary to obtain reliable and realistic results from the model. In this study, a linear programming model is developed for the chance constrained optimization of the water distribution network. The objective is to obtain minimum amount of injection mass subjected to maintaining more uniformly distributed chlorine concentrations within the limits while considering the randomness of chlorine concentration by probability distributions. Network hydraulics and chlorine concentration computations are done by the network simulation software, EPANET.

This dissertation primarily proposes data-driven methods to handle uncertainty in problems related to Enterprise-wide Optimization (EWO). Datadriven methods are characterized by the direct use of data (historical and/or forecast) in the construction of models for the uncertain parameters that naturally arise from real-world applications. Such uncertainty models are then incorporated into the optimization model describing the operations of an enterprise. Before addressing uncertainty in EWO problems, Chapter 2 deals with the integration of deterministic planning and scheduling operations of a network of batch plants. The main contributions of this chapter include the modeling of sequence-dependent changeovers across time periods for a unitspecific general precedence scheduling formulation, the hybrid decomposition scheme using Bilevel and Temporal Lagrangean Decomposition approaches, and the solution of subproblems in parallel. Chapters 3 to 6 propose different data analytics techniques to account for stochasticity in EWO problems. Chapter 3 deals with scenario generation via statistical property matching in the context of stochastic programming. A distribution matching problem is proposed that addresses the under-specification shortcoming of the originally proposed moment matching method. Chapter 4 deals with data-driven individual and joint chance constraints with right-hand side uncertainty. The distributions are estimated with kernel smoothing and are considered to be in a confidence set, which is also considered to contain the true, unknown distributions. The chapter proposes the calculation of the size of the confidence set based on the standard errors estimated from the smoothing process. Chapter 5 proposes the use of quantile regression to model production variability in the context of Sales & Operations Planning. The approach relies on available historical data of actual vs. planned production rates from which the deviation from plan is defined and considered a random variable. Chapter 6 addresses the combined optimal procurement contract selection and pricing problems. Different price-response models, linear and nonlinear, are considered in the latter problem. Results show that setting selling prices in the presence of uncertainty leads to the use of different purchasing contracts.

A incerteza é um elemento fundamental da realidade. Então, torna-se natural a busca por métodos que nos permitam representar o desconhecido em termos matemáticos. Esses problemas originam uma grande classe de programas probabilísticos reconhecidos como modelos de programação estocástica. Eles são mais realísticos que os modelos determinísticos, e tem por objetivo incorporar a incerteza em suas definições. Essa tese aborda os problemas probabilísticos da classe de problemas de multi-estágio com incerteza e com restrições probabilísticas e com restrições probabilísticas conjuntas. Inicialmente, nós propomos um modelo de administração de ativos e passivos multi-estágio estocástico para a indústria de fundos de pensão brasileira. Nosso modelo é formalizado em conformidade com a leis e políticas brasileiras. A seguir, dada a relevância dos dados de entrada para esses modelos de otimização, tornamos nossa atenção às diferentes técnicas de amostragem. Elas compõem o processo de discretização desses modelos estocásticos Nós verificamos como as diferentes metodologias de amostragem impactam a solução final e a alocação do portfólio, destacando boas opções para modelos de administração de ativos e passivos. Finalmente, nós propomos um “framework” para a geração de árvores de cenário e otimização de modelos com incerteza multi-estágio. Baseados na tranformação de Knuth, nós geramos a árvore de cenários considerando a representação filho-esqueda, irmão-direita o que torna a simulação mais eficiente em termos de tempo e de número de cenários. Nós também formalizamos uma reformulação do modelo de administração de ativos e passivos baseada na abordagem extensiva implícita para o modelo de otimização. Essa técnica é projetada pela definição de um processo de filtragem com “bundles”; e codifciada com o auxílio de uma linguagem de modelagem algébrica. A eficiência dessa metodologia é testada em um modelo de administração de ativos e passivos com incerteza com restrições probabilísticas conjuntas. Nosso framework torna possível encontrar a solução ótima para árvores com um número razoável de cenários. / Uncertainty is a key element of reality. Thus, it becomes natural that the search for methods allows us to represent the unknown in mathematical terms. These problems originate a large class of probabilistic programs recognized as stochastic programming models. They are more realistic than deterministic ones, and their aim is to incorporate uncertainty into their definitions. This dissertation approaches the probabilistic problem class of multistage stochastic problems with chance constraints and joint-chance constraints. Initially, we propose a multistage stochastic asset liability management (ALM) model for a Brazilian pension fund industry. Our model is formalized in compliance with the Brazilian laws and policies. Next, given the relevance of the input parameters for these optimization models, we turn our attention to different sampling models, which compose the discretization process of these stochastic models. We check how these different sampling methodologies impact on the final solution and the portfolio allocation, outlining good options for ALM models. Finally, we propose a framework for the scenario-tree generation and optimization of multistage stochastic programming problems. Relying on the Knuth transform, we generate the scenario trees, taking advantage of the left-child, right-sibling representation, which makes the simulation more efficient in terms of time and the number of scenarios. We also formalize an ALM model reformulation based on implicit extensive form for the optimization model. This technique is designed by the definition of a filtration process with bundles, and coded with the support of an algebraic modeling language. The efficiency of this methodology is tested in a multistage stochastic ALM model with joint-chance constraints. Our framework makes it possible to reach the optimal solution for trees with a reasonable number of scenarios.

A incerteza é um elemento fundamental da realidade. Então, torna-se natural a busca por métodos que nos permitam representar o desconhecido em termos matemáticos. Esses problemas originam uma grande classe de programas probabilísticos reconhecidos como modelos de programação estocástica. Eles são mais realísticos que os modelos determinísticos, e tem por objetivo incorporar a incerteza em suas definições. Essa tese aborda os problemas probabilísticos da classe de problemas de multi-estágio com incerteza e com restrições probabilísticas e com restrições probabilísticas conjuntas. Inicialmente, nós propomos um modelo de administração de ativos e passivos multi-estágio estocástico para a indústria de fundos de pensão brasileira. Nosso modelo é formalizado em conformidade com a leis e políticas brasileiras. A seguir, dada a relevância dos dados de entrada para esses modelos de otimização, tornamos nossa atenção às diferentes técnicas de amostragem. Elas compõem o processo de discretização desses modelos estocásticos Nós verificamos como as diferentes metodologias de amostragem impactam a solução final e a alocação do portfólio, destacando boas opções para modelos de administração de ativos e passivos. Finalmente, nós propomos um “framework” para a geração de árvores de cenário e otimização de modelos com incerteza multi-estágio. Baseados na tranformação de Knuth, nós geramos a árvore de cenários considerando a representação filho-esqueda, irmão-direita o que torna a simulação mais eficiente em termos de tempo e de número de cenários. Nós também formalizamos uma reformulação do modelo de administração de ativos e passivos baseada na abordagem extensiva implícita para o modelo de otimização. Essa técnica é projetada pela definição de um processo de filtragem com “bundles”; e codifciada com o auxílio de uma linguagem de modelagem algébrica. A eficiência dessa metodologia é testada em um modelo de administração de ativos e passivos com incerteza com restrições probabilísticas conjuntas. Nosso framework torna possível encontrar a solução ótima para árvores com um número razoável de cenários. / Uncertainty is a key element of reality. Thus, it becomes natural that the search for methods allows us to represent the unknown in mathematical terms. These problems originate a large class of probabilistic programs recognized as stochastic programming models. They are more realistic than deterministic ones, and their aim is to incorporate uncertainty into their definitions. This dissertation approaches the probabilistic problem class of multistage stochastic problems with chance constraints and joint-chance constraints. Initially, we propose a multistage stochastic asset liability management (ALM) model for a Brazilian pension fund industry. Our model is formalized in compliance with the Brazilian laws and policies. Next, given the relevance of the input parameters for these optimization models, we turn our attention to different sampling models, which compose the discretization process of these stochastic models. We check how these different sampling methodologies impact on the final solution and the portfolio allocation, outlining good options for ALM models. Finally, we propose a framework for the scenario-tree generation and optimization of multistage stochastic programming problems. Relying on the Knuth transform, we generate the scenario trees, taking advantage of the left-child, right-sibling representation, which makes the simulation more efficient in terms of time and the number of scenarios. We also formalize an ALM model reformulation based on implicit extensive form for the optimization model. This technique is designed by the definition of a filtration process with bundles, and coded with the support of an algebraic modeling language. The efficiency of this methodology is tested in a multistage stochastic ALM model with joint-chance constraints. Our framework makes it possible to reach the optimal solution for trees with a reasonable number of scenarios.

京都大学 / 新制・課程博士 / 博士(情報学) / 甲第24743号 / 情博第831号 / 新制||情||139(附属図書館) / 京都大学大学院情報学研究科システム科学専攻 / (主査)教授 大塚 敏之, 教授 加納 学, 教授 東 俊一 / 学位規則第4条第1項該当 / Doctor of Informatics / Kyoto University / DFAM

Le travail général de cette thèse consiste à étendre les outils analytiques et algébriques usuellement employés dans la résolution de problèmes combinatoires déterministes à un cadre combinatoire stochastique. Deux cadres distincts sont étudiés : les problèmes combinatoires stochastiques discrets et les problèmes stochastiques continus. Le cadre discret est abordé à travers le problème de la forêt couvrante de poids maximal dans une formulation Two-Stage à multi-scénarios. La version déterministe très connue de ce problème établit des liens entre la fonction de rang dans un matroïde et la formulation duale, via l'algorithme glouton. La formulation stochastique discrète du problème de la forêt maximale couvrante est transformée en un problème déterministe équivalent, mais du fait de la multiplicité des scénarios, le dual associé est en quelque sorte incomplet. Le travail réalisé ici consiste à comprendre en quelles circonstances la formulation duale atteint néanmoins un minimum égal au problème primal intégral. D'ordinaire, une approche combinatoire classique des problèmes de graphes pondérés consiste à rechercher des configurations particulières au sein des graphes, comme les circuits, et à explorer d'éventuelles recombinaisons. Pour donner une illustration simple, si on change d'une manière infinitésimale les valeurs de poids des arêtes d'un graphe, il est possible que la forêt couvrante de poids maximal se réorganise complètement. Ceci est vu comme un obstacle dans une approche purement combinatoire. Pourtant, certaines grandeurs analytiques vont varier de manière continue en fonction de ces variations infinitésimales, comme la somme des poids des arêtes choisies. Nous introduisons des fonctions qui rendent compte de ces variations continues, et nous examinons dans quels cas les formulations duales atteignent la même valeur que les formulations primales intégrales. Nous proposons une méthode d'approximation dans le cas contraire et nous statuons sur la NP complétude de ce type de problème.Les problèmes stochastiques continus sont abordés via le problème de sac à dos avec contrainte stochastique. La formulation est de type ``chance constraint'', et la dualisation par variable lagrangienne est adaptée à une situation où la probabilité de respecter la contrainte doit rester proche de $1$. Le modèle étudié est celui d'un sac à dos où les objets ont une valeur et un poids déterminés par des distributions normales. Dans notre approche, nous nous attachons à appliquer des méthodes de gradient directement sur la formulation en espérance de la fonction objectif et de la contrainte. Nous délaissons donc une possible reformulation classique du problème sous forme géométrique pour détailler les conditions de convergence de la méthode du gradient stochastique. Cette partie est illustrée par des tests numériques de comparaison avec la méthode SOCP sur des instances combinatoires avec méthode de Branch and Bound, et sur des instances relaxées. / The global purpose of this thesis is to study the conditions to extend analytical and algebraical properties commonly observed in the resolution of deterministic combinatorial problems to the corresponding stochastic formulations of these problems. Two distinct situations are treated : discrete combinatorial stochastic problems and continuous stochastic problems. Discrete situation is examined with the Two Stage formulation of the Maximum Weight Covering Forest. The well known corresponding deterministic formulation shows the connexions between the rank function of a matroid, the greedy algorithm , and the dual formulation. The discrete stochastic formulation of the Maximal Covering Forest is turned into a deterministic equivalent formulation, but, due to the number of scenarios, the associated dual is not complete. The work of this thesis leads to understand in which cases the dual formulation still has the same value as the primal integer formulation. Usually, classical combinatorial approaches aim to find particular configurations in the graph, as circuits, in order to handle possible reconfigurations. For example, slight modifications of the weights of the edges might change considerably the configuration of the Maximum Weight Covering Forest. This can be seen as an obstacle to handle pure combinatorial proofs. However, some global relevant quantities, like the global weight of the selected edges during the greedy algorithm, have a continuous variation in function of slight modifications. We introduce some functions in order to outline these continuous variations. And we state in which cases Primal integral problems have the same objective values as dual formulations. When it is not the case, we propose an approximation method and we examine the NP completeness of this problem.Continuous stochastic problems are presented with the stochastic Knapsack with chance constraint. Chance constraint and dual Lagrangian formulation are adapted in the case where the expected probability of not exceeding the knapsack capacity is close to $1$. The introduced model consists in items whose costs and rewards follow normal distributions. In our case, we try to apply direct gradient methods without reformulating the problem into geometrical terms. We detail convergence conditions of gradient based methods directly on the initial formulation. This part is illustrated with numerical tests on combinatorial instances and Branch and Bound evaluations on relaxed formulations.

Le travail général de cette thèse consiste à étendre les outils analytiques et algébriques usuellement employés dans la résolution de problèmes combinatoires déterministes à un cadre combinatoire stochastique. Deux cadres distincts sont étudiés : les problèmes combinatoires stochastiques discrets et les problèmes stochastiques continus. Le cadre discret est abordé à travers le problème de la forêt couvrante de poids maximal dans une formulation Two-Stage à multi-scénarios. La version déterministe très connue de ce problème établit des liens entre la fonction de rang dans un matroïde et la formulation duale, via l'algorithme glouton. La formulation stochastique discrète du problème de la forêt maximale couvrante est transformée en un problème déterministe équivalent, mais du fait de la multiplicité des scénarios, le dual associé est en quelque sorte incomplet. Le travail réalisé ici consiste à comprendre en quelles circonstances la formulation duale atteint néanmoins un minimum égal au problème primal intégral. D'ordinaire, une approche combinatoire classique des problèmes de graphes pondérés consiste à rechercher des configurations particulières au sein des graphes, comme les circuits, et à explorer d'éventuelles recombinaisons. Pour donner une illustration simple, si on change d'une manière infinitésimale les valeurs de poids des arêtes d'un graphe, il est possible que la forêt couvrante de poids maximal se réorganise complètement. Ceci est vu comme un obstacle dans une approche purement combinatoire. Pourtant, certaines grandeurs analytiques vont varier de manière continue en fonction de ces variations infinitésimales, comme la somme des poids des arêtes choisies. Nous introduisons des fonctions qui rendent compte de ces variations continues, et nous examinons dans quels cas les formulations duales atteignent la même valeur que les formulations primales intégrales. Nous proposons une méthode d'approximation dans le cas contraire et nous statuons sur la NP complétude de ce type de problème.Les problèmes stochastiques continus sont abordés via le problème de sac à dos avec contrainte stochastique. La formulation est de type ''chance constraint'', et la dualisation par variable lagrangienne est adaptée à une situation où la probabilité de respecter la contrainte doit rester proche de $1$. Le modèle étudié est celui d'un sac à dos où les objets ont une valeur et un poids déterminés par des distributions normales. Dans notre approche, nous nous attachons à appliquer des méthodes de gradient directement sur la formulation en espérance de la fonction objectif et de la contrainte. Nous délaissons donc une possible reformulation classique du problème sous forme géométrique pour détailler les conditions de convergence de la méthode du gradient stochastique. Cette partie est illustrée par des tests numériques de comparaison avec la méthode SOCP sur des instances combinatoires avec méthode de Branch and Bound, et sur des instances relaxées.

Cette thèse est dediée à l'alanyse numérique de méthodes numériques dans le domaine du contrôle optimal, et est composée de deux parties. La première partie est consacrée à des nouveaux résultats concernant des méthodes numériques pour le contrôle optimal de systèmes hybrides, qui peuvent être contrôlés simultanément par des fonctions mesurables et des sauts discontinus dans la variable d'état. La deuxième partie est dédiée è l'étude d'une application spécifique surl'optimisation de trajectoires pour des lanceurs spatiaux avec contraintes en probabilité. Ici, on utilise des méthodes d'optimisation nonlineaires couplées avec des techniques de statistique non parametrique. Le problème traité dans cette partie appartient à la famille des problèmes d'optimisation stochastique et il comporte la minimisation d'une fonction de coût en présence d'une contrainte qui doit être satisfaite dans les limites d'un seuil de probabilité souhaité. / This thesis is devoted to the analysis of numerical methods in the field of optimal control, and it is composed of two parts. The first part is dedicated to new results on the subject of numerical methods for the optimal control of hybrid systems, controlled by measurable functions and discontinuous jumps in the state variable simultaneously. The second part focuses on a particular application of trajectory optimization problems for space launchers. Here we use some nonlinear optimization methods combined with non-parametric statistics techniques. This kind of problems belongs to the family of stochastic optimization problems and it features the minimization of a cost function in the presence of a constraint which needs to be satisfied within a desired probability threshold.