This thesis focuses on ridge regression (RR) and least absolute shrinkage and selection operator (lasso). Ridge properties are being investigated in great detail which include studying the bias, the variance and the mean squared error as a function of the tuning parameter. We also study the convexity of the trace of the mean squared error in terms of the tuning parameter. In addition, we examined some special properties of RR for factorial experiments. Not only do we review ridge properties, we also review lasso properties because they are somewhat similar. Rather than shrinking the estimates toward zero in RR, the lasso is able to provide a sparse solution, setting many coefficient estimates exaclty to zero. Furthermore, we try a new approach to solve the lasso problem by formulating it as a bilevel problem and implementing a new algorithm to solve this bilevel program. / Graduate

Apresentamos dois novos métodos para a solução de problemas de otimização convexa em dois níveis não necessariamente diferenciáveis, i.e., mostramos que as sequências geradas por ambos os métodos convergem para o conjunto ótimo de uma função não suave sujeito a um conjunto que também envolve a minimização de uma função não diferenciável. Ambos os algoritmos dispensam qualquer tipo de resolução de subproblemas ou busca linear durante suas iterações. Ao final, para demonstrar que os métodos são viáveis, resolvemos um problema de reconstrução de imagens tomográficas / We present two new methods for solving bilevel convex optimization problems, where both functions are not necessarily differentiable, i.e., we show that the sequences generated by those methods converge to the optimal set of a nonsmooth function subject to a set that also involves a function minimization. Both algorithms do not require any kind of subproblems resolution or linear search during the iterations. At the end, to prove that our methods are viable, we solve a problem of tomographic image reconstruction

Optimal parametric design of a system must be able to respond quickly to short term needs as well as long term conditions. To this end, we present an Assess-Predict-Optimize (APO) strategy which allows for easy modification of a system’s characteristics and constraints, enabling quick design adaptation. There are three components to the APO strategy: Assess - extract necessary information from given data; Predict - predict future behavior of system; and Optimize – obtain optimal system configuration based on information from the other components. The APO strategy utilizes three key mathematical ingredients to yield real-time results which would certainly conform to given constraints: dimension reduction of the model, a posteriori error estimation, and optimization methods. The resulting formulation resembles a bilevel optimization problem with an inherent nonconvexity in the inner level. Using a simple infiltration-evaporation model to simulate an irrigation system, we demonstrate the APO strategy’s ability to yield real-time optimal results. The linearized model, described by a coercive elliptic partial differential equation, is discretized by the reduced-basis output bounds method. A primal-dual interior point method is then chosen to solve the resulting APO problem. / Singapore-MIT Alliance (SMA)

Apresentamos dois novos métodos para a solução de problemas de otimização convexa em dois níveis não necessariamente diferenciáveis, i.e., mostramos que as sequências geradas por ambos os métodos convergem para o conjunto ótimo de uma função não suave sujeito a um conjunto que também envolve a minimização de uma função não diferenciável. Ambos os algoritmos dispensam qualquer tipo de resolução de subproblemas ou busca linear durante suas iterações. Ao final, para demonstrar que os métodos são viáveis, resolvemos um problema de reconstrução de imagens tomográficas / We present two new methods for solving bilevel convex optimization problems, where both functions are not necessarily differentiable, i.e., we show that the sequences generated by those methods converge to the optimal set of a nonsmooth function subject to a set that also involves a function minimization. Both algorithms do not require any kind of subproblems resolution or linear search during the iterations. At the end, to prove that our methods are viable, we solve a problem of tomographic image reconstruction

Demand Response (DR) was proposed more than a decade ago to incentivise customers to shift their electricity usage from peak demand periods to off-peak demand periods and to curtail their electricity usage during peak demand periods. However, the lack of two-way communication infrastructure weakens the influence of DR and limits its applications. With the development of smart grid facilities (e.g. smart meters and the two-way communication infrastructure) that enable the interactions between the energy retailer and its customers, demand response shows great potential to reduce customers' bills, increase the retailer's profit and further stabilize the power systems. Given such a context, in this thesis we propose smart pricing based demand response programs to study the interactions between the energy retailer and its customers based on game-theory and machine learning techniques. We conduct the research in two different application scenarios: 1) For customers with home energy management system (HEMS) installed in their smart meters, the retailer will know the customers' energy consumption patterns by interacting with the HEMS. As a result, the smart pricing based demand response problem can be modelled as a Stackelberg game or bilevel optimization problem. Further, efficient solutions are proposed for the demand response problems and the existence of optimal solution to the Stackelberg game and the bilevel model is proved; 2) For customers without HEMS installed in their smart meters, the retailer will not know the energy consumption patterns of these customers and must learn customers' behaviour patterns via historical energy usage data. To realize this, two appliance-level machine learning algorithms are proposed to learn customers' consumption patterns. Further, distributed pricing algorithms are proposed for the retailer to solve the demand response problem effectively. Simulation results indicate the effectiveness of the proposed demand response models in both application scenarios.

<p>We analyse several facets of bilevel decision problems under uncertainty. These problems can be interpreted as an extension of stochastic programming problems where part of the uncertainty is attributed to the behaviour of another actor.</p><p>The field of decision making under uncertainty with bilevel features is quite new and most approaches focus on the interactions and relations between the decision makers. In contrast to these studies, the approach of bilevel stochastic programming pursued here stresses the stochastic programming aspect of the problem formulation. The framework enables a direct application of stochastic programming concepts and solution methods to the bilevel relationship between the actors. Thus more complex problem structures can be studied and the aspect of uncertainty can be treated adequately.</p><p>Our analysis covers both theoretical and more practically oriented issues. We study different formulations of one and two stage bilevel stochastic programming problems and state necessary optimality conditions for each of the problem instances. Additionally we present a solution algorithm utilising a stochastic quasi-gradient method. A further study is concerned with the uniqueness of the minima of a convex stochastic programming problem with uncertainty about the decision variables. We state conditions on the distribution of the parameters representing the uncertainty such that the minima of the optimisation problem are unique. We formulate a model of competition and collaboration of two different types of telecom service providers, the owner of a bottleneck facility and a virtual network operator. This represents an application of a bilevel stochastic programming formulation to a liberalised telecommunications environment. Furthermore, the utilisation of the bilevel stochastic programming framework and the developed solution concepts for the analysis of principal agent models is demonstrated. Also here the background of a regulated telecom environment, more specific the relations between a regulator and a regulated telecommunications company, was chosen.</p>

We analyse several facets of bilevel decision problems under uncertainty. These problems can be interpreted as an extension of stochastic programming problems where part of the uncertainty is attributed to the behaviour of another actor. The field of decision making under uncertainty with bilevel features is quite new and most approaches focus on the interactions and relations between the decision makers. In contrast to these studies, the approach of bilevel stochastic programming pursued here stresses the stochastic programming aspect of the problem formulation. The framework enables a direct application of stochastic programming concepts and solution methods to the bilevel relationship between the actors. Thus more complex problem structures can be studied and the aspect of uncertainty can be treated adequately. Our analysis covers both theoretical and more practically oriented issues. We study different formulations of one and two stage bilevel stochastic programming problems and state necessary optimality conditions for each of the problem instances. Additionally we present a solution algorithm utilising a stochastic quasi-gradient method. A further study is concerned with the uniqueness of the minima of a convex stochastic programming problem with uncertainty about the decision variables. We state conditions on the distribution of the parameters representing the uncertainty such that the minima of the optimisation problem are unique. We formulate a model of competition and collaboration of two different types of telecom service providers, the owner of a bottleneck facility and a virtual network operator. This represents an application of a bilevel stochastic programming formulation to a liberalised telecommunications environment. Furthermore, the utilisation of the bilevel stochastic programming framework and the developed solution concepts for the analysis of principal agent models is demonstrated. Also here the background of a regulated telecom environment, more specific the relations between a regulator and a regulated telecommunications company, was chosen.

This thesis addresses two important classes of optimization : multiobjective optimization and bilevel optimization. The investigation concerns their solution methods, applications, and possible links between them. First of all, we develop a procedure for solving Multiple Objective Linear Programming Problems (MOLPP). The method is based on a new characterization of efficient faces. It exploits the connectedness property of the set of ideal tableaux associated to degenerated points in the case of degeneracy. We also develop an approach for solving Bilevel Linear Programming Problems (BLPP). It is based on the result that an optimal solution of the BLPP is reachable at an extreme point of the underlying region. Consequently, we develop a pivoting technique to find the global optimal solution on an expanded tableau that represents the data of the BLPP. The solutions obtained by our algorithm on some problems available in the literature show that these problems were until now wrongly solved. Some applications of these two areas of optimization problems are explored. An application of multicriteria optimization techniques for finding an optimal planning for the distribution of electrical energy in Cameroon is provided. Similary, a bilevel optimization model that could permit to protect any economic sector where local initiatives are threatened is proposed. Finally, the relationship between the two classes of optimization is investigated. We first look at the conditions that guarantee that the optimal solution of a given BPP is Pareto optimal for both upper and lower level objective functions. We then introduce a new relation that establishes a link between MOLPP and BLPP. Moreover, we show that, to solve a BPP, it is possible to solve two artificial M0PPs. In addition, we explore Bilevel Multiobjective Programming Problem (BMPP), a case of BPP where each decision maker (DM) has more than one objective function. Given a MPP, we show how to construct two artificial M0PPs such that any point that is efficient for both problems is also efficient for the BMPP. For the linear case specially, we introduce an artificial MOLPP such that its resolution can permit to generate the whole feasible set of the leader DM. Based on this result and depending on whether the leader can evaluate or not his preferences for his different objective functions, two approaches for obtaining efficient solutions are presented

Ce travail présente une optimisation bi-niveau pour la conception de réseaux durables de ressources dans les parcs éco-industriels (EIP). Tout d'abord, les méthodes d'optimisation multiobjectif sont explorées afin de gérer la nature multicritère des problèmes de conception de réseaux dans les EIP. Ensuite, différents cas d’étude sont explorés et analysés afin de maintenir un équilibre concernant les coûts opératoires des usines, tout en minimisant la consommation des ressources naturelles. Ainsi, le problème est modélisé selon une structure bi-niveau reprenant les concepts de la théorie des jeux, où les usines des entreprises jouent un jeu de Nash entre elles, tout en étant dans une structure de jeu de Stackelberg avec l'autorité environnementale. Cette structure définit un modèle qui doit être transformé en un problème MOPEC (Multiple Optimization Problems with Equilibrium Constraints). Différents cas d’étude sont explorés : le premier cas est le réseau d'eau mono-polluant d’un EIP dans lequel l’influence des paramètres opératoires des usines est étudiée afin de déterminer ceux qui favorisent la symbiose entre les usines. Le réseau d'eau est composé d'un nombre fixe de procédés et d’unités de régénération où les concentrations maximales d’entrée et de sortie des polluants sont définies a priori. L'objectif est alors de déterminer quelles sont les allocations entre procédés et unités de régénération. Les résultats obtenus mettent en évidence les avantages de la structure du modèle proposée par rapport aux approches multiobjectif traditionnelles, en obtenant des gains économiques équilibrés d’usines différentes (gains entre 12-25%) tout en maintenant une faible consommation globale des ressources. Ensuite, d'autres études de cas sont abordées à l'aide de la structure bi-niveau : il s’agit d'inclure simultanément les réseaux d'énergie et d’eau dans une formulation multileader multi-follower où les deux «autorités » environnementales sont supposées jouer un jeu non-coopératif de Nash. Dans un premier cas, le gain économique est plus important en incluant des réseaux d'énergie dans la structure de l’EIP. La deuxième étude de cas industriel explore un modèle de réseau d’utilités offre-demande où l'autorité environnementale vise à minimiser les émissions totales de CO2 dans le parc. La conclusion des différents cas explorés montre des résultats extrêmement favorables en termes de coût et d’impact environnemental ce qui vise à encourager les entreprises à participer à l'EIP. / This work presents a bilevel programming framework for the design of sustainable resource networks in eco-industrial parks (EIP). First, multiobjective optimization methods are explored in order to manage the multi-criteria nature of EIP network design problems. Then, different case studies are modeled in order to minimize and maintain in equilibrium participating plants operating costs while minimizing resource consumption. Thus, the structure of the model is constituted by a bilevel programming framework where the enterprises’ plants play a Nash game between them while being in a Stackelberg game structure with the authority. This structure defines a model which, in order to be solved, has to be transformed into a MOPEC (Multiple Optimization Problems with Equilibrium Constraints) structure. Regarding the case studies, monocontaminant water networks in EIP are studied first, where the influence of plants operating parameters are studied in order to determine the most important ones to favor the symbiosis between plants. The water network is composed of a fixed number of process and water regeneration units where the maximal inlet and outlet contaminant concentrations are defined a priori. The aim is to determine which processes are interconnected and the water regeneration allocation. Obtained results highlight the benefits of the proposed model structure in comparison with traditional multiobjective approaches, by obtaining equilibrate different plants operating costs (i.e. gains between 12-25%) while maintaining an overall low resource consumption. Then, other case studies are approached by using the bilevel structure to include simultaneously energy networks in a multi-leader-multi-follower formulation where both environmental authorities are assumed to play a noncooperative Nash game. In the first case study, economic gain is proven to be more significant by including energy networks in the EIP structure. The second industrial case study explores a supply-demand utility network model where the environmental authority aims to minimize the total equivalent CO2 emissions in the EIP. In all cases, the enterprises’ plants are encouraged to participate in the EIP by the extremely favorable obtained results.

O objetivo deste trabalho é propor uma metodologia para a priorização de obras do sistema de distribuição de média tensão - até 36 kV - voltada ao cumprimento do índice de continuidade DEC e FEC imposto pela ANEEL, visando reduzir a quantidade de conjuntos que estão fora dos limites e que geram multas para a empresa frente ao órgão regulador e aos consumidores. Inicialmente, os diversos tipos de obras têm seu benefício calculado com o uso do Método do Payoff Simplificado, baseado no Método do Payoff COPEL, que consiste na extração somente da parcela relativa a interrupção, no DEC ou FEC, que determinada obra trará ao sistema. De posse deste benefício estimado, as obras foram analisadas de duas maneiras: geral e por conjunto. A análise Geral consiste em observar as obras propostas de maneira independente, preocupando-se com o benefício que elas trarão para a empresa como um todo. Na análise por conjunto, as obras são agrupadas por conjunto ANEEL, e o objetivo é a colocação da maior quantidade de conjuntos dentro dos limites de continuidade impostos pelo órgão regulador. A definição do objetivo apropriado é que irá orientar todo o processo de seleção das obras. Para isso são propostos modelos matemáticos, e para trabalhar com eles, foi utilizada como ferramenta a programação matemática. Foram realizadas simulações divididas em dois grupos: no primeiro, análise geral, a otimização é executada diretamente. Já no segundo, na análise por conjunto, é aplicada a programação multi-nível, mais especificamente, a programação em dois níveis (\"Bilevel Programming Problem\"), utilizando a programação inteira ou por metas (\"goal programming\"). Os resultados das simulações mostraram que o objetivo principal, que é tirar a maior quantidade de conjuntos da transgressão, foi atingido com menor orçamento com o uso da metodologia e dos modelos matemáticos empregados neste trabalho. A metodologia proposta pretende ser uma ferramenta adicional para as concessionárias de distribuição de energia elétrica que normalmente elaboram programas de obras específicos para redução de índices de continuidade ou quando pressionados pelo órgão regulador elaboram programas alternativos que competem pelo mesmo orçamento frente aos programas de obras tradicionais. / The purpose of this paper is to propose a methodology to prioritize planned works in the medium-voltage distribution system - up to 36 kV - aiming to adequate the DEC and FEC continuity index to the limits defined by the Brazilian regulatory agency (ANEEL) through the reduction of the number of sets out of target and consequently the reduction of monetary penalties to the utility imposed by the regulatory agency and consumers. At first every planned work has its benefit calculated by the Simplified Payoff Method which is based on COPEL Payoff Method and which consists in extracting just the interruption event from the DEC or FEC which a given work will bring to the system. Once you have got the estimated benefit, the planned works are analyzed in two different ways - general analysis and set analysis. General analysis consists in checking up proposed works independently, focusing on the benefit they will bring to the company as a whole. In the set analysis, works are grouped by \"ANEEL sets\" and the main aim is to gather the greatest number of sets into the continuity limits defined by the regulatory agency. The aims definition will lead the whole work selection process. To achieve that mathematical models are proposed and mathematical programming tools are used. Two groups of simulations were done - in the first one which is also called general analysis, optimization is executed directly. The second one called set analysis, is applied the bilevel programming using the integer programming or goal programming. The simulation results showed that the main aim which was to eliminate the greatest number of sets from the transgression was reached with a lower budget using the methodology and mathematical models. The proposed methodology intends to be an additional tool to the electricity distribution companies (utilities). These companies usually plan specific works to reduce the continuity index or when they are pressed by regulatory agencies, they plan alternative programs which compete by the same budget facing traditional work programs.