This thesis focuses on ridge regression (RR) and least absolute shrinkage and selection operator (lasso). Ridge properties are being investigated in great detail which include studying the bias, the variance and the mean squared error as a function of the tuning parameter. We also study the convexity of the trace of the mean squared error in terms of the tuning parameter. In addition, we examined some special properties of RR for factorial experiments. Not only do we review ridge properties, we also review lasso properties because they are somewhat similar. Rather than shrinking the estimates toward zero in RR, the lasso is able to provide a sparse solution, setting many coefficient estimates exaclty to zero. Furthermore, we try a new approach to solve the lasso problem by formulating it as a bilevel problem and implementing a new algorithm to solve this bilevel program. / Graduate

Apresentamos dois novos métodos para a solução de problemas de otimização convexa em dois níveis não necessariamente diferenciáveis, i.e., mostramos que as sequências geradas por ambos os métodos convergem para o conjunto ótimo de uma função não suave sujeito a um conjunto que também envolve a minimização de uma função não diferenciável. Ambos os algoritmos dispensam qualquer tipo de resolução de subproblemas ou busca linear durante suas iterações. Ao final, para demonstrar que os métodos são viáveis, resolvemos um problema de reconstrução de imagens tomográficas / We present two new methods for solving bilevel convex optimization problems, where both functions are not necessarily differentiable, i.e., we show that the sequences generated by those methods converge to the optimal set of a nonsmooth function subject to a set that also involves a function minimization. Both algorithms do not require any kind of subproblems resolution or linear search during the iterations. At the end, to prove that our methods are viable, we solve a problem of tomographic image reconstruction

Optimal parametric design of a system must be able to respond quickly to short term needs as well as long term conditions. To this end, we present an Assess-Predict-Optimize (APO) strategy which allows for easy modification of a system’s characteristics and constraints, enabling quick design adaptation. There are three components to the APO strategy: Assess - extract necessary information from given data; Predict - predict future behavior of system; and Optimize – obtain optimal system configuration based on information from the other components. The APO strategy utilizes three key mathematical ingredients to yield real-time results which would certainly conform to given constraints: dimension reduction of the model, a posteriori error estimation, and optimization methods. The resulting formulation resembles a bilevel optimization problem with an inherent nonconvexity in the inner level. Using a simple infiltration-evaporation model to simulate an irrigation system, we demonstrate the APO strategy’s ability to yield real-time optimal results. The linearized model, described by a coercive elliptic partial differential equation, is discretized by the reduced-basis output bounds method. A primal-dual interior point method is then chosen to solve the resulting APO problem. / Singapore-MIT Alliance (SMA)

Apresentamos dois novos métodos para a solução de problemas de otimização convexa em dois níveis não necessariamente diferenciáveis, i.e., mostramos que as sequências geradas por ambos os métodos convergem para o conjunto ótimo de uma função não suave sujeito a um conjunto que também envolve a minimização de uma função não diferenciável. Ambos os algoritmos dispensam qualquer tipo de resolução de subproblemas ou busca linear durante suas iterações. Ao final, para demonstrar que os métodos são viáveis, resolvemos um problema de reconstrução de imagens tomográficas / We present two new methods for solving bilevel convex optimization problems, where both functions are not necessarily differentiable, i.e., we show that the sequences generated by those methods converge to the optimal set of a nonsmooth function subject to a set that also involves a function minimization. Both algorithms do not require any kind of subproblems resolution or linear search during the iterations. At the end, to prove that our methods are viable, we solve a problem of tomographic image reconstruction

Demand Response (DR) was proposed more than a decade ago to incentivise customers to shift their electricity usage from peak demand periods to off-peak demand periods and to curtail their electricity usage during peak demand periods. However, the lack of two-way communication infrastructure weakens the influence of DR and limits its applications. With the development of smart grid facilities (e.g. smart meters and the two-way communication infrastructure) that enable the interactions between the energy retailer and its customers, demand response shows great potential to reduce customers' bills, increase the retailer's profit and further stabilize the power systems. Given such a context, in this thesis we propose smart pricing based demand response programs to study the interactions between the energy retailer and its customers based on game-theory and machine learning techniques. We conduct the research in two different application scenarios: 1) For customers with home energy management system (HEMS) installed in their smart meters, the retailer will know the customers' energy consumption patterns by interacting with the HEMS. As a result, the smart pricing based demand response problem can be modelled as a Stackelberg game or bilevel optimization problem. Further, efficient solutions are proposed for the demand response problems and the existence of optimal solution to the Stackelberg game and the bilevel model is proved; 2) For customers without HEMS installed in their smart meters, the retailer will not know the energy consumption patterns of these customers and must learn customers' behaviour patterns via historical energy usage data. To realize this, two appliance-level machine learning algorithms are proposed to learn customers' consumption patterns. Further, distributed pricing algorithms are proposed for the retailer to solve the demand response problem effectively. Simulation results indicate the effectiveness of the proposed demand response models in both application scenarios.

We present a novel bilevel programming formulation that aims to solve a resource allocation problem for request-response systems. Our formulation is motivated by potential inefficiencies in the allocation of computational resources to incoming user requests in such systems. In our experience, systems often operate with a surplus of resources despite potentially incurring unjustifiable cost. Our work attempts to optimize the tradeoff between the financial cost of resources and the opportunity cost of unfulfilled user demand. Our bilevel formulation consists of an \textit{upper} problem which has a constraint value appearing in the \textit{lower} problem. We derive efficient methods for finding global solutions to the upper problem in two settings; first with logarithmic utility functions, and then with a particular type of sigmoidal utility function. A solution to the model we describe (1) determines the optimal number of total resources to allocate and (2) determines the optimal distribution of such resources across the set of user requests.

O objetivo deste trabalho é propor uma metodologia para a priorização de obras do sistema de distribuição de média tensão - até 36 kV - voltada ao cumprimento do índice de continuidade DEC e FEC imposto pela ANEEL, visando reduzir a quantidade de conjuntos que estão fora dos limites e que geram multas para a empresa frente ao órgão regulador e aos consumidores. Inicialmente, os diversos tipos de obras têm seu benefício calculado com o uso do Método do Payoff Simplificado, baseado no Método do Payoff COPEL, que consiste na extração somente da parcela relativa a interrupção, no DEC ou FEC, que determinada obra trará ao sistema. De posse deste benefício estimado, as obras foram analisadas de duas maneiras: geral e por conjunto. A análise Geral consiste em observar as obras propostas de maneira independente, preocupando-se com o benefício que elas trarão para a empresa como um todo. Na análise por conjunto, as obras são agrupadas por conjunto ANEEL, e o objetivo é a colocação da maior quantidade de conjuntos dentro dos limites de continuidade impostos pelo órgão regulador. A definição do objetivo apropriado é que irá orientar todo o processo de seleção das obras. Para isso são propostos modelos matemáticos, e para trabalhar com eles, foi utilizada como ferramenta a programação matemática. Foram realizadas simulações divididas em dois grupos: no primeiro, análise geral, a otimização é executada diretamente. Já no segundo, na análise por conjunto, é aplicada a programação multi-nível, mais especificamente, a programação em dois níveis (\"Bilevel Programming Problem\"), utilizando a programação inteira ou por metas (\"goal programming\"). Os resultados das simulações mostraram que o objetivo principal, que é tirar a maior quantidade de conjuntos da transgressão, foi atingido com menor orçamento com o uso da metodologia e dos modelos matemáticos empregados neste trabalho. A metodologia proposta pretende ser uma ferramenta adicional para as concessionárias de distribuição de energia elétrica que normalmente elaboram programas de obras específicos para redução de índices de continuidade ou quando pressionados pelo órgão regulador elaboram programas alternativos que competem pelo mesmo orçamento frente aos programas de obras tradicionais. / The purpose of this paper is to propose a methodology to prioritize planned works in the medium-voltage distribution system - up to 36 kV - aiming to adequate the DEC and FEC continuity index to the limits defined by the Brazilian regulatory agency (ANEEL) through the reduction of the number of sets out of target and consequently the reduction of monetary penalties to the utility imposed by the regulatory agency and consumers. At first every planned work has its benefit calculated by the Simplified Payoff Method which is based on COPEL Payoff Method and which consists in extracting just the interruption event from the DEC or FEC which a given work will bring to the system. Once you have got the estimated benefit, the planned works are analyzed in two different ways - general analysis and set analysis. General analysis consists in checking up proposed works independently, focusing on the benefit they will bring to the company as a whole. In the set analysis, works are grouped by \"ANEEL sets\" and the main aim is to gather the greatest number of sets into the continuity limits defined by the regulatory agency. The aims definition will lead the whole work selection process. To achieve that mathematical models are proposed and mathematical programming tools are used. Two groups of simulations were done - in the first one which is also called general analysis, optimization is executed directly. The second one called set analysis, is applied the bilevel programming using the integer programming or goal programming. The simulation results showed that the main aim which was to eliminate the greatest number of sets from the transgression was reached with a lower budget using the methodology and mathematical models. The proposed methodology intends to be an additional tool to the electricity distribution companies (utilities). These companies usually plan specific works to reduce the continuity index or when they are pressed by regulatory agencies, they plan alternative programs which compete by the same budget facing traditional work programs.

<p>We analyse several facets of bilevel decision problems under uncertainty. These problems can be interpreted as an extension of stochastic programming problems where part of the uncertainty is attributed to the behaviour of another actor.</p><p>The field of decision making under uncertainty with bilevel features is quite new and most approaches focus on the interactions and relations between the decision makers. In contrast to these studies, the approach of bilevel stochastic programming pursued here stresses the stochastic programming aspect of the problem formulation. The framework enables a direct application of stochastic programming concepts and solution methods to the bilevel relationship between the actors. Thus more complex problem structures can be studied and the aspect of uncertainty can be treated adequately.</p><p>Our analysis covers both theoretical and more practically oriented issues. We study different formulations of one and two stage bilevel stochastic programming problems and state necessary optimality conditions for each of the problem instances. Additionally we present a solution algorithm utilising a stochastic quasi-gradient method. A further study is concerned with the uniqueness of the minima of a convex stochastic programming problem with uncertainty about the decision variables. We state conditions on the distribution of the parameters representing the uncertainty such that the minima of the optimisation problem are unique. We formulate a model of competition and collaboration of two different types of telecom service providers, the owner of a bottleneck facility and a virtual network operator. This represents an application of a bilevel stochastic programming formulation to a liberalised telecommunications environment. Furthermore, the utilisation of the bilevel stochastic programming framework and the developed solution concepts for the analysis of principal agent models is demonstrated. Also here the background of a regulated telecom environment, more specific the relations between a regulator and a regulated telecommunications company, was chosen.</p>

We analyse several facets of bilevel decision problems under uncertainty. These problems can be interpreted as an extension of stochastic programming problems where part of the uncertainty is attributed to the behaviour of another actor. The field of decision making under uncertainty with bilevel features is quite new and most approaches focus on the interactions and relations between the decision makers. In contrast to these studies, the approach of bilevel stochastic programming pursued here stresses the stochastic programming aspect of the problem formulation. The framework enables a direct application of stochastic programming concepts and solution methods to the bilevel relationship between the actors. Thus more complex problem structures can be studied and the aspect of uncertainty can be treated adequately. Our analysis covers both theoretical and more practically oriented issues. We study different formulations of one and two stage bilevel stochastic programming problems and state necessary optimality conditions for each of the problem instances. Additionally we present a solution algorithm utilising a stochastic quasi-gradient method. A further study is concerned with the uniqueness of the minima of a convex stochastic programming problem with uncertainty about the decision variables. We state conditions on the distribution of the parameters representing the uncertainty such that the minima of the optimisation problem are unique. We formulate a model of competition and collaboration of two different types of telecom service providers, the owner of a bottleneck facility and a virtual network operator. This represents an application of a bilevel stochastic programming formulation to a liberalised telecommunications environment. Furthermore, the utilisation of the bilevel stochastic programming framework and the developed solution concepts for the analysis of principal agent models is demonstrated. Also here the background of a regulated telecom environment, more specific the relations between a regulator and a regulated telecommunications company, was chosen.

This thesis addresses two important classes of optimization : multiobjective optimization and bilevel optimization. The investigation concerns their solution methods, applications, and possible links between them. First of all, we develop a procedure for solving Multiple Objective Linear Programming Problems (MOLPP). The method is based on a new characterization of efficient faces. It exploits the connectedness property of the set of ideal tableaux associated to degenerated points in the case of degeneracy. We also develop an approach for solving Bilevel Linear Programming Problems (BLPP). It is based on the result that an optimal solution of the BLPP is reachable at an extreme point of the underlying region. Consequently, we develop a pivoting technique to find the global optimal solution on an expanded tableau that represents the data of the BLPP. The solutions obtained by our algorithm on some problems available in the literature show that these problems were until now wrongly solved. Some applications of these two areas of optimization problems are explored. An application of multicriteria optimization techniques for finding an optimal planning for the distribution of electrical energy in Cameroon is provided. Similary, a bilevel optimization model that could permit to protect any economic sector where local initiatives are threatened is proposed. Finally, the relationship between the two classes of optimization is investigated. We first look at the conditions that guarantee that the optimal solution of a given BPP is Pareto optimal for both upper and lower level objective functions. We then introduce a new relation that establishes a link between MOLPP and BLPP. Moreover, we show that, to solve a BPP, it is possible to solve two artificial M0PPs. In addition, we explore Bilevel Multiobjective Programming Problem (BMPP), a case of BPP where each decision maker (DM) has more than one objective function. Given a MPP, we show how to construct two artificial M0PPs such that any point that is efficient for both problems is also efficient for the BMPP. For the linear case specially, we introduce an artificial MOLPP such that its resolution can permit to generate the whole feasible set of the leader DM. Based on this result and depending on whether the leader can evaluate or not his preferences for his different objective functions, two approaches for obtaining efficient solutions are presented