Thesis (MSc (Mathematics))--University of Stellenbosch, 2010. / ENGLISH ABSTRACT: The first approach initiated by Merton [Mer69, Mer71] to solve utility maximization portfolio problems in continuous time is based on stochastic control theory. The idea of Merton was to interpret the maximization portfolio problem as a stochastic control problem where the trading strategies are considered as a control process and the portfolio wealth as the controlled process. Merton derived the Hamilton-Jacobi-Bellman (HJB) equation and for the special case of power, logarithm and exponential utility functions he produced a closedform solution. A principal disadvantage of this approach is the requirement of the Markov property for the stocks prices. The so-called martingale method represents the second approach for solving utility maximization portfolio problems in continuous time. It was introduced by Pliska [Pli86], Cox and Huang [CH89, CH91] and Karatzas et al. [KLS87] in di erent variant. It is constructed upon convex duality arguments and allows one to transform the initial dynamic portfolio optimization problem into a static one and to resolve it without requiring any \Markov" assumption. A de nitive answer (necessary and su cient conditions) to the utility maximization portfolio problem for terminal wealth has been obtained by Kramkov and Schachermayer [KS99]. In this thesis, we study the convex duality approach to the expected utility maximization problem (from terminal wealth) in continuous time stochastic markets, which as already mentioned above can be traced back to the seminal work by Merton [Mer69, Mer71]. Before we detail the structure of our thesis, we would like to emphasize that the starting point of our work is based on Chapter 7 in Pham [P09] a recent textbook. However, as the careful reader will notice, we have deepened and added important notions and results (such as the study of the upper (lower) hedge, the characterization of the essential supremum of all the possible prices, compare Theorem 7.2.2 in Pham [P09] with our stated Theorem 2.4.9, the dynamic programming equation 2.31, the superhedging theorem 2.6.1...) and we have made a considerable e ort in the proofs. Indeed, several proofs of theorems in Pham [P09] have serious gaps (not to mention typos) and even aws (for example see the proof of Proposition 7.3.2 in Pham [P09] and our proof of Proposition 3.4.8). In the rst chapter, we state the expected utility maximization problem and motivate the convex dual approach following an illustrative example by Rogers [KR07, R03]. We also brie y review the von Neumann - Morgenstern Expected Utility Theory. In the second chapter, we begin by formulating the superreplication problem as introduced by El Karoui and Quenez [KQ95]. The fundamental result in the literature on super-hedging is the dual characterization of the set of all initial endowments leading to a super-hedge of a European contingent claim. El Karoui and Quenez [KQ95] rst proved the superhedging theorem 2.6.1 in an It^o di usion setting and Delbaen and Schachermayer [DS95, DS98] generalized it to, respectively, a locally bounded and unbounded semimartingale model, using a Hahn-Banach separation argument. The superreplication problem inspired a very nice result, called the optional decomposition theorem for supermartingales 2.4.1, in stochastic analysis theory. This important theorem introduced by El Karoui and Quenez [KQ95], and extended in full generality by Kramkov [Kra96] is stated in Section 2.4 and proved at the end of Section 2.7. The third chapter forms the theoretical core of this thesis and it contains the statement and detailed proof of the famous Kramkov-Schachermayer Theorem that addresses the duality of utility maximization portfolio problems. Firstly, we show in Lemma 3.2.1 how to transform the dynamic utility maximization problem into a static maximization problem. This is done thanks to the dual representation of the set of European contingent claims, which can be dominated (or super-hedged) almost surely from an initial endowment x and an admissible self- nancing portfolio strategy given in Corollary 2.5 and obtained as a consequence of the optional decomposition of supermartingale. Secondly, under some assumptions on the utility function, the existence and uniqueness of the solution to the static problem is given in Theorem 3.2.3. Because the solution of the static problem is not easy to nd, we will look at it in its dual form. We therefore synthesize the dual problem from the primal problem using convex conjugate functions. Before we state the Kramkov-Schachermayer Theorem 3.4.1, we present the Inada Condition and the Asymptotic Elasticity Condition for Utility functions. For the sake of clarity, we divide the long and technical proof of Kramkov-Schachermayer Theorem 3.4.1 into several lemmas and propositions of independent interest, where the required assumptions are clearly indicate for each step of the proof. The key argument in the proof of Kramkov-Schachermayer Theorem is an in nitedimensional version of the minimax theorem (the classical method of nding a saddlepoint for the Lagrangian is not enough in our situation), which is central in the theory of Lagrange multipliers. For this, we have stated and proved the technical Lemmata 3.4.5 and 3.4.6. The main steps in the proof of the the Kramkov-Schachermayer Theorem 3.4.1 are: We show in Proposition 3.4.9 that the solution to the dual problem exists and we characterize it in Proposition 3.4.12. From the construction of the dual problem, we nd a set of necessary and su cient conditions (3.1.1), (3.1.2), (3.3.1) and (3.3.7) for the primal and dual problems to each have a solution. Using these conditions, we can show the existence of the solution to the given problem and characterize it in terms of the market parameters and the solution to the dual problem. In the last chapter we will present and study concrete examples of the utility maximization portfolio problem in speci c markets. First, we consider the complete markets case, where closed-form solutions are easily obtained. The detailed solution to the classical Merton problem with power utility function is provided. Lastly, we deal with incomplete markets under It^o processes and the Brownian ltration framework. The solution to the logarithmic utility function as well as to the power utility function is presented. / AFRIKAANSE OPSOMMING: Die eerste benadering, begin deur Merton [Mer69, Mer71], om nutsmaksimering portefeulje probleme op te los in kontinue tyd is gebaseer op stogastiese beheerteorie. Merton se idee is om die maksimering portefeulje probleem te interpreteer as 'n stogastiese beheer probleem waar die handelstrategi e as 'n beheer-proses beskou word en die portefeulje waarde as die gereguleerde proses. Merton het die Hamilton-Jacobi-Bellman (HJB) vergelyking afgelei en vir die spesiale geval van die mags, logaritmies en eksponensi ele nutsfunksies het hy 'n oplossing in geslote-vorm gevind. 'n Groot nadeel van hierdie benadering is die vereiste van die Markov eienskap vir die aandele pryse. Die sogenaamde martingale metode verteenwoordig die tweede benadering vir die oplossing van nutsmaksimering portefeulje probleme in kontinue tyd. Dit was voorgestel deur Pliska [Pli86], Cox en Huang [CH89, CH91] en Karatzas et al. [KLS87] in verskillende wisselvorme. Dit word aangevoer deur argumente van konvekse dualiteit, waar dit in staat stel om die aanvanklike dinamiese portefeulje optimalisering probleem te omvorm na 'n statiese een en dit op te los sonder dat' n \Markov" aanname gemaak hoef te word. 'n Bepalende antwoord (met die nodige en voldoende voorwaardes) tot die nutsmaksimering portefeulje probleem vir terminale vermo e is verkry deur Kramkov en Schachermayer [KS99]. In hierdie proefskrif bestudeer ons die konveks dualiteit benadering tot die verwagte nuts maksimering probleem (van terminale vermo e) in kontinue tyd stogastiese markte, wat soos reeds vermeld is teruggevoer kan word na die seminale werk van Merton [Mer69, Mer71]. Voordat ons die struktuur van ons tesis uitl^e, wil ons graag beklemtoon dat die beginpunt van ons werk gebaseer is op Hoofstuk 7 van Pham [P09] se onlangse handboek. Die noukeurige leser sal egter opmerk, dat ons belangrike begrippe en resultate verdiep en bygelas het (soos die studie van die boonste (onderste) verskansing, die karakterisering van die noodsaaklike supremum van alle moontlike pryse, vergelyk Stelling 7.2.2 in Pham [P09] met ons verklaarde Stelling 2.4.9, die dinamiese programerings vergelyking 2.31, die superverskansing stelling 2.6.1...) en ons het 'n aansienlike inspanning in die bewyse gemaak. Trouens, verskeie bewyse van stellings in Pham cite (P09) het ernstige gapings (nie te praat van setfoute nie) en selfs foute (kyk byvoorbeeld die bewys van Stelling 7.3.2 in Pham [P09] en ons bewys van Stelling 3.4.8). In die eerste hoofstuk, sit ons die verwagte nutsmaksimering probleem uit een en motiveer ons die konveks duaale benadering gebaseer op 'n voorbeeld van Rogers [KR07, R03]. Ons gee ook 'n kort oorsig van die von Neumann - Morgenstern Verwagte Nutsteorie. In die tweede hoofstuk, begin ons met die formulering van die superreplikasie probleem soos voorgestel deur El Karoui en Quenez [KQ95]. Die fundamentele resultaat in die literatuur oor super-verskansing is die duaale karakterisering van die versameling van alle eerste skenkings wat lei tot 'n super-verskans van' n Europese voorwaardelike eis. El Karoui en Quenez [KQ95] het eers die super-verskansing stelling 2.6.1 bewys in 'n It^o di usie raamwerk en Delbaen en Schachermayer [DS95, DS98] het dit veralgemeen na, onderskeidelik, 'n plaaslik begrensde en onbegrensde semimartingale model, met 'n Hahn-Banach skeidings argument. Die superreplikasie probleem het 'n prag resultaat ge nspireer, genaamd die opsionele ontbinding stelling vir supermartingales 2.4.1 in stogastiese ontledings teorie. Hierdie belangrike stelling wat deur El Karoui en Quenez [KQ95] voorgestel is en tot volle veralgemening uitgebrei is deur Kramkov [Kra96] is uiteengesit in Afdeling 2.4 en bewys aan die einde van Afdeling 2.7. Die derde hoofstuk vorm die teoretiese basis van hierdie proefskrif en bevat die verklaring en gedetailleerde bewys van die beroemde Kramkov-Schachermayer stelling wat die dualiteit van nutsmaksimering portefeulje probleme adresseer. Eerstens, wys ons in Lemma 3.2.1 hoe om die dinamiese nutsmaksimering probleem te omskep in 'n statiese maksimerings probleem. Dit kan gedoen word te danke aan die duaale voorstelling van die versameling Europese voorwaardelike eise, wat oorheers (of super-verskans) kan word byna seker van 'n aanvanklike skenking x en 'n toelaatbare self- nansierings portefeulje strategie wat in Gevolgtrekking 2.5 gegee word en verkry is as gevolg van die opsionele ontbinding van supermartingale. In die tweede plek, met sekere aannames oor die nutsfunksie, is die bestaan en uniekheid van die oplossing van die statiese probleem gegee in Stelling 3.2.3. Omdat die oplossing van die statiese probleem nie maklik verkrygbaar is nie, sal ons kyk na die duaale vorm. Ons sintetiseer dan die duale probleem van die prim^ere probleem met konvekse toegevoegde funksies. Voordat ons die Kramkov-Schachermayer Stelling 3.4.1 beskryf, gee ons die Inada voorwaardes en die Asimptotiese Elastisiteits Voorwaarde vir Nutsfunksies. Ter wille van duidelikheid, verdeel ons die lang en tegniese bewys van die Kramkov-Schachermayer Stelling ref in verskeie lemmas en proposisies op, elk van onafhanklike belang waar die nodige aannames duidelik uiteengesit is vir elke stap van die bewys. Die belangrikste argument in die bewys van die Kramkov-Schachermayer Stelling is 'n oneindig-dimensionele weergawe van die minimax stelling (die klassieke metode om 'n saalpunt vir die Lagrange-funksie te bekom is nie genoeg in die geval nie), wat noodsaaklik is in die teorie van Lagrange-multiplikators. Vir die, meld en bewys ons die tegniese Lemmata 3.4.5 en 3.4.6. Die belangrikste stappe in die bewys van die die Kramkov-Schachermayer Stelling 3.4.1 is: Ons wys in Proposisie 3.4.9 dat die oplossing vir die duale probleem bestaan en ons karaktiriseer dit in Proposisie 3.4.12. Uit die konstruksie van die duale probleem vind ons 'n versameling nodige en voldoende voorwaardes (3.1.1), (3.1.2), (3.3.1) en (3.3.7) wat die prim^ere en duale probleem oplossings elk moet aan voldoen. Deur hierdie voorwaardes te gebruik, kan ons die bestaan van die oplossing vir die gegewe probleem wys en dit karakteriseer in terme van die mark parameters en die oplossing vir die duale probleem. In die laaste hoofstuk sal ons konkrete voorbeelde van die nutsmaksimering portefeulje probleem bestudeer vir spesi eke markte. Ons kyk eers na die volledige markte geval waar geslote-vorm oplossings maklik verkrygbaar is. Die gedetailleerde oplossing vir die klassieke Merton probleem met mags nutsfunksie word voorsien. Ten slotte, hanteer ons onvolledige markte onderhewig aan It^o prosesse en die Brown ltrering raamwerk. Die oplossing vir die logaritmiese nutsfunksie, sowel as die mags nutsfunksie word aangebied.

In this thesis, we solve a mean-variance portfolio optimization problem with portfolio constraints under a regime-switching model. Specifically, we seek a portfolio process which minimizes the variance of the terminal wealth, subject to a terminal wealth constraint and convex portfolio constraints. The regime-switching is modeled using a finite state space, continuous-time Markov chain and the market parameters are allowed to be random processes. The solution to this problem is of interest to investors in financial markets, such as pension funds, insurance companies and individuals. We establish the existence and characterization of the solution to the given problem using a convex duality method. We encode the constraints on the given problem as static penalty functions in order to derive the primal problem. Next, we synthesize the dual problem from the primal problem using convex conjugate functions. We show that the solution to the dual problem exists. From the construction of the dual problem, we find a set of necessary and sufficient conditions for the primal and dual problems to each have a solution. Using these conditions, we can show the existence of the solution to the given problem and characterize it in terms of the market parameters and the solution to the dual problem. The results of the thesis lay the foundation to find an actual solution to the given problem, by looking at specific examples. If we can find the solution to the dual problem for a specific example, then, using the characterization of the solution to the given problem, we may be able to find the actual solution to the specific example. In order to use the convex duality method, we have to prove a martingale representation theorem for processes which are locally square-integrable martingales with respect to the filtration generated by a Brownian motion and a finite state space, continuous-time Markov chain. This result may be of interest in problems involving regime-switching models which require a martingale representation theorem.

In this thesis, we solve a mean-variance portfolio optimization problem with portfolio constraints under a regime-switching model. Specifically, we seek a portfolio process which minimizes the variance of the terminal wealth, subject to a terminal wealth constraint and convex portfolio constraints. The regime-switching is modeled using a finite state space, continuous-time Markov chain and the market parameters are allowed to be random processes. The solution to this problem is of interest to investors in financial markets, such as pension funds, insurance companies and individuals. We establish the existence and characterization of the solution to the given problem using a convex duality method. We encode the constraints on the given problem as static penalty functions in order to derive the primal problem. Next, we synthesize the dual problem from the primal problem using convex conjugate functions. We show that the solution to the dual problem exists. From the construction of the dual problem, we find a set of necessary and sufficient conditions for the primal and dual problems to each have a solution. Using these conditions, we can show the existence of the solution to the given problem and characterize it in terms of the market parameters and the solution to the dual problem. The results of the thesis lay the foundation to find an actual solution to the given problem, by looking at specific examples. If we can find the solution to the dual problem for a specific example, then, using the characterization of the solution to the given problem, we may be able to find the actual solution to the specific example. In order to use the convex duality method, we have to prove a martingale representation theorem for processes which are locally square-integrable martingales with respect to the filtration generated by a Brownian motion and a finite state space, continuous-time Markov chain. This result may be of interest in problems involving regime-switching models which require a martingale representation theorem.

Some financial problems as minimizing the shortfall risk when hedging in incomplete markets lead to problems belonging to test theory. This paper considers a generalization of the Neyman-Pearson lemma. With methods of convex duality we deduce the structure of an optimal randomized test when testing a compound hypothesis against a simple alternative. We give necessary and sufficient optimality conditions for the problem.

This dissertation studies a class of path-dependent stochastic control problems with applications to Finance. In particular, we solve the open problem of the continuous time expected utility maximization with addictive consumption habit formation in incomplete markets under two independent scenarios. In the first project, we study the continuous time utility optimization problem with consumption habit formation in general incomplete semimartingale financial markets. Introducing the set of auxiliary state processes and the modified dual space, we embed our original problem into an abstract time-separable utility maximization problem with a shadow random endowment on the product space. We establish existence and uniqueness of the optimal solution using convex duality by defining the primal value function as depending on two variables, i.e., the initial wealth and the initial standard of living. We also provide market independent sufficient conditions both on the stochastic discounting processes of the habit formation process and on the utility function for the well-posedness of our original optimization problem. Under the same assumptions, we can carefully modify the classical proofs in the approach of convex duality analysis when the auxiliary dual process is not necessarily integrable. In the second project, we examine an example of the optimal investment and consumption problem with both habit-formation and partial observations in incomplete markets driven by It\^{o} processes. The individual investor develops addictive consumption habits gradually while only observing the market stock prices but not the instantaneous rates of return, which follow an Ornstein-Uhlenbeck process. Applying the Kalman-Bucy filtering theorem and Dynamic Programming arguments, we solve the associated Hamilton-Jacobi-Bellman(HJB) equation fully explicitly for this path dependent stochastic control problem in the case of power utility preferences. We provide the optimal investment and consumption policy in explicit feedback form using rigorous verification arguments. / text

In this thesis we consider the application of Fenchel's duality theory and gradient-based methods for the training and hyperparameter optimization of Support Vector Machines. We show that the dualization of convex training problems is possible theoretically in a rather general formulation. For training problems following a special structure (for instance, standard training problems) we find that the resulting optimality conditions can be interpreted concretely. This approach immediately leads to the well-known notion of support vectors and a formulation of the Representer Theorem. The proposed theory is applied to several examples such that dual formulations of training problems and associated optimality conditions can be derived straightforwardly. Furthermore, we consider different formulations of the primal training problem which are equivalent under certain conditions. We also argue that the relation of the corresponding solutions to the solution of the dual training problem is not always intuitive. Based on the previous findings, we consider the application of customized optimization methods to the primal and dual training problems. A particular realization of Newton's method is derived which could be used to solve the primal training problem accurately. Moreover, we introduce a general convergence framework covering different types of decomposition methods for the solution of the dual training problem. In doing so, we are able to generalize well-known convergence results for the SMO method. Additionally, a discussion of the complexity of the SMO method and a motivation for a shrinking strategy reducing the computational effort is provided. In a last theoretical part, we consider the problem of hyperparameter optimization. We argue that this problem can be handled efficiently by means of gradient-based methods if the training problems are formulated appropriately. Finally, we evaluate the theoretical results concerning the training and hyperparameter optimization approaches practically by means of several example training problems.

Some financial problems as minimizing the shortfall risk when hedging in incomplete markets lead to problems belonging to test theory. This paper considers a generalization of the Neyman-Pearson lemma. With methods of convex duality we deduce the structure of an optimal randomized test when testing a compound hypothesis against a simple alternative. We give necessary and sufficient optimality conditions for the problem.

Les familles exponentielles sont une classe de modèles omniprésente en statistique. D'une part, elle peut modéliser n'importe quel type de données. En fait la plupart des distributions communes en font partie : Gaussiennes, variables catégoriques, Poisson, Gamma, Wishart, Dirichlet. D'autre part elle est à la base des modèles linéaires généralisés (GLM), une classe de modèles fondamentale en apprentissage automatique. Enfin les mathématiques qui les sous-tendent sont souvent magnifiques, grâce à leur lien avec la dualité convexe et la transformée de Laplace. L'auteur de cette thèse a fréquemment été motivé par cette beauté. Dans cette thèse, nous faisons trois contributions à l'intersection de l'optimisation et des statistiques, qui tournent toutes autour de la famille exponentielle. La première contribution adapte et améliore un algorithme d'optimisation à variance réduite appelé ascension des coordonnées duales stochastique (SDCA), pour entraîner une classe particulière de GLM appelée champ aléatoire conditionnel (CRF). Les CRF sont un des piliers de la prédiction structurée. Les CRF étaient connus pour être difficiles à entraîner jusqu'à la découverte des technique d'optimisation à variance réduite. Notre version améliorée de SDCA obtient des performances favorables comparées à l'état de l'art antérieur et actuel. La deuxième contribution s'intéresse à la découverte causale. Les familles exponentielles sont fréquemment utilisées dans les modèles graphiques, et en particulier dans les modèles graphique causaux. Cette contribution mène l'enquête sur une conjecture spécifique qui a attiré l'attention dans de précédents travaux : les modèles causaux s'adaptent plus rapidement aux perturbations de l'environnement. Nos résultats, obtenus à partir de théorèmes d'optimisation, soutiennent cette hypothèse sous certaines conditions. Mais sous d'autre conditions, nos résultats contredisent cette hypothèse. Cela appelle à une précision de cette hypothèse, ou à une sophistication de notre notion de modèle causal. La troisième contribution s'intéresse à une propriété fondamentale des familles exponentielles. L'une des propriétés les plus séduisantes des familles exponentielles est la forme close de l'estimateur du maximum de vraisemblance (MLE), ou maximum a posteriori (MAP) pour un choix naturel de prior conjugué. Ces deux estimateurs sont utilisés presque partout, souvent sans même y penser. (Combien de fois calcule-t-on une moyenne et une variance pour des données en cloche sans penser au modèle Gaussien sous-jacent ?) Pourtant la littérature actuelle manque de résultats sur la convergence de ces modèles pour des tailles d'échantillons finis, lorsque l'on mesure la qualité de ces modèles avec la divergence de Kullback-Leibler (KL). Pourtant cette divergence est la mesure de différence standard en théorie de l'information. En établissant un parallèle avec l'optimisation, nous faisons quelques pas vers un tel résultat, et nous relevons quelques directions pouvant mener à des progrès, tant en statistiques qu'en optimisation. Ces trois contributions mettent des outil d'optimisation au service des statistiques dans les familles exponentielles : améliorer la vitesse d'apprentissage de GLM de prédiction structurée, caractériser la vitesse d'adaptation de modèles causaux, estimer la vitesse d'apprentissage de modèles omniprésents. En traçant des ponts entre statistiques et optimisation, cette thèse fait progresser notre maîtrise de méthodes fondamentales d'apprentissage automatique. / Exponential families are a ubiquitous class of models in statistics. On the one hand, they can model any data type. Actually, the most common distributions are exponential families: Gaussians, categorical, Poisson, Gamma, Wishart, or Dirichlet. On the other hand, they sit at the core of generalized linear models (GLM), a foundational class of models in machine learning. They are also supported by beautiful mathematics thanks to their connection with convex duality and the Laplace transform. This beauty is definitely responsible for the existence of this thesis. In this manuscript, we make three contributions at the intersection of optimization and statistics, all revolving around exponential families. The first contribution adapts and improves a variance reduction optimization algorithm called stochastic dual coordinate ascent (SDCA) to train a particular class of GLM called conditional random fields (CRF). CRF are one of the cornerstones of structured prediction. CRF were notoriously hard to train until the advent of variance reduction techniques, and our improved version of SDCA performs favorably compared to the previous state-of-the-art. The second contribution focuses on causal discovery. Exponential families are widely used in graphical models, and in particular in causal graphical models. This contribution investigates a specific conjecture that gained some traction in previous work: causal models adapt faster to perturbations of the environment. Using results from optimization, we find strong support for this assumption when the perturbation is coming from an intervention on a cause, and support against this assumption when perturbation is coming from an intervention on an effect. These pieces of evidence are calling for a refinement of the conjecture. The third contribution addresses a fundamental property of exponential families. One of the most appealing properties of exponential families is its closed-form maximum likelihood estimate (MLE) and maximum a posteriori (MAP) for a natural choice of conjugate prior. These two estimators are used almost everywhere, often unknowingly -- how often are mean and variance computed for bell-shaped data without thinking about the Gaussian model they underly? Nevertheless, literature to date lacks results on the finite sample convergence property of the information (Kulback-Leibler) divergence between these estimators and the true distribution. Drawing on a parallel with optimization, we take some steps towards such a result, and we highlight directions for progress both in statistics and optimization. These three contributions are all using tools from optimization at the service of statistics in exponential families: improving upon an algorithm to learn GLM, characterizing the adaptation speed of causal models, and estimating the learning speed of ubiquitous models. By tying together optimization and statistics, this thesis is taking a step towards a better understanding of the fundamentals of machine learning.

The aim of this thesis is to present new results concerning duality in scalar optimization. We show how the theory can be applied to optimization problems arising in the theory of risk measures, portfolio optimization and machine learning. First we give some notations and preliminaries we need within the thesis. After that we recall how the well-known Lagrange dual problem can be derived by using the general perturbation theory and give some generalized interior point regularity conditions used in the literature. Using these facts we consider some special scalar optimization problems having a composed objective function and geometric (and cone) constraints. We derive their duals, give strong duality results and optimality condition using some regularity conditions. Thus we complete and/or extend some results in the literature especially by using the mentioned regularity conditions, which are weaker than the classical ones. We further consider a scalar optimization problem having single chance constraints and a convex objective function. We also derive its dual, give a strong duality result and further consider a special case of this problem. Thus we show how the conjugate duality theory can be used for stochastic programming problems and extend some results given in the literature. In the third chapter of this thesis we consider convex risk and deviation measures. We present some more general measures than the ones given in the literature and derive formulas for their conjugate functions. Using these we calculate some dual representation formulas for the risk and deviation measures and correct some formulas in the literature. Finally we proof some subdifferential formulas for measures and risk functions by using the facts above. The generalized deviation measures we introduced in the previous chapter can be used to formulate some portfolio optimization problems we consider in the fourth chapter. Their duals, strong duality results and optimality conditions are derived by using the general theory and the conjugate functions, respectively, given in the second and third chapter. Analogous calculations are done for a portfolio optimization problem having single chance constraints using the general theory given in the second chapter. Thus we give an application of the duality theory in the well-developed field of portfolio optimization. We close this thesis by considering a general Support Vector Machines problem and derive its dual using the conjugate duality theory. We give a strong duality result and necessary as well as sufficient optimality conditions. By considering different cost functions we get problems for Support Vector Regression and Support Vector Classification. We extend the results given in the literature by dropping the assumption of invertibility of the kernel matrix. We use a cost function that generalizes the well-known Vapnik's ε-insensitive loss and consider the optimization problems that arise by using this. We show how the general theory can be applied for a real data set, especially we predict the concrete compressive strength by using a special Support Vector Regression problem.

The aim of this thesis is to present new results concerning duality in scalar optimization. We show how the theory can be applied to optimization problems arising in the theory of risk measures, portfolio optimization and machine learning. First we give some notations and preliminaries we need within the thesis. After that we recall how the well-known Lagrange dual problem can be derived by using the general perturbation theory and give some generalized interior point regularity conditions used in the literature. Using these facts we consider some special scalar optimization problems having a composed objective function and geometric (and cone) constraints. We derive their duals, give strong duality results and optimality condition using some regularity conditions. Thus we complete and/or extend some results in the literature especially by using the mentioned regularity conditions, which are weaker than the classical ones. We further consider a scalar optimization problem having single chance constraints and a convex objective function. We also derive its dual, give a strong duality result and further consider a special case of this problem. Thus we show how the conjugate duality theory can be used for stochastic programming problems and extend some results given in the literature. In the third chapter of this thesis we consider convex risk and deviation measures. We present some more general measures than the ones given in the literature and derive formulas for their conjugate functions. Using these we calculate some dual representation formulas for the risk and deviation measures and correct some formulas in the literature. Finally we proof some subdifferential formulas for measures and risk functions by using the facts above. The generalized deviation measures we introduced in the previous chapter can be used to formulate some portfolio optimization problems we consider in the fourth chapter. Their duals, strong duality results and optimality conditions are derived by using the general theory and the conjugate functions, respectively, given in the second and third chapter. Analogous calculations are done for a portfolio optimization problem having single chance constraints using the general theory given in the second chapter. Thus we give an application of the duality theory in the well-developed field of portfolio optimization. We close this thesis by considering a general Support Vector Machines problem and derive its dual using the conjugate duality theory. We give a strong duality result and necessary as well as sufficient optimality conditions. By considering different cost functions we get problems for Support Vector Regression and Support Vector Classification. We extend the results given in the literature by dropping the assumption of invertibility of the kernel matrix. We use a cost function that generalizes the well-known Vapnik's ε-insensitive loss and consider the optimization problems that arise by using this. We show how the general theory can be applied for a real data set, especially we predict the concrete compressive strength by using a special Support Vector Regression problem.