This dissertation aims to develop efficient algorithms with improved scalability and stability properties for large-scale optimization and optimization under uncertainty, and to bridge some of the gaps between modern optimization theories and recent applications emerging in the Big Data environment. To this end, the dissertation is dedicated to two important subjects -- i) Large-scale Convex Composite Optimization and ii) Error-in-Measurement Optimization. In spite of the different natures of these two topics, the common denominator, to be presented, lies in their accommodation for systematic use of saddle point techniques for mathematical modeling and numerical processing. The main body can be split into three parts. In the first part, we consider a broad class of variational inequalities with composite structures, allowing to cover the saddle point/variational analogies of the classical convex composite minimization (i.e. summation of a smooth convex function and a simple nonsmooth convex function). We develop novel composite versions of the state-of-the-art Mirror Descent and Mirror Prox algorithms aimed at solving such type of problems. We demonstrate that the algorithms inherit the favorable efficiency estimate of their prototypes when solving structured variational inequalities. Moreover, we develop several variants of the composite Mirror Prox algorithm along with their corresponding complexity bounds, allowing the algorithm to handle the case of imprecise prox mapping as well as the case when the operator is represented by an unbiased stochastic oracle. In the second part, we investigate four general types of large-scale convex composite optimization problems, including (a) multi-term composite minimization, (b) linearly constrained composite minimization, (c) norm-regularized nonsmooth minimization, and (d) maximum likelihood Poisson imaging. We demonstrate that the composite Mirror Prox, when integrated with saddle point techniques and other algorithmic tools, can solve all these optimization problems with the best known so far rates of convergences. Our main related contributions are as follows. Firstly, regards to problems of type (a), we develop an optimal algorithm by integrating the composite Mirror Prox with a saddle point reformulation based on exact penalty. Secondly, regards to problems of type (b), we develop a novel algorithm reducing the problem to solving a ``small series'' of saddle point subproblems and achieving an optimal, up to log factors, complexity bound. Thirdly, regards to problems of type (c), we develop a Semi-Proximal Mirror-Prox algorithm by leveraging the saddle point representation and linear minimization over problems' domain and attain optimality both in the numbers of calls to the first order oracle representing the objective and calls to the linear minimization oracle representing problem's domain. Lastly, regards to problem (d), we show that the composite Mirror Prox when applied to the saddle point reformulation circumvents the difficulty with non-Lipschitz continuity of the objective and exhibits better convergence rate than the typical rate for nonsmooth optimization. We conduct extensive numerical experiments and illustrate the practical potential of our algorithms in a wide spectrum of applications in machine learning and image processing. In the third part, we examine error-in-measurement optimization, referring to decision-making problems with data subject to measurement errors; such problems arise naturally in a number of important applications, such as privacy learning, signal processing, and portfolio selection. Due to the postulated observation scheme and specific structure of the problem, straightforward application of standard stochastic optimization techniques such as Stochastic Approximation (SA) and Sample Average Approximation (SAA) are out of question. Our goal is to develop computationally efficient and, hopefully, not too conservative data-driven techniques applicable to a broad scope of problems and allowing for theoretical performance guarantees. We present two such approaches -- one depending on a fully algorithmic calculus of saddle point representations of convex-concave functions and the other depending on a general approximation scheme of convex stochastic programming. Both approaches allow us to convert the problem of interests to a form amenable for SA or SAA. The latter developments are primarily focused on two important applications -- affine signal processing and indirect support vector machines.

This thesis considers the practical problem of constrained and unconstrained local optimization. This subject has been well studied when the objective function f is assumed to smooth. However, nonsmooth problems occur naturally and frequently in practice. Here f is assumed to be nonsmooth or discontinuous without forcing smoothness assumptions near, or at, a potential solution. Various methods have been presented by others to solve nonsmooth optimization problems, however only partial convergence results are possible for these methods. In this thesis, an optimization method which use a series of local and localized global optimization phases is proposed. The local phase searches for a local minimum and gives the methods numerical performance on parts of f which are smooth. The localized global phase exhaustively searches for points of descent in a neighborhood of cluster points. It is the localized global phase which provides strong theoretical convergence results on nonsmooth problems. Algorithms are presented for solving bound constrained, unconstrained and constrained nonlinear nonsmooth optimization problems. These algorithms use direct search methods in the local phase as they can be applied directly to nonsmooth problems because gradients are not explicitly required. The localized global optimization phase uses a new partitioning random search algorithm to direct random sampling into promising subsets of ℝⁿ. The partition is formed using classification and regression trees (CART) from statistical pattern recognition. The CART partition defines desirable subsets where f is relatively low, based on previous sampling, from which further samples are drawn directly. For each algorithm, convergence to an essential local minimizer of f is demonstrated under mild conditions. That is, a point x* for which the set of all feasible points with lower f values has Lebesgue measure zero for all sufficiently small neighborhoods of x*. Stopping rules are derived for each algorithm giving practical convergence to estimates of essential local minimizers. Numerical results are presented on a range of nonsmooth test problems for 2 to 10 dimensions showing the methods are effective in practice.

Assume that {φn} is an orthonormal uniformly bounded (ONB) sequence of complex-valued functions de ned on a measure space (Ω,Σ,µ), and f ∈ L1(Ω,Σ,µ). Let be the Fourier coefficients of f with respect to {φn} . R.E.A.C. Paley proved a theorem connecting the Lp-norm of f with a related norm of the sequence {cn}. Hardy and Littlewood subsequently proved that Paley’s result is best possible within its context. Their results were generalized by Dikarev, Macaev, Askey, Wainger, Sagher, and later by Tikhonov, Li yand, Booton and others.The present work continues the generalization of these results. / Includes bibliography. / Dissertation (Ph.D.)--Florida Atlantic University, 2014. / FAU Electronic Theses and Dissertations Collection

Made available in DSpace on 2014-06-11T19:22:18Z (GMT). No. of bitstreams: 0 Previous issue date: 2011-02-28Bitstream added on 2014-06-13T20:48:40Z : No. of bitstreams: 1 costa_tm_me_sjrp.pdf: 1425800 bytes, checksum: f5b08954e14201ee5211145299b1e813 (MD5) / Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES) / Neste trabalho, estamos interessados em apresentar uma abordagem relacionando a análise não suave com a otimização. Primeiramente, é realizado um estudo sobre conceitos da análise não suave, como cones normais, cone tangente de Bouligand, subdiferenciais proximal, estrita, limite e de clarke. Com esses conceitos exibimos uma série de resultados, por exemplo, uma caracterização par funções de Lipschitz, subdiferencais da soma, produto e máximo de funções semi-contínuas inferior, uma versão não suave dos multiplicadores de Lagrange, i.e., condições de primeira ordem para otimalidade de problemas de otimização não suaves. Também é feito um estudo sobre as condições de segunda ordem para otimalidade em problemas de otimização não suaves e para isso, foi necessário a apresentação de outros conceitos e propriedades como os de Hessiana generalizada, Jacobiana aproximada a Hessiana proximada. Após a apresentação desses resultados, é feita uma análise sobre dois Teoremas que fornecem, com abordagens distintas, condições suficiente de segunda ordem para problemas de otimização não suaves e este trabalho é finalizado com a aprsentação de um resultado que é considerado uma unificação desses dois Teoremas / In this work we are interested in the presentation of an approach relating Nonsmooth Analysis to Optimization. First we make a study about concepts of nonsmooth analysis such as, normal cone, Bouligand's tangent cone, proximal, strict and limiting Subdiferential, as well as Clarke's Suddifferential. After these, we exhibit a series of results, for example, a characterization of Lipschitz functions, Subdifferential sum, product and maxium rules of lower semicontinuous functions and a nonsmooth version of Lagrange's multiplier rule, that is, a first order necessary condition of optimality for nonsmooth optimization problems. We also made a study about second order optimality conditions for nonsmooth optimization problems. In order to do that, it was necessary to present other concepts and properties about generalized Hessian, approximate Jacobian and approximate Hessian. After presenting these concepts and results, an analysis of two theorems that provide, with different approches, second order conditions for optimality for nonsmooth problems is made. Finally, this dissertation is completed with the exposition of a result that is considered a unification of these two theorems

In this thesis, various numerical methods are developed to solve nonsmooth and in particular, nonconvex optimization problems. More speci

This thesis is concerned with the modeling and numerical study of nonsmooth dynamical systems (NSDS). The first part of the thesis deals with the modeling of some DC-DC power converters using the complementarity formalism. This mathematical theoretical framework allows us to ensure existence and uniqueness of solutions in a "natural" and synthetic way. Specifically, it works pretty well in power electronic converters because it incorporates generalized discontinuous conduction modes (GDCM), characterized by a reduction of the dimension of the effective dynamics. For systems with a single diode, analytical state-space conditions for the presence of a GDCM are stated and simulation results, showing a variety of behaviours, such as persistent or re-entering GDCM, are also presented. Furthermore, the analysis and simulation of a parallel resonant converter (PRC), which has four diodes, illustrate the convenience of the complementarity formalism to simulate electrical systems with a large number of ideal diodes. We also present the simulation of a boost converter with a sliding mode control, even though a general control theory for complementarity systems is not still developed.In the second part of the thesis we focus on the bifurcation analysis in NSDS, and in particular, we have studied different mechanical systems which involve impacts and dry-friction. It is known that nonsmooth or discontinuous dynamical systems can exhibit the bifurcations also exhibited by smooth systems. In addition to these, there are also some novel transitions so-called discontinuity-induced bifurcations (DIBs) which are unique to these systems. We have investigated the complex behaviour occurring in an impacting mechanical system. DIBs such as corner impact bifurcations and transitions from complete to uncomplete chattering motions have been analysed in detail. Another type of DIBs recently classified are the so-called sliding bifurcations. Such bifurcations are a characteristic feature of so-called Filippov systems. We present detailed examples of all the different sliding bifurcation scenarios in a dry friction oscillator using a measured friction characteristic firstly introduced by Popp. Furthermore, a codimension-two degenerate switching-sliding bifurcation is displayed. In this case of degenerate switching-sliding bifurcation two curves of codimension-one sliding bifurcations, crossing-sliding and adding-sliding, branch out from the codimension-two point. Also, a cusp smooth codimension-two bifurcation is shown and coexistence of periodic orbits in the region between both fold codimension-one curves is studied.We have also investigated the dynamic behaviour of the two-block Burridge model for earthquake simulations. Previous numerical studies investigated by Ruina verified that, with a friction force of Coulomb type, the system presents only periodic behaviour. We show that chaotic regions can be observed in a symmetric configuration even if a Coulomb friction is considered with the relaxation of one of the assumptions assumed in the seismological literature. Furthermore, we have studied the behaviour of the system with asymmetric configuration. Different periodic solutions and regions of chaos can be observed varying the asymmetry of the system. With respect to the bifurcation point of view, we have analysed several smooth bifurcations (smooth and DIBs) observed in this system.Chapter 6 of this thesis presents the SICONOS software platform dedicated to simulation of NSDS. We give an overview of the SICONOS software and the way NSDS are modeled and simulated within the platform. Routines for analysis (stability, bifurcations, invariant manifolds,.) of NSDS implemented in the platform are explained in detail. To conclude this part, several representative samples are shown in order to illustrate the SICONOS platform abilities.Conclusion and some open problems are presented in the last chapter.

Desirability Functions continue to attract attention of scientists and researchers working in the area of multi-response optimization. There are many versions of such functions, differing mainly in formulations of individual and overall desirability functions. Derringer and Suich&rsquo / s desirability functions being used throughout this thesis are still the most preferred ones in practice and many other versions are derived from these. On the other hand, they have a drawback of containing nondifferentiable points and, hence, being nonsmooth. Current approaches to their optimization, which are based on derivative-free search techniques and modification of the functions by higher-degree polynomials, need to be diversified considering opportunities offered by modern nonlinear (global) optimization techniques and related softwares. A first motivation of this work is to develop a new efficient solution strategy for the maximization of overall desirability functions which comes out to be a nonsmooth composite constrained optimization problem by nonsmooth optimization methods. We observe that individual desirability functions used in practical computations are of mintype, a subclass of continuous selection functions. To reveal the mechanism that gives rise to a variation in the piecewise structure of desirability functions used in practice, we concentrate on a component-wise and generically piecewise min-type functions and, later on, max-type functions. It is our second motivation to analyze the structural and topological properties of desirability functions via piecewise max-type functions. In this thesis, we introduce adjusted desirability functions based on a reformulation of the individual desirability functions by a binary integer variable in order to deal with their piecewise definition. We define a constraint on the binary variable to obtain a continuous optimization problem of a nonlinear objective function including nondifferentiable points with the constraints of bounds for factors and responses. After describing the adjusted desirability functions on two well-known problems from the literature, we implement modified subgradient algorithm (MSG) in GAMS incorporating to CONOPT solver of GAMS software for solving the corresponding optimization problems. Moreover, BARON solver of GAMS is used to solve these optimization problems including adjusted desirability functions. Numerical applications with BARON show that this is a more efficient alternative solution strategy than the current desirability maximization approaches. We apply negative logarithm to the desirability functions and consider the properties of the resulting functions when they include more than one nondifferentiable point. With this approach we reveal the structure of the functions and employ the piecewise max-type functions as generalized desirability functions (GDFs). We introduce a suitable finite partitioning procedure of the individual functions over their compact and connected interval that yield our so-called GDFs. Hence, we construct GDFs with piecewise max-type functions which have efficient structural and topological properties. We present the structural stability, optimality and constraint qualification properties of GDFs using that of max-type functions. As a by-product of our GDF study, we develop a new method called two-stage (bilevel) approach for multi-objective optimization problems, based on a separation of the parameters: in y-space (optimization) and in x-space (representation). This approach is about calculating the factor variables corresponding to the ideal solutions of each individual functions in y, and then finding a set of compromised solutions in x by considering the convex hull of the ideal factors. This is an early attempt of a new multi-objective optimization method. Our first results show that global optimum of the overall problem may not be an element of the set of compromised solution. The overall problem in both x and y is extended to a new refined (disjunctive) generalized semi-infinite problem, herewith analyzing the stability and robustness properties of the objective function. In this course, we introduce the so-called robust optimization of desirability functions for the cases when response models contain uncertainty. Throughout this thesis, we give several modifications and extensions of the optimization problem of overall desirability functions.

Many mathematical models generally consist of either a continuous system like that of a system of differential equations, or a discrete system such as a discrete game theoretic model; however, there exist phenomena in which neither modeling approach alone is sufficient for capturing the behaviour of the intended real world system. This leads to the need to explore the use of combinations of such discrete and continuous processes, namely the use of mathematical modeling with what are known as hybrid dynamical systems. In what follows, we provide a blueprint for one approach to study several classes of equilibrium problems in non-equilibrium states through the direct use of hybrid dynamical systems. The motivation of our work stems from the fact that the real world is rarely, if ever, in a state of perfect equilibrium and that the behaviour of equilibrium problems in non-equilibrium states is just as complex and interesting (if not more so) than standard equilibrium solutions. Our approach consists of an association of classes of traffic equilibrium problems, noncooperative games, minimization problems, and complementarity problems to a class of hybrid dynamical system called projected dynamical systems. The purposed connection between equilibrium problems and projected dynamical system is made possible through mutual connections to the robust framework of variational inequalities. The results of our work include theoretical contributions such as showing how evolution solutions (non-equilibrium solutions) can be analyzed from a theoretical point of view and how they relate to equilibrium solutions; computational methods for tracking and visualizing evolution solutions and the development of numerical algorithms for simulation; and applications such as the effect of population vaccination decisions in the spread of infectious disease, dynamic traffic networks, dynamic vaccination games, and nonsmooth electrical circuits.

The main purpose of this dissertation is to investigate necessary optimality conditions for a class of very general nonsmooth optimization problems called the mathematical program with geometric constraints (MPGC). The geometric constraint means that the image of certain mapping is included in a nonempty and closed set. We first study the conventional nonlinear program with equality, inequality and abstract set constraints as a special case of MPGC. We derive the enhanced Fritz John condition and from which, we obtain the enhanced Karush-Kuhn-Tucker (KKT) condition and introduce the associated pseudonormality and quasinormality condition. We prove that either pseudonormality or quasinormality with regularity implies the existence of a local error bound. We also give a tighter upper estimate for the Fr\'chet subdifferential and the limiting subdifferential of the value function in terms of quasinormal multipliers which is usually a smaller set than the set of classical normal multipliers. We then consider a more general MPGC where the image of the mapping from a Banach space is included in a nonempty and closed subset of a finite dimensional space. We obtain the enhanced Fritz John necessary optimality conditions in terms of the approximate subdifferential. One of the technical difficulties in obtaining such a result in an infinite dimensional space is that no compactness result can be used to show the existence of local minimizers of a perturbed problem. We employ the celebrated Ekeland's variational principle to obtain the results instead. We then apply our results to the study of exact penalty and sensitivity analysis. We also study a special class of MPCG named mathematical programs with equilibrium constraints (MPECs). We argue that the MPEC-linear independence constraint qualification is not a constraint qualification for the strong (S-) stationary condition when the objective function is nonsmooth. We derive the enhanced Fritz John Mordukhovich (M-) stationary condition for MPECs. From this enhanced Fritz John M-stationary condition we introduce the associated MPEC generalized pseudonormality and quasinormality condition and build the relations between them and some other widely used MPEC constraint qualifications. We give upper estimates for the subdifferential of the value function in terms of the enhanced M- and C-multipliers respectively. Besides, we focus on some new constraint qualifications introduced for nonlinear extremum problems in the recent literature. We show that, if the constraint functions are continuously differentiable, the relaxed Mangasarian-Fromovitz constraint qualification (or, equivalently, the constant rank of the subspace component condition) implies the existence of local error bounds. We further extend the new result to the MPECs. / Graduate / 0405

Thesis (Ph. D.)--West Virginia University, 1998. / Title from document title page. Document formatted into pages; contains xi, 116 p. : ill. (some col.) Includes abstract. Includes bibliographical references (p. 94-98).