Back to search

University of British Columbia

How to descend a rocky slope : numerical techniques for the solution of noisy optimization problems

Abstract

dc:description

The goal of this thesis is to develop new numerical techniques for the solution of noisy optimization problems. Noisy optimization problems occur in diverse fields, such as engineering design, machine learning, operations research, and geophysics. Thus, effective numerical techniques for solving noisy optimization problems are of practical interest in a variety of industrial applications. This thesis presents three numerical techniques for solving noisy optimization problems: Secant penalized Broyden-Fletcher-Goldfarb-Shanno (SP-BFGS), Neural Network Accelerated Implicit Filtering (NNAIF), and Linear Model Based Gradient Estimation (LMBGE). SP-BFGS is a new variant of the Broyden-Fletcher-Goldfarb-Shanno (BFGS) quasi-Newton method designed to perform well when gradient measurements are corrupted by noise. SP-BFGS provides a means of incrementally updating the new inverse Hessian approximation with a controlled amount of bias towards the previous inverse Hessian approximation, which allows one to replace the overwriting nature of the original BFGS update with an averaging nature that resists the destructive effects of noise and can cope with negative curvature measurements. I study the theoretical and empirical behaviour of SP-BFGS in this thesis. NNAIF intelligently combines the established literature on implicit filtering (IF) optimization methods with a neural network surrogate model of the objective function, resulting in accelerated derivative free methods for unconstrained optimization problems. I show that NNAIF directly inherits the convergence properties of IF optimization methods, and study the performance of NNAIF via numerical experiments with noisy problems from the Constrained and Unconstrained Testing Environment with safe threads (CUTEst) optimization benchmark set. LMBGE leverages techniques from linear inverse problems to estimate the gradient of a noisy objective function, and can be viewed as a noise robust extension of standard finite difference based approaches for gradient estimation. I demonstrate the use of LMBGE in the solution of the coverage directed generation (CDG) problem, an industrial problem encountered in the verification subfield of electrical and computer engineering.

Degree

thesis:*
Name thesis:degree_name
Doctor of Philosophy - PhD
Level thesis:degree_level
doctoral
Discipline thesis:degree_discipline
Geophysics
Grantor dc:publisher
University of British Columbia
Year dc:date
2023

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Irwin, Brian
Contributors dc:contributor
  • Haber, Eldad

Rights

dc:rights
Statement dc:rights
  • Attribution-NonCommercial-NoDerivatives 4.0 International
Language dc:language
eng

Identifiers

dc:identifier.*
Handle dc:identifier
http://hdl.handle.net/2429/86077

Chain of custody

source
Harvested from
University of British Columbia
Base URL
circle.library.ubc.ca/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Irwin, Brian. How to descend a rocky slope : numerical techniques for the solution of noisy optimization problems. doctoral thesis, University of British Columbia, 2023. http://hdl.handle.net/2429/86077