This research applies techniques from the field of optimization to spatial positioning and energy management in Unmanned Aerial Vehicles (UAVs). Two specific areas are treated: optimization of UAV view plans for 3D modeling of infrastructure, and trajectory optimization of solar powered high-altitude long-endurance (HALE) UAVs. Structure-from-Motion (SfM) is a computer vision technique for creating 3D models from 2D images. View planning is the process of planning image sets that will effectively model a given scene. First, a genetic algorithm based view planning approach is demonstrated. A novel terrain simulation environment is developed, and the algorithm is tested at multiple sites of interest. The genetic algorithm is compared quantitatively to traditional flights, and is found to yield terrain models with up to 43% greater accuracy than a standard grid flight pattern. Next a greedy heuristic planner is developed, and used to combine anomaly detection with automatic on-board 3D view planning for long linear infrastructure objects such as canals and pipelines. The proposed method is shown in simulation to decrease total flight time by up to 55%, while reducing the amount of image data to be processed by 89% and maintaining 3D model accuracy at areas of interest. The planning algorithm is then extended to select images of ground targets from an existing data set. The algorithm is tested on five different targets, and is shown to reduce processing time for target models by up to a factor of 50 with little decrease in accuracy. The second portion of the research demonstrates the use of nonlinear dynamic optimization to calculate energy optimal trajectories for a high-altitude, solar-powered Unmanned Aerial Vehicle (UAV). The objective is to maximize the total energy in the system while staying within a 3 km mission radius and meeting other system constraints. Solar energy capture is modeled using the vehicle orientation and solar position, and energy is stored both in batteries and in potential energy through elevation gain. Energy capture is maximized by optimally adjusting the angle of the aircraft surface relative to the sun. The UAV flight and energy system dynamics are optimized over a 24-hour period at an eight-second time resolution using Nonlinear Model Predictive Control (NMPC). Results of the simulated flights are presented for all four seasons, showing 8.2% increase in end-of-day battery energy for the most limiting flight condition of the wintersolstice.

One of the challenges of data mining is finding hyper-parameters for a learning algorithm that will produce the best model for a given dataset. Hyper-parameter optimization automates this process, but it can still take significant time. It has been found that hyperparameter optimization does not always result in induced models with significant improvement over default hyper-parameters, yet no systematic analysis of the role of hyper-parameter optimization in machine learning has been conducted. We propose the use of meta-learning to inform the decision to optimize hyper-parameters based on whether default hyper-parameter performance can be surpassed in a given amount of time. We will build a base of metaknowledge, through a series of experiments, to build predictive models that will assist in the decision process.

Several industries use what is called rotating workforce scheduling. This often means that employees are needed around the clock seven days a week, and that they have a schedule which repeats itself after some weeks. This thesis gives an introduction to this kind of scheduling and presents a review of previous work done in the field. Two different optimization models for rotating workforce scheduling are formulated and compared, and some examples are created to demonstrate how the addition of soft constraints to the models affects the scheduling outcome. Two large realistic cases, with constraints commonly used in many industries, are then presented. The schedules are in these cases analyzed in depth and evaluated. One of the models excelled as it provides good results within a short time limit and it appears to be a worthy candidate for rotating workforce scheduling.

In recent years, unprecedented growths in scale and dimensionality of data raise big computational challenges for traditional optimization algorithms; thus it becomes very important to develop efficient and effective optimization algorithms for solving numerous machine learning problems. Many traditional algorithms (e.g., gradient descent method) are black-box algorithms, which are simple to implement but ignore the underlying geometrical property of the objective function. Recent trend in accelerating these traditional black-box algorithms is to leverage geometrical properties of the objective function such as strong convexity. However, most existing methods rely too much on the knowledge of strong convexity, which makes them not applicable to problems without strong convexity or without knowledge of strong convexity. To bridge the gap between traditional black-box algorithms without knowing the problem's geometrical property and accelerated algorithms under strong convexity, how can we develop adaptive algorithms that can be adaptive to the objective function's underlying geometrical property? To answer this question, in this dissertation we focus on convex optimization problems and propose to explore an error bound condition that characterizes the functional growth condition of the objective function around a global minimum. Under this error bound condition, we develop algorithms that (1) can adapt to the problem's geometrical property to enjoy faster convergence in stochastic optimization; (2) can leverage the problem's structural regularizer to further improve the convergence speed; (3) can address both deterministic and stochastic optimization problems with explicit max-structural loss; (4) can leverage the objective function's smoothness property to improve the convergence rate for stochastic optimization. We first considered stochastic optimization problems with general stochastic loss. We proposed two accelerated stochastic subgradient (ASSG) methods with theoretical guarantees by iteratively solving the original problem approximately in a local region around a historical solution with the size of the local region gradually decreasing as the solution approaches the optimal set. Second, we developed a new theory of alternating direction method of multipliers (ADMM) with a new adaptive scheme of the penalty parameter for both deterministic and stochastic optimization problems with structured regularizers. With LEB condition, the proposed deterministic and stochastic ADMM enjoy improved iteration complexities of $\widetilde O(1/\epsilon^{1-\theta})$ and $\widetilde O(1/\epsilon^{2(1-\theta)})$ respectively. Then, we considered a family of optimization problems with an explicit max-structural loss. We developed a novel homotopy smoothing (HOPS) algorithm that employs Nesterov's smoothing technique and accelerated gradient method and runs in stages. Under a generic condition so-called local error bound (LEB) condition, it can improve the iteration complexity of $O(1/\epsilon)$ to $\widetilde O(1/\epsilon^{1-\theta})$ omitting a logarithmic factor with $\theta\in(0,1]$. Next, we proposed new restarted stochastic primal-dual (RSPD) algorithms for solving the problem with stochastic explicit max-structural loss. We successfully got a better iteration complexity than $O(1/\epsilon^2)$ without bilinear structure assumption, which is a big challenge of obtaining faster convergence for the considered problem. Finally, we consider finite-sum optimization problems with smooth loss and simple regularizer. We proposed novel techniques to automatically search for the unknown parameter on the fly of optimization while maintaining almost the same convergence rate as an oracle setting assuming the involved parameter is given. Under the Holderian error bound (HEB) condition with $\theta\in(0,1/2)$, the proposed algorithm also enjoys intermediate faster convergence rates than its standard counterparts with only the smoothness assumption.

L’entreprise Subsea Tech développe un ROV (véhicule sous-marin téléopéré - Remotely OperatedVehicle) léger (40 kg), nommé Tortuga 500, dont la particularité est la capacité de reconfiguration de ses actionneurs. Ses quatre propulseurs horizontaux, dits azimutaux, peuvent en effet pivoter autour de leur axe vertical en cours de mission pour permettre une optimisation de la poussée ou de la manœuvrabilité du véhicule. Le but de cette thèse est de maximiser, à chaque instant de la mission effectuée, la propulsion du ROV dans une direction souhaitée qui peut évoluer en cours de mission. Pour ce faire, une méthode d’optimisation locale et déterministe est utilisée pour calculer en ligne l’orientation optimale de chaque propulseur. Elle est initialisée par une méthode de recherche globale effectuée en amont. Les perturbations dues aux interactions entre les flux des propulseurs sont prises en compte dans la méthode d’optimisation. La poussée du véhicule est alors maximisée à chaque instant, et pour chaque direction souhaitée, tout en conservant un minimum de commandabilité latérale pour faire face aux perturbations extérieures (traînée de l’ombilical, courant). Plusieurs simulations, portant sur la poussée maximale atteignable par le véhicule, sa puissance consommée, et sa vitesse maximale lors d’un transit, permettent de comparer les performances de la méthode d’optimisation proposée avec celles de la traditionnelle configuration fixe dite "vectorielle" observée sur la majorité des autres ROVs du commerce. / The Subsea Tech company is developing a lightweight (40 kg) Remotely Operated Vehicle (ROV) called Tortuga 500, which can online reconfigure its four horizontal thrusters. These thrusters, also called "azimuth thrusters", can steer around their vertical axis during a mission in order to optimize the thrust or the manoeuvrability of the vehicle. This thesis focuses on the online thrust maximization of the ROV, along a variable desired direction. To do so, a local and deterministic optimization method is used and allows to get the optimal orientation of each thruster in real time. It is initialized by values previously computed by a sparse and offline research method. The interactions between thrusters due to cross flows are modelled and integrated into the optimization method. Thus, the vehicle thrust is maximized at every iteration step, and for every desired direction. Furthermore, a fixed minimum ratio of manoeuvrability is preserved during this process, to cope with external disturbances such as the tether’s drag and currents. Several simulations have been made and allow to compare the maximal reachable thrust, the consumed power, and the maximal achievable velocity, when it is configured with the proposed optimization method or with the "vectorized configuration" found on most other commercial ROVs.

As adaptive clinical trials (ACTs) receive growing attention and exhibit promising performance in practical trials during last decade, they also present challenges to drug supply chain management. As indicated by Burnham et al. (2015), the challenges include the uncertainty of maximum drug supply needed, the shifting of supply requirement, and rapid availability of new supply at decision points. To facilitate drug supply decision making and the development of mathematical analysis tools, we propose two trial supply chain optimization problems that represent different mindsets in response to trial adaptations. In the first problem, we treat the impacts of ACTs as exogenous uncertainties and study important aspects of trial supply, including drug wastage, resupply policy, trial length, and costs minimization, via a two-stage stochastic program. In the second problem, we incorporate the adaptation rules of ACTs with supply chain management and numerically study the impact of joint optimization on the trial and drug supply planning through a mixed-integer nonlinear program (MINLP). For solution approaches to the problems, we use progressive hedging algorithm (PHA) and particle swarm optimization (PSO) respectively, and take advantages of the problem structures to enhance the solution efficiency. With case studies, we see that the proposed models capture the features of ACT drug supply and the mechanisms of trial conduction well. The solutions not only reflect the impact of trial adaptations but also provide managerial suggestions, e.g. the prediction of needed production amount, storage capacity at clinical sites, and resupply schemes. The joint optimization also suggests a new angle and research extension in the field of ACT design and supply.

Minimizing cost and optimization of nonlinear problems are important for industries in order to be competitive. The need of optimization strategies provides significant benefits for companies when providing quotes for products. Accurate and easily attained estimates allow for less waste, tighter tolerances, and better productivity. The Nelder-Mead Simplex method with exterior penalty functions was employed to solve optimum machining parameters. Two case studies were presented for optimizing cost and time for a multiple tools scenario. In this study, the optimum machining parameters for milling operations were investigated. Cutting speed and feed rate are considered as the most impactful design variables across each operation. Single tool process and scalable multiple tool milling operations were studied. Various optimization methods were discussed. The Nelder-Mead Simplex method showed to be simple and fast.

The objective of this research is to develop new stochastic methods based on most probable points (MPPs) for general reliability analysis and reliability-based design optimization of complex engineering systems. The current efforts involves: (1) univariate method with simulation for reliability analysis; (2) univariate method with numerical integration for reliability analysis; (3) multi-point univariate for reliability analysis involving multiple MPPs; and (4) univariate method for design sensitivity analysis and reliability-based design optimization. Two MPP-based univariate decomposition methods were developed for component reliability analysis with highly nonlinear performance functions. Both methods involve novel function decomposition at MPP that facilitates higher-order univariate approximations of a performance function in the rotated Gaussian space. The first method entails Lagrange interpolation of univariate component functions that leads to an explicit performance function and subsequent Monte Carlo simulation. Based on linear or quadratic approximations of the univariate component function in the direction of the MPP, the second method formulates the performance function in a form amenable to an efficient reliability analysis by multiple one-dimensional integrations. Although both methods have comparable computational efficiency, the second method can be extended to derive analytical sensitivity of failure probability for design optimization. For reliability problems entailing multiple MPPs, a multi-point univariate decomposition method was also developed. In addition to the effort of identifying the MPP, the univariate methods require a small number of exact or numerical function evaluations at selected input. Numerical results indicate that the MPP-based univariate methods provide accurate and/or computationally efficient estimates of failure probability than existing methods. Finally, a new univariate decomposition method was developed for design sensitivity analysis and reliability-based design optimization subject to uncertain performance functions in constraints. The method involves a novel univariate approximation of a general multivariate function in the rotated Gaussian space; analytical sensitivity of failure probability with respect to design variables; and standard gradient-based optimization algorithms. In both reliability and sensitivity analyses, the proposed effort has been reduced to performing multiple one-dimensional integrations. Numerical results indicate that the proposed method provides accurate and computationally efficient estimates of the sensitivity of failure probability and leads to accurate design optimization of uncertain mechanical systems.

Mathematical modeling of human running is a challenging problem from analytical and computational points of view. Purpose of the present research is to develop and study formulations and computational procedures for simulation of natural human running. The human skeletal structure is modeled as a mechanical system that includes link lengths, mass moments of inertia, joint torques, and external forces. The model has 55 degrees of freedom, 49 for revolute joints and 6 for global translation and rotation. Denavit-Hartenberg method is used for kinematics analysis and recursive Lagrangian formulation is used for the equations of motion. The dynamic stability is achieved by satisfying the zero moment point (ZMP) condition during the ground contact phase. B-spline interpolation is used for discretization of the joint angle profiles. The joint torque square, impulse at the foot strike, and yawing moment are included in the performance measure. A minimal set of constraints is imposed in the formulation of the problem to simulate natural running motion. Normal running with arm fixed, slow jog along curves, and running with upper body motion are formulated. Simulation results are obtained for various cases and discussed. The cases are running with different foot locations, running with backpack, and running with different running speeds. Also, extreme cases are performed. Each case gives reasonable cause and effect results. Furthermore, sparsity of the formulation is studied. The results obtained with the formulation are validated with the experimental data. The proposed formulation is robust and can predict natural motion of human running.

The design of an aircraft wing often involves the use of mathematical methods for simultaneous aerodynamic and structural design. The goal of many of these methods is to minimize the drag on the wing. A variety of computer models exist for this purpose, but some require the use of expensive time and computational resources to give meaningful results. As an alternative, some mathematical methods have been developed that give reason ably accurate results without the need for a computer. However, most of these methods can only be used for wings with specific shapes and payload distributions. In this thesis, a hybrid mathematical/computational approach to wing design is developed that can be used for wings of any shape with any payload distribution. Specific mathematical expressions are found to predict the weight and drag for tapered wings and elliptic-shaped wings. The new approach and mathematical expressions are used to find the best distribution of lift on a variety of aircraft wing configurations to minimize drag during flight.