The increasing complexity of time-critical Cyber-Physical Systems (CPS) presents significant challenges in designing optimization algorithms that balance generality, scalability, and performance. Traditional approaches often compromise one or more of these properties: general metaheuristic algorithms lack scalability and performance guarantees, while problem-specific methods sacrifice generality for improved efficiency or optimality. However, due to the NP-hard nature of many real-time scheduling and optimization problems, it is highly unlikely to design optimization algorithms that are simultaneously general, scalable, and optimal. Therefore, this dissertation addresses these challenges by developing novel optimization frameworks tailored for time-critical CPS and try to improve the trade-off among the three factors. The first contribution focuses on general and scalable optimization techniques, introducing frameworks such as NORTH, which operates with black-box schedulability constraints while achieving very good scalability and reasonably good performance. Additionally, another optimization framework targets at general robotic working environments by performing dynamic resource allocation. It demonstrates 20–50\% improvements in safety-performance metrics with low computational overhead. The second contribution advances domain-specific optimization techniques by relaxing the general requirements. For instance, flexible Logical Execution Time (LET) optimization achieves significant improvements in end-to-end latency, time disparity, and jitter by leveraging symbolic operations and efficient exploration of solution spaces. Similarly, a novel scheduling approach for DAG-based task models minimizes worst-case end-to-end latency and time disparity through 1-opt solutions with polynomial runtime complexity, achieving up to 40\% performance gains over existing methods. These contributions push the boundaries of generality, scalability, and optimality in real-time systems optimization, providing practical solutions to complex scheduling and resource allocation problems. The proposed frameworks are validated through extensive experimental studies, demonstrating their applicability and impact across a range of real-world scenarios. / Doctor of Philosophy / Cyber-Physical Systems (CPS) are transformative technologies that seamlessly integrate computation, networking, and physical processes, forming the backbone of innovations like autonomous vehicles, smart grids, and advanced medical devices. Among them, time-critical CPS, or real-time systems, are uniquely challenging as they require not only logical correctness but also strict adherence to timing constraints. Failing to meet these constraints in applications like flight control or automotive systems can lead to catastrophic outcomes. The design of time-critical CPS involves solving complex optimization problems to balance competing goals such as latency, energy, reliability, and cost. However, the growing system complexity, driven by trends like parallel computing and heterogeneous platforms, makes achieving scalability, optimality, and general applicability in optimization algorithms increasingly difficult. Traditional optimization methods either lack scalability, fail to generalize across diverse problems, or provide optimal solutions. This dissertation addresses these challenges by proposing novel optimization frameworks tailored to time-critical CPS. It introduces general and scalable techniques capable of achieving good performance across diverse scenarios, including methods for optimizing systems with complex schedulability constraints and dynamic task environments. It also presents domain-specific solutions that prioritize optimality and scalability for specific problems, such as flexible Logical Execution Time (LET) scheduling and time-triggered scheduling for directed acyclic graph (DAG)-based systems. Extensive experiment evaluation show that these novel techniques significantly improved various metrics, such as energy, schedulability, end-to-end latency, time disparity, and safety-performance trade-offs.

There are many challenges facing the aerospace industry that can be addressed with new concepts, technologies, and materials. However, current design methods make it difficult to include these new ideas early in the design of aircraft. This is especially true in the structures discipline, which often uses weight-based methods based upon statistical regressions of historical data. A way to address this is to use physics-based structural analysis and design to create more detailed structural data. Thus, the overall research objective of this dissertation is to develop a physics-based structural analysis method to incorporate new concepts, technologies, and materials into the conceptual design phase. The design space of physics-based structural design problem is characterized as highly multimodal with numerous discontinuities; thus, a large number of alternatives must be explored. Current physics-based structural design methods tend to use high fidelity modeling and analysis tools that are computationally expensive. This dissertation proposes a modeling & simulation environment based on classical structural analysis methods. Using classical structural analysis will enable increased exploration of the design space by reducing the overall run time necessary to evaluate one alternative. The use of physics-based structural optimization using classical structural analysis is tested through experimentation. First the underlying hypotheses are tested in a canonical example by comparing different optimization algorithms ability to locate a global optimum identified through design space exploration. Then the proposed method is compared to a method based on higher fidelity finite element analysis as well as a method based on weight-based empirical data to validate the overall research objective.

In this dissertation, we focus on solving Nonlinear Least Squares problems using a supervised approach. In particular, we developed a Supervised Descent Method (SDM), performed thorough theoretical analysis, and demonstrated its effectiveness on optimizing analytic functions, and four other real-world applications: Inverse Kinematics, Rigid Tracking, Face Alignment (frontal and multi-view), and 3D Object Pose Estimation. In Rigid Tracking, SDM was able to take advantage of more robust features, such as, HoG and SIFT. Those non-differentiable image features were out of consideration of previous work because they relied on gradient-based methods for optimization. In Inverse Kinematics where we minimize a non-convex function, SDM achieved significantly better convergence than gradient-based approaches. In Face Alignment, SDM achieved state-of-the-arts results. Moreover, it was extremely computationally efficient, which makes it applicable for many mobile applications. In addition, we provided a unified view of several popular methods including SDM on sequential prediction, and reformulated them as a sequence of function compositions. Finally, we suggested some future research directions on SDM and sequential prediction.

Approved for public release, distribution is unlimited / problem, with engine thrust as the only possible perturbation. The optimal control problems are solved using the general purpose dynamic optimization software, DIDO. The dynamical model together with the fuel optimal control problem is validated by simulating several well known orbit transfers. By replicating the single satellite model, this thesis shows that a multi-satellite model which optimizes all vehicles concurrently can be easily built. The specific scenario under study involves the injection of multiple satellites from a common launch vehicle; however, the methods and model are applicable to spacecraft formation problems as well. / Major, United States Air Force

We use modern approach of stochastic dominance in portfolio optimization, where we want the portfolio to dominate a benchmark. Since the distribution of returns is often just estimated from data, we look for the worst distribution that differs from empirical distribution at maximum by a predefined value. First, we define in what sense the distribution is the worst for the first and second order stochastic dominance. For the second order stochastic dominance, we use two different formulations for the worst case. We derive the robust stochastic dominance test for all the mentioned approaches and find the worst case distribution as the optimal solution of a non-linear maximization problem. Then we derive programs to maximize an objective function over the weights of the portfolio with robust stochastic dominance in constraints. We consider robustness either in returns or in probabilities for both the first and the second order stochastic dominance. To the best of our knowledge nobody was able to derive such program before. We apply all the derived optimization programs to real life data, specifically to returns of assets captured by Dow Jones Industrial Average, and we analyze the problems in detail using optimal solutions of the optimization programs with multiple setups. The portfolios calculated using...

Bayesian Optimization (BO) methods are often used to optimize an unknown function f(���) that is costly to evaluate. They typically work in an iterative manner. In each iteration, given a set of observation points, BO algorithms select k ��� 1 points to be evaluated. The results of those points are then added to the set of observations and the procedure is repeated until a stopping criterion is met. The goal is to optimize the function f(���) with a small number of experiment evaluations. While this problem has been extensively studied, most existing approaches ignored some real world constraints frequently encountered in practical applications. In this thesis, we extend the BO framework in a number of important directions to incorporate some of these constraints. First, we introduce a constrained BO framework where instead of selecting a precise point at each iteration, we request a constrained experiment that is characterized by a hyper-rectangle in the input space. We introduce efficient sequential and non-sequential algorithms to select a set of constrained experiments that best optimize f(���) within a given budget. Second, we introduce one of the first attempts in batch BO where instead of selecting one experiment at each iteration, a set of k > 1 experiments is selected. This can significantly speedup the overall running time of BO. Third, we introduce scheduling algorithms for the BO framework when: 1) it is possible to run concurrent experiments; 2) the durations of experiments are stochastic, but with a known distribution; and 3) there is a limited number of experiments to run in a fixed amount of time. We propose both online and offline scheduling algorithms that effectively handle these constraints. Finally, we introduce a hybrid BO approach which switches between the sequential and batch mode. The proposed hybrid approach provides us with a substantial speedup against sequential policies without significant performance loss. / Graduation date: 2013

Optimal parametric design of a system must be able to respond quickly to short term needs as well as long term conditions. To this end, we present an Assess-Predict-Optimize (APO) strategy which allows for easy modification of a system’s characteristics and constraints, enabling quick design adaptation. There are three components to the APO strategy: Assess - extract necessary information from given data; Predict - predict future behavior of system; and Optimize – obtain optimal system configuration based on information from the other components. The APO strategy utilizes three key mathematical ingredients to yield real-time results which would certainly conform to given constraints: dimension reduction of the model, a posteriori error estimation, and optimization methods. The resulting formulation resembles a bilevel optimization problem with an inherent nonconvexity in the inner level. Using a simple infiltration-evaporation model to simulate an irrigation system, we demonstrate the APO strategy’s ability to yield real-time optimal results. The linearized model, described by a coercive elliptic partial differential equation, is discretized by the reduced-basis output bounds method. A primal-dual interior point method is then chosen to solve the resulting APO problem. / Singapore-MIT Alliance (SMA)

In the literature, a large number of metaheuristic search techniques have been proposed up to present time and some of those have been used in structural optimization. Scatter search is one of those techniques which has proved to be effective when solving combinatorial and nonlinear optimization problems such as scheduling, routing, financial product design and other problem areas. Scatter search is an evolutionary method that uses strategies based on a composite decision rules and search diversification and intensification for generating new trial points. Broodly speaking, this thesis is concerned with the use and application of scatter search technique in structural optimization. A newly developed optimization algorithm called modified scatter search is modified which is computerized in a software called SOP2012. The software SOP2012 is integrated with well-known structural analysis software SAP2000 using application programming interface for size optimum design of steel structures. Numerical studies are carried out using a test suite consisting of five real size design examples taken from the literature. In these examples, various steel truss and frame structures are designed for minimum weight according to design limitations imposed by AISC-ASD (Allowable Stress Design Code of American Institute of Steel Construction). The results reveal that the modified scatter search technique is very effective optimization technique for truss structures, yet its performance can be assessed ordinary for frame structures.

Modular and Reconfigurable Robots (MRRs) are those designed to address the increasing demand for flexible and versatile manipulators in manufacturing facilities. The term, modularity, indicates that they are constructed by using a limited number of interchangeable standardized modules which can be assembled in different kinematic configurations. Thereby, a wide variety of specialized robots can be built from a set of standard components. The term, reconfigurability, implies that the robots can be disassembled and rearranged to accommodate different products or tasks rather than being replaced. A set of MRR modules may consist of joints, links, and end-effectors. Different kinematic configurations are achieved by using different joint, link, and end-effector modules and by changing their relative orientation. The number of distinct kinematic configurations, attainable by a set of modules, varies with respect to the size of the module set from several tens to several thousands. Although determining the most suitable configuration for a specific task from a predefined set of modules is a highly nonlinear optimization problem in a hybrid continuous and discrete search space, a solution to this problem is crucial to effectively utilize MRRs in manufacturing facilities. The objective of this thesis is to develop novel optimization methods that can effectively search the Kinematic Configuration (KC) space to identify the most suitable manipulator for any given task. In specific terms, the goal is to develop and synthesize fast and efficient algorithms for a Task-Based Configuration Optimization (TBCO) from a given set of constraints and optimization criteria. To achieve such efficiency, a TBCO solver, based on Memetic Algorithms (MA), is proposed. MAs are hybrids of Genetic Algorithms (GAs) and local search algorithms. MAs benefit from the exploration abilities of GAs and the exploitation abilities of local search methods simultaneously. Consequently, MAs can significantly enhance the search efficiency of a wide range of optimization problems, including the TBCO. To achieve more optimal solutions, the proposed TBCO utilizes all the solutions of the Inverse Kinematics(IK) problem. Another objective is to develop a method for incorporating the multiple solutions of the IK problem in a trajectory optimization framework. The output of the proposed trajectory optimization method consists of a sequence of desired tasks and a single IK solution to reach each task point. Moreover, the total cost of the optimized trajectory is utilized in the TBCO as a performance measure, providing a means to identify kinematic configurations with more efficient optimized trajectories. The final objective is to develop novel IK solvers which are both general and complete. Generality means that the solvers are applicable to all the kinematic configurations which can be assembled from the available module inventory. Completeness entails the algorithm can obtain all the possible IK solutions.

Modular and Reconfigurable Robots (MRRs) are those designed to address the increasing demand for flexible and versatile manipulators in manufacturing facilities. The term, modularity, indicates that they are constructed by using a limited number of interchangeable standardized modules which can be assembled in different kinematic configurations. Thereby, a wide variety of specialized robots can be built from a set of standard components. The term, reconfigurability, implies that the robots can be disassembled and rearranged to accommodate different products or tasks rather than being replaced. A set of MRR modules may consist of joints, links, and end-effectors. Different kinematic configurations are achieved by using different joint, link, and end-effector modules and by changing their relative orientation. The number of distinct kinematic configurations, attainable by a set of modules, varies with respect to the size of the module set from several tens to several thousands. Although determining the most suitable configuration for a specific task from a predefined set of modules is a highly nonlinear optimization problem in a hybrid continuous and discrete search space, a solution to this problem is crucial to effectively utilize MRRs in manufacturing facilities. The objective of this thesis is to develop novel optimization methods that can effectively search the Kinematic Configuration (KC) space to identify the most suitable manipulator for any given task. In specific terms, the goal is to develop and synthesize fast and efficient algorithms for a Task-Based Configuration Optimization (TBCO) from a given set of constraints and optimization criteria. To achieve such efficiency, a TBCO solver, based on Memetic Algorithms (MA), is proposed. MAs are hybrids of Genetic Algorithms (GAs) and local search algorithms. MAs benefit from the exploration abilities of GAs and the exploitation abilities of local search methods simultaneously. Consequently, MAs can significantly enhance the search efficiency of a wide range of optimization problems, including the TBCO. To achieve more optimal solutions, the proposed TBCO utilizes all the solutions of the Inverse Kinematics(IK) problem. Another objective is to develop a method for incorporating the multiple solutions of the IK problem in a trajectory optimization framework. The output of the proposed trajectory optimization method consists of a sequence of desired tasks and a single IK solution to reach each task point. Moreover, the total cost of the optimized trajectory is utilized in the TBCO as a performance measure, providing a means to identify kinematic configurations with more efficient optimized trajectories. The final objective is to develop novel IK solvers which are both general and complete. Generality means that the solvers are applicable to all the kinematic configurations which can be assembled from the available module inventory. Completeness entails the algorithm can obtain all the possible IK solutions.