<p dir="ltr">Neural networks (NNs) effectively address the challenges of engineering design optimization by using data-driven models, thus reducing computational demands. However, their effectiveness depends heavily on hyperparameter optimization (HPO), which is a global optimization problem. While traditional HPO methods, such as manual, grid, and random search, are simple, they often fail to navigate the vast hyperparameter (HP) space efficiently. This work examines the effectiveness of integrating Bayesian optimization (BO) with multi-armed bandit (MAB) optimization for HPO in NNs. The thesis initially addresses HPO in one-shot sampling, where NNs are trained using datasets of varying sample sizes. It compares the performance of NNs optimized through traditional HPO techniques and a combination of BO and MAB optimization on the analytical Branin function and aerodynamic shape optimization (ASO) of an airfoil in transonic flow. Findings from the optimization of the Branin function indicate that the combined BO and MAB optimization approach leads to simpler NNs and reduces the sample size by approximately 10 to 20 compared to traditional HPO methods, all within half the time. This efficiency improvement is even more pronounced in ASO, where the BO and MAB optimization use about 100 fewer samples than the traditional methods to achieve the optimized airfoil design. The thesis then expands on adaptive HPOs within the framework of efficient global optimization (EGO) using a NN-based prediction and uncertainty (EGONN) algorithm. It employs the BO and MAB optimization for tuning HPs during sequential sampling, either every iteration (HPO-1itr) or every five iterations (HPO-5itr). These strategies are evaluated against the EGO as a benchmark method. Through experimentation with the analytical three-dimensional Hartmann function and ASO, assessing both comprehensive and selective tunable HP sets, the thesis contrasts adaptive HPO approaches with a static HPO method (HPO-static), which uses the initial HP settings throughout. Initially, a comprehensive set of the HPs is optimized and evaluated, followed by an examination of selectively chosen HPs. For the optimization of the three-dimensional Hartmann function, the adaptive HPO strategies surpass HPO-static in performance in both cases, achieving optimal convergence and sample efficiency comparable to EGO. In ASO, applying the adaptive HPO strategies reduces the baseline airfoil's drag coefficient to 123 drag counts (d.c.) for HPO-1itr and 120 d.c. for HPO-5itr when tuning the full set of the HPs. For a selected subset of the HPs, 123 d.c. and 121 d.c. are achieved by HPO-1itr and HPO-5itr, respectively, which are comparable to the minimum achieved by EGO. While the HPO-static method reduces the drag coefficient to 127 d.c. by tuning a subset of the HPs, which is a 15 d.c. reduction from its full set case, it falls short of the minimum of adaptive HPO strategies. Focusing on a subset of the HPs reduces time costs and enhances the convergence rate without sacrificing optimization efficiency. The time reduction is more significant with higher HPO frequencies as HPO-1itr cuts time by 66%, HPO-5itr by 38%, and HPO-static by 2%. However, HPO-5itr still requires 31% of the time needed by HPO-1itr for the full HP tuning and 56% for the subset HP tuning.</p>

Many problems in early vision can be formulated in terms of minimizing an energy or cost function. Examples are shape-from-shading, edge detection, motion analysis, structure from motion and surface interpolation (Poggio, Torre and Koch, 1985). It has been shown that all quadratic variational problems, an important subset of early vision tasks, can be "solved" by linear, analog electrical or chemical networks (Poggio and Koch, 1985). IN a variety of situateions the cost function is non-quadratic, however, for instance in the presence of discontinuities. The use of non-quadratic cost functions raises the question of designing efficient algorithms for computing the optimal solution. Recently, Hopfield and Tank (1985) have shown that networks of nonlinear analog "neurons" can be effective in computing the solution of optimization problems. In this paper, we show how these networks can be generalized to solve the non-convex energy functionals of early vision. We illustrate this approach by implementing a specific network solving the problem of reconstructing a smooth surface while preserving its discontinuities from sparsely sampled data (Geman and Geman, 1984; Marroquin 1984; Terzopoulos 1984). These results suggest a novel computational strategy for solving such problems for both biological and artificial vision systems.

This thesis investigates a technique to estimate the risk of the mean-variance (MV) portfolio optimization problem. We call this technique the Scaling technique. It provides a better estimator of the risk of the MV optimal portfolio. We obtain this result for a general estimator of the covariance matrix of the returns which includes the correlated sampling case as well as the independent sampling case and the exponentially weighted moving average case. This gave rise to the paper, [CMcS]. Our result concerning the Scaling technique relies on the moments of the inverse of compound Wishart matrices. This is an open problem in the theory of random matrices. We actually tackle a much more general setup, where we consider any random matrix provided that its distribution has an appropriate invariance property (orthogonal or unitary) under an appropriate action (by conjugation, or by a left-right action). Our approach is based on Weingarten calculus. As an interesting byproduct of our study - and as a preliminary to the solution of our problem of computing the moments of the inverse of a compound Wishart random matrix, we obtain explicit moment formulas for the pseudo-inverse of Ginibre random matrices. These results are also given in the paper, [CMS]. Using the moments of the inverse of compound Wishart matrices, we obtain asymptotically unbiased estimators of the risk and the weights of the MV portfolio. Finally, we have some numerical results which are part of our future work.

The most popular and fundamental portfolio optimization problem is Markowitz&#039 / s one period mean-variance portfolio selection problem. However, it is criticized because of its one period static nature. Further, the estimation of the stock price expected return is a particularly hard problem. For this purpose, there are a lot of studies solving the mean-variance portfolio optimization problem in continuous time. To solve the estimation problem of the stock price expected return, in 1992, Black and Litterman proposed the Bayesian asset allocation method in discrete time. Later on, Lindberg has introduced a new way of parameterizing the price dynamics in the standard Black-Scholes and solved the continuous time mean-variance portfolio optimization problem. In this thesis, firstly we take up the Lindberg&#039 / s approach, we generalize the results to a jump-diffusion market setting and we correct the proof of the main result. Further, we demonstrate the implications of the Lindberg parameterization for the stock price drift vector in different market settings, we analyze the dependence of the optimal portfolio from jump and diffusion risk, and we indicate how to use the method. Secondly, we present the Lagrangian function approach of Korn and Trautmann and we derive some new results for this approach, in particular explicit representations for the optimal portfolio process. In addition, we present the L2-projection approach of Schweizer for the continuous time mean-variance portfolio optimization problem and derive the optimal portfolio and the optimal wealth processes for this approach. While, deriving these results as the underlying model, the market parameterization of Lindberg is chosen. Lastly, we compare these three different optimization frameworks in detail and their attractive and not so attractive features are highlighted by numerical examples.

The basic planar cam mechanisms and link mechanisms are widely used in industrial automatic machines. In determining the design method and design procedure for the cam-link mechanism, the basic kinematic synthesis and motion curve generation method require effective design procedure and optimization method to determine the kinematic structure of the mechanism and its kinematic performance clearly. In order to determine the result of the multi-objective optimization problem for the cam-link mechanism, the genetic algorithm is defined as the problem solver and begins this dissertation. By considering the influences of the parameters in the evolving procedure and by defining the conditions of the parameters and variables properly, the best solutions of the multi-objective optimization problem can then be solved successfully. By comparing the curves for the motion synthesis of the cam-link mechanism, the existing motion functions with their kinematic characteristics used in cam mechanisms are introduced and the rational B-spline motion function is proposed. By using the genetic algorithm to approximate the motion curves that is combined with trigonometric functions, the flexibility of the rational B-spline is demonstrated. Furthermore, to minimize different kinematic characteristics of the single-objective minimization problems, these problems are also searched by using rational B-splines with genetic algorithm for having better results. For synthesizing different structures of cam-link mechanisms, first of all is to derive the kinematics of the two planar link mechanisms and four planar cam mechanisms and then the genetic algorithm is used here to find out the minimal cam dimension with the limits of the motion curves, the pressure angles, and the radius of curvatures. Then, the kinematic synthesis problem of the function generation slider-crank mechanisms as the slider starts at the toggle position is discussed. Through the analysis finds out that when using the traditional motion functions with acceleration continuity to synthesize the slider motion, the angular acceleration of the crank cannot be continuous. To achieve the acceleration continuity of the crank motion, the curve that contains the fourth derivatives of the displacement with respect to time are set to be zeros can fulfill the continuity requirement. Then using the structural synthesis design procedure, by following the input and output relations of the link mechanisms and cam mechanisms with design constraints to select the proper structures of the mechanisms. To apply the cam-link mechanism in real application, a machine containing a slider-crank mechanism as toggle mechanism is introduced. Through the design constraints of space and motion limits to find out the possible mechanism structure and define the dimensions and then analyze the kinematics and kinetostatics of the machine. By using the genetic algorithm to solve the multi-objective optimization problem, the parameters of the rational B-spline are adjusted to have optimal kinematics and minimal kinetostatics to reduce the contact stress and to improve the fatigue life of the cam follower. Due to the existing problem of the slider-crank mechanism that contains discontinuous acceleration at the toggle position, to prove the correctness of the theoretical results, the experimental tests are measured and verified with the theoretical results in high similarity. The results show that when the slider motion curves begin at the toggle position with boundary motion constraints up to fourth or more than fourth derivatives of the displacement with respect to time that are set to be zeros, the angular accelerations of the cranks are continuous. In summary, this dissertation provides suggestions of the kinematic characteristics for the designer to design cam-link mechanisms that contain a slider-crank mechanism as the toggle mechanism and design methods for the synthesis, analysis and experimental test of the simple function generation cam-link mechanism.

This thesis investigates a technique to estimate the risk of the mean-variance (MV) portfolio optimization problem. We call this technique the Scaling technique. It provides a better estimator of the risk of the MV optimal portfolio. We obtain this result for a general estimator of the covariance matrix of the returns which includes the correlated sampling case as well as the independent sampling case and the exponentially weighted moving average case. This gave rise to the paper, [CMcS]. Our result concerning the Scaling technique relies on the moments of the inverse of compound Wishart matrices. This is an open problem in the theory of random matrices. We actually tackle a much more general setup, where we consider any random matrix provided that its distribution has an appropriate invariance property (orthogonal or unitary) under an appropriate action (by conjugation, or by a left-right action). Our approach is based on Weingarten calculus. As an interesting byproduct of our study - and as a preliminary to the solution of our problem of computing the moments of the inverse of a compound Wishart random matrix, we obtain explicit moment formulas for the pseudo-inverse of Ginibre random matrices. These results are also given in the paper, [CMS]. Using the moments of the inverse of compound Wishart matrices, we obtain asymptotically unbiased estimators of the risk and the weights of the MV portfolio. Finally, we have some numerical results which are part of our future work.

Platooning is an interesting area which involve the possibility of decreasing the fuel consumption of heavy-duty vehicles. By reducing the inter-vehicle spacing in the platoon we can reduce air drag, which in turn reduces fuel consumption. Two fuel-efficient model predictive controllers for HDVs in a platoon has been formulated in this master thesis, both utilizing road grade preview information. The first controller is based on linear programming (LP) algorithms and the second on quadratic programming (QP). These two platooning controllers are compared with each other and with generic controllers from Scania. The LP controller proved to be more fuel-efficient than the QP controller, the Scania controllers are however more fuel-efficient than the LP controller.

Em ambientes de desastre urbano, grupos de agentes de resgate devem resolver tarefas de modo a minimizar os danos que podem ocorrer na cidade. Tais ambientes são dinâmicos e parcialmente observáveis, com características que dizem respeito à distância espacial, quantidade de recursos, à dificuldade da tarefa de desastre e à capacidade do agente de atendê-la. A comunicação entre os agentes pode ser ruidosa ou inexistente. Os sistemas multiagente são desenvolvidos para resolver problemas complexos e abrangentes, que estão além da capacidade de um único agente. Nesse contexto, os agentes são elementos computacionais autônomos que são responsáveis por uma parte da solução do problema. Os agentes são situados em um ambiente e podem ter habilidade social, interagindo com outros agentes para resolver as tarefas. Comumente, o domínio de desastre urbano é formalizado como um problema de alocação de tarefas e modelado como um problema de otimização de restrições distribuídas entre agentes heterogêneos, onde eles têm que escolher as tarefas que maximizam suas utilidades individuais ou minimizem seus custos individuais. Essa dissertação de mestrado propõe um modelo para formação de grupos de agentes baseado na minimização de uma métrica de distância. O modelo é formalizado como um problema de otimização de restrições distribuídas, usando algoritmos para troca de mensagens entre os agentes. O modelo chamado Formação de Grupos pela Minimização da Distância (FGMD) tem agentes autônomos que tem a capacidade de se auto-organizar sem a necessidade de um controle centralizado. Aplicamos o FGMD na plataforma RMASBench, que é um simulador para situações de desastre urbano. Comparou-se o FGMD com os algoritmos mais recentes de passagem de mensagens, tendo sido verificado que o FGMD use menos computação não-paralela. Com respeito a minimização dos danos na cidade, mostrou-se que é possível obter resultados melhores que as abordagens do estado-da-arte com leve aumento no esforço computacional. / In urban disaster environments, groups of rescue agents must solve tasks in order to minimize the damage that can occur in a city. Such environments are dynamic and partially observable, with features that correspond to spatial distance, amount of resources, difficulty of the disaster task, and the capability of the agent to handle it. The communication between the agents can be noisy or non-existent. Multiagent systems are developed to solve complex and comprehensive problems, that are beyond the capability of one single agent. In this context, the agents are autonomous computational elements that are responsible for a piece of the solution of the problem. The agents are situated in an environment, and may have social ability, interacting with other agents to solve the tasks. Commonly, the urban disaster domain is formalized as a task allocation problem, and modelled as a constraint optimization problem distributed among heterogeneous agents, where they have to choose the tasks that maximize their individual utilities or minimize their individual costs. This master thesis proposes a model for formation of groups of agents based in the minimization of a distance. The model is formalized as a distributed constraint optimization problem, using algorithms to exchange messages between agents. The model called Formation of Groups by Minimization of Distance (FGMD) has self-organizing autonomous agents without a centralized control. We applied the FGMD in the RMASBench platform, that is a simulator for urban disaster situations. We compare the FGMD with the most recent message passing algorithms, verifying that FGMD uses less non-parallel computation. With respect to the minimization of the damage in the city, we show that it is possible to obtain better results than the state-of-art approaches, with slightly increase of computational effort.

Em ambientes de desastre urbano, grupos de agentes de resgate devem resolver tarefas de modo a minimizar os danos que podem ocorrer na cidade. Tais ambientes são dinâmicos e parcialmente observáveis, com características que dizem respeito à distância espacial, quantidade de recursos, à dificuldade da tarefa de desastre e à capacidade do agente de atendê-la. A comunicação entre os agentes pode ser ruidosa ou inexistente. Os sistemas multiagente são desenvolvidos para resolver problemas complexos e abrangentes, que estão além da capacidade de um único agente. Nesse contexto, os agentes são elementos computacionais autônomos que são responsáveis por uma parte da solução do problema. Os agentes são situados em um ambiente e podem ter habilidade social, interagindo com outros agentes para resolver as tarefas. Comumente, o domínio de desastre urbano é formalizado como um problema de alocação de tarefas e modelado como um problema de otimização de restrições distribuídas entre agentes heterogêneos, onde eles têm que escolher as tarefas que maximizam suas utilidades individuais ou minimizem seus custos individuais. Essa dissertação de mestrado propõe um modelo para formação de grupos de agentes baseado na minimização de uma métrica de distância. O modelo é formalizado como um problema de otimização de restrições distribuídas, usando algoritmos para troca de mensagens entre os agentes. O modelo chamado Formação de Grupos pela Minimização da Distância (FGMD) tem agentes autônomos que tem a capacidade de se auto-organizar sem a necessidade de um controle centralizado. Aplicamos o FGMD na plataforma RMASBench, que é um simulador para situações de desastre urbano. Comparou-se o FGMD com os algoritmos mais recentes de passagem de mensagens, tendo sido verificado que o FGMD use menos computação não-paralela. Com respeito a minimização dos danos na cidade, mostrou-se que é possível obter resultados melhores que as abordagens do estado-da-arte com leve aumento no esforço computacional. / In urban disaster environments, groups of rescue agents must solve tasks in order to minimize the damage that can occur in a city. Such environments are dynamic and partially observable, with features that correspond to spatial distance, amount of resources, difficulty of the disaster task, and the capability of the agent to handle it. The communication between the agents can be noisy or non-existent. Multiagent systems are developed to solve complex and comprehensive problems, that are beyond the capability of one single agent. In this context, the agents are autonomous computational elements that are responsible for a piece of the solution of the problem. The agents are situated in an environment, and may have social ability, interacting with other agents to solve the tasks. Commonly, the urban disaster domain is formalized as a task allocation problem, and modelled as a constraint optimization problem distributed among heterogeneous agents, where they have to choose the tasks that maximize their individual utilities or minimize their individual costs. This master thesis proposes a model for formation of groups of agents based in the minimization of a distance. The model is formalized as a distributed constraint optimization problem, using algorithms to exchange messages between agents. The model called Formation of Groups by Minimization of Distance (FGMD) has self-organizing autonomous agents without a centralized control. We applied the FGMD in the RMASBench platform, that is a simulator for urban disaster situations. We compare the FGMD with the most recent message passing algorithms, verifying that FGMD uses less non-parallel computation. With respect to the minimization of the damage in the city, we show that it is possible to obtain better results than the state-of-art approaches, with slightly increase of computational effort.

Em ambientes de desastre urbano, grupos de agentes de resgate devem resolver tarefas de modo a minimizar os danos que podem ocorrer na cidade. Tais ambientes são dinâmicos e parcialmente observáveis, com características que dizem respeito à distância espacial, quantidade de recursos, à dificuldade da tarefa de desastre e à capacidade do agente de atendê-la. A comunicação entre os agentes pode ser ruidosa ou inexistente. Os sistemas multiagente são desenvolvidos para resolver problemas complexos e abrangentes, que estão além da capacidade de um único agente. Nesse contexto, os agentes são elementos computacionais autônomos que são responsáveis por uma parte da solução do problema. Os agentes são situados em um ambiente e podem ter habilidade social, interagindo com outros agentes para resolver as tarefas. Comumente, o domínio de desastre urbano é formalizado como um problema de alocação de tarefas e modelado como um problema de otimização de restrições distribuídas entre agentes heterogêneos, onde eles têm que escolher as tarefas que maximizam suas utilidades individuais ou minimizem seus custos individuais. Essa dissertação de mestrado propõe um modelo para formação de grupos de agentes baseado na minimização de uma métrica de distância. O modelo é formalizado como um problema de otimização de restrições distribuídas, usando algoritmos para troca de mensagens entre os agentes. O modelo chamado Formação de Grupos pela Minimização da Distância (FGMD) tem agentes autônomos que tem a capacidade de se auto-organizar sem a necessidade de um controle centralizado. Aplicamos o FGMD na plataforma RMASBench, que é um simulador para situações de desastre urbano. Comparou-se o FGMD com os algoritmos mais recentes de passagem de mensagens, tendo sido verificado que o FGMD use menos computação não-paralela. Com respeito a minimização dos danos na cidade, mostrou-se que é possível obter resultados melhores que as abordagens do estado-da-arte com leve aumento no esforço computacional. / In urban disaster environments, groups of rescue agents must solve tasks in order to minimize the damage that can occur in a city. Such environments are dynamic and partially observable, with features that correspond to spatial distance, amount of resources, difficulty of the disaster task, and the capability of the agent to handle it. The communication between the agents can be noisy or non-existent. Multiagent systems are developed to solve complex and comprehensive problems, that are beyond the capability of one single agent. In this context, the agents are autonomous computational elements that are responsible for a piece of the solution of the problem. The agents are situated in an environment, and may have social ability, interacting with other agents to solve the tasks. Commonly, the urban disaster domain is formalized as a task allocation problem, and modelled as a constraint optimization problem distributed among heterogeneous agents, where they have to choose the tasks that maximize their individual utilities or minimize their individual costs. This master thesis proposes a model for formation of groups of agents based in the minimization of a distance. The model is formalized as a distributed constraint optimization problem, using algorithms to exchange messages between agents. The model called Formation of Groups by Minimization of Distance (FGMD) has self-organizing autonomous agents without a centralized control. We applied the FGMD in the RMASBench platform, that is a simulator for urban disaster situations. We compare the FGMD with the most recent message passing algorithms, verifying that FGMD uses less non-parallel computation. With respect to the minimization of the damage in the city, we show that it is possible to obtain better results than the state-of-art approaches, with slightly increase of computational effort.