Artificial neural network (ANN) is a machine learning approach where parameters, i.e., frequency parameters and amplitude parameters, are learnt during the training process. Random features model is a special case of ANN that the structure of random features model is as same as ANN’s but the parameters’ learning processes are different. For random features model, the amplitude parameters are learnt during the training process but the frequency parameters are sampled from some distributions. If the frequency distribution of the random features model is well-chosen, both models can approximate data well. Adaptive random Fourier features with Metropolis sampling is an enhanced random Fourier features model which can select appropriate frequency distribution adaptively. This thesis studies Rectified Linear Unit and sigmoid features and combines them with the adaptive idea to generate another two adaptive random features models. The results show that using the particular set of hyper-parameters, adaptive random Rectified Linear Unit features model can also approximate the data relatively well, though the adaptive random Fourier features model performs slightly better. / I artificiella neurala nätverk (ANN), som används inom maskininlärning, behöver parametrar, kallade frekvensparametrar och amplitudparametrar, hittasgenom en så kallad träningsprocess. Random feature-modeller är ett specialfall av ANN där träningen sker på ett annat sätt. I dessa modeller tränasamplitudparametrarna medan frekvensparametrarna samplas från någon sannolikhetstäthet. Om denna sannolikhetstäthet valts med omsorg kommer båda träningsmodellerna att ge god approximation av givna data. Metoden Adaptiv random Fourier feature[1] uppdaterar frekvensfördelningen adaptivt. Denna uppsats studerar aktiveringsfunktionerna ReLU och sigmoid och kombinerar dem med den adaptiva iden i [1] för att generera två ytterligare Random feature-modeller. Resultaten visar att om samma hyperparametrar som i [1] används så kan den adaptiva ReLU features-modellen approximera data relativt väl, även om Fourier features-modellen ger något bättre resultat.

Derivative pricing, model calibration, and sensitivity analysis are the three main problems in financial modeling. The purpose of this study is to present an algorithm to improve the pricing process, the calibration process, and the sensitivity analysis of the double Heston model, in the sense of accuracy and efficiency. Using the optimized caching technique, our study reduces the pricing computation time by about 15%. Another contribution of this thesis is: a novel application of the Automatic Differentiation (AD) algorithms in order to achieve a more stable, more accurate, and faster sensitivity analysis for the double Heston model (compared to the classical finite difference methods). This thesis also presents a novel hybrid model by combing the heuristic method Differentiation Evolution, and the gradient method Levenberg--Marquardt algorithm. Our new hybrid model significantly accelerates the calibration process.

A hybrid design optimization method combining the stochastic method based on simultaneous perturbation stochastic approximation (SPSA) and the deterministic method of Broydon-Fletcher-Goldfarb-Shanno (BFGS) is developed in order to take advantage of the high efficiency of the gradient based methods and the global search capabilities of SPSA for applications in the optimal aerodynamic shape design of a three dimensional elliptic nozzle. The performance of this hybrid method is compared with that of SPSA, simulated annealing (SA) and gradient based BFGS method. The objective functions which are minimized are estimated by numerically solving the 3D Euler and Navier-Stokes equations using a TVD approach and a LU implicit scheme. Computed results show that the hybrid optimization method proposed in this study shows a promise of high computational efficiency and global search capabilities. / Singapore-MIT Alliance (SMA)

In this study, a simple and inexpensive membrane mechanical property measuring system was developed. By applying the force on a membrane and recording the corresponding out-of-plane displacement fields, then the Young¡¦s modules and Possion¡¦s ratio of the membrane can be obtained from those deformations through the inverse approach. Firstly, a loading frame was designed to fix the membrane and allow the membrane can be loaded and its deformations can be measured precisely. In order to measure the out-of-plane displacement fields of the loaded membrane, the digital image correlation (DIC) was used and an easier 3-D DIC measuring method was proposed in this study. The proposed 3-D DIC measuring method was verified by using a loaded cantilever beam with ESPI. The error was within in 10%. In this study, the smallest in-plane displacement that can be measured by proposed method is 2 £gm and the smallest out-of-plane displacement that that can be measured is 6£gm. In this study, in order to determine the mechanical properties of the membrane, digital image correlation, finite element method (FEM) and optimization method were combined with the measured out-of-plane displacement fields, then the Young¡¦s modules and Possion¡¦s ratio of the membrane were determined through the inverse approach. The FEM simulations were performed by using ANSYS. Several optimization theorems were adopted and their corresponding merits on this study were compared The obtained Young's modulus was compared with the results obtain from the nano-indentor and the error was within in 3% ~ 12%. Keyword: digital image correlation, membrane, Young¡¦s modules, Possion¡¦s ratio, finite element method, optimization method.

No desenvolvimento de projetos de iluminação de ambientes, um dos objetivos que se destaca é a busca pelo conforto visual, que emprega metodologias de resolução conhecidas, como o Método dos Lumens e o Método Ponto a Ponto. A luz visível está contida no espectro da radiação térmica e, portanto, o fluxo luminoso pode ser relacionado ao fluxo de radiação térmica. Determinar as posições e as competências das fontes de luz necessárias na superfície de projeto ganha importância quando o comportamento, em termos de uniformidade ou de fluxo radiante, é especificado. O presente trabalho visa a estabelecer diferentes valores de fluxo em duas regiões distintas da superfície do projeto. Por meio do posicionamento das fontes de luz, é estabelecido um fluxo maior na região denominada principal e um fluxo menor na região denominada secundária. A modelagem matemática da radiação térmica (Método das Radiosidades) é aplicada ao projeto de iluminação, considerando-se as características da visão humana e o comportamento das fontes de luz. Na modelagem, é considerada uma cavidade retangular tridimensional com superfícies cinza e com condição de parede fria, na qual o poder emissivo das paredes é nulo. As fontes de luz são representadas por unidades de malha no teto. A relação de equações é resolvida por metodologia inversa, usando o algoritmo de Otimização Extrema Generalizada (GEO). Este algoritmo é classificado como um método de otimização estocástica de busca global para a resolução de sistemas considerados inicialmente mal condicionados. A posição e a potência das fontes luminosas são determinadas pela resolução do sistema de equações, de forma a proporcionar um fluxo de radiação duas vezes maior na região principal em relação à região secundária. A função objetivo do processo consiste em minimizar a diferença entre o fluxo desejado e os valores de fluxo de radiação incidente nas duas regiões da superfície de projeto. Em virtude das características de simetria do problema, a relação é estabelecida para apenas um quarto da cavidade. Assim, por exemplo, aplicar a metodologia com 9 fontes de luz a um quarto da região resulta em 36 fontes de luz em toda a cavidade. Os resultados mostram que é possível encontrar um arranjo de fontes de luz preestabelecendo-se duas condições de potência. / In the development of environmental illumination projects, one of the main goals to be achieved is the visual comfort, which is usually done by known methodologies, like the Lumens Method and the Point by Point Method. Since the visible light is contained in the spectrum of thermal radiation, the luminous flux can be related to the thermal radiation flux. The determination of the position and power of the light sources required by the design surface gains an higher importance whenever a behavior is specified, should it be in terms of uniformity or in therms of radiant flux. In this work, we describe a method that allows the establishment of different flux values in two distinct regions of the design surface, which are referred by the names main region and secondary region. Through the spatial arrangement of the light sources, the method sets a more intense flux in the main region and a less intense one in the secondary region. The mathematical model of thermal radiation, known as Radiosity Method, is applied to the illumination design, along with the characteristics of the human vision and the behavior of light sources. In this model, a rectangular three-dimensional cavity is considered. It has gray surfaces and exhibits the conditions of a cold wall, in which the emissivity power of the walls is null. The light sources are represented by a mesh unit in the ceiling. The system of equations is solved by inversemethodology, using the Generalized Extremal Optimization (GEO) algorithm. This algoritm is classified as being a stochastic optimization method of global search to solve systems that are initially considered ill-conditioned. By solving this system, the position and power of light sources can be determined, and this is done in such a way that the flux radiation in the main region is twice more intense then the one in the secondary region. The target function of the whole process is to minimize the difference between the desired flux and the incident flux radiation values for each one of the two design surface regions. We further explore the problem symmetry, solving the equation system for only a quarter of the cavity. This way, if the methodology is applied with nine light sources into a quarter of the region, the entire cavity will behave as if it has 36 light sources. Our results show that, given two prescribed conditions of power, it is possible to find an arrangement of light sources.

General Purpose computing on Graphic Processing Unit (GPGPU) gained atten- tion in 2006 with NVIDIA’s first Tesla Graphic Processing Unit (GPU) which could perform high performance computing. Ever since, researchers have been working on software and hardware techniques to improve the efficiency of running general purpose applications on GPUs. The efficiency can be evaluated using metrics such as energy consumption and throughput and is defined based on the requirements of the system. I define it as obtaining high throughput by consuming minimum energy. GPUs are equipped with a large number of processing units, a high memory bandwidth, and different types of on-chip memory and caches. To run efficiently, an application should maximize the utilization of GPU resources. Therefore, a good correspondence between the computing and memory resources of the GPU and those of application is critical. Since an application’s requirements are fixed, the GPU’s configuration should be tuned to these requirements. Having models to study and predict the power consumption and throughput of running a GPGPU application on a given GPU configuration can help achieve high efficiency. The main purpose of this dissertation is to find a GPU configuration that best matches the requirements of a given application. I propose three models that predict a GPU configuration that runs an application with maximum throughput while consuming minimum energy. The first model is a fast, low-cost and effective approach to optimize resource allocation in future GPUs. The model finds the optimal GPU configuration for different available chip real-estate budgets . The second model considers the power consumption and throughput of a GPGPU application as functions of the GPU configuration parameters. The proposed model accurately predicts the power consumption and throughput of the modeled GPGPU application. I then propose to accelerate the process of building the model using optimization techniques and quantum annealing. I use the proposed model to explore the GPU configuration space of different applications. I apply multiobjective optimization technique to find the configurations that offer minimum power consumption and maximum throughput. Finally, using clustering and classification techniques, I develop models to re- late the power consumption and throughput of GPGPU applications to the code attributes. Both models could accurately predict the optimum configuration for any given GPGPU application. To build these models I have used different machine learning techniques and optimization methods such as Pareto Front and Knapsack optimization problem. I validated the model produced results with simulation results and showed that the models make accurate predictions. These models could be used by GPGPU programmers to identify the architectural parameters that most affect an application’s power consumption and throughput. This information could be translated into software optimization opportunities. Also, these models can be implemented as part of a compiler to help it to make the best optimization decisions. Moreover, GPU manufacturers could gain insight on architectural parameters which would profit GPGPU applications the most in terms of power and performance and hence invest on these. / Graduate

No desenvolvimento de projetos de iluminação de ambientes, um dos objetivos que se destaca é a busca pelo conforto visual, que emprega metodologias de resolução conhecidas, como o Método dos Lumens e o Método Ponto a Ponto. A luz visível está contida no espectro da radiação térmica e, portanto, o fluxo luminoso pode ser relacionado ao fluxo de radiação térmica. Determinar as posições e as competências das fontes de luz necessárias na superfície de projeto ganha importância quando o comportamento, em termos de uniformidade ou de fluxo radiante, é especificado. O presente trabalho visa a estabelecer diferentes valores de fluxo em duas regiões distintas da superfície do projeto. Por meio do posicionamento das fontes de luz, é estabelecido um fluxo maior na região denominada principal e um fluxo menor na região denominada secundária. A modelagem matemática da radiação térmica (Método das Radiosidades) é aplicada ao projeto de iluminação, considerando-se as características da visão humana e o comportamento das fontes de luz. Na modelagem, é considerada uma cavidade retangular tridimensional com superfícies cinza e com condição de parede fria, na qual o poder emissivo das paredes é nulo. As fontes de luz são representadas por unidades de malha no teto. A relação de equações é resolvida por metodologia inversa, usando o algoritmo de Otimização Extrema Generalizada (GEO). Este algoritmo é classificado como um método de otimização estocástica de busca global para a resolução de sistemas considerados inicialmente mal condicionados. A posição e a potência das fontes luminosas são determinadas pela resolução do sistema de equações, de forma a proporcionar um fluxo de radiação duas vezes maior na região principal em relação à região secundária. A função objetivo do processo consiste em minimizar a diferença entre o fluxo desejado e os valores de fluxo de radiação incidente nas duas regiões da superfície de projeto. Em virtude das características de simetria do problema, a relação é estabelecida para apenas um quarto da cavidade. Assim, por exemplo, aplicar a metodologia com 9 fontes de luz a um quarto da região resulta em 36 fontes de luz em toda a cavidade. Os resultados mostram que é possível encontrar um arranjo de fontes de luz preestabelecendo-se duas condições de potência. / In the development of environmental illumination projects, one of the main goals to be achieved is the visual comfort, which is usually done by known methodologies, like the Lumens Method and the Point by Point Method. Since the visible light is contained in the spectrum of thermal radiation, the luminous flux can be related to the thermal radiation flux. The determination of the position and power of the light sources required by the design surface gains an higher importance whenever a behavior is specified, should it be in terms of uniformity or in therms of radiant flux. In this work, we describe a method that allows the establishment of different flux values in two distinct regions of the design surface, which are referred by the names main region and secondary region. Through the spatial arrangement of the light sources, the method sets a more intense flux in the main region and a less intense one in the secondary region. The mathematical model of thermal radiation, known as Radiosity Method, is applied to the illumination design, along with the characteristics of the human vision and the behavior of light sources. In this model, a rectangular three-dimensional cavity is considered. It has gray surfaces and exhibits the conditions of a cold wall, in which the emissivity power of the walls is null. The light sources are represented by a mesh unit in the ceiling. The system of equations is solved by inversemethodology, using the Generalized Extremal Optimization (GEO) algorithm. This algoritm is classified as being a stochastic optimization method of global search to solve systems that are initially considered ill-conditioned. By solving this system, the position and power of light sources can be determined, and this is done in such a way that the flux radiation in the main region is twice more intense then the one in the secondary region. The target function of the whole process is to minimize the difference between the desired flux and the incident flux radiation values for each one of the two design surface regions. We further explore the problem symmetry, solving the equation system for only a quarter of the cavity. This way, if the methodology is applied with nine light sources into a quarter of the region, the entire cavity will behave as if it has 36 light sources. Our results show that, given two prescribed conditions of power, it is possible to find an arrangement of light sources.

A fundamental assumption behind most machine learning (ML) algorithms and analyses is the sequential execution. That is, any update to the ML model can be immediately applied and the new model is always available for the next algorithmic step. This basic assumption, however, can be costly to realize, when the computation is carried out across multiple machines, linked by commodity networks that are usually 104 times slower than the memory speed due to fundamental hardware limitations. As a result, concurrent ML computation in the distributed settings often needs to handle delayed updates and perform learning in the presence of staleness. This thesis characterizes learning with staleness from three directions: (1) We extend the theoretical analyses of a number of classical ML algorithms, including stochastic gradient descent, proximal gradient descent on non-convex problems, and Frank-Wolfe algorithms, to explicitly incorporate staleness into their convergence characterizations. (2)We conduct simulation and large-scale distributed experiments to study the empirical effects of staleness on ML algorithms under indeterministic executions. Our results reveal that staleness is a key parameter governing the convergence speed for all considered ML algorithms, with varied ramifications. (3) We design staleness-minimizing parameter server systems by optimizing synchronization methods to effectively reduce the runtime staleness. The proposed optimization of a bounded consistency model utilizes the additional network bandwidths to communicate updates eagerly, relieving users of the burden to tune the staleness level. By minimizing staleness at the framework level, our system stabilizes diverging optimization paths and substantially accelerates convergence across ML algorithms without any modification to the ML programs.

Orientador: Cláudio Kitano / Banca: Mauro Henrique de Paula / Banca: Aparecido Augusto de Carvalho / Resumo: A interferometria a laser é uma técnica consolidada para a caracterização de atuadores piezoelétricos. No entanto, este método requer um alinhamento óptico preciso e uma operação meticulosa. Há um grande interesse no desenvolvimento de sistemas de medição de deslocamento e vibração usando sensores reflexivos em fibra óptica devido a sua inerente simplicidade, tamanho reduzido, largura de banda ampla, limite de detecção extremamente baixo e capacidade de realizar medições sem contato ou afetar o sistema a ser ensaiado. Neste trabalho apresenta-se um arranjo simples do sensor reflexivo para se atingir resolução sub-micrométrica, utilizando-se fibras e componentes ópticos de baixo custo e circuitos eletrônicos simples. O sistema é constituído por duas fibras ópticas adjacentes (uma transmissora e outra receptora) e com extremidades emparelhadas, posicionadas na frente de uma superfície reflexiva vibratória. A luz proveniente de uma fonte óptica (no caso um laser) é acoplada à fibra transmissora e parte dos raios refletidos pela superfície móvel é capturada pela fibra receptora, que conduz a luz para um fotodetector. A tensão de saída do fotodetector é função da distância entre as extremidades das fibras e a superfície reflexiva. Apresenta-se uma formulação teórica da função de intensidade óptica refletida no plano a uma distância qualquer, juntamente com comparações entre características experimentais e teóricas do sensor reflexivo. Finalmente, atuadores piezoelétricos flextensionais, projetados com o método de otimização topológica, são caracterizados experimentalmente através da medição de seus deslocamentos sub-micrométricos, utilizando o sensor reflexivo. As respostas em freqüência dos piezoatuadores flextensionais são levantadas e o fenômeno de erro de trajetória e linearidade são discutidos. / Abstract: The laser interferometer method is a well-established technique for the characterization of piezoelectric actuators. However, this method requires precise optical alignment and meticulous operation. There is great interest in developing displacement and vibration measurement systems using reflective fiber optic displacement sensors (RFODS) because of their inherent simplicity, small size, wide frequency range, extremely low displacement detection limit, and ability to perform measurements without contact or affecting the vibrating system. This work presents a simple arrangement of RFODS to achieve sub-micrometer resolution, using low-cost fibers and optical components, and simple electronic circuits. The system is composed of two adjacent transmitting and receiving fibers at one end, located in front of a reflecting vibrating surface. The transmitting fiber is connected to a laser source, and part of the reflected rays by the moving surface is captured by the receiving fiber, which is connected to a light detector. The output voltage is a function of the distance between probe and vibrating surface. A theoretical formulation of the reflected light intensity function at distal end plane is presented, together with comparisons of experimental and ideal RFODS characteristics. Finally, piezoelectric flextensional actuators (PFAs), designed with the topology optimization method, are experimentally characterized by the measurement of their sub micrometric displacements using a RFODS. The frequency responses of the PFAs are evaluated, and the tracking error phenomenon and linearity are discussed. / Mestre

No desenvolvimento de projetos de iluminação de ambientes, um dos objetivos que se destaca é a busca pelo conforto visual, que emprega metodologias de resolução conhecidas, como o Método dos Lumens e o Método Ponto a Ponto. A luz visível está contida no espectro da radiação térmica e, portanto, o fluxo luminoso pode ser relacionado ao fluxo de radiação térmica. Determinar as posições e as competências das fontes de luz necessárias na superfície de projeto ganha importância quando o comportamento, em termos de uniformidade ou de fluxo radiante, é especificado. O presente trabalho visa a estabelecer diferentes valores de fluxo em duas regiões distintas da superfície do projeto. Por meio do posicionamento das fontes de luz, é estabelecido um fluxo maior na região denominada principal e um fluxo menor na região denominada secundária. A modelagem matemática da radiação térmica (Método das Radiosidades) é aplicada ao projeto de iluminação, considerando-se as características da visão humana e o comportamento das fontes de luz. Na modelagem, é considerada uma cavidade retangular tridimensional com superfícies cinza e com condição de parede fria, na qual o poder emissivo das paredes é nulo. As fontes de luz são representadas por unidades de malha no teto. A relação de equações é resolvida por metodologia inversa, usando o algoritmo de Otimização Extrema Generalizada (GEO). Este algoritmo é classificado como um método de otimização estocástica de busca global para a resolução de sistemas considerados inicialmente mal condicionados. A posição e a potência das fontes luminosas são determinadas pela resolução do sistema de equações, de forma a proporcionar um fluxo de radiação duas vezes maior na região principal em relação à região secundária. A função objetivo do processo consiste em minimizar a diferença entre o fluxo desejado e os valores de fluxo de radiação incidente nas duas regiões da superfície de projeto. Em virtude das características de simetria do problema, a relação é estabelecida para apenas um quarto da cavidade. Assim, por exemplo, aplicar a metodologia com 9 fontes de luz a um quarto da região resulta em 36 fontes de luz em toda a cavidade. Os resultados mostram que é possível encontrar um arranjo de fontes de luz preestabelecendo-se duas condições de potência. / In the development of environmental illumination projects, one of the main goals to be achieved is the visual comfort, which is usually done by known methodologies, like the Lumens Method and the Point by Point Method. Since the visible light is contained in the spectrum of thermal radiation, the luminous flux can be related to the thermal radiation flux. The determination of the position and power of the light sources required by the design surface gains an higher importance whenever a behavior is specified, should it be in terms of uniformity or in therms of radiant flux. In this work, we describe a method that allows the establishment of different flux values in two distinct regions of the design surface, which are referred by the names main region and secondary region. Through the spatial arrangement of the light sources, the method sets a more intense flux in the main region and a less intense one in the secondary region. The mathematical model of thermal radiation, known as Radiosity Method, is applied to the illumination design, along with the characteristics of the human vision and the behavior of light sources. In this model, a rectangular three-dimensional cavity is considered. It has gray surfaces and exhibits the conditions of a cold wall, in which the emissivity power of the walls is null. The light sources are represented by a mesh unit in the ceiling. The system of equations is solved by inversemethodology, using the Generalized Extremal Optimization (GEO) algorithm. This algoritm is classified as being a stochastic optimization method of global search to solve systems that are initially considered ill-conditioned. By solving this system, the position and power of light sources can be determined, and this is done in such a way that the flux radiation in the main region is twice more intense then the one in the secondary region. The target function of the whole process is to minimize the difference between the desired flux and the incident flux radiation values for each one of the two design surface regions. We further explore the problem symmetry, solving the equation system for only a quarter of the cavity. This way, if the methodology is applied with nine light sources into a quarter of the region, the entire cavity will behave as if it has 36 light sources. Our results show that, given two prescribed conditions of power, it is possible to find an arrangement of light sources.