No description available.

Neural network models such as Transformer-based BERT, mBERT and RoBERTa are achieving impressive performance (Devlin et al., 2019; Lewis et al., 2020; Liu et al., 2019; Raffel et al., 2020; Y. Sun et al., 2019), but we still know little about their inner working due to the complex technique like multi-head self-attention they implement. Attention is commonly taken as a crucial way to explain the model outputs, but there are studies argue that attention may not provide faithful and reliable explanations in recent years (Jain and Wallace, 2019; Pruthi et al., 2020; Serrano and Smith, 2019; Wiegreffe and Pinter, 2019). Bastings and Filippova (2020) then propose that saliency may give better model interpretations since it is designed to find which token contributes to the prediction, i.e. the exact goal of explanation.  In this thesis, we investigate the extent to which syntactic structure is reflected in BERT, mBERT and RoBERTa trained on English and Chinese by using a gradient-based saliency method introduced by Simonyan et al. (2014). We examine the dependencies that our models and baselines predict.  We find that our models can predict some dependencies, especially those that have shorter mean distance and more fixed position of heads and dependents, even though all our models can handle global dependencies in theory. Besides, BERT usually has higher overall accuracy on connecting dependents to their corresponding heads, followed by mBERT and RoBERTa. Yet all the three model in fact have similar results on individual relations. Moreover, models trained on English have better performances than models trained on Chinese, possibly because of the flexibility of Chinese language.

In the context of gradient-based numerical optimisation, the adjoint method is an e cient way of computing the gradient of the cost function at a computational cost independent of the number of design parameters, which makes it a captivating option for industrial CFD applications involving costly primal solves. The method is however a ected by instabilities, some of which are inherited from the primal solver, notably if the latter does not fully converge. The present work is an attempt at curbing primal solver limitations with the goal of indirectly alleviating adjoint robustness issues. To that end, a novel discretisation scheme for the steady-state incompressible Navier- Stokes problem is proposed: Mixed Hybrid Finite Volumes (MHFV). The scheme draws inspiration from the family of Mimetic Finite Di erences and Mixed Virtual Elements strategies, rid of some limitations and numerical artefacts typical of classical Finite Volumes which may hinder convergence properties. Derivation of MHFV operators is illustrated and each scheme is validated via manufactured solutions: rst for pure anisotropic di usion problems, then convection-di usion-reaction and nally Navier-Stokes. Traditional and novel Navier-Stokes solution algorithms are also investigated, adapted to MHFV and compared in terms of performance. The attention is then turned to the discrete adjoint Navier-Stokes system, which is assembled in an automated way following the principles of Equational Di erentiation, i.e. the di erentiation of the primal discrete equations themselves rather than the algorithm used to solve them. Practical/computational aspects of the assembly are discussed, then the adjoint gradient is validated and a few solution algorithms for the MHFV adjoint Navier-Stokes are proposed and tested. Finally, two examples of full shape optimisation procedures on internal ow test cases (S-bend and U-bend) are reported.

Numerical simulation is an important tool in understanding the electromagnetic field and how it interacts with the environment. Different topics for time-domain finite-difference (FDTD) and finite-element (FETD) methods for Maxwell's equations are treated in this thesis. Subcell models are of vital importance for the efficient modeling of small objects that are not resolved by the grid. A novel model for thin sheets using shell elements is proposed. This approach has the advantage of taking into account discontinuities in the normal component of the electric field, unlike previous models based on impedance boundary conditions (IBCs). Several results are presented to illustrate the capabilities of the shell element approach. Waveguides are of fundamental importance in many microwave applications, for example in antenna feeds. The key issues of excitation and truncation of waveguides are addressed. A complex frequency shifted form of the uniaxial perfectly matched layer (UPML) absorbing boundary condition (ABC) in FETD is developed. Prism elements are used to promote automatic grid generation and enhance the performance. Results are presented where reflection errors below -70dB are obtained for different types of waveguides, including inhomogeneous cases. Excitation and analysis via the scattering parameters are achieved using waveguide modes computed by a general frequency-domain mode solver for the vector Helmholtz equation. Huygens surfaces are used in both FDTD and FETD for excitation in waveguide ports. Inverse problems have received an increased interest due to the availability of powerful computers. An important application is non-destructive evaluation of material. A time-domain, minimization approach is presented where exact gradients are computed using the adjoint problem. The approach is applied to a general form of Maxwell's equations including dispersive media and UPML. Successful reconstruction examples are presented both using synthetic and experimental measurement data. Parameter reduction of complex geometries using simplified models is an interesting topic that leads to an inverse problem. Gradients for subcell parameters are derived and a successful reconstruction example is presented for a combined dielectric sheet and slot geometry.

Design optimization is a method that can be used to automate the design process to obtain better results. When applied to aeroelastic structures, design optimization often leads to the creation of highly flexible aeroelastic structures. There are, however, a number of conventional design procedures that must be modified when dealing with highly flexible aeroelastic structures. First, the deformed geometry must be the baseline for weight, structural, and stability analyses. Second, potential couplings between aeroelasticity and rigid-body dynamics must be considered. Third, dynamic analyses must be modified to handle large nonlinear displacements. These modifications to the conventional design process significantly increase the difficulty of developing an optimization framework appropriate for highly flexible aeroelastic structures. As a result, when designing these structures, often either gradient-free optimization is performed (which limits the optimization to relatively few design variables) or optimization is simply omitted from the design process. Both options significantly decrease the design exploration capabilities of a designer compared to a scenario in which gradient-based optimization is used. This dissertation therefore presents various contributions that allow gradient-based optimization to be more easily used to optimize highly flexible aeroelastic structures. One of our primary motivations for developing these capabilities is to accurately capture the design constraints of solar-regenerative high-altitude long-endurance (SR-HALE) aircraft. In this dissertation, we therefore present a SR-HALE aircraft optimization framework which accounts for the peculiarities of structurally flexible aircraft while remaining suitable for use with gradient-based optimization. These aircraft tend to be extremely large and light, which often leads to significant amounts of structural flexibility. Using this optimization framework, we design an aircraft that is capable of flying year-round at \SI{35}{\degree} latitude at \SI{18}{\kilo\meter} above sea level. We subject this aircraft to a number of constraints including energy capture, energy storage, material failure, local buckling, stall, static stability, and dynamic stability constraints. Critically, these constraints were designed to accurately model the actual design requirements of SR-HALE aircraft, rather than to provide a rough approximation of them. To demonstrate the design exploration capabilities of this framework, we also performed several parameters sweeps to determine optimal design sensitivities to altitude, latitude, battery specific energy, solar efficiency, avionics and payload power requirements, and minimum design velocity. Through this optimization framework, we demonstrate both the potential of gradient-based optimization applied to highly flexible aeroelastic structures and the challenges associated with it. One challenge associated with the gradient-based optimization of highly flexible aeroelastic structures, is the ability to accurately, efficiently, and reliably model the large deflections of these structures in gradient-based optimization frameworks. To enable large-scale optimization involving structural models with large deflections to be performed more easily, we present a finite-element implementation of geometrically exact beam theory which is designed specifically for gradient-based optimization. A key feature of this implementation of geometrically exact beam theory is its compatibility with forward and reverse-mode automatic differentiation, which allows accurate design sensitivities to be obtained with minimal development effort. Another key feature is its native support for unsteady adjoint sensitivity analysis, which allows design sensitivities to be obtained efficiently from time-marching simulations. Other features are also presented that build upon previous implementations of geometrically exact beam theory, including a singularity-free rotation parameterization based on Wiener-Milenkovi\'c parameters, an implementation of stiffness-proportional structural damping using a discretized form of the compatibility equations, and a reformulation of the equations of motion for geometrically exact beam theory from a fully implicit index-1 differential algebraic equation to a semi-explicit index-1 differential algebraic equation. Several examples are presented which verify the utility and validity of each of these features. Another challenge associated with the gradient-based optimization of highly flexible aeroelastic structures is the ability to reliably track and constrain individual dynamic stability modes across the design iterations of an optimization framework. To facilitate the development of mode-specific dynamic stability constraints in gradient-based optimization frameworks we develop a mode tracking method that uses an adaptive step size in order to maintain an arbitrarily high degree of confidence in mode correlations. This mode tracking method is then applied to track the modes of a linear two-dimensional aeroelastic system and a nonlinear three-dimensional aeroelastic system as velocity is increased. When used in a gradient-based optimization framework, this mode tracking method has the potential to allow continuous dynamic stability constraints to be constructed without constraint aggregation. It also has the potential to allow the stability and shape of specific modes to be constrained independently. Finally, to facilitate the development and use of highly flexible aeroelastic systems for use in gradient-based optimization frameworks, we introduce a general methodology for coupling aerodynamic and structural models together to form modular monolithic aeroelastic systems. We also propose efficient methods for computing the Jacobians of these coupled systems without significantly increasing the amount of time necessary to construct these systems. For completeness we also discuss how to ensure that the resulting system of equations constitutes a set of first-order index-1 differential algebraic equations. We then derive direct and adjoint sensitivities for these systems which are compatible with automatic differentiation so that derivatives for gradient-based optimization can be obtained with minimal development effort.

Thesis (MScEng (Department of Mechanical and Mechatronic Engineering))--Stellenbosch University, 2006. / ENGLISH ABSTRACT: Wind conditions in South Africa determine the need for a small-scale wind turbine to produce useable power at windspeeds below 7m/s. In this project, a range of windspeeds, within which optimal performance o the wind turbine is expected, was selected. The optimal performance was assessed in terms of the Coefficient of Power(Cp), which rates the turbines blade's ability to extract energy form the avalible wind stream. The optimisation methods employed allowed a means of tackling the multi-variable problem such that the aerodynamic characteristics of the blade were ideal throughout the wind speed range. The design problem was broken down into a two-dimensional optimisaion of the airfoils used at the radial stations, and a three-dimensional optimisation of the geometric features of the wind rotor. by means of blending various standard airfoil profiles, a new profile was created at each radial station. XFOIL was used for the two-dimensional analysis of these airfoils. Three-dimensional optimisn involved representation of the rotor as a simplified model and use of the Blade Element Momentum(BEM) method for analysis. an existimg turbine blade, on which the design specifications were modelled, was further used for comparative purposes throughout the project. The resulting blade design offers substantial improvements on the reference design. The application of optimisation methods has successfully aided the creation of a wind turbine blade with consistent peak performance over a range of design prints. / Sponsored by the Centre for Renewable and Sustainable Energy Studies, Stellenbosch University

Noise has become a main nuisance in urban areas to the point that according to the World Health Organization 40% of the European population is exposed to excessive noise levels, mainly due to ground transportation. There is therefore a need to find new ways to mitigate noise in urban areas. In this work, a possible device to achieve this goal is studied: a low-height noise barrier. It consists of a barrier typically less than one meter high placed close to a source, designed to decrease the noise level for nearby pedestrians and cyclists. This type of device is studied both numerically and experimentally. Tramway noise barriers are especially studied since the noise sources are in this case very close to the ground and can therefore be attenuated efficiently. The shape and the surface treatment of the barrier are optimized using a gradient-based method coupled to a 2D boundary element method (BEM). The optimization variables are the node coordinates of a control mesh and the parameters describing the surface impedance. Sensitivities are calculated efficiently using the adjoint state approach. Numerical results show that the shapes generated by the optimization algorithm tend to be quite irregular but provide a significant improvement of more than 5 dB (A) compared to simpler shapes. Utilizing an absorbing treatment on the source side of the barrier is shown to be efficient as well. This second point has been confirmed by scale model measurements. In addition, a full scale low height noise barrier prototype has been built and tested in situ close to a tramway track in Grenoble. Measurements show that the device provides more than 10 dB (A) of attenuation for a close receiver located at the typical height of human ears. These results therefore seem to confirm the applicability of such protections to efficiently decrease noise exposure in urban areas

Efficient energy use is primarily for any sensor networks to function for a longer time period. There have been many efficient schemes with various progress levels proposed by many researchers. Yet, there still more improvements are needed. This thesis is an attempt to make wireless sensor networks with further efficient on energy usage in the network with respect to rate of delivery of the messages. In sensor network architecture radio, sensing and actuators have influence over the power consumption in the entire network. While listening as well as transmitting, energy is consumed by the radio. However, if by reducing listening times or by reducing the number of messages transmitting would reduce the energy consumption. But, in real time scenario with critical information sensing network leads to information loss. To overcome this an adaptive routing technique should be considered. So, that it focuses on saving energy in a much more sophisticated way without reducing the performance of the sensing network transmitting and receiving functionalities. This thesis tackles on parts of the energy efficiency problem in a wireless sensor network and improving delivery rate of messages. To achieve this a routing technique is proposed. In this method, switching between two routing paths are considered and the switching decision taken by the server based on messages delivered comparative previous time intervals. The goal is to get maximum network life time without degrading the number of messages at the server. In this work some conventional routing methods are considered for implementing an approach. This approach is by implementing a shortest path, Gradient based energy routing algorithm and an observer component to control switching between paths. Further, controlled switching done by observer compared to normal initial switch rule. Evaluations are done in a simulation environment and results show improvement in network lifetime in a much more balanced way.

La présente étude concerne l’étude d’optimisation thermique d’une machine électrique. Un modèle nodal est utilisé pour la simulation du champ de température. Ce modèle résout l’équation de la chaleur en trois dimensions, en coordonnées cylindriques et en régime transitoire ou permanent. On prend en compte les deux mécanismes de transport les plus importants : La conduction et la convection. L’évaluation de ce modèle est effectuée par l’intermédiaire de 13 valeurs de débits de référence. C’est en faisant varier ces variables qu’on évalue la performance du refroidissement dans la machine. Avant de partir sur l’étude d’optimisation de cettegéométrie, on a lancé une étude d’optimisation d’un cas plus simple afin de mieux comprendre les différents outils d’optimisation disponibles. L’expérience acquise avec les cas simples est utilisée dans l’optimisation thermique de la machine. La machine est thermiquement évaluée sur la combinaison de deux critères : la température maximale et la température moyenne. Des contraintes ont été additionnées afin d’obtenir des résultats physiquement acceptables. Le problème est résolu à l’aide des méthodes de gradient (Active-set et Point-Intérieur) et des Algorithmes Génétiques. / This work relates the thermal optimization of an electrical machine. The lumped method is used to simulate the temperature field. This model solves the heat equation in three dimensions, in cylindrical coordinates and in transient or steady state. We consider two transport mechanisms: conduction and convection. The evaluation of this model is performed by means of 13 design variables that correspond to the main flow rates of the equipment. We analyse the machine cooling performance by varying these 13 flow rates. Before starting the study of such a complicated geometry, we picked a simpler case in order to better understand the variety of the available optimization tools. The experience obtained in the simpler case is applyed in the resolution of the thermal optimization problem of the electrical machine. This machine is evaluated from the thermal point of view by combining two criteria : the maximum and the mean temperature. Constraints are used to keep the problem consistent. We solved the problem using the gradient based methods (Active-set and Interior-Point) and the Genetic Algorithms.

The use of steady-state aerodynamic optimization methods in the computational fluid dynamic (CFD) community is fairly well established. In particular, the use of adjoint methods has proven to be very beneficial because their cost is independent of the number of design variables. The application of numerical optimization to airframe-generated noise, however, has not received as much attention, but with the significant quieting of modern engines, airframe noise now competes with engine noise. Optimal control techniques for unsteady flows are needed in order to be able to reduce airframe-generated noise. In this thesis, a general framework is formulated to calculate the gradient of a cost function in a nonlinear unsteady flow environment via the discrete adjoint method. The unsteady optimization algorithm developed in this work utilizes a Newton-Krylov approach since the gradient-based optimizer uses the quasi-Newton method BFGS, Newton's method is applied to the nonlinear flow problem, GMRES is used to solve the resulting linear problem inexactly, and last but not least the linear adjoint problem is solved using Bi-CGSTAB. The flow is governed by the unsteady two-dimensional compressible Navier-Stokes equations in conjunction with a one-equation turbulence model, which are discretized using structured grids and a finite difference approach. The effectiveness of the unsteady optimization algorithm is demonstrated by applying it to several problems of interest including shocktubes, pulses in converging-diverging nozzles, rotating cylinders, transonic buffeting, and an unsteady trailing-edge flow. In order to address radiated far-field noise, an acoustic wave propagation program based on the Ffowcs Williams and Hawkings (FW-H) formulation is implemented and validated. The general framework is then used to derive the adjoint equations for a novel hybrid URANS/FW-H optimization algorithm in order to be able to optimize the shape of airfoils based on their calculated far-field pressure fluctuations. Validation and application results for this novel hybrid URANS/FW-H optimization algorithm show that it is possible to optimize the shape of an airfoil in an unsteady flow environment to minimize its radiated far-field noise while maintaining good aerodynamic performance.