Le but de cette thèse est d'étudier la méthode de Newton pour résoudre numériquement les inclusions variationnelles, appelées aussi dans la littérature les équations généralisées. Ces problèmes engendrent en général des opérateurs multivoques. La première partie est dédiée à l'extension des approches de Kantorovich et la théorie (alpha, gamma) de Smale (connues pour les équations non-linéaires classiques) au cas des inclusions variationnelles dans les espaces de Banach. Ceci a été rendu possible grâce aux développements récents des outils de l'analyse variationnelle et non-lisse tels que la régularité métrique. La seconde partie est consacrée à l'étude de méthodes numériques de type-Newton pour les inclusions variationnelles en utilisant la différentiabilité généralisée d'applications multivoques où nous proposons de linéariser à la fois les parties univoques (lisses) et multivoques (non-lisses). Nous avons montré que, sous des hypothèses sur les données du problème ainsi que le choix du point de départ, la suite générée par la méthode de Newton converge au moins linéairement vers une solution du problème de départ. La convergence superlinéaire peut-être obtenue en imposant plus de conditions sur l'approximation multivaluée. La dernière partie de cette thèse est consacrée à l'étude des équations généralisées dans les variétés Riemaniennes à valeurs dans des espaces euclidiens. Grâce à la relation entre la structure géométrique des variétés et les applications de rétractions, nous montrons que le schéma de Newton converge localement superlinéairement vers une solution du problème. La convergence quadratique (locale et semi-locale) peut-être obtenue avec des hypothèses de régularités sur les données du problème. / This thesis is devoted to present some results in the scope of Newton-type methods applied for inclusion involving set-valued mappings. In the first part, we follow the Kantorovich's and/or Smale's approaches to study the convergence of Josephy-Newton method for generalized equation (GE) in Banach spaces. Such results can be viewed as an extension of the classical Kantorovich's theorem as well as Smale's (alpha, gamma)-theory which were stated for nonlinear equations. The second part develops an algorithm using set-valued differentiation in order to solve GE. We proved that, under some suitable conditions imposed on the input data and the choice of the starting point, the algorithm produces a sequence converging at least linearly to a solution of considering GE. Moreover, by imposing some stronger assumptions related to the approximation of set-valued part, the proposed method converges locally superlinearly. The last part deals with inclusions involving maps defined on Riemannian manifolds whose values belong to an Euclidean space. Using the relationship between the geometric structure of manifolds and the retraction maps, we show that, our scheme converges locally superlinearly to a solution of the initial problem. With some more regularity assumptions on the data involved in the problem, the quadratic convergence (local and semi-local) can be ensured.

Two new methods for the simulation of gas turbine fuel systems, one based on an inter-component volume (ICV) method, and the other based on the iterative Newton Raphson (NR) method, have been developed in this study. They are able to simulate the performance behaviour of each of the hydraulic components such as pumps, valves, metering unit of a fuel system, using physics-based models, which potentially offer more accurate results compared with those using transfer functions. A transient performance simulation system has been set up for gas turbine engines based on an inter-component volume (ICV). A proportional- integral (PI) control strategy is used for the simulation of engine control systems. An integrated engine and its control and hydraulic fuel systems has been set up to investigate their coupling effect during engine transient processes. The developed simulation methods and the systems have been applied to a model turbojet and a model turboshaft gas turbine engine to demonstrate the effectiveness of both two methods. The comparison between the results of engines with and without the ICV method simulated fuel system models shows that the delay of the engine transient response due to the inclusion of the fuel system components and introduced inter-component volumes is noticeable, although relatively small. The comparison of two developed methods applied to engine fuel system simulation demonstrate that both methods introduce delay effect to the engine transient response but the NR method is ahead than the ICV method due to the omission of inter-component volumes on engine fuel system simulation. The developed simulation methods are generic and can be applied to the performance simulation of any other gas turbines and their control and fuel systems. A sensitivity analysis of fuel system key parameters that may affect the engine transient behaviours has also been achieved and represented in this thesis. Three sets of fuel system key parameters have been introduced to investigate their sensitivities, which are, the volumes introduced for ICV method applied to fuel system simulation; the time constants introduced into those first order lags tosimulate the valve movements delay and fuel spray delay effect; and the fuel system key performance and structural parameters.

In the presented thesis work, meshfree method with distance fields was extended to obtain solution of nonlinear transient heat transfer problems. The thesis work involved development and implementation of numerical algorithms, data structure, and software. Numerical and computational properties of the meshfree method with distance fields were investigated. Convergence and accuracy of the methodology was validated by analytical solutions, and solutions produced by commercial FEM software (ANSYS 12.1). The research was focused on nonlinearities caused by temperature-dependent thermal conductivity. The behavior of the developed numerical algorithms was observed for both weak and strong temperature-dependency of thermal conductivity. Oseen and Newton-Kantorovich linearization techniques were applied to linearized the governing equation and boundary conditions. Results of the numerical experiments showed that the meshfree method with distance fields has the potential to produced fast accurate solutions. The method enables all prescribed boundary conditions to be satisfied exactly.

Machine learning is a major source of interesting optimization problems of current interest. These problems tend to be challenging because of their enormous scale, which makes it difficult to apply traditional optimization algorithms. We explore three avenues to designing algorithms suited to handling these challenges, with a view toward large-scale ML tasks. The first is to develop better general methods for unconstrained minimization. The second is to tailor methods to the features of modern systems, namely the availability of distributed computing. The third is to use specialized algorithms to exploit specific problem structure. Chapters 2 and 3 focus on improving quasi-Newton methods, a mainstay of unconstrained optimization. In Chapter 2, we analyze an extension of quasi-Newton methods wherein we use block updates, which add curvature information to the Hessian approximation on a higher-dimensional subspace. This defines a family of methods, Block BFGS, that form a spectrum between the classical BFGS method and Newton's method, in terms of the amount of curvature information used. We show that by adding a correction step, the Block BFGS method inherits the convergence guarantees of BFGS for deterministic problems, most notably a Q-superlinear convergence rate for strongly convex problems. To explore the tradeoff between reduced iterations and greater work per iteration of block methods, we present a set of numerical experiments. In Chapter 3, we focus on the problem of step size determination. To obviate the need for line searches, and for pre-computing fixed step sizes, we derive an analytic step size, which we call curvature-adaptive, for self-concordant functions. This adaptive step size allows us to generalize the damped Newton method of Nesterov to other iterative methods, including gradient descent and quasi-Newton methods. We provide simple proofs of convergence, including superlinear convergence for adaptive BFGS, allowing us to obtain superlinear convergence without line searches. In Chapter 4, we move from general algorithms to hardware-influenced algorithms. We consider a form of distributed stochastic gradient descent that we call Leader SGD, which is inspired by the Elastic Averaging SGD method. These methods are intended for distributed settings where communication between machines may be expensive, making it important to set their consensus mechanism. We show that LSGD avoids an issue with spurious stationary points that affects EASGD, and provide a convergence analysis of LSGD. In the stochastic strongly convex setting, LSGD converges at the rate O(1/k) with diminishing step sizes, matching other distributed methods. We also analyze the impact of varying communication delays, stochasticity in the selection of the leader points, and under what conditions LSGD may produce better search directions than the gradient alone. In Chapter 5, we switch again to focus on algorithms to exploit problem structure. Specifically, we consider problems where variables satisfy multiaffine constraints, which motivates us to apply the Alternating Direction Method of Multipliers (ADMM). Problems that can be formulated with such a structure include representation learning (e.g with dictionaries) and deep learning. We show that ADMM can be applied directly to multiaffine problems. By extending the theory of nonconvex ADMM, we prove that ADMM is convergent on multiaffine problems satisfying certain assumptions, and more broadly, analyze the theoretical properties of ADMM for general problems, investigating the effect of different types of structure.

Includes bibliographical references / In this thesis, the development of a strongly coupled, partitioned fluid-structure interactions (FSI) solver is outlined. Well established methods are analysed and new methods are proposed to provide robust, accurate and efficient FSI solutions. All the methods introduced and analysed are primarily geared towards the solution of incompressible, transient FSI problems, which facilitate the use of black-box sub-domain field solvers. In the first part of the thesis, radial basis function (RBF) interpolation is introduced for interface information transfer. RBF interpolation requires no grid connectivity information, and therefore presents an elegant means by which to transfer information across a non-matching and non-conforming interface to couple finite element to finite volume based discretisation schemes. The transfer scheme is analysed, with particular emphasis on a comparison between consistent and conservative formulations. The primary aim is to demonstrate that the widely used conservative formulation is a zero order method. Furthermore, while the consistent formulation is not provably conservative, it yields errors well within acceptable levels and converges within the limit of mesh refinement. A newly developed multi-vector update quasi-Newton (MVQN) method for implicit coupling of black-box partitioned solvers is proposed. The new coupling scheme, under certain conditions, can be demonstrated to provide near Newton-like convergence behaviour. The superior convergence properties and robust nature of the MVQN method are shown in comparison to other well-known quasi-Newton coupling schemes, including the least squares reduced order modelling (IBQN-LS) scheme, the classical rank-1 update Broyden's method, and fixed point iterations with dynamic relaxation. Partitioned, incompressible FSI, based on Dirichlet-Neumann domain decomposition solution schemes, cannot be applied to problems where the fluid domain is fully enclosed. A simple example often provided in the literature is that of balloon inflation with a prescribed inflow velocity. In this context, artificial compressibility (AC) will be shown to be a useful method to relax the incompressibility constraint, by including a source term within the fluid continuity equation. The attractiveness of AC stems from the fact that this source term can readily be added to almost any fluid field solver, including most commercial solvers. AC/FSI is however limited in the range of problems it can effectively be applied to. To this end, the combination of the newly developed MVQN method with AC/FSI is proposed. In so doing, the AC modified fluid field solver can continue to be treated as a black-box solver, while the overall robustness and performance are significantly improved. The study concludes with a demonstration of the modularity offered by partitioned FSI solvers. The analysis of the coupled environment is extended to include steady state FSI, FSI with free surfaces and an FSI problem with solid-body contact.

Asynchronous iterations often converge under different conditions than their syn- chronous counterparts. In this paper we will study the global convergence of Jacobi- Newton-like methods for nonlinear equationsF x = 0. It is a known fact, that the synchronous algorithm converges monotonically, ifF is a convex M-function and the starting valuesx0 andy0 meet the conditionF x04 04F y0 . In the paper it will be shown, which modifications are necessary to guarantee a similar convergence behavior for an asynchronous computation.

The Multiplicative Schwarz Preconditioned Inexact Newton (MSPIN) algorithm is presented as a complement to Additive Schwarz Preconditioned Inexact Newton (ASPIN). At an algebraic level, ASPIN and MSPIN are variants of the same strategy to improve the convergence of systems with unbalanced nonlinearities; however, they have natural complementarity in practice. MSPIN is naturally based on partitioning of degrees of freedom in a nonlinear PDE system by field type rather than by subdomain, where a modest factor of concurrency can be sacrificed for physically motivated convergence robustness. ASPIN, originally introduced for decompositions into subdomains, is natural for high concurrency and reduction of global synchronization. The ASPIN framework, as an option for the outermost solver, successfully handles strong nonlinearities in computational fluid dynamics, but is barely explored for the highly nonlinear models of complex multiphase flow with capillarity, heterogeneity, and complex geometry. In this dissertation, the fully implicit ASPIN method is demonstrated for a finite volume discretization based on incompressible two-phase reservoir simulators in the presence of capillary forces and gravity. Numerical experiments show that the number of global nonlinear iterations is not only scalable with respect to the number of processors, but also significantly reduced compared with the standard inexact Newton method with a backtracking technique. Moreover, the ASPIN method, in contrast with the IMPES method, saves overall execution time because of the savings in timestep size. We consider the additive and multiplicative types of inexact Newton algorithms in the field-split context, and we augment the classical convergence theory of ASPIN for the multiplicative case. Moreover, we provide the convergence analysis of the MSPIN algorithm. Under suitable assumptions, it is shown that MSPIN is locally convergent, and desired superlinear or even quadratic convergence can be obtained when the forcing terms are picked suitably. Numerical experiments show that MSPIN can be significantly more robust than Newton methods based on global linearizations, and that MSPIN can be more robust than ASPIN, and maintain fast convergence even for challenging problems, such as high-Reynolds number Navier-Stokes equations.

No description available.

Likelihood-based marginalized models using random effects have become popular for analyzing longitudinal categorical data. These models permit direct interpretation of marginal mean parameters and characterize the serial dependence of longitudinal outcomes using random effects [12,22]. In this paper, we propose model that expands the use of previous models to accommodate longitudinal nominal data. Random effects using a new covariance matrix with a Kronecker product composition are used to explain serial and categorical dependence. The Quasi-Newton algorithm is developed for estimation. These proposed methods are illustrated with a real data set and compared with other standard methods.

Syftet med denna studie är att undersöka om det utifrån det musikaliskainnehållet går att förstå och förklara viktiga aspekter som gör en låt till enlistetta. Valet av låtar, som utgör undersökningsmaterialet, baserades på att dessaskall ha legat etta på Billboard Hot-100, och att dessa skulle vara från olikatidsperioder: hela vägen från 80-talet och fram tills 2021. I denna studiebehandlas en tidsram på över 50 år. Från 80-talet valdes Olivia Newton-John -Physical, från 90-talet valdes Whitney Houston - I will always love you, från 00-talet valdes Kanye West- Golddigger, från 10-talet valdes Drake - God’s Planoch slutligen från 20-talet valdes BTS - Butter. En gemensam faktor dessa låtarhar, utöver att ha legat på första platsen på Billboard Hot-100, är att samtliga avlåtarna har legat etta på toppen i över tio veckor och de utvalda låtarna är fem avde endast 40 låtar som någonsin har åstadkommit det. Det finns otaliga analysersom kan utföras för att försöka att räkna ut framgången som låtarna har haft, menanalysen i denna studie fokuserar på de undersökta låtarnas uppbyggnad ochinnehåll. Studiens resultat visar att vissa övergripande mönster kan urskiljas. Entendens som iakttagits är att låtar som blir hits har någon säregen kvalitet somgör att de låter delvis likt tidigare låtar som har släppts men ändå jämförelsevisannorlunda. En annan viktig aspekt är att textinnehållet, för att en låt ska nåframgång och bli en listetta, bör fokusera på ämnet kärlek.