Over the years analysts have used the EM algorithm to obtain maximum likelihood estimates from incomplete data for various models. The general algorithm admits several appealing properties such as strong global convergence; however, the rate of convergence is linear which in some cases may be unacceptably slow. This work is primarily concerned with applying Anderson acceleration to the EM algorithm for Gaussian mixture models (GMM) in hopes of alleviating slow convergence. As preamble we provide a review of maximum likelihood estimation and derive the EM algorithm in detail. The iterates that correspond to the GMM are then formulated and examples are provided. These examples show how faster convergence is experienced when the data are well separated, whereas much slower convergence is seen whenever the sample is poorly separated. The Anderson acceleration method is then presented, and its connection to the EM algorithm is discussed. The work is then concluded by applying Anderson acceleration to the EM algorithm which results in reducing the number of iterations required to obtain convergence.

This thesis aims at developing efficient algorithms for solving some fundamental engineering problems in data science and machine learning. We investigate a variety of acceleration techniques for improving the convergence times of optimization algorithms.  First, we investigate how problem structure can be exploited to accelerate the solution of highly structured problems such as generalized eigenvalue and elastic net regression. We then consider Anderson acceleration, a generic and parameter-free extrapolation scheme, and show how it can be adapted to accelerate practical convergence of proximal gradient methods for a broad class of non-smooth problems. For all the methods developed in this thesis, we design novel algorithms, perform mathematical analysis of convergence rates, and conduct practical experiments on real-world data sets. / <p>QC 20191105</p>

L’eau des circuits d’une centrale nucléaire est purifiée à l’aide de résines échangeuses d’ions. La prédiction de leurs performances constitue une aide importante pour l’exploitation de ces réacteurs. Les méthodes du transport réactif sont particulièrement adaptées pour cela et constituent la base du code OPTIPUR, dédié à la modélisation de ces résines. Le travail présenté comporte trois axes principaux. Le premier est l’intégration d’une limitation au transfert de masse dans une colonne de résines échangeuses d’ions, avec une mobilité spécifique à chaque espèce chimique, dans le cadre d’un découplage des calculs de chimie et de transport. Ce modèle permet, sans paramètre ajustable, de reproduire assez fidèlement une série d’expériences réalisées précédemment par le CEA. Le second axe concerne les aspects numériques du transport réactif, avec l’utilisation de la méthode d’Anderson pour accélérer la convergence du couplage chimie-transport dans un schéma itératif. En utilisant les informations issues des itérations précédentes et sans modification majeure du code, la robustesse et les temps de calcul ont pu être nettement améliorés. La troisième thématique abordée est celle de l’équilibre d’échange d’ions. Les bases d’un modèle prenant en considération l’évolution de l’humidité de la résine, ainsi que son élasticité sont proposées ; les interactions entre groupes fonctionnels, contre-ions et eau sont considérées comme des équilibres chimiques. Les constantes d’équilibre sont ajustées à partir de mesures de la teneur en eau de la résine à différentes pressions de vapeur d’eau. Finalement, des coefficients de sélectivité apparents peuvent être calculés et comparés aux mesures disponibles. / In nuclear power plants, the water contained in different circuits is purified by passing through ion exchange resins. Prediction of the performance of these resins is an important help to the plant operators. To this end, the method of reactive transport modeling are well suited and is the basis of the OPTIPUR code that was designed to model the resins. The work presented in this manuscript covers three main aspects. The first one is the integration of a limitation to mass transfer in a ion exchange deep bed, taking into account a specific mobility for each chemical species, in the context of separated calculations for chemistry and transport. This model was shown to reproduce experimental data, without adjustable parameters. The second part of this work deals with the numerical aspects of reactive transport modelling. A method developped by Anderson was used to accelerate the convergence of the chemistry transport coupling in an iterative scheme. Using the information from previous iterations, and without major changes in the code, calculation times were largely decreased, as well as the number of calculations failures. The third topic is ion exchange equilibrium. The basis of a model that takes into account the change in the water content of the resin and its elasticity are described. The interactions between the fonctional groups, the counterions and water are considered as chemical reactions. The corresponding equilibrium constants are fitted to measurements of the water content of the resin at different relative humidity. Finaly, the selectivity coefficients can be calculated and compared to litterature values.