{"id":{"repo_id":"mcmaster","oai_identifier":"oai:macsphere.mcmaster.ca:11375/30425"},"canonical_url":"https://search.ndltd.org/etd/mcmaster/oai:macsphere.mcmaster.ca:11375/30425","repository":{"repo_id":"mcmaster","name":"McMaster University","base_url":"https://macsphere.mcmaster.ca/server/oai/request"},"display":{"title":"Online EM for Exponential Family Distributions","abstract":"The expectation-maximization (EM) algorithm is a widely utilized method for fitting probabilistic models with missing data or latent variables. However, applying the EM algorithm to high-dimensional datasets requires storing the entire dataset in computer memory, which is necessary at each iteration. Fortunately, online learning algorithms are available that overcome this issue by processing data sequentially and updating model parameters at each step. In this thesis, we introduce an approach for online training of mixture models of exponential family distributions through modifications to the EM algorithm. The first part of our contribution is the learning of Gaussian mixture models (GMMs) using alternating mirror descent updates. This method employs custom Bregman divergences for parameter updates and allows for the processing of one unit of data at each iteration. Our second contribution is an improved version of the EM algorithm for exponential family distributions. In this case, we replace the M-step updates with mirror descent updates and further enhance them by introducing a new Bregman divergence for the updates. Finally, we apply the improved EM algorithm to online principal component analysis (PCA) for exponential families, where the number of principal components is adaptively learned by pruning factors with low correlations to the observations. We evaluate the performance of our proposed approaches by comparing them with both simulation and real datasets.","abstract_html":"The expectation-maximization (EM) algorithm is a widely utilized method for fitting probabilistic models with missing data or latent variables. However, applying the EM algorithm to high-dimensional datasets requires storing the entire dataset in computer memory, which is necessary at each iteration. Fortunately, online learning algorithms are available that overcome this issue by processing data sequentially and updating model parameters at each step. In this thesis, we introduce an approach for online training of mixture models of exponential family distributions through modifications to the EM algorithm. The first part of our contribution is the learning of Gaussian mixture models (GMMs) using alternating mirror descent updates. This method employs custom Bregman divergences for parameter updates and allows for the processing of one unit of data at each iteration. Our second contribution is an improved version of the EM algorithm for exponential family distributions. In this case, we replace the M-step updates with mirror descent updates and further enhance them by introducing a new Bregman divergence for the updates. Finally, we apply the improved EM algorithm to online principal component analysis (PCA) for exponential families, where the number of principal components is adaptively learned by pruning factors with low correlations to the observations. We evaluate the performance of our proposed approaches by comparing them with both simulation and real datasets.","abstract_has_math":false,"creators":["Rahimipour, Shiva"],"institution":null,"degree_name":null,"degree_level":null,"degree_discipline":null,"degree_department":"Computational Engineering and Science","school":null,"contributors":[],"advisors":["McNicholas, Paul D."],"committee_chairs":[],"committee_members":[],"year":2024,"date_issued":"2024","date_published":"2024","updated_at":"2026-11-18T19:37:20Z","subjects":["Online learning","Gaussian mixture models","Exponential Family Distributions","Mirror descent algorithm","Principle Component Analysis"],"languages":["en_US"],"rights":[],"rights_urls":[],"identifier_entries":[{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://doi.org/10.71548/18340"],"render_values":[{"text":"https://doi.org/10.71548/18340","href":"https://doi.org/10.71548/18340","code":true}]}],"person_identifier_entries":[]},"links":{"outbound_url":"http://hdl.handle.net/11375/30425","outbound_label":"Handle","outbound_source":"dc:identifier.uri","record_url":"http://hdl.handle.net/11375/30425","record_label":"Repository record","record_kind":"handle","record_source":"dc:identifier.uri","download_url":null,"download_label":null,"download_source":null,"access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://macsphere.mcmaster.ca/server/oai/request?verb=GetRecord&metadataPrefix=dim&identifier=oai%3Amacsphere.mcmaster.ca%3A11375%2F30425","prefix":"dim"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.advisor","label":"Advisor","values":["McNicholas, Paul D."]},{"key":"dc:contributor.department","label":"Department","values":["Computational Engineering and Science"]},{"key":"dc:creator","label":"Author","values":["Rahimipour, Shiva"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.accessioned","label":"Dc Date Accessioned","values":["2024-10-15T12:41:00Z"]},{"key":"dc:date.available","label":"Dc Date Available","values":["2024-10-15T12:41:00Z"]},{"key":"dc:date.issued","label":"Date","values":["2024"]},{"key":"dc:type","label":"Dc Type","values":["Thesis"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Online learning","Gaussian mixture models","Exponential Family Distributions","Mirror descent algorithm","Principle Component Analysis"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language.iso","label":"Language (ISO)","values":["en_US"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.uri","label":"Identifier URI","values":["http://hdl.handle.net/11375/30425","https://doi.org/10.71548/18340"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["The expectation-maximization (EM) algorithm is a widely utilized method for fitting probabilistic models with missing data or latent variables. However, applying the EM algorithm to high-dimensional datasets requires storing the entire dataset in computer memory, which is necessary at each iteration. Fortunately, online learning algorithms are available that overcome this issue by processing data sequentially and updating model parameters at each step. In this thesis, we introduce an approach for online training of mixture models of exponential family distributions through modifications to the EM algorithm. The first part of our contribution is the learning of Gaussian mixture models (GMMs) using alternating mirror descent updates. This method employs custom Bregman divergences for parameter updates and allows for the processing of one unit of data at each iteration. Our second contribution is an improved version of the EM algorithm for exponential family distributions. In this case, we replace the M-step updates with mirror descent updates and further enhance them by introducing a new Bregman divergence for the updates. Finally, we apply the improved EM algorithm to online principal component analysis (PCA) for exponential families, where the number of principal components is adaptively learned by pruning factors with low correlations to the observations. We evaluate the performance of our proposed approaches by comparing them with both simulation and real datasets."]},{"key":"dc:description.degree","label":"Dc Description Degree","values":["Candidate in Philosophy"]},{"key":"dc:title","label":"Title","values":["Online EM for Exponential Family Distributions"]},{"key":"others:access_status","label":"Others Access Status","values":["open.access"]}]}],"canonical_facts":{"dc:contributor.advisor":["McNicholas, Paul D."],"dc:contributor.department":["Computational Engineering and Science"],"dc:creator":["Rahimipour, Shiva"],"dc:date.accessioned":["2024-10-15T12:41:00Z"],"dc:date.available":["2024-10-15T12:41:00Z"],"dc:date.issued":["2024"],"dc:description.abstract":["The expectation-maximization (EM) algorithm is a widely utilized method for fitting probabilistic models with missing data or latent variables. However, applying the EM algorithm to high-dimensional datasets requires storing the entire dataset in computer memory, which is necessary at each iteration. Fortunately, online learning algorithms are available that overcome this issue by processing data sequentially and updating model parameters at each step. In this thesis, we introduce an approach for online training of mixture models of exponential family distributions through modifications to the EM algorithm. The first part of our contribution is the learning of Gaussian mixture models (GMMs) using alternating mirror descent updates. This method employs custom Bregman divergences for parameter updates and allows for the processing of one unit of data at each iteration. Our second contribution is an improved version of the EM algorithm for exponential family distributions. In this case, we replace the M-step updates with mirror descent updates and further enhance them by introducing a new Bregman divergence for the updates. Finally, we apply the improved EM algorithm to online principal component analysis (PCA) for exponential families, where the number of principal components is adaptively learned by pruning factors with low correlations to the observations. We evaluate the performance of our proposed approaches by comparing them with both simulation and real datasets."],"dc:description.degree":["Candidate in Philosophy"],"dc:identifier.uri":["http://hdl.handle.net/11375/30425","https://doi.org/10.71548/18340"],"dc:language.iso":["en_US"],"dc:subject":["Online learning","Gaussian mixture models","Exponential Family Distributions","Mirror descent algorithm","Principle Component Analysis"],"dc:title":["Online EM for Exponential Family Distributions"],"dc:type":["Thesis"],"others:access_status":["open.access"]},"updated_at":"2026-11-18T19:37:20Z"}