{"id":{"repo_id":"cornell","oai_identifier":"oai:ecommons.cornell.edu:1813/110451"},"canonical_url":"https://search.ndltd.org/etd/cornell/oai:ecommons.cornell.edu:1813/110451","repository":{"repo_id":"cornell","name":"Cornell University","base_url":"https://ecommons.cornell.edu/server/oai/request"},"display":{"title":"ONLINE LEARNING OF MARKOV JUMP DYNAMIC SYSTEMS","abstract":"We study the online learning of Markovian Jump Linear Dynamical Systems. These are linear dynamical systems having several Markov modes with each mode represented by a pair of state dynamics matrices. The system’s mode switches with time according to an underlying Markov Chain. We focus on the problem of LQR control for such Markovian Jump Linear Dynamical Systems where the matrices governing the state transitions and the Markov Chain’s state transition probabilities are unknown but the number of modes of the system is known. In this setting, we propose an online learning algorithm which is expected to suffer a sublinear (on the time horizon length) regret when compared against the optimal control of the system in expectation. Where, in the optimal control setting, the system's parameters and the Markov Chain are known. We perform numerical simulations of this proposed algorithm learning a two-dimensional state space and control space with two modes and a simple Markov Chain between the two modes. Then we compare this algorithm against the known optimal solution of our system and expect to observe a sublinear regret.","abstract_html":"We study the online learning of Markovian Jump Linear Dynamical Systems. These are linear dynamical systems having several Markov modes with each mode represented by a pair of state dynamics matrices. The system’s mode switches with time according to an underlying Markov Chain. We focus on the problem of LQR control for such Markovian Jump Linear Dynamical Systems where the matrices governing the state transitions and the Markov Chain’s state transition probabilities are unknown but the number of modes of the system is known. In this setting, we propose an online learning algorithm which is expected to suffer a sublinear (on the time horizon length) regret when compared against the optimal control of the system in expectation. Where, in the optimal control setting, the system&#x27;s parameters and the Markov Chain are known. We perform numerical simulations of this proposed algorithm learning a two-dimensional state space and control space with two modes and a simple Markov Chain between the two modes. Then we compare this algorithm against the known optimal solution of our system and expect to observe a sublinear regret.","abstract_has_math":false,"creators":["Ray, Ritabrata"],"institution":"Cornell University","degree_name":"M.S., Electrical and Computer Engineering","degree_level":"Master of Science","degree_discipline":"Electrical and Computer Engineering","degree_department":null,"school":null,"contributors":[],"advisors":[],"committee_chairs":["Tong, Lang"],"committee_members":["Zhao, Qing"],"year":2021,"date_issued":"2021-08","date_published":"2021-08","updated_at":"2026-11-19T03:08:55Z","subjects":["LQR control","Online Learning","Optimal Control","Regret analysis","Reinforcement Learning","Upper Confidence Bound"],"languages":["en"],"rights":["Attribution-NonCommercial 4.0 International"],"rights_urls":["https://creativecommons.org/licenses/by-nc/4.0/"],"identifier_entries":[{"key":"dc:identifier.doi","label":"DOI","values":["https://doi.org/10.7298/b0s8-jb09"],"render_values":[{"text":"https://doi.org/10.7298/b0s8-jb09","href":"https://doi.org/10.7298/b0s8-jb09","code":true}]},{"key":"dc:identifier.bibid","label":"Dc Identifier Bibid","values":["15160130"],"render_values":[{"text":"15160130","href":null,"code":true}]},{"key":"dc:identifier.other","label":"Dc Identifier Other","values":["ProQuest Submission ID: 11299","ProQuest Publication ID: 28713620"],"render_values":[{"text":"ProQuest Submission ID: 11299","href":null,"code":true},{"text":"ProQuest Publication ID: 28713620","href":null,"code":true}]}],"person_identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/1813/110451","outbound_label":"Handle","outbound_source":"dc:identifier.uri","record_url":"https://hdl.handle.net/1813/110451","record_label":"Repository record","record_kind":"handle","record_source":"dc:identifier.uri","download_url":null,"download_label":null,"download_source":null,"access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://ecommons.cornell.edu/server/oai/request?verb=GetRecord&metadataPrefix=dim&identifier=oai%3Aecommons.cornell.edu%3A1813%2F110451","prefix":"dim"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.chair","label":"Chair","values":["Tong, Lang"]},{"key":"dc:contributor.committeemember","label":"Committee Member","values":["Zhao, Qing"]},{"key":"dc:creator","label":"Author","values":["Ray, Ritabrata"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.accessioned","label":"Dc Date Accessioned","values":["2021-12-20T20:34:36Z"]},{"key":"dc:date.available","label":"Dc Date Available","values":["2021-12-20T20:34:36Z"]},{"key":"dc:date.issued","label":"Date","values":["2021-08"]},{"key":"dc:relation.uri","label":"Dc Relation URI","values":["https://newcatalog.library.cornell.edu/catalog/15160130"]},{"key":"dc:type","label":"Dc Type","values":["dissertation or thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Electrical and Computer Engineering"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Master of Science"]},{"key":"thesis:degree_name","label":"Degree Name","values":["M.S., Electrical and Computer Engineering"]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["Cornell University"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["LQR control","Online Learning","Optimal Control","Regret analysis","Reinforcement Learning","Upper Confidence Bound"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language.iso","label":"Language (ISO)","values":["en"]},{"key":"dc:rights","label":"Dc Rights","values":["Attribution-NonCommercial 4.0 International"]},{"key":"dc:rights.uri","label":"Rights URI","values":["https://creativecommons.org/licenses/by-nc/4.0/"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.bibid","label":"Dc Identifier Bibid","values":["15160130"]},{"key":"dc:identifier.doi","label":"DOI","values":["https://doi.org/10.7298/b0s8-jb09"]},{"key":"dc:identifier.other","label":"Dc Identifier Other","values":["ProQuest Submission ID: 11299","ProQuest Publication ID: 28713620"]},{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://hdl.handle.net/1813/110451"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"cerif:openaire","label":"Cerif Openaire","values":["<Publication xmlns=\"https://www.openaire.eu/cerif-profile/1.1/\" id=\"91704db5-fa7f-4db7-be47-3198bd163aa3\"> <Type xmlns=\"https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types\">http://purl.org/coar/resource_type/c_1843</Type> <Language>en</Language> <Title>ONLINE LEARNING OF MARKOV JUMP DYNAMIC SYSTEMS</Title> <PublishedIn> <Publication> </Publication> </PublishedIn> <PublicationDate>2021-08</PublicationDate> <DOI>https://doi.org/10.7298/b0s8-jb09</DOI> <Authors> <Author> <DisplayName>Ray, Ritabrata</DisplayName> <Affiliation> <OrgUnit> </OrgUnit> </Affiliation> </Author> </Authors> <Editors> </Editors> <Publishers> <Publisher> <OrgUnit /> </Publisher> </Publishers> <License>https://creativecommons.org/licenses/by-nc/4.0/</License> <Keyword>LQR control</Keyword> <Keyword>Online Learning</Keyword> <Keyword>Optimal Control</Keyword> <Keyword>Regret analysis</Keyword> <Keyword>Reinforcement Learning</Keyword> <Keyword>Upper Confidence Bound</Keyword> <Abstract>We study the online learning of Markovian Jump Linear Dynamical Systems. These are linear dynamical systems having several Markov modes with each mode represented by a pair of state dynamics matrices. The system’s mode switches with time according to an underlying Markov Chain. We focus on the problem of LQR control for such Markovian Jump Linear Dynamical Systems where the matrices governing the state transitions and the Markov Chain’s state transition probabilities are unknown but the number of modes of the system is known. In this setting, we propose an online learning algorithm which is expected to suffer a sublinear (on the time horizon length) regret when compared against the optimal control of the system in expectation. Where, in the optimal control setting, the system's parameters and the Markov Chain are known. We perform numerical simulations of this proposed algorithm learning a two-dimensional state space and control space with two modes and a simple Markov Chain between the two modes. Then we compare this algorithm against the known optimal solution of our system and expect to observe a sublinear regret.</Abstract> <Access xmlns=\"http://purl.org/coar/access_right\" > </Access> </Publication>"]},{"key":"cris:virtual.author","label":"Cris Virtual Author","values":["Ray, Ritabrata"]},{"key":"cris:virtualsource.collection","label":"Cris Virtualsource Collection","values":["5893a6ea-7af3-41d7-abc6-04bcd26ab5df"]},{"key":"dc:description","label":"Description","values":["37 pages"]},{"key":"dc:description.abstract","label":"Abstract","values":["We study the online learning of Markovian Jump Linear Dynamical Systems. These are linear dynamical systems having several Markov modes with each mode represented by a pair of state dynamics matrices. The system’s mode switches with time according to an underlying Markov Chain. We focus on the problem of LQR control for such Markovian Jump Linear Dynamical Systems where the matrices governing the state transitions and the Markov Chain’s state transition probabilities are unknown but the number of modes of the system is known. In this setting, we propose an online learning algorithm which is expected to suffer a sublinear (on the time horizon length) regret when compared against the optimal control of the system in expectation. Where, in the optimal control setting, the system's parameters and the Markov Chain are known. We perform numerical simulations of this proposed algorithm learning a two-dimensional state space and control space with two modes and a simple Markov Chain between the two modes. Then we compare this algorithm against the known optimal solution of our system and expect to observe a sublinear regret."]},{"key":"dc:description.collection","label":"Dc Description Collection","values":["Cornell Theses and Dissertations"]},{"key":"dc:format.mimetype","label":"Dc Format Mimetype","values":["application/pdf"]},{"key":"dcterms:license","label":"Dcterms License","values":["https://hdl.handle.net/1813/59810"]},{"key":"dc:title","label":"Title","values":["ONLINE LEARNING OF MARKOV JUMP DYNAMIC SYSTEMS"]},{"key":"dspace:entity.type","label":"Dspace Entity Type","values":["Publication"]},{"key":"others:access_status","label":"Others Access Status","values":["open.access"]}]}],"canonical_facts":{"cerif:openaire":["<Publication xmlns=\"https://www.openaire.eu/cerif-profile/1.1/\" id=\"91704db5-fa7f-4db7-be47-3198bd163aa3\"> <Type xmlns=\"https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types\">http://purl.org/coar/resource_type/c_1843</Type> <Language>en</Language> <Title>ONLINE LEARNING OF MARKOV JUMP DYNAMIC SYSTEMS</Title> <PublishedIn> <Publication> </Publication> </PublishedIn> <PublicationDate>2021-08</PublicationDate> <DOI>https://doi.org/10.7298/b0s8-jb09</DOI> <Authors> <Author> <DisplayName>Ray, Ritabrata</DisplayName> <Affiliation> <OrgUnit> </OrgUnit> </Affiliation> </Author> </Authors> <Editors> </Editors> <Publishers> <Publisher> <OrgUnit /> </Publisher> </Publishers> <License>https://creativecommons.org/licenses/by-nc/4.0/</License> <Keyword>LQR control</Keyword> <Keyword>Online Learning</Keyword> <Keyword>Optimal Control</Keyword> <Keyword>Regret analysis</Keyword> <Keyword>Reinforcement Learning</Keyword> <Keyword>Upper Confidence Bound</Keyword> <Abstract>We study the online learning of Markovian Jump Linear Dynamical Systems. These are linear dynamical systems having several Markov modes with each mode represented by a pair of state dynamics matrices. The system’s mode switches with time according to an underlying Markov Chain. We focus on the problem of LQR control for such Markovian Jump Linear Dynamical Systems where the matrices governing the state transitions and the Markov Chain’s state transition probabilities are unknown but the number of modes of the system is known. In this setting, we propose an online learning algorithm which is expected to suffer a sublinear (on the time horizon length) regret when compared against the optimal control of the system in expectation. Where, in the optimal control setting, the system's parameters and the Markov Chain are known. We perform numerical simulations of this proposed algorithm learning a two-dimensional state space and control space with two modes and a simple Markov Chain between the two modes. Then we compare this algorithm against the known optimal solution of our system and expect to observe a sublinear regret.</Abstract> <Access xmlns=\"http://purl.org/coar/access_right\" > </Access> </Publication>"],"cris:virtual.author":["Ray, Ritabrata"],"cris:virtualsource.collection":["5893a6ea-7af3-41d7-abc6-04bcd26ab5df"],"dc:contributor.chair":["Tong, Lang"],"dc:contributor.committeemember":["Zhao, Qing"],"dc:creator":["Ray, Ritabrata"],"dc:date.accessioned":["2021-12-20T20:34:36Z"],"dc:date.available":["2021-12-20T20:34:36Z"],"dc:date.issued":["2021-08"],"dc:description":["37 pages"],"dc:description.abstract":["We study the online learning of Markovian Jump Linear Dynamical Systems. These are linear dynamical systems having several Markov modes with each mode represented by a pair of state dynamics matrices. The system’s mode switches with time according to an underlying Markov Chain. We focus on the problem of LQR control for such Markovian Jump Linear Dynamical Systems where the matrices governing the state transitions and the Markov Chain’s state transition probabilities are unknown but the number of modes of the system is known. In this setting, we propose an online learning algorithm which is expected to suffer a sublinear (on the time horizon length) regret when compared against the optimal control of the system in expectation. Where, in the optimal control setting, the system's parameters and the Markov Chain are known. We perform numerical simulations of this proposed algorithm learning a two-dimensional state space and control space with two modes and a simple Markov Chain between the two modes. Then we compare this algorithm against the known optimal solution of our system and expect to observe a sublinear regret."],"dc:description.collection":["Cornell Theses and Dissertations"],"dc:format.mimetype":["application/pdf"],"dc:identifier.bibid":["15160130"],"dc:identifier.doi":["https://doi.org/10.7298/b0s8-jb09"],"dc:identifier.other":["ProQuest Submission ID: 11299","ProQuest Publication ID: 28713620"],"dc:identifier.uri":["https://hdl.handle.net/1813/110451"],"dc:language.iso":["en"],"dc:relation.uri":["https://newcatalog.library.cornell.edu/catalog/15160130"],"dc:rights":["Attribution-NonCommercial 4.0 International"],"dc:rights.uri":["https://creativecommons.org/licenses/by-nc/4.0/"],"dc:subject":["LQR control","Online Learning","Optimal Control","Regret analysis","Reinforcement Learning","Upper Confidence Bound"],"dcterms:license":["https://hdl.handle.net/1813/59810"],"dc:title":["ONLINE LEARNING OF MARKOV JUMP DYNAMIC SYSTEMS"],"dc:type":["dissertation or thesis"],"dspace:entity.type":["Publication"],"others:access_status":["open.access"],"thesis:degree_discipline":["Electrical and Computer Engineering"],"thesis:degree_level":["Master of Science"],"thesis:degree_name":["M.S., Electrical and Computer Engineering"],"thesis:institution_name":["Cornell University"]},"updated_at":"2026-11-19T03:08:55Z"}