Back to search

Cornell University

ONLINE LEARNING OF MARKOV JUMP DYNAMIC SYSTEMS

Abstract

dc:description.abstract

We study the online learning of Markovian Jump Linear Dynamical Systems. These are linear dynamical systems having several Markov modes with each mode represented by a pair of state dynamics matrices. The system’s mode switches with time according to an underlying Markov Chain. We focus on the problem of LQR control for such Markovian Jump Linear Dynamical Systems where the matrices governing the state transitions and the Markov Chain’s state transition probabilities are unknown but the number of modes of the system is known. In this setting, we propose an online learning algorithm which is expected to suffer a sublinear (on the time horizon length) regret when compared against the optimal control of the system in expectation. Where, in the optimal control setting, the system's parameters and the Markov Chain are known. We perform numerical simulations of this proposed algorithm learning a two-dimensional state space and control space with two modes and a simple Markov Chain between the two modes. Then we compare this algorithm against the known optimal solution of our system and expect to observe a sublinear regret.

Degree

thesis:*
Name thesis:degree_name
M.S., Electrical and Computer Engineering
Level thesis:degree_level
Master of Science
Discipline thesis:degree_discipline
Electrical and Computer Engineering
Grantor
Cornell University
Year dc:date.issued
2021

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Ray, Ritabrata
Chair dc:contributor.chair
  • Tong, Lang
Committee member dc:contributor.committeemember
  • Zhao, Qing

Subjects

dc:subject × 6

Rights

dc:rights
Statement dc:rights
  • Attribution-NonCommercial 4.0 International
Language dc:language.iso
en

Identifiers

dc:identifier.*
Dc Identifier Bibid
15160130
Dc Identifier Other
ProQuest Submission ID: 11299
ProQuest Publication ID: 28713620

Chain of custody

source
Harvested from
Cornell University
Base URL
ecommons.cornell.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Ray, Ritabrata. ONLINE LEARNING OF MARKOV JUMP DYNAMIC SYSTEMS. Master of Science thesis, Cornell University, 2021. https://hdl.handle.net/1813/110451