Back to search

Monterey, CA; Naval Postgraduate School

ONLINE LEARNING OF MARKOVIAN SYSTEMS WITH CENSORED POISSON ARRIVALS

Abstract

dc:description.abstract

This thesis deals with online optimization of discrete performance measures in Markovian models with incomplete information. We consider a setting where a physical realization of the model is sequentially obtained over a number of periods. The information gathered to date is used in order to efficiently run the model in future days. The information is incomplete in two ways: (i) model parameters are initially unknown (the demand rates in our case), but can be estimated from the physical realizations; and (ii), the demands are censored when the system is in some boundary states. The method of Sample Average Approximation is used to solve the optimization problem. More precisely, in each period, sample paths are generated from the distributions estimated to date, and the best model configuration is determined with respect to these sample paths. Sequential observation of the system’s behavior allows for information to be gathered and a more informed decision to be made in each future round. The method developed in this thesis can be applied in a variety of contexts, where no information is known about the system beforehand, but can be observed at least partially in a sequential manner, such as assigning assets for surveillance of remote geographical regions for illicit activity. The motivating setting of this work is the operation of a bike-sharing system with fixed capacity stations, where an initial number of bikes must be set each day to minimize unsatisfied customers.

Degree

thesis:*
Name thesis:degree_name
Master of Science in Operations Research
Level thesis:degree_level
Masters
Discipline thesis:degree_discipline
Operations Research
Department dc:contributor.department
Operations Research (OR)
Grantor dc:publisher
Monterey, CA; Naval Postgraduate School
Year dc:date.issued
2021

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Gibbons Mac-Lean, Cedric G.
Advisors dc:contributor.advisor
  • Szechtman, Roberto
  • Grant , James , Lancaster University
Committee member dc:contributor.committeemember
  • Salmeron-Medrano, Javier

Rights

dc:rights
Statement dc:rights
  • Copyright is reserved by the copyright owner.
  • Approved for public release. Distribution is unlimited.

Identifiers

dc:identifier.*
Dc Identifier Programcode
360, Operations Analysis
Thesis Thesis Identifier
34987

Chain of custody

source
Harvested from
Naval Postgraduate School
Base URL
calhoun.nps.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Gibbons Mac-Lean, Cedric G.. ONLINE LEARNING OF MARKOVIAN SYSTEMS WITH CENSORED POISSON ARRIVALS. Masters thesis, Monterey, CA; Naval Postgraduate School, 2021. https://hdl.handle.net/10945/67720