Back to search

Cornell University

Off-policy Evaluation and Learning for Interactive Systems

Abstract

dc:description.abstract

Recent advances in reinforcement learning (RL) provide exciting potential for making agents learn, plan and act effectively in uncertain environments. Most existing algorithms in RL rely on known environments or the existence of a good simulator, where it is cheap to explore and collect the training data. However, this is not the case for human-centered interactive systems, in which online sampling or experimentation is costly, dangerous, or even illegal. This dissertation advocates an alternative data-driven approach that aims to evaluate and improve the performance of intelligent systems by only using the logged data from prior versions of the system (a.k.a. off-policy evaluation and learning). While such data is collected in large quantity as a byproduct of system operation, reasoning them is difficult since the data is biased and partial in nature. We present our key contributions in off-policy evaluation and learning for the contextual bandit setting, which is a state-less form of RL that is highly relevant to many real-world applications. This includes the discovery of a general family of counterfactual estimators for off-policy evaluation, which subsumes most estimators proposed to date; a principled optimization-based framework for automatically designing estimators, instead of manually constructing them; a data-driven model selection technique in off-policy policy evaluation settings; as well as various approaches for handling support-deficient data in the off-policy learning setting.

Degree

thesis:*
Name thesis:degree_name
Ph. D., Statistics
Level thesis:degree_level
Doctor of Philosophy
Discipline thesis:degree_discipline
Statistics
Grantor
Cornell University
Year dc:date.issued
2021

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Su, Yi
Chair dc:contributor.chair
  • Joachims, Thorsten
Committee members dc:contributor.committeemember
  • Sridharan, Karthik
  • Kallus, Nathan

Rights

dc:rights
Statement dc:rights
  • Attribution 4.0 International
Language dc:language.iso
en

Identifiers

dc:identifier.*
Dc Identifier Bibid
15160034
Dc Identifier Other
ProQuest Submission ID: 12733
ProQuest Publication ID: 28713331

Chain of custody

source
Harvested from
Cornell University
Base URL
ecommons.cornell.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Su, Yi. Off-policy Evaluation and Learning for Interactive Systems. Doctor of Philosophy thesis, Cornell University, 2021. https://hdl.handle.net/1813/110655