Back to search

Cornell University

STATISTICALLY EFFICIENT REINFORCEMENT LEARNING

Abstract

dc:description.abstract

My research focus is on developing algorithms and statistical theories of sequential decision making on the intersection of reinforcement learning (RL) and causal inference. RL is concerned with the ways agents learn to make sequential decisions in unknown environments. It has been one of the most vibrant research frontiers in machine learning over the last few years. We have empirical success in a variety of applications, especially for games such as AlphaGo (Silver et al., 2016). Despite its popularity, the real-world application of RL in fields such as biomedicine and social science is still limited. This is because these real-world applications do not have good simulators, and experimentation is often expensive and risky (e.g., running clinical trials, deploying new marketing strategies in companies) unlike for games. Although running new experiments can be difficult, fortunately, in an era of big data, we often have access to massive historical datasets such as web-logged data and large electronic health records. This motivated me to find ways to use offline data in a statistically efficient manner, which is a central topic in the subfield of offline RL and causal machine learning. However, there is a certain limitation in offline RL when the quality of the offline data is poor. In this scenario, we want to find the best policy by adaptively collecting data. This motivated me to find ways to collect the data and search for the best policy, which is a central topic in online RL. Since experiments are often costly, it again needs to be performed in a statistically efficient way. Hence, building statistically efficient RL algorithms in both offline and online settings is the key to bringing RL to a variety of real-world applications.

Degree

thesis:*
Name thesis:degree_name
Ph. D., Computer Science
Level thesis:degree_level
Doctor of Philosophy
Discipline thesis:degree_discipline
Computer Science
Grantor
Cornell University
Year dc:date.issued
2023

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Uehara, Masatoshi
Chair dc:contributor.chair
  • Kallus, Nathan
Committee members dc:contributor.committeemember
  • Sun, Wen
  • Joachims, Thorsten

Subjects

dc:subject × 2

Rights

dc:rights
Statement dc:rights
  • Attribution 4.0 International
Language dc:language.iso
en

Identifiers

dc:identifier.*
Dc Identifier Bibid
16219389
Dc Identifier Other
ProQuest Submission ID: 13799
ProQuest Publication ID: 30571678

Chain of custody

source
Harvested from
Cornell University
Base URL
ecommons.cornell.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Uehara, Masatoshi. STATISTICALLY EFFICIENT REINFORCEMENT LEARNING. Doctor of Philosophy thesis, Cornell University, 2023. https://hdl.handle.net/1813/114783