{"id":{"repo_id":"ets-quebec","oai_identifier":"oai:espace.etsmtl.ca:3424"},"canonical_url":"https://search.ndltd.org/etd/ets-quebec/oai:espace.etsmtl.ca:3424","repository":{"repo_id":"ets-quebec","name":"ETS (Quebec)","base_url":"https://espace.etsmtl.ca/cgi/oai2"},"display":{"title":"Evaluation of sample efficiency in offline reinforcement learning","abstract":"Reinforcement learning (RL) has shown great promise with algorithms learning in environments with large state and action spaces purely from scalar reward signals. A crucial challenge for current deep RL algorithms is that they require a tremendous amount of environment interactions for learning. This can be infeasible in situations where such interactions are expensive; such as in robotics. Offline RL algorithms try to address this issue by bootstrapping the learning process from existing logged data without needing to interact with the environment from the very beginning. While online RL algorithms are typically evaluated as a function of the number of environment interactions, there exists no single established protocol for evaluating offline RL methods. In this thesis, we propose a sequential approach to evaluate offline RL algorithms as a function of the training set size and thus by their data efficiency. Sequential evaluation provides valuable insights into the data efficiency of the learning process and the robustness of algorithms to distribution changes in the dataset while also harmonizing the visualization of the offline and online learning phases. Our approach is generally applicable and easy to implement. We compare several existing offline RL algorithms using this approach and present insights from a variety of tasks and offline datasets.","abstract_html":"Reinforcement learning (RL) has shown great promise with algorithms learning in environments with large state and action spaces purely from scalar reward signals. A crucial challenge for current deep RL algorithms is that they require a tremendous amount of environment interactions for learning. This can be infeasible in situations where such interactions are expensive; such as in robotics. Offline RL algorithms try to address this issue by bootstrapping the learning process from existing logged data without needing to interact with the environment from the very beginning. While online RL algorithms are typically evaluated as a function of the number of environment interactions, there exists no single established protocol for evaluating offline RL methods. In this thesis, we propose a sequential approach to evaluate offline RL algorithms as a function of the training set size and thus by their data efficiency. Sequential evaluation provides valuable insights into the data efficiency of the learning process and the robustness of algorithms to distribution changes in the dataset while also harmonizing the visualization of the offline and online learning phases. Our approach is generally applicable and easy to implement. We compare several existing offline RL algorithms using this approach and present insights from a variety of tasks and offline datasets.","abstract_has_math":false,"creators":["Sujit, Shivakanth"],"institution":"École de technologie supérieure","degree_name":"Mémoire de maîtrise électronique","degree_level":"masters","degree_discipline":null,"degree_department":null,"school":null,"contributors":[],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2023,"date_issued":"2023-12-06","date_published":"2023-12-06","updated_at":"2026-11-18T22:10:08Z","subjects":["apprentissage par renforcement hors ligne, méthodes d’évaluation"],"languages":["en"],"rights":[],"rights_urls":[],"identifier_entries":[{"key":"dc:identifier","label":"Identifier","values":["INSERT-YOUR-LIBRARY-ID-HERE3424"],"render_values":[{"text":"INSERT-YOUR-LIBRARY-ID-HERE3424","href":null,"code":true}]}],"person_identifier_entries":[]},"links":{"outbound_url":null,"outbound_label":null,"outbound_source":null,"record_url":null,"record_label":null,"record_kind":null,"record_source":null,"download_url":"https://espace.etsmtl.ca/id/eprint/3424/1/SUJIT_Shivakanth.pdf","download_label":"Download PDF","download_source":"dc:identifier","access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://espace.etsmtl.ca/cgi/oai2?verb=GetRecord&metadataPrefix=oai_etdms&identifier=oai%3Aespace.etsmtl.ca%3A3424","prefix":"oai_etdms"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:creator","label":"Author","values":["Sujit, Shivakanth"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2023-12-06"]},{"key":"dc:publisher","label":"Institution","values":["École de technologie supérieure"]},{"key":"dc:type","label":"Dc Type","values":["Electronic Thesis or Dissertation"]},{"key":"thesis:degree_level","label":"Degree Level","values":["masters"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Mémoire de maîtrise électronique"]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["École de technologie supérieure"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["apprentissage par renforcement hors ligne, méthodes d’évaluation"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["INSERT-YOUR-LIBRARY-ID-HERE3424","https://espace.etsmtl.ca/id/eprint/3424/1/SUJIT_Shivakanth.pdf"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Reinforcement learning (RL) has shown great promise with algorithms learning in environments with large state and action spaces purely from scalar reward signals. A crucial challenge for current deep RL algorithms is that they require a tremendous amount of environment interactions for learning. This can be infeasible in situations where such interactions are expensive; such as in robotics. Offline RL algorithms try to address this issue by bootstrapping the learning process from existing logged data without needing to interact with the environment from the very beginning. While online RL algorithms are typically evaluated as a function of the number of environment interactions, there exists no single established protocol for evaluating offline RL methods. In this thesis, we propose a sequential approach to evaluate offline RL algorithms as a function of the training set size and thus by their data efficiency. Sequential evaluation provides valuable insights into the data efficiency of the learning process and the robustness of algorithms to distribution changes in the dataset while also harmonizing the visualization of the offline and online learning phases. Our approach is generally applicable and easy to implement. We compare several existing offline RL algorithms using this approach and present insights from a variety of tasks and offline datasets."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Evaluation of sample efficiency in offline reinforcement learning"]}]}],"canonical_facts":{"dc:creator":["Sujit, Shivakanth"],"dc:date":["2023-12-06"],"dc:description":["Reinforcement learning (RL) has shown great promise with algorithms learning in environments with large state and action spaces purely from scalar reward signals. A crucial challenge for current deep RL algorithms is that they require a tremendous amount of environment interactions for learning. This can be infeasible in situations where such interactions are expensive; such as in robotics. Offline RL algorithms try to address this issue by bootstrapping the learning process from existing logged data without needing to interact with the environment from the very beginning. While online RL algorithms are typically evaluated as a function of the number of environment interactions, there exists no single established protocol for evaluating offline RL methods. In this thesis, we propose a sequential approach to evaluate offline RL algorithms as a function of the training set size and thus by their data efficiency. Sequential evaluation provides valuable insights into the data efficiency of the learning process and the robustness of algorithms to distribution changes in the dataset while also harmonizing the visualization of the offline and online learning phases. Our approach is generally applicable and easy to implement. We compare several existing offline RL algorithms using this approach and present insights from a variety of tasks and offline datasets."],"dc:format":["application/pdf"],"dc:identifier":["INSERT-YOUR-LIBRARY-ID-HERE3424","https://espace.etsmtl.ca/id/eprint/3424/1/SUJIT_Shivakanth.pdf"],"dc:language":["en"],"dc:publisher":["École de technologie supérieure"],"dc:subject":["apprentissage par renforcement hors ligne, méthodes d’évaluation"],"dc:title":["Evaluation of sample efficiency in offline reinforcement learning"],"dc:type":["Electronic Thesis or Dissertation"],"thesis:degree_level":["masters"],"thesis:degree_name":["Mémoire de maîtrise électronique"],"thesis:institution_name":["École de technologie supérieure"]},"updated_at":"2026-11-18T22:10:08Z"}