{"id":{"repo_id":"birmingham","oai_identifier":"oai:etheses.bham.ac.uk:12753"},"canonical_url":"https://search.ndltd.org/etd/birmingham/oai:etheses.bham.ac.uk:12753","repository":{"repo_id":"birmingham","name":"University of Birmingham","base_url":"https://etheses.bham.ac.uk/cgi/oai2"},"display":{"title":"Real-time energy management of the hybrid off-highway vehicle using reinforcement learning","abstract":"Rapid developments in transportation and mobility systems including off-highway vehicles have a significant impact on the environment and energy. Electrification of off-highway vehicles makes a significant contribution to decarbonisation with advancements in vehicle technologies. This Ph.D. research focuses on the development of an advanced energy management strategy (EMS) for the hybrid off-highway aircraft-towing tractor, harnessing reinforcement learning (RL), one of the most promising artificial intelligence technologies. Based on real-time modelling of the vehicle system using MATLAB/Simulink, the dedicated RL algorithms are developed to enhance the exploration capability and reduce the overestimation of the reward-action pairs so that they can improve the energy efficiency of the hybrid vehicle in learning cycles and real-world driving. The real-time control functionalities of the proposed energy system are tested and verified on the hardware-in-the-loop platform. By introducing three decay functions to the epsilon-greedy process of the RL-based EMS, the impacts of the exploration-to-exploitation ratio (E2E) on improving vehicle energy efficiency are investigated. A step-based decay function is proposed as a unified decay function incorporated with the vehicle EMS. Its contribution to the improvement of trustworthiness is demonstrated by a comparison with standard Q learning. An improvement rate of up to 6.21% higher than the standard Q-learning is achieved. Based on the performance analysis of the RL-based EMS with different decay functions, information-fused and synchronised ensemble learning frameworks are developed to enhance the optimality and robustness of the RL-based EMS. The random-based, maximum-based and weighted-based methods are studied and compared under the ensemble learning frameworks. The study suggests that the weighted-based method is the best of the three ensemble methods. This research leads to a new synchronised ensemble learning framework founded on the weighted-based method, with up to 1.06% and 5.06% higher vehicle energy efficiency than the maximum-based method and random-based method, respectively. Additionally, the weighted-based method achieves 1.15% higher vehicle energy efficiency than the standard Q-learning. Finally, by observing overestimation phenomena in the RL-based EMS under disturbed learning and undisturbed learning scenarios, a predictive double Q-learning framework with backup models and random action execution policy, respectively is developed and used to increase the efficiency of the learning experience and determine the final action to improve the robustness of the RL-based EMS for the hybrid off-highway vehicles.","abstract_html":"Rapid developments in transportation and mobility systems including off-highway vehicles have a significant impact on the environment and energy. Electrification of off-highway vehicles makes a significant contribution to decarbonisation with advancements in vehicle technologies. This Ph.D. research focuses on the development of an advanced energy management strategy (EMS) for the hybrid off-highway aircraft-towing tractor, harnessing reinforcement learning (RL), one of the most promising artificial intelligence technologies. Based on real-time modelling of the vehicle system using MATLAB/Simulink, the dedicated RL algorithms are developed to enhance the exploration capability and reduce the overestimation of the reward-action pairs so that they can improve the energy efficiency of the hybrid vehicle in learning cycles and real-world driving. The real-time control functionalities of the proposed energy system are tested and verified on the hardware-in-the-loop platform. By introducing three decay functions to the epsilon-greedy process of the RL-based EMS, the impacts of the exploration-to-exploitation ratio (E2E) on improving vehicle energy efficiency are investigated. A step-based decay function is proposed as a unified decay function incorporated with the vehicle EMS. Its contribution to the improvement of trustworthiness is demonstrated by a comparison with standard Q learning. An improvement rate of up to 6.21% higher than the standard Q-learning is achieved. Based on the performance analysis of the RL-based EMS with different decay functions, information-fused and synchronised ensemble learning frameworks are developed to enhance the optimality and robustness of the RL-based EMS. The random-based, maximum-based and weighted-based methods are studied and compared under the ensemble learning frameworks. The study suggests that the weighted-based method is the best of the three ensemble methods. This research leads to a new synchronised ensemble learning framework founded on the weighted-based method, with up to 1.06% and 5.06% higher vehicle energy efficiency than the maximum-based method and random-based method, respectively. Additionally, the weighted-based method achieves 1.15% higher vehicle energy efficiency than the standard Q-learning. Finally, by observing overestimation phenomena in the RL-based EMS under disturbed learning and undisturbed learning scenarios, a predictive double Q-learning framework with backup models and random action execution policy, respectively is developed and used to increase the efficiency of the learning experience and determine the final action to improve the robustness of the RL-based EMS for the hybrid off-highway vehicles.","abstract_has_math":false,"creators":["Shuai, Bin"],"institution":"University of Birmingham","degree_name":"d_ph","degree_level":"d_ph","degree_discipline":null,"degree_department":null,"school":null,"contributors":[],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2022,"date_issued":"2022-07","date_published":"2022-07","updated_at":"2026-11-19T09:31:51Z","subjects":["TJ Mechanical engineering and machinery","TL Motor vehicles. Aeronautics. Astronautics"],"languages":["eng"],"rights":[],"rights_urls":[],"identifier_entries":[{"key":"dc:identifier.grantnumber","label":"Dc Identifier Grantnumber","values":["The development of world-leading off-highway vehicular technologies (Grant 102253)"],"render_values":[{"text":"The development of world-leading off-highway vehicular technologies (Grant 102253)","href":null,"code":true}]}],"person_identifier_entries":[]},"links":{"outbound_url":null,"outbound_label":null,"outbound_source":null,"record_url":null,"record_label":null,"record_kind":null,"record_source":null,"download_url":"http://etheses.bham.ac.uk//id/eprint/12753/7/Shuai2022PhD.pdf","download_label":"Download PDF","download_source":"dc:identifier.uri","access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://etheses.bham.ac.uk/cgi/oai2?verb=GetRecord&metadataPrefix=uketd_dc&identifier=oai%3Aetheses.bham.ac.uk%3A12753","prefix":"uketd_dc"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.sponsor","label":"Sponsor","values":["other"]},{"key":"dc:creator","label":"Author","values":["Shuai, Bin"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2022-07-19"]},{"key":"dc:date.issued","label":"Date","values":["2022-07"]},{"key":"dc:publisher.department","label":"Dc Publisher Department","values":["College of Engineering & Physical Sciences","School of Engineering, Department of Mechanical Engineering"]},{"key":"dc:publisher.institution","label":"Dc Publisher Institution","values":["University of Birmingham"]},{"key":"dc:relation.isreferencedby","label":"Dc Relation Isreferencedby","values":["http://etheses.bham.ac.uk//id/eprint/12753/"]},{"key":"dc:type","label":"Dc Type","values":["Thesis"]},{"key":"dc:type.qualificationlevel","label":"Dc Type Qualificationlevel","values":["d_ph"]},{"key":"dc:type.qualificationname","label":"Dc Type Qualificationname","values":["d_ph"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["TJ Mechanical engineering and machinery","TL Motor vehicles. Aeronautics. Astronautics"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language.dcterms_iso639_2","label":"Language (ISO 639-2)","values":["eng"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.grantnumber","label":"Dc Identifier Grantnumber","values":["The development of world-leading off-highway vehicular technologies (Grant 102253)"]},{"key":"dc:identifier.uri","label":"Identifier URI","values":["http://etheses.bham.ac.uk//id/eprint/12753/7/Shuai2022PhD.pdf"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["Rapid developments in transportation and mobility systems including off-highway vehicles have a significant impact on the environment and energy. Electrification of off-highway vehicles makes a significant contribution to decarbonisation with advancements in vehicle technologies. This Ph.D. research focuses on the development of an advanced energy management strategy (EMS) for the hybrid off-highway aircraft-towing tractor, harnessing reinforcement learning (RL), one of the most promising artificial intelligence technologies. Based on real-time modelling of the vehicle system using MATLAB/Simulink, the dedicated RL algorithms are developed to enhance the exploration capability and reduce the overestimation of the reward-action pairs so that they can improve the energy efficiency of the hybrid vehicle in learning cycles and real-world driving. The real-time control functionalities of the proposed energy system are tested and verified on the hardware-in-the-loop platform. By introducing three decay functions to the epsilon-greedy process of the RL-based EMS, the impacts of the exploration-to-exploitation ratio (E2E) on improving vehicle energy efficiency are investigated. A step-based decay function is proposed as a unified decay function incorporated with the vehicle EMS. Its contribution to the improvement of trustworthiness is demonstrated by a comparison with standard Q learning. An improvement rate of up to 6.21% higher than the standard Q-learning is achieved. Based on the performance analysis of the RL-based EMS with different decay functions, information-fused and synchronised ensemble learning frameworks are developed to enhance the optimality and robustness of the RL-based EMS. The random-based, maximum-based and weighted-based methods are studied and compared under the ensemble learning frameworks. The study suggests that the weighted-based method is the best of the three ensemble methods. This research leads to a new synchronised ensemble learning framework founded on the weighted-based method, with up to 1.06% and 5.06% higher vehicle energy efficiency than the maximum-based method and random-based method, respectively. Additionally, the weighted-based method achieves 1.15% higher vehicle energy efficiency than the standard Q-learning. Finally, by observing overestimation phenomena in the RL-based EMS under disturbed learning and undisturbed learning scenarios, a predictive double Q-learning framework with backup models and random action execution policy, respectively is developed and used to increase the efficiency of the learning experience and determine the final action to improve the robustness of the RL-based EMS for the hybrid off-highway vehicles."]},{"key":"dc:format","label":"Dc Format","values":["text"]},{"key":"dcterms:accessrights","label":"Dcterms Accessrights","values":["public"]},{"key":"dc:title","label":"Title","values":["Real-time energy management of the hybrid off-highway vehicle using reinforcement learning"]}]}],"canonical_facts":{"dc:contributor.sponsor":["other"],"dc:creator":["Shuai, Bin"],"dc:date":["2022-07-19"],"dc:date.issued":["2022-07"],"dc:description.abstract":["Rapid developments in transportation and mobility systems including off-highway vehicles have a significant impact on the environment and energy. Electrification of off-highway vehicles makes a significant contribution to decarbonisation with advancements in vehicle technologies. This Ph.D. research focuses on the development of an advanced energy management strategy (EMS) for the hybrid off-highway aircraft-towing tractor, harnessing reinforcement learning (RL), one of the most promising artificial intelligence technologies. Based on real-time modelling of the vehicle system using MATLAB/Simulink, the dedicated RL algorithms are developed to enhance the exploration capability and reduce the overestimation of the reward-action pairs so that they can improve the energy efficiency of the hybrid vehicle in learning cycles and real-world driving. The real-time control functionalities of the proposed energy system are tested and verified on the hardware-in-the-loop platform. By introducing three decay functions to the epsilon-greedy process of the RL-based EMS, the impacts of the exploration-to-exploitation ratio (E2E) on improving vehicle energy efficiency are investigated. A step-based decay function is proposed as a unified decay function incorporated with the vehicle EMS. Its contribution to the improvement of trustworthiness is demonstrated by a comparison with standard Q learning. An improvement rate of up to 6.21% higher than the standard Q-learning is achieved. Based on the performance analysis of the RL-based EMS with different decay functions, information-fused and synchronised ensemble learning frameworks are developed to enhance the optimality and robustness of the RL-based EMS. The random-based, maximum-based and weighted-based methods are studied and compared under the ensemble learning frameworks. The study suggests that the weighted-based method is the best of the three ensemble methods. This research leads to a new synchronised ensemble learning framework founded on the weighted-based method, with up to 1.06% and 5.06% higher vehicle energy efficiency than the maximum-based method and random-based method, respectively. Additionally, the weighted-based method achieves 1.15% higher vehicle energy efficiency than the standard Q-learning. Finally, by observing overestimation phenomena in the RL-based EMS under disturbed learning and undisturbed learning scenarios, a predictive double Q-learning framework with backup models and random action execution policy, respectively is developed and used to increase the efficiency of the learning experience and determine the final action to improve the robustness of the RL-based EMS for the hybrid off-highway vehicles."],"dc:format":["text"],"dc:identifier.grantnumber":["The development of world-leading off-highway vehicular technologies (Grant 102253)"],"dc:identifier.uri":["http://etheses.bham.ac.uk//id/eprint/12753/7/Shuai2022PhD.pdf"],"dc:language.dcterms_iso639_2":["eng"],"dc:publisher.department":["College of Engineering & Physical Sciences","School of Engineering, Department of Mechanical Engineering"],"dc:publisher.institution":["University of Birmingham"],"dc:relation.isreferencedby":["http://etheses.bham.ac.uk//id/eprint/12753/"],"dc:subject":["TJ Mechanical engineering and machinery","TL Motor vehicles. Aeronautics. Astronautics"],"dcterms:accessrights":["public"],"dc:title":["Real-time energy management of the hybrid off-highway vehicle using reinforcement learning"],"dc:type":["Thesis"],"dc:type.qualificationlevel":["d_ph"],"dc:type.qualificationname":["d_ph"]},"updated_at":"2026-11-19T09:31:51Z"}