Back to search

University of Birmingham

Multi-scale energy management for multi-mode hybrid vehicles using multi-agent reinforcement learning

Abstract

dc:description.abstract

The automotive industry is undergoing a transformative shift driven by advancements in connectivity, intelligence, and electrification. While the rise of connected and autonomous vehicles (CAVs) and plug-in hybrid electric vehicles (PHEVs) presents new opportunities, it also introduces challenges including maintaining efficient inter-vehicle communication in dynamic traffic environments, coordinating energy flow among multiple propulsion sources under varying operating conditions, and ensuring the control effectiveness and practical applicability of hybrid powertrains. This research addresses these challenges by developing a reinforcement learning (RL) framework that integrates macro-level cooperative adaptive cruise control (CACC) and micro-level energy management strategies (EMSs) for multi-mode PHEVs, enabling intelligent and scalable energy optimization. At the macro (traffic) level, CACC is critical for ensuring the stability and energy efficiency of CAV platoons. A fully-decentralized multi-agent reinforcement learning (MARL) framework is introduced to enhance communication efficiency by incorporating the quantified stochastic gradient descent (QSGD) and a binary differential consensus (BDC) method. Compared to existing state-of-the-art MARL approaches, BDC-MARL achieves a 5.8% improvement in energy efficiency, maintains an average velocity of 15.26 m/s, and ensures consistent inter- vehicle spacing (IVS) of 20.76 m. Validation using real-world OpenACC data demonstrates its ability to balance platoon stability with energy optimization effectively. At the micro (vehicle) level, multi-mode PHEVs face the challenge of managing complex multi- input and multi-output (MIMO) energy flows, often leading to oversimplified and suboptimal solutions. To overcome this, a novel MARL-based framework is proposed by incorporating a hand-shaking strategy with a relevance ratio to improve coordination among energy control agents. By fine-tuning parameters such as learning rates and network configurations, this system achieves up to 4% energy savings compared to single-agent systems and up to 23.54% compared to traditional rule-based methods, validated in real-world driving conditions. From a theoretical perspective, this research further refines RL methods in EMSs for PHEVs by addressing overestimation bias. An ensemble strategy with in-target minimization and an update-to-data (UTD) mechanism reduces the bias from 1.18 to 5.2 × 10−4 while achieving up to 15.59% energy savings compared to conventional methods. Through theoretical analysis, bias visualization, and extensive testing on software-in-the-loop (SiL) and hardware-in-the-loop (HiL) platforms under real-world driving conditions, the practical applicability in improving energy savings is validated. By integrating macro and micro perspectives and aligning applications with theoretical analyses, this research provides efficient solutions for improving energy efficiency in CACC and PHEVs, contributing to the development of a sustainable and intelligent future in mobility.

Degree

thesis:*
Name dc:type.qualificationname
d_ph
Level dc:type.qualificationlevel
d_ph
Grantor dc:publisher.institution
University of Birmingham
Year dc:date.issued
2025

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Hua, Min

Subjects

dc:subject × 3

Rights

Language dc:language.dcterms_iso639_2
eng

Chain of custody

source
Harvested from
University of Birmingham
Base URL
etheses.bham.ac.uk/cgi/oai2
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Hua, Min. Multi-scale energy management for multi-mode hybrid vehicles using multi-agent reinforcement learning. d_ph thesis, University of Birmingham, 2025.