Back to search

Carleton University

Multi-Agent Robust Deep Reinforcement Learning for Voltage and Frequency Control in Electrical Distribution Systems with High Penetration of PVs and BESSs

Abstract

dc:description.abstract

As the distribution network expands and the penetration of photovoltaics (PVs) and battery energy storage systems (BESSs) is higher, its complexity increases. Multi-agent deep reinforcement learning (MADRL), being model-free, offers optimal control for large-scale systems, outperforming traditional methods in solving complex multi-variable problems. The integration of physical factors can enhance MADRL's robustness, particularly for voltage and frequency control. Thus, this thesis investigates voltage and frequency control in distribution networks, including fault conditions, with high penetration of PVs and BESSs based on multi-agent robust deep reinforcement learning algorithms. Firstly, the physics-guided multi-agent adversarial twin delayed deep deterministic (PG-MA2TD3) framework is proposed to address day-ahead energy trading and intra-day voltage regulation challenges. Within energy trading, it deploys a cost-minimization framework constrained by the dynamic behaviors of BESSs and distributed generators, enabling iterative negotiations among prosumers to attain optimal trading solutions in day-ahead stage. Subsequently, the PG-MA2TD3 algorithm is designed to resolve active voltage control (AVC) issue in intra-day stage by optimizing reactive power adjustments to mitigate line losses. Furthermore, to limit the action of the agent when approaching a dangerous state of BESSs in AVC scheme, a multi-agent twin delayed deep deterministic (MATD3) policy gradient algorithm with a physics-based shielding mechanism is developed. Additionally, a physics factor, global sensitivity of voltage (GSV), is embedded in the adversarial learning algorithm to measure the influence of the nodal voltage fluctuation on voltage violations on the other controlled nodes with PV inverters. The IEEE distribution networks with three-year electrical data in Portuguese are used to verify the effectiveness of designed algorithms for energy trading and active voltage control. Lastly, secondary control under fault conditions is investigated to eliminate the steady-state error and thereby ensure the control performance of AVC. The proposed MADRL algorithm can achieve frequency regulation and the state-of-charge (SoC) balancing simultaneously for BESSs subject to packet loss. This scheme leverages an asynchronous advantage actor-critic (A3C) based MADRL algorithm where the centralized learning is designed to ensure the performance of the entire system. Simulation results on a four-bus microgrid show that the proposed secondary controller can achieve simultaneous frequency regulation and SoC balancing.

Degree

thesis:*
Name thesis:degree_name
Doctor of Philosophy (Ph.D.)
Level thesis:degree_level
Doctoral
Discipline thesis:degree_discipline
Engineering, Electrical and Computer
Grantor dc:publisher
Carleton University
Year dc:date.issued
2024

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Chen, Pengcheng

Subjects

dc:subject × 2

Rights

dc:rights
Statement dc:rights
  • Copyright © 2024 the author(s). Theses may be used for non-commercial research, educational, or related academic purposes only. Such uses include personal study, distribution to students, research and scholarship. Theses may only be shared by linking to the Carleton University Institutional Repository and no part may be copied without proper attribution to the author; no part may be used for commercial purposes directly or indirectly via a for-profit platform; no adaptation or derivative works are permitted without consent from the copyright owner.
Language dc:language.iso
en

Identifiers

dc:identifier.*

Chain of custody

source
Harvested from
Carleton University
Base URL
carleton.scholaris.ca/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Chen, Pengcheng. Multi-Agent Robust Deep Reinforcement Learning for Voltage and Frequency Control in Electrical Distribution Systems with High Penetration of PVs and BESSs. Doctoral thesis, Carleton University, 2024. https://hdl.handle.net/20.500.14718/43353