Carleton University
Multi-Agent Robust Deep Reinforcement Learning for Voltage and Frequency Control in Electrical Distribution Systems with High Penetration of PVs and BESSs
Abstract
dc:description.abstractAs the distribution network expands and the penetration of photovoltaics (PVs) and battery energy storage systems (BESSs) is higher, its complexity increases. Multi-agent deep reinforcement learning (MADRL), being model-free, offers optimal control for large-scale systems, outperforming traditional methods in solving complex multi-variable problems. The integration of physical factors can enhance MADRL's robustness, particularly for voltage and frequency control. Thus, this thesis investigates voltage and frequency control in distribution networks, including fault conditions, with high penetration of PVs and BESSs based on multi-agent robust deep reinforcement learning algorithms. Firstly, the physics-guided multi-agent adversarial twin delayed deep deterministic (PG-MA2TD3) framework is proposed to address day-ahead energy trading and intra-day voltage regulation challenges. Within energy trading, it deploys a cost-minimization framework constrained by the dynamic behaviors of BESSs and distributed generators, enabling iterative negotiations among prosumers to attain optimal trading solutions in day-ahead stage. Subsequently, the PG-MA2TD3 algorithm is designed to resolve active voltage control (AVC) issue in intra-day stage by optimizing reactive power adjustments to mitigate line losses. Furthermore, to limit the action of the agent when approaching a dangerous state of BESSs in AVC scheme, a multi-agent twin delayed deep deterministic (MATD3) policy gradient algorithm with a physics-based shielding mechanism is developed. Additionally, a physics factor, global sensitivity of voltage (GSV), is embedded in the adversarial learning algorithm to measure the influence of the nodal voltage fluctuation on voltage violations on the other controlled nodes with PV inverters. The IEEE distribution networks with three-year electrical data in Portuguese are used to verify the effectiveness of designed algorithms for energy trading and active voltage control. Lastly, secondary control under fault conditions is investigated to eliminate the steady-state error and thereby ensure the control performance of AVC. The proposed MADRL algorithm can achieve frequency regulation and the state-of-charge (SoC) balancing simultaneously for BESSs subject to packet loss. This scheme leverages an asynchronous advantage actor-critic (A3C) based MADRL algorithm where the centralized learning is designed to ensure the performance of the entire system. Simulation results on a four-bus microgrid show that the proposed secondary controller can achieve simultaneous frequency regulation and SoC balancing.
Degree
thesis:*- Name thesis:degree_name
- Doctor of Philosophy (Ph.D.)
- Level thesis:degree_level
- Doctoral
- Discipline thesis:degree_discipline
- Engineering, Electrical and Computer
- Grantor dc:publisher
- Carleton University
- Year dc:date.issued
- 2024
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Chen, Pengcheng
Subjects
dc:subject × 2Rights
dc:rights- Statement dc:rights
-
- Copyright © 2024 the author(s). Theses may be used for non-commercial research, educational, or related academic purposes only. Such uses include personal study, distribution to students, research and scholarship. Theses may only be shared by linking to the Carleton University Institutional Repository and no part may be copied without proper attribution to the author; no part may be used for commercial purposes directly or indirectly via a for-profit platform; no adaptation or derivative works are permitted without consent from the copyright owner.
- Language dc:language.iso
- en