A MULTI-AGENT TWIN DELAYED DEEP DETERMINISTIC POLICY GRADIENT (MA-TD3) FRAMEWORK FOR PROACTIVE GRID ENERGY DISTRIBUTION AND SCHEDULING
Krunalkumar J. Gandhi, Dolly Thankachan
Abstract: The integration of intermittent renewable energy sources and the increase of dynamic demand profiles are placing great demands on modern power grids, requiring smart and adaptive control systems to be able to follow them. This paper presents a new type of Multi-Agent Twin Delayed Deep Deterministic Policy Gradient (MA-TD3) algorithm framework for energy distribution and scheduling optimization. Unlike centralized approaches, our methodology is based on a hierarchical multi-agent system in which agents representing specific areas of the grid (i.e. microgrids, storage farms, representers for demand response) coordinate together with a centralized critic and decentralized actors. This design increases scalability and addresses information privacy between all the stakeholders. The state space combines the use of multi-resolution prediction (solar irradiance/wind speed/load), real-time system telemetry and market signals. A composite reward function is designed to minimize Levelized Cost of Energy (LCOE) as well as penalize deviations of the grid frequency and favour renewable curtailment avoidance. We give full mathematical formulations of Markov game, policy networks, critic updates and centralized training with decentralized execution paradigm. Training is done in a high-fidelity digital twin environment based on the OpenAI gym interface with Pandapower and CityLearn simulators. Experimental results show operational cost improvement of 17.3% and renewable self-consumption improvement of 22.1% with respect to state-of-the-art benchmarks such as Model Predictive Control (MPC) and single-agent Deep Q-Network (DQN) methods and keeping voltage and frequency stability. The proposed framework presents an effective solution for the scalable and decentralized grid management.
Keywords: Reinforcement learning; Twin delayed DDPG (TD3); Multiagent systems; Energy scheduling; Smart grid; Digital twin; Renewable integration
DOI: 10.24874/PES08.03.012
Recieved: 20.05.2026 Revised: 09.07.2026 Accepted: 30.07.2026
UDC:
Reads: 5 
