Parametric study of adaptive reinforcement learning for battery operations in microgrids

dc.contributor.authorPanda, Deepak Kumar
dc.contributor.authorDas, Saptarshi
dc.contributor.authorAbusara, Mohammad
dc.date.accessioned2025-06-30T12:55:29Z
dc.date.available2025-06-30T12:55:29Z
dc.date.freetoread2025-06-30
dc.date.issued2025-09-01
dc.date.pubOnline2025-05-02
dc.description.abstractReinforcement learning (RL) has been increasingly used for efficient energy management systems (EMSs) in microgrids. The battery storage system in the microgrid can be controlled using efficient policies derived from RL. However, little attention has been paid so far to the parametric study, which is a fundamental step for efficient implementation of such RL algorithms. Unlike previous works which focused on the implementation of different RL algorithms, this paper mainly demonstrates the parametric sensitivity study of the RL algorithms. It involves investigating the effects of (1) controllable state discretization, (2) exogenous state discretization, (3) action discretization, (4) exploration and exploitation parameters, and (5) decision intervals. Moreover, the performance of the ε-greedy randomized RL algorithm is compared against the adaptive Q-learning, derived from the adaptive approximate dynamic programming (ADP). In many microgrids utilizing solar energy and battery storage, energy management still relies on manually tuned and inefficient algorithms. This is largely due to the sensitivity of RL algorithm parameters to factors such as the specific EMS problem, environment, action-state discretization, exploration parameter and time step. We show the univariate and multivariate kernel density estimate (KDE) plots to study the RL algorithms’ performance concerning the rewards and variation of the battery state of charge (SoC) and the net power imported from the grid. Overall, the deterministic adaptive RL performs better as compared to the randomized ε-greedy algorithms in terms of rewards and simulation time. Higher discretization levels in the action space affect the convergence rate while lower discretization levels in the state space influence the performance of the algorithm. The proposed parametric analysis can be easily adapted to other EMS in more complex microgrids.
dc.description.journalNameRenewable Energy
dc.description.sponsorshipThis work was partially supported by the ESIF ERDF Cornwall New Energy project, project number: 05R16P00282. SD was partially supported by the ERDF Deep Digital Cornwall project number: 05R18P02820.
dc.identifier.citationPanda DK, Das S, Abusara M. (2025) Parametric study of adaptive reinforcement learning for battery operations in microgrids. Renewable Energy, Volume 250, September 2025, Article number 123250en_UK
dc.identifier.eissn1879-0682
dc.identifier.elementsID673153
dc.identifier.issn0960-1481
dc.identifier.paperNo123250
dc.identifier.urihttps://doi.org/10.1016/j.renene.2025.123250
dc.identifier.urihttps://dspace.lib.cranfield.ac.uk/handle/1826/24043
dc.identifier.volumeNo250
dc.languageEnglish
dc.language.isoen
dc.publisherElsevieren_UK
dc.publisher.urihttps://www.sciencedirect.com/science/article/pii/S0960148125009127?via%3Dihub
dc.rightsAttribution 4.0 Internationalen
dc.rights.urihttp://creativecommons.org/licenses/by/4.0/
dc.subjectMicrogrid managementen_UK
dc.subjectBattery operationsen_UK
dc.subjectQ -learningen_UK
dc.subjectReinforcement learning convergenceen_UK
dc.subjectAdaptive RLen_UK
dc.subject40 Engineeringen_UK
dc.subject4008 Electrical Engineeringen_UK
dc.subject7 Affordable and Clean Energyen_UK
dc.subjectEnergyen_UK
dc.titleParametric study of adaptive reinforcement learning for battery operations in microgridsen_UK
dc.typeArticle
dc.type.subtypeJournal Article
dcterms.dateAccepted2025-04-24

Files

Original bundle

Now showing 1 - 1 of 1
Loading...
Thumbnail Image
Name:
Parametric_study_of_adaptive_reinforcement_learning-2025.pdf
Size:
33.15 MB
Format:
Adobe Portable Document Format
Description:
Published version

License bundle

Now showing 1 - 1 of 1
Loading...
Thumbnail Image
Name:
license.txt
Size:
1.63 KB
Format:
Plain Text
Description: