Abstract
A proximal policy optimization reinforcement learning system is proposed to handle the energy dispatch management of a sample microgrid. The microgrid in question has 3 participants of different classifications, signifying their relative importance and how sensitive they are to energy shortages. The energy within the microgrid is generated by these participants, which are individually equipped with a solar panel and a wind turbine for energy generation, and an energy storage system to store this energy. The environmental conditions, i.e. temperature, wind velocity and irradiation figures of Istanbul are considered to obtain accurate energy generation figures. The microgrid is designed to be grid connected in order to compensate for the uncertainties caused by the weather changes, hence service of the utility is accessed when energy produced & stored cannot respond to the demand. Information security of the participants is respected and to that end, direct energy generation, consumption and storage figures are not supplied to the agent, instead only supply and demand figures are transferred. The agent, using this information, after a period of training, optimizes the system for a reward scheme that rewards energy exports and punishes energy deficits and imports. The results verify the feasibility of proximal policy optimization in managing microgrid energy dispatch.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Sahin, A.: Progress and recent trends in wind energy. Prog. Energy Combust. Sci. 30(5), 501–543 (2004)
Duman, A.C., Güler, Ö.: Economic analysis of grid-connected residential rooftop PV systems in Turkey. Renew. Energy 148, 697–711 (2020)
Rezaeiha, A., Montazeri, H., Blocken, B.: A framework for preliminary large-scale urban wind energy potential assessment: roof-mounted wind turbines. Energy Convers. Manag. 214, 112770 (2020)
Vineetha, C.P., Babu, C.A.: Smart grid challenges, issues and solutions. In: 2014 International Conference on Intelligent Green Building and Smart Grid (IGBSG), Taipei, Taiwan, pp. 1–4 (2014)
Majzoobi, A., Khodaei, A.: Application of microgrids in supporting distribution grid flexibility. IEEE Trans. Power Syst. 32(5), 3660–3669 (2017)
Guerrero, J.M., Vasquez, J.C., Matas, J., de Vicuna, L.G., Castilla, M.: Hierarchical control of droop-controlled AC and DC microgrids—a general approach toward standardization. IEEE Trans. Ind. Electron. 58(1), 158–172 (2011)
Farrokhabadi, M., Canizares, C.A., Simpson-Porco, J.W., Nasr, E., Fan, L., Mendoza-Araya, P.A., et al.: Microgrid stability definitions, analysis, and examples. IEEE Trans. Power Syst. 35(1), 13–29 (2020)
Wu, D., Zheng, X., Xu, Y., Olsen, D., Xia, B., Singh, C., et al.: An open-source extendable model and corrective measure assessment of the 2021 texas power outage. Adv. Appl. Energy 4, 100056 (2021)
Murty, V.V.S.N., Kumar, A.: Multi-objective energy management in microgrids with hybrid energy sources and battery energy storage systems. Prot. Control Mod. Power Syst. 5(1) (2020). Article number: 2. https://doi.org/10.1186/s41601-019-0147-z
Qazi, H.S., Liu, N., Wang, T.: Coordinated energy and reserve sharing of isolated microgrid cluster using deep reinforcement learning. In: 2020 5th Asia Conference on Power and Electrical Engineering (ACPEE), Chengdu, China, pp. 81–86 (2020)
Kozlov, A.N., Tomin, N.V., Sidorov, D.N., Lora, E.E.S., Kurbatsky, V.G.: Optimal operation control of PV-biomass gasifier-diesel-hybrid systems using reinforcement learning techniques. Energies 13(10), 2632 (2020)
Muriithi, G., Chowdhury, S.: Optimal energy management of a grid-tied solar PV-battery microgrid: a reinforcement learning approach. Energies 14(9), 2700 (2021)
Shang, Y., et al.: Stochastic dispatch of energy storage in microgrids: An augmented reinforcement learning approach. Appl. Energy 261, 114423 (2020)
Al‐Gabalawy, M.: Advanced machine learning tools based on energy management and economic performance analysis of a microgrid connected to the utility grid. Int. J. Energy Res., 1–22 (2021). https://doi.org/10.1002/er.6764
Sadeghi, M., Mollahasani, S., Erol-Kantarci, M.: Power loss-aware transactive microgrid coalitions under uncertainty. Energies 13(21), 5782 (2020)
Fan, L., Zhang, J., He, Y., Liu, Y., Hu, T., Zhang, H.: Optimal scheduling of microgrid based on deep deterministic policy gradient and transfer learning. Energies 14(3), 584 (2021)
Samadi, E., Badri, A., Ebrahimpour, R.: Decentralized multi-agent based energy management of microgrid using reinforcement learning. Int. J. Electr. Power Energy Syst. 122, 106211 (2020)
Fang, X., Zhao, Q., Wang, J., Han, Y., Li, Y.: Multi-agent deep reinforcement learning for distributed energy management and strategy optimization of microgrid market. Sustain. Cities Soc. 74, 103163 (2021)
Guo, C., Wang, X., Zheng, Y., Zhang, F.: Optimal energy management of multi-microgrids connected to distribution system based on deep reinforcement learning. Int. J. Electr. Power Energy Syst. 131, 107048 (2021)
Wu, T., Wang, J.: Artificial intelligence for operation and control: the case of microgrids. Electricity J. 34(1), 106890 (2021)
Yin, L., Zhang, B.: Time series generative adversarial network controller for long-term smart generation control of microgrids. Appl. Energy 281, 116069 (2021)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, 2nd edn. MIT Press, Cambridge (2018)
Gu, S., Lillicrap, T., Ghahramani, Z., Turner, R.E., Levine, S.: Q-Prop: sample-efficient policy gradient with an off-policy critic. ArXiv (2017)
Open AI Proximal Policy Optimization. https://openai.com/blog/openai-baselines-ppo. Accessed 09 Feb 2021
Introduction to Deep Reinforcement Learning (Deep RL). https://www.youtube.com/watch?v=zR11FLZ-O9M&t=3487s. Accessed 09 Feb 2021
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms. arXiv (2017)
EU Energy Poverty Observatory, What is energy poverty? https://www.energypoverty.eu/about/what-energy-poverty. Accessed 09 Jan 2021
Photovoltaic Geographical Information System (PVGIS). https://ec.europa.eu/jrc/en/pvgis. Accessed 09 Feb 2021
Atia, R., Yamada, N.: Sizing and analysis of renewable energy and battery systems in residential microgrids. IEEE Trans. Smart Grid 7(3), 1204–1213 (2016)
Kim, R.-K., Glick, M.B., Olson, K.R., Kim, Y.-S.: MILP-PSO combined optimization algorithm for an islanded microgrid scheduling with detailed battery ESS efficiency model and policy considerations. Energies 13(8), 1898 (2020)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 IFIP International Federation for Information Processing
About this paper
Cite this paper
Gür, B., Kayakutlu, G. (2022). Deep Reinforced Learning for the Governance of a Sample Microgrid. In: Mercier-Laurent, E., Kayakutlu, G. (eds) Artificial Intelligence for Knowledge Management, Energy, and Sustainability. AI4KMES 2021. IFIP Advances in Information and Communication Technology, vol 637. Springer, Cham. https://doi.org/10.1007/978-3-030-96592-1_13
Download citation
DOI: https://doi.org/10.1007/978-3-030-96592-1_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-96591-4
Online ISBN: 978-3-030-96592-1
eBook Packages: Computer ScienceComputer Science (R0)
