Skip to main content
SpringerLink
Log in

Optimal Constrained Neuro-Dynamic Programming Based Self-learning Battery Management in Microgrids

  • Conference paper
  • First Online:
Neural Information Processing (ICONIP 2016)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 9949))

Included in the following conference series:

  • 3098 Accesses

Abstract

In this paper, a novel optimal self-learning battery sequential control scheme is investigated for smart home energy systems. Using the iterative adaptive dynamic programming (ADP) technique, the optimal battery control can be obtained iteratively. Considering the power constraints of the battery, a new non-quadratic form performance index function is established, which guarantees the value of the iterative control law not to exceed the maximum charging/discharging power of the battery to extend the service life of the battery. Simulation results are given to illustrate the performance of the presented method.

This work was supported in part by the National Natural Science Foundation of China under Grants 61233001, 61273140, 61304086, 61374105, 61503377, 61503379, 61304079, 61533017, and U1501251.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Abu-Khalaf, M., Lewis, F.L.: Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach. Automatica 41, 779–791 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  2. Boaro, M., Fuselli, D., Angelis, F.D., Liu, D., Wei, Q., Piazza, F.: Adaptive dynamic programming algorithm for renewable energy scheduling and battery management. Cogn. Comput. 5, 264–277 (2013)

    Article  Google Scholar 

  3. Fuselli, D., Angelis, F.D., Boaro, M., Liu, D., Wei, Q., Squartini, S., Piazza, F.: Action dependent heuristic dynamic programming for home energy resource scheduling. Int. J. Electr. Power Energy Syst. 48, 148–160 (2013)

    Article  Google Scholar 

  4. Huang, T., Liu, D.: A self-learning scheme for residential energy system control and management. Neural Comput. Appl. 22, 259–269 (2013)

    Article  Google Scholar 

  5. Jiang, Y., Jiang, Z.P.: Robust adaptive dynamic programming and feedback stabilization of nonlinear systems. IEEE Trans. Neural Netw. Learn. Syst. 25, 882–893 (2014)

    Article  Google Scholar 

  6. Lewis, F.L., Vrabie, D., Vamvoudakis, K.G.: Reinforcement learning and feedback control: using natural decision methods to design optimal adaptive controllers. IEEE Control Syst. 32, 76–105 (2012)

    Article  MathSciNet  Google Scholar 

  7. Lincoln, B., Rantzer, A.: Relaxing dynamic programming. IEEE Trans. Autom. Control 51, 1249–1260 (2006)

    Article  MathSciNet  Google Scholar 

  8. Modares, H., Lewis, F.L.: Optimal tracking control of nonlinear partially-unknown constrained-input systems using integral reinforcement learning. Automatica 50, 1780–1792 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  9. Song, R., Xiao, W., Zhang, H., Sun, C.: Adaptive dynamic programming for a class of complex-valued nonlinear systems. IEEE Trans. Neural Netw. Learn. Syst. 25, 1733–1739 (2014)

    Article  Google Scholar 

  10. Song, R., Lewis, F.L., Wei, Q., Zhang, H., Jiang, Z.P., Levine, D.: Multiple actor-critic structures for continuous-time optimal control using input-output data. IEEE Trans. Neural Netw. Learn. Syst. 26, 851–865 (2015)

    Article  MathSciNet  Google Scholar 

  11. Song, R., Lewis, F.L., Wei, Q., Zhang, H.: Off-policy actor-critic structure for optimal control of unknown systems with disturbances. IEEE Transactions on Cybernetics (2015, in press). doi:10.1109/TCYB.2015.2421338

    Google Scholar 

  12. Wei, Q., Liu, D., Yang, X.: Infinite horizon self-learning optimal control of nonaffine discrete-time nonlinear systems. IEEE Trans. Neural Netw. Learn. Syst. 26, 866–879 (2015)

    Article  MathSciNet  Google Scholar 

  13. Wei, Q., Wang, F., Liu, D., Yang, X.: Finite-approximation-error based discrete-time iterative adaptive dynamic programming. IEEE Trans. Cybern. 44, 2820–2833 (2014)

    Article  Google Scholar 

  14. Wei, Q., Liu, D.: Data-driven neuro-optimal temperature control of water gas shift reaction using stable iterative adaptive dynamic programming. IEEE Trans. Ind. Electron. 61, 6399–6408 (2014)

    Article  Google Scholar 

  15. Wei, Q., Liu, D., Shi, G., Liu, Y.: Optimal multi-battery coordination control for home energy management systems via distributed iterative adaptive dynamic programming. IEEE Trans. Ind. Electron. 42, 4203–4214 (2015)

    Article  Google Scholar 

  16. Wei, Q., Song, R., Yan, P.: Data-driven zero-sum neuro-optimal control for a class of continuous-time unknown nonlinear systems with disturbance using ADP. IEEE Trans. Neural Netw. Learn. Syst. 27, 444–458 (2016)

    Article  MathSciNet  Google Scholar 

  17. Wei, Q., Liu, D., Lin, H.: Value iteration adaptive dynamic programming for optimal control of discrete-time nonlinear systems. IEEE Trans. Cybern. 46, 840–853 (2016)

    Article  Google Scholar 

  18. Wei, Q., Liu, D.: A novel iterative \(\theta \)-adaptive dynamic programming for discrete-time nonlinear systems. IEEE Trans. Autom. Sci. Eng. 11, 1176–1190 (2014)

    Article  Google Scholar 

  19. Wei, Q., Liu, D.: Adaptive dynamic programming for optimal tracking control of unknown nonlinear systems with application to coal gasification. IEEE Trans. Autom. Sci. Eng. 11, 1020–1036 (2014)

    Article  Google Scholar 

  20. Wei, Q., Liu, D., Shi, G.: A novel dual iterative \(Q\)-learning method for optimal battery management in smart residential environments. IEEE Trans. Ind. Electron. 62, 2509–2518 (2015)

    Article  Google Scholar 

  21. Wei, Q., Liu, D., Lewis, F.L.: Optimal distributed synchronization control for continuous-time heterogeneous multi-agent differential graphical games. Inf. Sci. 317, 96–113 (2015)

    Article  Google Scholar 

  22. Wei, Q., Liu, D.: A novel policy iteration based deterministic Q-learning for discrete-time nonlinear systems. Sci. China Inf. Sci. 58, 1–15 (2015)

    Article  Google Scholar 

  23. Wei, Q., Lewis, F.L., Sun, Q., Yan, P., Song, R.: Discrete-time deterministic \(Q\)-learning: a novel convergence analysis. IEEE Transactions on Cybernetics (2016, in press)

    Google Scholar 

  24. Wei, Q., Liu, D., Lin, Q., Song, R.: Discrete-time optimal control via local policy iteration adaptive dynamic programming. IEEE Transactions on Cybernetics (2016, in press)

    Google Scholar 

  25. Werbos, P.J.: Advanced forecasting methods for global crisis warning and models of intelligence. General Syst. Yearbook 22, 25–38 (1977)

    Google Scholar 

  26. Werbos, P.J.: A menu of designs for reinforcement learning over time. In: Miller, W.T., Sutton, R.S., Werbos, P.J. (eds.) Neural Networks for Control, pp. 67–95. MIT Press, Cambridge (1991)

    Google Scholar 

  27. Yau, T., Walker, L.N., Graham, H.L., Raithel, R.: Effects of battery storage devices on power system dispatch. IEEE Trans. Power Apparatus Syst. 100, 375–383 (1981)

    Article  Google Scholar 

  28. Zhang, H., Qing, C., Luo, Y.: Neural-network-based constrained optimal control scheme for discrete-time switched nonlinear system using dual heuristic programming. IEEE Trans. Autom. Sci. Eng. 11, 839–849 (2014)

    Article  Google Scholar 

  29. Zhao, Q., Xu, H., Jagannathan, S.: Near optimal output feedback control of nonlinear discrete-time systems based on reinforcement neural network learning. IEEE/CAA J. Automatica Sin. 1, 372–384 (2014)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. The State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China

    Qinglai Wei

  2. School of Automation and Electrical Engineering, University of Science and Technology, Beijing, 100083, China

    Derong Liu

Authors
  1. Qinglai Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Derong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qinglai Wei .

Editor information

Editors and Affiliations

  1. The University of Tokyo , Tokyo, Japan

    Akira Hirose

  2. Kobe University , Kobe, Japan

    Seiichi Ozawa

  3. Okinawa Institute of Science and Technology Graduate University, Onna, Japan

    Kenji Doya

  4. Nara Institute of Science and Technology , Ikoma, Japan

    Kazushi Ikeda

  5. Kyungpook National University , Daegu, Korea (Republic of)

    Minho Lee

  6. Chinese Academy of Sciences , Beijing, China

    Derong Liu

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Wei, Q., Liu, D. (2016). Optimal Constrained Neuro-Dynamic Programming Based Self-learning Battery Management in Microgrids. In: Hirose, A., Ozawa, S., Doya, K., Ikeda, K., Lee, M., Liu, D. (eds) Neural Information Processing. ICONIP 2016. Lecture Notes in Computer Science(), vol 9949. Springer, Cham. https://doi.org/10.1007/978-3-319-46675-0_22

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-46675-0_22

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-46674-3

  • Online ISBN: 978-3-319-46675-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation