Skip to main content
SpringerLink
Log in

An Event-Triggered Heuristic Dynamic Programming Algorithm for Discrete-Time Nonlinear Systems

  • Conference paper
  • First Online:
Neural Information Processing (ICONIP 2017)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10634))

Included in the following conference series:

  • 4666 Accesses

Abstract

Event-triggered control means the control law of the systems will only be updated when the triggering condition is met, so that the computational burden is reduced. In this paper, a new triggering condition of the heuristic dynamic programming (HDP) algorithm is developed for discrete-time nonlinear systems. Two neural networks are constructed to estimate the value function and the control law. Besides, the Lyapunov stability of systems under the algorithm is proven. Finally, an example is presented to show the effectiveness of the algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Werbos, P.J.: Advanced forecasting methods for global crisis warning and models of intelligence. General Syst. Yearbook 22(6), 25–38 (1977)

    Google Scholar 

  2. Wei, Q., Liu, D., Lin, H.: Value iteration adaptive dynamic programming for optimal control of discrete-time nonlinear systems. IEEE Trans. Cybern. 46(3), 840–853 (2016)

    Article  Google Scholar 

  3. Wei, Q., Liu, D., Lewis, F.L., Liu, Y.: Mixed iterative adaptive dynamic programming for optimal battery energy control in smart residential microgrids. IEEE Trans. Industr. Electron. 64(5), 4110–4120 (2017)

    Article  Google Scholar 

  4. Sahoo, A., Xu, H., Jagannathan, S.: Near optimal event-triggered control of nonlinear discrete-time systems using neurodynamic programming. IEEE Trans. Neural Networks Learn. Syst. 27(9), 1801–1815 (2015)

    Article  MathSciNet  Google Scholar 

  5. Wei, Q., Song, R., Yan, P.: Data-driven zero-sum neuro-optimal control for a class of continuous-time unknown nonlinear systems with disturbance using ADP. IEEE Trans. Neural Networks Learn. Syst. 27(2), 444–458 (2016)

    Article  MathSciNet  Google Scholar 

  6. Wei, Q., Lewis, F.L., Sun, Q., Yan, P., Song, R.: Discrete-time deterministic Q-learning: a novel convergence analysis. IEEE Trans. Cybern. 47(5), 1224–1237 (2017)

    Article  Google Scholar 

  7. Lewis, F.L., Vrabie, D.: Reinforcement learning and adaptive dynamic programming for feedback control. IEEE Circuits Syst. Mag. 9(3), 32–50 (2009)

    Article  Google Scholar 

  8. Wei, Q., Lewis, F.L., Liu, D., Song, R., Lin, H.: Discrete-time local value iteration adaptive dynamic programming: Convergence analysis. IEEE Trans. Neural Networks Learn. Syst. (2016). doi:10.1109/TSMC.2016.2623766

  9. Wei, Q., Liu, D., Lin, Q., Song, R.: Discrete-time optimal control via local policy iteration adaptive dynamic programming. IEEE Trans. Cybern. (2016). doi:10.1109/TCYB.2016.2586082

  10. Werbos, P.J.: Foreword - ADP: the key direction for future research in intelligent control and understanding brain intelligence. IEEE Trans. Cybern. 38(4), 898–900 (2008)

    Article  Google Scholar 

  11. Wei, Q., Lewis, F.L., Shi, G., Song, R.: Error-tolerant iterative adaptive dynamic programming for optimal renewable home energy scheduling and battery management. IEEE Trans. Ind. Electron. (2017). doi:10.1109/TIE.2017.2711499

  12. Wei, Q., Shi, G., Song, R., Liu, Y.: Adaptive dynamic programming-based optimal control scheme for energy storage systems with solar renewable energy. IEEE Trans. Ind. Electron. (2017). doi:10.1109/TIE.2017.2674581

  13. Wei, Q., Liu, D., Liu, Y., Song, R.: Optimal constrained self-learning battery sequential management in microgrids via adaptive dynamic programming. IEEE/CAA J. Automatica Sinica 4(2), 168–176 (2017)

    Article  MathSciNet  Google Scholar 

  14. Lendaris, G.G.: Higher level application of ADP: a next phase for the control field? IEEE Trans. Syst. Man Cybern. B Cybern. 38(4), 901–12 (2008)

    Article  Google Scholar 

  15. Wei, Q., Wang, F., Liu, D., Yang, X.: Finite-approximation-error-based discrete-time iterative adaptive dynamic programming. IEEE Trans. Cybern. 44(12), 2820–2833 (2014)

    Article  Google Scholar 

  16. Wei, Q., Liu, D., Lin, Q., Song, R.: Adaptive dynamic programming for discrete-time zero-sum games. IEEE Trans. Neural Networks Learn. Syst. (2016). doi:10.1109/TNNLS.2016.2638863

  17. Wang, X., Lemmon, M.D.: Event-triggering in distributed networked control systems. IEEE Trans. Autom. Control 56(3), 586–601 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  18. Anta, A., Tabuada, P.: To sample or not to sample: self-triggered control for nonlinear systems. IEEE Trans. Autom. Control 55(9), 2030–2042 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  19. Eqtami, A., Dimarogonas, D.V., Kyriakopoulos, K.J.: Event-triggered control for discrete-time systems. In: American Control Conference, pp. 4719–4724 (2010)

    Google Scholar 

  20. Dimarogonas, D.V., Frazzoli, E., Johansson, K.H.: Distributed event-triggered control for multi-agent systems. IEEE Trans. Autom. Control 57(5), 1291–1297 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  21. Liu, W., Yang, C., Sun, Y., Qin, J.: Observer-based event-triggered tracking control of leader-follower systems with time delay. J. Syst. Sci. Complexity 29(4), 865–880 (2016)

    Article  MATH  MathSciNet  Google Scholar 

  22. Zhong, X., He, H.: An event-triggered ADP control approach for continuous-time system with unknown internal states. IEEE Trans. Cybern. 47(3), 683–694 (2017)

    Article  Google Scholar 

  23. Dong, L., Zhong, X., Sun, C., He, H.: Adaptive event-triggered control based on heuristic dynamic programming for nonlinear discrete-time systems. IEEE Trans. Neural Networks Learn. Syst. (2016). doi:10.1109/TNNLS.2016.2541020

  24. Si, J., Wang, Y.: On-line learning control by association and reinforcement. Neural Networks Official J. Int. Neural Network Soc. 12(2), 264–276 (2001)

    MathSciNet  Google Scholar 

  25. Wei, Q., Liu, D., Lin, Q.: Discrete-time local iterative adaptive dynamic programming: terminations and admissibility analysis. IEEE Trans. Neural Networks Learn. Syst. (2016). doi:10.1109/TNNLS.2016.2593743

  26. Liu, F., Sun, J., Si, J., Guo, W., Mei, S.: A boundedness result for the direct heuristic dynamic programming. Neural Networks Official J. Int. Neural Network Soc. 32(1), 229–235 (2012)

    Article  MATH  Google Scholar 

Download references

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China under Grants 61233001, 61722312, 61533017, 61374105 and 61673054.

Author information

Authors and Affiliations

  1. School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing, 100083, China

    Ziyang Wang

  2. The State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China

    Qinglai Wei

  3. University of Chinese Academy of Sciences, Beijing, 100049, China

    Qinglai Wei

  4. School of Automation, Guangdong University of Technology, Guangzhou, 510006, China

    Derong Liu

Authors
  1. Ziyang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qinglai Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Derong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qinglai Wei .

Editor information

Editors and Affiliations

  1. Guangdong University of Technology, Guangzhou, China

    Derong Liu

  2. Guangdong University of Technology, Guangzhou, China

    Shengli Xie

  3. South China University of Technology, Guangzhou, China

    Yuanqing Li

  4. Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Dongbin Zhao

  5. King Fahd University of Petroleum and Minerals, Dhahran, Saudi Arabia

    El-Sayed M. El-Alfy

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Wang, Z., Wei, Q., Liu, D. (2017). An Event-Triggered Heuristic Dynamic Programming Algorithm for Discrete-Time Nonlinear Systems. In: Liu, D., Xie, S., Li, Y., Zhao, D., El-Alfy, ES. (eds) Neural Information Processing. ICONIP 2017. Lecture Notes in Computer Science(), vol 10634. Springer, Cham. https://doi.org/10.1007/978-3-319-70087-8_76

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-70087-8_76

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-70086-1

  • Online ISBN: 978-3-319-70087-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation