Skip to main content
SpringerLink
Log in

Neural Control and Approximate Dynamic Programming

Encyclopedia of Systems and Control

  • 545 Accesses

Abstract

There has been great interest recently in “universal model-free controllers” that do not need a mathematical model of the controlled plant, but mimic the functions of biological processes to learn about the systems they are controlling online, so that performance improves automatically. Neural network (NN) control has had two major thrusts: approximate dynamic programming, which uses NN to approximately solve the optimal control problem, and NN in closed-loop feedback control.

Neural Feedback Control

The objective is to design NN feedback controllers that cause a system to follow, or track, a prescribed trajectory or path. Consider the dynamics of an n-link robot manipulator

$$\displaystyle\begin{array}{rcl} M(q)\ddot{q} + V _{m}(q,\dot{q})\dot{q} + G(q) + F(\dot{q}) +\tau _{d}& =& \tau \end{array}$$
(1)

with \(q(t) \in \mathbb{R}^{n}\) the joint variable vector, M(q) an inertia matrix, V m a centripetal/coriolis matrix, G(q) a gravity vector, and \(F(\cdot )\)representing...

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Bibliography

  • Abu-Khalaf M, Lewis FL (2005) Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach. Automatica 41(5):779–791

    Article  MATH  MathSciNet  Google Scholar 

  • Al-Tamimi A, Lewis FL, Abu-Khalaf M (2008) Discrete-time nonlinear HJB solution using approximate dynamic programming: convergence proof. IEEE Trans Syst Man Cybern Part B 38(4):943–949

    Article  Google Scholar 

  • Lewis FL, Liu D (2012) Reinforcement learning and approximate dynamic programming for feedback control. IEEE Press computational intelligence series. Wiley-Blackwell, Oxford

    Book  Google Scholar 

  • Lewis FL, Vamvoudakis KG (2011) Reinforcement learning for partially observable dynamic processes: adaptive dynamic programming using measured output data. IEEE Trans Syst Man Cybern Part B 41(1):14–25

    Article  Google Scholar 

  • Lewis FL, Jagannathan S, Yesildirek A (1999) Neural network control of robot manipulators and nonlinear systems. Taylor and Francis, London

    Google Scholar 

  • Lewis FL, Campos J, Selmic R (2002) Neuro-fuzzy control of industrial systems with actuator nonlinearities. Society of Industrial and Applied Mathematics Press, Philadelphia

    Book  Google Scholar 

  • Lewis FL, Vrabie D, Syrmos VL (2012a) Optimal control. Wiley, New York

    Book  MATH  Google Scholar 

  • Lewis FL, Vrabie D, Vamvoudakis KG (2012b) Reinforcement learning and feedback control: using natural decision methods to design optimal adaptive controllers. IEEE Control Syst Mag 32(6):76–105

    Article  MathSciNet  Google Scholar 

  • Slotine JJE, Li W (1987) On the adaptive control of robot manipulators. Int J Robot Res 6(3):49–59

    Article  Google Scholar 

  • Sutton RS, Barto AG (1998) Reinforcement learning – an introduction. MIT, Cambridge

    Google Scholar 

  • Vamvoudakis KG, Lewis FL (2010) Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica 46(5):878–888

    Article  MATH  MathSciNet  Google Scholar 

  • Vamvoudakis KG, Lewis FL (2011) Multi-player non zero sum games: online adaptive learning solution of coupled Hamilton-Jacobi equations. Automatica 47(8):1556–1569

    Article  MATH  MathSciNet  Google Scholar 

  • Vamvoudakis KG, Lewis FL (2012) Online solution of nonlinear two-player zero-sum games using synchronous policy iteration. Int J Robust Nonlinear Control 22(13):1460–1483

    Article  MATH  MathSciNet  Google Scholar 

  • Vamvoudakis KG, Lewis FL, Hudas GR (2012a) Multi-agent differential graphical games: online adaptive learning solution for synchronization with optimality. Automatica 48(8):1598–1611

    Article  MATH  MathSciNet  Google Scholar 

  • Vamvoudakis KG, Lewis FL, Johnson M, Dixon WE (2012b) Online learning algorithm for Stackelberg games in problems with hierarchy. In: Proceedings of the 51st IEEE conference on decision and control, Maui pp 1883–1889

    Google Scholar 

  • Vamvoudakis KG, Vrabie D, Lewis FL (2013) Online adaptive algorithm for optimal control with integral reinforcement learning. Int J Robust Nonlinear Control, Wiley. doi: 10.1002/rnc.3018

    MATH  Google Scholar 

  • Vrabie D, Pastravanu O, Lewis FL, Abu-Khalaf M (2009) Adaptive optimal control for continuous-time linear systems based on policy iteration. Automatica 45(2):477–484

    Article  MATH  MathSciNet  Google Scholar 

  • Vrabie D, Vamvoudakis KG, Lewis FL (2012) Optimal adaptive control and differential games by reinforcement learning principles. Control engineering series. IET Press, London

    Google Scholar 

  • Werbos PJ (1989) Neural networks for control and system identification. In: Proceedings of the IEEE conference on decision and control, Tampa

    Google Scholar 

  • Werbos PJ (1992) Approximate dynamic programming for real-time control and neural modeling. In: White DA, Sofge DA (eds) Handbook of intelligent control. Van Nostrand Reinhold, New York

    Google Scholar 

Download references

Acknowledgements

This material is based upon the work supported by NSF. Grant Number: ECCS-1128050, ARO. Grant Number: W91NF-05-1-0314, AFOSR. Grant Number: FA9550-09-1-0278.

Author information

Authors and Affiliations

  1. Arlington Research Institute, University of Texas, 76118, Fort Worth, TX, USA

    Dr. Frank L. Lewis

  2. Center for Control, Dynamical-systems and Computation (CCDC), University of California, 93106-9560, Santa Barbara, CA, USA

    Kyriakos G. Vamvoudakis

Authors
  1. Dr. Frank L. Lewis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kyriakos G. Vamvoudakis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Frank L. Lewis .

Editor information

Editors and Affiliations

  1. Electrical and Computer Engineering, Boston University, Boston, Massachusetts, USA

    John Baillieul

  2. Automation and Control Solutions, Honeywell, Golden Valley, Minnesota, USA

    Tariq Samad

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer-Verlag London

About this entry

Cite this entry

Lewis, F.L., Vamvoudakis, K.G. (2014). Neural Control and Approximate Dynamic Programming. In: Baillieul, J., Samad, T. (eds) Encyclopedia of Systems and Control. Springer, London. https://doi.org/10.1007/978-1-4471-5102-9_224-2

Download citation

  • DOI: https://doi.org/10.1007/978-1-4471-5102-9_224-2

  • Received:

  • Accepted:

  • Published:

  • Publisher Name: Springer, London

  • Online ISBN: 978-1-4471-5102-9

  • eBook Packages: Springer Reference EngineeringReference Module Computer Science and Engineering

Publish with us

Policies and ethics

Chapter history

  1. Latest

    Neuro-Inspired Control
    Published:
    20 August 2020

    DOI: https://doi.org/10.1007/978-1-4471-5102-9_224-3

  2. Neural Control and Approximate Dynamic Programming
    Published:
    08 December 2014

    DOI: https://doi.org/10.1007/978-1-4471-5102-9_224-2

  3. Original

    Neural Control and Approximate Dynamic Programming
    Published:
    12 April 2014

    DOI: https://doi.org/10.1007/978-1-4471-5102-9_224-1

Search

Navigation