Skip to main content
SpringerLink
Log in

Near-Optimal Control with Input Saturation

  • Chapter
  • First Online:
Deep Reinforcement Learning with Guaranteed Performance

Part of the book series: Studies in Systems, Decision and Control ((SSDC,volume 265))

  • 1332 Accesses

Abstract

In this chapter, a time-scale expansion-based scheme is presented for approximately solving the optimal control problem of continuous-time underactuated nonlinear systems subject to input constraints and system dynamics. By time-scale Taylor approximation of the original performance index, the optimal control problem is relaxed into an approximated optimal control problem. Based on the system dynamics, the problem is further reformulated as a quadratic program, which is solved by a projection neural network. Theoretical analysis on the closed-loop system synthesized by the controlled system and the projection neural network is conducted, which reveals that, under certain conditions, the closed-loop system possesses exponential stability and the original performance index converges to zero as time tends to infinity. In addition, two illustrative examples, which are based on a flexible joint manipulator and an underactuacted ship, are provided to validate the theoretical results and demonstrate the efficacy and superiority of the presented control scheme.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 139.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Liu, Y.-J., Tong, S.: Optimal control-based adaptive NN design for a class of nonlinear discrete-time block-triangular systems. IEEE Trans. Cybern. 46(11), 2670–2680 (2016)

    Article  Google Scholar 

  2. Liu, Y.-J., Gao, Y., Tong, S., Li, Y.: Fuzzy approximation-based adaptive backstepping optimal control for a class of nonlinear discrete-time systems with dead-zone. IEEE Trans. Fuzzy Syst. 24(1), 16–28 (2016)

    Article  Google Scholar 

  3. Lewis, F.L., Syrmos, V.L.: Optimal Control. Wiley, New York (1995)

    Google Scholar 

  4. Chen, Z., Jagannathan, S.: Generalized Hamilton–Jacobi–Bellman formulation-based neural network control of affine nonlinear discrete-time systems. IEEE Trans. Neural Netw. 19(1), 90–106 (2008)

    Article  Google Scholar 

  5. Mayne, D.Q.: Model predictive control: recent developments and future promise. Automatica 50(12), 2967–2986 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  6. Chen, W.H., Ballance, D.J., Gawthrop, P.J.: Optimal control of nonlinear systems: a predictive control approach. Automatica 39(4), 633–641 (2003)

    MathSciNet  MATH  Google Scholar 

  7. Hedjar, R., Toumi, R., Boucher, P., Dumur, D.: Finite horizon nonlinear predictive control by the Taylor approximation application to robot tracking trajectory. Int. J. Appl. Math. Comput. Sci. 15(4), 527–540 (2005)

    Google Scholar 

  8. Mayne, D.Q., Michalska, H.: Receding horizon control of nonlinear systems. IEEE Trans. Autom. Contro 35(7), 814–824 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  9. Han, D., Shi, L.: Guaranteed cost control of affine nonlinear systems via partition of unity method. Automatica 49(2), 660–666 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  10. Lian, C., Xu, X., Chen, H., He, H.: Near-optimal tracking control of mobile robots via receding-horizon dual heuristic programming. IEEE Trans. Cybern. 46(11), 2484–2496 (2016)

    Article  Google Scholar 

  11. Li, H., Shi, Y., Yan, W.: On neighbor information utilization in distributed receding horizon control for consensus-seeking. IEEE Trans. Cybern. 46(9), 2019–2027 (2016)

    Article  Google Scholar 

  12. Yang, Q., Jagannathan, S.: Reinforcement learning controller design for affine nonlinear discrete-time systems using online approximators. IEEE Trans. Syst. Man Cybern. B Cybern. 42(2), 377–390 (2012)

    Google Scholar 

  13. Du, H., Zhang, N.: Fuzzy control for nonlinear uncertain electrohydraulic active suspensions with input constraint. IEEE Trans. Fuzzy Syst. 17(2), 343–356 (2009)

    Article  Google Scholar 

  14. Liu, Y.-J., Gao, Y., Tong, S., Philip Chen, C.L.: A unified approach to adaptive neural control for nonlinear discrete-time systems with nonlinear dead-zone input. IEEE Trans. Neural Netw. Learn. Syst. 27(1), 39–150 (2016)

    Article  MathSciNet  Google Scholar 

  15. Ma, Y., Borrelli, F., Hencey, B., Coffey, B., Bengea, S., Haves, P.: Model predictive control for the operation of building cooling systems. IEEE Trans. Control Syst. Technol. 20(3), 796–803 (2012)

    Google Scholar 

  16. Han, H.G., Wu, X.L., Qiao, J.F.: Real-time model predictive control using a self-organizing neural network. IEEE Trans. Neural Netw. Learn. Syst. 24(9), 1425–1436 (2013)

    Google Scholar 

  17. Lu, C.H.: Design and application of stable predictive controller using recurrent wavelet neural networks. IEEE Trans. Ind. Electron. 56(9), 3733–3742 (2009)

    Article  Google Scholar 

  18. Peng, H., Nakano, K., Shioya, H.: Nonlinear predictive control using neural nets-based local linearization ARX model-stability and industrial application. IEEE Trans. Control Syst. Techno. 15(1), 130–143 (2017)

    Article  Google Scholar 

  19. Li, Z., Xiao, H., Yang, C., Zhao, Y.: Model predictive control of nonholonomic chained systems using general projection neural networks optimization. IEEE Trans. Syst. Man Cybern. Syst. 45(10), 1313–1321 (2015)

    Article  Google Scholar 

  20. Lu, P.: Constrained tracking control of nonlinear systems. Automatica 27(5), 305–314 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  21. Li, X., Rakkiyappan, R., Velmurugan, G.: Dissipativity analysis of memristor-based complex-valued neural networks with time-varying delays. Inform. Sci. 294, 645–665 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  22. Zhang, Y., Ge, S.S., Lee, T.H.: A unified quadratic-programming-based dynamical system approach to joint torque optimization of physically constrained redundant manipulators. IEEE Trans. Syst. Man Cybern. B 34(5), 2126–2132 (2004)

    Article  Google Scholar 

  23. Li, X., Regan, D.O., Akca, H.: Global exponential stabilization of impulsive neural networks with unbounded continuously distributed delays. IMA J. Appl. Math. 80(1), 85–99 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  24. Jin, L., Zhang, Y., Li, S., Zhang, Y.: Modified ZNN for time-varying quadratic programming with inherent tolerance to noises and its application to kinematic redundancy resolution of robot manipulators. IEEE Trans. Ind. Electron. 63(11), 6978–6988 (2016)

    Article  Google Scholar 

  25. Xiao, L.: A nonlinearly-activated neurodynamic model and its finite-time solution to equality-constrained quadratic optimization with nonstationary coefficients. Appl. Soft Comput. 40, 252–259 (2016)

    Article  Google Scholar 

  26. Qin, S., Xue, X.: A two-layer recurrent neural network for nonsmooth convex optimization problems. IEEE Trans. Neural Netw. Learn. Syst. 26(6), 1149–1160 (2015)

    Google Scholar 

  27. Zhang, Z., Zhang, Y.: Variable joint-velocity limits of redundant robot manipulators handled by quadratic programming. IEEE/ASME Trans. Mechatronics 18(2), 674–686 (2013)

    Article  Google Scholar 

  28. Li, X., Song, S.: Impulsive control for existence, uniqueness and global stability of periodic solutions of recurrent neural networks with discrete and continuously distributed delays. IEEE Trans. Neural Netw. 24, 868–877 (2013)

    Google Scholar 

  29. Li, S., Chen, S., Liu, B., Li, Y., Liang, Y.: Decentralized kinematic control of a class of collaborative redundant manipulators via recurrent neural networks. Neurocomputing 91, 1–10 (2012)

    Article  Google Scholar 

  30. Li, S., Cui, H., Li, Y., Liu, B., Lou, Y.: Decentralized control of collaborative redundant manipulators with partial command coverage via locally connected recurrent neural networks. Neural Comput. Appl. 23(3), 1051–1060 (2013)

    Article  Google Scholar 

  31. Li, S., He, J., Li, Y., Rafique, M.U.: Distributed recurrent neural networks for cooperative control of manipulators: A game-theoretic perspective. IEEE Trans. Neural Netw. Learn. Syst. 28(2), 415–426 (2017)

    Article  MathSciNet  Google Scholar 

  32. Jin, L., Li, S., La, H.M., Luo, X.: Manipulability optimization of redundant manipulators using dynamic neural networks. IEEE Trans. Ind. Electron. 64(6), 4710–4720 (2017)

    Article  Google Scholar 

  33. Li, Y., Li, S., Hannaford, B.: A novel recurrent neural network for improving redundant manipulator motion planning completeness. ICRA, 2956–2961 (2018)

    Google Scholar 

  34. Zhang, Y., Li, S.: A neural controller for image-based visual servoing of manipulators with physical constraints. IEEE Trans. Neural Netw. Learning Syst. 29(11), 5419–5429 (2018)

    Article  MathSciNet  Google Scholar 

  35. Li, S., Zhou, M., Luo, X.: Modified primal-dual neural networks for motion control of redundant manipulators with dynamic rejection of harmonic noises. IEEE Trans. Neural Netw. Learning Syst. 29(10), 4791–4801 (2018)

    Article  MathSciNet  Google Scholar 

  36. Li, S., Wang, H., Rafique, M.U.: A novel recurrent neural network for manipulator control with improved noise tolerance. IEEE Trans. Neural Netw. Learning Syst. 29(5), 1908–1918 (2018)

    Article  MathSciNet  Google Scholar 

  37. Jin, L., Li, S., Luo, X., Li, Y., Qin, B.: Neural dynamics for cooperative control of redundant robot manipulators. IEEE Trans. Ind. Inform. 14(9), 3812–3821 (2018)

    Article  Google Scholar 

  38. Li, J., Zhang, Y., Li, S., Mao, M.: New discretization-formula-based zeroing dynamics for real-time tracking control of serial and parallel manipulators. IEEE Trans. Ind. Inform. 14(8), 3416–3425 (2018)

    Article  Google Scholar 

  39. Chen, D., Zhang, Y., Li, S.: Tracking control of robot manipulators with unknown models: A jacobian-matrix-adaption method. IEEE Trans. Ind. Inform. 14(7), 3044–3053 (2018)

    Article  Google Scholar 

  40. Zhang, Y., Li, S., Gui, J., Luo, X.: Velocity-level control with compliance to acceleration-level constraints: A novel scheme for manipulator redundancy resolution. IEEE Trans. Ind. Inform. 14(3), 921–930 (2018)

    Article  Google Scholar 

  41. Xiao, L., Liao, B., Li, S., Zhang, Z., Ding, L., Jin, L.: Design and analysis of FTZNN applied to the real-time solution of a nonstationary Lyapunov equation and tracking control of a wheeled mobile manipulator. IEEE Trans. Ind. Inform. 14(1), 98–105 (2018)

    Article  Google Scholar 

  42. Zhang, Y., Chen, S., Li, S., Zhang, Z.: Adaptive projection neural network for kinematic control of redundant manipulators with unknown physical parameters. IEEE Trans. Ind. Electron. 65(6), 4909–4920 (2018)

    Article  Google Scholar 

  43. Zhang, Z., Lin, Y., Li, S., Li, Y., Yu, Z., Luo, Y.: Tricriteria optimization-coordination motion of dual-redundant-robot manipulators for complex path planning. IEEE Trans. Control Syst. Technol. 26(4), 1345–1357 (2018)

    Article  Google Scholar 

  44. Jin, L., Li, S., Hu, B., Yi, C.: Dynamic neural networks aided distributed cooperative control of manipulators capable of different performance indices. Neurocomputing 291, 50–58 (2018)

    Article  Google Scholar 

  45. Jin, L., Li, S., Yu, J., He, J.: Robot manipulator control using neural networks: a survey. Neurocomputing 285, 23–34 (2018)

    Article  Google Scholar 

  46. Chen, D., Zhang, Y., Li, S.: Zeroing neural-dynamics approach and its robust and rapid solution for parallel robot manipulators against superposition of multiple disturbances. Neurocomputing 275, 845–858 (2018)

    Article  Google Scholar 

  47. Li, S., Shao, Z., Guan, Y.: A dynamic neural network approach for efficient control of manipulators. IEEE Trans. Syst. Man Cybern. Syst. 49(5), 932–941 (2019)

    Article  Google Scholar 

  48. Zhang, Y., Li, S., Zhou, X.: Recurrent-neural-network-based velocity-level redundancy resolution for manipulators subject to a joint acceleration limit. IEEE Trans. Ind. Electron. 66(5), 3573–3582 (2019)

    Article  Google Scholar 

  49. Zhang, Z., Chen, S., Li, S.: Compatible convex-nonconvex constrained QP-based dual neural networks for motion planning of redundant robot manipulators. IEEE Trans. Control Syst. Technol. 27(3), 1250–1258 (2019)

    Article  Google Scholar 

  50. Xu, Z., Li, S., Zhou, X., Yan, W., Cheng, T., Huang, D.: Dynamic neural networks based kinematic control for redundant manipulators with model uncertainties. Neurocomputing 329, 255–266 (2019)

    Article  Google Scholar 

  51. Li, S., Zhang, Y., Jin, L.: Kinematic control of redundant manipulators using neural networks. IEEE Trans. Neural Netw. Learn. Syst. (In Press)

    Google Scholar 

  52. Zhang, Y., Wang, J., Xia, Y.: A dual neural network for redundancy resolution of kinematically redundant manipulators subject to joint limits and joint velocity limits. IEEE Trans. Neural Netw. 14(3), 658–667 (2003)

    Article  Google Scholar 

  53. Zhang, Z., Li, Z., Zhang, Y., Luo, Y., Li, Y.: Neural-dynamic-method-based dual-arm CMG scheme with time-varying constraints applied to humanoid robots. IEEE Trans. Neural Netw. Learn. Syst. 26(12), 3251–3262 (2015)

    Article  MathSciNet  Google Scholar 

  54. Zhang, Y., Li, W., Zhang, Z.: Physical-limits-constrained minimum velocity norm coordinating scheme for wheeled mobile redundant manipulators. Robotica 33(6), 1325–1350 (2015)

    Article  Google Scholar 

  55. Mohammed, A.M., Li, S.: Dynamic neural networks for kinematic redundancy resolution of parallel stewart platforms. IEEE Trans. Cybern. 46(7), 1538–1550 (2016)

    Article  Google Scholar 

  56. Jin, L., Zhang, Y.: G2-type SRMPC scheme for synchronous manipulation of two redundant robot Arms. IEEE Trans. Cybern. 45(2), 153–164 (2015)

    Article  MathSciNet  Google Scholar 

  57. Jin, L., Li, S., Xiao, L., Lu, R., Liao, B.: Cooperative motion generation in a distributed network of redundant robot manipulators with noises. IEEE Trans. Syst. Man Cybern. Syst. (In Press). https://doi.org/10.1109/TSMC.2017.2693400

    Article  Google Scholar 

  58. Li, S., Shao, Z., Guan, Y.: A dynamic neural network approach for efficient control of manipulators. IEEE Trans. Syst. Man Cybern. Syst. (In Press). https://doi.org/10.1109/TSMC.2017.2690460

    Article  Google Scholar 

  59. Jin, L., Li, S.: Distributed task allocation of multiple robots: a control perspective. IEEE Trans. Syst. Man Cybern. Syst. (In Press). https://doi.org/10.1109/TSMC.2016.2627579

    Article  Google Scholar 

  60. Chen, C.L.P., Wen, G.-X., Liu, Y.-J., Wang, F.-Y.: Adaptive consensus control for a class of nonlinear multiagent time-delay systems using neural networks. IEEE Trans. Neural Netw. Learn. Syst. 25(6), 1217–1226 (2014)

    Article  Google Scholar 

  61. Wang, H.Q., Liu, X.P., Liu, K.F.: Robust adaptive neural tracking control for a class of stochastic nonlinear interconnected systems. IEEE Trans. Neural Netw. Learn. Syst. 27(3), 510–523 (2016)

    Article  MathSciNet  Google Scholar 

  62. Chen, C.L.P., Liu, Y.-J., Wen, G.-X.: Fuzzy neural network-based adaptive control for a class of uncertain nonlinear stochastic systems. IEEE Trans. Cybern. 44(5), 583–593 (2014)

    Article  Google Scholar 

  63. Liu, Y.-J., Tang, L., Tong, S., Chen, C.L.P.: Adaptive NN controller design for a class of nonlinear MIMO discrete-time systems. IEEE Trans. Neural Netw. Learn. Syst. 26(5), 1007–1018 (2015)

    Article  MathSciNet  Google Scholar 

  64. Liu, Y.-J., Tang, L., Tong, S., Chen, C.L.P., Li, D.-J.: Reinforcement learning design-based adaptive tracking control with less learning parameters for nonlinear discrete-time MIMO systems. IEEE Trans. Neural Netw. Learn. Syst. 26(1), 165–176 (2015)

    Article  MathSciNet  Google Scholar 

  65. Li, T., Duan, S., Liu, J., Wang, L., Huang, T.: A spintronic memristor-based neural network with radial basis function for robotic manipulator control implementation. IEEE Trans. Syst. Man Cybern. Syst. 46(4), 582–588 (2016)

    Article  Google Scholar 

  66. Khalil, H.K.: Nonlinear Systems, 3rd edn. Prentice-Hall, New Jersey (2002)

    MATH  Google Scholar 

  67. Gao, Y., Wang, H., Liu, Y.-J.: Adaptive fuzzy control with minimal leaning parameters for electric induction motors. Neurocomputing 156, 143–150 (2015)

    Article  Google Scholar 

  68. Wang, H., Chen, B., Liu, X., Liu, K., Lin, C.: Robust adaptive fuzzy tracking control for pure-feedback stochastic nonlinear systems with input constraints. IEEE Trans. Cybern. 43(6), 2093–2104 (2013)

    Article  Google Scholar 

  69. Isidori, A.: Nonlinear Control Systems: An Introduction, 3rd edn. Springer, New York (1995)

    Book  MATH  Google Scholar 

  70. Xia, Y.: Further results on global convergence and stability of globally projected dynamical systems. J. Optim. Theory Appl. 122(3), 627–649 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  71. Wang, H., Shi, P., Li, H., Zhou, Q.: Adaptive neural tracking control for a class of nonlinear systems with dynamic uncertainties. IEEE Trans. Cybern. (In Press)

    Google Scholar 

  72. Chen, C.L.P., Wen, G.-X., Liu, Y.-J., Liu, Z.: Observer-based adaptive backstepping consensus tracking control for high-order nonlinear semi-strict-feedback multiagent systems. IEEE Trans. Cybern. 46(7), 1591–1601 (2016)

    Article  Google Scholar 

  73. Liu, Y.-J., Tong, S.: Barrier Lyapunov functions-based adaptive control for a class of nonlinear pure-feedback systems with full state constraints. Automatica 64, 70–75 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  74. He, W., Ge, W., Li, Y., Liu, Y.-J., Yang, C., Sun, C.: Model identification and control design for a humanoid robot. IEEE Trans. Syst. Man Cybern. Syst. 47(1), 45–57 (2017)

    Article  Google Scholar 

  75. Wang, H., Liu, X., Liu, K.: Adaptive fuzzy tracking control for a class of pure-feedback stochastic nonlinear systems with non-lower triangular structure. Fuzzy Set Syst. 302, 101–120 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  76. Chien, T., Chen, C., Huang, C.: Feedback linearization control and its application to mimo cancer immunotherapy. IEEE Trans. Control Syst. Technol. 18(4), 953–961 (2010)

    Article  Google Scholar 

  77. Zhang, Y., Li, S.: Predictive suboptimal consensus of multiagent systems with nonlinear dynamics. IEEE Trans. Syst. Man Cybern. Syst. (In Press). https://doi.org/10.1109/TSMC.2017.2668440

    Article  MathSciNet  Google Scholar 

  78. Allen, L.J.S.: An Introduction to Mathematical Biology. Pearson Education, New Jersey (2007)

    Google Scholar 

  79. Groves, K., Serrani, A.: Modeling and nonlinear control of a single-link flexible joint manipulator (2004)

    Google Scholar 

  80. Jiang, Z.: Global tracking control of underactuated ships by Lyapunov’s direct method. Automatica 38(2), 301–309 (2002)

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Cyber Security, Jinan University, Guangzhou, China

    Yinyan Zhang

  2. School of Information Science and Engineering, Lanzhou University, Lanzhou, China

    Shuai Li

  3. Guangdong Institute of Intelligent Manufacturing, Guangdong Academy of Science, Guangzhou, China

    Xuefeng Zhou

Authors
  1. Yinyan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shuai Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xuefeng Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shuai Li .

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Zhang, Y., Li, S., Zhou, X. (2020). Near-Optimal Control with Input Saturation. In: Deep Reinforcement Learning with Guaranteed Performance. Studies in Systems, Decision and Control, vol 265. Springer, Cham. https://doi.org/10.1007/978-3-030-33384-3_2

Download citation

Publish with us

Policies and ethics

Search

Navigation