The Relationship Between the Maximum Principle and Dynamic Programming

Abstract

In Chapters 3 and 4 we studied Pontryagin’s maximum principle (MP, for short) and Bellman’s dynamic programming (DP, for short). These two approaches serve as two of the most important tools in solving optimal control problems. Both MP and DP can be regarded as some necessary conditions of optimal controls (under certain conditions, they become sufficient ones). An interesting phenomenon one can observe from the literature is that to a great extent these two approaches have been developed separately and independently. Hence, a natural question arises: Are there any relations between these two? In this chapter we are going to address this question.