Inverse Reinforcement Learning Based on Behaviors of a Learning Agent

Abstract

Reinforcement learning agents can acquire the optimal policy to achieve their objectives based on trials and errors. An appropriate design of reward function is essential, because there are variety of reward functions for the same objective whereas different reward functions would give rise to different learning processes. There is no systematic way to determine a good reward function for a given environment and objective. One possible way is finding a reward function to imitate the learning strategy of a reference agent which is intelligent enough to efficiently adapt even variable environments. In this study, we extended the apprenticeship learning framework in order to imitate a learning reference agent, whose policy may change on the process of optimization. For the imitation above, we propose a new inverse reinforcement learning based on that agent’s history of states and actions. When mimicking a reference agent that was trained with a simple 2-state Markov decision process, the proposed method showed better performance than that by the apprenticeship learning.