Skip to main content
SpringerLink
Log in

Survey of imitation learning for robotic manipulation

  • Survey paper
  • Published:
International Journal of Intelligent Robotics and Applications Aims and scope Submit manuscript

Abstract

With the development of robotics, the application of robots has gradually evolved from industrial scenes to more intelligent service scenarios. For multitasking operations of robots in complex and uncertain environments, the traditional manual coding method is not only cumbersome but also unable to adapt to sudden changes in the environment. Imitation learning that avoids learning skills from scratch by using the expert demonstration has become the most effective way for robotic manipulation. The paper is intended to provide the survey of imitation learning of robotic manipulation and explore the future research trend. The review of the art of imitation learning for robotic manipulation involves three aspects that are demonstration, representation and learning algorithms. Towards the end of the paper, we highlight areas of future research potential.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  • Ajay, M., Zhu, Y., Li, F.: Robrurk: a crowdsourcing platform for robotic skill learning through imitation. In: Conference on Robot Learning, pp. 1–15 (2018)

  • Alibeigi, M., Ahmadabadi, M.N., Araabi, B.N.: A fast, robust, and incremental model for learning high-level concepts from human motions by imitation. IEEE Trans. Robot. 33(1), 153–168 (2017)

    Article  Google Scholar 

  • Amir, M., Matteo, R.: Robot learning from demonstrations: emulation learning in environments with moving obstacles. Robot. Auton. Syst. 101, 45–56 (2018)

    Article  Google Scholar 

  • Amor, H., Neumann, G., Kamthe, S., Kroemer, O., Peters, J.: Interaction primitives for human–robot cooperation tasks. In: IEEE International Conference on Robotics and Automation, pp. 2831–2837 (2014)

  • Andrew, J.: An invitation to imitation. Technical report, Robotics Institute, Carnegie Mellon University (2015)

  • Argall, B., Billard, A.: A survey of tactile human–robot interactions. Robot. Auton. Syst. 58, 1159–1176 (2010)

    Article  Google Scholar 

  • Attia, A., Dayan, S.: Global overview of imitation learning. https://arxiv.org/abs/1801.06503 (2018)

  • Baram, N., Anschel, O., Caspi, I., Mannor, S.: End-to-end differentiable adversarial imitation learning. In: International Conference on Machine Learning (ICML), pp. 390–399 (2017)

  • Cai, Q., Hong, M., Chen, Y., Wang, Z.: On the global convergence of imitation learning: a case for linear quadratic regulator. https://arxiv.org/abs/1901.03674 (2019)

  • Dermy, O., Charpillet, F., Ivaldi, S.: Multi-modal intention prediction with probabilistic movement primitives. In: International Workshop on Human-Friendly Robotics, pp. 181–196 (2017)

    Google Scholar 

  • Edmonds, M., Gao, F., Xie, X., Liu, H., Qi, S., Zhu, Y., Rothrock, B., Zhu, S.: Feeling the force: integrating force and pose for fluent discovery through imitation learning to open medicine bottles. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3530–3537 (2017)

  • Fahad, M., Chen, Z., Guo, Y.: Learning how to pedestrians navigate: a deep inverse reinforcement learning approach. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (2018)

  • Fang, B., Sun, F., Liu, H., Guo, D.: Development of a wearable device for motion capturing based on magnetic and inertial measurement units. Sci. Program. (2017a). https://doi.org/10.1155/2017/7594763

    Article  Google Scholar 

  • Fang, B., Sun, F., Liu, H., Guo, D.: Robotic teleoperation systems using a wearable multi-modal fusion device. Int. J. Adv. Robot. Syst. 1–11 (2017b)

  • Finn, C., Abbeel, P., Levine, S.: Model-agnostic meta-learning for fast adaptation of deep networks. In: International Conference on Machine Learning (ICML), pp. 1126–1135 (2017)

  • Finn, C., Levine, S., Abbeel, P.: Guided cost learning: deep inverse optimal control via policy optimization. In: International Conference on Machine Learning, pp. 49–58 (2016)

  • Gams, A., Nemec, B., Ijspeert, A.J., Ude, A.: Coupling movement primitives: interaction with the environment and bimanual tasks. IEEE Trans. Robot. 30(4), 816–830 (2014)

    Article  Google Scholar 

  • Gaspar, T., Nemec, B., Morimoto, J., Ude, A.: Skill learning and action recognition by arc-length dynamic movement primitives. Robot. Auton. Syst. 100, 225–235 (2018)

    Article  Google Scholar 

  • Gong, D., Zhao, J., Yu, J., Zuo, G.: Motion mapping of the heterogeneous master–slave system for intuitive telemanipulation. Int. J. Adv. Rob. Syst. 15, 1–9 (2018)

    Google Scholar 

  • Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: International Conference on Neural Information Processing Systems, pp. 2672–2680 (2014)

  • Henderson, P., Chang, W., Bacon, P., Meger, D., Pineau, J., Precup, D.: OptionGan: learning joint reward-policy options using generative adversarial inverse reinforcement learning. In: National conference on Artificial Intelligence (2018)

  • Ho, J., Ermon, S.: Generative adversarial imitation learning. Adv. Neural Inf. Process. Syst. (2016). https://arXiv.org/abs/1606.03476

  • Hussein, A., Gaber, M.M., Elyan, E., Jayne, C.: Imitation learning: a survey of learning methods. ACM Comput. Surv. (2017). https://doi.org/10.1145/3054912

    Article  Google Scholar 

  • Hwang, C., Chen, B., Syu, H., Wang, C., Karkoub, M.: Humanoid robot’s visual imitation of 3-D motion of a human subject using neural network based inverse kinematics. IEEE Syst. J. 10(2), 685–696 (2016)

    Article  Google Scholar 

  • Ijspeert, A., Nakanishi, J., Hoffmann, H., Pastor, P., Schaal, S.: Dynamical movement primitives: learning attractor models for motor behaviors. Neural Comput. 25(2), 328–373 (2013)

    Article  MathSciNet  Google Scholar 

  • Jin, H., Chen, Q., Chen, Z., Hu, Y., Zhang, J.: Multi-leap motion sensor based demonstration for robotic refine tabletop object manipulation task. Trans. Intell. Technol. 1, 104–113 (2016)

    Google Scholar 

  • Justin, F., Luo, K., Levine, S.: Learning robust rewards with adversarial inverse reinforcement learning. In: International Conference on Learning Representations, pp. 1–15 (2018)

  • Kuefler, A., Morton, J., Wheeler, T.: Imitating driver behavior with generative adversarial networks. In: IEEE Intelligent Vehicles Symposium, pp. 204–211 (2017)

  • Khansari-Zadeh, S.M., Billard, A.: Learning stable nonlinear dynamical systems. IEEE Trans. Robot. 27, 943–957 (2011)

    Article  Google Scholar 

  • Kumar, V., Gupta, A., Todorov, E., Levine, S.: Learning dexterous manipulation policies from experience and imitation. https://arxiv.org/abs/1611.05095 (2016)

  • Liu, H., Wang, L.: Gesture recognition for human–robot collaboration: a review. Int. J. Ind. Ergon. 68, 355–367 (2018)

    Article  Google Scholar 

  • Liu, H., Zhang, C., Zhu, Y., Jiang, C., Zhu, S.: Mirroring without overimitation: learning functionally equivalent manipulation actions. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 33 (2019). https://doi.org/10.1609/aaai.v33i01.33018025

    Article  Google Scholar 

  • Montaser, M., Waleed, D., Benjamin, R.: Transfer learning for prosthetics using imitation learning. https://arxiv.org/abs/1901.04772 (2019)

  • Osa, T., Pajarinen, J., Neumann, G., Bagnell, J.A., Abbeel, P., Peters, J.: An algorithmic perspective on imitation learning. Found. Trends Robot. 7(1), 1–179 (2017)

    Google Scholar 

  • Pastor, P., Hoffmann, H., Asfour, T., Schaal, S.: Learning and generalization of motor skills by learning from demonstration. In: IEEE International Conference on Robotics and Automation (2009)

  • Piot, B., Geist, M., Pietquin, O.: Bridging the gap between imitation learning and inverse reinforcement learning. IEEE Trans. Neural Netw. Learn. Syst. 28, 1814–1826 (2016)

    Article  MathSciNet  Google Scholar 

  • Rajeswaran, A., Kumar, V., Gupta, A., Vezzani, G., Schulman, J., Todorov, E., Levine, S.: Learning complex dexterous manipulation with deep reinforcement learning and demonstrations. Robot. Sci. Syst. (2018). https://doi.org/10.15607/RSS.2018.XIV.049

  • Ratliff, N., Bagnell, J., Zinkevich, M.. Maximum margin planning. In: International Conference on Machine learning (ICML), pp. 729–736 (2006)

  • Schreiber, G., Stemmer, A., Bischoff, R.: The fast research interface for the KUKA lightweight robot. In: The Workshop on IEEE ICRA 2010 Workshop on Innovative Robot Control Architectures for Demanding, pp. 15–21 (2010)

  • Sermanet, P., Lynch, C., Hsu, J., Levine, S.: Time-contrastive networks: self-supervised learning from video. In: IEEE International Conference on Robotics and Automation, pp. 1134–1141 (2018)

  • Torabi, F., Warnell, G., Stone, P.: Behavioral cloning from observation. In: International Joint Conference on Artificial Intelligence, pp. 4950–4957 (2018)

  • Wan, W., Lu, F., Wu, Z., Harada, K.: Teaching robots to do object assembly using multi-modal 3D vision. Neurocomputing 259, 85–93 (2017)

    Article  Google Scholar 

  • Wu, Y., Charoenphakdee, N., Bao, H., Tangkaratt, V., Sugiyama, M., Imitation learning from imperfect demonstration. https://arxiv.org/abs/1901.09387 (2019)

  • Yang, C., Lu, F., Wu, Z., Harada, K.: Development of a robotic teaching interface for human to human skill transfer. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 710–716 (2016)

  • Yang, C., Zeng, C., Liang, P., Li, Z., Li, R., Sun, C.: Interface design of a physical human–robot interaction system for human impedance adaptive skill transfer. In: IEEE Robotics and Automation Society, pp. 329–340 (2017)

    Article  Google Scholar 

  • Yang, C., Wang, X., Cheng, L., Ma, H.: Neural-learning-based telerobot control with guaranteed performance. IEEE Trans. Cybern. 47, 3148–3159 (2017)

    Article  Google Scholar 

  • Yang, C., Zeng, C., Fang, C., He, W., Li, Z.: A DMPs-based framework for robot learning and generalization of humanlike variable impedance skills. IEEE/ASME Trans. Mechatron. 23, 1193–1203 (2018)

    Article  Google Scholar 

  • Zhang, T., Mccarthy, Z., Jow, O., Lee, D., Goldberg, K., Abbeel, P.: Deep imitation learning for complex manipulation tasks from virtual reality teleoperation. In: IEEE International Conference on Robotics and Automation, pp. 5628–5635 (2018)

  • Ziebart, B., Maas, A., Bagnell, J.: Maximum entropy inverse reinforcement learning. In: AAAI Conference on Artificial Intelligence, pp. 1433–1438 (2008)

  • Zucker, M., Ratliff, N., Stolle, M., Chestnutt, J., Andrew Bagnell, J., Atkeson, C., Kuffner, J.: Optimization and learning for rough terrain legged locomotion. Int. J. Robot. Res. 30(2), 175–191 (2011)

    Article  Google Scholar 

Download references

Acknowledgements

This work is jointly supported by Foshan-Tsinghua industry-university-research cooperation collaborative innovation special fund no. 2018THFS04, Tsinghua University Initiative Scientific Research Program no. 2019Z08QCX15, National Natural Science Foundation of China under with Grant nos. 91848206, U1613212 and 61703284.

Author information

Authors and Affiliations

  1. Department of Computer Science and Technology, Beijing National Research Center for Information Science and Technology, The Institute for Artificial Intelligence, State Key Lab of Intelligent Technology and Systems, Tsinghua University, Beijing, 100084, People’s Republic of China

    Bin Fang, Di Guo & Fuchun Sun

  2. Key Lab of Industrial Computer Control Engineering of Hebei Province, Yanshan University, Qinhuangdao, 066004, People’s Republic of China

    Shidong Jia & Muhua Xu

  3. Princeton International School of Mathematics and Science, 19 Lambert Drive, Princeton, NJ, 08540, USA

    Shuhuan Wen

Authors
  1. Bin Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shidong Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Di Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Muhua Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shuhuan Wen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Fuchun Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bin Fang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fang, B., Jia, S., Guo, D. et al. Survey of imitation learning for robotic manipulation. Int J Intell Robot Appl 3, 362–369 (2019). https://doi.org/10.1007/s41315-019-00103-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s41315-019-00103-5

Keywords

Search

Navigation