Skip to main content
Log in

Fast motion planning from experience: trajectory prediction for speeding up movement generation

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

Trajectory planning and optimization is a fundamental problem in articulated robotics. Algorithms used typically for this problem compute optimal trajectories from scratch in a new situation. In effect, extensive data is accumulated containing situations together with the respective optimized trajectories—but this data is in practice hardly exploited. This article describes a novel method to learn from such data and speed up motion generation, a method we denote tajectory pediction. The main idea is to use demonstrated optimal motions to quickly predict appropriate trajectories for novel situations. These can be used to initialize and thereby drastically speed-up subsequent optimization of robotic movements. Our approach has two essential ingredients. First, to generalize from previous situations to new ones we need a situation descriptor—we construct features for such descriptors and use a sparse regularized feature selection approach to improve generalization. Second, the transfer of previously optimized trajectories to a new situation should not be made in joint angle space—we propose a more efficient task space transfer. We present extensive results in simulation to illustrate the benefits of the new method, and demonstrate it also with real robot hardware. Our experiments in diverse tasks show that we can predict good motion trajectories in new situations for which the refinement is much faster than an optimization from scratch.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. This is not to be confused with a reactive controller which maps the current sensor state to the current control signal—such a (temporally local) reactive controller could not explain trajectories which efficiently circumvent obstacles in an anticipatory way, as humans naturally do in complex situations.

References

  • Argall, B. D., Chernova, S., Veloso, M. M., & Browning, B. (2009). A survey of robot learning from demonstration. Robotics and Autonomous Systems, 57(5), 469–483.

    Article  Google Scholar 

  • Atkeson, C. G. (1993). Using local trajectory optimizers to speed up global optimization in dynamic programming. In: NIPS (pp. 663–670).

  • Branicky, M., Knepper, R., & Kuffner, J. (2008). Path and trajectory diversity: Theory and algorithms. In IEEE International Conference on Robotics and Automation (ICRA) (pp. 1359–1364).

  • Bruce, J., & Veloso, M. (2002). Real-time randomized path planning for robot navigation. In International Conference on Intelligent Robots and Systems (IROS), Switzerland.

  • Calinon, S., & Billard, A. (2005). Recognition and reproduction of gestures using a probabilistic framework combining PCA, ICA and HMM. In 22nd International Conference on Machine Learning (ICML) (pp. 105–112).

  • Call, J., & Carpenter, M. (2002). Three sources of information in social learning. In K. Dautenhahn & C. L. Nehaniv (Eds.), Imitation in animals and artifacts (pp. 211–228). Cambridge, MA: MIT Press.

  • Dyer, P., & McReynolds, S. R. (1970). The computation and theory of optimal control. New York: Elsevier.

    MATH  Google Scholar 

  • Elfes, A. (1989). Using occupancy grids for mobile robot perception and navigation. Computer, 22(6), 46–57.

    Article  Google Scholar 

  • Hiraki, K., Sashima, A., & Phillips, S. (1998). From egocentric to allocentric spatial behavior: A computational model of spatial development. Adaptive Behavior, 6(3–4), 371–391.

    Article  Google Scholar 

  • Jetchev, N. (2012). Learning representations from motion trajectories: Analysis and applications to robot planning and control. PhD Thesis, FU Berlin. Retrieved August 1, 2012 from http://www.diss.fu-berlin.de/diss/receive/FUDISS_thesis_000000037417.

  • Jetchev, N., & Toussaint, M. (2009). Trajectory prediction: Learning to map situations to robot trajectories. In 26th International Conference on Machine Learning (ICML) (pp. 449–456).

  • Jetchev, N., & Toussaint, M. (2010). Trajectory prediction in cluttered voxel environments. In International Conference on Robotics and Automation (ICRA) (pp. 2523–2528).

  • Jiang, X., & Kallmann, M. (2007). Learning humanoid reaching tasks in dynamic environments. In International Conference on Intelligent Robots and Systems (IROS) (pp. 1148–1153).

  • Kober, J., Oztop, E., & Peters, J. (2010). Reinforcement learning to adjust robot movements to new situations. In Robotics: Science and Systems.

  • Konidaris, G., & Barto, A. (2006). Autonomous shaping: Knowledge transfer in reinforcement learning. In: 23rd International Conference on Machine Learning (ICML) (pp. 489–496).

  • Lampariello, R., Nguyen-Tuong, D., Castellini, C., Hirzinger, G., & Peters, J. (2011). Trajectory planning for optimal robot catching in real-time. In IEEE International Conference on Robotics and Automation (ICRA) (pp. 3719–3726).

  • LaValle, S. M. (2006). Planning algorithms. Cambridge: Cambridge University Press. Retrieved January 5, 2012 from http://planning.cs.uiuc.edu/.

  • Martin, S., Wright, S., & Sheppard, J. (2007). Offline and online evolutionary bi-directional RRT algorithms for efficient re-planning in dynamic environments. In IEEE International Conference on Automation Science and Engineering (CASE) (pp. 1131–1136).

  • McGovern, A., & Sutton, R. S. (1998). Macro-actions in reinforcement learning: An empirical analysis. Technical Report 98–70. Amherst, MA: University of Massachusetts.

  • Muehlig, M., Gienger, M., Steil, J. J., & Goerick, C. (2009). Automatic selection of task spaces for imitation learning. In International Conference on Intelligent Robots and Systems (IROS) (pp. 4996–5002).

  • Nakhaei, A., & Lamiraux, F. (2008). Motion planning for humanoid robots in environments modeled by vision. In 8th IEEE-RAS International Conference on Humanoid Robots (pp. 197–204).

  • Peshkin, L., & de Jong, E. D. (2002). Context-based policy search: Transfer of experience across problems. In ICML-2002 Workshop on Development of Representations.

  • Pomerleau, D. A. (1991). Efficient training of artificial neural networks for autonomous navigation. Neural Computation, 3, 88–97.

    Article  Google Scholar 

  • Ratliff, N., Zucker, M., Bagnell, A., & Srinivasa, S. (2009). Chomp: Gradient optimization techniques for efficient motion planning. In IEEE International Conference on Robotics and Automation (ICRA).

  • Shon, A., Storz, J., & Rao, R. (2007). Towards a real-time Bayesian imitation system for a humanoid robot. In IEEE International Conference on Robotics and Automation (ICRA) (pp. 2847–2852).

  • Stolle, M., & Atkeson, C. (2007). Transfer of policies based on trajectory libraries. In International Conference on Intelligent Robots and Systems (IROS) (pp. 2981–2986).

  • Todorov, E., & Li, W. (2005). A generalized iterative LQG method for locally-optimal feedback control of constrained nonlinear stochastic systems. In Proceedings of the American Control Conference (Vol. 1, pp. 300–306).

  • Toussaint, M. (2009). Robot trajectory optimization using approximate inference. In 26th International Conference on Machine Learning (ICML) (pp. 1049–1056).

  • Ude, A., Gams, A., Asfour, T., & Morimoto, J. (2010). Task-specific generalization of discrete and periodic dynamic movement primitives. IEEE Transactions on Robotics, 26(5), 800–815.

    Article  Google Scholar 

  • Wagner, T., Visser, U., & Herzog, O. (2004). Egocentric qualitative spatial knowledge representation for physical robots. Robotics and Autonomous Systems, 49(1–2), 25–42.

    Article  Google Scholar 

  • Zacharias, F., Borst, C., & Hirzinger, G. (2007). Capturing robot workspace structure: Representing robot capabilities. In International Conference on Intelligent Robots and Systems (IROS) (pp. 3229–3236).

  • Zhang, J., & Knoll, A. (1995). An enhanced optimization approach for generating smooth robot trajectories in the presence of obstacles. In: Proceedings of the European Chinese Automation Conference (pp. 263–268).

  • Zucker, M., Kuffner, J., & Bagnell, J. A. D. (2008). Adaptive workspace biasing for sampling based planners. In IEEE International Conference on Robotics and Automation (ICRA).

Download references

Acknowledgments

This work was supported by the German Research Foundation (DFG), Emmy Noether fellowship TO 409/1-3.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Nikolay Jetchev.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Jetchev, N., Toussaint, M. Fast motion planning from experience: trajectory prediction for speeding up movement generation. Auton Robot 34, 111–127 (2013). https://doi.org/10.1007/s10514-012-9315-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-012-9315-y

Keywords

Navigation