Skip to main content
SpringerLink
Log in

Observation and Execution

  • Chapter
Advanced Bimanual Manipulation

Part of the book series: Springer Tracts in Advanced Robotics ((STAR,volume 80))

  • 1215 Accesses

Abstract

Assistive robotic systems in household or industrial production environments get more and more capable of performing also complex tasks which previously only humans were able to do. As robots are often equipped with two arms and hands, similar manipulations can be executed. The robust programming of such devices with a very large number of degrees of freedom (DOFs) compared with single industrial robot arms however is laborious if done joint-wise. Two major directions to overcome this problem have been previously proposed. The programming by demonstration (PbD) approach, where human arm and recently also hand motions are tracked, segmented and re-executed in an adaptive way on the robotic system and the high-level planning approach which tries to generate a task sequence on a logical level and attributes geometric information as necessary to generate artificial trajectories to solve the task. Here we propose to combine the best of both worlds. For the very complex motion generation for a robotic hand, a rather direct approach to assign manipulation actions from human demonstration to a human hand is taken. For the combination of different basic manipulation actions the task constraints are segmented from the demonstration action and used to generate a task oriented plan. This plan is validated against the robot kinematic and geometric constraints and then a geometric motion planner can generate the necessary robot motions to fulfill the task execution on the system.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Abdel-Malek, K., Yu, W.: Placement of robot manipulators to maximize dexterity. Journal of Robotics and Automation 19, 6–15 (2004)

    Google Scholar 

  2. Berenson, D., Srinivasa, S., Kuffner, J.: Addressing pose uncertainty in manipulation planning using task space regions. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, Saint Louis, MO (2009)

    Google Scholar 

  3. Bronstein, I.N., Semendjajew, K.A., Musiol, G., Mühlig, H.: Taschenbuch der Mathematik, 6th edn. Harri Deutsch (2005)

    Google Scholar 

  4. Butterfass, J., Grebenstein, M., Liu, H., Hirzinger, G.: DLR hand II: Next generation of a dextrous robot hand. In: IEEE International Conference on Robotics and Automation, Seoul (2001)

    Google Scholar 

  5. Cavallo, A., De Maria, G., Natale, C., Pirozzi, S.: Optoelectronic joint angular sensor for robotic fingers. Sensors and Actuators A: Physical 152, 203–210 (2009)

    Article  Google Scholar 

  6. Cerveri, P., Lopomo, N., Pedotti, A., Ferrigno, G.: Derivation of centers and axes of rotation for wrist and finger in a hand kinematic model: Methods and reliability results. Annals of Biomechanics Engineering 33, 402–412 (2005)

    Article  Google Scholar 

  7. Cerveri, P., Pedotti, A., Ferrigno, G.: Robust recovery of human motion from video using kalman filters and virtual humans. Human Movement Science 22, 377–404 (2003)

    Article  Google Scholar 

  8. Chang, L.Y., Pollard, N.: Method for determining kinematic parameters of the in vivo thumb carpometacarpal joint. IEEE Transactions on Biomedical Engineering 55, 1897–1906 (2008)

    Article  Google Scholar 

  9. Corato, F., Falco, P., Lösch, M., Maggio, E., Jäkel, R., Villani, L.: Original approaches to interpretation, learning and modeling, from the observation of human manipulation. In: Robotics: Science and Systems, Workshop on Understanding the Human Hand for Advancing Robotic Manipulation, Seattle, WA (2009)

    Google Scholar 

  10. Craig, J.: Introduction to Robotics: Mechanics and Control. Addison-Wesley, Reading (1989)

    MATH  Google Scholar 

  11. De Maria, G., Falco, P., Natale, C., Pirozzi, S.: Data fusion based on optical technology for observation of human manipulation. International Journal on Optomechatronics 6(1) (2012)

    Google Scholar 

  12. Denavit, J., Hartenberg, R.: A kinematic notation for lower-pair mechanisms based on matrices. Journal of Applied Mechanics 7, 215–221 (1955)

    MathSciNet  Google Scholar 

  13. Diankov, R.: Automated construction of robotic manipulation programs, Ph.D. thesis. Carnegie Mellon University (2010)

    Google Scholar 

  14. Diankov, R., Ratliff, N., Ferguson, D., Srinivasa, S., Kuffner, J.: Bispace planning: Concurrent multi-space exploration. In: Robotics: Science and Systems, Zurich (2008)

    Google Scholar 

  15. Diankov, R., Srinivasa, S., Ferguson, D., Kuffner, J.: Manipulation planning with caging grasps. In: 8th IEEE-RAS International Conference on Humanoid Robots, Daejeon (2008)

    Google Scholar 

  16. Dorfmuller-Ulhaas, K., Schmalstieg, D.: Finger tracking for interaction in augmented environments. In: IEEE/ACM International Symposium on Augmented Reality, New York (2001)

    Google Scholar 

  17. Falco, P., Natale, C.: On the stability of closed-loop inverse kinematics algorithms for redundant robots. IEEE Transactions on Robotics 27, 780–784 (2011)

    Article  Google Scholar 

  18. Fuchs, M., Borst, C., Giordano, P.R., Baumann, A., Krämer, E., Langwald, J., Gruber, R., Seitz, N., Plank, G., Kunze, K., Burger, R., Schmidt, F., Wimboeck, T., Hirzinger, G.: Rollin’ Justin — Design considerations and realization of a mobile platform for a humanoid upper body. In: IEEE International Conference on Robotics and Automation, Kobe (2009)

    Google Scholar 

  19. Gienger, M., Toussaint, M., Goerick, C.: Task maps in humanoid robot manipulation. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, Nice (2008)

    Google Scholar 

  20. Gienger, M., Toussaint, M., Jetchev, N., Bendig, A., Goerick, C.: Optimization of fluent approach and grasp motions. In: 8th IEEE-RAS International Conference on Humanoid Robots, Daejeon (2008)

    Google Scholar 

  21. Goldfeder, C., Ciocarlie, M., Dang, H., Allen, P.K.: The Columbia grasp database. In: IEEE International Conference on Robotics and Automation, Kobe (2009)

    Google Scholar 

  22. Guan, Y., Yokoi, K.: Reachable space generation of a humanoid robot using the monte carlo method. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, Beijing (2006)

    Google Scholar 

  23. Han, L., Trinkle, J., Li, Z.: Grasp analysis as linear matrix inequality problems. IEEE Transactions on Robotics and Automation 16, 663–674 (2000)

    Article  Google Scholar 

  24. Jung, S., Wohn, K.: Tracking and motion estimation of the articulated object: A hierarchical Kalman filter approach. Real-Time Imaging 3, 415–432 (1997)

    Article  Google Scholar 

  25. Klein, C.A., Blaho, B.E.: Dexterity measures for the design and control of kinematically redundant manipulators. International Journal of Robotics Research 6(2), 72–83 (1987)

    Article  Google Scholar 

  26. Konietschke, R., Frumento, S., Ortmaier, T., Hagn, U., Hirzinger, G.: Kinematic design optimization of an actuated carrier for the DLR multi-arm surgical system. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, Beijing (2006)

    Google Scholar 

  27. Larsen, E., Gottschalk, S., Lin, M.C., Manocha, D.: Fast proximity queries with swept sphere volumes. In: IEEE International Conference on Robotics and Automation, San Francisco, CA (2000)

    Google Scholar 

  28. Liu, Y.: Qualitative test and force optimization of 3-D frictional form-closure grasps using linear programming. IEEE Transactions on Robotics and Automation 15, 163–173 (1999)

    Article  Google Scholar 

  29. Miyata, N., Kouhci, M., Kurihara, T., Mochimaru, M.: Modeling of human hand link structure from optical motion capture data. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, Sendai (2004)

    Google Scholar 

  30. Murray, R., Li, Z., Sastry, S.: A Mathematical Introduction to Robotic Manipulation. CRC Press, Boca Raton (1994)

    MATH  Google Scholar 

  31. Natale, C.: Kinematic control of robots with noisy guidance systems. In: 18th IFAC World Congress, Milan (2011)

    Google Scholar 

  32. Ott, C., Eiberger, O., Friedl, W., Bäuml, B., Hillenbrand, U., Borst, C., Albu-Schäffer, A., Brunner, B., Hirschmüller, H., Kielhöfer, S., Konietschke, R., Suppa, M., Wimböck, T., Zacharias, F., Hirzinger, G.: A humanoid two-arm system for dexterous manipulation. In: 6th IEEE-RAS International Conference on Humanoid Robots, Genova (2006)

    Google Scholar 

  33. Palli, G., Pirozzi, S.: Force sensor based on discrete optoelectronic components and compliant frames. Sensors and Actuators A: Physical 165, 239–249 (2011)

    Article  Google Scholar 

  34. Palli, G., Pirozzi, S.: Miniaturized optical-based force sensors for tendon-driven robots. In: IEEE International Conference on Robotics and Automation, Shanghai (2011)

    Google Scholar 

  35. Park, F., Brockett, R.: Kinematic dexterity of robotic mechanisms. International Journal of Robotics Research 13, 1–15 (1994)

    Article  Google Scholar 

  36. Park, J.Y., Chang, P.H., Yang, J.Y.: Task-oriented design of robot kinematics using the grid method. Advanced Robotics 17, 879–907 (2003)

    Article  Google Scholar 

  37. Pettré, J., Laumond, J.P., Siméon, T.: A 2-stage locomotion planner for digital actors. In: Eurographics Symposium on Computer Animation, San Diego, CA (2003)

    Google Scholar 

  38. Pin, F.G., Culioli, J.C.: Optimal positioning of combined mobile platform-manipulator systems for material handling tasks. Journal of Intelligent and Robotic Systems 6, 165–182 (1992)

    Article  Google Scholar 

  39. Pitarch, E., Yang, J., Abdel-Malek, K.: Santos hand: A 25 degree-of-freedom model. In: Digital Human Modeling for Design and Engineering Symposium, Iowa City, IA (2005)

    Google Scholar 

  40. Pressure-Profile: Fingertps (2010), http://www.pressureprofile.com/products-fingertps

  41. Ristic, R., Arulampalam, S., Gordon, N.: Beyond the Kalman Filter. Artech House Publishers, Boston (2004)

    MATH  Google Scholar 

  42. Roa, M.A., Hertkorn, K., Borst, C., Hirzinger, G.: Reachable independent contact regions for precision grasps. In: IEEE International Conference on Robotics and Automation, Shanghai (2011)

    Google Scholar 

  43. Saff, E., Kuijlaars, A.: Distributing many points on the sphere. Mathematical Intelligence 19(1), 5–11 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  44. Sagardia, M., Hulin, T., Preusche, C., Hirzinger, G.: Improvements of the voxmap-pointshell algorithm – Fast generation of haptic data structures. In: 53rd Internationales Wissenschaftliches Kolloquium, Ilmenau (2007)

    Google Scholar 

  45. Shannon, C.E.: Communication in the presence of noise. Proceedings of the Institute of Radio Engineers 37(1), 10–21 (1949)

    MathSciNet  Google Scholar 

  46. Siciliano, B., Sciavicco, L., Villani, L., Oriolo, G.: Robotics: Modelling, Planning and Control. Springer, London (2009)

    Google Scholar 

  47. Stilman, M.: Task constrained motion planning in robot joint space. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, San Diego, CA (2007)

    Google Scholar 

  48. Stocco, L., Salcudean, S.E., Sassani, F.: Fast constrained global minimax optimization of robot parameters. Robotica 16, 595–605 (1998)

    Article  Google Scholar 

  49. Sturges, R.: A quantification of machine dexterity applied to an assembly task. International Journal of Robotics Research 9(3), 49–62 (1990)

    Article  Google Scholar 

  50. Townsend, W.: The Barrett hand grasper — Programmably flexible part handling and assembly. Industrial Robot 27(3), 181–188 (2000)

    Article  MathSciNet  Google Scholar 

  51. Vaughan, J., Rosenbaum, D., Meulenbroek, R.: Planning reaching and grasping movements: The problem of obstacle avoidance. Motor Control 5(2), 116–135 (2001)

    Google Scholar 

  52. Veber, M., Bajd, T., Munih, M.: Assessing joint angles in human hand via optical tracking device and calibrating instrumented glove. Meccanica 42, 451–463 (2007)

    Article  MATH  Google Scholar 

  53. Yoshikawa, T.: Foundations of Robotics: Analysis and Control. MIT Press, Cambridge (1990)

    Google Scholar 

  54. Zacharias, F., Borst, C., Beetz, M., Hirzinger, G.: Positioning mobile manipulators to perform constrained linear trajectories. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, Nice (2008)

    Google Scholar 

  55. Zacharias, F., Borst, C., Hirzinger, G.: Capturing robot workspace structure: Representing robot capabilities. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, San Diego, CA (2007)

    Google Scholar 

  56. Zacharias, F., Borst, C., Hirzinger, G.: Object-specific grasp maps for use in planning manipulation actions. In: German Workshop on Robotics, Braunschweig (2009)

    Google Scholar 

  57. Zhang, X., Lee, S., Braido, P.: Determining finger segmental centers of rotation in flexion-extension based on surface marker measurement. Journal of Biomechanics 36, 1097–1102 (2003)

    Article  Google Scholar 

  58. Zhu, X., Ding, H., Tso, S.: A pseudodistance function and its applications. IEEE Transactions on Robotics and Automation 20, 344–352 (2004)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Robotik und Mechatronik Zentrum, DLR, Münchnerstrasse 20, 82234, Wessling, Germany

    Christoph Borst, Franziska Zacharias, Florian Schmidt, Daniel Leidner, Maximo A. Roa, Katharina Hertkorn & Gerhard Grunwald

  2. Dipartimento di Ingegneria dell’Informazione, Seconda Università degli Studi di Napoli, via Roma 29, 81031, Aversa, Italy

    Pietro Falco & Ciro Natale

  3. Oxford Metrics Group, 14 Minns Business Park, West Way, Oxford, OX2 0JB, United Kingdom

    Emilio Maggio

Authors
  1. Christoph Borst
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Franziska Zacharias
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Florian Schmidt
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Daniel Leidner
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Maximo A. Roa
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Katharina Hertkorn
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Gerhard Grunwald
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Pietro Falco
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Ciro Natale
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Emilio Maggio
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christoph Borst .

Editor information

Editors and Affiliations

  1. Dipto. Informatica e, Sistemistica, Università di Napoli Federico II, Via Claudio 21, Napoli, 80125, Italy

    Bruno Siciliano

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer-Verlag GmbH Berlin Heidelberg

About this chapter

Cite this chapter

Borst, C. et al. (2012). Observation and Execution. In: Siciliano, B. (eds) Advanced Bimanual Manipulation. Springer Tracts in Advanced Robotics, vol 80. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-29041-1_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-29041-1_2

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-29040-4

  • Online ISBN: 978-3-642-29041-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation