Skip to main content
SpringerLink
Log in

On-line modeling for real-time 3D target tracking

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Model-based object tracking can provide autonomous mobile robotic systems with real-time 6-dof pose information, for example, enabling them to rendezvous with targets from a particular desired direction. Most existing model-based trackers, however, require the geometric model of the target to be known a priori, which may pose a practical problem in real-world environments. This paper presents a novel 3D modeler capable of building an approximate model of a target object on-line. The proposed technique rapidly constructs a 3D tessellated enveloping mesh and uses projective texture mapping to further model the target object’s surface features. Separation of the target object from background clutter is achieved via customizable interest filters. The resulting real-time object-tracking system was tested extensively via simulations and experiments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Yang, Q., Wang, L., Yang, R., Wang, S., Liao, M., Nistér D.: Real-time global stereo matching using hierarchical belief propagation. In: Proc. British Machine Vision Conf., vol. 3, pp. 989-998. Edinburgh, UK (2006)

  2. Alvarez, L., Deriche, R., Sánchez, J., Weickert, J.: Dense disparity map estimation respecting image discontinuities: A PDE and scale-space based approach (2000)

  3. Bhanu, B., Lee, S., Das, S.: Adaptive image segmentation using genetic and hybrid search methods. IEEE Trans. Aerospace Electron. Syst. 31(4), 1268–1291 (1995)

    Article  Google Scholar 

  4. Cadman, L., Tjahjadi, T.: Simultaneous feature tracking and three-dimensional object reconstruction from an image sequence. In: Int. Conf. on Image Processing, vol. 2, pp. 391–394. Thessaloniki (2001)

  5. Cascia, M.L., Scarloff, S., Athitsos, V.: Fast, realiable head tracking under varying illumination: An approach based on registration of texture-mapped 3d models. IEEE Trans. Pattern Anal. Mach. Intell. 22(4), 322–336 (2000)

    Article  Google Scholar 

  6. Collins, G., Dennis, L.: A system for video surveillance and monitoring. In: Int. Conf. on Automated Deduction, pp. 497–501. Pittsburgh (2000)

  7. Comport, A., Marchand, E., Chaumette, F.: A real-time tracker for markerless augmented reality. In: IEEE and ACM Int. Symposium on Mixed and Augmented Reality, pp. 36–45. Tokyo (2003)

  8. Comport, A., Marchand, E., Pressigout, M., Chaumette, F.: Real-time markerless tracking for augmented reality: the virtual visual servoing framework. IEEE Trans. Vis. Comput. Graphics 12(4), 615–628 (2006)

    Article  Google Scholar 

  9. Dankers, A., Zelinsky, A.: CeDAR: a real-world vision system. Mach. Vis. Appl. 16(1), 47–58 (2004)

    Article  Google Scholar 

  10. Davison, A.: Real-time simultaneous localisation and mapping with a single camera. In: International Conference on Computer Vision, pp. 1403–1410. Nice (2003)

  11. de Ruiter, H., Benhabib, B.: Colour-gradient redundancy for real-time spatial pose tracking in autonomous robot navigation. In: Canadian Conf. on Computer and Robotic Vision, pp. 20–28. Québec City (2006)

  12. Drummond, T., Cipolla, R.: Real-time visual tracking of complex scenes. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 932–946 (2002)

    Article  Google Scholar 

  13. Everitt, C.: Projective texture mapping (2006). http://developer.nvidia.com/object/Projective_Texture_Mapping%.html

  14. Franco, J.S., Ménier, C., Boyer, E., Raffin, B.: A distributed approach for real time 3D modeling. In: IEEE Workshop on Real Time 3D Sensors and Their Use. Washington DC (2004)

  15. Goulermas, J., Liatsis, P., Fernando, T.: A constrained nonlinear energy minimization framework for the regularization of the stereo correspondence problem. IEEE Trans. Circuits Syst. Video Technol. 15(4), 550–565 (2005)

    Article  Google Scholar 

  16. Hager, G.D., Belhumeur, P.N.: Real-time tracking of image regions with changes in geometry and illumination. In: IEEE Conf. on Computer Vision and Pattern Recognition, pp. 403–410. San Francisco (1996)

  17. Han, M., Xu, W., Tao, H., Gong, Y.: An algorithm for multiple object trajectory tracking. In: IEEE Conf. on Computer Vision and Pattern Recognition, vol. 1, pp. 864–871. Washington (2004)

  18. Harris, C., Stephens, M.: A combined corner and edge detector. In: 4th Alvey Vision Conf., vol. 15, pp. 141–151. Manchester (1988)

  19. Isard, M., Blake, A.: Condensation—conditional density propagation for visual tracking. Int. J. Comput. Vis. 29(1), 5–28 (1998)

    Article  Google Scholar 

  20. Jurie, F., Dhome, M.: Real time robust template matching. In: 13th British Machine Vision Conf., pp. 123–132. Cardiff (2002)

  21. Kahl, F., August, J.: Multi-view reconstruction of space curves. In: 9th IEEE Int. Conf. on Computer Vision, vol. 2, pp. 1017–1024 (2003)

  22. Kim, S., Kweon, I.: Robust model-based 3d object recognition by combining feature matching with tracking. In: Int. Conf. on Robotics and Automation, vol. 2, pp. 2123–2128. Taipei (2003)

  23. Krahnstoever, N., Sharma, R.: Appearance management and cue fusion for 3d model-based tracking. In: Conf. on Computer Vision and Pattern Recognition, vol. 2, pp. 249–254. Madison (2003)

  24. Kutulakos, K.N.: Approximate n-view stereo. In: 7th European Conf. on Computer Vision, pp. 67–83. Dublin (2000)

  25. Kyrki, V., Kragic, D.: Integration of model-based and model-free cues for visual object tracking in 3d. In: Int. Conf. on Robotics and Automation, pp. 1554–1560. Barcelona (2005)

  26. Lee, S., Jang, D., Kim, E., Hong, S., Han, J.: A real-time 3d workspace modeling with stereo camera. In: Int. Conf. on Robots and Systems, pp. 2140–2147. Edmonton (2005)

  27. Lowe, D.: Fitting parameterized three-dimensional models to images. IEEE Trans. Pattern Anal. Mach. Intell. 13(5), 441–450 (1991)

    Article  MathSciNet  Google Scholar 

  28. Lucas, B., Kanade, T.: An iterative image registration technique with application to stereo vision. In: 7th Int. Joint Conf. on Artificial Intelligence, pp. 674–479. Vancouver (1981)

  29. Marchand, E., Bouthemy, P., Chaumette, F.: A 2d–3d model-baed approach to real-time visual tracking. Image Vis. Comput. 19(7), 941–955 (2001)

    Article  Google Scholar 

  30. Martin, F., Horaud, R.: Multiple-camera tracking of rigid objects. Int. J. Robot. Res. 21(2), 97–113 (2002)

    Article  Google Scholar 

  31. McKenna, S., Jabri, S., Duric, Z., Wechsler, H.: Tracking interacting people. In: 4th IEEE Int. Conf. on Automatic Face and Gesture Recognition, pp. 348–353. Grenoble (2000)

  32. Moyung, T.J., Fieguth, P.: Incremental shape reconstruction using stereo image sequences. In: Int. Conf. on Image Processing, vol. 2, pp. 752–755. Vancouver (2000)

  33. Mulayim, A., Yilmaz, U., Atalay, V.: Silhouette-based 3-d model reconstruction from multiple images. IEEE Trans. Syst. Man Cybern. B Cybern. 33(4), 582–591 (2003)

    Article  Google Scholar 

  34. Pedrini, H.: Modeling dense range images through fast polygonal approaches. In: 11th Conf. on Image Analysis & Processing, pp. 448–453. Palermo (2001)

  35. de Ruiter, H., Benhabib, B.: Tracking of rigid bodies for autonomous surveillance. In: IEEE Int. Conf. on Mechatronics and Automation, vol. 2, pp. 928–933. Niagara Falls (2005)

  36. de Ruiter, H., Benhabib, B.: Real-time visual vehicle tracking for autonomous convoy control. In: Int. Conf. on Autonomous Robots and Agents, pp. 195–200. Palmerston North (2006)

  37. Schenker, P., Huntsberger, T., Pirjanian, P., Baumgartner, E., Tunstel, E.: Planetary rover developments supporting mars exploration, sample return and future human-robotic colonization. Auton. Robots 13(2–3), 103–126 (2003)

    Article  Google Scholar 

  38. Schneiderman, H., Nashman, M., Wavering, A., Lumia, R.: Vision-based robotic convoy driving. Mach. Vis. Appl. 8(6), 359–364 (1995)

    Google Scholar 

  39. Shreiner, D. (eds): OpenGL Reference Manual, 4th edn. Addison-Wesley, Boston (2004)

    Google Scholar 

  40. Simard, P., Ferrie, F.: Image-based model updating. In: 13th British Machine Vision Conf., pp. 193–202. Cardiff (2002)

  41. Sugar, T., McBeath, M., Suluh, A., Mundhra, K.: Mobile robot interception using human navigational principles: Comparison of active versus passive tracking algorithms. Auton. Robots 21(1), 43–54 (2006)

    Article  Google Scholar 

  42. Vincze, M., Ayromlou, M., Beltran, C., Gasteratos, A., Hoffgaard, S., Madsen, O., Ponweiser, W., Zillich, M.: A system to navigate a robot into a ship structure. Mach. Vis. Appl. 14(1), 15–25 (2003)

    Article  Google Scholar 

  43. Vincze, M., Schlemmer, M., Gemeiner, P., Ayromlou, M.: Vision for robotics: a tool for model-based object tracking. IEEE Robotics Autom. Mag. 12(4), 53–64 (2005)

    Article  Google Scholar 

  44. Wang, L., Gong, M., Gong, M., Yang, R.: How far can we go with local optimization in teal-time stereo matching. In: Proc. International Symposium on 3D Data Processing, Visualization, and Transmission, pp. 129–136. Chapel Hill (2006)

  45. Wang, S., Siskind, J.: Image segmentation with ratio cut. IEEE Trans. Pattern Anal. Mach. Intell. 25(6), 675–690 (2003)

    Article  Google Scholar 

  46. Webber, J., Malik, J.: Robust computation of optical flow in a multi-scale differential framework. In: 4th Int. Conf. on Computer Vision, pp. 12–20. Berlin (1993)

  47. Yang, R., Welch, G., Bishop, G.: Real-time consensus-based scene reconstruction using commodity graphics hardware. In: 10th Pacific Conf. on Computer Graphics and Applications, pp. 225–234. Beijing (2002)

Download references

Author information

Authors and Affiliations

  1. Department of Mechanical and Industrial Engineering, University of Toronto, 5 King’s College Road, Toronto, ON, M5S 3G8, Canada

    Hans de Ruiter & Beno Benhabib

Authors
  1. Hans de Ruiter
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Beno Benhabib
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hans de Ruiter.

Rights and permissions

Reprints and permissions

About this article

Cite this article

de Ruiter, H., Benhabib, B. On-line modeling for real-time 3D target tracking. Machine Vision and Applications 21, 17 (2009). https://doi.org/10.1007/s00138-008-0138-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-008-0138-y

Keywords

Search

Navigation