Skip to main content
SpringerLink
Log in

Appearance Learning for Infrared Tracking with Occlusion Handling

  • Chapter
  • First Online:
Machine Vision Beyond Visible Spectrum

Part of the book series: Augmented Vision and Reality ((Augment Vis Real,volume 1))

Abstract

This chapter discusses the issue of appearance learning for infrared target tracking with occlusion handling. The problem is cast in a co-inference framework, where both adaptive Kalman filtering (AKF) and particle filtering are integrated together to learn target appearance and to estimate target kinematics in a sequential manner. We propose a dual foreground–background appearance model that incorporates the pixel statistics in both foreground and background areas for an effective target representation. Appearance learning is formulated as an AKF problem that can be approached by either covariance or correlation methods for noise estimation. Moreover, occlusions can be easily detected by analyzing the Kalman filtering residuals. Experiments on real infrared imagery show that correlation-based AKF outperforms the covariance-based one as well as traditional histogram similarity-based approaches with near sub-pixel tracking accuracy and robust occlusion handling.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Available from the Center for Imaging Science at the Johns Hopkins University (http://cis.jhu.edu).

  2. 2.

    https://www.sdms.afrl.af.mil/main.php

  3. 3.

    http://jbrwww.che.wisc.edu/software/als/

References

  1. Arulampalam, M.S., Maskell, S., Gordon, N., Clapp, T.: A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking. IEEE Trans. Signal Process. 50(2), 174–188 (2002)

    Article  Google Scholar 

  2. Birchfield, S.: Elliptical head tracking using intensity gradients and color histograms. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, pp. 232–237 (1998)

    Google Scholar 

  3. Carew, B., Belanger, P.: Identification of optimum filter steady-state gain for systems with unknown noise covariances. IEEE Trans. Autom. Control 18(6), 582–587 (1973)

    Article  MathSciNet  MATH  Google Scholar 

  4. Collins, R.T., Liu, Y., Leordeanu, M.: Online selection of discriminative tracking features. IEEE Trans. Pattern Anal. Mach. Intell. 27(10), 1631–1643 (2005)

    Article  Google Scholar 

  5. Comaniciu, D., Ramesh, V., Meer, P.: Real-time tracking of non-rigid objects using mean shift. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 142–149 (2000)

    Google Scholar 

  6. Comaniciu, D., Ramesh, V., Meer, P.: Kernel-based object tracking. IEEE Trans. Pattern Anal. Mach. Intell. 25(5), 564–577 (2003)

    Article  Google Scholar 

  7. Dawoud, A., Alam, M.S., Bal, A., Loo, C.: Decision fusion algorithm for target tracking in infrared imagery. Opt. Eng. 44, 026401–18 (2005)

    Article  Google Scholar 

  8. Dawoud, A., Alam, M.S., Bal, A., Loo, C.: Target tracking in infrared imagery using weighted composite reference function-based decision fusion. IEEE Trans. Image Process. 15(2), 404–410 (2006)

    Article  Google Scholar 

  9. del Blanco, C.R., Jaureguizar, F., Garcìa, N., Salgado, L.: Robust automatic target tracking based on a Bayesian ego-motion compensation framework for airborne FLIR imagery. In: Sadjadi, F.A., Mahalanobis, A. (eds.) Polarimetric and Infrared Processing for ATR. Proceedings of the SPIE, vol. 7335 (2009)

    Google Scholar 

  10. Han, B., Zhu., Y., Comaniciu, D., Davis, L.: Kernel-based Bayesian filtering for object tracking. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 227–234 (2005)

    Google Scholar 

  11. Han, T.X., Liu, M., Huang, T.S.: A drifting-proof framework for tracking and online appearance learning. In: Proceedings of the IEEE Workshop on Applications of Computer Vision (2007)

    Google Scholar 

  12. Harger, G.D., Belhumeur, P.N.: Real-time tracking of image regions with changes in geometry and illumination. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, San Francisco, CA, pp. 403–410 (1996)

    Google Scholar 

  13. Jepson, A.D., Fleet, D.J., El-Maraghi, T.F.: Robust online appearance models for visual tracking. IEEE Trans. Pattern Anal. Mach. Intell. 25(10), 1296–1311 (2003)

    Article  Google Scholar 

  14. Johnston, C.M., Mould, N., Havlicek, J.P., Fan, G.: Dual domain auxiliary particle filter with integrated target signature update. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition Workshops, Miami, FL, pp. 54–59 (2009)

    Google Scholar 

  15. Khan, J.F., Alam, M.S.: Efficient target detection in cluttered FLIR imagery. In: Casasent, D.P., Chao, T.-H. (eds.) Optical Pattern Recognition XVI. Proceedings of the SPIE, vol. 5816, pp. 39–53 (2005)

    Google Scholar 

  16. Lankton, S., Malcolm, J., Nakhmani, M.A., Tannenbaum, A.: Tracking through changes in scale. In: Proceedings of the International Conference on Image Processing, pp. 241–244 (2008)

    Google Scholar 

  17. Latecki, L.J., Miezianko, R.: Object tracking with dynamic template update and occlusion detection. In: Proceedings of the International Conference on Pattern Recognition, Hong Kong, China, vol. 1, pp. 556–560 (2006)

    Google Scholar 

  18. Leichter, I., Lindenbaum, M., Rivlin, E.: Tracking by affine kernel transformations using color and boundary cues. IEEE Trans. Pattern Anal. Mach. Intell. 31(1), 164–171 (2009)

    Article  Google Scholar 

  19. Li, X., Hu, W.M., Zhang, Z.F., Zhang, X.Q., Zhu, M.L., Cheng, J.: Visual tracking via incremental log-Euclidean Riemannian subspace learning. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2008)

    Google Scholar 

  20. Li, X.R., Bar-Shalom, Y.: A recursive multiple model approach to noise identification. IEEE Trans. Aerosp. Electron. Syst. 30(3), 671–684 (1994)

    Article  Google Scholar 

  21. Maggio, E., Cavallaro, A.: Hybrid particle filter and mean shift tracker with adaptive transition model. In: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 2, pp. 221–224 (2005)

    Google Scholar 

  22. Matthews, L., Ishikawa, T., Baker, S.: The template update problem. IEEE Trans. Pattern Anal. Mach. Intell. 26(6), 810–815 (2004)

    Article  Google Scholar 

  23. Maybeck, P.S., Rogers, S.K.: Adaptive tracking of multiple hot-spot target IR images. IEEE Trans. Autom. Control 28(10), 937–943 (1983)

    Article  Google Scholar 

  24. Mehra, R.K.: On the identification of variances and adaptive Kalman filtering. IEEE Trans. Autom. Control 15(2), 175–184 (1970)

    Article  MathSciNet  Google Scholar 

  25. Mehra, R.K.: Approaches to adaptive filtering. IEEE Trans. Autom. Control 17(5), 693–698 (1972)

    Article  MathSciNet  MATH  Google Scholar 

  26. Mohamed, A.H., Schwarz, K.P.: Adaptive Kalman filtering for INS/GPS. J. Geodesy 73, 193–203 (1999)

    Article  MATH  Google Scholar 

  27. Mould, N.A., Nguyen, C.T., Johnston, C.M., Havlicek, J.P.: Online consistency checking for AM-FM target tracks. In: Bouman, C.A., Miller, E.L., Pollak, I. (eds.) Proceedings of the SPIE/IS&T Conference on Computational Imaging VI. Proceedings of the SPIE, vol. 6814 (2008)

    Google Scholar 

  28. Neethling, C., Young, P.: Comments on “identification of optimum filter steady-state gain for systems with unknown noise covariances”. IEEE Trans. Autom. Control 19(5), 623–625 (1974)

    Article  MathSciNet  MATH  Google Scholar 

  29. Nguyen, H.T., Smeulders, A.W.M.: Fast occluded object tracking by a robust appearance filter. IEEE Trans. Pattern Anal. Mach. Intell. 26(8), 1099–1104 (2004)

    Article  Google Scholar 

  30. Nguyen, H.T., Worring, M., van der Boomagaard, R.: Occlusion robust adaptive template tracking. In: Proceedings of the IEEE International Conference on Computer Vision, Vancouver, BC, vol. 1, pp. 678–683 (2001)

    Google Scholar 

  31. Noriega, G., Pasupathy, S.: Adaptive estimation of noise covariance matrices in real-time preprocessing of geophysical data. IEEE Trans. Geosci. Rem. Sens. 35(5), 1146–1159 (1997)

    Article  Google Scholar 

  32. Odelson, B.J., Lutz, A., Rawlings, J.B.: The autocovariance least-squares method for estimating covariances: application to model-based control of chemical reactors. IEEE Trans. Control Syst. Technol. 14(3), 532–540 (2006)

    Article  Google Scholar 

  33. Odelson, B.J., Rajamani, R.M., Rawlings, B.J.: A new autocovariance least-squares method for estimating noise covariances. Automatica 42(2), 303–308 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  34. Pan, J., Hu, B.: Robust object tracking against template drift. In: IEEE International Conference on Image Processing, pp. 353–356 (2007)

    Google Scholar 

  35. Peng, N.S., Yang, J., Liu, Z.: Mean shift blob tracking with kernel histogram filtering and hypothesis testing. Pattern Recognition Lett. 26(5), 605–614 (2005)

    Article  Google Scholar 

  36. Peng, Z., Zhang, Q., Guan, A.: Extended target tracking using projection curves and matching pel count. Opt. Eng. 46(6), 0664011–0664016 (2007)

    Article  Google Scholar 

  37. Porikli, F., Tuzel, O., Meer, P.: Covariance tracking using model update based on Lie algebra. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 728–735 (2006)

    Google Scholar 

  38. Powers, R.M., Pao, L.Y.: Using Kolmogorov–Smirnov tests to detect track-loss in the absence of truth data. In: Proceedings of the IEEE Conference on Decision and Control, pp. 3097–3104 (2005)

    Google Scholar 

  39. Shaik, J.S., Iftekharuddin, K.M.: Automated tracking and classification of infrared images. In: Proceedings of the International Joint Conference on Neural Networks, vol. 2, pp. 1201–1206 (2003)

    Google Scholar 

  40. She, K., Bebis, G., Gu, H., Miller, R.: Vehicle tracking using on-line fusion of color and shape features. In: Proceedings of the IEEE International Conference on Intelligent Transportation Systems, pp. 731–736 (2004)

    Google Scholar 

  41. Swain, M.J., Ballard, D.H.: Indexing via color histograms. In: Proceedings of the International Conference on Computer Vision, pp. 390–393 (1990)

    Google Scholar 

  42. Tuzel, O., Porikli, F., Meer, P.: Region covariance: a fast descriptor for detection and classification. In: Proceedings of the 9th European Conference on Computer Vision, pp. 589–600 (2006)

    Google Scholar 

  43. Venkataraman, V., Fan, G., Fan, X.: Target tracking with online feature selection in flir imagery. In: Workshop on Object Tracking and Classification Beyond the Visible Spectrum, pp. 1–8 (2007)

    Google Scholar 

  44. Wang, Z., Wu, Y., Wang, J., Lu, H.: Target tracking in infrared image sequences using diverse adaboostsvm. In: Proceedings of the International Conference on Innovative Computing, Information and Control, USA, pp. 233–236. IEEE Computer Society, Washington, DC (2006)

    Google Scholar 

  45. Wu, Y., Huang, T.S.: Robust visual tracking by integrating multiple cues based on co-inference learning. Int. J. Comput. Vis. 58(1), 55–71 (2004)

    Article  Google Scholar 

  46. Wu, Y., Yu, T., Hua, G.: Tracking appearances with occlusions. In: Proceedings of the Computer Vision and Pattern Recognition, vol. 1, pp. 789–795 (2003)

    Google Scholar 

  47. Yi, S., Zhang, L.: A novel multiple tracking system for UAV platforms. In: Henry, D.J. (ed.) ISR Systems and Applications III. Proceedings of the SPIE, vol. 6209 (2006)

    Google Scholar 

  48. Yilmaz, A., Li, X., Shah, M.: Contour-based object tracking with occlusion handling in video acquired using mobile cameras. IEEE Trans. Pattern Anal. Mach. Intell. 26(11), 1531–1536 (2004)

    Article  Google Scholar 

  49. Yilmaz , A., Shafique, K., Shah, M.: Tracking in airborne forward looking infrared imagery. Image Vis. Comput. 21(7), 623–635 (2003)

    Article  Google Scholar 

  50. Yilmaz, A., Javed, O., Shah, M.: Object tracking: a survey. ACM Comput. Surv. 38(4), 13 (2006)

    Article  Google Scholar 

  51. Zhang, C., Rui, Y.: Robust visual tracking via pixel classification and integration. In: Proceedings of the International Conference on Pattern Recognition, vol. 3, pp. 37–42 (2006)

    Google Scholar 

  52. Zivkovic, Z., Krose, B.: An EM like algorithm for color-histogram-based object tracking. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 798–803 (2004)

    Google Scholar 

Download references

Acknowledgments

The authors thank Dr. James B. Rawlings’s research group at the University of Wisconsin-Madison for providing the ALS code. Footnote 3 This work was supported in part by the U.S. Army Research Laboratory and the U.S. Army Research Office under Grants W911NF-04-1-0221 and W911NF-08-1-0293 and the 2009 Oklahoma NASA EPSCoR Research Initiation Grant (RIG).

Author information

Authors and Affiliations

  1. School of Electrical and Computer Engineering, Oklahoma State University, Stillwater, OK, USA

    Guoliang Fan & Vijay Venkataraman

  2. School of Software, Dalian University of Technology, Dalian, China

    Xin Fan

  3. School of Electrical and Computer Engineering, University of Oklahoma, Norman, OK, USA

    Joseph P. Havlicek

Authors
  1. Guoliang Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vijay Venkataraman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xin Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Joseph P. Havlicek
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guoliang Fan .

Editor information

Editors and Affiliations

  1. DynaVox Mayer-Johnson, Wharton Street 2100, Pittsburgh, 15203, Pennsylvania, USA

    Riad Hammoud

  2. School of Electrical &, Computer Engineering, Oklahoma State University, Stillwater, 74078-6028, Oklahoma, USA

    Guoliang Fan

  3. Defense Command, U.S. Army Space & Missile, Huntsville, Alabama, USA

    Robert W. McMillan

  4. Inst. Industrial Science, University of Tokyo, Komaba 4-6-1, Tokyo, 153-8505, Japan

    Katsushi Ikeuchi

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Fan, G., Venkataraman, V., Fan, X., Havlicek, J.P. (2011). Appearance Learning for Infrared Tracking with Occlusion Handling. In: Hammoud, R., Fan, G., McMillan, R., Ikeuchi, K. (eds) Machine Vision Beyond Visible Spectrum. Augmented Vision and Reality, vol 1. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-11568-4_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-11568-4_2

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-11567-7

  • Online ISBN: 978-3-642-11568-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation