Skip to main content
SpringerLink
Log in

Online multiple objects tracking with detection reliability prior constraint

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Multi-object tracking (MOT) is one popular topic in computer vision. It remains a challenging problem in complex scenes, especially of objects with similar appearance. In this case, many existing data association strategies, which link detections among consecutive frames according appearance and motion cues, may fail to track due to unreliable detections or confused appearance and motion. To solve this problem, this paper proposed a novel online multi-object tracking method with detection reliability prior constraint. Our method integrates the trajectory estimation and detection-prediction association into a unified framework. The detection reliability prior constraint is built with the Hankel matrix from object motion model. When we build the Hankel matrix, we adaptively select a set of previous frames to predict object states and calculate the associated weights between detections and candidate objects. Data association in MOT then is estimated by maximum a posteriori (MAP) in a Bayesian framework, accompanied with both previous trajectory and the current detection reliability. Experimental results using synthetic dataset and four public challenging datasets demonstrate that, the proposed method has a good tracking performance compared with the state-of-the-art multi-object trackers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Andriluka M, Roth S, Schiele B (2008) People-tracking-by-detection and people-detection-by-tracking. In: Computer vision and pattern recognition, 2008. CVPR 2008. IEEE Conference on. IEEE, pp 1–8

  2. Andriyenko A, Schindler K, Roth S (2012) Discrete-continuous optimization for multi-target tracking. In: Computer Vision and Pattern Recognition (CVPR), 2012 I.E. Conference on, IEEE, pp 1926–1933

  3. Bae S-H, Yoon K-J (2014) Robust online multi-object tracking based on tracklet confidence and online discriminative appearance learning. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 1218–1225

  4. Bae S-H, Yoon K-J (2014) Robust online multiobject tracking with data association and track management. IEEE Trans Image Process 23(7):2820–2833

    Article  MathSciNet  MATH  Google Scholar 

  5. Berclaz J, Fleuret F, Turetken E, Fua P (2011) Multiple object tracking using k-shortest paths optimization. IEEE Trans Pattern Anal Mach Intell 33(9):1806–1819

    Article  Google Scholar 

  6. Bernardin K, Stiefelhagen R (2008) Evaluating multiple object tracking performance: the CLEAR MOT metrics. EURASIP J Image and Video Processing 2008(1):1–10

    Article  Google Scholar 

  7. Breitenstein MD, Reichlin F, Leibe B, Koller-Meier E, Van Gool L (2011) Online multiperson tracking-by-detection from a single, uncalibrated camera. IEEE Trans Pattern Anal Mach Intell 33(9):1820–1833

    Article  Google Scholar 

  8. Brendel W, Amer M, Todorovic S (2011) Multiobject tracking as maximum weight independent set. In: Computer Vision and Pattern Recognition (CVPR), 2011 I.E. Conference on. IEEE, pp 1273–1280

  9. Chen Z, You X, Zhong B, Li J, Tao D (2016) Dynamically modulated mask sparse tracking. IEEE Trans Cybern 99:1–13

    Google Scholar 

  10. Chenouard N, Bloch I, Olivo-Marin J-C (2013) Multiple hypothesis tracking for cluttered biological image sequences. IEEE Trans Pattern Anal Mach Intell 35(11):2736–3750

    Article  Google Scholar 

  11. Collins RT (2012) Multitarget data association with higher-order motion models. In: Computer Vision and Pattern Recognition (CVPR), 2012 I.E. Conference on. IEEE, pp 1744–1751

  12. Dicle C, Camps OI, Sznaier M (2013) The way they move: tracking multiple targets with similar appearance. In: Proceedings of the IEEE international conference on computer vision. pp 2304–2311

  13. Dollár P, Appel R, Belongie S, Perona P (2014) Fast feature pyramids for object detection. IEEE Trans Pattern Anal Mach Intell 36(8):1532–1545

    Article  Google Scholar 

  14. Duan G, Ai H, Cao S, Lao S (2012) Group tracking: exploring mutual relations for multiple object tracking. Comput Vis ECCV 2012:129–143

    Google Scholar 

  15. Ellis A, Shahrokni A, Ferryman JM (2010) Pets2009 and winter-pets 2009 results: a combined evaluation. In: 12th IEEE International Workshop on Performance Evaluation of Tracking and Surveillance, IEEE, December 7-12, 2009, Snowbird, Utah, USA, pp 1-8

  16. Everingham L, Gool C, Williams K et al (2010) The pascal visual object classes (voc) challenge. Int J Comput Vis 88(2):303–338

    Article  Google Scholar 

  17. Fortmann T, Bar-Shalom Y, Scheffe M (1980) Joint probabilistic data association for multiple targets in clutter. In: Proc. Conf. on Information Sciences and Systems

  18. Hamid Rezatofighi S, Milan A, Zhang Z, Shi Q, Dick A, Reid I (2015) Joint probabilistic data association revisited. In: Proceedings of the IEEE international conference on computer vision, pp 3047–3055

  19. Han J, Zhang D, Cheng G, Guo L, Ren J (2015) Object detection in optical remote sensing images based on weakly supervised learning and high-level feature learning. IEEE Trans Geosci Remote Sens 53(6):3325–3337

    Article  Google Scholar 

  20. Hong Yoon J, Lee C-R, Yang M-H, Yoon K-J (2016) Online multi-object tracking via structural constraint event aggregation. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 1392–1400

  21. Hong Z, Chen Z, Wang C, Mei X, Prokhorov D, Tao D (2015) MUlti-store tracker (MUSTer): a cognitive psychology inspired approach to object tracking. In: Computer Vision and Pattern Recognition (CVPR), June 8-10, 2015, Boston, USA, pp 749–758

  22. Isard M, Blake A (1998) Condensation—conditional density propagation for visual tracking. Int J Comput Vis 29(1):5–28

    Article  Google Scholar 

  23. Jiang H, Fels S, Little JJ (2007) A linear programming approach for multiple object tracking. In: Computer vision and pattern recognition, 2007. CVPR'07. IEEE Conference on, 2007. IEEE, pp 1–8

  24. Kim C, Li F, Ciptadi A, Rehg JM (2015) Multiple hypothesis tracking revisited. In: IEEE international conference on computer vision. pp 4696–4704

  25. Kuhn HW (1955) The Hungarian method for the assignment problem. Nav Res Logist Q 2(1–2):83–97

    Article  MathSciNet  MATH  Google Scholar 

  26. Kuo C-H, Nevatia R (2011) How does person identity recognition help multi-person tracking? In: Computer Vision and Pattern Recognition (CVPR), 2011 I.E. Conference on, IEEE, pp 1217–1224

  27. Leibe B, Schindler K, Van Gool L (2007) Coupled detection and trajectory estimation for multi-object tracking. In: Computer Vision, 2007. ICCV 2007. IEEE 11th International Conference on. IEEE, pp 1–8

  28. Li Y, Huang C, Nevatia R (2009) Learning to associate: Hybridboosted multi-target tracker for crowded scene. In: Computer Vision and Pattern Recognition. CVPR 2009. IEEE Conference on, 2009. IEEE, pp 2953-296046. M

  29. Luo W, Xing J, Zhang X, Zhao X, Kim T-K (2014) Multiple object tracking: a literature review. arXiv preprint arXiv:14097618

  30. Milan A, Schindler K, Roth S (2013) Detection-and trajectory-level exclusion in multiple object tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 3682–3689

  31. Milan A, Roth S, Schindler K (2014) Continuous energy minimization for multitarget tracking. IEEE Trans Pattern Anal Mach Intell 36(1):58–72

    Article  Google Scholar 

  32. Oh S, Russell S, Sastry S (2009) Markov chain Monte Carlo data association for multi-target tracking. IEEE Trans Autom Control 54(3):481–497

    Article  MathSciNet  MATH  Google Scholar 

  33. Okuma K, Taleghani A, Nd F, Little JJ, Lowe DG (2004) A boosted particle filter: multitarget detection and tracking. Comput Vis ECCV 2004:28–39

    MATH  Google Scholar 

  34. Pirsiavash H, Ramanan D, Fowlkes CC (2011) Globally-optimal greedy algorithms for tracking a variable number of objects. In: Computer Vision and Pattern Recognition (CVPR), 2011 I.E. Conference on, IEEE, pp 1201–1208

  35. Qi Y, Zhang S, Qin L, Yao H, Huang Q, Lim J, Yang MH (2016) Hedged deep tracking. In: Computer Vision and Pattern Recognition, 2016. CVPR'16. IEEE Conference on, June 27-30, 2016, Las Vegas, USA, pp 4303-4311

  36. Reid D (1979) An algorithm for tracking multiple targets. IEEE Trans Autom Control 24(6):843–854

    Article  Google Scholar 

  37. Rezatofighi SH, Milani A, Zhang Z, Shi Q, Dick A, Reid I (2016) Joint probabilistic matching using m-best solutions. In: Computer Vision and Pattern Recognition, 2016. CVPR'16. IEEE Conference on, June 27-30, 2016, Las Vegas, USA, pp 136-145

  38. Wang X, Yang M, Zhu S (2013) Lin Y Regionlets for generic object detection. In: Proceedings of the IEEE international conference on computer vision, pp 17–24

  39. Wang B, Wang G, Chan KL, Wang L (2016) Tracklet association by online target-specific metric learning and coherent dynamics estimation. IEEE transactions on pattern analysis and machine intelligence, 2017, 39(3):589-602

  40. Xiang Y, Alahi A, Savarese S (2015) Learning to track: online multi-object tracking by decision making. In: IEEE international conference on computer vision, pp 4705–4713

  41. Yang B, Nevatia R (2012) Multi-target tracking by online learning of non-linear motion patterns and robust appearance models. In: Computer Vision and Pattern Recognition (CVPR), 2012 I.E. Conference on, IEEE, pp 1918–1925

  42. Yang B, Nevatia R (2014) Multi-target tracking by online learning a CRF model of appearance and motion patterns. Int J Comput Vis 107(2):203–217

    Article  MathSciNet  MATH  Google Scholar 

  43. Yang M, Pei M, Shen J, Jia Y (2015) Robust online multi-object tracking by maximum a posteriori estimation with sequential trajectory prior. In: International Conference on neural information processing, Springer, pp 623–633

  44. Yoon JH, Yang M-H, Lim J, Yoon K-J (2015) Bayesian multi-object tracking using motion context from multiple objects. In: Applications of Computer Vision (WACV), 2015 I.E. Winter Conference on, 2015. IEEE, pp 33–40

  45. Zhang D, Han J, Li C, Wang J, Li X (2016) Detection of co-salient objects by looking deep and wide. Int J Comput Vis 120(2):215–232

    Article  MathSciNet  Google Scholar 

  46. Zhao S, Yao H, Gao Y, Ji RR, Ding G (2017) Continuous probability distribution prediction of image emotions via multi-task shared sparse regression. IEEE Trans Multimedia 99:1–1

    Google Scholar 

Download references

Acknowledgment

This work was supported in part by National Natural Science Foundation of China (Grant No. 61673125 and 61703115), in part by the Frontier and Key Technology Innovation Special Funds of Guangdong Province (Grant No. 2014B090919002, 2016B090910003 and 2015B010917003) and Program of Foshan Innovation Team of Science and Technology (Grant No. 2015IT100072).

Author information

Authors and Affiliations

  1. School of Electromechanical Engineering, Guangdong University of Technology, Guangzhou, 510006, China

    Li He

  2. Department of Computing Science, University of Alberta, T6G2E8, Edmonton, Alberta, Canada

    Honghong Yang

  3. Department of Automation, Northwestern Polytechnical University, Xi’an, 710072, China

    Honghong Yang

Authors
  1. Honghong Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Li He
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Li He.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, H., He, L. Online multiple objects tracking with detection reliability prior constraint. Multimed Tools Appl 77, 23167–23191 (2018). https://doi.org/10.1007/s11042-017-5530-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-5530-z

Keywords

Search

Navigation