Skip to main content
SpringerLink
Log in

Enhanced TLD-based video object-tracking implementation tested on embedded platforms

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Object-tracking algorithms on embedded platforms are very important in many civilian and military applications. The Tracking–Learning–Detection (TLD) algorithm is considered one of the state-of-the-art online long-term object-tracking algorithms. The performance of running such computationally intensive algorithms on embedded platforms with limited computing resources is a challenge. This work proposes an enhanced TLD implementation, specifically designed for, and tested on, embedded platforms. In this new implementation, an extra-stage has been added to the TLD detector cascade, called a Region filter. This filter dynamically identifies the candidate region for the tracked object. Further, the two independent tracker and detector TLD components, and the two independent Forward–Backward (FB) and Normalized Cross Correlation (NCC) error measures in the tracker have been parallelized. Still further, the computations of Image Integral in the detector and the NCC in both the tracker and the detector have been optimized using a single instruction multiple data (SIMD) architecture. We evaluate our proposed implementation on the Apalis T30 embedded platform, using the same video sequences that the original TLD is evaluated on. Our results show that our enhanced implementation outperforms the baseline with an average speedup of 3.7 × in the total number of frames per second (fps), while achieving an average 91% of the Precision and 86.6% of the Recall metrics, across all sequences. Further, our enhanced implementation achieves an average speedup of 4.52 × and 1.86 × in the detector and tracker execution times, respectively. Moreover, it results in an average 66.3% energy saving, as compared to the original implementation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. E. Maggio and A. Cavallaro. Video Tracking: Theory and Practice. Wiley, 2011.

  2. Liu, B., Liu, Q., Zhang, T., Yang, Y.: MSSTResNet-TLD: A robust tracking method based on tracking–learning–detection framework by using multi-scale spatio-temporal residual network feature model. Neurocomputing 362, 175–194 (2019)

    Article  Google Scholar 

  3. Gurcan, I., Temizel, A.: Heterogeneous CPU–GPU tracking–learning–detection (H-TLD) for real-time object tracking. J. Real-Time Image Proc. 16, 339–353 (2019). https://doi.org/10.1007/s11554-015-0538-y

    Article  Google Scholar 

  4. Yang, X., Zhu, S., Xia, S. and Zhou, D. A new TLD target tracking method based on improved correlation filter and adaptive scale, The Visual Computer, pp.1–13. 2019.

  5. Moridvaisi, H., Razzazi, F., Pourmina, M.A., Dousti, M.: An extended KCF tracking algorithm based on TLD structure in low frame rate videos. Multimed. Tools. Appl. 79, 20995–21012 (2020). https://doi.org/10.1007/s11042-020-08867-w

    Article  Google Scholar 

  6. Ramesh, B., Zhang, S., Yang, H., Ussa, A., Ong, M., Orchard, G. and Xiang, C. “e-TLD: Event-based Framework for Dynamic Object Tracking,” arXiv preprint arXiv: 2009.00855. 2020.

  7. Zhen, X., Fei, S., Wang, Y., Du, W.: A visual object tracking algorithm based on improved TLD. Algorithms 13(1), 15 (2020)

    Article  Google Scholar 

  8. Kalal, Z., Mikolajczyk, K., Matas, J.: Tracking–learning–detection. IEEE Trans. Pattern. Anal. Mach. Intell. 34(7), 1409–1422 (2011)

    Article  Google Scholar 

  9. Toradex. https://www.toradex.com/computer-on-modules/apalis-arm-family/nvidia-tegra-3. Accessed on July, 2019

  10. OpenTLD, https://www.gnebehay.com/tld/, Accessed January 2018.

  11. TLD Vision, s.r.o., http://tldvision.com/, Accessed May 2020.

  12. S. Chang, Y. Zhong, Z. Quan, Y. Hong, J. Zeng and D. Du. “A Real-time Object Tracking and Image Stabilization System for Photographing in Vibration Environment using OpenTLD Algorithm,” in IEEE International Workshop on Advanced Robotics and its Social Impacts (ARSO). Pp. 141–145. IEEE.2016.

  13. T. Xu, C. Huang, Q. He, G. Guan and Y. Zhang. “An Improved TLD Target Tracking Algorithm,” in Proceedings of the IEEE International Conference on Information and Automation, ICIA. Pp. 2051–2055. IEEE.2016.

  14. R.Bartak and A.Vyskovsky. “Any Object Tracking and Following by a Flying Drone,” in Fourteenth Mexican International Conference on Artificial Intelligence, MICAI. Pp. 35–41. IEEE. 2015.

  15. Abughalieh, K.M., Sababha, B.H., Rawashdeh, N.A.: A Video-based object detection and tracking system for weight sensitive UAVs. Multimedia Tools Appl. 78(7), 9149–9167 (2019)

    Article  Google Scholar 

  16. C.Cigla, K.Emrecan Sahin, and F. Alim. “GPU based Video Object Tracking on PTZ Cameras,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops. pp. 654–662. 2018.

  17. C. Bui, N. Patel, D. Patel, S. Rogers, A. Sawant, R. Manwatkar and H. Tabkhi. “A Hardware/Software Co-Design Approach for Real-Time Object Detection and Tracking on Embedded Devices,” In SoutheastCon, Pp: 1-7, St. Petersburg, FL, USA, 2018

  18. V. Lehtola, H. Huttunen, F. Christophe and T. Mikkonen. “Evaluation of Visual Tracking Algorithms for Embedded Devices,” Scandinavian Conference on Image Analysis, SCIA, pp. 88–97. Springer.2017.

  19. A. E. Guevara, A.Hoak, J. T. Bernal and H. Medeiros. “Vision-based Self-contained Target Following Robot using Bayesian Data Fusion,” in International Symposium on Visual Computing. pp 846–857. Springer 2016.

  20. J. Atala, C.Bederián, A.Bordese, F.Gaich, G.Ingaramo, J. Medina, M. Rossetti, J. Sánchez, M.Tealdi and N.Wolovick. “Real-time Full HD Tracking–Learning–Detection on a 2-SMXGPU,” GPU Technology Conference (GTC). Poster. 2015.

  21. A. Nikitakis, T. Paganos and I. Papaefstathiou. “A novel embedded system for vision tracking,” in Proceeding DATE '14 Proceedings of the conference on Design, Automation & Test in Europe. pp: 1–4, Leuven, Belgium. IEEE. 2014.

  22. S. Dey, G. Kalliatakis, S. Saha, A.K. Singh, S. Ehsan, K. McDonald-Maier, “Mat-cnn-sopc: Motionless analysis of traffic using convolutional neural networks on system-on-a-programmable-chip,”in NASA/ESA Conference on Adaptive Hardware and Systems (AHS), pp. 291–298. IEEE. 2018

  23. Y-F. Li, C-C. Tsai, Y-T. Lai, and J-I. Guo. “A Multiple-Lane Vehicle Tracking Method for Forward Collision Warning System Applications,” in Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC). pp. 1061–1064. IEEE. 2017.

  24. D.Hulens, J. Verbeke and T.Goedemé. “How to choose the best embedded processing platform for on-board UAV image processing?,” Proceedings VISAPP. pp: 1–10. 2015.

  25. S.K. Teoh, V.V. Yap, C.S. Soh, and P. Sebastian. “Implementation and Optimization of Human Tracking System using ARM Embedded Platform,”in 4th International Conference on Intelligent and Advanced Systems ICIAS. Pp. 353–356. IEEE. 2012.

  26. B. D. Lucas and T. Kanade. “An iterative image registration technique with an application to stereo vision,” in Proceedings of the International Joint Conference on Artificial Intelligence, Pp: 674–679, 1981.

  27. Z. Kalal, K. Mikolajczyk, J. Matas. “Forward–Backward Error: Automatic Detection of Tracking Failures,” in International Conference on Pattern Recognition.Pp:2756–2759. IEEE. 2010.

  28. G.Nebehay. “Robust Object Tracking Based on Tracking–Learning–Detection,” Master thesis. Vienna University of Technology. 2012.

  29. C. M. Bishop. Pattern recognition and machine learning. Springer, 2006.

  30. Yocto Project (YP). https://www.yoctoproject.org. Accessed on July 2019.

  31. OpenEmbedded. http://www.openembedded.org/. Accessed Dec. 2019

  32. OpenCV. https://opencv.org/releases.html. Accessed on July 2019.

  33. NEON™. Version: 1.0. Programmer’s Guide. https://developer.arm.com/docs/den0018/a/neon-programmers-guide-version-10. Accessed on July, 2019.

  34. S. Ehsan, A. F. Clark and K. D. McDonald-Maier. “Novel Hardware Algorithms for Row-Parallel Integral Image Calculation,” in Digital Image Computing: Techniques and Applications. pp. 61–65. IEEE. 2009.

  35. Y. Zhang, S. Yin, P. Ouyang, L. Liu and S. Wei. “A Parallel Hardware Architecture for Fast Integral Image Computing,” in IEEE International Symposium on Circuits and Systems (ISCAS). pp. 2189–2192. IEEE.2014.

  36. Ehsan, S., Clark, A.F., Rehman, N.U., McDonald-Maier, K.D.: Integral images: efficient algorithms for their computation and storage in resource-constrained embedded vision systems. Sensors 15(7), 16804–16830 (2015)

    Article  Google Scholar 

  37. B. Bilgic, B. K.P. Horn and I. Masaki. “Efficient Integral Image Computation on the GPU,” In 2010 IEEE Intelligent Vehicles Symposium. pp. 528–533. IEEE. 2010.

  38. P. Viola and M. Jones. “Rapid Object Detection using a Boosted Cascade of Simple Features,” in Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition, CVPR. Vol. 1, pp. I–I. IEEE. 2001

Download references

Author information

Authors and Affiliations

  1. Computer Engineering Department, Yarmouk University, Irbid, 21163, Jordan

    Mwaffaq Otoom

  2. School of Computing and Informatics, Al Hussein Technical University, Amman, 11185, Jordan

    Malek Al-Louzi

Authors
  1. Mwaffaq Otoom
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Malek Al-Louzi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mwaffaq Otoom.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Otoom, M., Al-Louzi, M. Enhanced TLD-based video object-tracking implementation tested on embedded platforms. J Real-Time Image Proc 18, 937–952 (2021). https://doi.org/10.1007/s11554-020-01050-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-020-01050-2

Keywords

Search

Navigation