Skip to main content
SpringerLink
Log in

Sparse binarised statistical dynamic features for spatio-temporal texture analysis

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

The paper presents a new spatio-temporal learning-based descriptor called binarised statistical dynamic features (BSDF) for representation and classification of dynamic texture. The BSDF descriptor operates by applying three-dimensional spatio-temporal filters on local voxels of an image sequence where the filters are learned via an independent component analysis, maximising independence over spatial and temporal domains concurrently. The BSDF representation is formed by binarising filter responses which are then converted into codewords and summarised using histograms. A robust representation of the BSDF descriptor is finally obtained via a sparse representation approach yielding very discriminative features for classification. The effects of different hyper-parameters on performance including the number of filters, the number of scales, temporal depth, number of samples drawn are also investigated. The proposed approach is evaluated on the most commonly used dynamic texture databases and shown to perform very well compared to the existing methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. http://www.yongxu.org/lunwen.html

  2. Ali, W., Georgsson, F., Hellstrom, T.: Visual tree detection for autonomous navigation in forest environment. In: Intelligent Vehicles Symposium, 2008 IEEE, pp. 560–565 (2008). https://doi.org/10.1109/IVS.2008.4621315

  3. Arashloo, S.R., Amirani, M.C., Noroozi, A.: Dynamic texture representation using a deep multi-scale convolutional network. J. Vis. Commun. Image Represent. 43, 89–97 (2017). https://doi.org/10.1016/j.jvcir.2016.12.015

    Article  Google Scholar 

  4. Arashloo, S.R., Kittler, J.: Hierarchical image matching for pose-invariant face recognition. In: Cavallaro, A., Prince, S., Alexander D. (eds.) BMVC. British Machine Vision Association, London, UK (2009)

  5. Arashloo, S.R., Kittler, J.: Dynamic texture recognition using multiscale binarized statistical image features. IEEE Trans. Multimed. 16(8), 2099–2109 (2014). https://doi.org/10.1109/TMM.2014.2362855

    Article  Google Scholar 

  6. Baktashmotlagh, M., Harandi, M., Lovell, B.C., Salzmann, M.: Discriminative non-linear stationary subspace analysis for video classification. IEEE Trans. Pattern Anal. Mach. Intell. 36(12), 2353–2366 (2014). https://doi.org/10.1109/TPAMI.2014.2339851

    Article  Google Scholar 

  7. Beham, M.P., Roomi, S.M.M.: Anti-spoofing enabled face recognition based on aggregated local weighted gradient orientation. Signal Image Video Process. 12(3), 531–538 (2018). https://doi.org/10.1007/s11760-017-1189-1

    Article  Google Scholar 

  8. Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1798–1828 (2013). https://doi.org/10.1109/TPAMI.2013.50

    Article  Google Scholar 

  9. Cannons, K.J., Gryn, J.M., Wildes, R.P.: Visual Tracking Using a Pixelwise Spatiotemporal Oriented Energy Representation, pp. 511–524. Springer, Berlin (2010)

    Google Scholar 

  10. Chan, A.B., Vasconcelos, N.: Probabilistic kernels for the classification of auto-regressive visual processes. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 1, pp. 846–851 (2005). https://doi.org/10.1109/CVPR.2005.279

  11. Chan, K.L.: Saliency detection in video sequences using perceivable change encoded local pattern. Signal Image Video Process. 12(5), 975–982 (2018). https://doi.org/10.1007/s11760-018-1242-8

    Article  Google Scholar 

  12. Chan, T.H., Jia, K., Gao, S., Lu, J., Zeng, Z., Ma, Y.: Pcanet: a simple deep learning baseline for image classification? IEEE Trans. Image Process. 24(12), 5017–5032 (2015). https://doi.org/10.1109/TIP.2015.2475625

    Article  MathSciNet  MATH  Google Scholar 

  13. Chen, J., Zhao, G., Salo, M., Rahtu, E., Pietikainen, M.: Automatic dynamic texture segmentation using local descriptors and optical flow. IEEE Trans. Image Process. 22(1), 326–339 (2013). https://doi.org/10.1109/TIP.2012.2210234

    Article  MathSciNet  MATH  Google Scholar 

  14. Culibrk, D., Sebe, N.: Temporal dropout of changes approach to convolutional learning of spatio-temporal features. In: K.A. Hua, Y. Rui, R. Steinmetz, A. Hanjalic, A. Natsev, W. Zhu (eds.) ACM Multimedia, pp. 1201–1204. ACM (2014)

  15. Derpanis, K.G., Wildes, R.P.: Classification of traffic video based on a spatiotemporal orientation analysis. In: 2011 IEEE Workshop on Applications of Computer Vision (WACV), pp. 606–613 (2011). https://doi.org/10.1109/WACV.2011.5711560

  16. Derpanis, K.G.P., Wildes, R.: Spacetime texture representation and recognition based on a spatiotemporal orientation analysis. IEEE Trans. Pattern Anal. Mach. Intell. 34(6), 1193–1205 (2012). https://doi.org/10.1109/TPAMI.2011.221

    Article  Google Scholar 

  17. Derrac, J., Garca, S., Molina, D., Herrera, F.: A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm Evol. Comput. 1(1), 3–18 (2011). https://doi.org/10.1016/j.swevo.2011.02.002

    Article  Google Scholar 

  18. Dollar, P., Rabaud, V., Cottrell, G., Belongie, S.: Behavior recognition via sparse spatio-temporal features. In: 2005 IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance, pp. 65–72 (2005). https://doi.org/10.1109/VSPETS.2005.1570899

  19. Donoho, D.L., Tsaig, Y.: Fast solution of l1-norm minimization problems when the solution may be sparse. IEEE Trans. Inf. Theory 54(11), 4789–4812 (2008). https://doi.org/10.1109/TIT.2008.929958

    Article  MATH  Google Scholar 

  20. Doretto, G., Chiuso, A., Wu, Y.N., Soatto, S.: Dynamic textures. Int. J. Comput. Vis.: IJCV 51(2), 91–109 (2003)

    Article  MATH  Google Scholar 

  21. Dubois, S., Pteri, R., Mnard, M.: Characterization and recognition of dynamic textures based on the 2d+t curvelet transform. Signal Image Video Process. 9(4), 819–830 (2015). https://doi.org/10.1007/s11760-013-0532-4

    Article  Google Scholar 

  22. Fitzgibbon, A.W.: Stochastic rigidity: image registration for nowhere-static scenes. In: Proceedings. Eighth IEEE International Conference on Computer Vision, 2001. ICCV 2001, vol. 1, pp. 662–669 (2001). https://doi.org/10.1109/ICCV.2001.937584

  23. Ghanem, B., Ahuja, N.: Phase based modelling of dynamic textures. In: 2007 IEEE 11th International Conference on Computer Vision, pp. 1–8 (2007). https://doi.org/10.1109/ICCV.2007.4409094

  24. Ghanem, B., Ahuja, N.: Extracting a fluid dynamic texture and the background from video. In: IEEE Conference on Computer Vision and Pattern Recognition, 2008. CVPR 2008, pp. 1–8 (2008). https://doi.org/10.1109/CVPR.2008.4587547

  25. Haas, M., Rijsdam, J., Thomee, B., Lew, M.S.: Relevance feedback: perceptual learning and retrieval in bio-computing, photos, and video. In: Proceedings of the 6th ACM SIGMM International Workshop on Multimedia Information Retrieval, MIR ’04, pp. 151–156. ACM, New York, NY, USA (2004). https://doi.org/10.1145/1026711.1026737

  26. van Hateren, J.H., Ruderman, D.L.: Independent component analysis of natural image sequences yields spatio-temporal filters similar to simple cells in primary visual cortex. Proc. R. Soc. Biol. Sci. 265(1412), 2315–2320 (1998). https://doi.org/10.1098/rspb.1998.0577

    Article  Google Scholar 

  27. Hyvarinen, A.: Fast and robust fixed-point algorithms for independent component analysis. IEEE Trans. Neural Netw. 10(3), 626–634 (1999). https://doi.org/10.1109/72.761722

    Article  Google Scholar 

  28. Hyvrinen, A., Hurri, J., Hoyer, P.O.: Natural Image Statistics: A Probabilistic Approach to Early Computational Vision, 1st edn. Springer, Berlin (2009)

    Book  MATH  Google Scholar 

  29. Ji, H., Yang, X., Ling, H., Xu, Y.: Wavelet domain multifractal analysis for static and dynamic texture classification. IEEE Trans. Image Process. 22(1), 286–299 (2013). https://doi.org/10.1109/TIP.2012.2214040

    Article  MathSciNet  MATH  Google Scholar 

  30. Junejo, I.N., Bhutta, A.A., Foroosh, H.: Single-class svm for dynamic scene modeling. Signal Image Video Process. 7(1), 45–52 (2013). https://doi.org/10.1007/s11760-011-0230-z

    Article  Google Scholar 

  31. Kannala, J., Rahtu, E.: Bsif: Binarized statistical image features. In: 2012 21st International Conference on Pattern Recognition (ICPR), pp. 1363–1366 (2012)

  32. Kung, T.J., Richards, W.: Inferring “water” from images. In: Richards, W. (ed.) Natural Computation, Chap. 16, pp. 224–233. M.I.T. Press, Cambridge, MA (1988)

  33. Mumtaz, A., Coviello, E., Lanckriet, G.R.G., Chan, A.B.: Clustering dynamic textures with the hierarchical em algorithm for modeling video. IEEE Trans. Pattern Anal. Mach. Intell. 35(7), 1606–1621 (2013). https://doi.org/10.1109/TPAMI.2012.236

    Article  Google Scholar 

  34. Nanni, L., Brahnam, S., Lumini, A.: Local ternary patterns from three orthogonal planes for human action classification. Expert Syst. Appl. 38(5), 5125–5128 (2011). https://doi.org/10.1016/j.eswa.2010.09.137

    Article  Google Scholar 

  35. Osborne, M., Presnell, B., Turlach, B.: A new approach to variable selection in least squares problems. IMA J. Numer. Anal. 20(3), 389 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  36. Päivärinta, J., Rahtu, E., Heikkilä, J.: Volume Local Phase Quantization for Blur-Insensitive Dynamic Texture Classification, pp. 360–369. Springer, Berlin (2011)

    Google Scholar 

  37. Péteri, R., Fazekas, S., Huiskes, M.J.: DynTex : a comprehensive database of dynamic textures. Pattern Recogn. Lett. https://doi.org/10.1016/j.patrec.2010.05.009

  38. Qi, X., Li, C.G., Zhao, G., Hong, X., Pietikainen, M.: Dynamic texture and scene classification by transferring deep image features. Neurocomputing 171, 1230–1241 (2016). https://doi.org/10.1016/j.neucom.2015.07.071

    Article  Google Scholar 

  39. Qiao, Y., Weng, L.: Hidden markov model based dynamic texture classification. IEEE Signal Process. Lett. 22(4), 509–512 (2015). https://doi.org/10.1109/LSP.2014.2362613

    Article  Google Scholar 

  40. Quan, Y., Huang, Y., Ji, H.: Dynamic texture recognition via orthogonal tensor dictionary learning. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 73–81 (2015). https://doi.org/10.1109/ICCV.2015.17

  41. Ravichandran, A., Chaudhry, R., Vidal, R.: View-invariant dynamic texture recognition using a bag of dynamical systems. In: IEEE Conference on Computer Vision and Pattern Recognition, 2009. CVPR 2009, pp. 1651–1657 (2009). https://doi.org/10.1109/CVPR.2009.5206847

  42. Ravichandran, A., Chaudhry, R., Vidal, R.: Categorizing dynamic textures using a bag of dynamical systems. IEEE Trans. Pattern Anal. Mach. Intell. 35(2), 342–353 (2013). https://doi.org/10.1109/TPAMI.2012.83

    Article  Google Scholar 

  43. Rivera, A.R., Chae, O.: Spatiotemporal directional number transitional graph for dynamic texture recognition. IEEE Trans. Pattern Anal. Mach. Intell. 37(10), 2146–2152 (2015). https://doi.org/10.1109/TPAMI.2015.2392774

    Article  Google Scholar 

  44. Saisan, P., Doretto, G., Wu, Y.N., Soatto, S.: Dynamic texture recognition. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2001. CVPR 2001, vol. 2, pp. II-58–II-63 (2001). https://doi.org/10.1109/CVPR.2001.990925

  45. Thriault, C., Thome, N., Cord, M.: Dynamic scene classification: learning motion descriptors with slow features analysis. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2603–2610 (2013). https://doi.org/10.1109/CVPR.2013.336

  46. Wang, Y., Chun Zhu, S.: Modeling textured motion: particle, wave and sketch. In: IEEE International Conference on Computer Vision, ICCV’03, pp. 213–220 (2003)

  47. Wildes, R.P., Bergen, J.R.: Qualitative Spatiotemporal Analysis Using an Oriented Energy Representation, pp. 768–784. Springer, Berlin (2000)

    Google Scholar 

  48. Woolfe, F., Fitzgibbon, A.W.: Shift-invariant dynamic texture recognition. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV (2), Lecture Notes in Computer Science, vol. 3952, pp. 549–562. Springer, Berlin (2006)

    Google Scholar 

  49. Xie, J., Fang, Y.: Dynamic texture recognition with video set based collaborative representation. Image Vis. Comput. 55(Part 2), 86–92 (2016)

    Article  Google Scholar 

  50. Zhang, Z., Xu, Y., Yang, J., Li, X., Zhang, D.: A survey of sparse representation: algorithms and applications. IEEE Access 3, 490–530 (2015). https://doi.org/10.1109/ACCESS.2015.2430359

    Article  Google Scholar 

  51. Zhao, G., Barnard, M., Pietikainen, M.: Lipreading with local spatiotemporal descriptors. IEEE Trans. Multimed. 11(7), 1254–1265 (2009). https://doi.org/10.1109/TMM.2009.2030637

    Article  Google Scholar 

  52. Zhao, G., Pietikainen, M.: Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 915–928 (2007). https://doi.org/10.1109/TPAMI.2007.1110

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Faculty of Engineering, Bilkent University, Ankara, Turkey

    Shervin Rahimzadeh Arashloo

Authors
  1. Shervin Rahimzadeh Arashloo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shervin Rahimzadeh Arashloo.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rahimzadeh Arashloo, S. Sparse binarised statistical dynamic features for spatio-temporal texture analysis. SIViP 13, 575–582 (2019). https://doi.org/10.1007/s11760-018-1384-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-018-1384-8

Keywords

Search

Navigation