Skip to main content
SpringerLink
Log in

Non-local duplicate pooling network for salient object detection

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Many existing salient object detection methods are dedicated to fusing features from different levels of a pre-trained convolutional neural network (CNN). However, these methods can easily lead to internal discontinuities within the salient objects because of unreasonable feature fusion strategies and short-range dependencies resulting from common convolution and pooling operations. In this paper, we propose a novel non-local duplicate pooling (NLDP) network to overcome these internal discontinuities. NLDP begins by removing the first few convolutional layers of a classic CNN, which have small receptive fields and require large amounts of calculation. A novel duplicate pooling module (DPM) is then used to generate richer and more detailed saliency maps. This is achieved by constructing a double-pathway that can integrating partial feature maps. Within the DPM, a non-local module (NLM) is used to obtain long-range dependencies. This enhances the internal continuities between the saliency maps. Comprehensive experiments conducted on six benchmark datasets have confirmed the increased effectiveness and detection speed of our method in relation to other salient object detection methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. https://github.com/lucasb-eyer/pydensecrf

References

  1. Wang W, Shen J, Porikli F (2015) Saliency-aware geodesic video object segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3395–3402

  2. Wang W, Shen J, Sun H, et al. (2017) Video co-saliency guided co-segmentation. IEEE Trans Circ Syst Video Technol 28(8):1727–1736

    Article  Google Scholar 

  3. Hong S, You T, Kwak S, et al. (2015) Online tracking by learning discriminative saliency map with convolutional neural network. In: International conference on machine learning, pp 597–606

  4. Cheng MM, Zhang FL, Mitra NJ, et al. (2010) Repfinder: finding approximately repeated scene elements for image editing. ACM Transactions on Graphics (TOG) 29(4):1–8

    Article  Google Scholar 

  5. Zhang GX, Cheng MM, Hu SM, et al. (2009) A shape-preserving approach to image resizing. Computer Graphics Forum 28(7):1897–1906

    Article  Google Scholar 

  6. Craye C, Filliat D, Goudou JF (2016) Environment exploration for object-based visual saliency learning. In: 2016 IEEE international conference on robotics and automation (ICRA), pp 2303–2309

  7. Zhai Y, Shah M (2006) Visual attention detection in video sequences using spatiotemporal cues. In: Proceedings of the 14th ACM international conference on multimedia, pp 815–824

  8. Cheng MM, Mitra NJ, Huang X, et al. (2014) Global contrast based salient region detection. IEEE Trans Pattern Anal Mach Intell 37(3):569–582

    Article  Google Scholar 

  9. Achanta R, Estrada F, Wils P, et al. (2008) Salient region detection and segmentation. In: International conference on computer vision systems, pp 66–75

  10. Achanta R, Hemami S, Estrada F, et al. (2009) Frequency-tuned salient region detection. In: 2009 IEEE conference on computer vision and pattern recognition, pp 1597–1604

  11. Itti L (1998) A model of saliency-based visual attention for rapid scene analysis[J]. IEEE Trans 20. https://doi.org/10.1109/34.730558

  12. Rosin PL (2009) A simple method for detecting salient regions[J]. Pattern Recogn 42(11):2363–2371. https://doi.org/10.1016/j.patcog.2009.04.021

    Article  Google Scholar 

  13. Yu Z, Wong HS (2007) A rule based technique for extraction of visual attention regions based on real-time clustering[J]. IEEE Trans Multimed 9(4):766–784. https://doi.org/10.1109/TMM.2007.893351

    Article  Google Scholar 

  14. Siva P, Russell C, Xiang T, et al. (2013) Looking beyond the image: unsupervised learning for object saliency and detection. IEEE Conference on Computer Vision and Pattern Recognition, pp 3238–3245

  15. Li H, Ngan KN (2011) A co-saliency model of image pairs[J]. IEEE Trans Image Process 20 (12):3365–3375. https://doi.org/10.1109/TIP.2011.2156803

    Article  MathSciNet  Google Scholar 

  16. Fu H, Cao X, Tu Z (2013) cluster-based co-saliency detection. In IEEE Transactions on Image Processing 22:3766–3778. https://doi.org/10.1109/TIP.2013.2260166

    Article  MathSciNet  Google Scholar 

  17. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440

  18. Hou Q, Cheng MM, Hu X, et al. (2017) Deeply supervised salient object detection with short connections. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3203–3212

  19. Luo Z, Mishra A, Achkar A, et al. (2017) Non-local deep features for salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6609–6617

  20. Zhang P, Wang D, Lu H, et al. (2017) Amulet: aggregating multi-level convolutional features for salient object detection. In: Proceedings of the IEEE international conference on computer vision, pp 202–211

  21. He S, Jiao J, Zhang X, et al. (2017) Delving into salient object subitizing and detection. In: Proceedings of the IEEE international conference on computer vision, Venice, pp 1059–1067

  22. Hu X, Zhu L, Qin J, et al. (2018) Recurrently aggregating deep features for salient object detection. In: Thirty-second AAAI conference on artificial intelligence

  23. Amirul Islam M, Kalash M, Bruce NDB (2018) Revisiting salient object detection: simultaneous detection, ranking, and subitizing of multiple salient objects. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7142–7150

  24. Liu N, Han J (2016) Dhsnet: deep hierarchical saliency network for salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 678–686

  25. Zhang D, Han J, Zhang Y (2017) Supervision by fusion: towards unsupervised learning of deep salient object detector. In: Proceedings of the IEEE international conference on computer vision, pp 4048–4056

  26. Zhang L, Dai J, Lu H, et al. (2018) A bi-directional message passing model for salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1741–1750

  27. Wang T, Zhang L, Wang S, et al. (2018) Detect globally, refine locally: a novel approach to saliency detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3127–3135

  28. Zhang X, Wang T, Qi J, et al. (2018) Progressive attention guided recurrent network for salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 714–722

  29. Wang W, Shen J, Dong X, et al. (2018) Salient object detection driven by fixation prediction. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1711–1720

  30. Liu N, Han J, Yang MH (2018) Picanet: learning pixel-wise contextual attention for saliency detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3089–3098

  31. Chen S, Tan X, Wang B, et al. (2018) Reverse attention for salient object detection. In: Proceedings of the European conference on computer vision (ECCV), pp 234–250

  32. Fukushima K (1980) Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological Cybern 36(4):193–202

    Article  Google Scholar 

  33. LeCun Y, Boser B, Denker JS, et al. (1989) Backpropagation applied to handwritten zip code recognition. Neural Computation 1(4):541–551

    Article  Google Scholar 

  34. Gao P, Zhang Q, Wang F, Xiao L, Fujita H, Zhang Y (2020) Learning reinforced attentional representation for end-to-end visual tracking. Inf Sci 517:52–67. https://doi.org/10.1016/j.ins.2019.12.084

    Article  Google Scholar 

  35. Gao P, Yuan R, Wang F, et al. (2019) Siamese attentional keypoint network for high performance visual tracking[J]. Knowledge Based Systems 193. https://doi.org/10.1016/j.knosys.2019.105448

  36. Wang X, Girshick R, Gupta A, et al. (2018) Non-local neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7794–7803

  37. Pérez-Hernández F, Tabik S, Lamas A, et al. (2020) Object detection binary classifiers methodology based on deep learning to identify small objects handled similarly: application in video surveillance[J]. Knowledge-Based Systems. https://doi.org/10.1016/j.knosys.2020.105590

  38. Li G, Yu Y (2016) Deep contrast learning for salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 478–487

  39. Wu Z, Su L, Huang Q (2019) Cascaded partial decoder for fast and accurate salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3907–3916

  40. Zhou B, Khosla A, Lapedriza A, et al. (2014) Object detectors emerge in deep scene cnns. arXiv:1412.6856

  41. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556

  42. Liu X, Zhu X, Li M, et al. (2019) Multiple kernel k-means with incomplete kernels[J]. IEEE Trans Pattern Anal Mach Intell 42(5):1191–1204. https://doi.org/10.1109/TPAMI.2019.2892416

    Google Scholar 

  43. Yu X, Ye X, Gao Q (2020) Infrared handprint image restoration algorithm based on apoptotic mechanism[j]. IEEE Access 8:47334–47343. https://doi.org/10.1109/ACCESS.2020.2979018

    Article  Google Scholar 

  44. Liu S, Huang D (2018) Receptive field block net for accurate and fast object detection. In: Proceedings of the European conference on computer vision (ECCV), pp 385–400

  45. Borji A, Cheng MM, Hou Q, et al. (2019) Salient object detection: A survey. Computational Visual Media 5(2):117–150 https://doi.org/10.1007/s41095-019-0149-9

    Article  Google Scholar 

  46. Borji A, Cheng MM, Jiang H, et al. (2015) Salient object detection: a benchmark. IEEE Transactions on Image Processing 24(12):5706–5722

    Article  MathSciNet  Google Scholar 

  47. He K, Zhang X, Ren S, et al. (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  48. Liu JJ, Hou Q, Cheng MM, et al. (2019) A simple pooling-based design for real-time salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3917–3926

  49. Xie S, Tu Z (2015) Holistically-nested edge detection. In: Proceedings of the IEEE international conference on computer vision, pp 1395–1403

  50. Qin X, Zhang Z, Huang C, et al. (2019) Basnet: boundary-aware salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7479–7489

  51. Buades A, Coll B, Morel JM (2005) A non-local algorithm for image denoising. In: 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05), pp 60–65

  52. Vaswani A, Shazeer N, Parmar N, et al. (2017) Attention is all you need. In: Advances in neural information processing systems, pp 5998–6008

  53. Yan Q, Xu L, Shi J, et al. (2013) Hierarchical saliency detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1155–1162

  54. Li Y, Hou X, Koch C, et al. (2014) The secrets of salient object segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 280–287

  55. Yang C, Zhang L, Lu H, et al. (2013) Saliency detection via graph-based manifold ranking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3166–3173

  56. Li G, Yu Y (2015) Visual saliency based on multiscale deep features. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5455–5463

  57. Wang L, Lu H, Wang Y, et al. (2017) Learning to detect salient objects with image-level supervision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 136–145

  58. Liu T, Yuan Z, Sun J, et al. (2010) Learning to detect a salient object. IEEE Trans Pattern Anal Mach Intell 33(2):353–367

    Google Scholar 

  59. Wang W, Zhao S, Shen J, et al. (2019) Salient object detection with pyramid attention and salient edges. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1448–1457

  60. Feng M, Lu H, Ding E (2019) Attentive feedback network for boundary-aware salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1623–1632

  61. Wang W, Shen J, Cheng MM, et al. (2019) An iterative and cooperative top-down and bottom-up inference network for salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5968–5977

  62. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv:1412.6980

  63. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105

  64. Wang T, Borji A, Zhang L, et al. (2017) A stagewise refinement model for detecting salient objects in images. In: Proceedings of the IEEE international conference on computer vision, pp 4019–4028

  65. Krähenbühl P, Koltun V (2011) Efficient inference in fully connected crfs with gaussian edge potentials. In: Advances in neural information processing systems, pp 109–117

  66. Movahedi V, Elder JH (2010) Design and perceptual validation of performance measures for salient object segmentation. In: 2010 IEEE computer society conference on computer vision and pattern recognition-workshops, pp 49–56

Download references

Acknowledgements

This work is supported by National Natural Science Foundation of China (No. 62076062) and National Key Research and Development Program of China (No. 2017YFB1002801). It is also supported by Collaborative Innovation Center of Wireless Communications Technology.

Author information

Authors and Affiliations

  1. School of Cyber Science and Engineering, Southeast University, Nanjing, 211189, China

    Jun Jiao

  2. School of Computer Science and Engineering, Southeast University, Nanjing, 211189, China

    Jun Jiao & Hui Xue

  3. MOE Key Laboratory of Computer Network and Information Integration, Southeast University, Nanjing, 211189, China

    Jun Jiao & Hui Xue

  4. School of Computer Science and Engineering, Nanjing University of Science & Technology, Nanjing, 210094, China

    Jundi Ding

Authors
  1. Jun Jiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hui Xue
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jundi Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hui Xue.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jiao, J., Xue, H. & Ding, J. Non-local duplicate pooling network for salient object detection. Appl Intell 51, 6881–6894 (2021). https://doi.org/10.1007/s10489-020-02147-8

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-020-02147-8

Keywords

Search

Navigation