Skip to main content
SpringerLink
Log in

Attentive boundary aware network for multi-scale skin lesion segmentation with adversarial training

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Because of the large variation in appearance, the existence of artifacts, the low contrast, skin lesion segmentation remains a challenging task. In this paper, we propose a novel Scale Attention based Atrous Spatial Pyramid Pooling (Scale-Att-ASPP) module for skin lesion segmentation with attentive boundary aware. Our network is based on the Generative Adversarial Network (GAN), which includes the segmentation network and the critic network. In the segmentation network, we design the Scale-Att-ASPP module to automatically select the optimal scale of the skin lesion feature of the intermediate convolution layer (Inter-CL) in the encoding path, meanwhile, the irrelevant artifacts features are automatically diminished without using complex pre-processing. After introducing the output of the Scale-Att-ASPP module to the same level layer in the decoding path through skip connection in pixel-wise addition way, the more meaningful semantic segmentation is gained. The Jaccard distance loss is employed to solve the problem of label imbalance in skin lesion segmentation. Our network is adversarially trained on ISBI 2017 dataset by the multi-scale L1 loss introduced by the critic network, which guides the Scale-Att-ASPP module learning to focus on the optimal scale of the skin lesion feature. Finally, our network significantly improves the segmentation performance compared with other state-of-the-art methods, especially for the JAC and SEN scores. Besides, our proposed network works efficiently and shows robustness for different datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Ahn E, Bi L, Jung Y, Kim J, Li C, Fulham MJ, Feng DD (2015) Automated saliency-based lesion segmentation in dermoscopic images. In: EMBC 2015, Milan, Italy, 25–29 Aug. 2015, pp 3009–3012

  2. Ahn E, Kim J, Bi L, Kumar A, Li C, Fulham MJ, Feng DD (2017) Saliency-based lesion segmentation via background detection in dermoscopic images. IEEE J Biomed Health Inform 21(6):1685–1693

    Article  Google Scholar 

  3. Al-masni MA, Al-antari MA, Choi M-T, Han S-M, Kim T-S (2018) Skin lesion segmentation in dermoscopy images via deep full resolution convolutional networks. Comput Methods Prog Biomed 162:221–231

    Article  Google Scholar 

  4. Badrinarayanan V, Kendall A, Cipolla R (2017) SegNet: a deep convolutional encoder-decoder architecture for scene segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495

    Article  Google Scholar 

  5. Berseth M (2017) Isic 2017-skin lesion analysis towards melanoma detection. arXiv:1703.00523

  6. Bi L, Kim J, Ahn E, Feng D, Fulham M (2016) Automated skin lesion segmentation via image-wise supervised learning and multi-scale superpixel based cellular automata. In: ISBI 2016, Prague, Czech Republic, 13–16 April 2016, pp 1059–1062

  7. Bi L, Jinman K, Ahn E, Feng D (2017) Automatic skin lesion analysis using large-scale dermoscopy images and deep residual networks. arXiv:1703.04197

  8. Bissoto A, Perez F, Valle E, Avila S (2018) Skin lesion synthesis with generative adversarial networks. arXiv:1902.03253

  9. Celebi ME, Kingravi HA, Iyatomi H, Aslandogan YA, Stoecker WV, Moss RH, Malters JM, Grichnik JM, Marghoob AA, Rabinovitz HS, Menzies SW (2008) Border detection in dermoscopy images using statistical region merging. Skin Res Technol 14(3):347–353

    Article  Google Scholar 

  10. Chen L-C, Yang Y, Wang J, Xu W, Yuille AL (2016) Attention to scale: Scale-aware semantic image segmentation. In: CVPR 2016, Las Vegas, NV, USA, 27–30 June, 2016, pp 3640–3649

  11. Chen L-C, Papandreou G, Schroff F, Adam H (2017) Rethinking atrous convolution for semantic image segmentation. arXiv:1706.05587

  12. Chen L-C, Zhu Y, Papandreou G, Schroff F, Adam H (2018) Encoder-decoder with atrous separable convolution for semantic image segmentation. In: ECCV 2018, 8–14 September 2018, Munich, Germany, pp.833–851

  13. Codella NCF, Gutman D, Celebi ME, Helba B, Marchetti MA, Dusza SW, Kalloo A, Liopyris K, Mishra NK, Kittler H, Halpern A (2018) Skin lesion analysis toward melanoma detection: a challenge at the 2017 international symposium on biomedical imaging (ISBI), hosted by the international skin imaging collaboration (ISIC). ISBI 2018:168–172

    Google Scholar 

  14. Dakhia A, Wang T, Lu H (2019) Multi-scale pyramid pooling network for salient object detection. Neurocomputing 333:211–220

    Article  Google Scholar 

  15. Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009) Imagenet: a large-scale hierarchical image database, in Proc. IEEE Conf Comput Vis Pattern Recognit 2009:248–255

    Google Scholar 

  16. Farabet C, Couprie C, Najman L, LeCun Y (2013) Learning hierarchical features for scene labeling. IEEE Trans Pattern Anal Mach Intell 35(8):1915–1929

    Article  Google Scholar 

  17. Gutman D, Codella NC, Celebi E, Helba B, Marchetti M, Mishra N, Halpern A (2016) Skin lesion analysis toward melanoma detection: a challenge at the international symposium on biomedical imaging (isbi) 2016″, hosted by the international skin imaging collaboration (ISIC). arXiv:1605.01397

  18. Hariharan B, Arbeláez P, Girshick R, Malik J (2015) Hypercolumns for object segmentation and fine-grained localization. In: CVPR 2015, 7–12 June 2015, Boston, USA, pp 447–456

  19. He K, Zhang X, Ren S, Sun J (2016) Deep Residual Learning for Image Recognition," 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, pp. 770–778, https://doi.org/10.1109/CVPR.2016.90.

  20. Hu J, Shen L, Albanie S, Sun G, Wu E (2018) Squeeze-and-Excitation Networks. In: CVPR 2018, Salt Lake City, USA, 19–21 June, 2018, pp 7132–7141

  21. Kamnitsas K, Ledig C, Newcombe VFJ, Simpson JP, Kane AD, Menon DK, Rueckert D, Glocker B (2017) Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med Image Anal 36:61–78

    Article  Google Scholar 

  22. Lin BS, Michael K, Kalra S, Tizhoosh HR (2017) Skin lesion segmentation: U-Nets versus clustering. In: SSCI 2017, November 2017, pp 1–7

  23. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: CVPR 2015, 7–12 June 2015, Boston, USA, pp 3431–3440

  24. Ma Z, Tavares JMRS (2016) A novel approach to segment skin lesions in dermoscopic images based on a deformable model. IEEE J Biomed Health Inform 20(2):615–623

    Article  Google Scholar 

  25. Mendonça T, Ferreira PM, Marques JS, Marcal AR, Rozeira J (2013) PH2-A dermoscopic image database for research and benchmarking. In: EMBC 2013, 3–7 July 2013, pp. 5437–5440

  26. Mete M, Sirakov NM (2010) Lesion detection in demoscopy images with novel density-based and active contour approaches. BMC Bioinformatics 11(Suppl 6):S23

    Article  Google Scholar 

  27. Noh H, Hong S, Han B (2015) Learning deconvolution network for semantic segmentation. In: ICCV 2015, 7–13 December 2015, Santiago, Chile, pp 1520–1528

  28. Oktay O, Schlemper J, Le Folgoc L (2018) Attention U-Net: Learning where to look for the pancreas. In: MIDL 2018, 4–6 July, 2018, pp 1–10

  29. Oliveira RB, Papa JP, Pereira AS, Tavares JMRS (2018) Computational methods for pigmented skin lesion classification in images: review and future trends. Neural Comput Applic 29(3):613–636

    Article  Google Scholar 

  30. Rahman M, Alpaslan N, Bhattacharya P (2016) Developing a retrieval based diagnostic aid for automated melanoma recognition of dermoscopic images. In: IEEE Applied Imagery Pattern Recognition Workshop, Washington, DC, USA, 18–20 Oct. 2016, pp 1–7

  31. Rogers HW, Weinstock MA, Feldman SR, Coldiron BM (2015) Incidence estimate of nonmelanoma skin cancer (keratinocyte carcinomas) in the U.S. population, 2012. JAMA Dermatol 151(10):1081–1086

    Article  Google Scholar 

  32. Ronneberger O, Fischer P, Brox T (2015) U-Net: Convolutional Networks for Biomedical Image Segmentation. In: MICCAI 2015, 5–9 October 2015, Munich, Germany, pp 234–241

  33. Salimans T, Goodfellow I, Zaremba W, Cheung V, Radford A, Chen X (2016) Improved techniques for training GANs. In: NIPS 2016, Barcelona, Spain, June 2016

  34. Sarker Md. MK, Rashwan HA, Akram F (2018) SLSDeep: Skin Lesion Segmentation Based on Dilated Residual and Pyramid Pooling Networks. In: MICCAI 2018, Granada, Spain, 16–20 September 2018

  35. Shelhamer E, Long J, Darrell T (2017) Fully convolutional networks for semantic segmentation. IEEE Trans Pattern Anal Mach Intell 39(4):1–1

    Article  Google Scholar 

  36. Siegel RL, Miller KD, Jemal A (2016) Cancer statistics, 2016. CA Cancer J Clin 66:7–30

    Article  Google Scholar 

  37. Tang J, Hou X, Yang C, Li Y, Xin Y, Guo W, Wei Z, Liu Y, Jiang G (2017) Recent developments in nanomedicine for melanoma treatment. Int J Cancer 141(4):646–653

    Article  Google Scholar 

  38. Wang F, Jiang M, Qian C, Yang S, Li C, Zhang H, Wang X, Tang X (2017) Residual Attention Network for Image Classification. In: CVPR 2017, 21–26 July, 2017, pp 6450–6458

  39. Wang TC, Liu MY, Zhu JY, Tao A, Kautz J, Catanzaro B (2018) High resolution image synthesis and semantic manipulation with conditional GANs. In: CVPR 2018, Salt Lake City, 19-21 June, 2018, pp 8798-8807

  40. Xue Y, Xu T, Han Z, Rodney Long L, Huang X (2018) SegAN: adversarial network with multi-scale L1 loss for medical image segmentation. Neuroinformatics 16(6):383–392

    Article  Google Scholar 

  41. Yu L, Chen H, Dou Q, Qin J, Heng PA (2017) Automated melanoma recognition in dermoscopy images via very deep residual networks. IEEE Trans Med Imaging 36(4):994–1004

    Article  Google Scholar 

  42. Yuan Y, Lo Y-C (2019) Improving Dermoscopic image segmentation with enhanced convolutional-Deconvolutional networks. IEEE J Biomed Health Inform 23(2):519–526

    Article  MathSciNet  Google Scholar 

  43. Yuan Y, Chao M, Lo Y-C (2017) Automatic skin lesion segmentation using deep fully convolutional networks with Jaccard distance. IEEE Trans Med Imaging 36(9):1876–1886

    Article  Google Scholar 

  44. Yüksel ME, Borlu M (2009) Accurate segmentation of dermoscopic images by image thresholding based on type-2 fuzzy logic. IEEE Trans Fuzzy Syst 17(4):976–982

    Article  Google Scholar 

  45. Zhang J, Xie Y, Xia Y, Shen C (2019) Attention residual learning for skin lesion classification. IEEE Trans Med Imaging 38(9):2092–2103

    Article  Google Scholar 

  46. Zhao H, Shi J, Qi X, Wang X, Jia J (2017) Pyramid scene parsing network. In: CVPR 2017, Honolulu, HI, USA, 21–26 July, 2017, pp 2881–2890

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China (61771056), National Key R&D Program of China (2017YFC0110700).

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Beijing Institute of Technology, Beijing, China

    Zenghui Wei, Feng Shi, Hong Song & Weixing Ji

  2. School of Information Engineering, Yellow River Conservancy Technical Institute, Kaifeng, China

    Zenghui Wei

  3. School of Biomedical Engineering, Sun Yat-sen University, Guangzhou, China

    Guanghui Han

Authors
  1. Zenghui Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Feng Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hong Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Weixing Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Guanghui Han
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Zenghui Wei or Hong Song.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wei, Z., Shi, F., Song, H. et al. Attentive boundary aware network for multi-scale skin lesion segmentation with adversarial training. Multimed Tools Appl 79, 27115–27136 (2020). https://doi.org/10.1007/s11042-020-09334-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09334-2

Keywords

Search

Navigation