Skip to main content
SpringerLink
Log in

A GAN-based Denoising Method for Chinese Stele and Rubbing Calligraphic Image

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Chinese calligraphic images have important artistic and historical values. However, subjected to hundreds of years of natural weathering, corrosion and man-made destruction, Chinese calligraphic images inevitably contain some special noise, such as dotted noise, flake corrosion noise and scratch noise. How to denoise this special noise is a challenge for digital preservation of Chinese calligraphic. In this paper, we propose an end-to-end calligraphic image denoising algorithm based on a well-designed generative adversarial network. The generator contains a recurrent network and a denoising autoencoder. By introducing an attention mechanism, we use a recurrent network with multiple progressive network units to generate a noise attention map. Through the noise attention map, the denoising autoencoder can restore the noisy calligraphic image into a clean image with reduced noise or even no noise. The extensive experiments results show that the results of our method are better than those of other comparison methods in terms of visual effects, PSNR and SSIM.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Buades, A., Coll, B., Morel, J.M.: Non-local means denoising. Image Process. Line 1, 208–212 (2011). https://doi.org/10.5201/ipol.2011.bcm_nlm

    Article  MATH  Google Scholar 

  2. Cao, S., Song, B.: Visual attentional-driven deep learning method for flower recognition. Math. Biosci. Eng. MBE 18(3), 1981–1991 (2021). https://doi.org/10.3934/mbe.2021103

    Article  MATH  Google Scholar 

  3. Chen, J., Benesty, J., Huang, Y., Doclo, S.: New insights into the noise reduction Wiener filter. IEEE Trans. Audio Speech Lang. Process. 14(4), 1218–1234 (2006). https://doi.org/10.1109/TSA.2005.860851

    Article  Google Scholar 

  4. Chen, J., Chen, J., Chao, H., Yang, M.: Image blind denoising with generative adversarial network based noise modeling. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3155–3164 (2018).

  5. Chen, R., Yu, W., Wang, R., Liu, G., Shao, Y.: Interferometric phase denoising by pyramid nonlocal means filter. IEEE Geosci. Remote Sens. Lett. 10(4), 826–830 (2013). https://doi.org/10.1109/LGRS.2012.2225594

    Article  Google Scholar 

  6. Ding, B., Long, C., Zhang, L., Xiao, C.: ARGAN: Attentive recurrent generative adversarial network for shadow detection and removal. In: Proceedings of the IEEE/CVF international conference on computer vision, pp. 10213–10222 (2019).

  7. Friebe, M., Kaup, A.: Low complexity two classes gauss weighting filter for noise reduction in mobile received analog tv signals. In: 2005 13th European signal processing conference, IEEE pp. 1–4 (2005).

  8. Fu, J., Liu, J., Tian, H., Li, Y., Bao, Y., Fang, Z., Lu, H.: Dual attention network for scene segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 3146–3154 (2019).

  9. Fu, X., Liang, B., Huang, Y., Ding, X., Paisley, J.: Lightweight pyramid networks for image deraining. IEEE Trans. Neural Netw. Learn. Syst. 31(6), 1794–1807 (2019). https://doi.org/10.1109/TNNLS.2019.2926481

    Article  Google Scholar 

  10. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. Advances in neural information processing systems, 27 (2014).

  11. Guo, H., Zheng, K., Fan, X., Yu, H., Wang, S.: Visual attention consistency under image transforms for multi-label image classification. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 729–739 (2019).

  12. Guo, S., Yan, Z., Zhang, K., Zuo, W., Zhang, L.: Toward convolutional blind denoising of real photographs. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1712–1722, (2019).

  13. Huang, T., Yang, G.J.T.G.Y., Tang, G.: A fast two-dimensional median filtering algorithm. IEEE Trans. Acoust. Speech Signal Process. 27(1), 13–18 (1979). https://doi.org/10.1109/TASSP.1979.1163188

    Article  Google Scholar 

  14. Kandemir, C., Kalyoncu, C., Toygar, Ö.: A weighted mean filter with spatial-bias elimination for impulse noise removal. Digit. Signal Process. 46, 164–174 (2015). https://doi.org/10.1016/j.dsp.2015.08.012

    Article  MathSciNet  Google Scholar 

  15. Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., Shi, W.: Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4681–4690 (2017).

  16. Li, R., Pan, J., Li, Z., Tang, J.: Single image dehazing via conditional generative adversarial network. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp. 8202–8211 (2018).

  17. Liu, X.W., Liu, C.Y.: An optional gauss filter image denoising method based on difference image fast fuzzy clustering. Appl. Mech. Mater. 411, 1348–1352 (2013). https://doi.org/10.4028/www.scientific.net/AMM.411-414.1348

    Article  Google Scholar 

  18. Mao, X., Shen, C., Yang, Y.B.: Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. Adv. Neural Inf. Process. Syst. 29, 2802–2810 (2016)

    Google Scholar 

  19. Qin, Y., Jia, L., Feng, J., An, M., Diao, L.: Proceedings of the 2015 International Conference on Electrical and Information Technologies for Rail Transportation: Transportation, vol. 378. Springer, Berlin (2016)

    Book  Google Scholar 

  20. Shi, Z., Xu, B., Zheng, X., Zhao, M.: A Chinese character structure preserved denoising method for Chinese tablet calligraphy document images based on KSVD dictionary learning. Multimed. Tools Appl. 76(13), 14921–14936 (2017). https://doi.org/10.1007/s11042-016-4284-3

    Article  Google Scholar 

  21. Shi, Z., Xu, B., Zheng, X., Zhao, M.: An integrated method for ancient Chinese tablet images de-noising based on assemble of multiple image smoothing filters. Multimed. Tools Appl. 75(19), 12245–12261 (2016). https://doi.org/10.1007/s11042-016-3421-3

    Article  Google Scholar 

  22. Tian, C., Xu, Y., Li, Z., Zuo, W., Fei, L., Liu, H.: Attention-guided CNN for image denoising. Neural Netw. 124, 117–129 (2020). https://doi.org/10.1016/j.neunet.2019.12.024

    Article  Google Scholar 

  23. Wang, C., Xu, C., Wang, C., Tao, D.: Perceptual adversarial networks for image-to-image transformation. IEEE Trans. Image Process. 27(8), 4066–4079 (2018). https://doi.org/10.1109/TIP.2018.2836316

    Article  MathSciNet  MATH  Google Scholar 

  24. Wang, Q., Huang, Y., Jia, W., He, X., Blumenstein, M., Lyu, S., Lu, Y.: FACLSTM: ConvLSTM with focused attention for scene text recognition. Sci. China Inf. Sci. 63(2), 1–14 (2020). https://doi.org/10.1007/s11432-019-2713-1

    Article  MathSciNet  Google Scholar 

  25. Wang, X., Yu, K., Wu, S., Gu, J., Liu, Y., Dong, C., Qiao, Y., Change Loy, C.: Esrgan: Enhanced super-resolution generative adversarial networks. In: Proceedings of the European conference on computer vision (ECCV) workshops, pp. 1–16 (2018).

  26. Xiao, L., He, K., Zhou, J.L., Wu, D.: Image noise removal on improvement adaptive medium filter. Laser J. 30(2), 44–46 (2009)

    Google Scholar 

  27. Xiao, T., Xu, Y., Yang, K., Zhang, J., Peng, Y., Zhang, Z.: The application of two-level attention models in deep convolutional neural network for fine-grained image classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 842–850 (2015).

  28. Xingjian, S. H. I., Chen, Z., Wang, H., Yeung, D. Y., Wong, W. K., Woo, W. C.: Convolutional LSTM network: a machine learning approach for precipitation nowcasting. In: Advances in neural information processing systems, pp. 802–810, (2015).

  29. Yu, F., Koltun, V.: Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:https://arxiv.org/abs/1511.07122 (2015).

  30. Zhang, H., Sindagi, V., Patel, V.M.: Image de-raining using a conditional generative adversarial network. IEEE Trans. Circuits Syst. Video Technol. 30(11), 3943–3956 (2019). https://doi.org/10.1109/TCSVT.2019.2920407

    Article  Google Scholar 

  31. Zhang, J., Guo, M., Fan, J.: A novel CNN structure for fine-grained classification of Chinese calligraphy styles. Int. J. Doc. Anal. Recogn. (IJDAR) 22(2), 177–188 (2019). https://doi.org/10.1007/s10032-019-00324-1

    Article  Google Scholar 

  32. Zhang, J., Guo, M., Fan, J.: A novel generative adversarial net for calligraphic tablet images denoising. Multimed. Tools Appl. 79(1), 119–140 (2020). https://doi.org/10.1007/s11042-019-08052-8

    Article  Google Scholar 

  33. Zhang, J.S., Yu, J.H., Mao, G.H., Ye, X.Z.: Denoising of Chinese calligraphy tablet images based on run-length statistics and structure characteristic of character strokes. J. Zhejiang Univ.-Sci. A 7(7), 1178–1186 (2006). https://doi.org/10.1631/jzus.2006.A1178

    Article  MATH  Google Scholar 

  34. Zhang, K., Zuo, W., Zhang, L.: FFDNet: toward a fast and flexible solution for CNN-based image denoising. IEEE Trans. Image Process. 27(9), 4608–4622 (2018). https://doi.org/10.1109/TIP.2018.2839891

    Article  MathSciNet  Google Scholar 

  35. Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a gaussian denoiser: residual learning of deep CNN for image denoising. IEEE Trans. Image Process. 26(7), 3142–3155 (2017). https://doi.org/10.1109/TIP.2017.2662206

    Article  MathSciNet  MATH  Google Scholar 

  36. Zheng, X., Miao, Q., Shi, Z., Fan, Y., Shui, W.: A new artistic information extraction method with multi channels and guided filters for calligraphy works. Multimed. Tools Appl. 75(14), 8719–8744 (2016). https://doi.org/10.1007/s11042-015-2788-x

    Article  Google Scholar 

  37. Liu, Q., Zhang, C., Guo, Q., Xu, H., Zhou, Y.: Adaptive sparse coding on PCA dictionary for image denoising. Vis. Comput. 32(4), 535–549 (2016). https://doi.org/10.1007/s00371-015-1087-x

    Article  Google Scholar 

  38. Jiang, X., Jin, Y., Yao, Y.: Low-dose CT lung images denoising based on multiscale parallel convolution neural network. Vis. Comput. 37(8), 2419–2431 (2021). https://doi.org/10.1007/s00371-020-01996-1

    Article  Google Scholar 

  39. Zhang, L., Long, C., Zhang, X., Xiao, C.: Ris-gan: Explore residual and illumination with generative adversarial networks for shadow removal. Proc. AAAI Conf. Artif. Intell. 34(07), 12829–12836 (2020). https://doi.org/10.1609/aaai.v34i07.6979

    Article  Google Scholar 

  40. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778 (2016).

  41. Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: International Conference on medical image computing and computer-assisted intervention (pp. 234–241) (2015) doi: https://doi.org/10.1007/978-3-319-24574-4_28

  42. Chu, J., Guo, Z., Leng, L.: Object detection based on multi-layer convolution feature fusion and online hard example mining. IEEE Access 6, 19959–19967 (2018). https://doi.org/10.1109/ACCESS.2018.2815149

    Article  Google Scholar 

  43. Zhang, Y., Chu, J., Leng, L., Miao, J.: Mask-refined R-CNN: A network for refining object details in instance segmentation. Sensors 20(4), 1010 (2020). https://doi.org/10.3390/s20041010

    Article  Google Scholar 

  44. Chen, Y., Liu, L., Tao, J., Xia, R., Zhang, Q., Yang, K., Xiong, J., Chen, X.: The improved image inpainting algorithm via encoder and similarity constraint. Vis. Comput. 37(7), 1691–1705 (2021). https://doi.org/10.1007/s00371-020-01932-3

    Article  Google Scholar 

  45. Gou, Y., Li, B., Liu, Z., Yang, S., Peng, X.: CLEARER: Multi-scale neural architecture search for image restoration. Advances in Neural Information Processing Systems, 33 (2020).

Download references

Funding

This study was funded by the NSFC under Grant No.61972315.

Author information

Authors and Affiliations

  1. School of Telecommunication and Information Engineering, Xi’an University of Posts and Telecommunications, Shaanxi, 710121, China

    Xuanhong Wang & Kun Wu

  2. School of Information Science and Technology, Northwest University, Shaanxi, 710127, China

    Ying Zhang, Yun Xiao & Pengfei Xu

Authors
  1. Xuanhong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kun Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ying Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yun Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Pengfei Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yun Xiao.

Ethics declarations

Conflict of interest

To the best of our knowledge, the named authors have no conflict of interest, financial or otherwise.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, X., Wu, K., Zhang, Y. et al. A GAN-based Denoising Method for Chinese Stele and Rubbing Calligraphic Image. Vis Comput 39, 1351–1362 (2023). https://doi.org/10.1007/s00371-022-02410-8

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-022-02410-8

Keywords

Search

Navigation