Abstract
Anime sketch coloring is to fill the color into the anime sketches to obtain the colorful anime images and it is a new research direction in deep learning technology. Currently, generative adversarial networks (GANs) have been used for anime sketch coloring and achieved some results. However, the colorful images generated by the anime sketch coloring methods based on GANs generally have poor coloring effects. In this paper, an efficient anime sketch coloring method based on swish-gated residual U-net (SGRU) is proposed to solve the above problems. In SGRU, the proposed swish layer and swish-gated residual blocks (SGRBs) effectively filter the information transmitted by each level and speed up the convergence of the network. The perceptual loss and the per-pixel loss are used to constitute the final loss of SGRU. The final loss function reflects the coloring results more realistically and can control the effect of coloring more effectively. SGRU can automatically color the sketch without providing any coloring hints in advance and can be trained end-to-end with the sketch and the corresponding color image. Experimental results show that our method performs better than other state-of-the-art coloring methods, and can achieve the colorful images with higher visual quality.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Anonymous, The Danbooru Community, Branwen, G., Gokaslan, A.: Danbooru2017: a large-scale crowdsourced and tagged anime illustration dataset. https://www.gwern.net/Danbooru2017
Ba, J.L., Kiros, J.R., Hinton, G.E.: Layer normalization. CoRR abs/1607.06450 (2016). https://arxiv.org/abs/1607.06450v1
Chen, Q., Koltun, V.: Photographic image synthesis with cascaded refinement networks. In: Proceedings of International Conference on Computer Vision 2017 (ICCV 2017), Venice, Italy, pp. 1511–1520, October 2017
Dong, H., Yang, G., Liu, F., Mo, Y., Guo, Y.: Automatic brain tumor detection and segmentation using U-Net based fully convolutional networks. In: Valdés Hernández, M., González-Castro, V. (eds.) MIUA 2017. CCIS, vol. 723, pp. 506–517. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-60964-5_44
He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of 2015 IEEE International Conference on Computer Vision (ICCV 2015), Santiago, Chile, pp. 1026–1034 (2015)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of 29th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, United states, pp. 770–778 (2016)
Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of 30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017), Honolulu, HI, United States, pp. 5967–5976, July 2017
Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 694–711. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_43
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: Proceedings of the 3rd International Conference for Learning Representations (ICLR 2015), San Diego, CA, United States, pp. 1–15, May 2015
Lin, B.S., Michael, K., Kalra, S., Tizhoosh, H.R.: Skin lesion segmentation: U-nets versus clustering. In: Proceedings of 2017 IEEE Symposium Series on Computational Intelligence (SSCI 2017), Honolulu, HI, United States, pp. 1–7, November 2017
Liu, Y., Qin, Z., Wan, T., Luo, Z.: Auto-painter: cartoon image generation from sketch by using conditional Wasserstein generative adversarial networks. Neurocomputing 311, 78–87 (2018)
Ramachandran, P., Zoph, B., Le, Q.V.: Searching for activation functions. CoRR abs/1710.05941 (2017). http://arxiv.org/abs/1710.05941
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Schmidhuber, J.: Deep learning in neural networks: an overview. Neural Netw. 61, 85–117 (2015)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR abs/1409.1556 (2014). http://arxiv.org/abs/1409.1556
Wang, K., Gou, C., Duan, Y., Lin, Y., Zheng, X., Wang, F.Y.: Generative adversarial networks: introduction and outlook. IEEE/CAA J. Autom. Sinica 4(4), 588–598 (2017)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)
Yonetsuji, T.: Paintschainer. https://paintschainer.preferred.tech/index_en.html
Zhang, L., Ji, Y., Lin, X.: Style transfer for anime sketches with enhanced residual U-net and auxiliary classifier GAN. In: Proceedings of Asian Conference on Pattern Recognition (ACPR 2017), Nanjing, China, November 2017
Zhang, Z., Liu, Q., Wang, Y.: Road extraction by deep residual U-net. IEEE Geosci. Remote Sens. Lett. 15(5), 749–753 (2018)
Zhao, H., Sun, N.: Improved U-net model for nerve segmentation. In: Zhao, Y., Kong, X., Taubman, D. (eds.) ICIG 2017. LNCS, vol. 10667, pp. 496–504. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-71589-6_43
Acknowledgment
The work described in this paper was support by National Natural Science Foundation of China Foundation No. 61300127. Any conclusions or recommendations stated here are those of the authors and do not necessarily reflect official positions of NSFC.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Liu, G., Chen, X., Hu, Y. (2019). Anime Sketch Coloring with Swish-Gated Residual U-Net. In: Peng, H., Deng, C., Wu, Z., Liu, Y. (eds) Computational Intelligence and Intelligent Systems. ISICA 2018. Communications in Computer and Information Science, vol 986. Springer, Singapore. https://doi.org/10.1007/978-981-13-6473-0_17
Download citation
DOI: https://doi.org/10.1007/978-981-13-6473-0_17
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-6472-3
Online ISBN: 978-981-13-6473-0
eBook Packages: Computer ScienceComputer Science (R0)