Abstract
Color edition is an important issue in image processing and graphic design. This paper presents a deep color decomposition based framework for image recoloring, allowing users to achieve professional color edition through simple interactive operations. Different from existing methods that perform palette generation and color decomposition separately, our method directly generates the recolored images by a light-weight CNN. We first formulate the generation of color palette as an unsupervised clustering problem, and employ a fully point-wise CNN to extract the most representative colors from the input image. Particularly, a pixel scrambling strategy is adopted to map the continuous image color to a compact discrete palette space, facilitating the CNN focus on color-relevant features. Then, we devise a deep color decomposition network to obtain the projected weights of input image on the basis colors of the generated palette space, and leverage them for image recoloring in a user-interacted manner. In addition, a novel aesthetic constraint derived from color harmony theory is proposed to augment the color reconstruction from user-specified colors, resulting in an aesthetically pleasing visual effect. Qualitative comparisons with existing methods demonstrate the effectiveness of our proposed method.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Reinhard, E., Adhikhmin, M., Gooch, B., Shirley, P.: Color transfer between images. IEEE Comput. Graph. Appl. 21(5), 34–41 (2001)
Gupta, R.K., Chia, A.Y.S., Rajan, D., Ng, E.S., Zhiyong, H.: Image colorization using similar images. In: Proceedings of the 20th ACM International Conference on Multimedia, pp. 369–378. ACM (2012)
Levin, A., Lischinski, D., Weiss, Y.: Colorization using optimization. In: ACM Transactions on Graphics (tog), vol. 23, pp. 689–694. ACM (2004)
Li, Y., Ju, T., Hu, S.M.: Instant propagation of sparse edits on images and videos. In: Computer Graphics Forum. vol. 29, pp. 2049–2054. Wiley Online Library (2010)
Ding, X., Xu, Y., Deng, L., Yang, X.: Colorization using quaternion algebra with automatic scribble generation. In: Schoeffmann, K., Merialdo, B., Hauptmann, A.G., Ngo, C.-W., Andreopoulos, Y., Breiteneder, C. (eds.) MMM 2012. LNCS, vol. 7131, pp. 103–114. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-27355-1_12
Chen, X., Zou, D., Li, J., Cao, X., Zhao, Q., Zhang, H.: Sparse dictionary learning for edit propagation of high-resolution images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2854–2861 (2014)
Matsui, Y.: Challenge for manga processing: sketch-based manga retrieval. In: Proceedings of the 23rd ACM International Conference on Multimedia, pp. 661–664. ACM (2015)
Ci, Y., Ma, X., Wang, Z., Li, H., Luo, Z.: User-guided deep anime line art colorization with conditional adversarial networks. In: 2018 ACM Multimedia Conference on Multimedia Conference, pp. 1536–1544. ACM (2018)
Chang, H., Fried, O., Liu, Y., DiVerdi, S., Finkelstein, A.: Palette-based photo recoloring. ACM Trans. Graph. (TOG) 34(4), 139 (2015)
Aksoy, Y., Aydin, T.O., Smolić, A., Pollefeys, M.: Unmixing-based soft color segmentation for image manipulation. ACM Trans. Graph. (TOG) 36(2), 19 (2017)
Zhang, Q., Xiao, C., Sun, H., Tang, F.: Palette-based image recoloring using color decomposition optimization. IEEE Trans. Image Process. 26(4), 1952–1964 (2017)
Tan, J., Lien, J.M., Gingold, Y.: Decomposing images into layers via RGB-space geometry. ACM Trans. Graph. (TOG) 36(1), 7 (2017)
Tan, J., Echevarria, J., Gingold, Y.: Efficient palette-based decomposition and recoloring of images via rgbxy-space geometry. In: SIGGRAPH Asia 2018 Technical Papers, p. 262. ACM (2018)
Tan, J., Echevarria, J., Gingold, Y.: Palette-based image decomposition, harmonization, and color transfer. arXiv preprint (2018). arXiv:1804.01225
Wah, C., Branson, S., Welinder, P., Perona, P., Belongie, S.: The caltech-ucsd birds-200-2011 dataset (2011)
Zhang, J., Cao, Y., Wang, Y., Wen, C., Chen, C.W.: Fully point-wise convolutional neural network for modeling statistical regularities in natural images. arXiv preprint (2018). arXiv:1801.06302
Hordley, S.D., Finlayson, G.D.: Re-evaluating colour constancy algorithms. In: Proceedings of the 17th International Conference on Pattern Recognition, 2004, ICPR 2004, vol. 1, pp. 76–79. IEEE (2004)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Tanno, R., Matsuo, S., Shimoda, W., Yanai, K.: DeepStyleCam: a real-time style transfer app on iOS. In: Amsaleg, L., Guðmundsson, G.Þ., Gurrin, C., Jónsson, B.Þ., Satoh, S. (eds.) MMM 2017. LNCS, vol. 10133, pp. 446–449. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-51814-5_39
Tokumaru, M., Muranaka, N., Imanishi, S.: Color design support system considering color harmony. In: 2002 IEEE World Congress on Computational Intelligence, 2002 IEEE International Conference on Fuzzy Systems, FUZZ-IEEE 2002. Proceedings (Cat. No. 02CH37291). vol. 1, pp. 378–383. IEEE (2002)
Cho, W., et al.: Text2colors: guiding image colorization through text-driven palette generation. arXiv preprint (2018). arXiv:1804.04128
Cho, J., Yun, S., Mu Lee, K., Young Choi, J.: Palettenet: image recolorization with given color palette. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 62–70 (2017)
Lin, S., Fisher, M., Dai, A., Hanrahan, P.: Layerbuilder: layer decomposition for interactive image and video color editing. arXiv preprint (2017). arXiv:1701.03754
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Li, Z., Zha, Z., Cao, Y. (2020). Deep Palette-Based Color Decomposition for Image Recoloring with Aesthetic Suggestion. In: Ro, Y., et al. MultiMedia Modeling. MMM 2020. Lecture Notes in Computer Science(), vol 11961. Springer, Cham. https://doi.org/10.1007/978-3-030-37731-1_11
Download citation
DOI: https://doi.org/10.1007/978-3-030-37731-1_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-37730-4
Online ISBN: 978-3-030-37731-1
eBook Packages: Computer ScienceComputer Science (R0)