Skip to main content
SpringerLink
Log in

Painting-91: a large scale database for computational painting categorization

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Computer analysis of visual art, especially paintings, is an interesting cross-disciplinary research domain. Most of the research in the analysis of paintings involve medium to small range datasets with own specific settings. Interestingly, significant progress has been made in the field of object and scene recognition lately. A key factor in this success is the introduction and availability of benchmark datasets for evaluation. Surprisingly, such a benchmark setup is still missing in the area of computational painting categorization. In this work, we propose a novel large scale dataset of digital paintings. The dataset consists of paintings from 91 different painters. We further show three applications of our dataset namely: artist categorization, style classification and saliency detection. We investigate how local and global features popular in image classification perform for the tasks of artist and style categorization. For both categorization tasks, our experimental results suggest that combining multiple features significantly improves the final performance. We show that state-of-the-art computer vision methods can correctly classify 50 % of unseen paintings to its painter in a large dataset and correctly attribute its artistic style in over 60 % of the cases. Additionally, we explore the task of saliency detection on paintings and show experimental findings using state-of-the-art saliency estimation algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. The dataset is available at http://www.cat.uab.cat/~joost/painting91.html.

  2. The software at: www.smivision.com/en/gaze-and-eye-tracking-systems/products/begaze-analysis-software.html.

References

  1. Arora, R., Elgammal, A.: Towards automated classification of fine-art painting style: a comparative study. In: Proceedings of the ICPR (2012)

  2. Bosch, A., Zisserman, A., Munoz, X.: Representing shape with a spatial pyramid kernel. In: Proceedings of the CIVR (2007)

  3. Carneiro, G., da Silva, N.P., Bue, A.D., Costeira, J.P.: Artistic image classification: an analysis on the printart database. In: Proceedings of the ECCV (2012)

  4. Cinzia Di Dio, G.R., Macaluso, E.: The golden beauty: brain response to classical and renaissance sculptures. Plos One 2(11), e1201 (2007)

  5. Condorovici, R.G., Vranceanu, R., Vertan, C.: Saliency map retrieval for artistic paintings inspired from human understanding. In: Proceedings of the SPAMEC (2011)

  6. Csurka, G., Bray, C., Dance, C., Fan, L.: Visual categorization with bags of keypoints. In: Proceedings of the Workshop on Statistical Learning in Computer Vision, ECCV (2004)

  7. Danelljan, M., Khan, F.S., Felsberg, M., van de Weijer, J.: Adaptive color attributes for real-time visual tracking. In: Proceedings of the CVPR (2014)

  8. Deng, J., Dong, W., Socher, R., Jia, L., Kai, L., Li, F.F.: Imagenet: a large-scale hierarchical image database. In: Proceedings of the CVPR (2009)

  9. Elfiky, N., Khan, F.S., van de Weijer, J., Gonzalez, J.: Discriminative compact pyramids for object and scene recognition. Pattern Recognit. 45(4), 1627–1636 (2012)

    Article  MATH  Google Scholar 

  10. Everingham, M., Gool, L.J.V., Williams, C.K.I., Winn, J.M., Zisserman, A.: The pascal visual object classes (voc) challenge. Int. J. Comput. Vis. 88(2), 303–338 (2010)

    Article  Google Scholar 

  11. Gehler, P.V., Nowozin, S.: On feature combination for multiclass object classification. In: Proceedings of the ICCV (2009)

  12. Goguen, J.: Art and the Brain. Imprint Academic, UK (1999)

  13. Guo, Z., Zhang, L., Zhang, D.: A completed modeling of local binary pattern operator for texture classification. Trans. Image Process. 19(6), 1657–1663 (2010)

    Article  MathSciNet  Google Scholar 

  14. Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. In: Proceedings of the NIPS (2006)

  15. Hou, X., Zhang, L.: Saliency detection: a spectral residual approach. In: Proceedings of the CVPR (2007)

  16. Hou, X., Harel, J., Koch, C.: Image signature: highlighting sparse salient regions. Pattern Anal. Mach. Intell. 34(1), 194–201 (2012)

    Article  Google Scholar 

  17. Itti, L., Cristof, K., Ernst, N.: A model of saliency-based visual attention for rapid scene analysis. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998)

    Article  Google Scholar 

  18. Jiang, S., Huang, Q., Ye, Q., Gao, W.: An effective method to detect and categorize digitized traditional chinese paintings. Pattern Recognit. Lett. 27(7), 734–746 (2006)

    Article  Google Scholar 

  19. Johnson, R., Hendriks, E., Berezhnoy, I.J., Brevdo, E., Hughes, S.M., Daubechies, I., Li, J., Postma, E., Wang, J.Z.: Image processing for artist identification. IEEE Signal Process. Mag. 25(4), 37–48 (2008)

    Article  Google Scholar 

  20. Judd, T., Ehinger, K.A., Durand, F., Torralba, A.: Learning to predict where humans look. In: Proceedings of the ICCV (2009)

  21. Khan, F.S., Anwer, R.M., van de Weijer, J., Bagdanov, A.D., Vanrell, M., Lopez, A.M.: Color attributes for object detection. In: Proceedings of the CVPR (2012a)

  22. Khan, F.S., van de Weijer, J., Ali, S., Felsberg, M.: Evaluating the impact of color on texture recognition. In: Proceedings of the CAIP (2013b)

  23. Khan, F.S., van de Weijer, J., Bagdanov, A.D., Vanrell, M.: Portmanteau vocabularies for multi-cue image representations. In: Proceedings of the NIPS (2011)

  24. Khan, F.S., van de Weijer, J., Vanrell, M.: Top-down color attention for object recognition. In: Proceedings of the ICCV (2009)

  25. Khan, F.S., van de Weijer, J., Vanrell, M.: Modulating shape features by color attention for object recognition. Int. J. Comput. Vis. 98(1), 49–64 (2012b)

    Article  Google Scholar 

  26. Khan, F.S., Anwer, R.M., van de Weijer, J., Bagdanov, A., Lopez, A., Felsberg, M.: Coloring action recognition in still images. Int. J. Comput. Vis. 105(3), 205–221 (2013a)

    Article  Google Scholar 

  27. Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: Proceedings of the CVPR (2006)

  28. Lazebnik, S., Schmid, C., Ponce, J.: A sparse texture representation using local affine regions. Pattern Anal. Mach. Intell. 27(8), 1265–1278 (2005)

    Article  Google Scholar 

  29. Liu, T., Yuan, Z., Sun, J., Wang, J., Zheng, N., Tang, X., Shum, H.Y.: Learning to detect a salient object. Pattern Anal. Mach. Intell. 32(2), 353–367 (2011)

    Google Scholar 

  30. Lowe, D.G.: Distinctive image features from scale-invariant points. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  31. Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. Pattern Anal. Mach. Intell. 27(10), 1615–1630 (2005)

    Article  Google Scholar 

  32. Murray, N., Marchesotti, L., Perronnin, F.: Ava: a large-scale database for aesthetic visual analysis. In: Proceedings of the CVPR (2012)

  33. Mutch, J., Lowe, D.G.: Object class recognition and localization using sparse features with limited receptive fields. Int. J. Comput. Vis. 80(1), 45–57 (2008)

    Article  Google Scholar 

  34. Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002)

    Article  Google Scholar 

  35. Oliva, A., Torralba, A.B.: Modeling the shape of the scene: a holistic representation of the spatial envelope. Int. J. Comput. Vis. 42(3), 145–175 (2001)

    Article  MATH  Google Scholar 

  36. Quiroga, R.Q., Pedreira, C.: How do we see art: an eye-tracker study. Front. Hum. Neurosci. 5, 98 (2011)

  37. Ramachandran, V., Hirstein, W.: The science of art: a neurological theory of aesthetic experience. J. Conscious. Stud. 6(7), 15–51 (1999)

    Google Scholar 

  38. Rojas, D., Khan, F.S., van de Weijer, J., Gevers, T.: The impact of color on bag-of-words based object recognition. In: Proceedings of the ICPR (2010)

  39. Sablatnig, R., Kammerer, P., Zolda, E.: Hierarchical classification of paintings using face- and brush-stroke models. In: Proceedings of the ICPR (1998)

  40. Shamir, L., Macura, T., Orlov, N., Eckley, D.M., Goldberg, I.G.: Impressionism, expressionism, surrealism: automated recognition of painters and schools of art. ACM Trans. Appl. Percept. 7(2), 8–16 (2010)

    Article  Google Scholar 

  41. Shamir, L.: Computer analysis reveals similarities between the artistic styles of Van Gogh and Pollock. Leonardo 45(2), 149–154 (2012)

    Article  Google Scholar 

  42. Shamir, L., Tarakhovsky, J.A.: Computer analysis of art. J. Comput. Cult. Herit. 5(2), 1–7 (2012)

    Article  Google Scholar 

  43. Shechtman, E., Irani, M.: Matching local self-similarities across images and videos. In: Proceedings of the CVPR (2007)

  44. Shen, J.: Stochastic modeling western paintings for effective classification. Pattern Recognit. 42(2), 293–301 (2009)

    Article  MATH  Google Scholar 

  45. Siddiquie, B., Vitaladevuni, S.N.P., Davis, L.S.: Combining multiple kernels for efficient image classification. In: Proceedings of the WACV (2009)

  46. Vikram, T., Tscherepanow, M., Wrede, B.: A saliency map based on sampling an image into random rectangular regions of interest. Pattern Recognit. 45(9), 3114–3124 (2012)

  47. van de Sande, K.E.A., Gevers, T., Snoek, C.G.M.: Evaluating color descriptors for object and scene recognition. Pattern Anal. Mach. Intell. 32(9), 1582–1596 (2010)

    Article  Google Scholar 

  48. van de Weijer, J., Schmid, C., Verbeek, J.J., Larlus, D.: Learning color names for real-world applications. Trans. Image Process. 18(7), 1512–1524 (2009)

    Article  MathSciNet  Google Scholar 

  49. Xiao, J., Hays, J., Ehinger, K.A., Oliva, A., Torralba, A.: Sun database: large-scale scene recognition from abbey to zoo. In: Proceedings of the CVPR (2010)

  50. Zhang, J., Marszalek, M., Lazebnik, S., Schmid, C.: Local features and kernels for classification of texture and object catergories: a comprehensive study. Int. J. Comput. Vis. 73(2), 213–218 (2007)

    Article  Google Scholar 

  51. Zujovic, J., Gandy, L., Friedman, S., Pardo, B., Pappas, T.: Classifying paintings by artistic genre: an analysis of features and classifiers. In: Proceedings of the MMSP (2009)

Download references

Acknowledgments

This work has been supported by SSF through a grant for the project CUAS, by VR through a grant for the project ETT, through the Strategic Area for ICT research ELLIIT, and CADICS.

Author information

Authors and Affiliations

  1. Computer Vision Laboratory, Linköping University, Linköping, Sweden

    Fahad Shahbaz Khan & Michael Felsberg

  2. Norwegian Colour and Visual Computing Laboratory, Gjovik University College, Gjøvik, Norway

    Shida Beigpour

  3. Computer Vision Centre Barcelona, Universitat Autonoma de Barcelona, Catalonia, Spain

    Joost van de Weijer

Authors
  1. Fahad Shahbaz Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shida Beigpour
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Joost van de Weijer
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Michael Felsberg
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fahad Shahbaz Khan.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Khan, F.S., Beigpour, S., van de Weijer, J. et al. Painting-91: a large scale database for computational painting categorization. Machine Vision and Applications 25, 1385–1397 (2014). https://doi.org/10.1007/s00138-014-0621-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-014-0621-6

Keywords

Search

Navigation