Skip to main content

Advertisement

SpringerLink
Log in

Sentic Album: Content-, Concept-, and Context-Based Online Personal Photo Management System

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

The world of online personal photo management has come a long way in the past few years, but today, there are still huge gaps in annotating, organizing, and retrieving online pictures in such a way that they can be easily queried and visualized. Existing content-based image retrieval systems apply statistics, pattern recognition, signal processing, and computer vision techniques but these are still too weak to ‘bridge the semantic gap’ between the low-level data representation and the high-level concepts the user associates with images. Image meta search engines, on the other hand, rely on tags associated with online pictures but results are often too inaccurate since they mainly depend on keyword-based rather than concept-based algorithms. Sentic Album is a novel content-, concept-, and context-based online personal photo management system that exploits both data and metadata of online personal pictures to intelligently annotate, organize, and retrieve them. Many salient features of pictures, in fact, are only noticeable in the viewer’s mind, and the cognitive ability to grasp such features is a key aspect for accordingly analyzing and classifying personal photos. To this end, Sentic Album exploits not just colors and texture of online images (content), but also the cognitive and affective information associated with their metadata (concept), and their relative timestamp, geolocation, and user interaction metadata (context).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. http://picasa.google.com.

  2. http://flickr.com.

  3. http://facebook.com.

  4. http://myspace.com.

  5. http://google.com/images.

  6. http://images.search.yahoo.com.

  7. http://bing.com/images.

  8. http://semedia.dibet.univpm.it/heo/heo.owl.

  9. http://code.google.com/apis/gdata.

  10. http://pythonware.com/products/pil.

  11. http://python.org.

  12. http://w3.org/TR/mediaont-10.

  13. http://www.foaf-project.org.

  14. http://simile-widgets.org/exhibit.

References

  1. Bach J, Fuller C, Gupta A, Hampapur A, Horowitz B, Humphrey R, Jain R, Shu C. Virage image search engine: an open framework for image management. In: Sethi I, Jain R, editors. Storage and retrieval for still image and video databases, vol. 2670. Bellingham: SPIE; 1996. p. 76–87.

    Google Scholar 

  2. Bianchi-Berthouze N. K-DIME: an affective image filtering system. IEEE Multimedia. 2003; 10(3):103–6.

    Article  Google Scholar 

  3. Bonanno G, Papa A, O’Neill K, Westphal M, Coifman K. The importance of being flexible: the ability to enhance and suppress emotional expressions predicts long-term adjustment. Psychol Sci. 2004; 15:482–7.

    Article  PubMed  Google Scholar 

  4. Burke A, Heuer F, Reisberg D. Remembering emotional events. Memory Cogn. 1992; 20:277–90.

    Article  CAS  Google Scholar 

  5. Cambria E, Benson T, Eckl C, Hussain A. Sentic PROMs: application of sentic computing to the development of a novel unified framework for measuring health-care quality. Expert Systems with Applications, Elsevier. 2012. doi:10.1016/j.eswa.2012.02.120.

  6. Cambria E, Grassi M, Hussain A, Havasi C. Sentic computing for social media marketing. Multimedia Tools Appl. 2011. doi:10.1007/s11042-011-0815-0.

  7. Cambria E, Hussain A. Sentic computing: techniques, tools, and applications. Berlin: Springer; 2012.

    Google Scholar 

  8. Cambria E, Hussain A, Durrani T, Havasi C, Eckl C, Munro J. Sentic computing for patient centered application. In: Proceedings of IEEE ICSP. Beijing; 2010. p. 1279–82.

  9. Cambria E, Hussain A, Havasi C, Eckl C. AffectiveSpace: blending common sense and affective knowledge to perform emotive reasoning. In: Proceedings of CAEPIA. Seville; 2009. p. 32–41.

  10. Cambria E, Livingstone A, Hussain A. The Hourglass of Emotions. LNCS, Cognitive Behavioral Systems. Springer-Verlag, Berlin Heidelberg; 2012.

  11. Cambria E, Mazzocco T, Hussain A, Eckl C. Sentic medoids: organizing affective common sense knowledge in a multi-dimensional vector space. In: Advances in neural networks. Lecture notes in computer science, vol. 6677. Berlin: Springer; 2011. p. 601–10.

  12. Cambria E, Olsher D, Kwok K. Sentic panalogy: swapping affective common sense reasoning strategies and foci. In: Proceedings of CogSci. Sapporo; 2012.

  13. Cambria E, Olsher D, Kwok K. Sentic activation: a two-level affective common sense reasoning framework. In: Proceedings of AAAI. Toronto; 2012.

  14. Chi P, Lieberman H. Intelligent assistance for conversational storytelling using story patterns. In: IUI. Palo Alto 2011.

  15. Christianson S, Loftus E. Remembering emotional events: the fate of detailed information. Cogn Emot. 1991;5:81–108.

    Article  Google Scholar 

  16. Damasio A. Descartes’ error: emotion, reason, and the human brain. New York: Grossett/Putnam; 1994.

    Google Scholar 

  17. Datta R, Wang J. ACQUINE: aesthetic quality inference engine—real-time automatic rating of photo aesthetics. In: Proceedings of the international conference on multimedia information retrieval. Philadelphia; 2010.

  18. Decherchi S, Gastaldo P, Redi J, Zunino R, Cambria E. Circular-ELM for the reduced-reference assessment of perceived image quality. Neurocomputing. (in press).

  19. Duda R, Hart P. Pattern classification and scene analysis. New York: Wiley; 1973.

    Google Scholar 

  20. Eckart C, Young G. The approximation of one matrix by another of lower rank. Psychometrika. 1936;1(3):211–8.

    Article  Google Scholar 

  21. Elliott CD. The affective reasoner: A process model of emotions in a multi-agent system. Ph.D. thesis, Northwestern University, Evanston; 1992.

  22. Fellbaum C. WordNet: An electronic lexical database (language, speech, and communication). Cambridge: The MIT Press; 1998.

    Google Scholar 

  23. Flickner M, Sawhney H, Niblack W, Ashley J, Huang Q, Dom B, Gorkani M, Hafner J, Lee D, Petkovic D, Steele D, Yanker P. Query by image and video content: the QBIC system. Computer. 1995;28(9):23–32.

    Article  Google Scholar 

  24. Frankel C, Swain MJ, Athitsos V. WebSeer: an image search engine for the world wide web. Technical report. Chicago: University of Chicago; 1996.

    Google Scholar 

  25. Garey M, Johnson D. Computers and intractability: a guide to the theory of NP-completeness. San Francisco: Freeman; 1979.

    Google Scholar 

  26. Goertzel B, Silverman K, Hartley C, Bugaj S, Ross M. The baby webmind project. In: Proceedings of AISB. Birmingham; 2000.

  27. Grassi M. Developing HEO human emotions ontology. In: Fierrez J, Ortega-Garcia J, Esposito A, Drygajlo A, Faundez-Zanuy M, editors. Biometric ID management and multimodal communication. Lecture notes in computer science, vol. 5707. Berlin: Springer; 2009. p. 244–51.

  28. Hanjalic A. Extracting moods from pictures and sounds: towards truly personalized TV. IEEE Signal Process Mag. 2006;23(2):90–100.

    Article  Google Scholar 

  29. Hartigan J, Wong M. Algorithm AS 136: a k-means clustering algorithm. J R Stat Soc. 1979;28(1):100–8.

    Google Scholar 

  30. Havasi C, Speer R, Alonso J. ConceptNet 3: a flexible, multilingual semantic network for common sense knowledge. In: Proceedings of RANLP. Borovets; 2007.

  31. Havasi C, Speer R, Holmgren J. Automated color selection using semantic knowledge. In: Proceedings of AAAI CSK. Arlington; 2010.

  32. Havasi C, Speer R, Pustejovsky J, Lieberman H. Digital intuition: applying common sense using dimensionality reduction. IEEE Intell Syst. 2009;24(4):24–35.

    Article  Google Scholar 

  33. Hu M, Liu B. Mining opinion features in customer reviews. In: Proceedings of AAAI. San Jose; 2004.

  34. Huang J, Ravi S, Mitra M, Zhu W, Zabih R. Image indexing using color correlograms. In: Proceedings of IEEE CVPR, 1997. p. 762–8.

  35. Itten J. The art of color: the subjective experience and objective rationale of color. New York: Wiley; 1973.

    Google Scholar 

  36. Jing F, Wang C, Yao Y, Deng K, Zhang L, Ma WY. IGroup: web image search results clustering. In: Proceedings of ACM Multimedia. Santa Barbara; 2006.

  37. Kaufman L, Rousseeuw P. Finding groups in data: an introduction to cluster analysis. London: Wiley; 1990.

    Book  Google Scholar 

  38. Keelan B. (2002) Handbook of image quality. New York: Marcel Dekker; 2002.

    Book  Google Scholar 

  39. Lakoff G. Women, fire, and dangerous things. Chicago: University of Chicago Press; 1990.

    Google Scholar 

  40. Laney C, Campbell H, Heuer F, Reisberg D. Memory for thematically arousing events. Memory Cogn. 2004;32(7):1149–59.

    Article  Google Scholar 

  41. Lansdale M, Edmonds E. Using memory for events in the design of personal filing systems. Int J Man Mach Stud. 1992;36(1):97–126.

    Article  Google Scholar 

  42. Lascu A, Cambria E, Grassi M. Human semiotics ontology. In: Proceedings of ICMC. Venice; 2011. p. 152.

  43. Lee B, Hendler J, Lassila O. The semantic web. Scientific American 2001.

  44. Lempel R, Soffer A. PicASHOW: pictorial authority search by hyperlinks on the web. In: Proceedings of WWW. Hong Kong; 2001.

  45. Lew M, Sebe N, Djeraba C, Jain R. Content-based multimedia information retrieval: state of the art and challenges. ACM Trans Multimedia Comput Commun Appl. 2006;2(1):1–19.

    Article  Google Scholar 

  46. Lieberman H, Rosenzweig E, Singh P. ARIA: an agent for annotating and retrieving images. IEEE Comput. 2001;34(7):57–62.

    Article  Google Scholar 

  47. Lieberman H, Selker T. Out of context: computer systems that adapt to, and learn from, context. IBM Syst J. 2000;39(3):617–32.

    Article  Google Scholar 

  48. Lu W, Zeng K, Tao D, Yuan Y, Gao X. No-reference image quality assessment in contourlet domain. Neurocomputing. 2012;73(4–6):784–94.

    Article  Google Scholar 

  49. Machajdik J, Hanbury A. Affective image classification using features inspired by psychology and art theory. In: Proceedings of the international conference on multimedia. Florence; 2010.

  50. Minsky M. The emotion machine: commonsense thinking, artificial intelligence, and the future of the human mind. New York: Simon & Schuster; 2006.

    Google Scholar 

  51. Motik B, Sattler U, Studer R. Query answering for OWL-DL with rules. 2004. p. 549–63.

  52. Nakazato M, Manola L, Huang T. ImageGrouper: Search, annotate and organize images by groups. In: Chang S, Chen Z, Lee S editors. Recent advances in visual information systems. Lecture notes in computer science, vol. 2314. Berlin: Springer; 2002. p. 93–105.

  53. Narwaria M, Lin W. Objective image quality assessment based on support vector regression. IEEE Trans Neural Netw. 2010;12(3):515–19.

    Article  Google Scholar 

  54. O’Hare N, Lee H, Cooray S, Gurrin C, Jones G, Malobabic J, O’Connor N, Smeaton A, Uscilowski B. MediAssist: Using content-based analysis and context to manage personal photo collections. In: Proceedings of CIVR, Tempe; 2006. p. 529–32.

  55. Ortony A, Clore G, Collins A. The cognitive structure of emotions. Cambridge: Cambridge University Press; 1988.

    Book  Google Scholar 

  56. Pantic M. Affective computing. In: Encyclopedia of multimedia technology and networking, vol. 1. Idea Group Reference; 2005. p. 8–14.

  57. Park H, Jun C. A simple and fast algorithm for k-medoids clustering. Exp Syst Appl. 2009;36(2):3336–41.

    Article  Google Scholar 

  58. Plutchik R. The nature of emotions. Am Sci. 2001;89(4):344–50.

    Google Scholar 

  59. Porkaew K, Chakrabarti K. Query refinement for multimedia similarity retrieval in MARS. In: Proceedings of ACM international conference on multimedia. New York: ACM; 1999. p. 235–8.

  60. Redi J, Gastaldo P, Heynderickx I, Zunino R. Color distribution information for the reduced-reference assessment of perceived image quality. IEEE Trans Circuits Syst Video Technol. 2012;20(12):1757–69.

    Article  Google Scholar 

  61. Reisberg D, Heuer F. Memory for emotional events. Memory Emot. 2004:3–41.

  62. Richards J, Butler E, Gross J. Emotion regulation in romantic relationships: the cognitive consequences of concealing feelings. J. Social Pers. Relation. 2003;20:599–620.

    Article  Google Scholar 

  63. Sebe N, Tian Q, Loupias E, Lew MS, Huang TS. Evaluation of salient point techniques. In: Proceedings of the international conference on image and video retrieval. London: Springer; 2002. p. 367–77.

  64. Smith J, Chang S. An image and video search engine for the world-wide web. In: Symposium on electronic imaging: science and technology 1997.

  65. Somasundaran S, Wiebe J, Ruppenhofer J. Discourse level opinion interpretation. In: Proceedings of COLING. Manchester; 2008.

  66. Strapparava C, Valitutti A. WordNet-Affect: An affective extension of WordNet. In: Proceedings of LREC. Lisbon; 2004.

  67. Urban J, Jose J, Van Rijsbergen C. An adaptive approach towards content-based image retrieval. Multimedia Tools Appl. 2006;31:1–28.

    Article  Google Scholar 

  68. Urban J, Jose JM. EGO: a personalized multimedia management and retrieval tool. Int J Intell Syst. 2006;21(7):725–45.

    Article  Google Scholar 

  69. Valdez P, Mehrabian A. Effects of color on emotions. J Exp Psychol General. 1994;123(4):394–409.

    Article  PubMed  CAS  Google Scholar 

  70. Vesterinen E. Affective computing. In: Digital media research seminar. Helsinki; 2001.

  71. Wall M, Rechtsteiner A, Rocha L. Singular value decomposition and principal component analysis. In: Berrar D, Dubitzky W, Granzow M (eds) A practical approach to microarray data analysis. Berlin: Springer; 2003. pp. 91–109.

    Chapter  Google Scholar 

  72. Wang W, He Q. A survey on emotional semantic image retrieval. In: Proceedings of IEEE ICIP, 2008. p. 117–20.

  73. Wessel I, Merckelbach H. The impact of anxiety on memory for details in spider phobics. Appl Cogn Psychol. 1997;11:223–31.

    Article  Google Scholar 

  74. Wiebe J, Wilson T, Cardie C. Annotating expressions of opinions and emotions in language. Lang Resour Eval. 2005;39(2):165–210.

    Article  Google Scholar 

  75. Wilson T, Wiebe J, Hoffmann P. Recognizing contextual polarity in phrase-level sentiment analysis. In: Proceedings of HLT/EMNLP. Vancouver; 2005.

Download references

Acknowledgments

This work has been part-funded by Hewlett–Packard Labs India, the UK Engineering and Physical Sciences Research Council (EPSRC Grant Reference: EP/G501750/1) and Sitekit Solutions Ltd. (UK). We would like to thank Praphul Chandra for providing the application case study and the HP industrial placement opportunity.

Author information

Authors and Affiliations

  1. Temasek Laboratories, National University of Singapore, 117411, Singapore, Singapore

    Erik Cambria

  2. Department of Computing Science and Mathematics, University of Stirling, Stirling, FK9 4LA, UK

    Amir Hussain

Authors
  1. Erik Cambria
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Amir Hussain
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Erik Cambria.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Cambria, E., Hussain, A. Sentic Album: Content-, Concept-, and Context-Based Online Personal Photo Management System. Cogn Comput 4, 477–496 (2012). https://doi.org/10.1007/s12559-012-9145-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-012-9145-4

Keywords

Search

Navigation