Abstract
Photographs taken by human beings differ from the images that taken by a lifeless device, such as a surveillance camera or a visual sensor on a robot, in that human being intentionally shoot photographs to express his/her feeling or photo-realistically record a scene. This creation process is accomplished by adjusting two factors: the setting of parameters on a camera and the position between the camera and the object which he or she is interested in. In this paper, this procedure is learned using the machine learning technique so that what the interest of the photographer is and what the core content of a photo wants to display can be reversely calculated. A photo retrieval system was built upon the category of interesting regions and the metadata is used for help. The research also explore the argument how local or global feature affects the performance of image retrieval. A novel stochastic segmentation algorithm called region restricted EM algorithm was applied in order to construct the interesting regions. Experimental evaluation on over 7000+ photos taken by 200+ different models of cameras with variety of interests has shown the robustness of our technique.
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-009-0346-0/MediaObjects/11042_2009_346_Fig1_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-009-0346-0/MediaObjects/11042_2009_346_Fig2_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-009-0346-0/MediaObjects/11042_2009_346_Fig3_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-009-0346-0/MediaObjects/11042_2009_346_Fig4_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-009-0346-0/MediaObjects/11042_2009_346_Fig5_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-009-0346-0/MediaObjects/11042_2009_346_Fig6_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-009-0346-0/MediaObjects/11042_2009_346_Fig7_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-009-0346-0/MediaObjects/11042_2009_346_Fig8_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-009-0346-0/MediaObjects/11042_2009_346_Fig9_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-009-0346-0/MediaObjects/11042_2009_346_Fig10_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-009-0346-0/MediaObjects/11042_2009_346_Fig11_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-009-0346-0/MediaObjects/11042_2009_346_Fig12_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-009-0346-0/MediaObjects/11042_2009_346_Fig13_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-009-0346-0/MediaObjects/11042_2009_346_Fig14_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-009-0346-0/MediaObjects/11042_2009_346_Fig15_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-009-0346-0/MediaObjects/11042_2009_346_Fig16_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-009-0346-0/MediaObjects/11042_2009_346_Fig17_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-009-0346-0/MediaObjects/11042_2009_346_Fig18_HTML.gif)
Similar content being viewed by others
References
Achanta R, Estrada F, Wils P, Ssstrunk S (2008) Salient region detection and segmentation. Lect Notes Comput Sci 5008/2008:66–75
Bouveyron C, Girard S, Schmid C (2007) High-dimensional data clustering. Comput Stat Data Anal 52:502–519
Boutell M, Luo J (2004) Bayesian fusion of camera metadata cues in semantic scene classification. In: IEEE computer society conference on computer vision and pattern recognition, vol 2, pp 623–630
Brox T, Weickert J (2004) Level set based image segmentation with multiple regions. Lect Notes Comput Sci 3175/2004:415–423
Brox T, Rousson M, Deriche R, Weickert J (2003) Unsupervised segmentation incorporating colour, texture, and motion. 2756/2003:353–360
Comaniciu D, Meer P (2002) Mean shift: a robust approach toward feature space analysis. IEEE Trans Pattern Anal Mach Intell 24:603–619
Datta R, Joshi D, Li J, Wang JZ (2008) Image retrieval: ideas, influences, and trends of the new age. ACM Comput Surv 40(5):1–60
Dempster AP, Laird NM, Rubin DB (1997) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc Ser B 39:1–38
Deng Y, Manjunath BS (2001) Unsupervised segmentation of color-texture regions in image and video. IEEE Trans Pattern Anal Mach Intell 8:800–810
Desimone R, Duncan J (1995) Neural mechanisms of selective visual attention. Annu Rev Neurosci 18:193–222
Fred ALN, Leitao JMN (2003) A new cluster isolation criterion based on dissimilarity increments. IEEE Trans Pattern Anal Mach Intell 25:944–958
Fussenegger M, Deriche R, Pinz A (2006) A Multiphase level set based segmentation framework with pose invariant shape priors, ACCV (2). Springer, Hyderabad, pp 395–404
Kato Z (1999) Bayesian color image segmentation using reversible jump Markov chain Monte Carlo. University of Szeged, Institute of Informatics
Lee TCM (2000) A minimum description length based image segmentation procedure, and its comparison with a cross-validation based segmentation procedure. J Am Stat Assoc 95:259–270
LIBSVM: a library for support vector machines (2001). Chih-Chung Chang and Chih-Jen Lin. http://www.csie.ntu.edu.tw/~cjlin/libsvm
Luo J, Savakis AE, Singhal A (2005) A Bayesian network-based framework for semantic image understanding. Pattern Recogn 38:919–934
Ma Y, Derksen H, Hong W, Wright J (2007) Segmentation of multivariate mixed data via lossy data coding and compression. IEEE Trans Pattern Anal Mach Intell 29:1546–1562
Mansouri A-R, Mitiche A, Vazquez C (2006) Multiregion competition: a level set extension of region competition to multiple region image partitioning. Comput Vis Image Underst 101:137–150
Privitera CM, Stark LW (2000) Algorithms for defining visual region-of-interest: comparison with eye fixations. IEEE Trans Pattern Anal Mach Intell 22:970–982
Rissanen J (1989) Stochastic complexity in statistical inquiry. World Scientific, Singapore
Schwarz G (1978) Estimating the dimension of a model. Ann Stat 6:461–464
Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intell 22:888–905
Shih FY, Cheng S (2005) Automatic seeded region growing for color image segmentation. Image Vis Comput 23:877–886
Shyu C, Brodley C, Kak A, Kosaka A, Aisen A, Broderick L (1998) In: IEEE workshop on content-based access of image and video libraries. IEEE Computer Society, Washington, DC, pp 30–34
Siagian C, Itti L (2007) Rapid biologically-inspired scene classification using features shared with visual attention. IEEE Trans Pattern Anal Mach Intell 29:300–312
Smeulders A, Worring M, Santini S, Gupta A, Jain R (2000) Content-based image retrieval at the end of the early years. IEEE Trans Pattern Anal Mach Intell 22:1349–1380
Torralba A, Oliva A, Castelhano MS, Henderson JM (2006) Contextual guidance of eye movements and attention in real-world scenes: the role of global features on object search. Psychol Rev 113:766–786
Ueda N, Nakano R, Ghahramani YZ, Hiton GE (2000) SMEM algorithm for mixture models. Neural Comput 12:2109–2128
Walther D, Koch C (2006) Modeling attention to salient proto-objects. Neural Netw 19:1395–1407
Yang S, Kim S-K, Ro YM (2007) Semantic home photo categorization. IEEE Trans Circuits Syst Video Technol 17:324–335
Zhang Z, Chen C, Sun J, Chan KL (2003) EM algorithm for Gaussian mixtures with split-and-merge operation. Pattern Recogn 36:1973–1983
Zhu SC, Yuille A (1996) Region competition: unifying snakes, region growing, and Bayes/MDL for multiband image segmentation. IEEE Trans Pattern Anal Mach Intell 18:884–900
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Li, Z., Fan, J. Exploit camera metadata for enhancing interesting region detection and photo retrieval. Multimed Tools Appl 46, 207–233 (2010). https://doi.org/10.1007/s11042-009-0346-0
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-009-0346-0