Abstract
Automatic image annotation (AIA) is an effective technique to bridge the semantic gap between low level image features and high level semantics. However, most of the existing AIA approaches failed to consider the use of unlabeled data. In this paper, we present an interactive semi-supervised approach for AIA by integrating graph propagation model and kernel canonical correlation analysis (KCCA) together. We aim to jointly utilize the keywords associated with labeled and selected unlabeled images to annotate the residual unlabeled images. Toward this goal, we firstly estimate the annotations of unlabeled images by the consistency-driven graph propagation model. Then, the KCCA is applied to seek the semantic consistency between the two concurrent visual and textual features. In addition, the unlabeled image with highest semantic consistency is selected into the training set. Thus, with the enlarged training set, the potential of the semantic consistency between visual and textual representations could be boosted. Some experiments carried out on two standard databases validate the effectiveness of the proposed method.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Barnard, K., Duygulu, P., Forsyth, D., de Freitas, N., Blei, D.M., Jordan, M.I.: Matching words and pictures. J. Mach. Learn. Res., 1107–1135 (2003)
Monay, F., Gatica-Perez, D.: Plsa-based image auto-annotation: constraining the latent space. In: Proceedings of the 12th Annual ACM International Conference on Multimedia, pp. 348–351. ACM (2004)
Yakhnenko, O., Honavar, V.: Annotating images and image objects using a hierarchical dirichlet process model. In: Proceedings of the 9th International Workshop on Multimedia Data Mining: Held in Conjunction with the ACM SIGKDD 2008, pp. 1–7. ACM (2008)
Feng, S., Manmatha, R., Lavrenko, V.: Multiple bernoulli relevance models for image and video annotation. In: CVPR, vol. 2, pp. 1002–1009 (2004)
Carneiro, G., Chan, A.B., Moreno, P.J., Vasconcelos, N.: Supervised learning of semantic classes for image annotation and retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 29, 394–410 (2007)
Grangier, D., Bengio, S.: A discriminative kernel-based approach to rank images from text queries. TPAMI 30, 1371–1384 (2008)
Xu, X., Jiang, Y., Peng, L., Xue, X., Zhou, Z.: Ensemble approach based on conditional random field for multi-label image and video annotation. In: MM, pp. 1377–1380. ACM (2011)
Makadia, A., Pavlovic, V., Kumar, S.: A New Baseline for Image Annotation. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, pp. 316–329. Springer, Heidelberg (2008)
Zhang, S., Huang, J., Huang, Y., Yu, Y., Li, H., Metaxas, D.: Automatic image annotation using group sparsity. In: CVPR, pp. 3312–3319 (2010)
He, J., Li, M., Zhang, H., Tong, H., Zhang, C.: Manifold-ranking based image retrieval. In: Proceedings of the 12th Annual ACM International Conference on Multimedia, pp. 9–16. ACM (2004)
Liu, J., Li, M., Liu, Q., Lu, H., Ma, S.: Image annotation via graph learning. Pattern Recognition 42, 218–228 (2009)
Hotelling, H.: Relations Between Two Sets of Variates. Biometrika 28, 321–377 (1936)
Hardoon, D.R., Saunders, C., Szedmak, S., Shawe-Taylor, J.: A Correlation Approach for Automatic Image Annotation. In: Li, X., Zaïane, O.R., Li, Z.-H. (eds.) ADMA 2006. LNCS (LNAI), vol. 4093, pp. 681–692. Springer, Heidelberg (2006)
Zhou, D., Bousquet, O., Lal, T.N., Weston, J., Schölkopf, B.: Learning with local and global consistency. In: Advances in Neural Information Processing Systems 16, pp. 321–328. MIT Press (2004)
Wang, C., Jing, F., Zhang, L., Zhang, H.J.: Image annotation refinement using random walk with restarts. In: MM, MULTIMEDIA 2006, New York, NY, USA, pp. 647–650 (2006)
Duygulu, P., Barnard, K., de Freitas, J.F.G., Forsyth, D.: Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002, Part IV. LNCS, vol. 2353, pp. 97–112. Springer, Heidelberg (2002)
Oliva, A., Torralba, A.: Modeling the shape of the scene: A holistic representation of the spatial envelope. In: ICCV, vol. 42, pp. 145–175 (2001)
Tong, H., He, J., Li, M., Ma, W.Y., Zhang, H.J., Zhang, C.: Manifold-ranking-based keyword propagation for image retrieval. EURASIP J. Appl. Signal Process., 190 (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Xiao, Y., Zhu, Z., Liu, N., Zhao, Y. (2012). An Interactive Semi-supervised Approach for Automatic Image Annotation. In: Lin, W., et al. Advances in Multimedia Information Processing – PCM 2012. PCM 2012. Lecture Notes in Computer Science, vol 7674. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-34778-8_70
Download citation
DOI: https://doi.org/10.1007/978-3-642-34778-8_70
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-34777-1
Online ISBN: 978-3-642-34778-8
eBook Packages: Computer ScienceComputer Science (R0)