Skip to main content
SpringerLink
Log in

Complex-query web image search with concept-based relevance estimation

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

Complex queries are widely used in current Web applications. They express highly specific information needs, but simply aggregating the meanings of primitive visual concepts does not perform well. To facilitate image search of complex queries, we propose a new image reranking scheme based on concept relevance estimation, which consists of Concept-Query and Concept-Image probabilistic models. Each model comprises visual, web and text relevance estimation. Our work performs weighted sum of the underlying relevance scores, a new ranking list is obtained. Considering the Web semantic context, we involve concepts by leveraging lexical and corpus-dependent knowledge, such as Wordnet and Wikipedia, with co-occurrence statistics of tags in our Flickr corpus. The experimental results showed that our scheme is significantly better than the other existing state-of-the-art approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10

Similar content being viewed by others

References

  1. Bai, J., Song, D., Bruza, P., Nie, J.-Y., Cao, G.: Query expansion using term relationships in language models for information retrieval. ACM CIKM, pp. 688–695 (2005)

  2. Balasubramanian, N., Kumaran, G., Carvalho, V.: Exploring reductions for long web queries. SIGIR, pp. 571–578 (2010)

  3. Bendersky, M., Croft, W.B.: Discovering key concepts in verbose queries. SIGIR, pp. 491–498 (2008)

  4. Bendersky, M., Croft, W.B.: Analysis of long queries in a large scale search log., Proceedings of workshop on Web Search Click Data, pp. 8–14 (2009)

  5. Bendersky, M., Metzler, D., Croft, W.B.: Learning concept importance using a weighted dependence model. Proceedings of the ACM international conference on Web search and data mining, pp. 31–40 (2010)

  6. Bendersky, M., Metzler, D., Croft, W.B.: Parameterized concept weighting in verbose queries. SIGIR, pp. 605–614 (2011)

  7. Chen, X., Yuan, J., Nie, L., Zha, Z., Yan, S., Chua, T.-S.: TRECVID 2010 known-item search by NUS. TRECVID (2010)

  8. Cilibrasi, R.L., Vitanyi, P.M.B.: The google similarity distance. IEEE Trans. Knowl. Data Eng. 19(3), 370–383 (2007)

    Article  Google Scholar 

  9. Delgado, D., Magalhaes, J., Correia, N.: Assisted news reading with automated illustration. ACM Multimedia, pp. 1647–1650 (2010)

  10. Han, Y., Wu, F., Tian, Q.: Image annotation by input–output structural grouping sparsity. IEEE Trans. Image Proc. 21(6), 3066–3079 (2012)

    Article  MathSciNet  Google Scholar 

  11. Han, Y., Yang, Y., Yan, Y.: Semisupervised feature selection via spline regression for video semantic recognition. IEEE Trans. Neural Netw. Learn. Syst. 26(2), 252–264 (2014)

    Google Scholar 

  12. Hauptmann, A., Yan, R., Lin, W.-H., Christel, M., Wactlar, H.: Can high-level concepts fill the semantic gap in video retrieval? A case study with broadcast news. IEEE Trans. Multimed. 9(5), 958–966 (2007)

    Article  Google Scholar 

  13. Hong, C., Zhu, J.: Hypergraph-based multi-example ranking with sparse representation for transductive learning image retrieval. Neurocomputing 101, 94–103 (2013)

    Article  Google Scholar 

  14. Hong, R.-C., Pan, J., Hao, S., Wang, M., Xue, F., Wu, X.: Image quality assessment based on matching pursuit. Inf. Sci. 273, 196–211 (2014)

    Article  Google Scholar 

  15. Hong, R.-C., Wang, M., Gao, Y., Tao, D., Li, X., Wu, X.: Image annotation by multiple-instance learning with discriminative feature mapping and selection. IEEE Trans. Cybern. 44(5), 669–680 (2014)

    Article  Google Scholar 

  16. Hsu, W.H., Kennedy, L.S., Chang, S.-F.: Video search reranking through random walk over document-level context graph. ACM Multimedia, pp. 971–980 (2007)

  17. Kumaran, G., Allan, J.: A case for shorter queries, and helping users create them., Proceedings of Human Language Technologies: The Annual Conference of the North American Chapter of the Association for Computational Linguistics, pp. 220–227 (2006)

  18. Kumaran, G., Allan, J.: Effective and efficient user interaction for long queries. SIGIR, pp. 11–18 (2008)

  19. Lease, M., Allan, J., Croft, W.B.: Regression rank: Learning to meet the opportunity of descriptive queries. ECIR, pp. 90–101 (2009)

  20. Li, X., Snoek, C.G.M., Worring, M., Smeulders, A.W.M.: Harvesting social images for bi-concept search. IEEE Trans. Multimed. 14(4), 1520–9210 (2010)

    Google Scholar 

  21. Liu, Y., Mei, T., Hua, X.-S., Tang, J.: Learning to video search rerank via pseudo preference feedback. ICME, pp. 297–300 (2008)

  22. Liu, D., Hua, X.-S., Yang, L., Wang. M., Zhang. H. J.: Tag ranking. WWW, pp. 351-360 (2009)

  23. Morioka, N., Wang, J.: Robust visual reranking via sparsity and ranking constraints. ACM Multimedia, pp. 533–542 (2011)

  24. Natsev, A., Haubold, A., Xie, L., Yan, R.: Semantic concept-based query expansion and re-ranking for multimedia retrieval. ACM Multimedia, pp. 991–1000 (2007)

  25. Nie, L., Wang, M., Zha, Z., Li, G., Chua, T.-S.: Multimedia answering: enriching text QA with media information. SIGIR, pp. 695–704 (2011)

  26. Nie, L., Yan, S., Wang, M., Hong, R., Chua, T.-S.: Harvesting visual concepts for image search with complex queries. ACM Multimedia, pp. 59–68 (2012)

  27. Snoek, C.G.M., Worring, M.: Concept-based video retrieval. Found. Trends Inf. Retr. 2(4), 215–322 (2008)

    Article  Google Scholar 

  28. Snoek, C.G.M., Huurnink, B., Hollink, L., de Rijke, M., Schreiber, G., Worring, M.: Adding semantics to detectors for video retrieval. IEEE Trans. Multimed. 9(5), 975–986 (2007)

    Article  Google Scholar 

  29. Szummer, M., Jaakkola, T.: Partially labeled classification with markov random walks. Adv. Neural Inf. Proc. Syst. 14, 945–952 (2002)

    Google Scholar 

  30. Tang, S., Li, J.-T., Li, M., Xie, Cheng., Liu, Y.-Z., Tao, K., Xu, S.-X.: TRECVID 2008 high-level feature extraction By MCG-ICT-CAS. In: Proc. The TRECVID Workshop, 566 (2008)

  31. Tian, X., Yang, L., Wang, J., Yang, Y., Wu, X., Hua, X.-S.: Bayesian video search reranking. ACM Multimedia, pp. 131–140 (2008)

  32. Wang, M., Hua, X.S., Tang, J.H., Hong, R.C.: Beyond distance measurement: constructing neighborhood similarity for video annotation. IEEE Trans. Multimed. 11(3), 465–476 (2009)

    Article  Google Scholar 

  33. Wang, M., Yang, K., Hua, X.-S., Zhang, H.-J.: Towards a relevant and diverse search of social images. IEEE Trans. Multimed. 12(8), 829–842 (2010)

    Article  Google Scholar 

  34. Wei, X.-Y., Jiang, Y.-G., Ngo, C.-W.: Concept-driven multi-modality fusion for video search. IEEE Trans. Circ. Syst. Video Technol. 21(1), 62–73 (2011)

    Article  Google Scholar 

  35. Yan, R., Hauptmann, A., Jin, R.: Multimedia search with pseudo-relevance feedback. ICVR, pp. 238–247 (2003)

  36. Yang, Y., Zhuang, Y.T., Wu, F.: Harmonizing hierarchical manifolds for multimedia document semantics understanding and cross-media retrieval. IEEE Trans. Multimed. 10(3), 437–446 (2008)

    Article  Google Scholar 

  37. Yang, Y., Nie, F., Xu, D.: A multimedia retrieval framework based on semi-supervised ranking and relevance feedback. IEEE Trans. Pattern Anal. Mach. Intell. 34(4), 723–742 (2012)

    Article  MathSciNet  Google Scholar 

  38. Yang Y., Ma Z., Xu Z.: How related exemplars help complex event detection in web videos?. Computer Vision (ICCV), 2013 I.E. International Conference, pp. 2104–2111 (2013)

  39. Yu, J., Rui, Y., Chen, B.: Exploiting click constraints and multi-view features for image re-ranking. IEEE Trans. Multimed. 16(1), 159–168 (2014)

    Article  Google Scholar 

  40. Yu, J., Rui, Y., Tao, D.: Click prediction for web image reranking using multimodal sparse coding. IEEE Trans. Image Proc. 23(5), 2019–2032 (2014)

    Article  MathSciNet  Google Scholar 

  41. Yu, J., Tao, D., Wang, M., et al.: Learning to rank using user clicks and visual features for image retrieval. IEEE Trans. Cybern. 45(4), 767–779 (2014)

    Article  Google Scholar 

  42. Yuan, J., Zha, Z.-J., Zheng, Y.-T., Wang, M., Zhou, X., Chua, T.-S.: Learning concept bundles for video search with complex queries. ACM Multimedia, pp. 453–462 (2011)

  43. Yuan, J., Zha, Z.-J., Zheng, Y.-T., Wang, M., Zhou, X., Chua, T.-S.: Utilizing related samples to enhance interactive concept-based video search. IEEE Trans. Multimed. 13(6), 1520–9210 (2011)

    Article  Google Scholar 

  44. Zha, Z.-J., Wang, M., Zheng, Y.-T., Yang, Y.: Interactive video indexing with statistical active learning. IEEE Trans. Multimed. 14(1), 17–27 (2012)

    Article  Google Scholar 

  45. Zhu S., Wang G., Ngo C.-W.: On the sampling of web images for learning visual concept classifiers. ACM CIVR, pp. 50–57 (2010)

Download references

Acknowledgments

This work was supported by the National Natural Science Foundation of China (NSFC) under grant 61305062 and the Anhui Provincial Natural Science Foundation under grant 1308085QF102.

Author information

Authors and Affiliations

  1. School of Computer and Information, Hefei University of Technology, 193 Tunxi Road, Hefei, Anhui, 230009, China

    Dan Guo & Pengfei Gao

Authors
  1. Dan Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pengfei Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dan Guo.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Guo, D., Gao, P. Complex-query web image search with concept-based relevance estimation. World Wide Web 19, 247–264 (2016). https://doi.org/10.1007/s11280-015-0357-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11280-015-0357-x

Keywords

Search

Navigation