Abstract
The fourth CLEF campaign, CLEF 2003, was marked by a large increase in the number of experiments submitted by participants. More complex tasks (multilingual retrieval on documents written in eight different languages), new document sources, and a new, ninth document language, Russian, led to a wide variety of work being conducted in this campaign. This paper discusses the tracks that have been offered since the beginnings of CLEF, and that still have the highest participation, i.e. the ad-hoc and domain-specific tracks. Collectively, these tracks attracted 33 participating groups who submitted more than 400 experiments. This overview gives a brief description of the tracks and tasks, and summarizes the principal research results. A discussion of the validity of the results, in terms of the completeness of the relevance assessments used in the test collection and of the statistical significance of some of the results, is also given.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Blustein, J.: IR STAT PAK, URL: http://www.csd.uwo.ca/~jamie/IRSP-overview.html
Braschler, M.: CLEF 2000 - Overview of Results. In: Peters, C. (ed.) CLEF 2000. LNCS, vol. 2069, pp. 89–101. Springer, Heidelberg (2001)
Braschler, M.: CLEF 2001 - Overview of Results. In: Peters, C., Braschler, M., Gonzalo, J., Kluck, M. (eds.) CLEF 2001. LNCS, vol. 2406, pp. 9–26. Springer, Heidelberg (2002)
Braschler, M.: CLEF 2002 – Overview of Results. In: Peters, C., Braschler, M., Gonzalo, J. (eds.) CLEF 2002. LNCS, vol. 2785, pp. 9–27. Springer, Heidelberg (2003)
Chen, A., Gey, F.C.: Combining Query Translation and Document Translation in Cross-Language Retrieval. In: Peters, C., Gonzalo, J., Braschler, M., Kluck, M. (eds.) CLEF 2003. LNCS, vol. 3237, pp. 108–121. Springer, Heidelberg (2004)
Clough, P., Sanderson, M.: The CLEF 2003 Cross Language Image Retrieval Track. In: Peters, C., Gonzalo, J., Braschler, M., Kluck, M. (eds.) CLEF 2003. LNCS, vol. 3237, pp. 581–593. Springer, Heidelberg (2004)
Cross-Language Evaluation Forum: Results of the CLEF 2003 Cross-Language System Evaluation Campaign. In: Working Notes for the CLEF 2003 Workshop, vol. II: Appendices. Electronic version, Available at: www.clef-campaign.org
Federico, M., Jones, G.J.F.: The CLEF 2003 Cross-Language Spoken Document Retrieval Track. In: Peters, C., Gonzalo, J., Braschler, M., Kluck, M. (eds.) CLEF 2003. LNCS, vol. 3237, pp. 646–652. Springer, Heidelberg (2004)
Harman, D., Braschler, M., Hess, M., Kluck, M., Peters, C., Schäuble, P., Sheridan, P.: CLIR Evaluation at TREC. In: Peters, C. (ed.) CLEF 2000. LNCS, vol. 2069, pp. 7–23. Springer, Heidelberg (2001)
Hull, D.A.: Using Statistical Testing in the Evaluation of Retrieval Experiments. In: Proceedings of the 16th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Pittsburg, USA, pp. 329–338 (1993)
Kamps, J., Monz, C., de Rijke, M., Sigurbjörnsson, B.: Language-Dependent and Language-Independent Approaches to Cross-Lingual Text Retrieval. In: Peters, C., Gonzalo, J., Braschler, M., Kluck, M. (eds.) CLEF 2003. LNCS, vol. 3237, pp. 152–165. Springer, Heidelberg (2004)
Magnini, B., Romagnoli, S., Vallin, A., Herrera, J., Peñas, A., Peinado, V., Verdejo, M.F., de Rijke, M.: The Multiple Language Question Answering Track at CLEF 2003. In: Peters, C., Gonzalo, J., Braschler, M., Kluck, M. (eds.) CLEF 2003. LNCS, vol. 3237, pp. 471–486. Springer, Heidelberg (2004)
Oard, D.W., Gonzalo, J.: The CLEF 2003 Interactive Track. In: Peters, C., Gonzalo, J., Braschler, M., Kluck, M. (eds.) CLEF 2003. LNCS, vol. 3237, pp. 425–434. Springer, Heidelberg (2004)
Peters, C., Braschler, M.: European Research Letter: Cross-language System Evaluation: The CLEF campaigns. Journal of the American Society for Information Science and Technology 52(12), 1067–1072 (2001)
Savoy, J.: Report on CLEF-2003 Multilingual Tracks. In: Peters, C., Gonzalo, J., Braschler, M., Kluck, M. (eds.) CLEF 2003. LNCS, vol. 3237, pp. 64–73. Springer, Heidelberg (2004)
Tague-Sutcliffe, J.: The Pragmatics of Information Retrieval Experimentation, Revisited. In: Readings in Information Retrieval, pp. 205–216. Morgan Kaufmann Publishers, San Francisco (1997)
Tague-Sutcliffe, J., Blustein, J.: A Statistical Analysis of the TREC-3 Data. In: Proc. TREC-3, NIST Special Publication 500-226, p. 385 (1994)
Voorhees, E., Buckley, C.: The Effect of Topic Set Size on Retrieval Experiment Error. In: Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 316–323 (2002)
Vorhees, E., Harman, D.: Overview of the Eighth Text REtrieval Conference (TREC-8). In: Proc. TREC-8, NIST Special Publication 500-246, pp. 1–24 (1999)
Zobel, J.: How reliable are the results of large-scale information retrieval experiments? In: Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 307–314 (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Braschler, M. (2004). CLEF 2003 – Overview of Results. In: Peters, C., Gonzalo, J., Braschler, M., Kluck, M. (eds) Comparative Evaluation of Multilingual Information Access Systems. CLEF 2003. Lecture Notes in Computer Science, vol 3237. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30222-3_5
Download citation
DOI: https://doi.org/10.1007/978-3-540-30222-3_5
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-24017-4
Online ISBN: 978-3-540-30222-3
eBook Packages: Springer Book Archive