Skip to main content
SpringerLink
Log in

On URL Normalization

  • Conference paper
Computational Science and Its Applications – ICCSA 2005 (ICCSA 2005)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 3481))

Included in the following conference series:

Abstract

Since syntactically different URLs could represent the same resource in WWW, there are on-going efforts to define the URL normalization in the standard communities. This paper considers the three additional URL normalization steps beyond ones specified in the standard URL normalization. The idea behind our work is that in the URL normalization we want to minimize false negatives further while allowing false positives in a limited level. Two metrics are defined to analyze the effect of each step in the URL normalization. Over 170 million URLs that were collected in the real web pages, we did an experiment, and interesting statistical results are reported in this paper.

This work was supported by Korea Research Foundation Grant. (KRF-2004-005-D00172)

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Berners-Lee, T., Fielding, R., Masinter, L.: Uniform Resource Identifiers (URI): Generic Syntax (2004), http://gbiv.com/protocols/uri/rev-2002/rfc2396bis.html

  2. Burner, M.: Crawling Towards Eternity: Building an Archive of the World Wide Web. Web Techniques Magazine 2(5), 37–40 (1997)

    Google Scholar 

  3. Heydon, A., Najork, M.: Mercator: A Scalable, Extensible Web Crawler. International Journal of WWW 2(4), 219–229 (1999)

    Google Scholar 

  4. Kim, S.J., Lee, S.H.: Implementation of a Web Robot and Statistics on the Korean Web. In: Chung, C.-W., Kim, C.-k., Kim, W., Ling, T.-W., Song, K.-H. (eds.) HSI 2003. LNCS, vol. 2713, pp. 341–350. Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  5. Kim, S.J., Lee, S.H.: How Web Pages Change: An Empirical Study, submitted for publications (2004)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computing, Soongsil University, Seoul, Korea

    Sang Ho Lee & Seok Hoo Hong

  2. School of Computer Science and Engineering, Seoul National University, Seoul, Korea

    Sung Jin Kim

Authors
  1. Sang Ho Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sung Jin Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Seok Hoo Hong
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Mathematics and Computer Science, University of Perugia, via Vanvitelli, 1, I-06123, Perugia, Italy

    Osvaldo Gervasi

  2. Department of Computer Science, University of Calgary, 2500 University Drive N.W., T2N 1N4, Calgary, AB, Canada

    Marina L. Gavrilova

  3. William Norris Professor, Head of the Computer Science and Engineering Department, University of Minnesota, USA

    Vipin Kumar

  4. Department of Chemistry, University of Perugia, Via Elce di Sotto, 8, P.O. Box, I-06123, Perugia, Italy

    Antonio Laganà

  5. Institute of High Performance Computing, IHCP, 1 Science Park Road, 01-01 The Capricorn, Singapore Science Park II, 117528, Singapore

    Heow Pueh Lee

  6. School of Computing, Soongsil University, Seoul, Korea

    Youngsong Mun

  7. Clayton School of IT, Monash University, 3800, Clayton, Australia

    David Taniar

  8. OptimaNumerics Ltd, P.O. Box, Belfast, United Kingdom

    Chih Jeng Kenneth Tan

Rights and permissions

Reprints and permissions

Copyright information

© 2005 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Lee, S.H., Kim, S.J., Hong, S.H. (2005). On URL Normalization. In: Gervasi, O., et al. Computational Science and Its Applications – ICCSA 2005. ICCSA 2005. Lecture Notes in Computer Science, vol 3481. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11424826_115

Download citation

  • DOI: https://doi.org/10.1007/11424826_115

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-25861-2

  • Online ISBN: 978-3-540-32044-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation