Abstract
This paper describes an open architecture for distributed Internet search engines and the experience derived from implementation of a conforming prototype. The architecture enables competing collaboration of independent information retrieval service providers. It allows integration of multiple cheap private servers into a powerful distributed system, which still guards independence and commercial interests of every player. Special emphasis was made on demonstrating the ability of the architecture to make effective use of latest advances in information retrieval technology. Prototype implementation has proved the feasibility of the approach. It has also exposed a wide area of optimisations desirable at the component level. The source code of the prototype is publicly available.
This work was undertaken as a part of OASIS project (INCO Copernicus PL96 1116) funded by the Commission of European Communities
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Anders Ardo and Sigfrid Lunberg. A regional distributed WWW search and indexing service — the DESIRE way. In Proc. of the Seventh International World Wide Web Conference. Elsevier Science, 1998.
C.M. Bowman, P.B. Danzig, D.R. Hardy, U. Manber, and M.F Schwartz. Scalable internet resource discovery: Research problems and approaches. Communications of the ACM, 37(8):98–107, 1994.
C.M Bowman, P.B. Danzig, D.R. Hardy, U. Manber, and M.F. Schwartz. The harvest information discovery and access system. Computer Networks and ISDN Systems, 28(1–2):119–125, 1995.
James Callan, Zhihong Lu, and Bruce Croft. Searching distributed collections with inference networks. In Proc. of the 18th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 1995.
Andrzej Duda and Mark Sheldon. Content routing in a network of WAIS servers. In Proc. of the 14th International Conference on Distributed Computing Systems, pages 124–132. IEEE Computer Society Press, 1994.
B. Fritzke. A growing neural gas network learns topologies. Advances in Neural Information Processing Systems, 7:625–632, 1995.
L. Gravano, C. K. Chang, H. Garcia-Molina, and A. Paepcke. Starts: Stanford proposal for internet meta-searching. In Proc. of the International Conference on Management of Data, 1997.
L. Gravano, H. Garcia-Molina, and A. Tomasic. Precision and recall of GlOSS estimators for database discovery. In Proc. of the 3rd International Conference on Parallel and Distributed Information Systems (PDIS’94), 1994.
[9] OMG Group. The Common Object Request Broker: Architecture and Specification. July 1995.
T. Koch, A. Ard, A. Bremmer, and S. Lundberg. The building and maintenance of robot based internet search services: A review of current indexing and data collection methods, 1998.
T. Kohonen. Self-Organizing Maps. Springer-Verlag, 1995.
T.M. Martinetz and K.J. Schulten. A “neural-gas” network learns topologies. Artificial Neural Networks, pages 397–402, 1991.
Igor Nekrestyanov, Tadhg O’Meara, and Ekaterina Romanova. Building topic-specific collections with intelligent agents. In Proc. of the Sixth International Conference on Intelligence in Services and Networks, April 1999.
OASIS project consortium. Distributed search algorithms specification. INCO Copernicus PL961116 deliverable D3.4.
G. Salton and M.J. McGill. Introduction to Modern Information Retrieval. McGraw-Hill, New York, 1983.
E. Schikuta and M. Erhart. The bang-clustering system: Grid-based data analysis. Advances in Intelligent Data Analysis (IDA-97), pages 513–524, 1997.
H. Speckmann. Analyse mit fraktale Dimensionen und Parallelesierung von Kohonens selbstorganisierender Karte. PhD thesis, University of Tuebingen, 1995.
R. Weiss, B. Velez, M. Sheldon, C. Namprempre, P. Szilagyi, A. Duda, and D. Gifford. HyPursuit: A hierarchical network search engine that exploits content-link hypertext clustering. In Hypertext’96, The Seventh ACM Conference on Hypertext, pages 180–193. ACM Press, 1996.
L. Xu, A. Krzyzak, and E. Oja. Rival penalized competitive learning for clustering analysis, rbf net, and curve detection. IEEE Transactions on Neural Networks, 4(4), July 1993.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1999 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bessonov, M., Heuser, U., Nekrestyanov, I., Patel, A. (1999). Open Architecture for Distributed Search Systems. In: Zuidweg, H., Campolargo, M., Delgado, J. (eds) Intelligence in Services and Networks Paving the Way for an Open Service Market. IS&N 1999. Lecture Notes in Computer Science, vol 1597. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-48888-X_7
Download citation
DOI: https://doi.org/10.1007/3-540-48888-X_7
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-65895-5
Online ISBN: 978-3-540-48888-0
eBook Packages: Springer Book Archive