A free-text retrieval index-server on a massively parallel machine

Abstract

Several advancements in recent years have made possible the development of products based on parallel hardware platforms suitable for the conventional commercial marketplace. Probably the single most important factor is the acceptance of client/server models by large organizations in place of traditional main-frame solutions. In the client/server world all server systems are almost by definition “open” even if they are based on exotic hardware, since they can be replaced at any time by another server with a similar software interface.

The searching for indexes in a free-text retrieval system is a perfect problem for a parallel computer in a client/server environment. Such a system has been developed for the SIMD architecture of the 4096-processor DAP system. The paper discusses the main characteristics of index-servers in general and how such systems can benefit from implementation on massively parallel systems. Examples of performance improvements are given which show over 100 times improvements over conventional implementations. Integration with existing document and image retrieval systems is also dealt with.