Research and Design of the Clustering System Based on the Column-Store

Abstract

With the rapid development of the network and the increase of the information on the web, rapid access to the database and data mining become very important. Column-store has the advantage of quick read speed, saving the disk I/O, and can be read by uncompressed, which is helpful to acquire knowledge in the massive data. So based on the traditional data mining module, introduce the column store technology. Information base and knowledge base all adopt column store to store and access, and also provide the access interface between the store module and its upper layer module. Then, compute the Minkowski distance and use k-medoids methods in the data clustering module. On the base of the access advantage of the column store and k-medoids methods, this system can improve the speed and the quality of the clustering. The innovation is the application of column store in the clustering system, and provide completed data access interface, using k-medoids methods can detect clusters of arbitrary shape. Computing the Minkowski distance can improve the efficiency of the dissimilarity of objects and clustering speed.