Small File Read Performance Optimization Based on Redis Cache in Distributed Storage System

Abstract

At present, small files stored in most of the distributed storage systems account for a large proportion, how to improve the storage efficiency of small file has long been a hot issue in the academic community. In the existing methods, researchers usually integrate small files into content-related large files for association storage. The shortcoming is that if we read some small files in a large file, the entire large file must be read into the cache space, which greatly reduces the utilization of the cache space. To solve this defect, this paper designs a scheme based on the combination of cache replacement optimization algorithm and multi-level cache dynamic elimination mechanism. We compute the content heat value of the cache object by the cache replacement optimization algorithm, and then through the multi-level cache dynamic elimination mechanism we expel the object with low content heat value in the cache space to reserve enough space for the new arrived object. This proposed method not only improves the cache space utilization, but also increases the cache data search efficiency and hit rate. Series of experiments and their result show that the cache hit rate obtained by our proposed method is 14% higher than the LRU cache replacement algorithm and 52% higher than the LFU cache replacement algorithm, which can optimize the reading performance of small files based on Redis cache.