Abstract
In this paper we present two improvements to the partitioning process: 1) A new dynamic buffer management strategy is employed to increase the average block size of I/O-transfers to temporary files, and 2) An optimal switching between three different variants of the partitioning methods that ensures minimal partitioning cost. The expected performance gain resulting from the new management strategy is about 30% for a reasonable resource configuration. The performance gain decreases with increasing available buffer space. The different partitioning strategies (partial partitioning or hybrid hashing, one pass partitioning, and multipass partitioning) are analyzed, and we present the optimal working range for these, as a function of operand volume and available memory.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
References
K. Bratbergsengen. Hashing Methods and Relational Algebra Operations. In Proceedings of the 10th International Conference on VLDB, 1984.
K. Bratbergsengen, R. Larsen, O. Risnes, and T. Aandalen. A Neighbor Connected Processor Network for Performing Relational Algebra Operations. In Fifth Workshop on Computer Architecture for Non-Numeric Processing, March 11–14, 1980 (SIGIR Vol. XV No. 2, SIGMOD Vol.X No. 4), 1980.
D. L. Davidson and G. Graefe. Memory-Contention Responsive Hash Joins. In Proceedings of the 20th International Conference on VLDB, 1994.
D. DeWitt, R. Katz, F. Olken, L. Shapiro, M. Stonebraker, and D. Wood. Implementation Techniques for Main Memory Database Systems. In Proc. ACM SIGMOD Conf., 1984.
G. Graefe. Query Evaluation Techniques for Large Databases. ACM Computing Surveys, 25(2), 1993.
G. Graefe. Volcano — An Extensible and Parallel Query Evaluation System. IEEE Transactions on Knowledge and Data Engineering, 6(1), 1994.
M. Kitsuregawa, H. Tanaka, and T. Motooka. Application of Hash to Data Base Machine and its Architecture. New Generation Computing, 1(1), 1983.
D. Knuth. The Art of Computer Programming. Sorting and Searching. Addison-Wesley Publishing Company Inc., 1973.
M. Nakayama, M. Kitsuregawa, and M. Takagi. Hash-Partitioned Join Method Using Dynamic Destaging Strategy. In Proceedings of the 14th International Conference on VLDB, 1988.
L. D. Shapiro. Join Processing in Database Systems with Large Main Memories. ACM Transactions on Database Systems, 11(3), 1986.
H. Zeller and J. Gray. An Adaptive Hash Join Algorithm for Multiuser Environments. In Proceedings of the 16th International Conference on VLDB, 1990.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1997 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bratbergsengen, K., Nørvåg, K. (1997). Improved and optimized partitioning techniques in database query processing. In: Small, C., Douglas, P., Johnson, R., King, P., Martin, N. (eds) Advances in Databases. BNCOD 1997. Lecture Notes in Computer Science, vol 1271. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-63263-8_5
Download citation
DOI: https://doi.org/10.1007/3-540-63263-8_5
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-63263-4
Online ISBN: 978-3-540-69254-6
eBook Packages: Springer Book Archive