Abstract
For efficient query processing, a relational table should be indexed in multiple ways; for efficient database loading, indexes should be omitted. This research introduces new techniques called zones filters, zone indexes, adaptive merging, and partition filters. The new data structures can be created as side effects of the load process, with all required analyses accomplished while a moderate amount of new data still remains in the buffer pool. Traditional sorting and indexing are not required. Nonetheless, query performance matches that of Netezza’s zone maps where those apply, exceeds it for the many predicates for which zone maps are ineffective, and can be comparable to query processing with traditional indexing, as demonstrated in our simulations.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Ailamaki, A., De Witt, D.J., Hill, M.D.: Marios Sk- ounakis: Weaving relations for cache performance. In: VLDB 2001, pp. 169–180 (2001)
Bloom, B.H.: Space time trade-offs in hash coding with allowable errors. ACM Commun. 13(7), 422–426 (1970)
Boral, H., DeWitt, D.J.: Database machines: An idea whose time passed? In: A critique of the future of database machines. In: IWDM 1983, pp. 166–187 (1983)
Bayer, R., Unterauer, K.: Prefix B-trees. ACM TODS 2(1), 11–26 (1977)
Cao, Y., Das, G.C., Chan, C.Y., Tan, K.-L.: Optimizing complex queries with multiple relation instances. In: SIGMOD 2008, pp. 525–538 (2008)
Fernandez, P.M.: Red Brick Warehouse: A read-mostly RDBMS for Open SMP platforms. In: SIGMOD 1994, p. 492 (1994)
Graefe, G.: Sorting and indexing with partitioned B-trees. In: CIDR 2003 (2003)
Graefe, G.: Hierarchical locking in B-tree indexes. In: BTW 2007, pp. 18–42 (2007)
Graefe, G.: The five-minute rule twenty years later, and how flash memory changes the rules. In: DaMoN 2007, p. 6 (2007)
Graefe, G.: Integrating PAX and NSM page formats. Hewlett-Packard Laboratories (2008) (unpublished manuscript)
Graefe, G., Kuno, H.: Self-selecting, self-tuning, incrementally optimized indexes. To appear in EDBT (2010)
Graefe, G., Kuno, H.: Two adaptive indexing techniques: improvements and performance evaluation (submitted)
Graefe, G., Kuno, H.: Adaptive indexing for relational keys. To appear in SMDB (2010)
Grtner, A., Kemper, A., Kossmann, D., Zeller, B.: Efficient bulk deletes in relational databases. In: ICDE 2001, pp. 183–192 (2001)
Graefe, G., Larson, P.-k.: B-Tree indexes and CPU caches. In: ICDE 2001, pp. 349–358 (2001)
Idreos, S., Kersten, M.L., Manegold, S.: Database cracking. In: CIDR 2007, pp. 68–78 (2007)
Idreos, S., Kersten, M.L., Manegold, S.: Updating a cracked database. In: SIGMOD 2007, pp. 413–424 (2007)
Idreos, S., Kersten, M., Manegold, S.: Self-organizing tu-ple reconstruction in column stores. In: SIGMOD 2009, pp. 297–308 (2009)
Idreos, S., Manegold, S., Graefe, G., Kuno, H.: Adaptive indexing. Submitted for publication (2010)
Jermaine, C., Datta, A., Omiecinski, E.: A novel index supporting high volume data warehouse insertion. In: VLDB 1999, pp. 235–246 (1999)
Kersten, M.L., Manegold, S.: Cracking the database store. In: CIDR 2005 (2005)
Lomet, D.B.: The evolution of effective B-tree page organization and techniques: a personal account. SIGMOD Record 30(3), 64–69 (2001)
Lang, C.A., Bhattacharjee, B., Malkemus, T., Wong, K.: Increasing buffer-locality for multiple index based scans through intelligent placement and index scan speed control. In: VLDB 2007, pp. 1298–1309 (2007)
Lehman, T.J., Carey, M.J.: A study of index structures for main memory database management systems. In: VLDB 1986, pp. 294–303 (1986)
Leslie, H., Jain, R., Birdsall, D., Yaghmai, H.: Efficient search of multi-dimensional B-trees. In: VLDB 1995, pp. 710–719 (1995)
Monash, C.: Kognito WX2 overview (January 2008), http://www.dbms2.com/2008/01/26/kognitio-wx2
Moerkotte, G.: Small materialized aggregates: A light weight index structure for data warehousing. In: VLDB 1998, pp. 476–487 (1998)
Merrett, T.H., Kambayashi, Y., Yasuura, H.: Scheduling of Page- Fetches in Join Operations. In: VLDB 1981, pp.488–498 (1981)
Muth, P., O’Neil, P., Pick, A., Weikum, G.: The LHAM log-structured history data access method. VLDB J. 8(3-4), 199–221 (2000)
Murphy, M.C., Rotem, D.: Multiprocessor Join Scheduling. IEEE TKDE 5(2), 322–338 (1993)
Nyberg, C., Barclay, T., Cvetanovic, Z., Gray, J., Lomet, D.B.: AlphaSort: A Cache-Sensitive Parallel External Sort. VLDB J. 4(4), 603–627 (1995)
Rao, J., Ross, K.A.: Making B+-trees cache conscious in main memory. In: SIGMOD 2000, pp. 475–486 (2000)
Slezak, D., Wroblewski, J., Eastwood, V., Synak, P.: Brighthouse: an analytic data warehouse for ad-hoc queries. PVLDB 1(2), 1337–1345 (2008)
Valduriez, P.: Join Indices. ACM TODS 12(2), 218–246 (1987)
Zukowski, M., Hman, S., Nes, N., Boncz, P.A.: Cooperative scans: Dynamic bandwidth sharing in a DBMS. In: VLDB 2007, pp. 723–734 (2007)
Zhou, J., Larson, P.-k., Freytag, J.C.: Wolfgang Lehner: Efficient exploitation of similar subexpressions for query processing. In: SIGMOD 2007, pp. 533–544 (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Graefe, G., Kuno, H. (2010). Fast Loads and Queries. In: Hameurlain, A., Küng, J., Wagner, R., Bach Pedersen, T., Tjoa, A.M. (eds) Transactions on Large-Scale Data- and Knowledge-Centered Systems II. Lecture Notes in Computer Science, vol 6380. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-16175-9_2
Download citation
DOI: https://doi.org/10.1007/978-3-642-16175-9_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-16174-2
Online ISBN: 978-3-642-16175-9
eBook Packages: Computer ScienceComputer Science (R0)