Abstract
We design an automatic test suite generation tool Artemis for functionality test of Online Analytical Processing Databases (OLAP DBs). This is the first work which accomplishes the work of DB test by integrating three artifacts, i.e., data generation, workload generation and oracle generation, but promises the scalability, effectiveness and efficiency. The key idea of our approach is to design a deterministic random data generation mechanism, based on which we can instantiate the parameterized queries and calculate the oracles simultaneously by resolving the constraint chains along query trees. Since we provide deterministic random functions for data generations corresponding to a predefined schema, repetitive test and data migration become a trivial job. Random workload generation and automatic oracle calculation instead of differential comparison make abundant and massive scale of test possible. We finally provide extensive experiments to show the performance of Artemis.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abdul Khalek, S., Khurshid, S.: Automated SQL query generation for systematic testing of database engines. In ASE 2010, pp. 329–332 (2010)
Alexandrov, A., Tzoumas, K., Markl, V.: Myriad: scalable and expressive data generation. In: VLDB 2012, vol. 5, issue 12, pp. 1890–1893 (2012)
Bati, H., Giakoumakis, L., Herbert, S., Surna, A.: A genetic approach for random testing of database systems. In: VLDB 2007, pp. 1243–1251 (2007)
Binnig, C., Kossmann, D., Lo, E.: Reverse query processing. In: ICDE 2007, pp. 506–515 (2007)
Binnig, C., Kossmann, D., Lo, E., Saenzbadillos, A.: Automatic result verification for the functional testing of a query language. In: ICDE 2008, pp. 1534–1536 (2008)
Binnig, C., Kossmann, D., Lo, E., Özsu, M.T.: Qagen generating query-aware test databases. In: SIGMOD 2007, pp. 341–352 (2007)
Hamlin, A., Herzog, J.: A test-suite generator for database systems. In: HPEC 2014, pp. 1–6 (2014)
He, S., Manns, G., Saunders, J., Wang, W., Pollock, L., Soffa, M.L.: A statistics-based performance testing methodology for cloud applications. In: ESEC/FSE 2019 (2019)
Hoag, J.E., Thompson, C.W.: A parallel general-purpose synthetic data generator. SIGMOD Rec 36(1), 19–24 (2007)
Jackson, D.: Alloy: a lightweight object modelling notation. ACM Trans. Softw. Eng. Methodol. 11(2), 256–290 (2002)
Khalek, S.A., Elkarablieh, B., Laleye, Y.O., Khurshid, S.: Query-aware test generation using a relational constraint solver. In: ASE 2008, pp. 238–247 (2008)
Khalek, S.A., Khurshid, S.: Systematic testing of database engines using a relational constraint solver. In: ICST 2011, pp. 50–59 (2011)
Li, Y., Zhang, R., Yang, X., Zhang, Z., Zhou, A.: Touchstone: generating enormous query-aware test databases. In: ATC 2018, pp. 575–586 (2018)
Lo, E., Cheng, N., Lin, W.W.K., Hon, W.K., Choi, B.: Mybenchmark: generating databases for query workloads. VLDB J. 23(6), 895–913 (2014)
Mishra, C., Koudas, N., Zuzarte, C.: Generating targeted queries for database testing. In: SIGMOD 2008, pp. 499–510 (2008)
Poess, M., Stephens, J.M.: Generating thousand benchmark queries in seconds. in: VLDB 2004, pp. 1045–1053 (2004)
Rigger, M., Su, Z.: Testing database engines via pivoted query synthesis. arXiv: Databases (2020)
Slutz, D.R.: Massive stochastic testing of SQL. In: VLDB 1998, pp. 618–622 (1998)
Torlak, E.: Scalable test data generation from multidimensional models. In: SIGSOFT FSE 2012, p. 36 (2012)
Zhou, J., Aghili, N., Ghaleini, E.N., Bui, D.T., Tahir, M.M., Koopialipoor, M.: A Monte Carlo simulation approach for effective assessment of flyrock based on intelligent system of neural network. Eng. Comput. 36(2), 1–11 (2020). https://doi.org/10.1007/s00366-019-00726-z
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Mi, K., Zhang, C., Qian, W., Zhang, R. (2021). Artemis: An Automatic Test Suite Generator for Large Scale OLAP Database. In: Wolf, F., Gao, W. (eds) Benchmarking, Measuring, and Optimizing. Bench 2020. Lecture Notes in Computer Science(), vol 12614. Springer, Cham. https://doi.org/10.1007/978-3-030-71058-3_5
Download citation
DOI: https://doi.org/10.1007/978-3-030-71058-3_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-71057-6
Online ISBN: 978-3-030-71058-3
eBook Packages: Computer ScienceComputer Science (R0)