Abstract
Benchmarking is a common practice for the evaluation of database computer systems. By executing certain benchmarks, manufacturers and researchers are able to highlight the characteristics of a certain system and are also able to rank the system against the rest. On the other hand, at the moment, BigData is a hot topic. It concerns dealing efficiently with information that is challenging to handle, due to volume, velocity or variety. As more and more platforms are proposed to deal with BigData, it becomes important to have benchmarks that can be used to evaluate performance characteristics of such platforms. At the same time, Decision Support applications are related to BigData, as they need to efficiently deal with huge datasets. In this paper we describe benchmarks representing Decision Support Systems (TPC-H, SSB, TPC-DS), and benchmarks for the Big Data class (YCSB, BigBench, and BigFrame), in order to help users to choose the most appropriate one for their needs. We also characterize the relationship between Big Data benchmarks and Decision Support benchmarks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
5 Steps To Benchmarking Managed NoSQL - DynamoDB Vs Cassandra, http://highscalability.com/blog/2013/4/3/5-steps-to-benchmarking-managed-nosql-dynamodb-vs-cassandra.html (accessed November 2, 2013)
Bog, A.: Benchmarking Transaction and Analytical Processing Systems. In: The Creation of a Mixed Workload Benchmark and its Application. Springer (2009) ISBN: 978-3-642-38069-3
Ghazal, A., Rabl, T., Hu, M., Raab, F., Poess, M., Croletteand, A., Jacobsen, H.A.: BigBench: Towards an Industry Standard Benchmark for Big Data Analytics. In: SIGMOD Conference, pp. 1197–1208 (2013)
Thanopoulou, A., Carreira, P., Galhardas, H.: Benchmarking with TPC-H on Off-the-Shelf Hardware. In: ICEIS (1), pp. 205–208 (2012)
An overview of the TPC-H benchmark on HP ProLiant servers and server blades, ftp://ftp.hp.com/pub/c-products/servers/benchmarks/HP_ProLiant_tpcc_Overview.pdf (accessed October 14, 2013)
Cooper, B.F., Silberstein, A., Tam, E., Ramakrishnan, R., Sears, R.: Benchmarking Cloud Serving Systems with YCSB. In: SoCC, pp. 143–154 6 (2010)
Big Data Insights, http://www.sas.com/big-data/ (accessed November 1, 2013)
Big Data, http://wikibon.org/wiki/v/Enterprise_Big-data (accessed December 10, 2013)
BigFrame User Guide, https://github.com/bigframeteam/BigFrame/wiki/BigFrame-User-Guide (accessed November 1, 2013)
Decision Support, http://www.slideshare.net/sursayantan92/decision-support-systemdss (accessed December 10, 2013)
Li, F., Ooi, B.C., Ozsu, M.T., Wu, S.: Distributed Data Management Using MapReduce. ACM Comput. Surv. 46(3), 31 (2014)
Poess, M., Nambiar, R.O., Walrath, D.: Why You Should Run TPC-DS:A Workload Analysis. In: VLDB, pp. 1138–1149 (2007)
O’Neil, P., O’Neil, E., Chen, X.: The star schema benchmark (SSB). University of Massachusetts, Boston, Technical Report (2007)
TPC-DS, http://www.tpc.org/information/sessions/tpc_ds.pdf (accessed October 12, 2013)
TPC-DS, http://www.tpc.org/tpcds/default.asp (accessed October 12, 2013), 4
TPC-H Benchmark, http://cs338-s13.cs.uwaterloo.ca/ (accessed October 18, 2013)
YCSB, https://github.com/brianfrankcooper/YCSB/wiki/core-workloads (accessed November 2, 2013)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Barata, M., Bernardino, J., Furtado, P. (2014). Survey on Big Data and Decision Support Benchmarks. In: Decker, H., Lhotská, L., Link, S., Spies, M., Wagner, R.R. (eds) Database and Expert Systems Applications. DEXA 2014. Lecture Notes in Computer Science, vol 8645. Springer, Cham. https://doi.org/10.1007/978-3-319-10085-2_15
Download citation
DOI: https://doi.org/10.1007/978-3-319-10085-2_15
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-10084-5
Online ISBN: 978-3-319-10085-2
eBook Packages: Computer ScienceComputer Science (R0)