Abstract
Execution plans constitute the traditional interface between DBMS front-ends and back-ends; similar networks of interconnected operators are found also outside database systems. Tasks like adapting execution plans for distributed or heterogeneous runtime environments require a plan transformation mechanism which is simple enough to produce predictable results while general enough to express advanced communication schemes required for instance in skew-resistant partitioning. In this paper, we describe the BobolangNG language designed to express execution plans as well as their transformations, based on hierarchical models known from many environments but enhanced with a novel compile-time mechanism of component multiplication. Compared to approaches based on general graph rewriting, the plan transformation in BobolangNG is not iterative; therefore the consequences and limitations of the process are easier to understand and the development of distribution strategies and experimenting with distributed plans are easier and safer.
F. Zavoral—This paper was supported by the Czech Science Foundation (GAČR) project P103-14-14292P and by the Charles University Grant Agency (GAUK) project PRVOUK-P46.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
References
Bruno, N., Kwon, Y., Wu, M.C.: Advanced join strategies for large-scale distributed computation. Proc. VLDB Endow. 7(13), 1484–1495 (2014). http://dx.doi.org/10.14778/2733004.2733020
Falt, Z., Bednárek, D., Kruliš, M., Yaghob, J., Zavoral, F.: Bobolang: a language for parallel streaming applications. In: Proceedings of the 23rd International Symposium on High-Performance Parallel and Distributed Computing, pp. 311–314. ACM (2014)
Isard, M., Budiu, M., Yu, Y., Birrell, A., Fetterly, D.: Dryad: distributed data-parallel programs from sequential building blocks. In: ACM SIGOPS Operating Systems Review, vol. 41, pp. 59–72. ACM (2007)
Ke, Q., Isard, M., Yu, Y.: Optimus: a dynamic rewriting framework for data-parallel execution plans. In: Proceedings of the 8th ACM European Conference on Computer Systems, pp. 15–28. ACM (2013)
Neumeyer, L., Robbins, B., Nair, A., Kesari, A.: S4: distributed stream computing platform. In: Proceedings of the IEEE International Conference on Data Mining Workshops (ICDMW), pp. 170–177. IEEE (2010)
Levandoski, J.J., Eldawy, A., Mokbel, M.F., Khalefa, M.E.: Flexible and extensible preference evaluation in database systems. ACM Trans. Database Syst. (TODS) 38(3), 17 (2013)
Murray, D.G., McSherry, F., Isaacs, R., Isard, M., Barham, P., Abadi, M.: Naiad: a timely dataflow system. In: Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles, pp. 439–455. ACM (2013)
Murray, D.G., Schwarzkopf, M., Smowton, C., Smith, S., Madhavapeddy, A., Hand, S.: Ciel: a universal execution engine for distributed data-flow computing. In: Proceedings of 8th ACM/USENIX Symposium on Networked Systems Design and Implementation, pp. 113–126 (2011)
Murray, D.G., Hand, S.: Scripting the cloud with skywriting. HotCloud 10, 12 (2010)
Nagendra, M., Candan, K.S.: Efficient processing of skyline-join queries over multiple data sources. ACM Trans. Database Syst. (TODS) 40(2), 10 (2015)
Oscar, B., Sam, R., Ian, O., Jimmy, L.: Summingbird: a framework for integrating batch and online mapreduce computations. Proc. VLDB Endow. 7(13), 1441–1451 (2014)
Gulisano, V., Jimenez-Peris, R., Patino-Martinez, M., Valduriez, P.: Streamcloud: a large scale data streaming system. In: Proceedings of the 30th International Conference on Distributed Computing Systems (ICDCS), pp. 126–137. IEEE (2010)
Van Emden, M.H., Kowalski, R.A.: The semantics of predicate logic as a programming language. J. ACM 23(4), 733–742 (1976). http://doi.acm.org/10.1145/321978.321991
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Bednárek, D., Kruliš, M., Yaghob, J., Zavoral, F. (2016). Creating Distributed Execution Plans with BobolangNG. In: Carretero, J., Garcia-Blas, J., Ko, R., Mueller, P., Nakano, K. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2016. Lecture Notes in Computer Science(), vol 10048. Springer, Cham. https://doi.org/10.1007/978-3-319-49583-5_6
Download citation
DOI: https://doi.org/10.1007/978-3-319-49583-5_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-49582-8
Online ISBN: 978-3-319-49583-5
eBook Packages: Computer ScienceComputer Science (R0)