Abstract
Data intensive applications such as network monitoring, financial applications; sensor-based applications etc. need to be supported by general-purpose systems rather than customized implementations. They have a continuous, unpredictable and unbounded flow of data as input, referred as streams. The fact that data comes as a stream with varying input rates (instead of accessing data stored on a disk in a predictable way) and that quality of service (QoS) requirements are stringent for these applications warrants a re-examination of the fundamental architecture of a DBMS. This paper describes the basic processing model and architecture of MavStream – a new Data Stream Management System (DSMS) being developed at UT Arlington. The architecture of MavStream is the primary focus of this paper. The user can give a continuous query from a graphical user interface (GUI), which is instantiated, scheduled, and executed by the MavStream server. We first provide an overview of the basic model and architecture and then describe some of the components of the system. We provide some experimental results to demonstrate the utility of the system and the effect of different scheduling strategies and buffer sizes on the performance and output.
This work was supported, in part, by the following NSF grants: IIS-0326505, EIA-0216500, MRI 0421282, and IIS 0534611.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Carney, D., Çetintemel, U., Cherniack, M., Convey, C., Lee, S., Seidman, G., Stonebraker, M., Tatbul, N., Zdonik, S.: Monitoring Streams- A new class of data management applications. In: Proc. of the 28th Intl. Conf. on VLDB (2002)
Babcock, B., Babu, S., Datar, M., Motwani, R., Widom, J.: Models and Issues in Data Stream Systems. In: Proc. of PODS (2002) (Invited paper)
Chandrasekharan, S., Franklin, M.J.: Streaming Queries over Streaming Data. In: Proc. of the 28th Intl. Conf. VLDB, pp. 203–214 (2002)
Chandrasekharan, S., Cooper, O., Deshpande, A., Franklin, M.J., Hellerstein, J.M., Hong, W., Krishnamurthy, S., Madden, S., Raman, V., Reiss, F., Shah, M.A.: TelegraphCQ: Continuous Dataflow Processing for an Uncertain World. In: Proc. of CIDR (2003)
Avnur, R., Hellerstein, J.M.: Eddies: Continuously adaptive query processing. In: Proceedings of ACM SIGMOD, pp. 261–272 (2000)
Madden, S., Franklin, M.J.: Fjording the stream: An architecture for queries over streaming sensor data. In: Proc. of the Intl. Conf. on Data Engineering, pp. 555–566 (2002)
Bonnet, P., Gehrke, J., Seshadri, P.: Towards sensor database systems. In: Tan, K.-L., Franklin, M.J., Lui, J.C.-S. (eds.) MDM 2001. LNCS, vol. 1987, pp. 3–14. Springer, Heidelberg (2000)
Chen, J., DeWitt, D.J., Tian, F., Wang, Y.: NiagaraCQ: A scalable continuous query system for internet databases. In: Proc. of ACM SIGMOD (2000)
Abadi, D., Carney, D., Çetintemel, U., Cherniack, M., Convey, C., Lee, S., Stonebraker, M., Tatbul, N., Zdonik, S.: Aurora: A New Model and Architecture for Data Stream Management. VLDB Journal 2(12), 120–139 (2003)
Carney, D., Çetintemel, U., Rasin, A., Zdonik, S., Cherniack, M., Stonebraker, M.: Operator Scheduling in a Data Stream Manager. In: Proc. of VLDB (2003)
The STREAM Group. STREAM: The Stanford Stream Data Manager. IEEE Data Engineering Bulletin (March 2003)
Motwani, R., Widom, J., Arasu, A., Babcock, B., Babu, S., Datar, M., Manku, G., Olston, C., Rosenstein, J., Varma, R.: Query Processing Resource Management and Approximation in a Data Stream Management System. In: Proc. of CIDR (2003)
Gilani, A.: Design and implementation of stream operators, query instantiator and stream buffer manager. MS Thesis, CSE Department, UT Arlington (2003) [online], http://www.cse.uta.edu/research/publications/Downloads/CSE-2003-37.pdf
Sonune, S.: Design and implementation of windowed operators and scheduler for stream data. MS Thesis, CSE Department, University of Texas at Arlington (2003) [online], http://www.cse.uta.edu/research/publications/Downloads/CSE-2003-38.pdf
Cook, D.J., et al.: MavHome: An Agent-Based Smart Home. In: Proc. of the Conference on Pervasive Computing (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gilani, A., Sonune, S., Kendai, B., Chakravarthy, S. (2006). The Anatomy of a Stream Processing System. In: Bell, D.A., Hong, J. (eds) Flexible and Efficient Information Handling. BNCOD 2006. Lecture Notes in Computer Science, vol 4042. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11788911_20
Download citation
DOI: https://doi.org/10.1007/11788911_20
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-35969-2
Online ISBN: 978-3-540-35971-5
eBook Packages: Computer ScienceComputer Science (R0)