Abstract
Spatial join aggregate(SJA) is a commonly used but time-consuming operation in spatial database. Since it involves both the spatial join and the aggregate operation, performing SJA is a challenging task especially facing the deluge of spatial data. A popular model nowadays for massive data processing is the shared-nothing cluster using MapReduce. Thus, to explore SJA in MapReduce, a Map-Reduce-Filter-Merge(MRFM) algorithm is proposed.Map step divides the total SJA task into disjoint sets, then Reduce step aggregate each set individually, a Filter operation will filter those aggregate results of single assignment spatial objects.Finally, Merge step further aggregate the partial results of multiple assignment spatial objects using an efficient merge algorithm. Extensive experiments in large real spatial data have demonstrated the efficiency, effectiveness and scalability of the proposed methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Tao, Y., Papadias, D.: Range aggregate processing in spatial databases. IEEE Transactions on Knowledge and Data Engineering 16(12), 1555–1570 (2004)
Jurgens, M., Lenz, H.: The Ra*-Tree: An Improved R-Tree with Materialized Data for Supporting Range Queries on OLAP-Data. In: Proc. DEXA Workshop (1998)
Papadias, D., Kalnis, P., Zhang, J., Tao, Y.: Efficient OLAP Operations in Spatial Data Warehouses. In: Jensen, C.S., Schneider, M., Seeger, B., Tsotras, V.J. (eds.) SSTD 2001. LNCS, vol. 2121, p. 443. Springer, Heidelberg (2001)
Gray, J., Bosworth, A., Layman, A., Pirahesh, H.: Data Cube: A Relational Aggregation Operator Generalizing Group-By, Cross- Tabs and Subtotals. In: Proc. Intl. Conf. Data Eng. (1996)
Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)
Dittrich, J.P., Seeger, B.: Data redundancy and duplicate detection in spatial join processing. In: ICDE, pp. 535–546 (2000)
UC Bureau, Census 2010 Tiger/Line data (2010)
Pavlo, A., Paulson, E., et al.: A comparison of approaches to large-scale data analysis. In: SIGMOD Conference, pp. 165–178 (2009)
Jiang, D., Ooi, B.C., Shie, L., Wu, S.: The Performance of MapReduce: An Indepth Study. In: VLDB 2010 (2010)
Yang, H., Dasdan, A., et al.: Map-Reduce-Merge: simplified relational data processing on large clusters. In: SIGMOD Conference, pp. 1029–1040 (2007)
White, T.: Hadoop: The Definitive Guide. Yahoo! Press, Sebastopol (2009)
Wu, X., Carceroni, R., et al.: Automatic alignment of large-scale aerial rasters to road-maps, Geographic Information Systems. In: Proceedings of the 15th Annual ACM International Symposium on Advances in Geographic Information Systems, Article No. 17 (2007)
Cary, A., Sun, Z., Hristidis, V., Rishe, N.: Experiences on Processing Spatial Data with MapReduce. In: Winslett, M. (ed.) SSDBM 2009. LNCS, vol. 5566, pp. 302–319. Springer, Heidelberg (2009)
Zhang, S., Han, J., Lin, Z., et al.: SJMR: Parallelizing Spatial Join with MapReduce on Clusters. In: SSDBM Conference (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Liu, Y., Chen, L., Jing, N., Xiong, W. (2012). MRFM: An Efficient Approach to Spatial Join Aggregate. In: Bao, Z., et al. Web-Age Information Management. WAIM 2012. Lecture Notes in Computer Science, vol 7419. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33050-6_15
Download citation
DOI: https://doi.org/10.1007/978-3-642-33050-6_15
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33049-0
Online ISBN: 978-3-642-33050-6
eBook Packages: Computer ScienceComputer Science (R0)