Abstract
Currently, the size of data becomes much larger and the distributed data processing is getting very important to manage the huge size of data. The MapReduce well known as Google’s data processing environment is the most popular distributed platform with good scalability and fault tolerance. Many traditional algorithms in the single machine environment are being adopted to the MapReduce platform. In this paper we analyze a novel algorithm to generate wavelet synopses on the distributed MapReduce framework. Wavelet synopsis is one of the most popular dimensionality reduction methods and has been studied in various areas such as query optimization, approximate query answering, feature selection, etc. In the proposed algorithm, the wavelet synopsis can be calculated by a single MapReduce phase, and, by minimizing the amount of data communicated through the network of the distributed MapReduce platform, all computations are processed within almost linear time complexity. We theoretically study the properties of constructing wavelet synopsis on partitioned data sets and the correctness of the proposed algorithm.
Similar content being viewed by others
References
Cao P, Wang Z (2004) “Efficient top-k query calculation in distributed networks.” In Proceeding of PODC Conference
Chakrabarti K, Garofalakis MN, Rastogi R, Shim K (2000) “Approximate query processing using wavelets.” In Proceeding of VLDB Conference
Dean J, Ghemawat S (2004) “Mapreduce: simplified data processing on large clusters.” In Proceeding of OSDI Conference
Fallahpour M, Megías D (2011) High capacity audio watermarking using the high frequency band of the wavelet domain. Multimed Tools Appl 52(2–3):485–498
Gilbert AC, Kotidis Y, Muthukrishnan S, Strauss M (2001) “Surfing wavelets on streams: one-pass summaries for approximate aggregate queries.” In Proceeding of VLDB Conference
Guha S, Harb B (2006) “Approximation algorithms for wavelet transform coding of data streams.” In Proceeding of SODA Conference
Guha S, Kim C, Shim K (2004) “XWAVE: optimal and approximate extended wavelets for streaming data.” In Proceeding of VLDB Conference
Haar A (1910) Zur Theorie der orthogonalen Funktionensysteme. Math Ann 69:331–371
Jestes J, Yi K, Li F (2011) Building wavelet histograms on large data in MapReduce. PVLDB 5(2):109–120
Jiang J, Guo BF, Ipson S (2006) Shape-based image retrieval for JPEG-2000 compressed image databases. Multimed Tools Appl 29(2):93–108
Karras P, Mamoulis N (2007) “The haar+ tree: a refined synopsis data structure.” In Proceeding of ICDE Conference
Kim C (2012) An algorithm to generate wavelet synopsis in the MapReduce framework. J KIISE Database 39(6):346–353
Li Z, Liu G (2012) Video scene analysis in 3D wavelet transform domain. Multimed Tools Appl 56(3):419–437
Matias Y, Vitter JS, Wang M (1998) “Wavelet-based histograms for selectivity estimation.” In Proceeding of ACM SIGMOD Conference
Mattias Y, Vitter JS, Wang M (2000) “Dynamic maintenance of wavelet-based histograms.” In Proceeding of VLDB Conference
Natsev A, Rastogi R, Shim K (1999) “WALRUS: a similarity retrieval algorithm for image databases.” In Proceeding of ACM SIGMOD Conference
Omidyeganeh M, Ghaemmaghami S, Shirmohammadi S (2013) Application of 3D-wavelet statistics to video analysis. Multimed Tools Appl 65(3):441–465
Tsai M-J (2011) Dynamic energy enabled differentiation (DEED) image watermarking based on human visual system and wavelet tree classification. Multimed Tools Appl 52(2–3):385–406
Vitter JS, Wang M (1999) “Approximate computation of multidimensional aggregates of sparse data using wavelets.” In Proceeding of ACM SIGMOD Conference
Vitter JS, Wang M, Iyer BR (1998) “Data cube approximation and histograms via wavelets.” In Proceeding of CIKM Conference
Acknowledgements
This work was supported by the Gachon University research fund of 2011. (GCU-2011-R077).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Kim, C. Theoretical analysis of constructing wavelet synopsis on partitioned data sets. Multimed Tools Appl 74, 2417–2432 (2015). https://doi.org/10.1007/s11042-014-1908-3
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-014-1908-3