Skip to main content
SpringerLink
Log in

Scalable management of storage for massive quality-adjustable sensor data

  • Published:
Computing Aims and scope Submit manuscript

Abstract

The quality-adjustable nature of sensor data and gradually decreasing access pattern foster a new data archiving scheme that can cope with rapidly increasing data generation rates by sensors. We propose a scalable quality management of massive sensor data, which handles less frequent data access through discarding supplementary layers as time elapses for efficient usage of storage space. The efficacy of our scheme is shown by its capability to offer multiple fidelity levels compactly utilizing spatio-temporal correlation, without compromising key features of sensor data. In order to store a huge amount of data from various sensor types efficiently, we also study the optimal storage configuration strategy using analytical models that can capture characteristics of our scheme. This strategy helps storing sensor data blocks while minimizing total distortion under a given total rate budget.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Satyanarayanan M (2010) Mobile computing: the next decade. Paper presented at the 1st ACM workshop on mobile cloud computing and services: social networks and beyond, San Francisco

  2. Campbell AT, Eisenman SB, Lane ND, Miluzzo E, Peterson RA (2006) People-centric urban sensing. Paper presented at the 2nd annual international workshop on wireless internet, Boston

  3. Roddick JF, Hoel E, Egenhofer MJ, Papadias D, Salzberg B (2004) Spatial, temporal and spatio-temporal databases—hot issues and directions for PhD research. SIGMOD Rec 33(2):126–131. doi:10.1145/1024694.1024724

    Article  Google Scholar 

  4. Kitsuregawa M, Nishida T (2010) Special issue on information explosion. New Gener Comput 28(3):207–215. doi:10.1007/s00354-010-0086-8

    Article  MATH  Google Scholar 

  5. Sweeney L (2001) Information explosion. In: Zayatz L, Doyle P, Theeuwes J, Lane J (eds) Confidentiality, disclosure, and data access: theory and practical applications for statistical agencies. Urban Institute, Washington

  6. Gantz JF, Chute C (2008) The diverse and exploding digital universe: an updated forecast of worldwide information growth through 2011. 2008 IDC

  7. Hilbert M, López P (2011) The world’s technological capacity to store, communicate, and compute information. Science 332(6025):60–65. doi:10.1126/science.1200970

    Article  Google Scholar 

  8. Sampson A, Nelson J, Strauss K, Ceze L (2013) Approximate storage in solid-state memories. Paper presented at the 46th IEEE/ACM international symposium on microarchitecture (MICRO), Davis

  9. Esmaeilzadeh H, Sampson A, Ceze L, Burger D (2012) Architecture support for disciplined approximate programming. Paper presented at the 17th international conference on architectural support for programming languages and operating systems (ASPLOS), London

  10. Ganesan D, Estrin D, Heidemann J (2003) Dimensions: why do we need a new data handling architecture for sensor networks? SIGCOMM Comput Commun Rev 33(1):143–148. doi:10.1145/774763.774786

    Article  MATH  Google Scholar 

  11. Ganesan D, Greenstein B, Estrin D, Heidemann J, Govindan R (2005) Multiresolution storage and search in sensor networks. ACM Trans Storage 1(3):277–315. doi:10.1145/1084779.1084780

    Article  Google Scholar 

  12. Srisooksai T, Keamarungsi K, Lamsrichan P, Araki K (2012) Practical data compression in wireless sensor networks: a survey. J Netw Comput Appl 35(1):37–59. doi:10.1016/j.jnca.2011.03.001

    Article  Google Scholar 

  13. Vuran MC, Akan ÖB, Akyildiz IF (2004) Spatio-temporal correlation: theory and applications for wireless sensor networks. Comput Netw 45(3):245–259. doi:10.1016/j.comnet.2004.03.007

    Article  MATH  Google Scholar 

  14. Smith H (2011) Data center storage: cost-effective strategies, implementation, and management. CRC Press, Boca Raton

    Book  Google Scholar 

  15. Sensirion Home Page. http://www.sensirion.com/en/home/. Accessed 12 Jan 2015

  16. Cao H, Wolfson O, Trajcevski G (2006) Spatio-temporal data reduction with deterministic error bounds. VLDB J 15(3):211–228

    Article  Google Scholar 

  17. Palmer M (2005) Seven principles of effective RFID data management. Progress Software

  18. Cohen E, Kaplan H (2001) Aging through cascaded caches: performance issues in the distribution of web content. SIGCOMM Comput Commun Rev 31(4):41–53

    Article  MATH  Google Scholar 

  19. Rimoldi B (1994) Successive refinement of information: characterization of the achievable rates. IEEE Trans Inf Theory 40(1):253–259. doi:10.1109/18.272493

    Article  MathSciNet  Google Scholar 

  20. Equitz WHR, Cover TM (1991) Successive refinement of information. IEEE Trans Inf Theory 37(2):269–275. doi:10.1109/18.75242

    Article  MathSciNet  Google Scholar 

  21. Schwarz H, Marpe D, Wiegand T (2007) Overview of the scalable video coding extension of the H.264/AVC standard. IEEE Trans Circuits Syst Video Technol 17(9):1103–1120. doi:10.1109/TCSVT.2007.905532

    Article  Google Scholar 

  22. Vidyo Home Page. http://www.vidyo.com. Accessed 12 Jan 2015

  23. Sayood K (2005) Introduction to data compression, 3rd edn. Morgan Kaufmann, San Francisco

    Google Scholar 

  24. Xiong Z, Ramchandran K, Orchard MT, Zhang Y-Q (1999) A comparative study of DCT- and wavelet-based image coding. IEEE Trans Circuits Syst Video Technol 9(5):692–695. doi:10.1109/76.780358

    Article  Google Scholar 

  25. Iverson V, McVeigh J, Reese B (2004) Real-time H.264/AVC codec on Intel architectures. Paper presented at the 11th IEEE international conference on image processing (ICIP), Singapore

  26. Lappalainen V, Hallapuro A, Hämäläinen TD (2003) Performance of H. 26L video encoder on general-purpose processor. J VLSI Signal Process Syst Signal Image Video Technol 34(3):239–249

    Article  Google Scholar 

  27. Diao Y, Ganesan D, Mathur G, Shenoy PJ (2007) Rethinking data management for storage-centric sensor networks. Paper presented at the 3rd biennial conference on innovative data systems research (CIDR), Pacific Grove

  28. SensorScope: sensor networks for environmental monitoring. http://lcav.epfl.ch/op/edit/sensorscope-en. Accessed 12 Jan 2015

  29. RD models derivation Home Page. http://pail.unist.ac.kr/papers/DELEE_COMPUTING_RD.pdf. Accessed 12 Jan 2015

  30. Li D, Sun X (2006) Nonlinear integer programming. Springer, New York

    MATH  Google Scholar 

  31. Shoham Y, Gersho A (1988) Efficient bit allocation for an arbitrary set of quantizers. IEEE Trans Acoust Speech Signal Process 36(9):1445–1453. doi:10.1109/29.90373

    Article  MATH  Google Scholar 

  32. Everett H (1963) Generalized Lagrange multiplier method for solving problems of optimum allocation of resources. Oper Res 11(3):399–417

    Article  MathSciNet  Google Scholar 

  33. Forney GD Jr (1973) The viterbi algorithm. Proc IEEE 61(3):268–278. doi:10.1109/PROC.1973.9030

    Article  MathSciNet  Google Scholar 

  34. Wang Y-C, Hsieh Y-Y, Tseng Y-C (2009) Multiresolution spatial and temporal coding in a wireless sensor network for long-term monitoring applications. IEEE Trans Comput 58(6):827–838

    Article  MathSciNet  Google Scholar 

  35. Palpanas T, Vlachos M, Keogh E, Gunopulos D, Truppel W (2004) Online amnesic approximation of streaming time series. Paper presented at the 20th international conference on data engineering (ICDE), Boston

  36. Schneidewind NF (1993) Software reliability model with optimal selection of failure data. IEEE Trans Softw Eng 19(11):1095–1104

    Article  Google Scholar 

  37. Chakrabarti K, Garofalakis M, Rastogi R, Shim K (2001) Approximate query processing using wavelets. VLDB J 10(2–3):199–223. doi:10.1007/s007780100049

    Google Scholar 

  38. Cesari M, Favalli L, Folli M (2009) Quality modeling for the medium grain scalability option of H.264/SVC. Paper presented at the 5th international ICST mobile multimedia communications conference (MOBIMEDIA), London

  39. Mansour H, Krishnamurthy V, Nasiopoulos P (2008) Rate and distortion modeling of medium grain scalable video coding. Paper presented at the 15th IEEE international conference on image processing (ICIP), San Diego

  40. Dai M, Loguinov D, Radha HM (2006) Rate-distortion analysis and quality control in scalable internet streaming. IEEE Trans Multimed 8(6):1135–1146. doi:10.1109/TMM.2006.884626

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MEST) (No. 2012R1A1A2044653, No. 2011-0000228). Dongeun Lee wishes to express his gratitude to UNIST (Ulsan National Institute of Science and Technology) for support, where he is currently a postdoctoral research associate.

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Seoul National University, Seoul, 151-744, Korea

    Dongeun Lee, Junhee Ryu & Heonshik Shin

Authors
  1. Dongeun Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Junhee Ryu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Heonshik Shin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dongeun Lee.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lee, D., Ryu, J. & Shin, H. Scalable management of storage for massive quality-adjustable sensor data. Computing 97, 769–793 (2015). https://doi.org/10.1007/s00607-015-0465-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00607-015-0465-6

Keywords

Mathematics Subject Classification

Search

Navigation