Abstract
An IoT cloud environment consists of connected physical devices communicating with the cloud, sending telemetry data and accepting actuation information. For sensors, the data flow is from the physical devices to the cloud. The IoT edge device is responsible for collecting this data and forwarding it to the cloud environment for processing. The time it takes for the data to be made available for processing in the cloud is critical, and the network connectivity, bandwidth and latency are the bottlenecks. In this work, we created a flow controller which adaptively controls the flow of the data from the edge device to the cloud. While rate limiting is a trivial technique to control data flow, it is crucial how the edge devices dynamically control the data rate by re-configuring the IoT devices to send data based on the current network condition and load on the Edge device. We tested this system with a simulated data flow from 10 sensors to a Raspberry Pi Device which performed the rate limiting.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abadi, D.J., et al.: Aurora: a new model and architecture for data stream management. VLDB J. 12(2), 120–139 (2003)
Agarwal, S., Mozafari, B., Panda, A., Milner, H., Madden, S., Stoica, I.: BlinkDB: Queries with bounded errors and bounded response times on very large data. In: Proceedings of the 8th ACM European Conference on Computer Systems, pp. 29–42. ACM (2013)
Akidau, T., et al.: Millwheel: fault-tolerant stream processing at internet scale. Proc. VLDB Endowment 6(11), 1033–1044 (2013)
Akidau, T., et al.: The dataflow model: a practical approach to balancing correctness, latency, and cost in massive-scale, unbounded, out-of-order data processing. Proc. VLDB Endowment 8(12), 1792–1803 (2015)
Bahreini, T., Grosu, D.: Efficient placement of multi-component applications in edge computing systems. In: Proceedings of the Second ACM/IEEE Symposium on Edge Computing, p. 5. ACM (2017)
Beck, M., Bhatotia, P., Chen, R., Fetzer, C., Strufe, T., et al.: Privapprox: privacy-preserving stream analytics. In: 2017 \(\{\)USENIX\(\}\) Annual Technical Conference (\(\{\)USENIX\(\}\{\)ATC\(\}\) 17). pp. 659–672 (2017)
Boeing: Boeing 787s to create half a terabyte of data per flight, says virgin atlantic. https://www.computerworlduk.com/data/boeing-787s-create-half-terabyte-of-data-per-flight-says-virgin-atlantic-3433595/. Accessed 08 Apr 2019
Bonomi, F., Milito, R., Zhu, J., Addepalli, S.: Fog computing and its role in the internet of things. In: Proceedings of the First Edition of the MCC Workshop on Mobile Cloud Computing, MCC 2012, pp. 13–16. ACM, New York (2012). https://doi.org/10.1145/2342509.2342513
Carbone, P., Katsifodimos, A., Ewen, S., Markl, V., Haridi, S., Tzoumas, K.: Apache Flink: stream and batch processing in a single engine. Bull. IEEE Comput. Soc. Tech. Committee Data Eng. 36(4), 28–38 (2015)
Chandrasekaran, S., et al.: TelegraphCQ: continuous dataflow processing for an uncertain world. In: CIDR, vol. 2, p. 4 (2003)
Cheng, B., Longo, S., Cirillo, F., Bauer, M., Kovacs, E.: Building a big data platform for smart cities: experience and lessons from Santander. In: IEEE International Congress on Big Data, pp. 592–599. IEEE (2015)
Chippa, V.K., Chakradhar, S.T., Roy, K., Raghunathan, A.: Analysis and characterization of inherent application resilience for approximate computing. In: Proceedings of the 50th Annual Design Automation Conference, p. 113. ACM (2013)
Goiri, I., Bianchini, R., Nagarakatte, S., Nguyen, T.D.: ApproxHadoop: bringing approximations to mapreduce frameworks. In: ACM SIGARCH Computer Architecture News, vol. 43, pp. 383–397. ACM (2015)
Hunkeler, U., Truong, H.L., Stanford-Clark, A.: MQTT-S - A publish/subscribe protocol for wireless sensor networks. In: 3rd International Conference on Communication Systems Software and Middleware and Workshops (COMSWARE 2008), pp. 791–798. IEEE (2008)
James, P.M., Dawson, R.J., Harris, N., Joncyzk, J.: Urban observatory environment. Newcastle University, pp. 154300–154319 (2014)
Kandula, S., et al.: Quickr: lazily approximating complex adhoc queries in bigdata clusters. In: Proceedings of the 2016 International Conference on Management of Data, pp. 631–646. ACM (2016)
Light, R.A., et al.: Mosquitto: server and client implementation of the MQTT protocol. J. Open Source Softw. 2(13), 265 (2017)
Lin, W., Qian, Z., Xu, J., Yang, S., Zhou, J., Zhou, L.: Streamscope: continuous reliable distributed processing of big data streams. In: 13th \(\{\)USENIX\(\}\) Symposium on Networked Systems Design and Implementation (\(\{\)NSDI\(\}\) 2016), pp. 439–453 (2016)
Lukić, M., Mihajlović,, Mezei, I.: Data flow in low-power wide-area IoT applications. In: 2018 26th Telecommunications Forum (TELFOR). pp. 1–4 (Nov 2018). https://doi.org/10.1109/TELFOR.2018.8611848
Meyerson, J.: The go programming language. IEEE Softw. 31(5), 104–104 (2014). https://doi.org/10.1109/MS.2014.127
Mohammadi, M., Al-Fuqaha, A., Sorour, S., Guizani, M.: Deep learning for IoT big data and streaming analytics: a survey. IEEE Commun. Surv. Tutorials 20(4), 2923–2960 (2018)
Morabito, R., Beijar, N.: Enabling data processing at the network edge through lightweight virtualization technologies. In: IEEE International Conference on Sensing, Communication and Networking (SECON Workshops), pp. 1–6, June 2016. https://doi.org/10.1109/SECONW.2016.7746807
Murray, D.G., McSherry, F., Isaacs, R., Isard, M., Barham, P., Abadi, M.: Naiad: a timely dataflow system. In: Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles, pp. 439–455. ACM (2013)
Networking, C.V.: Cisco global cloud index: Forecast and methodology 2015–2020. White paper (2016)
O’Keeffe, D., Salonidis, T., Pietzuch, P.: Frontier: resilient edge processing for the internet of things. Proc. VLDB Endowment 11(10), 1178–1191 (2018)
Quoc, D.L., Chen, R., Bhatotia, P., Fetze, C., Hilt, V., Strufe, T.: Approximate stream analytics in Apache Flink and Apache Spark streaming. arXiv preprint arXiv:1709.02946 (2017)
Ranjan, R., et al.: The next grand challenges: integrating the internet of things and data science. IEEE Cloud Comput. 5(3), 12–26 (2018)
Sajjad, H.P., Danniswara, K., Al-Shishtawy, A., Vlassov, V.: Spanedge: towards unifying stream processing over central and near-the-edge data centers. In: 2016 IEEE/ACM Symposium on Edge Computing (SEC), pp. 168–178. IEEE (2016)
Szydlo, T., Brzoza-Woch, R., Sendorek, J., Windak, M., Gniady, C.: Flow-based programming for IoT leveraging fog computing. In: IEEE 26th International Conference on Enabling Technologies: Infrastructure for Collaborative Enterprises (WETICE), pp. 74–79, June 2017. https://doi.org/10.1109/WETICE.2017.17
Toshniwal, A., et al.: Storm@ Twitter. In: Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data, pp. 147–156. ACM (2014)
Upton, E., Halfacree, G.: Raspberry Pi User Guide. Wiley, New York (2014)
Venkataraman, S., et al.: Drizzle: fast and adaptable stream processing at scale. In: Proceedings of the 26th Symposium on Operating Systems Principles, pp. 374–389. ACM (2017)
Wen, Z., Bhatotia, P., Chen, R., Lee, M., et al.: ApproxIoT: approximate analytics for edge computing. In: IEEE 38th International Conference on Distributed Computing Systems (ICDCS), pp. 411–421. IEEE (2018)
Zaharia, M., Das, T., Li, H., Hunter, T., Shenker, S., Stoica, I.: Discretized streams: Fault-tolerant streaming computation at scale. In: Proceedings of the Twenty-fourth ACM Symposium on Operating Systems Principles, pp. 423–438. ACM (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Thekkummal, N.B., Jha, D.N., Puthal, D., James, P., Ranjan, R. (2020). Coordinated Data Flow Control in IoT Networks. In: Brandic, I., Genez, T., Pietri, I., Sakellariou, R. (eds) Algorithmic Aspects of Cloud Computing. ALGOCLOUD 2019. Lecture Notes in Computer Science(), vol 12041. Springer, Cham. https://doi.org/10.1007/978-3-030-58628-7_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-58628-7_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-58627-0
Online ISBN: 978-3-030-58628-7
eBook Packages: Computer ScienceComputer Science (R0)