Abstract
Today’s robotic systems are increasingly turning to computationally expensive models such as deep neural networks (DNNs) for tasks like localization, perception, planning, and object detection. However, resource-constrained robots, like low-power drones, often have insufficient on-board compute resources or power reserves to scalably run the most accurate, state-of-the art neural network compute models. Cloud robotics allows mobile robots the benefit of offloading compute to centralized servers if they are uncertain locally or want to run more accurate, compute-intensive models. However, cloud robotics comes with a key, often understated cost: communicating with the cloud over congested wireless networks may result in latency or loss of data. In fact, sending high data-rate video or LIDAR from multiple robots over congested networks can lead to prohibitive delay for real-time applications, which we measure experimentally. In this paper, we formulate a novel Robot Offloading Problem—how and when should robots offload sensing tasks, especially if they are uncertain, to improve accuracy while minimizing the cost of cloud communication? We formulate offloading as a sequential decision making problem for robots, and propose a solution using deep reinforcement learning. In both simulations and hardware experiments using state-of-the art vision DNNs, our offloading strategy improves vision task performance by between 1.3 and 2.3\(\times \) of benchmark offloading strategies, allowing robots the potential to significantly transcend their on-board sensing accuracy but with limited cost of cloud communication.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
The post ROS Ate My Network Bandwidth! details similar (ROS Answers, 2017) behaviors.
We provide offloading DNN models and an OpenAI gym (Brockman et al., 2016) offloading simulator at https://github.com/StanfordASL/cloud_robotics. An extended technical report is available at https://arxiv.org/abs/1902.05703.
Source code for on-device distributed inference can be found at: https://bitbucket.org/sandeep_chinchali/edgetpu_dev_board_release/src/distributed_inference/.
References
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., & Isard, M., et al. (2016). Tensorflow: A system for large-scale machine learning. In Proceedings of the OSDI. Savannah.
Achiam, J., Held, D., Tamar, A., & Abbeel, P. (2017). Constrained policy optimization. In International conference on machine learning (pp. 22–31).
Altman, E. (1999). Constrained Markov decision processes. CRC Press.
Amos, B., Ludwiczuk, B., & Satyanarayanan, M.. (2016). Openface: A general-purpose face recognition library with mobile applications. Technical report, CMU-CS-16-118, CMU School of Computer Science.
Bajcsy, R. (1988). Active perception. In Proceedings of the IEEE.
Bellman, R. (1957). A Markovian decision process. DTIC Document: Technical report.
Blundell, C., Cornebise, J., Kavukcuoglu, K., & Wierstra, D. (2015). Weight uncertainty in neural networks. International conference on machine learning (ICML).
Bozcuoğlu, A. K., Kazhoyan, G., Furuta, Y., Stelter, S., Beetz, M., Okada, K., et al. (2018). The exchange of knowledge using cloud robotics. IEEE Robotics and Automation Letters, 3(2), 1072–1079.
Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., & Zaremba, W. (2016). Openai gym. arXiv:1606.01540.
Camacho, E. F., & Alba, C. B. (2013). Model predictive control. Springer.
Chinchali, S., Pergament, E., Nakanoya, M., Cidon, E., Zhang, E., Bharadia, D., et al. (2020). International symposium on experimental robotics (ISER), Malta, Valetta.
Chinchali, S., Sharma, A., Harrison, J., Elhafsi, A., Kang, D., Pergament, E., et al. (2019). Network offloading policies for cloud robotics: A learning-based approach. In Robotics: Science and systems, Freiburg im Breisgau, Germany.
Chinchali, S. P., Cidon, E., Pergament, E., Chu, T., & Katti, S. (2018). Neural networks meet physical networks: Distributed inference between edge devices and the cloud. In ACM workshop on hot topics in networks (HotNets).
Chow, Y., Nachum, O., Duenez-Guzman, E., & Ghavamzadeh, M. (2018). A lyapunov-based approach to safe reinforcement learning. In Neural information processing systems (NIPS).
Edge TUP. (2019). Retrieved September 1, 2019, from https://cloud.google.com/edge-TUP/.
Fetch Robotics. (2019). Introducing the fetch cloud robotics platform. Retrieved May 14, 2019, from https://fetchrobotics.com/products-technology/cloud-robotics-platform-for-warehouse-automation/.
Forouzan, B. A., & Fegan, S. C. (2002). TCP/IP protocol suite. McGraw-Hill.
Gal, Y., Islam, R., & Ghahramani, Z. (2017). Deep bayesian active learning with image data.
Goldberg, K., & Kehoe, B.. (2013). Cloud robotics and automation: A survey of related work. EECS Department, University of California, Berkeley, Tech. Rep. UCB/EECS-2013-5.
Harrison, J., Sharma, A., & Pavone, M. (2018). Meta-learning priors for efficient online Bayesian regression. In Workshop on the algorithmic foundations of robotics (WAFR).
Higuera, J. C. G., Xu, A., Shkurti, F., & Dudek, G. (2012). Socially-driven collective path planning for robot missions. In IEEE conference on computer and robot vision.
Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780.
Intel. (2016). Data is the new oil in the future of automated driving. Retrieved January 30, 2019, from https://newsroom.intel.com/editorials/krzanich-the-future-of-automated-driving/#gs.LoDUaZ4b.
Jain, A., Das, D., Gupta, J. K., & Saxena, A. (2015). Planit: A crowdsourcing approach for learning to plan paths from large scale preference feedback. In IEEE international conference on robotics and automation (ICRA).
Kalva, H. (2006). The h. 264 video coding standard. IEEE Multimedia, 13(4), 86–90.
Kang, D., Emmons, J., Abuzaid, F., Bailis, P., & Zaharia, M. (2017a). Noscope: Optimizing neural network queries over video at scale. In Proceedings of the VLDB Endow.
Kang, Y., Hauswald, J., Gao, C., Rovinski, A., Mudge, T., Mars, J., et al. (2017b). Neurosurgeon: Collaborative intelligence between the cloud and mobile edge. ACM SIGPLAN Notices, 52(4), 615–629.
Kehoe, B., Matsukawa, A., Candido, S., Kuffner, J., & Goldberg, K. (2013). Cloud-based robot grasping with the google object recognition engine. In 2013 IEEE international conference on robotics and automation (ICRA) (pp. 4263–4270). IEEE.
Kehoe, B., Patil, S., Abbeel, P., & Goldberg, K. (2015). A survey of research on cloud robotics and automation. IEEE Transactions on Automation Science and Engineering, 12(2), 398–409.
Kuffner, J. (2010). Cloud-enabled robots. In IEEE-RAS international conference on humanoid robots. IEEE.
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014). Microsoft coco: Common objects in context. In European conference on computer vision (pp. 740–755). Springer.
Mao, H., Netravali, R., & Alizadeh, M. (2017). Neural adaptive video streaming with pensieve. In Proceedings of the conference of the ACM special interest group on data communication (pp. 197–210). ACM.
Marsh, S. (2018). Amazon alexa crashes after christmas day overload. Retrieved January 20, 2019, from https://www.theguardian.com/technology/2018/dec/26/amazon-alexa-echo-crashes-christmas-day-overload.
Marshall, A. (2021). Starsky robotics unleashes its truly driverless truck in florida. Wired Magazine. https://www.wired.com/story/starsky-robotics-truck-self-driving-florida-test.
Mitchell, B. (2018). Learn exactly how fast a wi-fi network can move. Retrieved January 31, 2019, https://www.lifewire.com/how-fast-is-a-wifi-network-816543.
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T., Harley, T., Silver, D., & Kavukcuoglu, K. (2016). Asynchronous methods for deep reinforcement learning. In International conference on machine learning (pp. 1928–1937).
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M.. (2013). Playing atari with deep reinforcement learning. arXiv preprintarXiv:1312.5602
Mohanarajah, G., Hunziker, D., D’Andrea, R., & Waibel, M. (2015a). Rapyuta: A cloud robotics platform. IEEE Transactions on Automation Science and Engineering, 12(2), 481–493.
Mohanarajah, G., Usenko, V., Singh, M., D’Andrea, R., & Waibel, M. (2015b). Cloud-based collaborative 3d mapping in real-time with low-cost robots. IEEE Transactions on Automation Science and Engineering, 6, 66.
Padhye, J., Firoiu, V., & Towsley, D. (1999). A stochastic model of tcp reno congestion avoidence and control.
Pakha, C., Chowdhery, A., & Jiang, J. (2018). Reinventing video streaming for distributed vision analytics. In 10th USENIX workshop on hot topics in cloud computing (HotCloud 18). USENIX Association. https://www.usenix.org/conference/hotcloud18/presentation/pakha.
Penmetcha, M., & Min, B.-C. (2021). A deep reinforcement learning-based dynamic computational offloading method for cloud robotics. In IEEE Access.
Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Wheeler, R., & Ng, A. Y. (2009). Ros: An open-source robot operating system. In ICRA workshop on open source software, Kobe, Japan (Vol. 3, p. 5).
Rahman, A., Jin, J., Cricenti, A., Rahman, A., & Yuan, D. (2016). A cloud robotics framework of optimal task offloading for smart city applications. In IEEE global communications conference (GLOBECOM).
Riazuelo, L., Civera, J., & Montiel, J. M. M. (2014). C2tam: A cloud framework for cooperative tracking and mapping. Robotics and Autonomous Systems, 62(4), 401–413.
Riiser, H., Vigmostad, P., Griwodz, C., & Halvorsen, P. (2013). Commute path bandwidth traces from 3g networks: Analysis and applications. In Proceedings of the 4th ACM multimedia systems conference (MMSys’13) (pp. 114–118). ACM. ISBN 978-1-4503-1894-5.
ROS Answers. (2017). Ros ate my network bandwidth! Retrieved January 31, 2019, from https://answers.ros.org/question/256080/ros-ate-my-network-bandwidth/.
Salmerón-Garcı, J., Íñigo-Blasco, P., Dı, F., Cagigas-Muniz, D., et al. (2015). A tradeoff analysis of a cloud-based robot navigation assistant using stereo image processing. IEEE Transactions on Automation Science and Engineering, 12(2), 444–454.
Sandler, M., & Howard, A. (2018). Mobilenetv2: The next generation of on-device computer vision networks. https://ai.googleblog.com/2018/04/mobilenetv2-next-generation-of-on.html.
Schroff, F., Kalenichenko, D., & Philbin, J. (2015). Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 815–823).
Sugiura, K., & Zettsu, K. (2015). Rospeex: A cloud robotics platform for human-robot spoken dialogues. In IEEE international conference on intelligent robots and systems (IROS).
Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. IEEE Transactions on Neural Networks, 9(5), 1054.
Szepesvári, C. (2010). Algorithms for reinforcement learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 4(1), 1–103.
Tan, M., & Le, Q. V. (2019). Efficientnet: Rethinking model scaling for convolutional neural networks. arXiv preprint arXiv:1905.11946
Tanwani, A. K., Anand, R., Gonzalez, J. E., & Goldberg, K. (2020). Rilaas: Robot inference and learning as a service. IEEE Robotics and Automation Letters, 5(3), 4423–4430.
Tanwani, A. K., Mor, N., Kubiatowicz, J., Gonzalez, J. E., & Goldberg, K. (2019). A fog robotics approach to deep robot learning: Application to object recognition and grasp planning in surface decluttering. arXiv preprint arXiv:1903.09589
Tian, N., Chen, J., Ma, M., Zhang, R., Huang, B., Goldberg, K., & Sojoudi, S. (2018). A fog robotic system for dynamic visual servoing. arXiv preprintarXiv:1809.06716
Verizon. (2013). 4g lte speeds vs. your home network. Retrieved January 31, 2019, https://www.verizonwireless.com/articles/4g-lte-speeds-vs-your-home-network/.
Verma, L., Fakharzadeh, M., & Choi, S. (2013). Wifi on steroids: 802.11 ac and 802.11 ad. IEEE Wireless Communications, 20(6), 30–35.
Wan, J., Tang, S., Yan, H., Li, D., Wang, S., & Vasilakos, A. V. (2016). Cloud robotics: Current status and open issues. IEEE Access, 4, 2797–2807.
Wu, H., Lou, L., Chen, C.-C., Hirche, S., & Kuhnlenz, K. (2013). Cloud-based networked visual servo control. In IEEE transactions on industrial electronics.
Xiangyu, Z., Xinyu, Z., Mengxiao, L., & Jian, S. (2017). Shufflenet: An extremely efficient convolutional neural network for mobile devices. In Computer vision and pattern recognition.
Youtube. (2021). Youtube: Recommended upload encoding settings. https://support.google.com/youtube/answer/1722171?hl=en.
Acknowledgements
Toyota Research Institute (“TRI”) provided funds to assist the authors with their research but this article solely reflects the opinions and conclusions of its authors and not TRI or any other Toyota entity. The NASA University Leadership initiative (Grant #80NSSC20M0163) also provided funds to assist the authors with their research, but this article solely reflects the opinions and conclusions of its authors and not any NASA entity.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Special Issue on Robotics: Science and Systems 2019.
Rights and permissions
About this article
Cite this article
Chinchali, S., Sharma, A., Harrison, J. et al. Network offloading policies for cloud robotics: a learning-based approach. Auton Robot 45, 997–1012 (2021). https://doi.org/10.1007/s10514-021-09987-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10514-021-09987-4