Abstract
Smart mobile devices have access to huge amounts of data appropriate to deep learning models, which in turn can significantly improve the end-user experience on mobile devices. But massive data collection required for machine learning introduce obvious privacy issues. To this end, the notion of federated learning (FL) was proposed, which leaves the training data distributed on the mobile devices, and learns a shared model by aggregating locally-computed updates. However, in many applications one or more Byzantine devices may suffice to let current coordination learning mechanisms fail with unpredictable or disastrous outcomes. In this paper, we provide a proof-of-concept for managing security issues in federated learning systems via blockchain technology. Our approach uses decentralized programs executed via blockchain technology to establish secure learning coordination mechanisms and to identify and exclude Byzantine members. We studied the performance of our blockchain-based approach in a collective deep-learning scenario both in the presence and absence of Byzantine devices and compared our results to those obtained with an existing collective decision approach. The results show a clear advantage of the blockchain approach when Byzantine devices are part of the members.
Supported by Natural Science Fund of Shaanxi Province #K05074.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Poushter, J.: Smartphone ownership and internet usage continues to climb in emerging economies. Pew Research Center Report (2016)
McMahan, H.B., Moore, E., Ramage, D., Hampson, S., et al.: Communication-efficient learning of deep networks from decentralized data (2016). arXiv preprint: arXiv:1602.05629
Nakamoto, S.: Bitcoin: a peer-to-peer electronic cash system (2008). https://bitcoin.org/bitcoin.pdf
Buterin, V.: A next-generation smart contract and decentralized application platform. Ethereum project white paper (2014). https://github.com/ethereum/wiki/wiki/White-Paper
Wood, G.: Ethereum: a secure decentralised generalised transaction ledger. Ethereum project yellow paper (2014). http://gavwood.com/paper.pdf
Hannun, A., Case, C., Casper, J., et al.: DeepSpeech: scaling up end-to-end speech recognition (2014). arXiv:1412.5567
He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on ImageNet classification (2015). arXiv:1502.01852
Graves, A., Mohamed, A.R., Hinton, G.: Speech recognition with deep recurrent neural networks. In: ICASSP (2013)
Hinton, G., Deng, L., Yu, D., Dahl, G., et al.: Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Signal Process. Mag. 29(6), 82–97 (2012)
Krizhevsky, A., Sutskever, I., Hinton, G.: ImageNet classification with deep convolutional neural networks. In: NIPS (2012)
Simard, P., Steinkraus, D., Platt, J.: Best practices for convolutional neural networks applied to visual document analysis. In: Document Analysis and Recognition (2013)
Taigman, Y., Yang, M., Ranzato, M., Wolf, L.: DeepFace: closing the gap to human-level performance in face verification. In: CVPR (2014)
Lindell, Y., Pinkas, B.: Privacy preserving data mining. In: Bellare, M. (ed.) CRYPTO 2000. LNCS, vol. 1880, pp. 36–54. Springer, Heidelberg (2000). https://doi.org/10.1007/3-540-44598-6_3
Du, W., Han, Y., Chen, S.: Privacy-preserving multivariate statistical analysis: linear regression and classification. In: SDM, vol. 4, pp. 222–233 (2004)
Vaidya, J., Clifton, C.: Privacy preserving association rule mining in vertically partitioned data. In: KDD (2002)
Vaidya, J., Kantarcoğlu, M., Clifton, C.: Privacy-preserving Naive Bayes classification. VLDB 17(4), 879–898 (2008)
Jagannathan, G., Wright, R.: Privacy-preserving distributed k-means clustering over arbitrarily partitioned data. In: KDD (2005)
Dwork, C., Rothblum, G., Vadhan, S.: Boosting and differential privacy. In: FOCS (2010)
Chaudhuri, K., Sarwate, A., Sinha, K.: A near-optimal algorithm for differentially-private principal components. JMLR 14(1), 2905–2943 (2013)
Chaudhuri, K., Monteleoni, C.: Privacy-preserving logistic regression. In: NIPS (2009)
Zhang, J., Zhang, Z., Xiao, X., Yang, Y., Winslett, M.: Functional mechanism: regression analysis under differential privacy. VLDB 5(11), 1364–1375 (2012)
Rubinstein, B., Bartlett, P., Huang, L., Taft, N.: Learning in a large function space: privacy-preserving mechanisms for SVM learning. J. Priv. Confidentiality 4(1), 4 (2012)
Sarwate, A., Chaudhuri, K.: Signal processing and machine learning with differential privacy: algorithms and challenges for continuous data. IEEE Signal Process. Mag. 30(5), 86–94 (2013)
Chaudhuri, K., Monteleoni, C., Sarwate, A.: Differentially private empirical risk minimization. JMLR 12, 1069–1109 (2011)
Wainwright, M., Jordan, M., Duchi, J.: Privacy aware learning. In: NIPS (2012)
Shokri, R., Shmatikov, V.: Privacy-preserving deep learning. In: Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pp. 1310–1321. ACM (2015)
Hamm, J., Cao, P., Belkin, M.: Learning privately from multiparty data. In: Proceedings of the 33rd International Conference on Machine Learning, pp. 555–563 (2016)
Papernot, N., Abadi, M., Erlingsson, U., Goodfellow, I., Talwar, K.: Semi-supervised knowledge transfer for deep learning from private training data. In: Proceedings of the 5th International Conference on Learning Representations (2017)
Hitaj, B., Ateniese, G., Pérez-Cruz, F.: Deep models under the GAN: information leakage from collaborative deep learning. CoRR, vol. abs/1702.07464 (2017)
Shi, W., Cao, J., Zhang, Q., Li, Y., Xu, L.: Edge computing: vision and challenges. IEEE Internet Things J. 3(5), 637–646 (2016)
Georgiev, P., Lane, N.D., Rachuri, K.K., Mascolo, C.: DSP.Ear: leveraging co-processor support for continuous audio sensing on smartphones. In: Proceedings of the 12th ACM Conference on Embedded Network Sensor Systems, pp. 295–309. ACM (2014)
Peters, G.W., Panayi, E.: Understanding modern banking ledgers through blockchain technologies: future of transaction processing and smart contracts on the internet of money. In: Tasca, P., Aste, T., Pelizzon, L., Perony, N. (eds.) Banking Beyond Banks and Money. NEW, pp. 239–278. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-42448-4_13
Kuo, T.T., Kim, H.-E., Ohno-Machado, L.: Blockchain distributed ledger technologies for biomedical and health care applications. J. Am. Med. Inform. Assoc. 24(6), 1211–1220 (2017)
Kuo, T.T., Ohno-Machado, L.: ModelChain: decentralized privacy-preserving healthcare predictive modeling framework on private blockchain networks (2018). arXiv preprint: arXiv:1802.01746
Topol, E.J.: Money back guarantees for non-reproducible results? BMJ 353, i2770 (2016)
Baxendale, G.: Can blockchain revolutionise EPRs? ITNOW 58(1), 38–39 (2016)
Taylor, P.: Applying blockchain technology to medicine traceability (2016)
Brodersen, C., Kalis, B., Leong, C., et al.: Applying blockchain technology to medicine traceability (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhu, X., Li, H., Yu, Y. (2019). Blockchain-Based Privacy Preserving Deep Learning. In: Guo, F., Huang, X., Yung, M. (eds) Information Security and Cryptology. Inscrypt 2018. Lecture Notes in Computer Science(), vol 11449. Springer, Cham. https://doi.org/10.1007/978-3-030-14234-6_20
Download citation
DOI: https://doi.org/10.1007/978-3-030-14234-6_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-14233-9
Online ISBN: 978-3-030-14234-6
eBook Packages: Computer ScienceComputer Science (R0)