Abstract
Machine learning lies at the core of many modern applications, extracting valuable information from data acquired from numerous sources. It has produced a disruptive change in society, providing new functionality, improved quality of life for users, e.g., through personalization, optimized use of resources, and the automation of many processes. However, machine learning systems can themselves be the targets of attackers, who might gain a significant advantage by exploiting the vulnerabilities of learning algorithms. Such attacks have already been reported in the wild in different application domains. This chapter describes the mechanisms that allow attackers to compromise machine learning systems by injecting malicious data or exploiting the algorithms’ weaknesses and blind spots. Furthermore, mechanisms that can help mitigate the effect of such attacks are also explained, along with the challenges of designing more secure machine learning systems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Machine learning is a field of computer science that gives software tools the ability to progressively improve their performance on a specific task without being explicitly programmed.
- 2.
Big data refers to extremely large datasets that, when analyzed, can reveal patterns, trends, and associations, but cannot be processed with traditional data processing tools due to data velocity, volume, value, variety, and veracity.
- 3.
Unsupervised machine learning refers to machine learning tasks that infer a function to describe hidden structure from unlabeled data.
- 4.
A decision boundary is a hypersurface that partitions the underlying vector space into multiple sets, one for each class.
- 5.
Artificial neural networks are computing systems inspired by the biological neural networks of brains.
- 6.
A loss function is a function that maps values of one or more variables onto a real number representing the cost associated with those values.
- 7.
Bilevel optimization is an optimization that embeds (nests) a problem within another problem.
- 8.
- 9.
Mean square error is the average of the squares of errors. It is a measure of estimator quality, is always non-negative, and the closer its value to zero the better.
References
Muñoz González L, Lupu EC (2018) The secret of machine learning. ITNow 60(1):38–39. https://doi.org/10.1093/itnow/bwy018
McDaniel P, Papernot N, Celik ZB (2016) Machine learning in adversarial settings. IEEE Secur Priv 14(3):68–72. https://doi.org/10.1109/MSP.2016.51
Huang L, Joseph AD, Nelson B, Rubinstein BI, Tygar J (2011) Adversarial machine learning. In: Chen Y, Cárdenas A.A, Greenstadt R, Rubinstein B (eds) Proceedings of the 4th ACM Workshop on Security and Artificial Intelligence. ACM, New York, pp 43–58. https://doi.org/10.1145/2046684.2046692
Barreno M, Nelson B, Joseph AD, Tygar J (2010) The security of machine learning. Mach Learn 81(2):121–148. https://doi.org/10.1007/s10994-010-5188-5
Barreno M, Nelson B, Sears R, Joseph AD, Tygar JD (2006) Can machine learning be secure? In: Lin, F-C, Lee, D-T, Lin B-S, Shieh S, Jajodia S (eds) Proceedings of the 2006 ACM Symposium on Information, Computer and Communications Security. ACM, New York, pp 16–25. https://doi.org/10.1145/1128817.1128824
Biggio B, Fumera G, Roli F (2014) Security evaluation of pattern classifiers under attack. IEEE T Knowl Data En 26(4):984–996. https://doi.org/10.1109/TKDE.2013.57
Muñoz-González L, Biggio B, Demontis A, Paudice A, Wongrassamee V, Lupu EC, Roli F (2017) Towards poisoning of deep learning algorithms with back-gradient optimization. In: Thuraisingham B, Biggio B, Freeman DM, Miller B, Sinha A (eds) Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, pp 27–38. https://doi.org/10.1145/3128572.3140451
Carlini N, Wagner D (2017) Towards evaluating the robustness of neural networks. In: 2017 IEEE Symposium on Security and Privacy. IEEE Computer Society, Los Alamitos, CA, USA, pp 39–57. https://doi.org/10.1109/SP.2017.49
Szegedy C, Zaremba W, Sutskever I, Bruna J, Erhan D, Goodfellow I, Fergus R (2013) Intriguing properties of neural networks. arXiv:1312.6199
Papernot N, McDaniel P, Jha S, Fredrikson M, Celik ZB, Swami A (2016) The limitations of deep learning in adversarial settings. In: Proceedings of the 2016 IEEE European Symposium on Security and Privacy. IEEE Computer Society, Los Alamitos, CA, USA, pp 372–387. https://doi.org/10.1109/EuroSP.2016.36
Papernot N, McDaniel P, Goodfellow I, Jha S, Celik ZB, Swami A (2017) Practical black-box attacks against machine learning. In: Karri R, Sinanoglu O, Sadeghi A-R, Yi X (eds) Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security. ACM, New York, pp 506–519. https://doi.org/10.1145/3052973.3053009
Paudice A, Muñoz-González L, György A, Lupu EC (2018) Detection of adversarial training examples in poisoning attacks through anomaly detection. arXiv:1802.03041
Joseph AD, Laskov P, Roli F, Tygar JD, Nelson B (eds.) (2013) Machine learning methods for computer security. Dagstuhl Manif 3(1):1–30. http://drops.dagstuhl.de/opus/volltexte/2013/4356/pdf/dagman-v003-i001-p001-12371.pdf
Nelson B, Barreno M, Chi FJ, Joseph AD, Rubinstein BI, Saini U, Sutton CA, Tygar JD, Xia K (2008) Exploiting machine learning to subvert your spam filter. In: Proceedings of the 1st Usenix Workshop on Large-Scale Exploits and Emergent Threats, article no. 7. USENIX Association, Berkeley, CA, USA. https://www.usenix.org/legacy/event/leet08/tech/full_papers/nelson/nelson.pdf
Biggio B, Nelson B, Laskov P (2012) Poisoning attacks against support vector machines. In: Langford J, Pineau J (eds) Proceedings of the 29th International Conference on Machine Learning, pp 1807–1814. arXiv:1206.6389
Mei S, Zhu X (2015) Using machine teaching to identify optimal training-set attacks on machine learners. In: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence. AAAI Press, Palo Alto, CA, USA, pp 2871–2877. https://www.aaai.org/ocs/index.php/AAAI/AAAI15/paper/viewFile/9472/9954
Xiao H, Biggio B, Brown G, Fumera G, Eckert C, Roli F (2015) Is feature selection secure against training data poisoning? In: Bach F, Blei D (eds) Proceedings of the 32nd International Conference on Machine Learning, pp 1689–1698
Do CB, Foo CS, Ng AY (2007) Efficient multiple hyperparameter learning for log-linear models. In: Proceedings of the 20th International Conference on Neural Information Processing Systems. Curran Associates, Red Hook, NY, USA, pp 377–384
Pearlmutter BA (1994) Fast exact multiplication by the Hessian. Neural Comput 6(1):147–160. https://doi.org/10.1162/neco.1994.6.1.147
Domke J (2012) Generic methods for optimization-based modeling. In: Proceedings of the 15th International Conference on Artificial Intelligence and Statistics, pp 318–326. http://proceedings.mlr.press/v22/domke12/domke12.pdf
Papernot N, McDaniel, P, Goodfellow I (2016) Transferability in machine learning: from phenomena to black-box attacks using adversarial samples. arXiv:1605.07277
Feng J, Xu H, Mannor S, Yan S (2014) Robust logistic regression and classification. In: Ghahramani Z, Welling M, Cortes C, Lawrence ND, Weinberger KQ (eds) Proceedings of the 27th International Conference on Neural Information Processing Systems, vol 1. MIT Press, Cambridge, pp 253–261
Steinhardt J, Koh PWW, Liang PS (2017) Certified defenses for data poisoning attacks. In: Guyon I, Luxburg UV, Bengio S, Wallach H, Fergus R, Vishwanathan S, Garnett R (eds) Advances in neural information processing systems 30 (NIPS 2017). Curran Associates, Red Hook, NY, USA, pp 3520–3532. http://papers.nips.cc/paper/6943-certified-defenses-for-data-poisoning-attacks.pdf
Paudice A, Muñoz-González L, Lupu EC (2018) Label sanitization against label flipping poisoning attacks. arXiv:1803.00992
Koh PW, Liang P (2017) Understanding black-box predictions via influence functions. In: Proceedings of the 34th International Conference on Machine Learning, pp 1885–1894. arXiv:1703.04730v2
Goodfellow I, Shlens J, Szegedy C (2015) Explaining and harnessing adversarial examples. arXiv:1412.6572
Evtimov I, Eykholt K, Fernandes E, Kohno T, Li B, Prakash A, Rahmati A, Song D (2017) Robust physical-world attacks on deep learning models. arXiv:1707.08945
Melis M, Demontis A, Biggio B, Brown G, Fumera G, Roli F (2017) Is deep learning safe for robot vision? Adversarial examples against the iCub Humanoid. In: ICCV Workshop on Vision in Practice on Autonomous Robots, Venice, Italy, 23 Oct 2017. arXiv:1708.06939
Grosse K, Manoharan P, Papernot N, Backes M, McDaniel P (2017) On the statistical detection of adversarial examples. arXiv:1702.06280
Gong Z, Wang W, Ku WS (2017) Adversarial and clean data are not twins. arXiv:1704.04960
Carlini N, Wagner D (2017) Adversarial examples are not easily detected: bypassing ten detection methods. In: Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security. ACM, New York, pp. 3–14. https://doi.org/10.1145/3128572.3140444
Papernot N, McDaniel P, Wu X, Jha S, Swami A (2016) Distillation as a defense to adversarial perturbations against deep neural networks. In: 2016 IEEE Symposium on Security and Privacy. IEEE Computer Society, Los Alamitos, CA, USA, pp 582–597. https://doi.org/10.1109/SP.2016.41
Carlini N, Wagner D (2016) Defensive distillation is not robust to adversarial examples. arXiv:1607.04311
Bhagoji AN, Cullina D, Mittal P (2017) Dimensionality reduction as a defense against evasion attacks on machine learning classifiers. arXiv:1704.02654v2
Maclaurin D, Duvenaud D, Adams R (2015) Gradient-based hyperparameter optimization through reversible learning. In: Bach F, Blei D (eds) Proceedings of the 32nd International Conference on Machine Learning, pp 2113–2122
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Muñoz-González, L., Lupu, E.C. (2019). The Security of Machine Learning Systems. In: Sikos, L. (eds) AI in Cybersecurity. Intelligent Systems Reference Library, vol 151. Springer, Cham. https://doi.org/10.1007/978-3-319-98842-9_3
Download citation
DOI: https://doi.org/10.1007/978-3-319-98842-9_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-98841-2
Online ISBN: 978-3-319-98842-9
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)