Abstract
Machine-learning augments today’s IDS capability to cope with unknown malware. However, if an attacker gains partial knowledge about the IDS’s classifier, he can create a modified version of his malware, which can evade detection. In this article we present an IDS based on various classifiers using system calls executed by the inspected code as features. We then present a camouflage algorithm that is used to modify malicious code to be classified as benign, while preserving the code’s functionality, for decision tree and random forest classifiers. We also present transformations to the classifier’s input, to prevent this camouflage - and a modified camouflage algorithm that overcomes those transformations. Our research shows that it is not enough to provide a decision tree based classifier with a large training set to counter malware. One must also be aware of the possibility that the classifier would be fooled by a camouflage algorithm, and try to counter such an attempt with techniques such as input transformation or training set updates.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
We used Windows XP and not newer versions, in-order to allow computer viri that use exploits found on this OS but patched afterward to run on our IDS either, thus detecting both new and old (but still used) malware.
- 3.
Tracing only the first seconds of a program execution might not detect certain malware types, like “logic bombs” that commence their malicious behavior only after the program has been running some time. However, this can be mitigated both by classifying the suspension mechanism as malicious or by tracing the code operation throughout the program execution life-time, not just when the program starts.
- 4.
- 5.
- 6.
The description and the source code of this virus are available at: http://vxheaven.org/lib/vpe01.html.
- 7.
References
Baldi, P., Brunak, S., Chauvin, Y., Andersen, C.A., Nielsen, H.: Assessing the accuracy of prediction algorithms for classification: an overview. Bioinformatics 16(5), 412–424 (2000)
Biggio, B., Rieck, K., Ariu, D., Wressnegger, C., Corona, I., Giacinto, G., Rol., F.: Poisoning behavioral malware clustering. In: Proceedings of the 7th ACM Workshop on Artificial Intelligence and Security (2014)
Firdausi, I., Lim, C., Erwin, A.: Analysis of machine learning techniques used in behavior based malware detection. In: Proceedings of 2nd International Conference on Advances in Computing, Control and Telecommunication Technologies, pp. 201–203 (2010)
Forrest, S., Hofmeyr, S., Somayaji, A., Longsta, T.: A sense of self for Unix processes. In: IEEE Symposium on Security and Privacy, pp. 120–128. IEEE Press, USA (1996)
Forrest, S., Hofmeyr, S., Somayaji, A.: The evolution of system-call monitoring. In: Proceedings of the Annual Computer Security Applications Conference, pp. 418–430 (2008)
Gambs, S., Gmati, A., Hurfin, M.: Reconstruction attack through classifier analysis. In: Proceedings of the 26th Annual IFIP WG 11.3 Working Conference on Data and Applications Security and Privacy, pp. 274–281 (2012)
Hamlen, K.W., Mohan, V., Masud, M.M., Khan, L., Thuraisingham, B.: Exploiting an antivirus interface. Comput. Stand. Interfaces 31(6), 1182–1189 (2009)
Kolter, J.Z., Maloof, M.A.: Learning to detect malicious executables in the wild. In: Proceedings of the 10th International Conference on Knowledge Discovery and Data Mining, pp. 470–478 (2004)
Navarro, G.: A guided tour to approximate string matching. ACM Comput. Surv. 33(1), 31–88 (2001)
Ming, J., Xin, Z., Lan, P., Wu, D., Liu, P., Mao, B.: Replacement attacks: automatically impeding behavior-based malware specifications. In: Malkin, T., et al. (eds.) ACNS 2015. LNCS, vol. 9092, pp. 497–517. Springer, Heidelberg (2015). doi:10.1007/978-3-319-28166-7_24
Moser, A., Kruegel, C., Kirda, E.: Limits of static analysis for malware detection. In: 23rd Annual Computer Security Applications Conference, pp. 421–430 (2007)
Moskovitch, R., Gus, I., Pluderman, S., Stopel, D., Fermat, Y., Shahar, Y., Elovici, Y.: Host based intrusion detection using machine learning. In: Proceedings of Intelligence and Security Informatics, pp. 107–114 (2007)
Raffetseder, T., Kruegel, C., Kirda, E.: Detecting system emulators. In: Garay, J.A., Lenstra, A.K., Mambo, M., Peralta, R. (eds.) ISC 2007. LNCS, vol. 4779, pp. 1–18. Springer, Heidelberg (2007)
Rozenberg, B., Gudes, E., Elovici, Y., Fledel, Y.: Method for detecting unknown malicious executables. In: Kirda, E., Jha, S., Balzarotti, D. (eds.) RAID 2009. LNCS, vol. 5758, pp. 378–379. Springer, Heidelberg (2009)
Somayaji, A., Forrest, S.: Automated response using system-call delays. In: Proceedings of the 9th USENIX Security Symposium, pp. 185–198 (2000)
Sufatrio, Yap, R.H.C.: Improving host-based IDS with argument abstraction to prevent mimicry attacks. In: Valdes, A., Zamboni, D. (eds.) RAID 2005. LNCS, vol. 3858, pp. 146–164. Springer, Heidelberg (2006)
Tandon, G., Chan, P.: On the learning of system call attributes for host-based anomaly detection. Int. J. Artif. Intell. Tools 15(6), 875–892 (2006)
Wagner, D., Soto, P.: Mimicry attacks on host-based intrusion detection systems. In: Proceedings of the 9th ACM Conference on Computer and Communications Security, pp. 255–264 (2002)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Rosenberg, I., Gudes, E. (2016). Evading System-Calls Based Intrusion Detection Systems. In: Chen, J., Piuri, V., Su, C., Yung, M. (eds) Network and System Security. NSS 2016. Lecture Notes in Computer Science(), vol 9955. Springer, Cham. https://doi.org/10.1007/978-3-319-46298-1_14
Download citation
DOI: https://doi.org/10.1007/978-3-319-46298-1_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-46297-4
Online ISBN: 978-3-319-46298-1
eBook Packages: Computer ScienceComputer Science (R0)