Abstract
The ever-increasing amount and variety of malware on the Internet have presented significant challenges to the interconnected network community. The emergence of unseen malware variants has resulted in a different distribution of features and labels in the training and testing datasets. For widely used machine learning-based detection methods, the issue of dataset shift will render the trained model ineffective in the face of new data. However, it is a laborious and tedious undertaking whether relearning features to describe new data or collecting large amounts of labeled samples to retrain the classifiers. To address these problems, this paper proposes TransNet, a framework based on deep transfer learning for unseen malware variants detection. We first convert the raw traffic represented by sessions containing data from all layers of the OSI model into fixed-size RGB images through data preprocessing. Afterward, based on the ResNet-50 model pre-trained on the ImageNet, we replace Batch Normalization with Transferable Normalization as the normalization layer to construct our deep transfer learning model. In this way, our approach leverages deep learning to avoid the problem of traditional machine learning in relying on expert knowledge and uses transfer learning to address the issue of domain shift. We test the effectiveness of different methods with a thorough set of experiments. TransNet achieves 95.89% accuracy and 96.09% F-measure on two public datasets from the real-world environment, which is higher than comparative methods. Meantime, our method ranks first on all ten subtasks, showing that it can detect unseen malware variants with stable and excellent performance. Moreover, the distribution discrepancy computed by our method is much smaller than other approaches, which illustrates that our method successfully reduces the shift of data distributions.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Anderson, B., McGrew, D.: Identifying encrypted malware traffic with contextual flow data. Proc. ACM Workshop Artif. Intell. Secur. 2016, 35–46 (2016)
Wang, T.S., Lin, H.T., Cheng, W.T., et al.: DBod: Clustering and detecting DGA-based botnets using DNS traffic analysis. Comput. Secur. 64, 1–15 (2017)
Kovanen, T., David, G., Hämäläinen, T.: Survey: intrusion detection systems in encrypted traffic. In: Galinina, O., Balandin, S., Koucheryavy, Y. (eds.) NEW2AN/ruSMART -2016. LNCS, vol. 9870, pp. 281–293. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46301-8_23
Wang, W., Sheng, Y., Wang, J., et al.: HAST-IDS: Learning hierarchical spatial-temporal features using deep neural networks to improve intrusion detection. IEEE Access 6, 1792–1806 (2017)
Zhao, S., Ma, X., Zou, W., Bai, B.: DeepCG: classifying metamorphic malware through deep learning of call graphs. In: Chen, S., Choo, K.-K.R., Fu, X., Lou, W., Mohaisen, A. (eds.) SecureComm 2019. LNICST, vol. 304, pp. 171–190. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-37228-6_9
Tencent Security: https://s.tencent.com/research/report/790.html. August 2019
Kumar, V., Sangwan, O.P.: Signature based intrusion detection system using SNORT. Int. J. Comput. Appl. Inf. Technol. 1(3), 35–41 (2012)
Timcenko, V., Gajin, S.: Ensemble classifiers for supervised anomaly based network intrusion detection. In: 2017 13th IEEE International Conference on Intelligent Computer Communication and Processing (ICCP), pp. 13–19. IEEE (2017)
AlAhmadi, B.A., Martinovic, I.: Malclassifier: alware family classification using network flow sequence behaviour. In: 2018 APWG Symposium on Electronic Crime Research (eCrime), pp. 1–13. IEEE (2018)
Kohout, J, Pevný, T.: Unsupervised detection of malware in persistent web traffic. In: 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1757–1761. IEEE (2015)
Alom, M.Z., Taha, T.M.: Network intrusion detection for cyber security using unsupervised deep learning approaches. In: 2017 IEEE National Aerospace and Electronics Conference (NAECON), pp. 63–69. IEEE (2017)
Bartos, K, Sofka, M, Franc, V.: Optimized invariant representation of network traffic for detecting unseen malware variants. In: 25th USENIX Security Symposium (USENIX Security 16), pp. 807–822 (2016)
Li, H., Chen, Z., Spolaor, R., et al.: DART: detecting unseen malware variants using adaptation regularization transfer learning. In: ICC 2019–2019 IEEE International Conference on Communications (ICC), pp. 1–6. IEEE (2019)
Zhao, J., Shetty, S., Pan, J.W.: Feature-based transfer learning for network security. In: MILCOM 2017–2017 IEEE Military Communications Conference (MILCOM), pp. 17–22. IEEE (2017)
Zhao, J., Shetty, S., Pan, J.W., et al.: Transfer learning for detecting unknown network attacks. EURASIP J. Inf. Secur. 2019(1), 1 (2019)
LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)
Deng, J., Dong, W., Socher, R., et al.: Imagenet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)
Rezende, E., Ruppert, G., Carvalho, T., et al.: Malicious software classification using transfer learning of resnet-50 deep neural network. In: 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA), pp. 1011–1014. IEEE (2017)
Rezende, E., Ruppert, G., Carvalho, T., Theophilo, A., Ramos, F., Geus, P.: Malicious software classification using VGG16 deep neural network’s bottleneck features. In: Latifi, S. (ed.) Information Technology - New Generations. AISC, vol. 738, pp. 51–59. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-77028-4_9
He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Wang, X., Jin, Y., Long, M., et al.: Transferable normalization: towards improving transferability of deep neural networks. In: Advances in Neural Information Processing Systems, pp. 1951–1961 (2019)
Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning, pp. 448–456 (2015)
Hubballi, N., Suryanarayanan, V.: False alarm minimization techniques in signature-based intrusion detection systems: a survey. Comput. Commun. 49, 1–17 (2014)
Long, M., Zhu, H., Wang, J., et al.: Deep transfer learning with joint adaptation networks. In: Proceedings of the 34th International Conference on Machine Learning, vol. 70, pp. 2208–2217. JMLR. org (2017)
Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2009)
Long, M., Wang, J., Ding, G., et al.: Adaptation regularization: a general framework for transfer learning. IEEE Trans. Knowl. Data Eng. 26(5), 1076–1089 (2013)
Wiesler, S., Ney, H.: A convergence analysis of log-linear training. In: Advances in Neural Information Processing Systems, pp. 657–665 (2011)
Quionero-Candela, J., Sugiyama, M., Schwaighofer, A., et al.: Dataset Shift in Machine Learning. The MIT Press, Cambridge (2009)
Student: The probable error of a mean. Biometrika 6(1), 1–25 (1908)
Wang, W., Zhu, M., Zeng, X., et al.: Malware traffic classification using convolutional neural network for representation learning. In: 2017 International Conference on Information Networking (ICOIN), pp. 712–717. IEEE (2017)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp. 1097–1105 (2012)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Stratosphere: Stratosphere Laboratory Datasets. (2015). Retrieved March 13, 2020, fromhttps://www.stratosphereips.org/datasets-overview
Borgwardt, K.M., Gretton, A., Rasch, M.J., et al.: Integrating structured biological data by kernel maximum mean discrepancy. Bioinformatics 22(14), e49–e57 (2006)
Acknowledgements
This work is supported by The National Natural Science Foundation of China (No.61702501) and The National Key Research and Development Program of China (No.2016QY05X1000, No.2018YFB1800200).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Rong, C., Gou, G., Cui, M., Xiong, G., Li, Z., Guo, L. (2020). TransNet: Unseen Malware Variants Detection Using Deep Transfer Learning. In: Park, N., Sun, K., Foresti, S., Butler, K., Saxena, N. (eds) Security and Privacy in Communication Networks. SecureComm 2020. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 336. Springer, Cham. https://doi.org/10.1007/978-3-030-63095-9_5
Download citation
DOI: https://doi.org/10.1007/978-3-030-63095-9_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63094-2
Online ISBN: 978-3-030-63095-9
eBook Packages: Computer ScienceComputer Science (R0)