Skip to main content
SpringerLink
Log in

DBI, debuggers, VM: gotta catch them all

How to escape or fool debuggers with internal architecture CPU flaws?

  • Original Paper
  • Published:
Journal of Computer Virology and Hacking Techniques Aims and scope Submit manuscript

Abstract

Many developers try to protect their creations (malware, video games, etc...) from different methods of analysis, first by detecting or avoiding them. To achieve this, they use a wide variety of techniques from exploiting flaws in analysis tools through code obfuscation (self-modifying code, for instance) to the use of documented API (IsDebuggerPresent). Most of the time these methods only work on one kind of tool and they fail to treat all of them at the same time. Countermeasures of the detection methods could consist in fixing the bug exploited in the analysis tool or directly modifying results returned by API calls or handling self-modifying code in a smart way. But all of these detection methods have countermeasures which leads to a never-ending war between detection and fooling detection. The aim of this paper is to propose a new technique of detection which is supposed to handle different types of analysis environment by exploiting uncovered properties from CPU.

In this paper, we will describe a new method to protect software from dynamic analysis. This method works by detecting anomalies in the execution flow of a given thread based on on the actualization of the CPU’s cache. As a direct consequence, we can detect debuggers, Dynamic Binary Instrumentation (DBI) framework as well as virtual machines (VM). Without using dedicated exploits or specific flaws, our method is generic enough to be the same for each analysis environment which is detected since it is based on properties from the hardware on which it is executed. In addition, it does need neither any admin rights nor ring 0 accesses. Implementation of our method fits in dozens of assembler instructions, following operational requirements for offensive shellcodes. Indeed, it exploits some uncovered properties of the CPU’s cache from AMD and Intel CPU vendors. After having precisely detailed the operation of the algorithm we use and what kind of events are detected in each case, we will present at limits and different ways to use it.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. Until now, Flow is a private DBI tool which will be published soon.

  2. https://github.com/radareorg/radare2/

References

  1. Menéndez, H., Llorente, J.: Mimicking anti-viruses with machine learning and entropy profiles. Entropy 21(05), 513 (2019)

    Article  Google Scholar 

  2. Brosch, T., Morgenstern, M.: Runtime packers: the hidden problem? In: Proceedings of Black Hat, 01 (2006)

  3. Guo, F., Ferrie, P., tzi-cker Chiueh.: A study of the packer problem and its solutions. In: RAID, vol. 5230, pp. 98–115 (2008)

  4. Lita, Catalin, Cosovan, Doina, Gavrilut, Dragos: Anti-emulation trends in modern packers: a survey on the evolution of anti-emulation techniques in upa packers. J. Comput. Virol. Hacking Tech. 14(02), 107 (2017)

    Article  Google Scholar 

  5. Afianian, A., Niksefat, S., Sadeghiyan, B., Baptiste, D.: Malware dynamic analysis evasion techniques: a survey. ACM Comput. Surv. 52(11), 1–28 (2019)

    Google Scholar 

  6. Sihwail, R., Omar, K., Ariffin, K.A.Z.: A survey on malware analysis techniques: static, dynamic, hybrid and memory analysis. Int. J. Adv. Sci. Eng. Inf. Technol. 8(01), 1662 (2018)

    Article  Google Scholar 

  7. Gao, Y., Lu, Z., Luo, Y.: Survey on malware anti-analysis. In: 5th International Conference on Intelligent Control and Information Processing, ICICIP 2014-Proceedings, pp. 270–275 (2015)

  8. Microsoft.: IsDebuggerPresent function (2018). Last accessed on 2020-10-04

  9. Park, J., Jang, Y.-H., Hong, S., Park, Y.: Automatic detection and bypassing of anti-debugging techniques for microsoft windows environments. Adv. Electr. Comput. Eng. 19(05), 23–28 (2019)

    Article  Google Scholar 

  10. Gagnon, M., Taylor, S., Ghosh, A.: Software protection through anti-debugging. Secur. Privacy IEEE 5(06), 82–84 (2007)

    Article  Google Scholar 

  11. Lukan, D.: Anti-debugging: Detecting system debugger, 02 (2013)

  12. Xie, P., Lu, X., Wang, Y., Su, J., Li, M.: An automatic approach to detect anti-debugging in malware analysis. In: ISCTCS, vol. 320, pp. 436–442 (2013)

  13. Qi, Zhengwei, Li, Bingyu, Lin, Qian, Miao, Yu., Xia, Mingyuan, Guan, Haibing: Spad: software protection through anti-debugging using hardware-assisted virtualization. J. Inf. Sci. Eng. 28, 813–827 (2012)

    Google Scholar 

  14. FrançSois, P., Baptiste, D.: Exploiting flaws in windbg: how to escape or fool debuggers from existing flaws. J. Comput. Virol. Hacking Tech, 10.1007/s11416-020-00347-x (2020)

  15. Marhusin, M.F., Larkin, H., Lokan, C., Cornforth, D.: An evaluation of api calls hooking performance. In: Proceedings - 2008 International Conference on Computational Intelligence and Security, CIS 2008, vol. 1: pp. 315–319 (2008)

  16. Sun, H-M., Lin, Y-H., Wu, M-F.: Api monitoring system for defeating worms and exploits in ms-windows system. In: Proceedings of the 11th Australasian Conference on Information Security and Privacy, ACISP’06, pages 159–170, Berlin, Heidelberg. Springer-Verlag (2006)

  17. Ortega, A.: Pafish (paranoid fish), 07 (2012)

  18. Ortega, A.: Al-khaser v0.79, 11 (2015)

  19. Karvandi, S.: Defeating malware’s anti-vm techniques (cpuid-based instructions), 06 (2018)

  20. Rutkowska, J.: Subverting vistatm kernel forfun and profit, 08 (2006)

  21. Quist, D., Smith, V., Offensive Computing.: detecting the presence of virtual machines using the local data table. Offensive Comput., 25(04) (2006)

  22. Rutkowska, J.: Red pill... or how to detect vmm using (almost) one cpu instruction, 11 (2007)

  23. Leon, R., Kiperberg, M., Algawi, A., Resh, A., Zaidenberg, N.: Creating modern blue pills and red pills. In: European Conference on Cyber Warfare and Security, vol. 1: p. 9 (2019)

  24. Tuzel, T., Bridgman, M., Zepf, J., Lengyel, T.K., Temkin, K.J.: Who watches the watcher? detecting hypervisor introspection from unprivileged guests. Digital Investig. 26, S98–S106 (2018)

    Article  Google Scholar 

  25. Korkin, I.: Two challenges of stealthy hypervisors detection: Time cheating and data fluctuations. J. Digital Forensics Secur. Law, 25, 05 (2015)

  26. Desnos, A., Filiol, E., Lefou, I.: Detecting (and creating!) a hvm rootkit (aka bluepill-like). J. Comput. Virol. 7(02), 23–49 (2011)

    Article  Google Scholar 

  27. Ali, M., Shiaeles, S., Ghita, B.V., Papadaki, M.: Agent-based vs agent-less sandbox for dynamic behavioral analysis. In: 2018 Global Information Infrastructure and Networking Symposium, p. 5 (2018)

  28. Ben-Yehuda, M.: Machine virtualization:efficient hypervisors, stealthy malware, 03 (2013)

  29. Scalability, fidelity and stealth in the DRAKVUF dynamic malware analysis system. In: 30th Annual Computer Security Applications Conference, 12 2014

  30. Aaraj, N., Raghunathan, A., Jha, N.K.: Dynamic binary instrumentation-based framework for malware defense. In DIMVA, vol. 5137, 07 (2008)

  31. D’Elia, D.C., Coppa, E., Nicchi, S., Palmaro, F., Cavallaro, L.: Sok: Using dynamic binary instrumentation for security (and how you may get caught red handed). In: ACM Asia Conference on Information, Computer and Communications Security (ASIACCS 2019), p. 14 (2019)

  32. Kim, D., Kim, S., Ryou, J.: Design and implementation of user-level dynamic binary instrumentation on arm architecture. J. Supercomput. 74, 3583 (2016)

    Article  Google Scholar 

  33. Zhao, V.: Evaluation of dynamic binary instrumentation approaches: Dynamic binary translation vs. dynamic probe injection. Master’s thesis, Williams College, 06 (2018)

  34. Rodriguez, R.J., Artal, J., Merseguer, J.: Performance evaluation of dynamic binary instrumentation frameworks. Latin America Trans. IEEE (Revista IEEE America Latina), 12:1572–1580 (2014)

  35. Kirsch, J., Zhechev, Z., Bierbaumer, B. and Kittel, T.: PwIN - Pwning Intel piN: Why DBI is unsuitable for security applications. In: European Symposium on Research in Computer Security pp. 363–382. ESORICS, : Barcelona. Spain (2018)

  36. Zhechev, Z.: Security evaluation of dynamic binary instrumentation engines. Master’s thesis, University of Munich, 06 (2018)

  37. Julian, K., Zhechko, Z.: Pwning intel pin - reconsidering intel pin in context of security. In: REcon. REcon Montreal 2018, June (2018)

  38. Polino, M., Continella, A., Mariani, S., D’Alessio, S., Fontana, L., Gritti, F., Zanero, S.: Measuring and defeating anti-instrumentation-equipped malware. In: Proceedings of the Conference on Detection of Intrusions and Malware and Vulnerability Assessment (DIMVA)

  39. Bougacha, A.: Detecting valgrind, 09 (2012)

  40. Intel.: Intel® 64 and IA-32 Architectures Software Developer’s Manual Volume 3A: 8.1.3 Handling Self- and Cross-Modifying Code. Intel (2016)

  41. Microsoft.: SpinLock, 03 (2017). Last accessed on 2020-10-04

  42. Microsoft.: What is .NET?, 02 (2002). Last accessed on 2020-10-04

  43. Osnat Levi.: Pin - a dynamic binary instrumentation tool, 06 (2012)

  44. Tessier, C., Hubain, C.: Qbdi - quarkslab dynamic binary instrumentation home page, 09 (2015)

  45. Kalleberg, K.T., Ravnas, O.A.V.: Testing interoperability with closed-source software through scriptable diplomacy. (FOSDEM ’16), 01 (2016)

  46. Nethercote, N., Seward, J.: Valgrind: A framework for heavyweight dynamic binary instrumentation. (PLDI ’07) ACM (2007)

  47. Fiedor, J., Vojnar, T.: Anaconda: A framework for analysing multi-threaded c/c++ programs on the binary level. In: Shaz Qadeer and Serdar Tasiran, editors, RV, volume 7687 of Lecture Notes in Computer Science, pages 35–41. Springer (2012)

  48. Bruening, Z., Amarasinghe.: Transparent dynamic instrumentation. In: (VEE ’12). ACM (2012)

  49. Microsoft.: Structured Exception Handling (C/C++), 08 (2018). Last accessed on 2020-10-04

  50. Intel.: Pin - Command Line Switches, 05 (2018)

  51. Chatterjee, N., Majumdar, S., Sahoo, S., Das, P.: Debugging multi-threaded applications using pin-augmented gdb (pgdb), 07 (2015)

  52. Gdb: The gnu project debugger

  53. Ambavkar, P.: Debugging on linux. Int. Organ. Sci. Res. J. Eng. (IOSRJEN)February 2012, page 7, 02 (2012)

  54. Debugging in visual studio, 11 (2016). Last accessed on 2020-10-04

  55. x64dbg: An open-source x64/x32 debugger for windows

  56. ollydbg

  57. Intel.: Intel® 64 and IA-32 Architectures Software Developer’s Manual Volume 3A: chapter 23 - introduction to virtual machine extensions. Intel (2016)

  58. Biswas, Kamanashis, Islam, Md: Hardware virtualization support in intel, amd and ibm power processors. Int. J. Comput. Sci. Inf. Secur. 4, 09 (2009)

    Google Scholar 

  59. Intel.: Intel® 64 and IA-32 Architectures Software Developer’s Manual Volume 3C (2016)

  60. Intel.: Intel® 64 and IA-32 Architectures Software Developer’s Manual Volume 2A (2016)

  61. vmware

  62. Virtualbox

  63. Bellard, F.: Qemu, a fast and portable dynamic translator. In: USENIX Annual Technical Conference, FREENIX Track(2005)

  64. Microsoft.: Hyper-V Technology Overview, 11 (2016). Last accessed on 2020-10-04

  65. Microsoft.: Introduction to Hyper-V on Windows 10, 06 (2018). Last accessed on 2020-10-04

  66. Lipp, M., Schwarz, M., Gruss, D., Prescher, T., Haas, W., Fogh, A., Horn, J., Mangard, S., Kocher, P., Genkin, D., Yarom, Y., Hamburg, M.: Meltdown: Reading kernel memory from user space. In: 27th USENIX Security Symposium (USENIX Security 18) (2018)

  67. Kocher, P., Horn, J., Fogh, A., Genkin, D., Gruss, D., Haas, W., Hamburg, M., Lipp, M., Mangard, S., Prescher, T., Schwarz, M., Yarom, Y.: Spectre attacks: Exploiting speculative execution. In 40th IEEE Symposium on Security and Privacy (S&P’19), (2019)

Download references

Author information

Authors and Affiliations

  1. ESIEA, Laval, France

    François Plumerault & Baptiste David

Authors
  1. François Plumerault
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Baptiste David
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Baptiste David.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Plumerault, F., David, B. DBI, debuggers, VM: gotta catch them all. J Comput Virol Hack Tech 17, 105–117 (2021). https://doi.org/10.1007/s11416-020-00371-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11416-020-00371-x

Keywords

Search

Navigation