Research Article

Multi-path exploration guided by taint and probability against evasive malware

¹ National Engineering Research Center for Big Data Technology and System, Wuhan, 430074, China
² Services Computing Technology and System Lab, Cluster and Grid Computing Lab, Wuhan, 430074, China
³ Hubei Key Laboratory of Distributed System Security, Hubei Engineering Research Center on Big Data Security, Wuhan, 430074, China
⁴ School of Cyber Science and Engineering, Huazhong University of Science and Technology, Wuhan, 430074, China
⁵ School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, 430074, China
⁶ Jinyinhu Laboratory, Wuhan, 430040, China

^* Corresponding authors (email: wzqiang@hust.edu.cn)

Received: 8 May 2023
Revised: 20 June 2023
Accepted: 11 August 2023

Abstract

Static analysis is often impeded by malware obfuscation techniques, such as encryption and packing, whereas dynamic analysis tends to be more resistant to obfuscation by leveraging concrete execution information. Unfortunately, malware can employ evasive techniques to detect the analysis environment and alter its behavior accordingly. While known evasive techniques can be explicitly dismantled, the challenge lies in generically dismantling evasions without full knowledge of their conditions or implementations, such as logic bombs that rely on uncertain conditions, let alone unsupported evasive techniques, which contain evasions without corresponding dismantling strategies and those leveraging unknown implementations. In this paper, we present Antitoxin, a prototype for automatically exploring evasive malware. Antitoxin utilizes multi-path exploration guided by taint analysis and probability calculations to effectively dismantle evasive techniques. The probabilities of branch execution are derived from dynamic coverage, while taint analysis helps identify paths associated with evasive techniques that rely on uncertain conditions. Subsequently, Antitoxin prioritizes branches with lower execution probabilities and those influenced by taint analysis for multi-path exploration. This is achieved through forced execution, which forcefully sets the outcomes of branches on selected paths. Additionally, Antitoxin employs active anti-evasion countermeasures to dismantle known evasive techniques, thereby reducing exploration overhead. Furthermore, Antitoxin provides valuable insights into sensitive behaviors, facilitating deeper manual analysis. Our experiments on a set of highly evasive samples demonstrate that Antitoxin can effectively dismantle evasive techniques in a generic manner. The probability calculations guide the multi-path exploration of evasions without requiring prior knowledge of their conditions or implementations, enabling the dismantling of unsupported techniques such as C2 and significantly improving efficiency compared to linear exploration when dealing with complex control flows. Additionally, taint analysis can accurately identify branches related to logic bombs, facilitating preferential exploration.