© The Author(s) 2025. Published by EDP Sciences and China Science Publishing & Media Ltd.

1. Introduction

Memory data object security is a significant security threat within system endogenous security issues [1]. It pertains to attackers exploiting or manipulating data objects in memory to carry out malicious activities. Such memory data attacks typically fall into two categories: (1) Control-flow hijacking attacks [2–5]: attackers manipulate variables that affect program control flow (e.g., return addresses) to alter the program’s intended behavior; (2) Data-oriented attacks (DOAs) [6, 7]: attackers leak or utilize non-control data within the program (e.g., permission data, keys) to achieve malicious objectives such as privilege escalation or information disclosure. However, efficient protection measures for control data (e.g., Control-Flow Integrity (CFI) [8, 9]) make it difficult for attackers to execute control-flow hijacking attacks. Consequently, attackers resort to manipulating non-control data to carry out DOAs, as evidenced by attacks such as Direct Data Manipulation (DDM [6]), Data-Oriented Programming (DOP [7]), which demonstrate that manipulating non-control data objects can lead to complex and significant malicious activities. Therefore, DOAs have emerged as a critical research direction in memory security.

Existing research has been able to mitigate the threat of DOAs. Technologies such as data object encryption [10–12] and DSLR [13, 14] increase the difficulty for attackers to exploit non-control data objects. However, these studies still employ a broad data object protection strategy, resulting in significant performance overhead and making them challenging to deploy in existing program systems, as we described in Section 7. Therefore, in this paper, we aim to propose a low-overhead key-area defense strategy against DOAs.

To achieve this goal, we first analyze the key area within the program–specifically, the path from the program’s entry point to the code where a vulnerability could be exploited. In particular, we identify two prerequisites that DOAs must satisfy to launch an attack: generating appropriate program inputs to reach the vulnerability and exploiting data objects that are non-control data. Based on these prerequisites, we propose an algorithm to compute constraints on attacker inputs. Then, under the assumption that attackers can successfully execute DOAs, we propose a key-area CMD strategy [1] (a solution that addresses endogenous security issues by employing a Dynamic Heterogeneous Redundancy (DHR) architecture) to defend against DOAs. This architecture not only helps prevent attackers from exploiting known vulnerabilities to carry out DOAs but also protects against attacks that leverage unknown vulnerabilities to execute DOAs [1]. Specifically, in the DHR architecture, we utilize large language models (LLMs) [15, 16] to rewrite key-area code in order to translate unsafe executors into safe executors, thereby defending potential DOAs.

In conclusion, this paper makes several contributions:

• We propose a formalized expression algorithm that can determine whether DOA attackers are able to construct inputs to reach the exploited vulnerability points.

• We develop a key-area CMD strategy that rewrites the code from the input to the vulnerability point in the program, effectively preventing attackers from triggering the vulnerability.

• We design experiments on real-world applications and simulation to validate the rationality of the proposed DOA formalized expression algorithm and the effectiveness of the key-area CMD strategy in preventing DOAs.

2. Background

2.1. Endogenous security and cyberspace mimic defense

Endogenous security problems [1] refer to inherent contradictions within information systems that cannot be eliminated. Traditional security measures, which often focus on patching vulnerabilities and sealing off potential threats may inadvertently introduce new security risks. Endogenous security problems arise from the interaction between external attacks and internal system vulnerabilities.

CMD technology [17], pioneered by Wu et al., addresses endogenous security issues by employing DHR architecture. This approach enables the system to defend against both known and unknown vulnerabilities and backdoors. CMD combines software techniques and system structures to provide proactive defense mechanisms without relying solely on traditional security measures like firewalls and antivirus software.

As shown in Figure 1, in CMD, the core is the dynamic heterogeneous redundancy architecture, which consists of several key components:

• Strategy Scheduling: This component serves as both a central control unit and a feedback controller. It activates execution units, performs cleaning and repair operations, and executes other designated tasks based on predefined strategies.

• Policy Distribution: Responsible for determining whether external inputs should be connected to designated heterogeneous execution units within the service set, based on instructions from the strategy scheduling component.

• Policy Voting: This component applies consistency or majority decision-making strategies to the output vectors of multiple execution units and reports any abnormal states to the strategy scheduling component.

• Heterogeneous Resource Pool: A collection of all functional elements of heterogeneous behaviors that can meet the system’s demands. The strategy scheduling component generates new execution units or updates existing ones by selecting elements from this pool according to predefined reconstruction schemes.

Figure 1. Dynamic heterogeneous redundancy (DHR) architecture of CMD

2.2. Memory data object security

Attacks posing a threat to memory data security can be categorized based on the type of data being manipulated. When targeting control data, attackers exploit memory vulnerabilities to inject malicious code into writable and executable memory regions (known as code injection attacks [2]) or piece together fragments of a program’s code (gadgets) to achieve malicious objectives (known as code reuse attacks [3–5]). For example, attackers commonly utilize techniques like return into libc [18], Return-Oriented Programming (ROP) [19], or Jump-Oriented Programming (JOP) [20] to construct exploitable attack code. However, the widespread adoption of protection mechanisms for control data, such as Data Execution Prevention (DEP) [21], StackGuard [22], Address Space Layout Randomization (ASLR) [23], and CFI [8, 9], makes it challenging for attackers to manipulate control data. Consequently, attackers are compelled to seek alternative attack vectors, focusing more attention on non-control data. In non-control data operations (also known as DOAs [6, 7]), attackers employ two methods: DDM [6] or the utilization of instruction sequences (data-oriented gadgets) to construct malicious code. DDM involves leveraging memory vulnerabilities to read and write memory at specific locations. However, DDM encounters difficulties in executing complex operations until the emergence of DOP [7], which demonstrates that attacks targeting non-controlled data can achieve Turing-complete capabilities, enabling attackers to perform arbitrary computations.

The relationship between DOAs, endogenous security, and CMD. Endogenous security problems refer to inherent contradictions within information systems that cannot be fully eradicated. Traditional security approaches, typically centered around patching vulnerabilities and mitigating potential threats, might inadvertently introduce new security risks. In the context of DOAs which manipulate memory data, endogenous security problems arise from the interplay between external attacks and internal system vulnerabilities. Adversaries exploit these vulnerabilities to manipulate or compromise memory data, posing significant risks to data integrity and system confidentiality. Despite efforts to patch vulnerabilities and seal off potential threats, the inherent complexities and dependencies within information systems can create unintended consequences, leading to the emergence of new security challenges.

In the context of DOAs and their relationship with endogenous security problems, CMD presents a viable solution. DOAs often exploit internal system vulnerabilities to manipulate or compromise sensitive data, exacerbating endogenous security issues. However, CMD’s DHR architecture introduces diversity and complexity into the system, making it inherently more resilient against such attacks. By leveraging CMD, organizations can deploy proactive defense mechanisms that adapt to evolving threats, including DOAs. The dynamic nature of CMD’s architecture ensures that even in the face of unknown vulnerabilities and backdoors, the system remains robust and capable of mitigating the impact of DOAs. This highlights the potential of CMD as an effective strategy to address both endogenous security issues and the challenges posed by DOAs, ultimately enhancing the overall security posture of information systems.

3. Threat model

This paper aims to investigate low-cost defense mechanisms for safeguarding non-control data within programs. The primary requirement of DOAs is the ability to arbitrarily manipulate or write memory data through one or more memory vulnerabilities to alter a program’s execution behavior or achieve sub-attack goals, such as stitching or selecting program fragments. Thus, in our threat model, we assume an adversary capable of exploiting vulnerabilities to manipulate or corrupt memory.

Moreover, we assume that the system integrates defense techniques aimed at controlling data: CFI [8]. CFI is designed to thwart control-flow hijacking attacks, such as code injection attacks [2] and code reuse attacks [3–5]. Its fundamental concept involves confining a program’s control flow transfers within the boundaries of the Control-Flow Graph (CFG). Specifically, CFI scrutinizes the operands of call, jmp, and ret operations, which are categorized as control data. Presently, CFI protection is implemented within the Clang/LLVM compiler framework and is activated by default [24].

Unknown DOAs Identifying potentially security-sensitive data within a program entails recognizing data that adversaries could exploit for malicious actions, such as unauthorized access or data manipulation within memory. Likewise, pinpointing potential target data involves identifying information that adversaries might aim to manipulate, generally speaking, security-sensitive data is also part of the target data. The significance of this identification process lies in fortifying defenses against unknown DOAs. By understanding the landscape of sensitive and target data within a program, security measures can be tailored to safeguard these critical assets. This proactive approach enables us to preemptively implement a key-area CMD strategy, thereby mitigating the risk posed by emerging and evolving threats targeting data integrity and confidentiality.

4. DOAs formalized expression

To effectively prioritize defense against potential data attacks and operations within a program, we must assess the feasibility of attackers exploiting vulnerabilities for correlated attacks. Moreover, we assume attackers cannot tamper with control data, relying on system-embedded defense technologies as described in Section 3. Based on these considerations, we outline the requirements for successfully executing DOAs as follows:

• Requirement 1: Attackers can create inputs that reach the vulnerability trigger point.

• Requirement 2: The vulnerability trigger point manipulates non-control data objects.

For requirement 1, attackers need to construct inputs that can reach the vulnerability trigger point, necessitating inputs that satisfy relevant path constraints. Regarding requirement 2, it mandates that the data objects manipulated by the statement containing the vulnerability are non-control data. Therefore, we utilize Algorithm 1 to determine the input constraints crafted by attackers that can trigger the vulnerability. Drawing from symbolic execution [25–30], we assume the program’s state is determined by a triple S = (PC, Π, Δ), where PC denotes the next instruction, Π represents the path constraint indicating expressions generated during execution due to branching at PC, and Δ is a symbolic store containing a collection of concrete or symbolic values.

As shown in Algorithm 1, it takes as input the starting PC where the user’s input resides (start_PC), the PC where the program vulnerability resides (vul_PC), and the target program P. The algorithm first determines whether the data object manipulated by vul_PC is non-control data. If it’s control data, we assume the attack_constraint is an empty set, indicating the absence of input constraints satisfying DOAs. If the data object manipulated by vul_PC is non-control data, the rest of the code is utilized to assess the reachability of the path from program input to the vulnerability point. If reachable, the path constraint is returned; otherwise, an empty set is returned. Within the while S ≠ ∅ loop, as shown in Line 7, the algorithm iterates through states using the next function to read the next state (i.e., program counter PC, path condition Π, and variable assignment mapping Δ). The switch typeAt(PC) statement (Line 8) selects different execution operations based on the current type of the program counter PC. For assignment operations (Line 9), the value of expression e is computed and assigned to variable v. The corresponding program state S is modified, where succ(PC) denotes the address of the next instruction, and the path constraint remains unchanged, but a new symbol v is added to the symbolic store Δ. For conditional statements, the algorithm follows the branch’s decision. If the condition is ¬e, it executes goto PC’ and adds the new state to the state set S. If e holds true, it proceeds to the next instruction and adds the new state (i.e., succ(PC), updated path condition Π ∩ e, and the same variable assignment mapping Δ) to the state set S. For assertion checks, the assertion e is evaluated. When the program reaches vul_PC, symbolic execution analysis stops, and the current input constraints (attack_constraint) are outputted.

Overall, Algorithm 1 enables us to determine whether an attacker can construct an input that reaches exploitable vulnerability points. Specifically, if the set attack_constraint is non-empty, we infer that the attacker possesses the capability to trigger relevant vulnerabilities. Next, we will provide an example of DOA to demonstrate how this algorithm can be used to determine the attacker’s input constraints.

1 void do_authentication (char *user, …){
2      int authenticated = 0;
3      …
4      while (! authenticated){
5      /* Get a packet from the client */
6      type = packet_read ();
7      // calls detect_attack () internally
8      switch (type){
9      …
10     case SSH_CMSG_AUTH_PASSWORD:
11          if (auth_password(user, password))
12              authenticated =1;
13     case …
14     }
15     if (authenticated) break;
16     }
17     /* Perform session preparation */
18 do_authentication (pw);
19 }

Code 1. Source code of do_authentication().

In Code 1, we show an integer overflow vulnerability [31] exists in multiple SSH server implementations. This vulnerability occurs when an excessively large encrypted SSH packet is transmitted to the server. The issue arises when the server attempts to copy a 32-bit integer packet size value into a 16-bit integer variable. If the packet size is sufficiently large, the 16-bit integer can overflow and be set to zero. For Line 4 – Line 16, the SSH server reads packets from users and extracts the user and password fields. If the user and password match, the SSH server considers the user legitimate and sets their authenticated status to 1 (authenticated=1). If the user and password do not match, the SSH server continues the authentication process. However, if attackers do not know a user’s password but still wishes to operate with authenticated status (authenticated=1), they can exploit the vulnerability in Line 6. Line 6 reads input packet using function packet_read(), which internally invokes the vulnerable function detect_attack(). Through the integer overflow vulnerability, they can directly overwrite the authenticated status to 1, bypassing the program’s authentication mechanism. Hence, the process of calculating the constraints of attacking Code 1’s input needs to be satisfied using Algorithm 1 can be depicted as follows (as shown in Algorithm 2). The general flow of Algorithm 2 is similar to Algorithm 1. Once the attacker sets authenticated to 1, the current value of the attack_constraint variable is outputted (Line 9 – Line 11). Otherwise, the algorithm consistently verifies the user and password matching using SSH_CMSG_AUTH_PASSWORD (Line 13 – Line 18).

5. Key-area CMD strategy

In this section, we propose a key-area CMD strategy to mitigate attackers’ exploitation of vulnerabilities at low cost. As indicated in Section 4, attackers may potentially craft program inputs to trigger vulnerabilities. Therefore, safeguarding the path from program inputs to vulnerable points is deemed crucial in preventing attackers from triggering relevant vulnerabilities.

Figure 2. Key-area DHR architecture with unsafe and safe executors

As illustrated in Figure 2, we devise a DHR architecture to protect the relevant code snippets from the program’s entry point to the vulnerable points. If attackers manage to craft inputs that reach the vulnerable points, we consider the code snippets from the inputs to the vulnerable points as unsafe. To address this, we construct a new safe code snippet as a new execution entity. Thus, on the basis of this DHR architecture, there exist two execution entities: the safe executor and the unsafe executor, and these two entities are functionally equivalent. Specifically, within this DHR architecture, the policy distribution component distributes program inputs to the unsafe and safe executors respectively. After performing their respective operations, the outputs of the two executors are arbitrated by the policy voting module, which outputs inconsistent results if any. In case of inconsistent outputs, the strategy scheduling component executes cleaning and repair operations. As proposed in [32], we observed that as the lines of code (LoC) increases, it becomes more difficult for the LLMs to successfully translate the code, and it is beyond the scope of our study to improve the effectiveness of translating code with LLMs. Therefore, in the experiments on real-world applications, we focused on translating the functions containing the vulnerabilities rather than the relevant code snippets from the programs’ entry points to the vulnerability points to reduce the LoC that needs to be translated.

Figure 3. LLM to generate secure code snippets

Unsafe and safe executor We leverage LLMs [15, 16] such as GPT (Generative Pre-trained Transformer) [15] to translate code to Rust. These models excel in understanding and generating human-like text across various programming languages, making them suitable for tasks like code translation and transformation.

As shown in Figure 3, the LLM translates potentially unsafe language code (e.g., C code) into relatively safer language code (e.g., Rust, which is a relatively recent programming language designed for building safe and efficient low-level software [33]). After translation, it undergoes compilation to determine whether the generated safe language code can run properly. Upon successful execution, the LLM compares the semantics of the safe language code with the original unsafe language code. Specifically, LLM generates test cases that aim to cover the program as thoroughly as possible. These test cases are then run on both the generated code and the original code to ensure semantic equivalence. If the semantics match, the semantic transformation process concludes; if not, the code is fed back into the LLM for further language transformation. In this process, the LLM serves as a bridge for converting code from one language to another while ensuring safety and semantic consistency. The compilation step validates the functionality of the translated code, and the semantic comparison step confirms the fidelity of the translation.

1 // main -> my_malloc
2 void *my_malloc(unsigned int size)
3 {
4      if (size > 512)
5           return 0; // Unsafe operation
6      return malloc(size);
7 }

Code 2. Unsafe C code.

1 // main -> my_malloc
2 fn my_malloc(size: u32) -> Result <Vec <u8>, &’static str >{
3    if size > 512 {
4        Err(“Allocation_size_is_too_large”)
5    } else {
6        // Safely allocate a buffer and initialize it with zeros
7        Ok(vec! [0; size ])
8    }
9 }

Code 3. Safe Rust code.

An example In this example, we illustrate an unsafe operation from the program entry (function main) to the invocation of the my_malloc function in Code 2. Specifically, in Line 5, when the size is greater than 512, the wrapped malloc returns 0 (i.e., a NULL pointer), indicating a failed memory allocation. In such a scenario, if the subsequent code fails to check whether the allocation result is NULL and directly uses such a pointer for read or write operations, it may lead to undefined behavior, such as accessing invalid memory or triggering a segmentation fault.

To prevent such unsafe occurrences, we’ll rewrite the code snippet where the main function calls the my_malloc function using LLM, as shown in Code 3. In Code 3, when size > 512, the program throws an error, thereby avoiding subsequent operations referencing null pointers, among other issues. In the DHR architecture, Code 2 and Code 3 form two execution paths. If these paths yield inconsistent results during certain operations, it indicates potential exploitation of the unsafe operation in Code 2.

Discussion about asynchronous and synchronization issues To ensure the correctness of our framework with asynchronous operations, we do not introduce new asynchronous operations. Instead, it leverages existing synchronization mechanisms within the program, such as locks and semaphores [34, 35], to ensure correctness. These mechanisms help prevent race conditions and data inconsistencies by ensuring that only one operation can occur within critical sections at a time. Also, when using multi-version comparison, there is synchronization overhead. We can address this in several ways: (1) Utilizing more efficient synchronization mechanisms such as lock-free data structures, fine-grained locking, or reader-writer locks. These technologies minimize waiting times for locks, thereby reducing overhead. Alternatively, (2) employing a sliding window model to compare synchronization overhead within specific time intervals, tailored to the system’s characteristics and performance requirements. Or (3) batching multiple comparison operations together and processing them in segments. This reduces the overhead associated with frequent synchronizations, making the process more efficient.

6. Evaluation

In this section, we evaluate the effectiveness and overhead of the key-area DHR architecture in defending against DOAs through real-world applications and simulation experiments.

6.1. Real-world applications testing

6.1.1. Testing setup

We leverage GPT-4 to translate unsafe programs, which takes as input (1) the unsafe function that contains vulnerabilities and its related code blocks, (2) the budget which represents the number of attempts allowed for the LLM to translate the code until the translated code matches the original code’s semantics. Among them, we extract the code blocks the function to be translated on from the application source code, such as called functions (including transitive calls), data structures, global variables, macro definitions, standard libraries, etc., and feed them to LLM. Then LLM translates the unsafe function with its related code blocks into a Rust program. Finally, it is necessary to determine whether the two programs have the same semantics, and if they do not, LLM is asked to re-translate, and the process is repeated until the semantics are the same or the budget is exhausted.

Benchmarks. Given C’s susceptibility to unsafe memory access, its prevalent usage similar to Rust-in low-level programming tasks, and the limited reliance on third-party libraries that complicate program translation, we selected C as the source program language for our study. We collect a set of test applications used in existing studies which include [32] and [36], focusing on those applications that included representative CVE, which we used as benchmarks. We ultimately identified 9 test applications and the corresponding function included representative CVE, as shown in Table 1.

Configured Hyperparameters. To control the randomness of GPT-4’s output, we set the temperature (hyperparameters of GPT-4) to 0.2. Other hyperparameters of GPT-4 are set to the default value. We define the success of the code translation by evaluating both the success of the compilation (indicating that the translated code runs correctly) and the preservation of semantics (ensuring that the translated code maintains the same semantics as the original code). To verify semantic equivalence, we fuzzed [37, 38] (a testing technique involving generating inputs and running the program) the code before and after the translation. If the translated code produces identical outputs for identical inputs, we consider the semantics to be equivalent. Due to inherent differences between C and Rust, “identical inputs and outputs” here refer to those that are equivalent after serialization [39] (which is the process of converting data structures to uniform format). The fuzzing is conducted for a duration of 5 minutes, and the budget is set to 5, 10, and 20.

6.1.2. Real-world applications effectiveness testing

We evaluate the effectiveness of our DHR architecture. We use the fuzz tools to test whether the programs’ outputs are identical before and after translation for the same input, and we consider the translation is success if the translated program compiles and can not find any counterexample during fuzzing.

The result is demonstrated in Table 2. Overall, four out of nine functions are successfully translated. We find that the effect of code translation is strongly related to the structure of the original code, and we mainly consider the LoC and the depth of function with CVEs in the experiment. We find that the depth of function with CVEs is directly related to whether the translated program can be compiled. In the experimental results, when the depth of function with CVEs is greater than 3, the translated program can hardly be compiled. However, for function BZ2_compress, the LoC is 431 and the depth is 2, and the translated program can be successfully compiled. For the translated programs that are successfully compiled, we evaluate whether their semantics are consistent with the corresponding original programs under different budgets. As shown in Table 2, the larger the LoC, the easier it is to observe counterexamples (instances where the output of the original code differs from that of the transformed code.). Additionally, with the increase in budget, there is no significant improvement in translation effectiveness, we consider this to be because we simply ask LLM to re-translate a different version rather than provide feedback (e.g., counterexamples), which is beyond the scope of this study and will be considered in our future work.

Furthermore, we evaluate the effectiveness of defending against vulnerabilities with our DHR architecture. For Rust programs that have been successfully translated in Table 2, we fuzz them and their corresponding original version to obverse if they trigger corresponding CVEs. When the original programs trigger their respective vulnerabilities and the translated programs do not, we consider this a successful defense through the DHR architecture. However, for CVE in function hash_link_ref of snudown, we are not sure if our DHR architecture works even though we do not trigger the corresponding vulnerabilities in translated programs because we also do not trigger its vulnerability in the original version.

6.1.3. Real-world applications overhead testing

In this section, we discuss the performance and memory overhead introduced by our DHR architecture when testing real programs.

Since we successfully translate four applications, our analysis will focus on the overhead of these four applications under the DHR architecture, specifically including:

• Generating performance overhead: Performance loss in this phase mainly results from the process of generating Rust secure code snippets using the GPT-4, especially when dealing with complex functions containing CVE vulnerabilities. The iterative process of multi-round code translation and semantic consistency verification significantly increases time costs, particularly for benchmark programs with a high number of code lines and recursion levels.

• Synchronizing performance overhead: This performance loss is due to the synchronization mechanisms in multi-threaded concurrent execution. As shown in Figure 4, we define this overhead as the time increase for the secure executor compared to the unsafe executor when performing the same task. Notably, when the unsafe executor’s execution time exceeds that of the secure executor, the synchronization overhead can be considered zero.

• Voting performance overhead: Policy voting plays a crucial role in verifying the consistency of execution strategies. This process involves comparing the execution results produced by both the safe and unsafe executors. By thoroughly analyzing the differences between these results, we can identify any discrepancies that may arise and take cleaning and repair operations to ensure the program’s overall security and stability. This step is essential for maintaining the integrity of the execution process and preventing potential vulnerabilities that could compromise the system.

Figure 4. The runtime overhead of the key-area DHR architecture in different applications

However, generating performance overhead refers to the time cost that the DHR architecture introduces before execution while synchronizing performance overhead and voting performance overhead are the costs incurred during runtime. We record the overheads for generating secure code snippets for four real programs in Table 3, and Figure 4 shows a comparison of runtime overheads between the DHR architecture and the initial programs without DHR architecture (including synchronizing performance overhead and voting performance overhead).

For generating performance overhead, Table 3 shows that the time overhead for this phase is notable. However, it is important to note that while generating performance overhead introduces significant time costs, these are necessary pre-execution costs for the DHR architecture and do not affect its runtime overhead. Figure 4 illustrates that the runtime overhead caused by the synchronizing performance overhead and the voting performance overhead for the DHR architecture is approximately 1 to 3 times that of the initial performance overhead, which is within an acceptable range.

Regarding the memory overhead of the actual benchmark programs, since GPT-4 is deployed in the cloud, we do not consider the memory consumption during the LLM code translation process. The experimental results (as shown in Table 4) further indicate that the difference in runtime memory overhead between the translated and original programs is relatively limited, although the overall design of the DHR architecture inevitably introduces additional memory demands.

6.2. Simulation testing

In this section, we utilize simulation experiments to simulate the process of DOAs and validate the prerequisite conditions that an attacker’s input needs to satisfy in order to trigger DOAs. Additionally, after triggering DOAs, we verify whether the key-area CMD policy can effectively resist such attacks.

6.2.1. Testing setup

Simulation of a successful attack. We simulate the process by which attackers navigate from vulnerabilities to successful attacks (under our DHR architecture). As illustrated in Figure 5, attackers initially discover and exploit vulnerabilities in C programs in an attempt to launch attacks. If the exploited vulnerability pertains to control data, we assert that such attacks can be detected and resisted by existing defenses, such as CFI [8]. Attackers then need to seek new vulnerabilities to manipulate non-control data. Subsequently, attackers must successfully target both unsafe (C language) and safe (Rust language) executors to achieve their objectives. Otherwise, we contend that attackers cannot succeed in their attacks, meaning that the DHR architecture can defend against these instances of DOAs. In this process, we introduce three variables to represent relevant probabilities: the probability of successfully attacking the unsafe (C language) executor (P_C), the probability of successfully attacking the safe (Rust language) executor (P_Rust), and the probability of successful defense by the DHR architecture (P_defense).

Figure 5. Attack vulnerabilities and DHR architecture defense flowchart

Simulation of a program with vulnerabilities. Then, we need to abstract a program. The program here refers to a potentially vulnerable code snippet, typically a function, which is the scale handled by the LLM for code generation. As shown in Algorithm 1, programs are abstracted into three types of statements: assignment operations, conditional statements, and assertion checks. Assignment operations include general assignments, arithmetic operations, logic operations, and other related operations. Conditional statements can be subdivided into judgment statements (e.g., if statements) and loop statements (e.g., while statements), which form the programs’ framework. As discussed in Section 4 the assertion checks serve as potential trigger points for vulnerabilities. Moreover, we use two lists: a statement list and an end-position list. The statement list uses numerical indicators for different statement types: ‘0’ for assignments, ‘1’ for assertions, ‘2’ for conditional statements, and ‘3’ for loops. The end-position list uses Boolean values to indicate where code blocks end, such as endif or endwhile. For example, as shown in Code 4, a program with seven statements has a statement list of [0, 0, 2, 0, 3, 1, 1] and an end-position list of [F, F, F, T, F, T], showing where conditional blocks conclude.

1 // The statement list and end -position list of this program is $[0, 0, 2, 0, 3, 1, 1]$ and $[F, F, F, T, F, T]$.
2 void sample_code(char input1 [10], int intput2)
3 {
4      char output [6]; // assignment operation
5      int a = input2; // assignment operation
6      if( a > 1){ // judgment statement
7            int b = a; // assignment operation
8      }
9      while(a < 1){ // loop statement
10         strncpy(output, input1, sizeof(input1)); // assertion check, vulnerability 1
11     }
12     strncpy(output, input1, sizeof(input1)); // assertion check, vulnerability2
13 }

Code 4. Vulnerable sample code.

It is worth noting that, when the program is analyzed using techniques such as symbolic execution, the issue of path explosion may arise if the analysis is trapped in a loop statement, thereby preventing the attacker from triggering the vulnerability within a limited time. To facilitate an analysis of whether a vulnerability can be triggered within a limited time, we assume in our experiments that any vulnerabilities occurring after the first loop code block cannot be triggered within a limited time. As illustrated in Code 4, both Line 10 and Line 12 represent potential vulnerabilities. However, since Line 10 falls within the first loop code block while Line 12 lies outside it, it can be inferred that only the vulnerability in Line 10 can be triggered within a limited time, whereas triggering the vulnerability corresponding to Line 12 is not possible within such constraints.

The comprehensive simulation of various forms of vulnerable C programs involves setting different values for program size n, the occurrence probability of four types of statements p₁, and the probability of code block termination p₂. Under each configuration, we generate 1000 vulnerable random C programs and analyze the attack success probability for each program. The average attack success probability of these analyses is considered as the overall attack success probability P_C for the particular configuration.

6.2.2. Simulation effectiveness testing

In this paper, we propose key-area DHR architecture, aiming to enhance the system’s security performance against DOAs through a dynamic heterogeneous redundancy mechanism. As shown in Figure 5, utilizing a probability flowchart, we use P_defense to describe the possibility of successful DHR defenses. In this section, we will provide a detailed explanation of how P_defense is calculated.

In this section, we primarily analyze the effectiveness of defense strategies based on the DHR architecture. Drawing from the probabilities of randomly generated programs in previous experiments and the likelihood of attacks reaching vulnerability points, we conduct a simulation of the defense logic within the DHR architecture. Based on Figure 5, as shown in equation (1), P_defense is equal to the probability that attackers cannot simultaneously succeed in compromising both unsafe and safe executors.

$$\begin{array}{c}\hfill P_{\mathrm{defense}}=1-(1-P_C)\ast (1-P_{\mathrm{Rust}})\end{array}$$(1)

To our knowledge, since there is currently no research indicating the probability of successful attacks after C is translated into Rust, we make the assumption based on the experience from several studies [40–42] that this probability is 0.1, i.e., (P_Rust = 0.1). Following this probability, we then simulate the attack and defense processes to calculate the probability of successful DHR defense. The values of the number of successful attacks and successful defenses obtained from 10 000 attacks and defenses in 100 different simulation programs are shown in Figure 6 below. Figure 7 shows the probability of an initial successful attack and the probability of a successful attack after defense. It can be observed that the success rate of the attack significantly decreases after the DHR defense strategy. Additionally, the probability of the DHR defense successfully countering DOAs is approximately 90.94%.

Figure 6. The number of DOAs attacks and successful defenses in different programs

Figure 7. Initial attack success probability and DOAs success probability

6.2.3. Simulation overhead testing

As we discuss in experiments on real-world application, compared to merely executing the initial program (i.e., unsafe executor), there are three progresses introduce performance overhead: (1) the execution of the safe executor, (2) the execution of the policy voting, (3) the execution of the generating.

For the performance overhead caused by the execution of the safe executor, since the safe executor and the unsafe executor can run concurrently in different threads, we only need to consider the synchronization overhead. For the performance overhead caused by the execution of the policy voting, only a small amount of performance overhead is introduced because policy voting is not a high computational complexity operation.

The results of the simulation experiment are shown in Figure 8, the cyan dashed line demonstrates the ratio of the key-area DHR architecture’s running time to the initial programs’ running time. It is not difficult to find that although the ratio is very high in some extreme cases, its value is between 1 and 2 in most cases and generally declines with the increase of the initial program’s running time. Although the key-area DHR architecture introduces additional performance overhead compared to the initial program, considering the security enhancement it brings, a slight sacrifice in efficiency is acceptable.

Figure 8. The performance overhead of the key-area DHR architecture in different programs

For the generation of a safe executor, we discuss its performance overhead separately here because it is the preparation phase for our key-area DHR architecture rather than a component of our key-area DHR architecture. As shown in Figure 3, generating the safe executor may involve multiple rounds of code translation. Each translation needs to generate code with an LLM and verify semantic consistency with fuzz tools. Taking the GPT-4 model as an example, the response time for translating 100 lines of random unsafe code is generally between 500 milliseconds and 10 seconds. The duration of fuzzing is 5 minutes, which is much longer than the translation time. For the sake of discussion, we assume that the running time for each round of code translation is 10 minutes, without loss of generality. For a given initial program, the total performance overhead of generating a safe executor depends on the number of times code translation is performed. The result is shown in Figure 9.

Figure 9. The performance overhead of code translation

We also measure the memory overhead of the key-area DHR architecture, we ignore the memory overhead caused by LLM, because LLM is deployed in the cloud in the key-area DHR architecture, and the random initial programs’ memory overhead are between 100 KB and 1000 KB. As shown in Figure 10, the key-area DHR architectrue inevitably introduces additional memory overhead due to the execution of the safe executor compared to the memory overhead of the initial program.

Figure 10. The memory overhead of the key-area DHR architecture in different programs

7. Related work

Our work builds upon the study of DOAs. These studies demonstrate that DOAs indeed pose a significant threat to program security. However, to our knowledge, there has been no research on the formal expression of DOAs. In Section 4, we propose an algorithm for expressing attack input constraints to formalize DOAs.

Numerous studies have focused on resisting DOAs through various techniques. For instance, data object encryption [10–12] establishes equivalence classes of data objects using a point-to-graph approach. Objects accessed through the same pointer are considered part of the same equivalence class, enhancing data confidentiality and thwarting DOAs. DSLR [13, 14] introduces randomization for all struct-type data, adding an extra layer of unpredictability to prevent attackers from exploiting data structures. Pointer integrity research [43] safeguards control data, such as function pointers and return addresses, preventing control-flow hijack attacks commonly associated with DOAs. PAC [44] employs cryptographic MACs to protect pointers from arbitrary manipulation by attackers, preserving data integrity and thwarting unauthorized access. Pointer boundary checking techniques [45–47] verify the address range of each pointer during program runtime, ensuring pointers are within specified boundaries to prevent unauthorized memory access and data manipulation. DFI [48, 49] technology constructs a data-flow graph (DFG) representing the define-use relationship of variables. DFI instruments ensure that each variable is only modified by legitimate writes before each read instruction at runtime, mitigating unauthorized data alterations and ensuring data integrity. Overall, these techniques collectively fortify software systems against DOAs by enhancing data confidentiality, integrity, and preventing unauthorized access and manipulation of critical data structures.

However, employing such broad-spectrum defense strategies comes with significant performance implications. For instance, runtime data object encryption, which applies to all pointer-type data, incurs a non-negligible performance overhead. On average, it leads to a 42.12% runperformance overhead, as reported in [12]. Similarly, implementing runtime DSLR [14] results in substantial performance overhead, ranging from approximately 100% to 120% in programs such as gzip, gap, and twolf. SoftBound [50] introduces an average performance overhead of 67% and a memory overhead of 200% in standard benchmarks. Similarly, complete DFI, which is resistant to DOP, imposes heavy performance overhead. For example, interproc DFI exhibits a performance overhead of 103%. Thus, while theoretically effective in preventing all possible DOA exploits, complete pointer protection comes at a considerable cost. Therefore, in this paper, we employ the key-area CMD strategy to defend against DOAs with minimal overhead, and our experiments confirm the effectiveness of this defense mechanism.

8. Conclusion

In this paper, we analyze the prerequisite conditions for initiating DOAs and establish a formalized expression algorithm to demonstrate how to construct input constraints that trigger vulnerabilities. Furthermore, we propose a key-area CMD strategy to counter DOAs. In this strategy, we generate safe code snippets by rewriting unsafe code snippets, thereby constructing heterogeneous executors. It is worth noting that this paper explores the application of key-area CMD in the field of DOAs. CMD is a general method for addressing endogenous security issues, and key-area CMD could potentially be applied across various domains of endogenous security. We plan to do this research in future work.

Acknowledgments

We thank all anonymous reviewers for their helpful comments and suggestions.

Funding

This work was supported by the National Key R&D Program of China (2022YFB3102800).

Conflicts of interest

The authors declare that they have no conflict of interest.

Data availability statement

No data are associated with this article.

Author contribution statement

Prof. Chen contributed to the framework design and the experiment design, serving as the first author. Jin Wei was involved in the manuscript writing and some experimental studies, as well as proofreading and editing the final paper. Zhuyang Yu and Jiwei Chen contributed to the manuscript writing of the evaluation section, assisted with the revision of the manuscript.

References

Pin Chen is currently a researcher at Fudan University. His main research areas include software and system security, cyberspace endogenous security, intelligent vehicle security, vulnerability discovery, blockchain security, and anti-fraud.

Jin Wei is a Ph.D. candidate at Fudan University, focusing on software and system security, cyberspace endogenous security, and vulnerability discovery.

Zhuyang Yu received his M.D degree in computer technology from Chongqing University in 2024. He is currently a D.Eng candidate at Fudan University. His main research interests include vulnerability discovery, intelligent vehicle security and system security.

Jiwei Chen received his B.S. degree in electronic and information engineering from the Hefei University of Technology, Anhui, China, in 2021. He is currently pursuing the M.S. degree in Engineering at the School of Computer Science and Technology, Fudan University, Shanghai, China. His current research interests include safety of intelligent vehicles.

All Tables

All Figures

	Figure 1. Dynamic heterogeneous redundancy (DHR) architecture of CMD
In the text

	Figure 2. Key-area DHR architecture with unsafe and safe executors
In the text

	Figure 3. LLM to generate secure code snippets
In the text

	Figure 4. The runtime overhead of the key-area DHR architecture in different applications
In the text

	Figure 5. Attack vulnerabilities and DHR architecture defense flowchart
In the text

	Figure 6. The number of DOAs attacks and successful defenses in different programs
In the text

	Figure 7. Initial attack success probability and DOAs success probability
In the text

	Figure 8. The performance overhead of the key-area DHR architecture in different programs
In the text

	Figure 9. The performance overhead of code translation
In the text

	Figure 10. The memory overhead of the key-area DHR architecture in different programs
In the text