{'fraction_non_alphanumeric': 0.034375121175695064, 'fraction_numerical': 0.011943076505486835, 'mean_word_length': 4.782399103139014, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
ROPNN: Detection of ROP Payloads Using Deep Neural Networks

Return-oriented programming (ROP) is a code reuse attack that chains short snippets of existing code (known as gadgets) to perform arbitrary operations on target machines. Existing detection mechanisms against ROP often rely on certain heuristic rules and/or require instrumentations to the program or the compiler. As a result, they exhibit low detection efficiency and/or have high runtime overhead. In this paper, we present ROPNN, which innovatively combines address space layout guided disassembly and deep neural networks, to detect ROP payloads in HTTP requests, PDF files, and images, etc. The disassembler treats application input data as code pointers to potential gadgets and aims to find any potential gadget chains. The identified potential gadget chains are then classified by the deep neural network as benign or malicious. We propose novel methods to generate the two training datasets, respectively, and process huge amount (TB-level) of raw input data to obtain sufficient training data. Our experiments show that ROPNN has high detection rate (98.3%) while maintaining very low false positive rate (0.01%). To show ROPNN is usable in practical scenario, we also test it against ROP exploits that are collected in-the-wild, created manually or created by ROP exploit generation tools Ropper and ROPC. ROPNN successfully detects all of the 80 exploits. Meanwhile, ROPNN is completely non-intrusive and does not incur any runtime overhead to the protected program.


INTRODUCTION
Due to broad deployment of W⊕X or Data Execution Prevention (DEP) [36,37], code injection attacks (e.g., shellcode injection) are no longer very viable. Code reuse attacks, especially return-oriented programming (ROP) attacks [8,10,51,52], have recently risen to play the role code injection attacks used to play. A ROP attack proceeds with two phases. First, the attacker identifies a set of particular machine instruction sequences (i.e., "gadgets") that are elaborately selected from a binary executable or a shared library. Each gadget typically ends in a return instruction. Second, enabled by exploiting a (buffer overflow) vulnerability, the attacker overwrites part of the stack with the addresses and register operands of these gadgets; the addresses and register operands are placed at particular locations (in the stack) so that these gadgets will be executed sequentially if the control flow is directed to the first one. By chaining gadgets together, the attacker is often able to perform arbitrary operations on the target machine [52].
Since ROP attacks are a major threat to high-profile targets, extensive researches have been conducted to defend against ROP attacks, and the existing defenses appear to focus on two perspectives, namely prevention and detection.
From the perspective of prevention, two dominant families of defenses are ASLR (Address Space Layout Randomization) and CFI (Control Flow Integrity), respectively. ASLR can prevent classic ROP attacks; however, as ROP attacks evolve (e.g., JIT-ROP [53]), ASLR techniques must evolve as well (e.g., XnR [3], Isomeron [16]). Due to the arm race, we'd better assume that a never-seen-before ROP attack could bypass the state-of-the-art ASLR. Theoretically, CFI can prevent any ROP attack. However, on the one hand, theoretically safe CFI is not a practical solution; on the other hand, many sophisticated ROP attacks [9,15,17,39] can bypass real-world CFI implementations.
Since not all ROP attacks can be prevented and not all effective prevention mechanisms are practical, intrusion detection also plays a critical role in defending against ROP attacks. This work is directly motivated by the main limitations of the existing detection methods.
Although lots of research has been done, existing detection methods are still quite limited in meeting four highly-desired requirements: • (R1) can detect most ROP attacks; • (R2) close to zero false positive rate; • (R3) acceptable runtime overhead; • (R4) non-intrusive (e.g., does not require a lot of binary instrumentation) and easy to deploy.
As a result, although many detection methods have been proposed, they are either not very effective or not widely deployed.
(1F) Statistical-based detection [19,44]. We briefly summarize the limitations of these detection methods regarding the four requirements in Table 1 as follows. (a) Class 1A and 1C detection methods could result in low detection rates; specific heuristics or signatures are found not very hard to bypass. (b) Class 1B, 1D, and 1E detection methods could cause substantial runtime overhead. (c) Class 1A, 1B, 1D, and 1F detection methods may cause substantial changes to existing (legacy) application software and even the running environment, thus they are not transparent. In particular, Class 1A, 1B and 1F detection methods may require a substantial amount of instructions inserted into the (legacy) application software. Class 1D detection methods require the system admin to set up a speculative code execution platform, which could be a daunting task for system admins in small and even mediumsized enterprises. A detailed discussion of the existing detection methods is provided in Section 2.2.
To overcome the above limitations, in this paper we propose Ropnn, a novel detection method which combines Address Space Layout guided (ASL-guided) disassembly (based on addresses identified in application input data) and a customized deep neural network. The main reason we combine ASL-guided disassembly with deep learning is that deep learning does not require traditional feature engineering and can automatically discover the features needed for detection or classification from minimally preprocessed data [31]. For more detailed motivations of this combination, our insights are given in Section 3.
Regarding how our detection method works, firstly, our method is used to build a new kind of network Intrusion Detection System (IDS) which can be deployed in the same way as a conventional network IDS such as Snort. It is noteworthy that Ropnn can be deployed on a machine other than the protected server, making Ropnn non-intrusive and have no runtime overhead for the protected program (see R3 and R4), which is an advantage over many other methods. Secondly, once deployed, our method works in the following manner: when network packets arrive and a reassembled protocol data unit (PDU) is obtained, our method takes two steps. (Step 1) Our method does ASL-guided PDU (i.e., application input data) disassembly and obtains a set of potential gadget chains. (Step 2) The potential gadget chains obtained in Step 1 are fed into a neural network classifier. The classifier identifies each potential gadget chains as either "ROP payload" or "benign data".
Ropnn is faced with several challenges. Firstly, a deep neural network must be trained with proper input data. Since ROP payloads only contain addresses of ROP gadget chains (please refer to section 2.1), we cannot train a classifier to directly distinguish ROP payloads from benign data. Instead, we propose ASL-guided disassembly (Section 4) and create gadget-chain-like instruction sequences based on the addresses identified in benign data. In section 5, we propose a viable method to generate sufficient real gadget chains. Simply put, the two datasets are both machine instruction sequences. Now that we have a benign dataset and a malicious dataset and we train a classifier on them.
Secondly, training a deep neural network requires huge amount of training data. In our experiments, we process TB-level of raw input data to generate sufficient benign training samples.
Thirdly, we need to design a proper architecture of the deep neural network. We propose to use convolutional neural network (ConvNet) as our classifier as it is good at capturing spatial structure and local dependencies. Specifically, we use a ConvNet with three convolutional layers. In the evaluation section, we show ConvNet outperforms Multiple Layer Perceptron (MLP) and Support Vector Machine (SVM) significantly.
Last but not least, a neural network cannot directly process instruction sequences. In Section 6, we propose to convert every byte in the sequence into its hex value (0-255), and then use one-hot encoding to represent the data. We also apply padding to deal with sequences that have different lengths.
In summary, the contributions of this paper are as follows: • We propose Ropnn, a novel method for a network IDS to use in detecting ROP payloads. It combines ASL-guided disassembly and deep learning to classify reassembled PDU into either "ROP payload" or "benign data". • To the best of our knowledge, Ropnn is the first intrusion detection method that applies deep learning to mitigate the threat of ROP attacks. It sheds light on the application of powerful deep learning in cybersecurity. • We design and implement Ropnn on Linux system. We test it with several programs (e.g., Nginx). The evaluation results show that Ropnn achieves very high detection rate (98.3%) and very low false positive rate (0.01%). More importantly, it can detect real-world ROP exploits collected in-the-wild and generated by exploit generation tools. We collect and create 20 real ROP exploits for 4 vulnerable programs and Ropnn successfully detects all of them. Meanwhile, Ropnn is completely non-intrusive and does not incur any runtime overhead to the protected program.

BACKGROUND 2.1 ROP Attacks
In a typical ROP attack, an attacker carefully crafts a gadget chain and embeds it in a network packet, which exploits a certain vulnerability in the server program. Figure 1 shows the relationship of the network packet payload, the overflown stack, and the program's memory address space layout. This is adapted from the blind ROP attack [33], which targets a stack buffer overflow vulnerability in Nginx 1.4.0. As we can see, the addresses of gadgets (e.g., 0x804c51c, 0x804c69a) are embedded in the network packet. This is the crucial clue for us to detect ROP payloads in network packets. Broadly speaking, the addresses of ROP gadgets are always present in ROP payloads. It motivates us to search for addresses of potential gadgets in input data and then disassemble them. On the other hand, since only address of ROP gadgets are present in the ROP payloads, we cannot train a classifier to directly distinguish ROP payloads from benign data -there are not sufficient information for the neural network to learn from. Conversely, when we train the classifier on two set of instruction sequences (derived from ROP payloads and benign data), there are richer information for the neural network to deal with.

Existing Detection Methods against ROP Attacks
In this subsection, we classify the existing detection methods against ROP attacks into five categories and explain why they fail to meet the four requirements set in Section 1. Since we focus on detection methods, pure prevention methods (e.g., G-free [40], ASLR) are excluded here and discussed in Section 9.
Heuristic-based detection methods are based on the observation that the execution of ROP gadgets differs from the execution of benign code. DROP [11] checks whether the frequency of executed return instructions exceeds a certain threshold. kBouncer [42] and ROPecker [12] detect abnormal pattern of indirect branches. Although these heuristic rules can detect traditional ROP attacks, they are incomplete and can be bypassed by carefully constructed gadget chains. Göktaş et al. point out in [22] that some defensive mechanisms; e.g., kBouncer and ROPecker, have thresholds for the lengths of gadget chains and the sizes of gadget thus can be bypassed when the attackers use gadgets that violate these assumptions. Besides, some heuristic-based detections, e.g., DROP [11], insert new instructions into a program to do the detection, which leads to considerable runtime overhead.
Control Flow Control (CFI) [1,7,15,62,63] takes the opposite approach: instead of detecting anomalous execution trace, it forces all control flow transfers to follow a valid path defined in a static control flow graph (CFG). Theoretically, CFI can detect any ROP attacks since they inevitably violate the CFG. However, theoretically safe CFI is not a practical solution in the real world. As demonstrated in [2,43], fine-grained CFI provides strong security guarantee but at the same time introduces runtime overhead significantly high; e.g., 52% in [35]. In the real world, many practical CFI implementations have been proposed, such as clang compiler [60] and Microsoft Visual Studio [26], however, many sophisticated ROP attacks [9,15,17,39] can bypass CFI implementations. Meanwhile, CFI requires instrumentation to the program binary or even the compiler (to generate precise CFG), which leads to high runtime overhead and deployment complexity.
There are also signature-based detection against ROP, i.e., n-ROPdetector [59]. n-ROPdetector checks whether a set of addresses of API functions appear in network traffic. While this method can detect some ROP attacks, it can be easily bypassed by attacks using other API functions. Moreover, the address of critical APIs can be masqueraded to evade the detection. For example, an attacker can break a specific address into the sum of two integers, and calculate the address using certain gadgets at runtime.
Speculative-code-execution-based detection (ROPscan [45]) first searches network traffic or files for any address that points to nonrandomized modules. Then ROPscan treats it as the first gadget in a potential gadget chain, and starts speculative code execution from it. ROPscan heuristically determines that there is a ROP gadget chain if four or more consecutive gadget chains can be successfully executed. However, speculative code execution is considerably complex to configure, making it difficult to deploy in a production environment. Moreover, the fact that any input must be speculatively executed from every byte incurs considerable runtime overhead, discouraging deployment on highly loaded servers.
Check my profile [56] first searches the data region for any pointers that point to an address in code region. Starting from such an address, it then applies certain heuristic to search for a gadget-chainlike instruction sequence. Finally, it checks whether the instruction sequence exhibits the behavior of making an API call. Although calling an API function is typical behavior of any meaningful ROP attack and thus a reasonable heuristic, it can be bypassed by inserting irrelevant gadgets between the several gadgets making an API call. Moreover, standard techniques to evade API hooking can be applied to avoid detection.
Statistical-based detection methods [19,44] first extract certain information about the program execution and then train a statistical machine learning model to distinguish ordinary execution from ROP gadget chain execution. In particular, HadROP [44] collects micro-architectural events from hardware performance counters (HPC) and then trains an SVM model as a classifier. EigenROP [19] collects microarchitecture-independent program characteristics and then uses a directional statistics-based algorithm to identify deviations from the expected program characteristics during execution. They can detect various types of ROP attacks. However, they are both intrusive and require certain instrumentation to the OS kernel or the program executable to acquire such information. Moreover, they use relatively small training and testing dataset, which leads to doubt about their accuracy. Figure 2 illustrates the architecture of our proposed ROPNN. Overall, our approach has two phases: the training phase and the production phase. During the training phase, we first collect two training datasets, namely the gadget-chain-like instruction sequences and the real gadget chains. We get the gadget-chain-like instruction sequences by Address Space Layout (ASL)-guided disassembly (of the protected program's binary) based on the valid addresses identified in the benign input data. We extract real gadget chains from the protected program by chaining individual gadgets.

OVERVIEW
ASL-guided disassembly is based on the observation that even benign input data could contain bytes whose value happen to constitute a valid address. These addresses can point to gadget-like instruction sequences. Furthermore, several addresses can appear close to each other in a way that looks like a gadget chain. This motivates us to train a classifier to distinguish these instruction sequences from real gadget chains.
After the two training datasets are collected, we build a deep neural network to classify the two datasets. Specifically, we choose to use convolutional neural network (ConvNet), which is good at learning spatial structures and capturing the local dependencies in data. The ConvNet classifier has very high detection rate (98.3%) and very low false positive rate (0.01%).
During the production phase, after a PDU is obtained (re-assembled), we first identify any valid addresses contained in the PDU. Then we use the addresses to perform ASL-guided disassembly, which will result in a set of potential gadget chains. After that, we send each of them to the trained neural network. A warning is raised if any of these potential gadget chains is classified as real ROP attack. Conversely, if the ASL-guided disassembly does not produce a potential gadget chain, or the chain is classified as benign by the neural network, the input data is considered benign.
Regarding why we combine ASL-guided disassembly and deep learning classification, our main observations are as follows: O1: ROP feature extraction is a daunting challenge in applying many classic machine learning techniques. As we mention in Section 6, a clear merit of deep learning is that it does not need the ROP features to be manually extracted.
O2: This unique combination enables us to solve the ROP payload detection problem without any software instrumentation (i.e., not intrusive at all). The only thing that the Ropnn needs to know about is a memory dump of the protected program. If we train a classifier to distinguish ordinary execution trace from ROP gadget chain execution trace, we will have to rely on software instrumentation to monitor the execution trace, which is against our goal.
O3: If the ROP payload detection completely depends on ASLguided disassembly, there would probably be too many false positives. Although benign inputs should never contain gadget chains, it is possible that benign data contain pointers to potential gadgets and some of them happen to form potential gadget chains. If we treat all of them as threats and reject them, we can cause substantial denial of service.
O4: If the ROP payload detection is directly based on deeplearning-based classification of input data (e.g., network packets), there would probably be too many false positives as well. ROP payloads only contain the addresses of gadgets, which include very limited information. ROP payloads are different from traditional buffer overflow exploit payloads in that they do not contain any executable code. If we directly use benign input data and ROP payloads to train the neural network, it is hard to build an accurate classifier.
O5: The combination provides a viable way to obtain labels. Whether high-quality labels can be obtained is a most challenging hurdle when building a neural network. In our experiments, we use a large amount (TB level) of benign input data to gather sufficient training data.
O6: Support Vector Machine (SVM) and Multi-Layer Perceptron (MLP) are two widely applied classifiers. However, we find them not suitable for our task. Our evaluation results (Section 7) show that SVM and MLP suffer from 22.2% and 42.4% false positive rates, respectively. In contrast, our deep neural network achieves 0.01% false positive rate.

ASL-GUIDED DISASSEMBLY AND GADGET-CHAIN-LIKE INSTRUCTION SEQUENCES GENERATION
This section describes the details about the ASL-guided disassembly. ASL-guided disassembly treats bytes in data as addresses and checks if they point to gadget-like or gadget-chain-like instruction sequences. In training phase, We use ASL-guided disassembly to collect gadget-chain-like instruction sequences as training data for the neural network. In production phase, we use ASL-guided disassembly to identify any potential gadget chain in input data.

Disassembly of Individual Addresses
We first create a memory dump of the protected program. Then we consider every four bytes (on 32-bit system) in the input data as an address. We require the address to be in the text segment of a loaded module. Otherwise, it cannot be the start of a ROP gadget.
Here we do not limit our search space to non-randomized modules because attackers can bypass the ASLR in multiple ways. We start disassembling from any identified addresses using Capstone [47]. The disassembling can stop in two ways: (1). An invalid or privileged instruction is encountered or the disassembly reaches the end of code segment. In this case, the current address is ignored in subsequent analysis. (2). An indirect branch, i.e., ret, jmp, or call is encountered. Then the instruction sequence (from the instruction at the starting address all the way to the indirect branch) is considered a gadget-like instruction sequence.

"Chaining" of Gadget-Like Instruction Sequences
Having obtained the set of gadget-like instruction sequences, we need to figure out how they could be chained together, in a similar way as an attacker chains gadgets into a gadget chain. Specifically, if we find an address at offset n in the data which points to a gadget-like instruction sequence, we check if anyone of the next ten addresses, i.e., address at n + 4, n + 8, n + 12,...,n + 40 in the data also points to a gadget-like instruction sequence. if so, we "chain" it together with the previous gadget-like instruction sequence and repeat the process. Otherwise, we end the "chaining" process. For any "chain" that has at least two addresses, we collect the corresponding gadget-like instruction sequences and concatenate them into a gadget-chain-like instruction sequence. Note the threshold ten is determined according to the observation that most gadgets only pop less than five elements from the stack to the registers, and ten should be able to capture the next address in a "chain". We repeat the process on every address to collect all possible gadget-chain-like instruction sequence. In this way, we get the first training dataset. To efficiently implement the above algorithm, we start four parallel threads to analyze different addresses. Moreover, if an address is already examined and found to be pointing to a gadget-like instruction sequence, we record it in a global table. In this way, the analysis is not repeated on the same address. Moreover, we also process multiple files simultaneously to utilize all CPU cores.
The reader may take Figure 1 as an example of the ASL-guided disassembly during the production phase. We first identify the three addresses in the data (0x0804c69a, 0x080bcbec, 0x0804c51c). Then we start disassembling from these addresses, until a stopping criterion is met. For this particular example, we end up with a potential gadget chain (pop esi; pop edi; ret; pop eax; ret; pop esi; pop edi; pop ebp; ret;). In training phase, however, when we collect gadget-chain-like-instruction sequences, we process benign input data using the same approach.
It is noteworthy that we start ASL-guided disassembly from EV-ERY bytes of the input data, i.e., we ignore the alignment and do not limit our search to offsets divisible by four. For example, in Figure  1, we also examine the four bytes starting from n+1 (0x6af09441), n+2(0xf0944141), n+3(0x94414141). We start disassembling from these addresses, however, they are unmapped or do not lead to gadget-like instruction sequences. Thus they are ignored in subsequent analysis. The reason behind this is that we are dealing with data, so code or memory alignment actually does not apply, and any four-bytes could be an address.

REAL GADGET CHAIN GENERATION
The real gadget chain dataset is created by chaining real individual gadgets together. The gadgets can be extracted from existing binary files or libraries; e.g., Nginx, standard GNU libc. There are several existing tools to automate gadget chains generation; e.g., rp++ [54], ROPgadget [48], ropper [49], PSHAPE [20], ROPER [57]. However, the existing tools cannot be directly used in our paper due to two main reasons: 1. the number of generated gadget chains is small; 2. the generated gadget chains might cause crashes due to accesses of unmapped memory. We create our own gadget chain generation tool based on ROPgadget [48] and PSHAPE [20] to overcome the two limitations aforementioned.
Many existing tools usually build ROP exploits for one predefined scenario; e.g., execve or mprotect, which will lead to a small number of real gadget chains. Another reason is that existing tools usually use gadgets whose lengths are as short as possible to reduce side-effects on other registers, the stack, or flags. To generate a large number of real gadget chains, we combine both short and long gadgets. We borrow the idea from ROPER to form chains by permuting the gadgets. Permutation will easily lead to state explosion, so we also limit the maximum length of gadget chains and the maximum size of gadgets.
The gadgets are added to the chain in such a way that the analyst can initialize all registers the chain dereferences, as its execution may otherwise lead to a crash. To avoid crashes, we have to solve the side effect of gadgets. Take two gadgets "mov [esi], 0x1; ret;" and "mov eax, 0x1; jmp [esi];" for instance. There exists a side effect caused by the gadget "mov eax, 0x1; jmp [esi];" unintendedly changing the value of EAX, and a register usage conflict for ESI, which is used for setting a memory as 0x1 and setting the target address for jump instructions. To solve the side effects, we remove all the gadgets that contain the memory usages, and make sure that no two gadgets read one register without write operation between them.
We can generate a huge amount of real gadget chains in this way. However, if we contain too many real gadget chains in the training data, the neural network will tend to classify more samples as real gadget chain, which can lead to higher false positives. To avoid this, we generate the same amount of real gadget chains as gadget-chain-like instruction sequences.

NEURAL NETWORK CLASSIFICATION
As mentioned in observation O1 in Section 3, it is challenging to manually extract features of ROP. We leverage the unique advantage of deep neural networks to deal with the challenge. That is, deep neural networks do not require traditional feature engineering and can automatically discover the features needed for detection or classification from raw data [31]. Neural network models with several layers of neurons were developed in the 1960s and 1970s [50], and around the same time, the efficient gradient descent method for training neural networks of arbitrary depth called backpropagation was developed [50]. Deep neural networks have been successfully applied to various fields; e.g., computer vision [29], speech recognition [24] and object classification [31].
However, deep neural networks have never been applied to the detection of ROP attacks. This problem can be formulated as a classification problem where our goal is to discriminate ROP gadget chains from gadget-chain-like instruction sequences. But there are two main challenges when applied to ROP detection. First, deep neural networks work well in classification when trained with a huge amount of data with proper labels. For example, deep neural networks perform worse than some traditional classifiers when the dataset has less than 100 samples [32]. In contrast, as mentioned in Section 4.2 and Section 5, it is challenging to get a large number of ROP gadget chains and gadget-chain-like instruction sequences. In fact, we have to disassemble TB level of input data to get enough instruction sequences (please refer to Section 7.1). Second, specific deep neural networks work much better when data has some specific structures; e.g., convolutional neural networks work well in image classification since the data has spatial structures and recurrent neural networks with long short-term memory [25] can deal with temporal structures in spoken language. It is difficult to find suitable neural network architectures for security applications because security data could be miscellaneous and cannot provide guidance of constructing neural networks in general. For example, the data in binary files are just raw binary bits which could be addresses, constants or other information, which cannot be directly used.
In Sections 4 and 5, we proposed ASL-guided disassembly and ROP gadget chains generation tool to get a large amount of gadgetchain-like instruction sequences and ROP gadget chains. So the first challenge aforementioned has been addressed. To leverage the power of deep neural networks, we now need to first properly preprocess and represent the data (Section 6.1), and then design a specific network architecture that is suitable for our task (Section 6.2).

Data Representation and Preprocessing
In order to make our two datasets (i.e., the gadget-chain-like instruction sequences and the real gadget chains) tractable for the neural network, we need to convert every instructions in them into numerical data. Recall that instructions are binary data by nature. We first convert every byte in an instruction sequence into its hex value (0-255). For example, the instruction sequence "mov eax, 1 ; pop edx ; ret" is converted to "[0xb8, 0x10, 0x00, 0x5a, 0xc3]".
We can simply scale these values to 0-1 by a division of 255 and then input them into the neural network. However, this is inappropriate for our data. Since neural networks multiplies the input data with certain weight parameters, such representation leads to an implicit relative order between different byte values. For example, in the above example, the neural network implicitly assumes the "ret" instruction (0xc3) is larger than "pop edx" (0x5a), which is meaningless regarding instructions.
To address this problem, we instead use one-hot encoding. We represent each byte value by a 256 × 1 vector, with all but one positions to be zero, and the position that corresponds to the byte value to be one. For example, the "ret" (0xc3, 195) will be represented by {0,...,0,1,0,...,0}, where we have 195 zeros in front of the one, and 60 zeros behind the one.
In other words, for an instruction sequence that has n bytes, it is represented by {X 1 , X 2 , ..., X n }, with each X i being a 256 × 1 one-hot vector.
Since instruction sequences usually have different lengths, padding is applied to make them have the same length. We first find the longest instruction sequence (in bytes). For shorter ones, we append the one-byte nop (0x90) instruction at the end of them until they all reach the same length.

Architecture of the Deep Neural Network
After the data preprocessing, a customized neural network will be used to classify whether a potential gadget chain is benign or real. The architecture of our neural network is designed to be a convolutional neural network (ConvNet), which is a particular kind of feed-forward neural networks and is good at learning data with spatial structures and capturing their local dependencies. In gadget chains, gadgets are chained with orders and adjacent instructions in a chain have meaningful connections with each other. Different permutations of the same gadgets could have totally different results. For example, the two gadget chains "pop edx; ret; add eax, edx; ret; sub eax, edx; ret;" and "add eax, edx; ret; pop edx; ret; sub eax, edx; ret;" both set the eax, but the spatial structures are different. The first gadget keeps eax unchanged, while the second gadget chain adds the differences between the values of edx after setting edx (pop edx; ret). With ConvNet, we can capture the local relation of the first two instructions by calculating the weighted sum of their hex values of these two instructions; e.g. the weighted sum of "pop edx; ret;" is different from that of "add eax, edx; ret;". In contrast, a multilayer perceptron (MLP, which is a fully connected neural network) cannot capture those spatial structures. The order of the instructions in the arrays does not affect MLP because MLP will calculate the weighted sum of all instructions at each layer. And reordered sequences with same instructions will always give the same result. Thus we use a ConvNet to preserve the spatial information in the inputs. We tailor the convolution layer in the network to be 1D convolution and choose appropriate convolution filter sizes based on the average number of instructions in the ROP gadgets. The 1D convolutional layers are demonstrated in Figure 3 1 .
The architecture of our neural network is illustrated in Figure 4, which is a zoom-in version of the ConvNet illustrated in Figure 2. After the input layer, which just passes on the arrays to the hidden layer, we use a convolutional layer with 64 random filters with  max (x, 0). After the ReLU activation, we apply a 50% dropout to prevent overfitting when using smaller datasets. Then we repeat this Convolution-BN-ReLU-Dropout structure for two more time. In particular, we use 32 convolutional kernels with the size of 5 and 16 convolutional kernels with size 3 for these two layers. Then the second covolutional layer is feed into a fully connected layer which uses a softmax activation function as a classifier. It is worth mentioning that we do not include pooling layers which are widely used in image classification tasks, because our entire input is meaningful and downsampling our input vector would yield a completely different gadget chain.
This network includes some modern techniques in deep learning such as BN [27] and dropout [55] to improve the performance. BN is a method to reduce internal covariance shift by normalizing the layer inputs in the neural network [27], and has been shown to improve learning. Dropout is a simple way to prevent a neural network from overfitting by randomly dropping a percentage of neurons during the training phase. These methods have been widely used in recent years and seen great successes.

Training ConvNet
As mentioned in Section 3, after collecting training samples (gadgetchain-like instruction sequences and real gadget chains) for a specific program, we train the ConvNet. When training the ConvNet, an optimization procedure called stochastic gradient descent (SGD) of learning rate 0.01 with momentum [46] is applied. In particular, we input a mini-batch of the training samples into the network, then compute the outputs, the errors and the gradients for the data. Then the gradients are calculated by the backpropagation algorithm to determine how the network should update its internal weights; e.g., w 1 ,w 2 and w 3 in Figure 3, that are used to compute the representation in each layer from the representation in the previous layer. We repeat this process for many mini-batches until the errors convergence or the maximum training epoch is reached.

EVALUATION
In this section, we focus on the performance of Ropnn in realworld programs. In particular, we want to answer the following questions. 1. Can Ropnn detect most ROP payloads in multiple programs with high detection rate and low false positive rate? 2. Can Ropnn outperform SVM and MLP on detection rate and false positive rate? 3. Is the performance of Ropnn acceptable? We train a ConvNet each of four different programs, namely nginx 1.4.0, poppler 0.8.4, proftpd 1.3.0a and foxit 1.0.0.0925, respectively. One specific ConvNet should be trained for one program because the (training + testing) dataset for one program is totally different from that for other programs. The instructions used in different programs are usually different and we expect to get different instruction sequences in different datasets. Then we compare Ropnn with traditional deep neural networks; e.g, MLP and traditional machine learning methods; e.g., SVM, to demonstrate the superiority of our customized neural network. Finally, we evaluate the performance throughput.

How is Big Data Used in Our Evaluation?
Collecting abundant training data is one of the most challenging tasks when utilizing the neural network. We use the following three input datasets to generate the gadget-chain-like instruction sequences: (A) a medium size HTTP traffic dataset in [61]; (B) all PDF files from arXiv [21] (800 GB in total); (C) all images in ImageNet [18] (1.2 TB in total). We use all (A)(B)(C) for Nginx to do ASL-guided disassembly (described in Section 4) and generate gadget-chain-like instruction sequences. Similarly, we use all (B) for the two pdf viewers (i.e., poppler and foxit), and all (B)(C) for the ftp server (i.e., proftpd) to do the same job. To quickly process the huge amount of data, we use a Google Cloud Platform instance with 96 vCPUs. The input data, generation time, and the number of generated gadget-chain-like instruction sequences are shown in Table 2. As a brief summary, we generate 13k to 76k gadget-chainlike instruction sequences for different programs, respectively.
It can be verified here that we do need a classifier in Ropnn. Otherwise, if we simply do ASL-guided disassembly on input data and treat all potential gadget chains as real gadget chains, all gadgetchain-like instruction sequences generated above will become false positives.
As explained in Section 5, for each program, we generate the same number of real gadget chains as gadget-chain-like instruction sequences. The generation is rapid and is done on local workstation. Now that we have four datasets for the four programs, respectively. And each of them contains two smaller datasets: the gadget-chainlike instruction sequences and the real gadgets chains, which have the same number of samples.

Can Ropnn Accurately Detect ROP Payloads against Commonly Used Programs?
In this subsection, we evaluate the accuracy of Ropnn against the four commonly used programs.
To start with, we evaluate the detection rate and false positive rate of Ropnn for Nginx. In particular, we study how the number of training samples affects the performance of Ropnn. We randomly choose 80%, 60%, 40% and 20% of Nginx dataset as training data. And respectively, the rest 20%, 40%, 60% and 80% of Nginx dataset are chosen as test data. To clarify, 80% of Nginx dataset means 80% of the gadget-chain-like instruction sequences and 80% of the real gadget chains in the Nginx dataset.
We ran the experiment under each setting for five times and take the average value. The results are shown in Table 3. When using 80% of the dataset as training data, we spend 45 minutes in training and reach highest detection rate (98.3%) and lowest false positive rate (0.01%). We can see the detection rates are higher   In image classification, millions of samples are required to train neural networks. In contrast, less than 100K samples are sufficient to successfully classify potential gadget chains. We observe that the training time increases linearly proportional to the size of training data. However, once the neural network is trained, the test speed is fast and less sensitive to the amount of training data. In the first row, it only takes 0.6 second to classify all test data (20% of all samples, 16,270 in total).
We further evaluate Ropnn on three other commonly used programs to show our approach is widely applicable. From the results of Nginx in the last subsection, we know that larger training dataset leads to better accuracy. So for the other three programs, we select 80% of the dataset as training data to achieve the best performance. We make 5-fold cross validation and the evaluation result is shown in Table 4. We reach 99.2% detection rate and 0.7% false positive rate for poppler 0.8.4, 95.8% detection rate and 0.21% false positive rate for proftpd 1.3.0a, and 97.6% detection rate and 0.46% false positive rate for foxit 1.0.0.0925, respectively. As we can see from the results, Ropnn can work well on different programs. This indicates Ropnn is widely applicable.

Can Ropnn Accurately Detect Real-World ROP Exploits?
In this subsection, we test Ropnn against real-world ROP exploits that are collected in-the-wild or generated by ROP exploit generation tools (i.e., Ropper [49] and ROPC [41]). The motivations behind these experiments are two-fold. Firstly, in the previous subsection, we show that the trained deep neural network of Ropnn is an accurate classifier with high detection rate and low false positive rate. However, despite our efforts to make the real gadget chain dataset as real as possible, there is still a chance that they differ from real-world ROP exploits to some degree. Therefore, we need a more direct evaluation here to show Ropnn is able to detect real-world ROP exploits (that are not part of the training data). Secondly, since the real gadget chains are directly generated (not from ASL-guided disassembly), we also need to show the ASL-guided disassembly is capable of correctly identify the addresses of gadgets in a real gadget chain, which is the basis for our approach.
For each vulnerable program, we first obtain at least one ROP exploit in-the-wild. For nginx, we use the attack script published in BROP [6]. For the rest three, we use the exploits published in Exploite-DB [14]. After that, we manually mutate the exploit to generate 4-5 more samples for testing. For example, we can exchange the order of several ROP gadgets without changing the behavior of the exploit. We also substitute gadgets with new gadgets that have same effects.
To further create test samples, we use Ropper [49] to generate ROP exploits that execute mprotect or execve. Ropper can generate different exploits because we can block the used gadgets and force it to generate new ones. Meanwhile, we create ROP exploits with ROPC [41], which is a ROP compiler that can compile scripts written in a special type of language (ROP Language, ROPL) into a gadget chain.
For each of the four vulnerable programs, we collect and create 20 ROP exploits and test them against Ropnn. We first observe that the ASL-guided disassembly successfully extracts the gadget addresses embedded in the payload. Subsequently, the DNN correctly classifies all of them as real gadget chains. This demonstrates the ASL-guided disassembler and the neural network synergize well and the system works as designed. Ropnn is able to detect real-world ROP exploits.   Traditional deep neural networks (e.g, MLP) and traditional machine learning methods (e.g., SVM) are widely used for classification in many application domains. For example, the paper [38] apply SVM in remote sensing image classification and the papers [58] apply MLP in medical image classification. In this section, we compare the performance of Ropnn with that of combining ASL-guided disassembly and other well-known classifiers, i.e., SVM and MLP. Specifically, we use an SVM with radial basis function (RBF) kernels, and an MLP with three fully-connected layers with 32 units in each layer. And we use 80% of Nginx dataset for training. The results are shown in Table 5.
The combination of ASL-guided disassembly and MLP leads to 96.9% detection rate and 22.2% false positive rate. The combination of ASL-guided disassembly and SVM leads to 77.6% detection rate and 42.4% false positive rate. These two false positive rates are too high for practical application because they can easily cause denial of service. The comparison results show that Ropnn significantly outperforms both MLP and SVM in both detection rate and false positive rate. The results confirm our conjecture of the potential spatial features in the instruction sequences and also validate that our data preprocessing and neural network architecture can capture the spatial features. Conversely, as mentioned earlier, MLP and SVM are unable to learn such information, and eventually produce a considerably high false positive rate.

Throughput of Ropnn
We consider the throughput of Ropnn on Nginx in this section. We can calculate from Table 2 that the disassembler works at a speed of 665 Mb/s, which can match the traffic on a server that is not too busy. Moreover, since the ASL-guided disassembly can be parallelized very well, we can split the workload across multiple servers running Ropnn, further increasing the throughput. Besides, since we deploy our program as IDS, even if the peak traffic is too much to handle, it can catch up with the amount of traffic during non-peak hours.
We run our neural network on NVIDIA Titan Xp. It takes 45 minutes to train the neural network. After training, it can classify more than 27, 100 potential gadget chains in one second. We observe the number of potential gadget chains is rather small. On average, in the 665 Mb data processed within one second, only 13 potential gadget chains are fed into the neural network for classification. Thus, the performance of the entire IDS is bounded by the speed of disassembly and the overall throughput should be considered 665 Mb/s. We also test our disassembly module on commodity hardware, where PDF readers and image viewers are installed. We run Ropnn on a laptop computer that has Intel i7 7700 CPU and Nvidia GTX 1050, and the overall throughput of the entire system is 100 Mb/s. The neural network can classify 1941 potential gadget chains in one second. This opens up the possibility of deploying Ropnn as a local intrusion detection system.

DISCUSSION AND LIMITATIONS
Readers may wonder how can Ropnn work if the traffic is encrypted (e.g., HTTPS). Encryption can hide the addresses of ROP gadgets and hinder the ASL-guided disassembly. The solution is to deploy Ropnn between the point of encryption and the protected program. For example, when a reverse proxy is deployed in front of a web-server to provide encryption, then Ropnn should be deployed between the reverse proxy and the web-server. Now Ropnn sees the unencrypted HTTP traffic and works in its normal way.
Our IDS can cooperate well with Address Space Layout Randomization (ASLR). The two gadget chain datasets only contain instructions, not addresses. Moreover, after the IDS is deployed, the disassembler always works on the current image of the running program. So the result from the disassembler is also accurate. Given these two observations combined, we find that our IDS can work in conjunction with ASLR, further raising the bar for attackers.
Readers may notice our criteria for a potential gadget in Section 4.1 is relatively broad and it may include some non-gadgets. The logic behind this is that we do not want let any real ROP gadgets evade the disassembly engine and further avoid the detection. Moreover, not setting up complex heuristic rules make our approach more robust and harder to bypass. But this does not lead to too many false positives. As discussed earlier, the false positive rate (which could lead to denial of service) is considerably low (0.01% of input potential gadget chains). Meanwhile, the number of potential gadget chains is rather small (40, 674 out of 2TB benign data). So it is estimated that on a busy server which receives 1 Gb/s traffic constantly, roughly 20 false positives will be reported in a day.
It is worth mentioning that our ConvNet is trained for a specific program. In case of a patch or an update, whether the neural  network needs to be re-trained depends on the detailed change of the program. If a part (instructions, functions, etc) of the program is removed and this section happens to be present in some gadget chains, then these gadget chains should be removed from the dataset. If some instructions or functions are inserted into the program, it is very likely that the original gadget chains are still valid and there is no need to regenerate the dataset or re-train the neural network. Moreover, we can keep track of the addresses of gadgets in a gadget chain and check if the gadget shifts certain offset in case of a patch or an update. Then we can update the two gadget chain datasets accordingly and re-train the neural network. Since training our neural network is not very time consuming, this kind of infrequent re-train is affordable. ROP attack is getting more and more complex and has several variants. Our IDS should be able to detect polymorphic ROP attacks [34] successfully because there always have to be some unmasqueraded ROP gadgets to unpack, decrypt, or arrange the real underlying attack. Although our IDS cannot capture the masqueraded ROP payload, the un-masqueraded ROP attack can be detected. On the other hand, some recent variations of ROP, which only borrow the idea of ROP, but differ from ROP to some degree, including LOP [30], which uses the entire function as basic building blocks, may evade our detection.

OTHER RELATED WORK
Besides detection of ROP attack, there are also other methods to thwart ROP attacks. Among them, G-Free [40] tries to remove the usable gadgets from the binary to stop ROP. Its strongest form is possible to prevent all types of ROP. However, G-Free requires compiler instrumentation to achieve its goal, making it difficult to apply on COTS.
Another genre of work focuses on randomization [4,5,13,23,28], which randomizes runtime address space layout and makes it difficult for an attacker to know the address of usable gadgets. Attackers often rely on remote memory disclosure vulnerability to subvert the effort of randomization. JIT-ROP [53] can bypass finegrained ASLR by exploiting remote memory disclosure vulnerability to learn about the address space layout and compile a gadget chain just-in-time to execute an arbitrary operation. In response, some recent works adopt more complex ideas to mitigate the problem. Isomeron [16] combines code randomization with execution path randomization to mitigate conventional ROP and JIT-ROP attacks. XnR [3] thwarts JIT-ROP by preventing the inadvertent reading of code while the code itself can still be executed. Still, memory randomization is an active research topic against ROP attacks.

CONCLUSION
In this paper, we present Ropnn, a novel intrusion detection method that leverages the power of deep neural networks to classify potential gadget chains produced by a ASL-guided disassembler. We avoid using heuristic rules as the final criteria to determine a malicious gadget chain. Instead, we exert very loose assumptions on the disassembler and let the neural network learn to differentiate gadget-chain-like instruction sequences (disassembled based on valid addresses contained in benign input data) and ROP gadget chains. We design a suitable architecture of the deep neural network for our task. We show that Ropnn has both high detection rate and very low false positive rate. It also successfully detects all 80 real-world ROP exploits collected or create by us. Meanwhile, it is non-intrusive and does not incur runtime overhead. We also test and discover that deep neural network outperforms SVM and MLP in terms of accuracy. We argue that Ropnn is a practical widelydeployable detection method against ROP attacks.