{'fraction_non_alphanumeric': 0.03433182875119683, 'fraction_numerical': 0.008525965440204258, 'mean_word_length': 4.409124537607892, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Analysis of Attacker Behavior in Compromised Hosts During Command and Control

Traditional reactive approach of blacklisting botnets fails to adapt to the rapidly evolving landscape of cyberattacks. An automated and proactive approach to detect and block botnet hosts will immensely benefit the industry. Behavioral analysis of botnet is shown to be effective against a wide variety of attack types. Current works, however, focus solely on analyzing network traffic from and to the bots. In this work we take a different approach of analyzing the chain of commands input by attackers in a compromised host. We have deployed several honeypots to simulate Linux shells and allowed attackers access to the shells to collect a large dataset of commands. We have further developed an automated mechanism to analyze these data. For the automation we have developed a system called CYbersecurity information Exchange with Privacy (CYBEX-P). Finally, we have done a sequential analysis on the dataset to show that we can successfully predict attacker behavior from the shell commands without analyzing network traffic like previous works.


I. INTRODUCTION
A key component in cyberattacks is a bot -a compromised host or a botnet [1], [2] -a set of such hosts. A host becomes a bot when an attacker gains access to its shell. To do that the attacker can plant a malware (e.g. mirai [3], Torpig [4], Conficker [5] etc.) or directly perform brute-force attack to crack the password. Attackers use botnet to distribute malware, perform DDoS attacks, host phishing websites, perform brute-force attacks etc. Bots make up a large portion of the cybersecurity market. So it is desirable to detect and block a bot in any network.

A. Motivation & Challenges
Currently the most popular defense against botnet is manually blacklisting their IP addresses. However, numerous hosts are compromised every day. At the same time, many bots become benign as the owner regains control of the host. So, it is impossible to list all their IPs. Moreover, blacklisting is a reactive approach. An IP shows up in a blacklist only after it has done some harm.
As a result, the industry would greatly benefit from a proactive defense mechanism against botnets. An intelligent system should detect a zero-day bot from its behavior not the IP. If a bot is detected in an early phase of the kill chain, it cannot do any harm to anybody else. This research is supported by the National Science Foundation (NSF), USA, Award #1739032.
There is extensive research on behavioral analysis of botnet. Intrusion detection systems (IDS) [6] use network signatures to detect bots. While they work really well for known patterns they cannot adapt to the new attacks. It also takes a long time to detect an attack pattern, analyze it and create its signature before adding it to an IDS.
Another popular approach is using anomalies in network traffic [7]- [9] to detect bots. Some works take it further to detect anomalies in DNS traffic [10]- [13]. However, network traffic data is cumbersome to work with and often encrypted.

B. Contribution
In this work, we take a novel approach of analyzing the chain of commands input by attackers in a compromised hosts, rather than using network traffic data. A safe but effective approach of documenting commands input in bots is to use honeypots [14].
We have allowed attackers access to a simulated shell of the Cowrie [15] honeypot. After gaining access to the honeypot's shell, attackers input several commands. Attackers try to gain information about the system or gain access to a privileged shell. After trying out several commands the attacker logs out of the shell. We call this sequence of commands input during one particular login session a command chain.
We then create a model from these command chains based on their frequency occurrence. Finally, we try to predict the next command to be input by the attacker from our dataset. We have achieved an accuracy of 94-99% in predicting the next command input by the attacker. This shows that attacker behavior is predictable using command chains only. To the best of our knowledge, no previous work has looked into this behavior of botnets and our approach is a novel one.

II. RELATED WORK
Extensive work was done on botnet behavior. Some of them used dataset similar to ours. Shrivastava et al. [16] captured different attacks on IoT devices using the Cowrie [15] honeypot. They employed various machine learning algorithms, including Naive Bayes, Random Forest and Support Vector Machine (SVM) to classify these attack. Surnin et al. studied different techniques for SSH detection and proposed a methodology for probabilistic estimation of honeypot detection. Dowling et al. [17] used reinforcement learning to achieve a similar goal of concealing honeypot functionality.
Several works studied the Mirai [3] botnet using Cowrie honeypot. Kambourakis et al. [18] provided details on how the Mirai botnet malware spreads, and discussed defensive strategies. Said et al. [19] discussed techniques to classify binary samples as Mirai based on their syntactic and behavioral properties. Lingenfelter et al. [20] analyzed variation among IoT Botnets using medium interaction honeypots. However, none of these works used our approach of analyzing the shell commands during the command and control stage. Our work differs by proposing an automated methodology to analyze their behavior by analyzing the shell commands input during the command and control stage.
Several previous works modeled the attacker behavior in IoT network. Deshmukh et al. [21] proposes Fusion Hidden Markov Model (FHMM) for modeling attacker behavior. FHMM is more noise resistant and provides faster performance than Deep Recurrent Neural Network (DeepRNN) with comparative accuracy.
Finally, Rade et al. [22] modeled honeypot data using semisupervised Markov Chains and Hidden Markov Models (HMM). They also explored Long Short-Term Memory (LSTM) for attack sequence modeling. They concluded that LSTM provides better accuracy than HMM. As shown in section VII, we have used normalized Levenshtein distance [23] to get a faster performance and a better accuracy. We also propose an automation framework to better capture the evolving nature of the threat landscape.

III. SYSTEM ARCHITECTURE
In this work, we have developed an automated framework to analyze and fingerprint attacker behavior in any compromised host. Our system uses CYbersecurity information Exchange with Privacy (CYBEX-P) [24] infrastructure as a service (IaaS). CYBEX-P is a cloud based platform for organizations to share heterogeneous cyberthreat data. CYBEX-P accepts all kinds of human or machine generated data including firewall logs, emails, malware signatures etc. Our system has 7 modules -(1) Honeypots, (2) Frontend, (3) Input, (4) API, (5) Archive, (6) Analytics, and (7) Report. These modules share various components as shown in Fig. 1.

1) Honeypots:
We have setup 4 instances of the Cowrie honeypot all around the world. The locations are -Amsterdam, London, Mumbai and San Jose. All of them login SSH login attempts and corresponding commands input upon successful login. We have a diverse choice of four honeypots in four locations for better analysis and correlation.
2) Frontend Module: The frontend module ( , in Fig.  1) is a webapp for users to interact with CYBEX-P. This module allows users -(1) to register with and login to CYBEX-P, (2) to configure the data sources, (3) to view the data, (4) to generate reports, and (5) to visualize the data.
3) Input module: The input module ( , , , , in Fig. 1) handles all the data incoming to CYBEX-P. Machine data is automatically sent via a connector ( ) to the collector ( ) using real time websockets. Afterwards, the collector posts the raw data to our API ( ) endpoint. To ensure privacy, it uses the transport layer security (TLS) protocol [25] during collection and posting.
4) API module: The API module ( , in Fig. 1) consists of the API server ( ) and the cache data lake ( ). It acts as the gateway for all data into and out of CYBEX-P. It serves two primary purposes: 1) The input module (subsection III-3) puts raw data into the system using the API. 2) The report module (subsection III-7) sends reports back to users using the API. 5) Archive module: The archive module ( , in Fig. 1) resides in the archive cluster and consists primarily of a set of parsing scripts. As mentioned earlier, the cache data lake ( ) is encrypted with the public key of the archive server ( ). The archive server -(1) gets the encrypted data from the cache data lake (2) decrypts the data using own private key (3) parses the data into TAHOE, and (4) stores the data in the archive DB ( ).
6) Analytics module: The analytics module ( , in Fig.  1) works on the archived data to transform, enrich, analyze or correlate them. It has various sub modules, some of which described here. a) Filter sub-module: An analytics filter parses a specific event from raw user data. Multiple filters can act on the same raw data and vice-versa. For example, one filter can extract a file download event from a piece of data while another filter can extract a DNS query event from the same data.
b) Sequential Analysis sub-module: This is a specialized sub-module that performs sequential analysis of the data based on the timestamp. It also correlates events in a session. A session is the time duration when one user is logged in. 7) Report Module: Users use the report module ( , , , , , , in Fig. 1) to generate and view reports. They request reports via the frontend client ( , ). The API ( ) stores the requests in the cache data lake ( ). The report server ( ) handles those requests by getting relevant data from the archive DB ( ) and aggregating them into reports. It then stores the reports in the report DB ( ). Users can access the reports on demand.

IV. DATASET A. Data Source -Cowrie Honeypot
Cowrie [15] is a medium to high interaction SSH and Telnet honeypot designed to log brute force attacks and the shell interaction performed by the attacker. In medium interaction mode (shell) it emulates a UNIX system in Python, in high interaction mode (proxy) it functions as an SSH and telnet proxy to observe attacker behavior to another system. In this work we use Cowrie in the medium interaction mode.
In our setup, cowrie only allows SSH logins into our honeypot. An attacker can login to the system using any username and password combination. Cowrie logs all the interactions including the source IP address of the attacker, the SSH parameters, the downloaded files and the commands input while the attacker is logged in. Cowrie structures collected data into events. Fig. 2 shows the attributes of a Cowrie event. Here • eventid: Describes the type of the data, in the Fig. 2, cowrie.command.input means this event contains a command input by the attacker in the shell. • timestamp: The time when the event was recorded by the honeypot. • msg: The log message generated by Cowrie. • src ip: Source IP of the attacker.

B. Cowrie Data as Events
• session: Cowrie maintains a session id to group events logged during one login session by an attacker. • sensor: Host-name of the honeypot server. • command: This field is present only for eventid cowrie.command.input and contains the exact command input by the attacker. Cowrie generates about 20 different 'eventid's. Many of these events are related to the SSH session, key exchange and data logging and do not carry valuable data. For this work, we are primarily interested in the 'eventid' cowrie.command.input, which stores the exact command input by an attacker into the shell.

C. Data Statistics
For this work we have collected more than 18 million Cowrie events, spanning about 700 thousand sessions, from 27 August 2019 to 8 September 2019. Although we have 4 honeypot installations our initial analysis shows that all of them faced the same type of attack. The datasets in all the honeypots were dominated by the Mirai [18] botnet. This is because Cowrie honeypot emulates an IoT device and during 2019 Mirai was dominating IoT devices.
We have grouped all the events from all the honeypots together in one dataset because they were generated by the same malware (Mirai). Our initial analysis showed that there was no qualitative difference between the data generated from the different honeypots. So we have not done any comparative analysis in this work, rather analyzed all the data together.
V. ANALYSIS METHODOLOGY As mentioned in section III, We have used CYBEX-P [24] to automate the entire procedure of data collection to data analysis for this work. In other words we have used CYBEX-P infrastructure as a service (IaaS) here. This work is closely coupled with the development of CYBEX-P.
Along with CYBEX-P, we have further developed TAHOE a graph-based cyberthreat language (CTL). TAHOE offers several advantages over traditional CTLs. Firstly, TAHOE can store all types of structured data. Secondly, queries in TAHOE format are faster than in other CTLs. Finally, TAHOE scales well for analyzing data -a major limitation of other CTLs. Furthermore, it structures threat data as JavaScript Object Notation (JSON) which is very versatile compared to SQL.
1) Data Generation: Each Cowrie honeypot ( in Fig.  1) simulates a generic IoT device. They generate data in the format of Fig. 2. The honeypots log these data pieces into a file in respective server. We call each such log message a raw document.
2) Data Input: Each of our honeypot installations have a connector agent ( in Fig. 1). The connector is basically a script that reads the raw data from log files and sends them to the CYBEX-P collector ( ) via a websocket. The data in transport are encrypted via TLS.
3) Data Collection: The collector then posts the data to the API ( ). The API encrypts the data with the public key of the archive cluster ( ) and stores the encrypted data in the cache data lake ( ).  4) Data Archiving: The archive cluster ( ), then pulls the data from the cache data lake ( ), decrypts the data using its private key, converts the cowrie events into TAHOE raw format and stores them in the archive database ( ). TAHOE raw basically puts a wrapper around the Cowrie event. Fig.  3 shows the structure of a TAHOE raw document.
5) Data Analytics: The analytics cluster ( ) parses the TAHOE raw data into TAHOE events and sessions. It reads the raw data from the archive database ( ), processes the data and writes the results back in the archive database. Fig. 4 shows structure of a TAHOE event.
6) Command Sequence: A TAHOE session is a special document that connects related events. Note that the event itself contains no information about any other event. However, a session contains references to all events that were generated during one Cowrie login session. Fig. 5 highlights how a TAHOE session groups related command events. Here, events 1, 2 and 3 were all recorded during one login session of an attacker. TAHOE maintains this relationship by connecting them to the same session node.
Also, Fig. 4 shows that a TAHOE event has a field called timestamp that stores the exact time when the command was input into the shell. Using these information we can sort the events in a session by their timestamps to get a sequence of commands input by an attacker. Table I shows the 5 most popular command sequences along with their frequency in our dataset.

VI. EXPERIMENTAL VERIFICATION
To verify that our system works, we have done predictive analysis on command chains. Specifically, we have used Levenshtein distance [23] to predict the next command input by a bot. The high prediction accuracy of our model, as shown in subsection VII shows that botnet behavior is predictable and our system can be used to analyze that.

A. Training a Model
As explained in subsection V-6 we form chains out of commands input into the Cowrie shell. An example of a command chain is shell -/mnt/.ptmx -systemchmod -/boot/.ptmx -cat. Here, the attacker inputs 6 commands, in this particular order, before logging out. Let's replace these commands with integers in this example. Then the chain becomes 1-2-3-4-5-6.
We have split the chains into sub-chains for two reasons -(1) to obtain more data out of our dataset, (2) because our initial observation revealed that sub-chains of lengths 3 to 11 are common in many command chains in our dataset. However, this is purely a design choice and can be easily modified later.
We have then created a model out of these command-chains by storing them in a nested hash table. The keys are the subchains except the last command. The corresponding value is a second dictionary. The keys of the second dictionary are the final command and values are the frequency of the entire sequence. Fig. 6 shows the training methodology for subchains of length 5. This can easily be extended for other subchains of lengths 3 to 11. The methodology is explained below: 1) Fetch all sessions from the database.
2) For each session, fetch all of its related events.
3) Select only events with sub type='shell command'. 4) Store the events in an array. 5) Sort the array by the timestamp of the events. 6) Assign an unique integer to each command. This array contains a sub-chain of a certain length (5 in this case). 7) Feed the array into the edit distance training model. The model stores each sub-chain, the next command input by the attacker and the frequency of this sub-chain in our dataset. After doing the above operations, we end up with a number of arrays, each containing 5 commands. One such array represents one sub-chain of commands input by the attacker. We also store their frequency or the number of times this subchain is seen in our dataset.

B. Why Levenshtein distance?
Levenshtein distance is a type of edit distance which quantifies how dissimilar two sequences (or strings) are. This is an industry standard technique to compare two sequences when one sequence can be obtained from the other by insertion, deletion or substitution of certain elements. There are other specialized types of edit distances like Longest common subsequence [26] which only considers insertion and deletion but not substitution. Another example is Hamming distance [27] which only considers substitution, so it works for sequences of same length only. However in our case the two sequences can be totally different in content and length requiring the use of Levenshtein distance.

C. Testing our Model
We have split our total dataset into training and testing sets in a ratio of 80%/20%. For testing, we have used normalized Levenshtein distance [23]. Levenshtein distance is a metric of similarity between two strings. We use this to detect the similarity between two chain of commands. It is a type of edit distance. A normalized Levensthein distance is calculated by dividing the total Levenshtein distance by the total length of both the sequences. Now, to test a command chain, we perform the same operations as Fig. 6. At the end of each loop we get an array of 5 commands. The first 4 commands form a sub-chain whereas the 5 th command is the actual next command. We then calculate the edit distance of the sub-chain with all the sub-chains in our model. We select the sub-chain with the least edit distance as the match. If there are multiple matches we consider the one that has the highest frequency. A hash table lookup of the matched sub-chain gives us the predicted next command from our model. We can then test the accuracy of our system by comparing our predicted next command with the actual next command.
VII. RESULT  Table II shows the accuracy of the predictions of our training model. This same result is plotted in Fig. 7. The result shows that, even after the attacker inputs only 3 commands into the system, our methodology can predict with great certainty what the next command input by the attacker will be. This means that predictive analysis of attacker behavior works in this dataset. As the attacker keeps inputting more commands into the system, then length of the sequence increases and so does the prediction accuracy of our system. Table II and Fig. 7 also compares the accuracy of our model with Long Short Term Memory (LSTM). As seen our model (Levenshtein distance) has much better accuracy than LSTM. Furthermore, Table II shows the superior performance of our methodology compared to LSTM. The training time of LSTM is much slower than our model and is not practical for real systems. Fig. 8 compares the time complexity of LSTM with that of Levenshtein distance. It shows that edit distance is much faster than LSTM during both training and testing phase. Fig. 9 shows the time complexity of our training and testing procedures. Note that, the x-axis of Fig. 9 is sample size not sub-chain length. The result shows that the process is linear and thus can be scaled horizontally. This is important for a realtime system that collects data from thousands of hosts. As the whole process from data collection to behavioral analysis scales linearly, we can use clustering or map-reduce to support virtually infinite hosts.

VIII. CONCLUSION AND FUTURE WORK
In this work we have outlined a robust framework for automated analysis of botnet behavior. To do this, We have used a novel approach of analyzing chain of commands input by attackers into the shell. We have also incorporated a sequential analysis to verify that our approach can correctly predict attacker behavior. A prediction accuracy of 94 − 99% proves the validity of our approach.
In future we will look into real-time classification by using both benign and malicious datasets. We would then try to make the methodology more versatile by utilizing dataset generated by other malware not only Mirai. We will also cross validate our procedure with a newer dataset collected over a different time span.