{'fraction_non_alphanumeric': 0.024751847940865893, 'fraction_numerical': 0.005491024287222809, 'mean_word_length': 4.843040473840079, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Authorship Identification of Source Code Segments Written by Multiple Authors Using Stacking Ensemble Method

Source code segment authorship identification is the task of identifying the author of a source code segment through supervised learning. It has vast importance in plagiarism detection, digital forensics, and several other law enforcement issues. However, when a source code segment is written by multiple authors, typical author identification methods no longer work. Here, an author identification technique, capable of predicting the authorship of source code segments, even in the case of multiple authors, has been proposed which uses a stacking ensemble classifier. This proposed technique is built upon several deep neural networks, random forests and support vector machine classifiers. It has been shown that for identifying the author group, a single classification technique is no longer sufficient and using a deep neural network-based stacking ensemble method can enhance the accuracy significantly. The performance of the proposed technique has been compared with some existing methods which only deal with the source code segments written precisely by a single author. Despite the harder task of authorship identification for source code segments written by multiple authors, our proposed technique has achieved promising results evidenced by the identification accuracy, compared to the related works which only deal with code segments written by a single author.


I. INTRODUCTION
Source code segment author identification is a major research topic in the field of software forensics. It has many uses such as plagiarism detection, law enforcement, copyright infringement etc. [8], [12]. Frantzeskou [7] mentioned that source code author identification is useful against cyber attacks in the form of viruses, trojan horses, logic bombs, fraud, credit card cloning, and authorship disputes or proof of authorship in court. There are certain patterns that developers sub-consciously reflect in their codes based on their particular coding style while still following the guidelines, standards, rules, and grammars of a language or framework [12]. These pieces of information can be used to identify the author of the source code segment.
In recent years, open source software development has entered a new era. A lot of big companies like Google, Microsoft, and many others are maintaining their projects open source.
Alongside, small and mid-level projects are being written by a group of authors. In these cases, trivial author identification schemes no longer work. When someone contributes to an open source project, the writing style of the original author of the source code segment is no longer unique and it makes the author identification task harder. Even worse case is when a project is equally contributed by a number of authors. The writing style is then the aggregation of all the authors. We aimed to solve this problem and proposed an approach to identify the author of a source code even when it is contributed by more than one author. In this paper, we have proposed an author identification technique using a stacking ensemble method composed of several deep neural networks(DNN), random forests and support vector machines(SVM).

A. Problem Definition
Authorship identification is the task of having some samples of code for several programmers and determining the likelihood of a new piece of code having been written by each programmer [9]. As the name suggests, authorship identification of source code segments written by multiple authors is identifying the author-label when the number of authors of source code segment is more than one. This can be of two types. One is the source code segment can be written by mostly one author and have small contributions from several other authors. Another is a source code segment can be directly written by a group of authors and have a roughly equal contribution from each of them. Both of these happen in opensource software and projects which are very popular nowadays.

B. Motivation
Authorship identification of source code segment has a vast application area including plagiarism detection, authorship dispute, software forensics, malicious code tracking, criminal prosecution, software intellectual property infringement, corporate litigation, and software maintenance [8], [13], [16], [19]- [21]. In the case of authorship dispute, authorship identification can be a solution. Given the source code segment and the candidate owners, the likelihood of each candidate of being the author of the source code can be determined [12]. Again, Kothari [12] identified that author identification is useful for detecting the author of the malicious code. Software companies can also use authorship identification system to keep track of programs and modules for better maintenance [21]. Though source code segments are much more restrictive and formal than spoken or written language, they inhibit a large degree of flexibility [7]. According to Shevertalov [18], using differences in the way programmers express their idea, their programming style can be captured. This programming style, in turn, can be used for author identification. Although, a large number of works already done regarding source code segment author identification, according to Frantzeskou [8], the future of source code segment author identification is in collaborative projects to which we aimed at.
The remaining sections are organized as follows. Section II contains a briefing on background topics regarding this work. Section III contains a summary of the related works. In section IV, we discuss our author identification technique for multiple authors. In section V, the experimental results of our proposed technique are analyzed and compared with that of some related works. Finally, in section VI, the conclusion is stated with possible future direction of this work.

A. Ensemble Method
By combining several methods, ensembling method helps to improve the results of machine learning. An ensemble is often more accurate than any of the single classifiers in the ensemble. According to Maclin [14], an ensemble consists of a set of individually trained classifiers whose predictions are combined, while classifying instances, by the ensemble method. These meta-algorithm combines several machine learning techniques into one predictive model. In our work, we used stacking ensemble in order to improve our prediction performance.

B. Random Forests
Random forest is an ensemble learning method where each classifier in the ensemble is a decision tree classifier. This collection of classifiers is called a forest. During classification, each of the decision trees gives its vote and the result is based on the majority of the votes.

III. RELATED WORKS
Numerous works are available on source code segment author identification using a variety of features and classifiers. However, very few of them use machine learning techniques to identify the author of source code segments.
According toĎuračík [6], there are several approaches to identify the author of source code segment. The first one is text-based and uses plain text as an input. The second level is token or metric based.

A. Text Based Approaches
The first approach, which treats source code segment as plain text, is a form of natural language processing. This approach cannot make use of the programmatic structure of source code segment.
Frantzeskou et al. [8] proposed a technique called Source Code Author Profiles(SCAP) for author identification. They generated byte level n-gram author profile and compared with previously calculated author profiles. Burrows [4] mentioned, the SCAP method truncates the author profiles that are greater than the maximum profile length causing a bias towards the truncated profiles.
Burrows et al. [3] proposed an approach using information retrieval. They generated n-gram tokens from the source code segments and indexed them in a search engine to query the author of source code and return a ranking list of authors which matched the n-gram token of the source code segment with 67% accuracy.

B. Metric Based Approaches
Frantzeskou [8] pointed out that metric-based author identification is divided into two steps. The first step is extracting the code metrics that represent the author's style and the second part is using those metrics to generate a model that is capable of labeling a source code segment by corresponding author name. However, a large amount of time is required to gather all possible metrics and examine to choose only the metrics responsible for differing the authors' style.
Lange and Spiros [13] assumed that the code metrics histogram should vary from author to author as of their coding style. From a number of source code metrics, an optimum set was selected using genetic algorithms(GA) and used as input for the nearest neighbor(NN) classifier. This method achieved 55% accuracy. According to Yang [20], some of the features of this paper are unbounded. For example, the indentation category.
Shevertalov el al. [18] proposed a technique based on GA. The metrics are extracted from the source code segment to make a histogram which is sampled using GA. The author profile is produced using categorized histogram samples. For files, they achieved 54% accuracy and for projects, they achieved 75% accuracy. Yang [20] mentioned that the details of the final feature set are not mentioned in this paper. So, the feature set is non-reproducible.
Bandara and Wijayarathna [1] used the deep Neural Network for source code segment author identification. The converted source code metrics they used to feed a neural network are identical to that of Lange et al. [13]. Their deep neural network consisted of three restricted Boltzmann machine (RBM) layers and one output layer. They achieved 93% accuracy.
Zhang et al. [21] used SVM to identify the author of source code segment. They categorized their feature into four groups namely -programming layout feature, programming style feature, programming structure feature and programming logic feature. They used sequential minimal optimization(SMO) as the classifier for SVM and achieved 98% and 80% accuracy for two different datasets.

IV. AUTHOR IDENTIFICATION OF SOURCE CODES
WRITTEN BY MULTIPLE AUTHORS Our developed author identification approach consists of four phases. Firstly, source code metrics are extracted from the source code segments in the training set. These extracted metrics are then converted to feature vectors.
Secondly, these feature vectors are fed to five individual base classifiers along with corresponding class-labels to train the author signatures to the base classifiers. In the case of open source contribution, class-label means the owner of the source code segment and in the case of a group of authors, the whole group is considered as the class label. By author signature, the coding style of a particular class-label is meant. Caruana [5] showed that, in general, for classification problem, random forest, DNN, decision tree, and SVM are the top four algorithms. Hence, our chosen classifiers are DNN, random forest with CART decision trees [2], random forest with C4.5 decision trees [10], C-SVM and ν-SVM.
Thirdly, each of the classifiers generates the posterior classprobability according to their predictions. These outputs are called meta-features. Meta-features are used as the input for a meta-classifier. Then the meta-classifier is trained based on the meta-features and output. This approach is known as stacking ensemble. Another deep neural network is used as the metaclassifier. Figure 1 shows a block diagram of the architecture of the stacking ensemble method we have designed. Finally, to identify the author of a new source code segment, that is from the test set, the same metrics are extracted from the test source code segments and are converted to feature vectors. These feature vectors are fed to the meta-classifier via the base classifiers. Using the experience from the training, the metaclassifier along with the base classifiers predict the class labels of the test source code segments. Figure 2 shows the block diagram of the proposed approach for author identification of source codes written by multiple authors.
In the following sub-sections, the building blocks of the author identification approach are described. Some careful considerations are needed while choosing the dataset. Data must be collected from a diverse population of programmers and should provide enough information about the authors so that a clear distinction can be computed from author to author and valid comparison of their programming style can be made. In addition, the dataset must be close to real-world data as well as open for academic study [13].
In our study, we have generated our dataset based on open source codes from github.com. All the source codes have a permissive license like MIT or BSD. The dataset contains 6063 python source code segments from 8 authors/ author groups which are considered as individual classes. Each source code segment contains roughly 226 lines on average. Source code segments of each author are roughly split into 2:1 ratio to make the training and testing set.
Each class label consists of authors and contributors. By author, we mean the true owner of the projects. This could be a single author or a group of authors. By contributors, we mean a group of people who are not the owner of the project but willingly contribute to the project by writing or editing a segment of it. The number of authors and the number of contributors per class-label is listed in table I.

B. Metric Extraction
Previously, Shevertalov, Lange, Bandara, and Zhang [1], [13], [18], [21] used source code metrics for author identification. From a set of probable code metrics, Lange selected the optimal set of code metrics using the genetic algorithm. Bandara used almost the same set of source code metrics. We have used the same set of metrics for our author identification approach only except the access modifier metric. The access modifier feature is present only in a limited number of programming languages and makes the whole system language dependent. Table II shows the set of metrics to be used and corresponding descriptions.

Metric Name Metric Description
Line Length This metric measures the number of characters in one source code line.

Line Words
This metric measures the number of words in one source code line.

Comments Frequency
This metric calculates the relative frequency of line comment, block comment and optionally doccomment used by the programmers.

Identifiers Length
This metric calculates the length of each identifier of programs.

Inline Space-Tab
This metric calculates the whitespaces that occur on the interior areas of non-whitespace lines.

Trail Space-Tab
This metric measures the whitespace and tab occurrence at the end of each non-whitespace line.

Space-Tab
This metric calculates the indentation whitespaces used at the beginning of each non-whitespace line.

Underscores
This metric measures the number of underscore characters used in identifiers.
After extracting the metrics, we have counted the number of occurrences for each possible values for each of the metrics. For example, for underscore metrics, we have counted the number of words with no underscore, one underscore, two underscores etc. These counts have been fed to the base classifiers.

C. Base Classifiers
There are total five base classifiers in our author identification system. They are -DNN, random forest based on CART, random forest based on C4.5, C-SVM and ν-SVM. Each of the base classifiers is described below: 1) Deep Neural Network: The DNN model used as the base classifier consists of 14 layers. Data are fed to the DNN as batches of 32 entries. They are one input layer, followed by eight fully connected layers, a dropout layer, a fully connected layer, a dropout layer, a fully connected layer and finally the output layer.
In the fully connected layers, ReLU activation function and in the output layer softmax activation function are used. Categorical cross-entropy is chosen as the loss function. Adam [11] optimizer is used to optimize the network.
2) Random Forest: The second base classifier is a random forest with one hundred decision trees. Classification and Regression Tree(CART) [2] algorithm is used to build the trees which selects the split node based on Gini impurity.
The third base classifier is another random forest with one hundred decision trees. Decision trees in the third base classifier are built with the C4.5 [10] algorithm. This algorithm chooses the split node based on the entropy ratio.
3) Support Vector Machine: The fourth base classifier is a C-support vector classifier. It is a support vector machine where C is a penalty parameter for the error term.
The fifth base classifier is a ν-support vector classifier. It is a support vector machine where ν is the upper bound of training error and the lower bound of the number of support vectors.

D. Meta Classifier
We have used another deep neural network as the metaclassifier. The outputs of the base classifiers (meta-features) are fed to the meta-classifier to learn the mapping from the meta-features to the actual output.
The neural network consists of 19 layers. They are one input layer, followed by eight fully connected layers, a dropout layer, two fully connected layers, a dropout layer, a fully connected layer, a dropout layer, a fully connected layer, a dropout layer, a fully connected layer, a dropout layer, and finally the output layer. The output from this output layer is the final output of our author identification system for source code segment written by multiple authors.
The activation functions of the network are ReLU for fully connected layers and softmax for the output layer. The loss function used in the meta-classifier is categorical crossentropy. Stochastic Gradient Descent(SGD) is used as the optimizer of the meta-classifier.

E. Training
We have implemented our author identification system for source code segment written by multiple authors in multi-class classification category. Here, a unique list of authors(or groups of authors) of the source code segments in the training set is treated as classes. The author identification system produces its confidence for each class of being the actual class of given Roughly, 67% source code segments from each class formed the training dataset and rest are used for testing. The training set contains 4034 files and the test set contains 2039 source code segments.
The training stage of our system is divided into three phases -feature extraction from the source code segments, training the base classifiers and training the meta-classifier. Figure 3 shows the steps followed in our author identification system for source code segment written by multiple authors.
First of all, the source code metrics mentioned in table II are extracted from source code segments. Then the extracted metrics are converted to feature vectors as mentioned in section IV-B. These feature vectors are fed to each of the base classifiers as input.
The base classifiers run according to their own learning algorithm to learn to identify the writing style of each class. During this training phase, several configurations of each of the base classifiers, specially DNN, are used to find out which configuration works best for the training set.
After completing the training of each of the base models, the posterior probability for each input in the training set is generated. This produced a 5 × |classes| sized feature vector for each of the input feature vectors where |classes| is the number of classes. These feature vectors are known as metafeatures. Meta features are fed to the meta-classifier along with the class labels through which the meta-classifier learned to predict the actual class from the meta-features.

A. Experimental Setup
While implementing our author identification system for source code segment written by multiple contributors, we have used keras as the framework for deep neural networks and Scikit Learn [17] as the library for general purpose machine learning. For data pre-processing and visualization, we have used numpy and pandas [15] library. We have developed a feature extractor that extracts the features mentioned in table II from the source codes.
For C-SVM, the parameter C is a penalty for the error term. For ν-SVM, the parameter ν is an upper bound to the training error and lower bound to the number of support vectors. During the experiment, we found that for both the random forests, a hundred trees were sufficient to converge to the highest accuracy. After numerous iterations, we reached to a decision that the set of values stated in table III classifies the source code segments most accurately.
Accuracy and f1-score were used to evaluate the accuracy of our method. Accuracy is the ratio between the number of correctly identified samples and the number of total samples. F1-score is the harmonic mean of precision and recall. Micro averaging was used to compute the f1-score.

B. Results of The Base Classifiers
Table IV contains the accuracies for the five base models of our stacking ensemble method.

C. Results of The Meta Classifier
After training the meta-classifier by the meta-features, we have achieved 87% accuracy with f1-score 0.86. Identifying the authorship of source codes is more difficult when the number of authors is more than one as the writing style of the source code is then inconsistent from segment to segment. Table V shows a comparison between the type of features, language independence, the capability of handling multiple authorship, number of classes and the total number of source code segments used in training and testing. From that table, we can see that even after dealing with source code segments written by multiple authors, our method has achieved an accuracy that is pretty close to that of the methods that deal with single authors. Our chosen set of metrics is compact and is still able to achieve a satisfactory accuracy. Alongside a number of works suffer from choosing a set of metrics that are not language independent. So, the main contribution of this work is the identification of multiple authors using language independent set of metrics.

VI. CONCLUSION
Here, we have proposed a new approach for identifying the author of source code segment where the number of authors of the source code segment is more than one. The main challenge of this work is to select the base estimators from a large number of possible combinations. Again, as several classifiers need to be trained, each classifier needs to be finetuned individually to produce a good final result. On the other hand, the problem of identifying the authorship of source code segments is harder when the number of authors is more than one.
We have developed a stacking ensemble classifier that consists of five base classifiers and a meta-classifier which uses a relatively small set of code metrics that are relatively easy to compute and language independent as well.
In spite of the fact that our stacking ensemble method achieved a satisfactory accuracy, this still can be improved. Even though our code metrics are language independent, we only tested with python source code segments. Future works may test on other languages and check how the set of metrics works for other languages. Other sets of metrics can also be examined to see how they contribute to the writing style of source code segments.