Source code authorship attribution is a process of identifying the author of a given code. With increasing number of software submissions on open source repositories like Github, Codalab, Kaggle, Codeforces online judge, etc. the authors can copy other's code for their products. The application area of this method includes the detection of plagiarized code and prevent legal issues. In this work, we have applied the tf-idf based mechanism for representing the given source code. Word and character n-grams are used for the generation of code vectors. Finally, the generated vectors are fed to diferent available machine learning classifiers for the prediction task. We have applied this methodology to the dataset released by organizers of the AI-SOCO, a shared task of FIRE-2020. The problem statement for the task is "Given the predefined set of source code and their authors, the task is to build a system to determine which one of these authors wrote a given unseen before source code." An accuracy of 82.95% is achieved on the test data, which attained 10th position in the competition.
Nowadays, the identification of the author of a piece of code has become very crucial in many
cases. Some of the cases are authorship dispute, malware attacks, logic bomb, proof of authorship,
frauds, etc. [
In this section, we have discussed the basic concepts of abstract syntax tree, and the stacked classifier.
An abstract syntax tree (AST) is a representation of a program’s code in the form of a rooted
tree. Nodes of an AST correspond to diferent code constructs (e.g., math operations and
variable declarations). Children of a node correspond to smaller constructs that comprise its
corresponding code. Diferent constructs are represented with diferent node types [
Stacked Generalization is an ensemble machine learning algorithm, where diferent machine learning algorithms are ensembled together to get the results. Stacked Model uses a metalearning algorithm to learn how to combine the predictions from two or more machine learning algorithms. The benefit of stacking is that better results are obtained by combining the outputs of multiple classifiers.
In stacking, diferent base machine learning classifiers are combined together with the help of a meta-classifier. The individual classification models are trained based on the complete training set. Then the outputs for these classifiers are fed as inputs to the meta-classifier where it predicts the final result. The meta-classifier can either be trained on the predicted class labels or probabilities from the ensemble.
In this section, we have discussed the data, its pre-processing, vectorization, and the classification. 3.1. Data The dataset is composed of source code collected from the open submissions in the Codeforces online judge. There are 100 source code of 1,000 users, a total of 100,000 codes. The data is divided into three parts, training, development, and test set having 50000, 25000, and 25000 codes of C++ programming language, respectively. All codes are correct, bug-free, compile-ready, and for a unique problem.
The data is preprocessed using the clang library for the generation of Abstract syntax tree for the source code. The writing style and syntax of code are diferent from the natural texts, thus the normal tokenization process should not be usual for splitting codes. We have used AST for the above purpose. Firstly, the language of the source code is found through the GuessLang library of python. All the codes are written in C or C++, thus we have used the clang library to generate the abstract syntax tree. AST traversal provides the list of tokens for the codes. 3.3. Vectorization tf-idf Vectorizer is used for generating the vector representation of the given source code. The list of tokens are extracted from the AST. Word and character n-grams are generated from the token list. While building the vocabulary, we have ignored the terms having document frequency strictly higher than 0.6 and strictly lower than 0.0. In this way, the code vectors are calculated using the traditional tf-idf mechanism. We have used word uni-grams, bi-grams, tri-grams, and character bi-grams, tri-grams, and quad-grams for the experimental purpose.
The representation vectors for the given codes are fed to diferent available machine learning classifiers. We have also used a stacked model for classification purpose. A stacking of three diferent classifiers i) extra tree classifier, ii) random forest, and iii) XG-Boost Classifier is carried out. The input is first classified using the three diferent classifiers, and then the outputs produced by the classifiers are fed to SVM for final prediction. Thus, SVM is used as the meta classifier over these classifiers. This architecture is shown in Figure 1. 150 estimators are used with the extra tree classifier and the random forest classifier. SVM classifier takes the output of all the three classifiers and then predicts the author for the input given. Stacking Classifier from the mlxtend library is used to stack the models.
In this section, we have discussed the results achieved on the development data, using diferent approaches. An accuracy of 75.1% is achieved by random forest, using the word uni-gram based vectors. The performance for the unigrams is shown in Table 1. The accuracy value has increased with the use of uni-gram and bi-grams, shown in Table 2. An accuracy of 83.20% is achieved using the stacked model for the word n-grams (n=1, 2). A decrease in performance is recorded with the vectors created from the combinations of word uni-grams, bi-grams, and tri-grams, as shown in Table 3. Character n-grams are also used for generating the feature representations. Word-based features outperform the character-based features. The experimental results are shown in Table 4. Thus, it is concluded that the word bi-gram-based representation with stacked model performs the best on the development dataset.
Accuracy values of 81.58% and 82.95% are achieved using the bi-grams-based representations by random forest and the stacked model, respectively. The results for the test data are shown in table 5.
Source code authorship identification is a process of identifying, who wrote a piece of code, given some set of probable authors. In the current work, we have used a tf-idf mechanism for generating the feature vector for the codes. The abstract syntax tree is used for tokenizing the codes and splitting it into tokens. Experimental results illustrate that word bi-grams perform better than other combinations of word and character n-grams.
The sources for the implementation are available via • Clang • Tf-idf. • Source code link.