We participated in the PR-SOCO task hosted in FIRE 2016 and tried some basic approaches which we look to improve in the future. We de ned some simple features from the source code which, in our opinion, were indicative of the manner in which the code was written and which might give some clues about the personality of the programmer. We built a multiple linear regression model from the training data and applied this model on the test data. The results show that our method produces good prediction performances for Neuroticism, Extroversion and Openness.
Much work has been done on predicting user
personality based on text written in a natural language (e.g,
Facebook status updates [
The rest of the paper is arranged as follows: We describe the proposed methodology in Section 2. We present the results in Section 3. We conclude in Section 4. 2. 2.1
We used four features (explanatory variables) for multiple linear regression. Here each of the BIG5 traits is the dependent variable. The feature values were extracted from the source code of each program le. The features are as follows (examples are shown in Table 1): 1. Multi-line comments (MLC): This is the number of genuine comment words in multi-line comments, i.e., between /* and */ found in the program code. In Table 1, we see a case of genuine comment under Positive example. We have not considered the cases where lines of code were commented, as shown under Negative example. To extract this feature from a source code le, 1https://en.wikipedia.org/wiki/Neuroticism as seen on 26th October, 2016 2https://en.wikipedia.org/wiki/Extraversion and introversion as seen on 26th October, 2016 3https://en.wikipedia.org/wiki/Linear regression as seen on 26th October, 2016 we rst read the lines within /* and */. Then we eliminated any instances of program code by searching for a regular expression containing ;= as symbols and functions of the form [a-zA-Z][a-zA-Z]*[ ]*( (e.g., System.out.println(\Even");) used in a Java code. This feature value was normalized by dividing it by the total number of words in the program le. 2. Single-line comments (SLC): This is the number of genuine single-line comment words in single line comments, i.e., comments following \//" (as shown in Table 1, under Positive example). Here also, we have not considered the cases where lines of code were commented (as shown in Table 1, under Negative example). To extract this feature value, we simply determined the number of words following \//" in the code. Then we eliminated the occurrences of program code by the procedure used for the feature MLC. This feature value was normalized by dividing it by the total number of words in the program le. 3. Non-existent spaces (NES): This is the number of lines containing non-existent spaces, e.g., i=1; i<=casos; as shown in Table 1, under Negative example as opposed to i = 1; i< = casos; as shown in Table 1, under Positive example. We have considered this feature since the presence of spaces is supposed to be a good programming practice. This feature was extracted by identifying the lines of code satisfying the regular expression [a-z][a-z]* [a-z][a-z]*[=<>+] (e.g., int i=1). This feature value was normalized by dividing it by the total number of lines in the program le. 4. Import Speci c (IS): This is the number of instances where the programmer exported the speci c libraries only (e.g., cases of import java.io.FileNotFoundException as opposed to import java.io.*). These examples are also shown in Table 1. We have considered this feature as this is supposed to be a good programming practice to use speci c libraries which reduce compilation time. In addition, the choice of speci c libraries may indicate the experience and pro ciency in programming. This is because a good programmer is supposed to know the speci c libraries which will be useful. On the other hand, an inexperienced programmer is more likely to \import" all the libraries to somehow get the job done. This feature was extracted by considering all the instances of \import" not ending with a \*". This feature value also was normalized with respect to the total number of lines in the program le. 2.2
For each BIG5 trait, we de ne a multiple linear regression model4 for a program code p, given as follows: scoreBIG5(p) =
+ 1M LC(p) + 2SLC(p) + 3N ES(p) + 4IS(p) (1)
We calculate the values of parameters and i, i = 1, 2, 3, 4 from the training data using the linear regression imple4https://en.wikipedia.org/wiki/Linear regression as seen on 11th October, 2016 mentation in R.5 Here, scoreBIG5 is the dependent variable and MLC, SLC, NES and IS are the explanatory variables. 3.
We submitted two runs as follows: 1. Run1.txt: The values of the dependent variables were generated on the test data using the regression equation (1) learned from the training data. 2. Run2.txt: For this run, for each BIG5 trait, we calculated the values of the dependent variables given by the linear regression equation (1) on the training set. We then calculated the error between the predicted value and the actual value for each of the 49 training samples. We removed the samples in the training set with the three highest error values. We then trained the linear regression on the new training set and calculated the regression coefficients. Finally, values of the dependent variables were calculated on the test data. The purpose of this run is to remove some outliers from the training set.
The performances of these two runs are shown in Tables 2 and 3. Table 2 reports the results in terms of RMSE. The table also reports two official baselines (bow and mean) and the best results reported among all the submitted runs (Reported best ).6 In RMSE, our run Run1.txt produced the best performance for Extroversion. This run also produced good performances for Neuroticism and Openness when compared with the baselines.
Table 3 reports the results in terms of Pearson ProductMoment Correlation (PC). The table also reports two official baselines (bow and mean) and the best results reported among all the submitted runs (Reported best ). In PC, our run Run1.txt produced the best performance for Neuroticism. This run produced good performances also for Extroversion and Openness when compared with the baselines.
Table 4 shows the regression coefficient values learned from the training data for each BIG5 trait, used for Run1. Since our predictions for Neuroticism, Extroversion and Openness are promising, we try to draw some inferences from Table 4 for these traits, as follows.
Neuroticism: The negative value of high magnitude of 2 indicates that a person who frequently provides Single Line Comments (SLC) in her code is likely to exhibit a low level of Neuroticism. This agrees with our intuition that a Neurotic person is not organized in her coding. However a positive value (though of relatively lower magnitude) of 1 indicates that a person who provides Multi Line Comments (MLC) is likely to have a high level of Neuroticism. Also, a negative value of 3 indicates that a person who does not provide necessary spaces in the code is likely to have a low level of Neuroticism. These two coefficient values somewhat contradict our intuition that a Neurotic person is necessarily chaotic in nature while writing a code. But negative value of high magnitude of 4 indicates that a person who tends to import libraries selectively, is likely to have a low level of Neuroticism, which again agrees with our intuition. 5https://www.r-bloggers.com/ r-tutorial-series-multiple-linear-regression/ 6These values are reported at http://www.autoritas.es/ prsoco/evaluation/
Extroversion: The positive values of 1, 2 and 4 indicates that a person who tends to provide genuine comments (both Multi Line and Single Line) and import speci c libraries in her code is likely to have high Extrovertion. But, the positive value 3 indicates that an Extrovert may not provide appropriate spaces in her code. The value of 2 is much higher than the other coefficients, which implies that a person with a tendency of providing genuine Single Line Comments is likely to have high Extrovertion.
Openness: The observations about Openness are similar to those about Extroversion.
However, the prediction results show that our features are possibly not suitable indicators for Agreeableness and Conscientiousness. 4.
We see that these simple and intuitive features yield promising prediction results for Neuroticism, Extroversion and Openness, as inferred from samples of written program code.
We gain some interesting insights into the relationship of these three traits with these features. For example, Neuroticism has a strong negative correlation with the tendency of writing genuine Single Line Comments, Extroversion has a strong (positive) correlation with the tendency of writing genuine Single Line Comments etc. We look to explore other features in future. However, these features are not adequate for predicting Agreeableness and Conscientiousness. 5. SLC NES IS
Positive example Negative example
/** /*System.out.println(\Even"); * Make the hash table logically empty. printQ(qEven); */ System.out.println(\Odd"); printQ(qOdd);*/ // Create a new double-sized, empty table //String[] ss = linea.readLine().split(\ "); for (int i=1; i<=casos; i++) for (int i = 1; i< = casos; i++) import java.io.FileNotFoundException import java.io.*
Method Run1.txt
Run2.txt Baseline (bow) Baseline (mean) Reported best