Optimizing Authorship Profiling of Online Messages Adeola O. Opesade Africa Regional Centre for Information Science, University of Ibadan, Nigeria morecrown@gmail.com ABSTRACT group of the author of an anonymous text. Its application in Authorship profiling is of growing importance in the current forensics and digital security has made it to be of growing information age, partly due to its application in digital forensics. importance in the present information age. Methodologies of Methodologies of profiling like any other authorship analysis profiling like any other authorship analysis consist majorly of consist majorly of feature extraction and application of analytical feature extraction and application of analytical techniques. Choice techniques. Choice of feature sets and analytical techniques may of feature sets and analytical techniques may significantly affect significantly affect the performance of authorship analysis. Hence, the performance of authorship analysis [1]; thus, studies into a need for methods that can help improve on the success of optimization of authorship profiling of online messages can assist authorship profiling undertakings. The present study sought in improving the success of identifying sources of security threats through experiments, the writing features, analytical technique and perpetrated through web-based channels. number of class labels that can help improve the effectiveness of profiling the country of affiliation of authors of online messages. A number of previous studies ([1]; [22]; [3]) have investigated The experiment showed that the most effective model was some parameters that could affect the effectiveness of authorship achieved when all feature set types in our study were used within a attribution undertakings. These studies, however, focused on two-class dataset that was analysed with the Neural Network authorship identification problem and not on authorship profiling. (Multilayer Perceptron) machine learning scheme. The study Considering the potential of authorship profiling in investigating recommends a need for further studies in finding models that can transnational digital breaches, the present study seeks to find maximize both effectiveness and efficiency in profiling the through experiments the writing-style features, classification authorship of online messages. techniques as well as possible number of class options that can maximize the effectiveness of profiling the authorship of electronic messages. The following research questions were pursued in order CCS Concepts to achieve the purpose the study: • General and reference ➝ Cross-computing tools and techniques ➝Experimentation Research Question 1: Which feature type set maximizes the effectiveness of profiling the country of affiliation of writers of Keywords online messages? Authorship profiling, Machine learning, Computational linguistics, Natural Language Processing, Nigerian English Research Question 2: Which classification scheme maximizes the effectiveness of profiling the country of affiliation of writers of online messages? 1. INTRODUCTION Electronic messages are extensively used to distribute information Research Question 3: Which class labelling option maximizes the over such channels as e-mail, Internet newsgroups, Internet chat effectiveness of profiling the country of affiliation of writers of rooms, Internet forums and other user generated contents on the online messages? Web. These messages are quite different from other forms of writings particularly, because of their brevity. Unfortunately, Research Question 4: What is the performance of the resultant unethical hands and criminals exploit the convenience of these model in classifying electronic messages to writers' countries of media to carry out their obnoxious goals. Digital forensics require affiliation? the use of scientifically derived and proven methods towards the preservation, collection, validation, identification, analysis, interpretation, documentation and presentation of digital evidence 2. LITERATURE REVIEW derived from digital sources for litigation purposes. 2.1 Authorship Attribution Problems Authorship attribution is a process of examining the characteristics Authorship profiling is one of the major classes of authorship of a piece of writing in order to draw conclusions about its author. attribution problems. It seeks the demographic or psychological Authorship attribution problems vary in complexity. They have been categorized into three major classes, namely, authorship identification, authorship profiling and authorship verification. The most straightforward version of these three is the identification problem which involves the determination of the actual author of a given text among a small set of candidate authors. Given a set of writings of a number of authors, the task in authorship CoRI’16, Sept 7–9, 2016, Ibadan, Nigeria. identification is to assign a new piece of writing to one of them [4]. In authorship verification, there is no closed candidate set but there is one suspect and the challenge is to determine if the suspect is or is not the author. In this case, examples of the writing of a single 96 author are given and the task is to verify that a given target text Unlike in the choice of feature sets, researchers are less varied in was or was not written by this author. Hence, verification can be their choices of analytical techniques. While older studies tend to thought of as a one-class classification problem and it is favour the use of Principal Component Analysis, the more recent significantly more difficult than basic authorship identification ones tend towards the use of Support Vector Machine. Most problem [5]. previous studies reported the use of only a single analytical technique. Considering such statement as made by [15]. In authorship profiling (also known as authorship characterization problem) there is no candidate set at all; the challenge is to provide Experience shows that no single machine as much demographic or psychological information as possible learning scheme is appropriate to all data about the author. Unlike the identification problem, authorship mining problems. The universal learner is profiling does not begin with a set of writing samples from known an idealistic fantasy. Real datasets vary and candidate authors. Instead, it exploits the sociolinguistic to obtain accurate models, the bias of the observation that different groups of people speaking or writing in a learning algorithm must match the structure particular genre and in a particular language, use that language of the domain. Data mining is an differently; that is, they vary in how often they use certain words experimental science (pg 365). or syntactic constructions in addition to variation in pronunciation or intonation [6]. Profiling problem is concerned with determining Choice of machine learning scheme should be based on the result such characteristics as gender, educational and cultural of a prior experiment that validates its suitability to the dataset. backgrounds, language familiarity and so on of the author that produced a piece of work. This is a harder problem than the 2.3 Related Authorship Studies identification problem since it characterizes the writing style of a A number of previous studies have shown relative performances of set of writers rather than the unique style of a single person [7]. a number of feature types and analytical techniques in authorship analyses. [3] studied the results of authorship identification using Despite variations in the complexities of authorship problems, many authors and limited data on learning. Their result showed choices of appropriate linguistic features and analytical techniques that systematically increasing the amount of authors under are paramount. investigation led to a significant decrease in performance. Their study also revealed that providing a more heterogeneous set of 2.2 Authorship Attribution Methods features improves the system significantly. [1] investigated the One of the main components of authorship attribution methods is types of writing-style features and classification techniques that the extraction of linguistic features that represent the writing style were effective for identifying the authorship of online messages. of an author or author group. Language, like genetics, can be They reported that the accuracy kept increasing as more types of characterized by a very large set of potential features that may or features were used and that Support Vector Machine (SVM) may not show up in any specific sample, and that may or may not outperformed Neural Networks (NN), which in turn outperformed have obvious large-scale impact. By identifying the features the C4.5 classifier. The best accuracy was achieved when SVM characteristic of a group or individual of interest, and then finding and all feature types were used but classifier performance reduced those features in an anonymous document, one can support a as the number of authors increased. [2] through experiment finding that the document was written by that person or a member demonstrated that inclusion of stylistic idiosyncrasy features to of that group [8]. The various feature sets, otherwise known as letter n-grams, function words and to a combination of n-grams feature metrics in computational linguistics can be classified into and function words consistently led to improved accuracy for four main classes, which are the lexical, syntactical, content- identifying the native language of the author of a given English specific and structural features [9]. Researchers vary in their language text. choices of linguistic features; while some used feature(s) that belong to a single class (for example, [10]; [11]; [12]; and [9], The studies of [3] and [1] are situated within the identification others (such as [6]; [2]; [4]; [3]; [7]; [1]; [13]; [14]) used features domain of authorship attribution problems because they started across multiple feature classes. with a close number of candidate authors, while that of [2] was a profiling problem. However, their focus was majorly to show the The second component is the application of analytical techniques ability of idiosyncrasies in detecting writer's native language. It to feature sets for supervised or unsupervised learning. Different therefore, did not address some of the salient issues covered by [1] analytical techniques have been used in previous authorship which are relative performances of analytical techniques and effect attribution studies. These techniques can be classified into three, of increasing the number of candidate authors. Also, the corpus namely, the unitary invariant, multivariate and machine learning used by [2] was the International Corpus of Learner English approaches [8]. Machine learning examines previous examples and (ICLE) which had between 579 and 846 words. These numbers their outcomes and learns how to reproduce these and make were quite high for an online message, which are usually very generalisations about new cases. Machine learning algorithms short. The present study focuses on shorter texts which characterise differ in terms of level of data and abilities to resolve data online messages. Therefore, the present study seeks to find the ambiguities such as noise or missing data. Machine learning writing-style (linguistic) features, classification techniques as well techniques include rule based algorithms such as OneR, neural as possible number of class options that can maximize the networks such as Multilayer Perceptron, statistical modelling effectiveness of profiling the native language of the author of an algorithm such as Naive Bayes, decision trees such as J48, linear online message. models such as linear regression and Support Vector Machine and instance-based learning algorithm such as Nearest Neighbour. 97 3. EXPERIMENTATION FOR OPTIMIZING reliable characteristic of attribution domain [21]. Certain features were extracted in the present study, based on their relevance as AUTHORSHIP PROFILING OF ONLINE determined from relevant literature on authorship attibution and MESSAGES Nigerian Englishes ([16]; [17]). Extracted features were syntactic 3.1 Problem formulation features comprising the twenty (20) most frequent function words Given a number of online messages written in English language by in the topix.com corpus, Idiosyncratic features comprising nationals of selected African countries, namely, Cameroon, Ghana, frequency of occurrence of spelling errors, adverb-verb part of Liberia, Nigeria and Sierra-Leone. The goal is to find the types of speech (POS) bigram distribution and article omission/inclusion writing-style features, the classification technique as well as distribution. Structural features comprising lexical diversity; and possible number of class options that can maximize the content specific features consisting of twenty (20) most frequent effectiveness of profiling the linguistic origin of anonymous noun, adjective, verb and adverb unigrams in the topix.com corpus. electronic texts written by the nationals of any of the selected The features extracted and their denotations are as shown in Table countries. 2. Table 2: Extracted Linguistic Features 3.2 Research Method Feature type Feature metric Denotation A multistage sampling technique was used to select a representative sample of electronic texts from the population of Lexical Vocabulary richness F1 texts contained in the relevant country pages of the website www.topix.com. To get the texts that could be useful for a Syntactic Probabilities of occurrence of F2 supervised learning approach of the study, each text was opened, most occurring function words read and assessed based on the number of words contained and a Idiosyncrasies Probabilities of occurrence of F3 sense of affiliation to the respective country as depicted in the article deletion, verb -adverb content. A comment was considered to be affiliated to (and sequence and spelling errors. labelled to be from) a particular country if it was found in that Content Noun unigrams, adjective F4 country's forum and if it contained such phrases as 'our country', specific unigrams, verb unigrams, 'our beloved country' and other related ones in its discourse. adverb unigrams. Initially the researcher targeted selecting texts with a hundred or The decision to extract twenty most frequent features (function more words; however, this was reduced to texts with twenty (20) word, noun, adjective, verb and adverb unigrams) was as a result or more words because of the scarcity of large texts on the of a prior experiment which showed that the summation of the discussion forums. The numbers of texts selected for the study in frequencies of occurrence of the twenty most frequent features November 2011 and based on the assessment criteria are as shown accounted for at least 60% of the cumulative frequency of all in Table 1. features extracted in each case. Table 1: Training Data Set 3.3 Experimental Setup i. Class Labelling: According to the study of [3] learner’s Country's forum website No. of Pages No. of performance changes with number of candidate authors. To find pages selected selected out the effect of varying the number of classes on the texts classification performance in the present study, the dataset was www.topix.com/forum/worl 31 2,8,13,2 425 copied into three different files having all parameters being the d/nigeria 5 same except the class labels. The class labels were controlled as www.topix.com/forum/worl 9 2,3,6.9 317 presented in Table 3. d/ghana www.topix.com/forum/worl 4 1-4 130 Table 3: Dataset Class Labelling Options d/liberia File No of Class Labels Remark www.topix.com/forum/worl 4 1-4 241 Name Class d/cameroon Labels www.topix.com/forum/worl 4 1-4 357 Dataset1 5 Nigeria, Labelling according to d/sierra-leone Ghana, texts’ original classes. Total no. of Texts 1,470 Cameroon, Liberia, Sierra- 3.2.1 Text Pre-processing and Processing Leone The corpora were subjected to pre-processing in order to put them Dataset 2 3 Nigeria, Labelling informed by in the format expected by the relevant software for text processing. Ghana, Non- language similarities The pre-processing tasks included deletion of e-mail headers, Ghana-Nigeria between the selected removal of control codes, text aggregation, and removal of non- countries as found in a ASCII characters. Text processing was achieved by extracting previous study [21]. linguistic features from the sampled texts using computer codes Dataset 3 2 Nigeria, Non- Testing a 2-class written by the researcher in Python 2.6.4 programming language, Nigeria labelling scheme which based on the natural language toolkit (NLTK) version 2.0. Some of can enable the the specific issues handled in the course of text processing were identification of online tokenization, part of speech tagging and linguistic feature texts from a country extraction. from those of other Although there is no agreement on a best set of features for a wide countries put together. range of application domains, selected feature metrics must be 98 The texts in Dataset 1 bear their original class labels, that is, the and it is an engineering tool that is widely applied to numerous actual countries of affiliation of the writers as determined from the engineering problems for designing and testing all types of forums and the texts. There are therefore five different class labels, engineering and physical systems ([18]; [19]). The result of the dot representing the five country sources of the texts. Dataset 2 has products of the two measures is as presented in Appendix 2. The three class labels; texts from Nigeria and Ghana bear their original table in Appendix 2 presents the performances of our models country source labels while those from the other three countries taking into consideration the two performance measures. We were combined and labelled 'Non-Ghana-Nigeria'. This was consider this table more representative of the models' informed by a previous study that showed varying degrees of performances because it combines the strengths and weaknesses of similarity in the English language usage among the selected the two performance measures. Answers to research questions will, countries. Dataset 3 labelled texts from Nigeria as Nigeria while therefore, be based on the content of this table. texts from the other four countries were combined under the label ' Non-Nigeria'. This was done to achieve a two-class dataset option. 4. RESULTS AND DISCUSSION Research Question 1: Which feature set type maximizes the Experiments were carried out using the Experimenter interface of effectiveness of profiling the country of affiliation of writers of the open source Waikato Environment for Knowledge Analysis online messages? (WEKA) machine learning tool. In this study, four machine learning algorithm implementations in WEKA namely naïve Figure 1 is a derivative of the table in Appendix 2, it shows the Bayes, SMO (SVM implementation), J48 and Multilayer product of percent correct and kappa statistic values derived for the perceptron (Neural network implementation) were used. The feature set types in our experiment. The results are presented experiment was carried out to compare the performances classifier successively for Naive Bayes, SMO, J48 and Neural Network. models in the phase of: a. Changing the number of classes. b. Changing the linguistic feature sets. c. Changing classifier algorithms. Each of the three datasets (Dataset 1, Dataset 2 and Dataset 3) with each of the feature set types (F1, F2, F3, F4) and all their possible combinations (F1+F2, F1+F2+F3, F1+F2+F3+F4, F1+F2+F4, F1+F3, F1+F4, F2+F3, F2+F3+F4, F2+F4, F3+F4, F3+F4+F1) were analysed using the four machine learning algorithms. Ten fold cross validation was used to evaluate the models' performances based on percent correct (percentage of all datasets Figure 1: Comparison of feature sets performances that are classified correctly) and Kappa statistic (measure of the agreement between predicted and observed categorization, while Across all the three datasets, the feature set that combined all correcting for agreement that happens by chance. feature types (F1+F2+F3+F4) performed best. This is followed by (F2+F4), (F2+F3+F4) and (F1+F2+F3), while the performance of 3.4 Evaluation of the Experiments F1 was the least. Our result shows that inclusion of all features Tables in Appendix 1 show the percent correct and kappa statistic from all the four types (lexical, syntactic, idiosyncrasies and values derived for each of the datasets in our experiment. The content specific) produced the most effective model. Again the results are presented successively for Naive Bayes, SMO, J48 and result was consistent with those of [20] and[2] and [1] pg 365 who multilayer perceptron. It could be observed from the tables that the reported that combining feature types in their studies gave a better percent correct values appear to be highest for Dataset 3 while result. Using vocabulary richness only produced the poorest result Kappa statistics appear to be highest for Dataset 2. This probably because of the short length of online messages in the observation cuts across virtually all features sets and classifiers. study. This implies that classifiers were better able to classify Dataset 3 Research Question 2: Which classification scheme maximizes correctly compared to other datasets while classifications achieved the effectiveness of profiling the country of affiliation of in Dataset 2 gave better agreement between predicted and observed writers of online messages? categorization having corrected for agreement that happened by Figure 2 shows the relative performances of the four classifiers chance. Worthy to be noted is the result of SMO in Dataset 3, across all feature types (F1+F2+F3+F4) and datasets. although the percent correct values were relatively high, Kappa statistics were all zero. Lack of coherence in the directions of the two performance measures led us to using the product of the two measures (percent correct and kappa statistic) as a basis for comparing models' performances. This decision to use the product was informed by the theory of Dimensional Analysis which is a problem-solving method that uses the fact that any number or expression can be multiplied by one without changing its value. One can only meaningfully add or subtract quantities of the same type but can multiply or divide quantities of different types. When two measurements are multiplied together the product is of a type depending on the types Figure 2: Relative performances of the four classifiers across of the measurements. This analysis is routinely applied in physics all feature and data sets. 99 classifying electronic messages to writers' countries of affiliation. Neural Network (multilayer perceptron) performed best when Separate two-class label file was created for each country, resulting compared to the other three classifiers. Its performance was in a dataset for each country, where all attributes except the class particularly the highest on the feature set (F1+F2+F3+F4) attribute were the same. The class attribute for a particular country contained in our two-class option dataset (Dataset 3). Most had instances labelled either as 'the country name' such as (Nigeria, previous studies considered SVM most appropriate in authorship Ghana, Cameroon) or as 'non country name' such as (Non-Nigeria, attribution (though most times without carrying out a prior Non-Ghana, Non-Cameroon). Tables 4 shows the effectiveness of experiment). [1] however, reported that there were no significant profiling authors' countries of affiliation by the resultant model. performance differences between SVM and neural networks. It could be observed that SVM implementation (SMO) outperformed Table 4: Effectiveness of Profiling Authors' Countries of the other three classifiers when the texts contained their natural Affiliation class labels (Dataset 1) and performed most terribly on Dataset 3. Country Percent Kappa PC*KS This corroborates the submission of [15] that no single machine Correct Statistics learning scheme is appropriate to all data mining problems because Nigeria 75.80 0.34 25.95 real datasets vary and to obtain accurate models, the bias of the Cameroon 73.80 0.10 7.68 learning algorithm must match the structure of the domain. Ghana 78.40 0.27 21.54 Meaning that the structure of our Dataset 3 is most amenable to Liberia 88.20 0.04 3.23 neural network than any of the other machine learning schemes Sierra Leone 70.80 0.28 19.59 (Naive Bayes, SMO, J48) in our study. Worthy of note also is the PC*KS denotes Percent correct* Kappa statistics usefulness of our application of the dimensional analysis principle which informed the multiplication of the two performance Application of our optimization method resulted in a remarkable measures in our study. For example, if our comparison had been improvement in the profiling of each country from the others. The based on percent correct (in Appendix 1) only, we might have study showed that we could achieve a percent correct ranging erroneously rated the performance of SMO relatively high on between 70.8% and 88.2% at Kappa statistics ranging between Dataset 3. 0.04 and 0.34 compared to the highest possible percent correct value of 43.8% at kappa statistics of 0.26% if our method was not Research Question 3: Which class labelling option maximizes applied. This however is a trade-off on the efficiency of the the effectiveness of profiling the country of affiliation of profiling process because we needed to create separate labels for writers of online messages? the class attribute. The extent of improvement in model Fig. 3 shows the percent correct values derived for each of the performance however can be said to outweigh the additional effort. datasets in our experiment using the most precise classification The detailed performance of the model is as shown in Table 5. scheme (Neural Network) and all feature sets (F1+F2+F3+F4) only. The results are presented successively for Naive Bayes, Table 5: Detailed Prediction Performance of the Resultant SMO, J48 and Neural Network. Model TP FP Preci- Re- F- RO Rate Rate sion call score C Area Nigerian 0.380 0.080 0.671 0.380 0.485 0.72 1 Non-Nigerian 0.920 0.620 0.776 0.920 0.842 0.72 1 Weighted Average 0.758 0.458 0.744 0.758 0.735 0.72 1 Cameroon 0.299 0.182 0.230 0.299 0.260 0.65 2 Non-Cameroon 0.818 0.701 0.865 0.818 0.841 0.65 Figure 3: Column Chart of Classifier Performances with 2 Weighted Average 0.738 0.621 0.767 0.738 0.751 0.65 Varied Class Labelling Options 2 The figure shows that the dataset having two class options (Dataset Ghanaian 0.333 0.092 0.5 0.333 0.400 0.67 3) performed best followed by the one having three class options 1 Non-Ghanaian 0.908 0.667 0.832 0.908 0.868 0.67 (Dataset 2) and lastly the one having the instances labelled 1 naturally, having five classes (Dataset 1). The result is consistent Weighted Average 0.784 0.543 0.760 0.784 0.767 0.67 with those of [3] and [1] that reported that authorship attribution 1 success improves with reduction in the number of authors or author Liberian 0.036 0.013 0.250 0.036 0.063 0.67 classes. In the specific however, the present result shows that if we 1 can reduce an authorship profiling problem to a two-class one, we Non-Liberian 0.987 0.964 0.892 0.987 0.937 0.67 can get an appreciable improvement in the effectiveness of 1 authorship profiling task. Weighted Average 0.882 0.859 0.822 0.822 0.841 0.67 1 Research Question 4: What is the performance of the resultant Sierra-Leonean 0.582 0.256 0.39 0.582 0.467 0.74 model in classifying electronic messages to writers' countries of (SL) 8 affiliation? Non SL 0.744 0.418 0.863 0.744 0.799 0.74 Using the TrainTestSplitMaker component of WEKA's knowledge 8 Weighted Average 0.708 0.383 0.759 0.708 0.726 0.74 flow interface to evaluate the performance of our model in 8 100 The resultant model performed well when we consider the [5] Koppel, M., Schler, J., Argamon, S. 2009. Computational weighted averages of the performance measures of each dataset. It methods in authorship attribution. Journal of the American could however, be observed that the model was better at Society for Information Science and Technology 60(1). 9-26. identifying texts that were not from the country as against those [6] Argamon, S., Koppel,M., Pennebaker, J.W. and Schler, J. that were from the country in each case. It could also be observed 2009. Automatically Profiling the Author of an Anonymous that the performance of the model in predicting each country's Text. Communications of the ACM 52(2). 119-123. texts vary directly with the number of each country's texts in the [7] De Vel, O. , Anderson, A., Corney, M. and Mohay, G. 2001. study corpus. The best performance was achieved in profiling Mining E-mail Content for Author Identification Forensics. Nigerian electronic texts from Non Nigeria texts, followed by that Special Interest Group on Management of Data (ACM of Sierra Leone and then Ghana. Thus, it could be deduced that SIGMOD) Record 30(4). 55-64. performance of our model could be much improved with bigger [8] Juola, P. 2007. Future trends in authorship attribution. sub-corpora sizes. International Federation for Information Processing 24(2). 119-132. 5. CONCLUSION [9] Iqbal, F., Hadjidj, R., Fung, B.C.M and Debbabi, M. 2008. A The study through experiments sought the number of class options, novel approach of mining write-prints for authorship feature set types and machine learning scheme that maximize the attribution in e-mail forensics. 2008 Digital Forensic effectiveness of identifying the countries of affiliation of authors of Research Workshop. Elsevier Ltd. Retrieved Nov. 16, 2009, online messages composed in English language. The online from www.elsevier.com/locate/diin. 2008.05.001 messages in our corpus were collected from online forums of five [10] Holmes, D.I. 2003. Stylometry and the civil war: the case of African countries with average length of 52 to 102 words. Using a the Pickett letters. CHANCE 16(2) 18-25. product of percent correct and kappa statistics as our bases for [11] Binongo, J.N.G. 2003. Who wrote the 15th book of Oz? An model justification, the experiment showed that we achieved the application of multivariate analysis to authorship attribution. most effective model when all feature set types, contained in a CHANCE 16(2) . 9-17. two-class dataset was analysed with the neural network (multilayer [12] Binongo, J.N.G. and Smith M.W.A. 1999. The application of perceptron) machine learning scheme. Application of the principal component analysis to stylometry. Literary and parameters of the most effective model (derived from the Linguistic Computing 14(4). 445-466. experiment) to profiling the countries of affiliation of authors of [13] Abbasi, A. and Chen, H. 2006. Visualizing authorship for the online messages resulted in about a hundred percent identification. Lecture Notes in Computer Science (LNCS) improvement in effectiveness. 3975. Eds. Mehrotra, S., Zeng, D.D., Chen, H., Thuraisingham, B., Wang, F. Berlin: Springer-Verlag. 60– The study achieved greater effectiveness but with a trade-off on 71. efficiency. We look forward to having a model that can maximize [14] Abbasi, A. and Chen, H. 2008. Writeprints: a stylometric both effectiveness and efficiency in profiling the authorship of approach to identity level identification and similarity online messages, and this constitutes a need for further studies. detection in cyberspace. ACM Transactions on Information This approach in its present state can be very appropriate if a group Systems. 26 (2). doi: 10.1145/1344411.1344413. is suspected and the purpose of authorship attribution is to affirm [15] Witten, I.H. and Frank, E. 2005. Data mining: practical one's thought about the suspect's group of affiliation. machine learning tools and techniques. 2nd ed. USA: Morgan Kaufmann publishers. [16] Kujore, O. 1985. English usage: some notable Nigerian 6. REFERENCES variations. 1-112. Nigeria: Evans Brothers Nigeria [1] Zheng, R., Li, J., Chen, H. and Huang, Z. 2006. A framework Publishers Limited. for authorship identification of online messages: writing- [17] Jowitt, D. 1991. Nigerian English usage: An Introduction. 1- style features and classification techniques. Journal of the 277. Nigeria: Longman. American Society for Information Science and Technology, [18] Balaguer P 2013 Application of Dimensional Analysis in 57(3). 378–393. Systems Modeling and Control Design, The [2] Koppel, M., Schler, J. and Zigdon, K. 2005. Automatically Institution of Engineering and Technology; determining an anonymous author’s native language. [19] Szirtes T 2007 Applied Dimensional Analysis and Modeling. Lecture Notes in Computer Science (LNCS) 3495. Eds. Elsevier/Butterworth-Heinemann Amsterdam; New Kantor, P.B., Muresan, G., Roberts, F., Zeng, D.D. and York. Wang, F : ISI 2005, Berlin: Springer-Verlag. 209 – 217. [20] Ma, J., Teng, G., Zhang, Y., Li, y. and Li Y (2009) A [3] Luyckx, K. and Daelemans, W. 2008. Authorship attribution Cybercrime Forensic Method for Chinese Web Information and verification with many authors and limited data. In: Authorship Analysis. In: PAISI 2009, LNCS 5477 pp. 14- Proceedings of the 22nd International Conference on 24. H. Chen et al. (Eds.). Springer-Verlag Berlin Heidelberg Computational Linguistics held in Manchester from 18-22 [21] Opesade, A., Adegbola, T., & Tiamiyu, M. (2013). August 2008. 513–520. Comparative Analysis of Idiosyncrasy, Content and [4] Koppel , M., Schler , J., Argamon, S. and Messeri, E. 2006. Function Word Distributions in the English Language Authorship attribution with thousands of candidate authors. Variants of Selected African Countries. International Journal In: Proceedings of the 29th annual international ACM SIGIR of Computational Linguistics Research Vol. 4(3) pp.130- (Special Interest Group on Information Retrieval) 143. conference on research and development in information retrieval. Aug. 6-11 2006, Seattle, Washington, USA. 101 Appendix 1: Experiment Result Naive Bayes SMO Dataset 1 Dataset 2 Dataset 3 Dataset 1 Dataset 2 Dataset 3 Feature PC KS PC KS PC KS PC KS PC KS PC KS F1+F2+F3 +F4 34.96 0.18 58.49 0.33 67.34 0.31 43.65 0.25 62.12 0.34 71.09 0.00 FI 31.87 0.06 50.14 0.09 71.09 0.00 31.52 0.05 49.52 0.00 71.09 0.00 F1+F2 36.29 0.20 60.11 0.34 69.41 0.29 42.53 0.24 58.54 0.26 71.09 0.00 F1+F2+F3 36.77 0.20 60.31 0.35 69.78 0.32 42.48 0.24 60.33 0.30 71.09 0.00 F1+F2+F4 34.80 0.18 58.39 0.33 66.97 0.30 42.84 0.24 60.54 0.30 71.09 0.00 F1+F3 32.37 0.11 52.63 0.21 69.77 0.15 32.73 0.07 49.48 0.00 71.09 0.00 F1+F4 32.10 0.15 55.01 0.28 65.59 0.27 34.07 0.10 49.86 0.06 71.09 0.00 F2 36.06 0.20 59.77 0.33 70.74 0.30 41.83 0.23 58.48 0.26 71.09 0.00 F2+F3 36.69 0.20 60.51 0.35 70.63 0.31 42.32 0.23 59.57 0.28 71.09 0.00 F2+F3+F4 34.64 0.18 58.86 0.33 67.93 0.31 43.76 0.26 61.72 0.33 71.09 0.00 F2+F4 34.65 0.18 58.73 0.33 67.46 0.30 42.50 0.24 59.89 0.29 71.09 0.00 F3 31.79 0.10 53.24 0.18 71.46 0.09 31.86 0.06 49.50 0.00 71.09 0.00 F3+F4 32.43 0.15 55.97 0.29 66.14 0.27 35.05 0.12 51.58 0.10 71.09 0.00 F3+F4+F1 32.64 0.15 55.44 0.28 65.23 0.26 34.50 0.11 52.17 0.11 71.09 0.00 F4 31.53 0.14 55.37 0.28 65.93 0.26 34.88 0.12 49.75 0.06 71.09 0.00 PC = Percent Correct KS = Kappa Statistic Experiment Result Continued Tree (J48) Multilayer Perceptron (Neural Network) Feature Dataset 1 Dataset 2 Dataset 3 Dataset 1 Dataset 2 Dataset 3 F1+F2+F3+F4 PC KS PC KS PC KS PC KS PC KS PC KS FI 35.11 0.13 49.92 0.15 71.09 0.00 31.76 0.07 50.01 0.13 71.09 0.00 F1+F2 38.32 0.20 55.59 0.28 72.10 0.24 40.28 0.22 60.31 0.33 72.73 0.30 F1+F2+F3 37.66 0.20 55.05 0.28 71.74 0.24 40.16 0.22 61.05 0.35 74.16 0.33 F1+F2+F4 37.88 0.20 55.65 0.29 70.80 0.24 41.22 0.23 60.32 0.34 72.91 0.32 F1+F3 31.58 0.11 51.93 0.20 72.37 0.09 32.73 0.10 52.52 0.19 70.39 0.03 F1+F4 34.61 0.15 55.34 0.28 70.43 0.05 38.69 0.18 57.62 0.29 69.86 0.16 F2 37.87 0.20 55.57 0.28 72.20 0.26 40.18 0.21 59.50 0.32 71.90 0.28 F2+F3 36.97 0.19 55.41 0.28 71.63 0.25 41.02 0.23 61.19 0.35 73.98 0.33 F2+F3+F4 37.76 0.20 56.08 0.30 71.11 0.25 41.22 0.24 60.37 0.34 73.81 0.33 F2+F4 37.84 0.20 56.18 0.30 71.14 0.25 40.60 0.22 60.30 0.34 72.50 0.30 F3 29.48 0.07 52.54 0.17 70.59 0.04 31.64 0.08 52.90 0.17 70.44 0.04 F3+F4 35.49 0.17 54.41 0.27 69.78 0.17 38.06 0.18 57.67 0.30 70.46 0.19 F3+F4+F1 34.71 0.16 54.12 0.27 69.94 0.18 38.48 0.19 57.69 0.30 70.12 0.20 F4 35.41 0.16 55.06 0.27 70.09 0.02 38.46 0.18 57.03 0.29 70.49 0.12 PC = Percent Correct KS = Kappa Statistic 102 Appendix 2: Products of Percent Correct and Kappa Statistics Naive Bayes SMO J48 Multilayer Perceptron (PC*KS) (PC*KS) (PC*KS) (PC*KS) Datase Datas Datas Datase Datas Datase Datase Dataset Dataset Dataset Dataset Datase Feature t1 et 2 et 3 t1 et 2 t3 t1 2 3 1 2 t3 F1+F2+F 3+F4 6.29 19.30 20.88 10.91 21.12 0.00 7.55 16.17 17.77 9.95 21.29 25.22 FI 1.91 4.51 0.00 1.58 0.00 0.00 4.56 7.49 0.00 2.22 6.50 0.00 F1+F2 7.26 20.44 20.13 10.20 15.22 0.00 7.66 15.57 17.30 8.86 19.90 21.82 F1+F2+F 3 7.35 21.11 22.33 10.20 18.10 0.00 7.53 15.41 17.22 8.84 21.37 24.47 F1+F2+F 4 6.26 19.27 20.09 10.28 18.16 0.00 7.58 16.14 16.99 9.48 20.51 23.33 F1+F3 3.56 11.05 10.47 2.29 0.00 0.00 3.47 10.39 6.51 3.27 9.98 2.11 F1+F4 4.82 15.40 17.71 3.41 2.99 0.00 5.19 15.50 3.52 6.96 16.71 11.18 F2 7.21 19.72 21.22 9.62 15.20 0.00 7.57 15.56 18.77 8.44 19.04 20.13 F2+F3 7.34 21.18 21.90 9.73 16.68 0.00 7.02 15.51 17.91 9.43 21.42 24.41 F2+F3+F 4 6.24 19.42 21.06 11.38 20.37 0.00 7.55 16.82 17.78 9.89 20.53 24.36 F2+F4 6.24 19.38 20.24 10.20 17.37 0.00 7.57 16.85 17.79 8.93 20.50 21.75 F3 3.18 9.58 6.43 1.91 0.00 0.00 2.06 8.93 2.82 2.53 8.99 2.82 F3+F4 4.86 16.23 17.86 4.21 5.16 0.00 6.03 14.69 11.86 6.85 17.30 13.39 F3+F4+F 1 4.90 15.52 16.96 3.80 5.74 0.00 5.55 14.61 12.59 7.31 17.31 14.02 F4 4.41 15.50 17.14 4.19 2.99 0.00 5.67 14.87 1.40 6.92 16.54 8.46 PC*KS denotes Percent correct* Kappa statistic 103