Social network platforms include several tasks such as advertisements, political communications, and so on, producing vast amounts of data spread over the network. Consequently, verifying the truthfulness of such data and the accounts generating them becomes necessary. In particular, malicious users often create fake accounts and followers for harmful activities, potentially producing negative societal implications. In order to increase the capability to identify fake accounts correctly, this discussion paper presents a new feature engineering strategy that exploits relaxed functional dependencies (rfds) to enhance the capability of existing machine learning strategies in discriminating fake accounts. In particular, experimental results conducted using several machine learning models on account datasets of both the Twitter and Instagram platforms emphasize the efectiveness of the proposed approach in fake account discrimination activities.
Social networks enable sharing information among users of all ages, at every moment, and in every part of the world. Social interaction platforms like Instagram, Twitter, Tumblr, etc., have a significant impact on the daily life of their users and the entire society.
A fundamental aspect to be monitored over a social network is the popularity of a profile, witnessed by the number of its friends or followers. A Twitter or Instagram profile with many followers is considered influential, hence it provides a better reputation to the profile’s owner and attract better-paid advertisements. Consequently, a common practice of several social network users is to buy fake followers to appear more influential, also because they can be bought at an extremely low price (a few dollars for hundreds of fake followers). If this practice was merely used to support individual vanity, it would be harmless, but if it aimed at making an account more reliable and influential, it might be dangerous.
In general, it is possible to find many types of anomalous accounts in social networks, such
as Spammers, Bots, Cyborgs, and Trolls. Spammer accounts tend to recommend fake contents
and/or dangerous links. Bot algorithms tend to manage accounts to simulate human behavior,
trying to automatically perform typical human actions. With respect to them, Cyborgs are both
managed by humans, hence they are not necessarily malicious. For instance, a politician might
not handle his/her account personally and might rely on a stuf of people together with some
Bots. Finally, Trolls are algorithms aiming to disrupt conversations and activities of others.
Related work. In the literature, we find several automatic techniques for identifying spams
and bots [
In this discussion paper, we describe the feature engineering strategy proposed in [
The remainder of the paper is organized as follows. Section 2 presents the new proposed feature engineering strategy and the associated FEM metric. Section 3 presents experimental results, whereas Section 4 concludes the paper and provides directions for future work.
In general, although the addition of new features can potentially increase the training time of
classification algorithms, it could lead to the creation of more concise and accurate classifiers.
Moreover, meaningful features could contribute to the understanding of the learned concept
[
Aiming to add new meaningful features based on (rfds), we defined a new function, named tuple Frequency Account in Validation (FAV), which permits to account for the number of times a tuple is involved in the validation of an (rfd) when it is coupled with other tuples. A more formal definition of the FAV function is provided below.
Definition 1 (Tuple Frequency Account in Validation (FAV)). Given a relational database schema ℛ, denfied on a set of attributes (ℛ) = {1, . . . }, an instance of ℛ with tuples, an (rfd) : Φ→1−− − Ψ≤ Φ2 holding on , and a tuple , the tuple frequency in validating (1) (2) can be defined as: () = ∑︀=1 |= (, )
− 1 where |= (, ) is a boolean function defined by the following formula: |= (, ) = {︃1, if (, ) satisfies , 0, otherwise with ̸= In other words, () counts the number of tuples that satisfy when compared with .
The FAV function distributes the validation of an (rfd) throughout the tuples of the dataset.
In this way, it is possible to characterize how much each sample (a tuple) is involved in the
validation of an (rfd) , while maintaining the semantics of preserved. The proposed feature
engineering methodology exploits the FAV function to add new features related to the discovered
(rfds) to the account dataset. Consequently, an (rfd) characterizing fake accounts should yield
higher FAV values for fake account tuples; vice versa, it is expected that such accounts should
present low FAV values for (rfds) that do not characterize fake accounts. For more details about
(rfd) concepts, refer to [
Since the training set will necessarily include tuples of both fake and real accounts, the defined metrics consider both ) the class of each tuple, named tuple type, and ) the category of accounts from which a (rfd) has been discovered, named (rfd) type. Then, according to them, it is possible to evaluate the expected FAV values. More specifically, we expected that a ( rfd) holding on fake accounts only should provide FAV values that are high for fake accounts and low for real ones. Vice versa, a (rfd) holding on real accounts only should provide FAV values that are low for fake accounts and high for real ones. For this reason, we can state that the most significant FAV-based features are those that show the extremes of this behavior, by assigning a value of 1 when the tuple type and the (rfd) type match, and a value of 0 when the types do not match. However, having this kind of behavior is unrealistic, since it would define a perfect classification criterion for assigning an account to its proper category.
By following the previous considerations, we defined novel metrics, named FAV-based feature Evaluation Metrics (FEM), which allow the evaluation of the meaningfulness of FAV-based features in order to define a ranking and filtering strategy devoted to the minimization of the number of the newly added features in the classification models. A more formal definition of FEM metrics is provided below.
Definition 2 (FAV-based feature Evaluation Metrics (FEM)). Given a relational database schema ℛ, denfied on a set of attributes (ℛ) = {1, . . . }, an instance of ℛ with tuples, an (rfd) : Φ→1−− − Ψ≤ Φ2 holding on , and the FAV-based feature generated from it , the evaluation of the meaningfulness of in discriminating account types can be defined as: = ∑︀∈ | − ()| (3) where represents the expected value obtained according to the (rfd) type and the tuple type.
In other words, the proposed FEM metrics provide a value in the range [
Notice that, the metrics can be used for both ranking the FAV-based features and for ifltering them when it is combined with an input threshold to form a selection constraint. In the latter case, only the FAV-based features satisfying the constraint ≤ will be then used during the classification process.
The experimental session started with the definition of the dataset to be considered in our
evaluation. In particular, we have merged fake, verified, and real account datasets described
in [
According to the resulting (rfds), for each comparison threshold we constructed two datasets. The first one has been computed by adding the new FAV-based features as explained above, whereas the second one by only using the new FAV-based features (i.e., by removing the original features).
Each new dataset has been randomly split into training and test datasets with a proportion of 80% and 20%, respectively, and the efectiveness of supervised classification models has been evaluated in terms of precision (P), recall (R), and accuracy (A). Thus, we analyzed how the classification scores vary between the original dataset (named Baseline), the one augmented with the new features, and the one containing only the FAV-based features.
The experimental evaluation has involved Decision Tree [12], Random Forest [13], Support Vector Classification (SVC) [ 14], and Logistic Regression [15] as supervised classification models, by considering their versions available in the Scikit-learn1 python library.
Experimental results of each classifier over the diferent configurations are shown in Figure 1. In particular, it is possible to notice that rows show the used classifiers, whereas columns show evaluation metrics. Additionally, each plot in Figure 1 highlights how the performances change when using original features augmented with FAV values (denoted by 16 + in Figure 1), or FAV-based features only (denoted by in Figure 1), by considering diferent comparison thresholds (denoted by in each plot). Moreover, we also compared performances achieved on these datasets w.r.t. those achieved on the original dataset (denoted by, in Figure 1).
In what follows, we discuss how the application of the proposed feature engineering strategy afects the performances of the trained classification models. Decision tree By applying the DT model in the context of fake account discrimination (see Figure 1) it is possible to notice that DT outperforms the baseline for each used evaluation metric when using original features combined with FAV values, as well as FAV values only, with all comparison thresholds. Specifically, the proposed feature engineering strategy enhances the capabilities of the DT model in the discrimination task because most of the added features have been selected in the tree construction, since they permit to infer more discriminative patterns with respect to those achievable with the baseline features. Furthermore, it is possible to notice that the DT model achieves the same results with all evaluation metrics when trained with the 16 + or the feature set w.r.t. all comparison thresholds. In detail, by performing further analysis, we observed that the model only selects FAV-based features to build the DT structure, considering the original features not beneficial for the training phase. Random forest By applying the RF model in the context of fake account discrimination (see Figure 1), it is possible to notice that it outperforms the baseline for each evaluation metric, except on the recall, when using original features combined with FAV values, or the FAV values only, with all comparison thresholds. Moreover, it is possible to notice that the RF model achieves the same results for all evaluation metrics when trained on the 16 + or feature set with all comparison thresholds, except on the recall metric that does not present variations. This is what we expected, having observed that the proposed feature engineering strategy enhances the capabilities of the DT model in discriminating fake accounts, hence also RF indirectly benefits from it.
Support Vector Classification By applying the SVC model in the context of fake account discrimination we observed that it achieves the best results for each evaluation metrics, with all comparison thresholds, when trained on the feature set, except for the recall metric that presents a slight decrease (see Figure 1). In particular, by performing a thorough analysis of iftting problems, we found that when trained with original or 16 + feature set, such a model undergoes overfitting (with kernel set to Radial Basis Function - RBF) and underfitting (with kernel set to Sigmoid) phenomena. This is probably due to the fact that the original features do not permit to compute a hyperplane capable of discriminating accounts. Instead, using FAV values only implicitly guarantees the exploitation of semantic properties that permit a better discrimination capability when the SVC model is employed.
Logistic regression By applying the LR model in the context of fake account discrimination (see Figure 1), it is possible to notice that LR outperforms the baseline for each evaluation metrics when exploiting original features combined with FAV values, or FAV values only, with all comparison thresholds. In general, we have observed that our feature engineering strategy helps the LR model in the discrimination task, since most of the added features have either positive or negative weights, hence afecting the classification process.
K-nearest neighbors By applying the KNN model in the context of fake account discrimination (see Figure 1), it is possible to notice that KNN outperforms the baseline for each evaluation metrics when exploiting the FAV values only, with all comparison thresholds. On the other hand, when trained on the 16 + feature set, no improvement w.r.t. the baseline is reported, for each evaluation metrics, except for the recall. In general, since the KNN model classifies a new account by comparing it through a distance metric with each training account, good results for each evaluation metrics highlight that the proposed feature engineering strategy produces FAV values having a strong correlation among accounts of the same class.
In this section, we investigate the impact of filtering relevant FAV-based features in order to decrease the training time of classification models, while maintaining high performances in terms of fake account discrimination. In particular, Figure 2 reports accuracy (Accuracy), number of selected features (#Features), and training time (Time (s), expressed in seconds) by varying the comparison and the feature selection thresholds for each classification model.
Moreover, feature selection thresholds reported on the x-axis are shared among all row plots and
their maximum value (1) represents the selection of all FAV-based features. Instead, the other
values represent a specific threshold to filter no relevant features w.r.t. comparison threshold.
As can be seen in Figure 2, first-row plots show that the accuracy is maintained relatively
high even if a significant number of features is removed. In particular, we achieve best results
when considering the 1 as a comparison threshold, which preserves the accuracy trend
of all models and considers a restricted number of FAV-based features in the training set (see
second-row plots in Figure 2). Additionally, third-row plots illustrate that the 1 presents
the shortest time recorded for the training phase of each classification model. In conclusion, we
can observe that the combination of 1 and 0.3 as comparison and selection thresholds,
respectively, provide the best trade-of in terms of accuracy, training time, and number of
features involved in the training phase. It is important to notice that, a more detailed discussion
and further evaluations of the proposed feature engineering approach are shown in [
This work presents a new feature engineering strategy that exploits relaxed functional dependencies (rfds) to enhance the capability of existing machine learning strategies in discriminating fake accounts. Evaluation results achieved over diferent machine learning models demonstrated that not only the proposed strategy permits to improve classification performances, but it never negatively afects the application of models.
In the future, other than planning similar studies on diferent social network platforms, we would like to exploit other types of data dependencies, like graph dependencies, since they can potentially detect additional useful behavioral models to help discriminating fake accounts [16].
This Publication was produced with the co-funding of the European union - Next Generation EU: NRRP Initiative, Mission 4, Component 2, Investment 1.3 – Partnerships extended to universities, research centers, companies and research D.D. MUR n. 341 del 5.03.2022 – Next Generation EU (PE0000014 - "Security and Rights In the CyberSpace - SERICS" - CUP: H93C22000620001). [12] P. H. Swain, H. Hauska, The decision tree classifier: Design and potential, IEEE Transactions on Geoscience Electronics 15 (1977) 142–147. [13] M. Pal, Random forest classifier for remote sensing classification, International journal of remote sensing 26 (2005) 217–222. [14] S. S. Keerthi, S. K. Shevade, C. Bhattacharyya, K. R. Murthy, A fast iterative nearest point algorithm for support vector machine classifier design, IEEE transactions on neural networks 11 (2000) 124–136. [15] F. O. Redelico, F. Traversaro, M. d. C. García, W. Silva, O. A. Rosso, M. Risk, Classification of normal and pre-ictal eeg signals using permutation entropies and a generalized linear model as a classifier, Entropy 19 (2017) 72. [16] W. Fan, Y. Wu, J. Xu, Functional dependencies for graphs, in: Proceedings of the 2016 International Conference on Management of Data, ACM, San Francisco, California, USA, 2016, pp. 1843–1857.