1 Uş a k U ni versit y, Dist a nc e E d uc ati o n V oc ati o n al Sc h o ol , De p art me nt of C o m p uter Pr o gr a m mi n g , Uş a k, T ur k e y 2 G a zi U ni versit y , F ac ult y of Tec h n ol o g y , D e p art m e nt of C o m p uter E n gi neeri n g , A n k ar a, T ur k e y A b str a ct T o d a y, t h e us e of s o ci al m e di a sit es s u c h as F a c e b o o k, I nst a gr a m, T witt er is i n cr e asi n g d a y b y d a y. S h ar es o n s o ci al m e di a c a n e v e n c h a n g e t h e a g e n d a b y r e a c hi n g h u g e m ass es. O n T witt er, a s o ci al m e di a sit e, t h e a g e n d a t o pi cs c a n b e f oll o w e d t hr o u g h t h e s e cti o n c all e d Tr e n d -T o pi c. T his Tr e n d - T o pi c s e cti o n m a y b e m a ni p ul at e d b y s p a m m ers fr o m ti m e t o ti m e. I n or d er t o a v oi d s u c h u n w a nt e d sit u ati o ns, it is n e c ess ar y t o d et er mi n e w h et h er t h e us er is s p a m or n ot. M a c hi n e l e ar ni n g al g orit h ms c a n cl assif y w h et h er a us er is s p a m or n ot. Wit h m a c hi n e l e ar ni n g al g orit h ms, s u c c essf ul r es ults ar e als o o bt ai n e d i n sit u ati o ns s u c h as i m a g e pr o c essi n g, s p e e c h, v oi c e r e c o g niti o n a n d m al w ar e d et e ct i o n. I n t his st u d y, m a c hi n e l e ar ni n g al g orit h ms N aiv e B a y es, K N e ar est N ei g h b or s, R a n d o m F or est, j 4 8, M ultil a y er P er c e ptr o n w er e us e d t o cl assif y us ers. As a r es ult of t h e e v al u ati o ns, R a n d o m F or est al g orit h m, o n e of t h e m a c hi n e l e ar ni n g al g orit h ms us e d, m a d e t h e m ost s u c c essf ul cl assifi c ati o n wit h a n a c c ur a c y r at e of 8 8 % . K e y w or d s 1 T witt er, s p a m d et e cti o n, m a c hi n e l e ar ni n g 1. I ntr o d u cti o n I V U S 2 0 2 2: 2 7t h I nt er n ati o n al C o nf er e n c e o n I nf or m ati o n T e c h n ol o g y E M AI L: ali er b e y @ g m ail . c o m ( A. Er b e y); n b aris ci @ g a zi . ed u.tr ( N. B arış çı) O R CI D: 0 0 0 0 - 00 0 2 - 09 3 0 - 40 8 1 ( A. Er b e y); 0 0 0 0 - 00 0 2 - 87 6 2 - 50 9 1 ( N. B arış çı) © 2 0 2 2 C o p yri g ht f or t his p a per b y its a ut h ors. Use per mitte d u n der Creati ve CPWErooUrckResehdoinpgs hIStSpN:/c1e6u1r3-w-0s.o7r3g CCoEm mUo nRs LiWcoernskesAthtriobputi o nPr4.o0 cI neteer ndaitino ngasl(C(C CB YE 4.U0).R-W S. or g)
T o d a y, wit h t h e wi d e s pr e a d u s e of t h e i nt er n et
a n d t h e i n cr e as e i n t h e us e of m o bil e d e vi c es,
o nli n e s o ci al n et w or ki n g sit es, s o ci al n et w or ks
s u c h as F a c e b o o k, T witt er a n d Li n k e dI n ar e
b e c o mi n g m or e a n d m or e p o p ul ar [
Tr e n d T o pi c s e cti o n c a n b e dir e ct e d t o t h e
a g e n d a i n a n u n d esir a bl e w a y wit h m es s a g es s e nt
fr o m ti m e t o ti m e, o ut of p ur p os e. T h e h e a v y us e
of s o ci al m e di a h as f a cilit at e d t h e n e gl e ct of t h es e
e n vir o n m e nt s b y m ali ci o us p e o pl e [
W h e n w e l o o k at t h e e xisti n g st u di es i n d et e cti n g s p a m wit h T witt er p ost s, w e s e e t h at t h er e i s a l ot of w or k. T h es e st u di es ar e cl ust er e d i n c ert ai n ar e as. T witt er s p a m d et e cti o n st u di es ar e m ai nl y h a n dl e d i n t hr e e gr o u ps. T h es e ar e: a) t h os e w h o o nl y e x a mi n e t h e t w e et s b y t e xt mi ni n g, b) t h os e w h o a n al y z e t h e t w e et t e xt b y as s o ci ati n g wit h t h e us er w h o s e nt t h e t w e et, c) t h os e w h o e x a mi n e t h e r el ati o ns of us er s wit h s p a m.
T e xt mi ni n g -b as e d r es e ar c h m ostl y f o c us es o n
t w e et t e xt. I n t h es e st u di e s, r es e ar c h er s fir st
e xtr a ct f e at ur es a n d t h e n cl as sif y t h e m wit h
al g orit h ms s u c h as N ai v e B a y es a n d j 4 8. F e at ur e
extraction sometimes follows feature selection to
improve classification accuracy and reduce
training time. Gupta and Kumar [
Some research takes user characteristics into account when deciding whether a user is spam. These features can be account age, number of followers/followers, follower/followers’ rate, format of the profile page. However, since these features can be easily changed by the user, they are considered to be less reliable.
Because user characteristics can change easily, some researchers have studied the relationships between spammers and real users. By examining their following / following relationships, they created a network for each user. Setting up these networks can be costly in terms of computation time, power and data collection time.
In the following sections of this study, obtaining the spammy dataset, classification, detection of spammy users and feature selection processes are carried out. In the last section, the obtained results are evaluated.
We collect the dataset before making the classification. The data collection process has an important role in the classification process. 2.1.
In this study, user characteristics and the method of evaluating users' tweet attributes were chosen in order to classify spam. The reason for choosing this method is that it is less costly in terms of data collection and it is seen to give better results regardless of the tweet content, as it depends on user characteristics.
For training, a topic was selected from the Turkey Trend topic list, since a data set with spam users should be obtained. 15000 tweets were collected from this trending topic with the Twitter public API. Then, repetitive data, news content, tweets containing URL only were removed from this dataset and the remaining 3798 tweets were classified as spam and not spam. As a result, 3798 tweets were classified as 1666 spam and 2132 non-spam users.
In order to evaluate which users can send spam tweets after classification, the last maximum of 100 tweets of each user were collected. Since some users did not have 100 tweets, a total of 305.604 tweets were reached. Then, these tweets were processed for each user, and some unnecessary data such as smileys were removed from the text of the tweet. Because tweets are unofficial texts, some autocorrect libraries were used and typos were corrected. Then, as a more complex process, some new features are obtained from this tweet data, such as how often the user tweets, the average number of characters of the user's tweets, or the unique words tweeted in those 100 tweets, represented as columns in the final dataset has been done.
As a result of the data collection and preprocessing stage, a dataset consisting of 3798 rows representing each user and 22 columns in total, including features such as the age of the user, whether he is a verified account, whether he entered a URL on the profile page, was obtained.
The next part after data collection is to determine whether the user is a spam user with different machine learning algorithms.
Weka software was used to classify the
obtained data as spam or not spam. Weka is a
program developed for machine learning and text
mining intended to assist in the application of
machine learning techniques [
In this study, Naive Bayes algorithm, k Nearest Neighbor algorithm, Random Forest algorithm, j48 algorithm and Multilayer Perceptron classification algorithms were used in Weka software.
Considering the studies in the literature, the algorithms used in other studies are shown in Table 1.
The reason for choosing the NB, KNN, RF , j48 and MLP algorithms used in this study is to try to create a combination of algorithms that are widely used in the literature and in addition to them, less used algorithms. The reason for choosing the most used algorithms is to make comparisons with previous studies. The reason for choosing the less used algorithms is to create an alternative to the frequently used algorithms.
The Naive Bayes (NB) algorithm is a simple
probabilistic classifier that calculates a probability
set by counting the frequency and combinations
of values in a given data set. It is a classification
algorithm that classifies data by calculating it with
probability principles. Naive Bayes is a popular
algorithm used commercially or open source for
email spam filtering [
In 1968, Cover and Hart proposed the k
Nearest Neighbor (KNN) algorithm, which they
have been working on for a long time [
Breiman [
The purpose of the Decision Tree Algorithm is
to determine how the feature vector behaves for a
few samples [
Multilayer perceptron (MLP) is an algorithm
that can be effectively used for classification
purposes and has been used a lot recently. In
general, back propagation algorithm learning
technique based on slope drop method is used in
MLP. With this technique, the error between the
desired output and the produced output is
minimized [
The data obtained during the classification of the data was divided into 80% training data and 20% test data and evaluated in NB, KNN, RB, j48 and MLP algorithms. It is known that 1666 of 3798 users are spam users and 2132 of them are non-spam users in the dataset. The number of users to test is 760 people, which is 20% of the data. Looking at whether users are spam with the Naive Bayes algorithm, the Naive Bayes algorithm classified users with an accuracy rate of 76%.
When the complexity matrix of the NB algorithm is examined, the data are shown in Table 2.
According to the complexity matrix of the algorithm in Table 2; Of the 760 people in the 20% test data, 382 people who were spam were classified as spam, and 60 people who were spam were classified as non-spam. 122 non-spam were classified as spam, while 196 non-spam were classified as non-spam. When we look at the KNN algorithm, one of the machine learning algorithms, it is seen that it classifies users with an accuracy rate of 74%.
The complexity matrix of the KNN algorithm is as shown in Table 3.
According to the complexity matrix of the KNN algorithm in Table 3; Of the 760 people in the 20% test data, 357 people who were spam were classified as spam, and 85 people who were spam were classified as non-spam. 107 non-spam were classified as spam, while 211 non-spam were classified as non-spam.
When the Random Forest algorithm was used in the study, an accuracy rate of 88% was achieved.
According to the RF algorithm complexity matrix in Table 4; Of the 760 people in the 20% test data, 406 people who were spam were classified as spam, and 36 people who were spam were classified as non-spam. 53 non-spam classified as spam, 265 non-spam classified as non-spam.
85% accuracy rate was observed with the J48 algorithm. The complexity matrix of the J48 algorithm is as shown in Table 5.
According to the complexity matrix of the j48 algorithm in Table 5; Of the 760 people in the 20% test data, 382 people who were spam were classified as spam, and 60 people who were spam were classified as non-spam. 50 non-spam classified as spam, 268 non-spam classified as non-spam.
The MLP algorithm found an accuracy rate of 82%. The complexity matrix of the MLP algorithm is as shown in Table 6.
According to the complexity matrix of the MLP algorithm in Table 6; Of the 760 people in the 20% test data, 390 people who were spam were classified as spam, and 52 people who were spam were classified as non-spam. 82 non-spam people were classified as spam, while 236 nonspam were classified as not spam.
Feature selection is one of the important steps
of pattern recognition, machine learning and data
mining. Its purpose is to eliminate irrelevant and
redundant variables in order to understand the
data, reduce the computational requirement,
reduce the dimensionality effect, and improve the
performance of the predictor [
When the data is re-evaluated after the feature selection, the performances of the algorithms are seen in Figure 1.
As shown in Figure 1, it has obtained similar results with feature selection and without feature selection.
In this study, SPAM users on Twitter were tried to be detected. In this study, the data set consists of 3798 user information and different machine learning algorithms are used to classify users. The Random Forest algorithm achieved the highest accuracy rate of 88%. The NB algorithm achieved 76%, KNN 74%, j48 83% and MLP 80% correct classification rates. When the algorithms were applied again after the feature selection was made, it was observed that the accuracy rate decreased in other algorithms except the NB algorithm.
In addition to the accuracy rates in the algorithms, the number of users whose real class is negative but classified as positive in the complexity matrix is also important. Since these users are classified as spam even though they are not spam, they will suffer if the algorithm is trusted. This will create an undesirable situation. When we look at the results, it is seen that the j48 algorithm gives the lowest rate with 50 users.
As a suggestion for future research, different optimizations of feature extraction and different machine learning algorithms methods can be tried on the collected data and more successful classification results can be achieved.