Modeling of online social networks for automated monitoring system

Abstract

Monitoring using keywords is necessary step in solving the problem of detection of users' illegal behavior such as drug use, extremist propaganda in online social networks. Analysis of text posts is difficult because of using jargon and making mistakes in communications. In paper model of online social networks for automated monitoring system is presented. This model focuses not on communications between users but on text posts. Features of Russian text posts are given. Problem of text posts obfuscation by users involved in illicit fields of activities is discussed.

Keywords

• online social networks
• monitoring
• text analysis
• information retrieval
• fuzzy search

1. Introduction

M . Every post can be

k – text j is text of the post m j , text j  wj1, wj2 ,..., wjg , w ji is the i-th word in text; – typeh Type , Type  type1, type2 , type3 is a set of post types, where type1 is original post (which means that user

Data Science / Yu.B. Savva, Yu.V. Davydova – parent j is a user’s or community’s identifier. If type of current post is a repost or a comment then parent contains identifier of author who posted original message.

P   p1, p2 ,..., pic  is a set of pages of OSNs, number of pages is equal to number of users’ and communities’ identifiers as every page belongs to user or community. Page is defined as follows: pk  ik , ttq , ck , M k  mkz z  1..x , ik  I , x  mc , q  1..2 ,

, ttk  tt1 ck   in  I , n  1..ic

, y textkz ,WFq 

g r y textkz ,WFq     d (wzki , wfqj ) , d (wzki , wfqj )   ,

i1 j1 where: – ik is the identifier of user or community of current page pk ; – ttk TT , TT  tt1, tt2 is a set of pages type. tt1 is a personal page and tt2 is a community page; – ck is a set of user’s identifiers. If current page pk is a personal page then ck is an empty set as page pk belongs to one user. If pk is a community page then ck keeps user’s identifiers who are owners or managers of community (it can be one user, so ck keeps one element);

– M k is a group of posts which are posted on the page pk . It can be empty Mk   , that means OSNs page doesn’t contain any posts at the moment.

Set of keywords is given L  l1,l2 ,...,llc , where lc – the number keywords. Every keyword is represented by its grammatical, semantic information and word forms (according to inflection rules in Russian language) ls  GRs , SMs ,WFs . This keywords storage model was described in [ 4 ]. In this work we are focused on word forms of keywords. They was defined as a language WF over the alphabet A . W F  A  .

The goal of automated monitoring system of OSNs is to find set of pages PF  P which contains required amount of keywords, therefore these pages are indicators of potential illegal actions of their owners. Conceptually it can be presented as   x lc   follows: PF   ik , ttq , ck , M k  mkz z  1..x |    f mkz , lq       k  1..ic ,

  z1 q1   where: – f  mkz ,lq  is a function which is defined as f mkz ,lq   ytextkz ,WFq  ; –  is a threshold of presence of keywords in text posts of current user. It can be defined by decision maker. is a function of fuzzy search matching, conceptually it can be presented as follows: where: – d (wzki , wfqj ) is a distance measure, which shows similarity between two words wzki and wfqj . Initial states are: d (0, wfqj )  wfqj and d (wzki , 0)  wk ,

zi –  is a threshold of distance measure, it can be defined by decision maker. The less is the value of distance measure, the higher is similarity between words. That means that current word in text post is a keyword written with mistakes with great probability. By choosing the value of threshold of distance measure, decision maker can manage the levels of precision and recall of information retrieval. The less is the value the more precise search is, but in this case, some relevant text posts will be lost and recall will be lower.

As the result of Russian text posts analysis from OSNs it was revealed that users use informal style of writing and often neglect the language rules.

3. Features of Russian text posts of OSNs users

Text posts in OSNs have the following features: - use of conversational style in writing, slang and jargon use, abbreviations use; - short length of average text post with weak formal syntactic relations; - use of smileys, different special symbols; - intentional and unintentional garble of words, including spelling errors and typos; - borrowings from English language, like “4u” (For You).

These features characterize modern informal communications, where there is a high speed of information exchange and additional expression. Thus, text posts of OSNs users can be considered as unstructured sequences of letters symbols and images combining with each other. This fact should be taken into account in text analysis. As for communications in illegal fields of activity, additional features should be noted. To avoid detecting by law-enforcement agencies people use jargon. The main difficulty of text analysis in case of jargon use consists in high degree of homonony. Words of common used lexics may be organized in collocations and thus get new semantics as a result. There is a constant appearance of new jargon. The most glaring example is jargon in the field of illicit traffic of narcotic drugs as new substances appear rather quickly. Also it should be considered that OSNs users involved into illegal activities obfuscate posts with the same aim to prevent their detection.

Data Science / Yu.B. Savva, Yu.V. Davydova

4. Problem of text posts obfuscation and methods dealing with it

First mention of obfuscation method appeared in [ 5 ]. Authors suggested confusing of program code by adding extra variables and constructions with aim to prevent algorithm analysis and deter reverse engineering. Also obfuscation can be used to optimize code. Analysis of obfuscation methods of computer program is given in [ 6 ], deobfuscation methods are presented in [ 7 ]. Later obfuscation was applied to creating spam emails, spam messages on different web sites. In this case obfuscation allows to pass through content filtering. Obfuscated words can’t be found during exact matching between words from message and words from dictionary. Dictionary contains words, which are indicators of spam messages.

Text posts in OSNs can be obfuscated by users involved in illegal fields of activity, for instance terrorist and extremist propaganda, illicit drug sales. In this case as it was discussed in [ 8 ] users obfuscate their posts to prevent effective linguistic analysis of texts and avoid detection of their destructive actions and influence on other OSNs users. For text obfuscation generally the following methods are used: - intentional garble of words, including spelling errors, typos, wrong word boundaries (space insertions and deletions); - letter substitution by digits, symbols which look like substituted letters; - insertion extra not meaningful symbols; - transliteration use.

Thus, text posts deobfuscation is the actual and difficult issue. Solution by computer means is not a trivial task as there are many ways of obfuscation of even one word. Thereby such methods as spell checking, deleting non-alphabetic symbols and constructing variants by possible substitutions are not so effective. Applying Hidden Markov Model to the task of spam emails deobfuscation showed good results [ 9 ]. Also, statistical models can be useful, for instance, model based on Bayesian rule, ngram model [ 10, 11 ].

5. Using model of OSNs in automated monitoring system

Automated monitoring system includes the following main subsystems: - data collection; - fuzzy text search which includes linguistic knowledge base, keywords database, algorithmic search and deobfuscation modules; - results processing and report generation modules; - database of text posts and database of search index.

According to model, data collection subsystem gathers identifiers and text posts with additional attributes like type of messages, time and date of posting. This information is stored in database of text messages. Decision maker can specify settings of OSNs crawl strategy.

Subsystem of fuzzy text search takes information from database of text posts and implements the goal of automated monitoring system, trying to detect illegal behavior by using linguistic knowledge base and keywords database. At first stage tokenization is held, text is deobfuscated if it requires. The second stage is fuzzy text search using keywords. The use of linguistic knowledge base helps to make information retrieval not so sensitive to mistakes. Linguistic knowledge base contains information about inflection paradigms, models of mistakes, typos. Keywords database stores grammatical, semantic information and word forms of keywords lexemes. In case some text post contains threshold amount of keywords, it is indexed and is sent to database of search index. Processes of gathering information by data collection subsystem and searching by subsystem of fuzzy text search are parallel. Report generation modules show different slices of results to user of monitoring system such as topic distribution, age and location distribution of OSNs users and some others. Results are grouped according to threshold of similarity distance measure.

6. Results and discussions

At the present time automated monitoring system is to be used in detection of drugs use propaganda and illicit drug sales in OSNs [ 12 ], though system can be used in different fields, it depends on keywords database. Linguistic database of keywords used in the field of illicit traffic of narcotic drugs and psychotropic substances was developed [ 13 ]. It allows to store not only word forms but semantics of jargon. Deobfuscation method using Hidden Markov Model was developed [ 14 ], example of algorithm is presented at 0

Corpus of text posts is gathered from OSN Vkontakte. Currently algorithms of fuzzy search using keywords from developed linguistic database and models for linguistic knowledge base are developed. Features of algorithms and default values for distance measure should be tested on text corpus and corrected in case of need as they are a kind of empirical data because natural language is not a good formalized object [ 10, 11 ].

7. Conclusion

For providing information and psychological security of users, it is necessary to organize online social networks monitoring. Monitoring process has many difficulties like short messages in OSNs, informal communications using jargon, text posts obfuscation. To detect users’ illicit activities and destructive influence effective text analysis and search by keywords should be organized. Thus, in OSNs modeling emphasis should be on text posts, corresponding model was presented in this paper. Main

Data Science / Yu.B. Savva, Yu.V. Davydova subsystems of automated monitoring system using aspects of the model were described. The results of the work done were discussed and features of future work were given.

References

[1] Davydova YuV . To the issue of need for automation of threats search process in virtual social networks and communities. Actual problems in modern science in XXI century: proceedings of the 6th international scientific-practical conference . Makhachkala: “Aprobaciya” Publisher , 2014 ; 25 - 26 . (in Russian)

[2] The top 25 social media monitoring tools . URL: http://keyhole.co/blog/the-top-25 - social-media - monitoring-tools / (19.01 . 2017 ).

[3] Gubanov DA , Novikov DA , Chhartishvili AG . Online social networks: models of information influence, control and confrontation . Moscow: “Fizmatlit” Publisher, 2010 ; 228 p. (in Russian)

[4] Savva YuB , DavydovaYuV. Linguistic database for monitoring system of online social networks in providing information and psychological security. European integration: justice, freedom and security: proceedings of VII scientific and professional conference with international participation: in 3 volumes . Belgrade: “ Criminalistic-Police Academy ” Publisher, 2016 ; 1 : 145 - 154 .

[5] Diffie W , Hellman M . New directions in cryptography . IEEE Transactions on Information Theory 1976 ; IT-22( 6 ): 644 - 654 .

[6] Korobejnikov AG , Kutuzov IM , Kolesnikov PYu . Analysis of obfuscation methods. Cybernetics and programming 2012 ; 1 : 31 - 37 . (in Russian)

[7] Kasperski K , Rokko E. The art of disassembling . SPb: BHV-Peterburg , 2008 ; 892 p. (in Russian)

[8] Savva YuB , Eryomenko VT , Davydova YuV. About the problem of the linguistic analysis of the slang in the problem of the automated search of threats of spread of drug addiction on virtual social networks . Information systems and Technologies 2015 ; 6 ( 92 ): 68 - 75 . (in Russian)

[9] Honglak L , Andrew YNg . Spam Deobfuscation using Hidden Markov Model . Proceedings of the Second Conference on Email and Anti-Spam , 2005 . URL: http://ai.stanford.edu/~ang/papers/ceas05-spamdeobfuscation. pdf (11.07 . 2016 ).

[10] Ingersoll GS , Morton TS , Farris AL . Taming text. How to find, organize and manipulate it . NY: Manning Publications Co., 2013 ; 320 p.

[11] Manning CD , Raghavan P , Schutze H . Introduction to information retrieval . Cambridge: Cambridge University Press, 2008 ; 496 p.

[12] Savva YuB , Eryomenko VT , Davydova YuV . Design of information system identification of persons which participate illicit in field of narcotic drugs and psychotropic substances in the virtual social networks using the database jargon . Information systems and Technologies 2016 ; 1 ( 93 ): 68 - 75 . (in Russian)

[13] Savva YuB , Davydova YuV. Certificate of state registration database no. 2016620197. Jargon in the field of illicit traffic of narcotic drugs and psychotropic substances . Registered 10 February 2016 .

[14] Nikol 'skaya AN, Savva YuB. About the problem of opening of obfuscated Russian-language texts of participants of online social networks . Information systems and Technologies 2016 ; 6 ( 98 ): 44 - 55 . (in Russian)