This paper presents the results of a study on determining the rating of politicians based on a dataset collected from Twitter, comparing it with opinion polls. It is shown in the example of the analysis of the rating of the President of Ukraine Volodymyr Zelenskyy for the period from January 2019 to March 2021 that the differences in the rating can reveal events that influenced its change. Based on the specific reasons for the drop in rating, you can provide recommendations on how to stop the drop. On the other hand, the search for rating growth reasons can be used to determine the ways of increasing the respective politician's rating. To avoid misleading information and to verify the accuracy, detected Twitter events were compared to Google Trends and their consistency was confirmed.
Politicians are carefully considering the ratings. A lot of money is spent on sociological polls. Their implementation requires the involvement of various specialists and institutes that conduct thorough research, spending a lot of time on it. But times are changing. The massive diffusion of social networks provides opportunities for researching electoral preferences of different categories of potential voters. An automated algorithm could significantly reduce the time, money and number of people involved in sociological polls.
Such polls often make it difficult to understand which events have affected the rise or fall of the rating. If the poll is conducted once a month, then during this period there are usually many events. Which of these events had a decisive influence on the opinion of a particular respondent can only be known if the authors of the poll have provided a corresponding specific question on such an event or a group of relevant events.
By researching social media data, it could be aggregated for different periods, it can be a month, a week, a day or even an hour. Statistical counting of the number of messages on the social network will provide important information, indicating which topics are most discussed. This responsiveness and flexibility allow us to highlight key events and recommend how to respond to them to improve society’s response.
In this paper, we present the challenges and solutions in the following structure. Section 2 contains the literature review on topics like our study. Section 3 explains the approach to determining the political rating and the reasons for its changes. Section 4 presents the experiment, results, and discussion. The outcome is compared with sociological polls and Google Trends. Section 5 contains conclusions and future work.
Perhaps, Walter Lippmann was the first to theoretically substantiate the influence of the classical
mass media (press, radio, cinema, or, inferentially, television) on the political preferences of citizens
back in 1922 [
The idea of using social networks to calculate political ratings also is not new. Interestingly, the first
studies of this kind (see, for example, [
Researchers often use such popular social network as Facebook. Stephen R. Neely in [
The vast majority of articles that explore the relationship between the popularity of a politician or
some political force and their social activity or the activity of their supporters/opponents in online social
networks are based on an analysis of a snapshot of relevant messages. The first article in which such an
analysis is done based on data collected over four years was published only last year [
In social networks, users can act alone and even unite into groups. In [
Objectives: This study sets in a certain sense the inverse problem and it intends (i) to confirm the possibility, based on the activity of some users of a social network during a sufficiently long period (from six months), to determine how popular this or that political force is in the electoral sense, and (ii) to find out and analyze: is it then possible to identify the events that led to a change in the corresponding political rating? 3. Approach to determining the political rating and the reasons for its changes
To determine the political rating of Ukrainian President Volodymyr Zelenskyy, the Twitter dataset
from our previous research [
Another improvement in the formula was the greater importance of retweets compared to likes.
Therefore, we multiply them by an empirically selected coefficient equal to 2. The same multiplier for
retweets was used by researchers of Donald Trump’s activity in [
The next important change in the formula is the use of the sum of the natural logarithms of each component instead of the product. This change allowed us to consider all the terms separately and apply the coefficient equal to 4 so that the influence of the account was not dominant compared to the other two terms. The coefficient was chosen empirically to obtain the best result. Since the tweet score can be less than one, instead of the usual logarithm we use the logarithm of 1 + tweet score. log 1 log log 4 (2)
To detect dates of opinion changes the data is arranged into shorter periods. Initially to weeks. Once the algorithm detects the week with anomaly rating changes – rapid growth or decline of the chart, then the weeks are split by days. For each period rating changes, which may be not only one day but several days in a row, statistics of the most common words were collected. For each word, except for stop words (a set of commonly used words in any language), a number of occurrences in popular news got calculated. The words which occurred in the news the most were called keywords and got stored as potentially important in terms of political rating impact. To avoid false keywords selected in the previous step they could be double-checked in Google trends. Once the dates of tweets and Google Trends fit then specific news that affected the rating received. On some dates, the political rating may be affected not only by one event. Sometimes ratings changed after several positive or negative news. Therefore, it is important to collect all popular news.
The new approach allowed us to achieve results 4% better than with the original formula. The total deviation of the rating is 16%. Calculated ratings from Twitter data and sociological poll results are presented in Figure 1. Black bars represent the results of the algorithm, and the dashed grey line shows sociological polls’ result2.
2 https://ratinggroup.ua/files/ratinggroup/reg_files/rg_ukraine_covid_cati_ix_wave_022021_press.pdf
4.2.
To recognize the reasons for the fall in rating, data aggregation was carried out not by months, but by weeks. The first detected drop in the rating is a consequence of the beginning of the COVID-19 epidemic in Ukraine. Many people became ill, and on March 25th in the year 2020, the Cabinet of Ministers of Ukraine imposed a 30-day state of emergency across the country due to the spread of coronavirus disease. Examples of tweets with negative scores for observed dates are presented in Table 1. Calculated by proposed model rating changes for this period are presented in Figure 2. Table 1
decision of the Cabinet of Ministers, by which he actually introduced a state of emergency in the country, bypassing the President and the Verkhovna Rada. No "good intentions" can be the basis for violating the Constitution of Ukraine.
model
Another rating loss was detected when on July 14, 2020, the Council legalized the gambling business. Rating changes for this period are presented in Figure 3. 4.3.
To recognize the reasons for the growth of rating, data aggregation was carried out not by months or weeks, but by days. The first detected growth of rating started on the 25th of January year 2021 and exposed interesting reasons on how algorithms could be improved. Volodymyr Zelenskyy was born on the 25th of January and received on that date and a few days after a lot of good congratulatory words. Our algorithms accidentally detected such behavior as the rating grows. To improve the proposed approach, the major dates from personal politician life such as birthdays, marriage or family celebrations should be removed.
The next rating growth was detected on the 2nd of February year 2021. On that date, The President of Ukraine Volodymyr Zelenskyy put into effect the decision of the National Security and Defense Council on the application of sanctions against the People’s Deputy Taras Kozak and the TV channels 112 Ukraine, NewsOne and ZIK, which were blocked. The deputy and his TV channels carried out antiUkrainian activities.
The last rating growth was detected on the 16th of February year 2021. This one is interesting because is a result of multiple news on the same date which is alone not so powerful as the previous one but gives a significant growth as an aggregated result. On that date: Volodymyr Zelenskyy had an official visit to the United Arab Emirates and agreed on agreements and memoranda worth more than three billion dollars, cooperation in various fields, from defense to agriculture, foreign direct investment in Ukraine and readiness to increase interstate trade several times. Examples of tweets related to this news are presented in the first two rows of Table 2; Volodymyr Zelenskyy announced the reduction of the powers of the Kyiv District Administrative Court. An example of a tweet related to the announcement is presented in the third row of Table 2; The speaker of parliament Dmitry Razumkov said that deputies will start to be left without mandates for “button-pressing”. An example of a tweet related to the speaker’s words is presented in the fourth row of Table 2. Table 2
2021‐0‐15 13:47:18 2021‐02‐14 18:44:28 2021‐02‐13 16:35:01 2021‐02‐13 10:29:44
cooperation with the UAE. Among the initiatives are weeks of Ukrainian cinema and days of folk art in the
4
When several news items fall upon the same date, it is important to understand how much each of them affected the result. To do this, we calculated the statistics of the frequency of news data. The results in Figure 5 show that the news about the United Arab Emirates was much more resonant than the other two.
Google Trends could be used to verify that the right news is selected. This tool will allow checking which news has been popular in each period and whether it coincides with the results obtained based on data from Twitter. Figures 6-8 show that news detected by an algorithm using Twitter data fell on the same dates as they appeared in google search in Ukraine. Some news could start one or two days earlier than got discussed on Twitter. The administrative court and button-pressing are highly discussable topics in Ukraine, even if they were not the absolute maximum on the 14th of February they were still on the local maximum.
The conducted research proposed several algorithms to determine the rating of politicians, detect dates when news affected ratings the most and identify specific news which influenced grows or fall of the rating. Experimental results conducted on Ukrainian President Volodymyr Zelenskyy’s page on Twitter show that the proposed approach allows not only to detect of ratings and their changes but detect news that influenced such changes the most.
The result is slightly different from sociological polls. There are several explanations for this: Twitter is not very popular in Ukraine; Not all segments of the population participating in the elections use this social network.
For example, there are very few elderly people; Twitter is used by people who do not yet have the right to vote ― minors.
In future work proposed model will be evaluated on a calculation of political rating for French elections candidates – Emmanuel Macron and Marine Le Pen. The conducted research confirms that it is possible to identify the events that led to a change in the corresponding political rating. In future work, we plan to develop a system of recommendations for politicians or commercial brands based on the identified key events on how to react to news or informational attacks in order to avoid or decrease the loss of rating.