1 English. Understanding the events and the dominant thought is of great help to convey the desired message to our potential audience, be it marketing or political propaganda. Succeeding while the event is still ongoing is of vital importance to prepare alerts that require immediate action. A micro message platform like Twitter is the ideal place to be able to read a large amount of data linked to a theme and selfcategorized by its users using hashtags and mentions. In this research, I will show how a simple translator can be used to bring styles, vocabulary, grammar, and other characteristics to a common factor that leads each of us to be unique in the way we express ourselves.
Italiano. Comprendere gli eventi e il pensiero dominante è di grande aiuto per veicolare alla nostra potenziale audience il messaggio desiderato sia esso di marketing o di propaganda politica.
Riuscirci mentre l'evento è ancora in corso è di vitale importanza per predisporre alert che richiedono un intervento immediato. Una piattaforma di micro messaggi come Twitter è il luogo ideale per poter leggere una grande quantità di dati legata ad un tema, e spesso auto categorizzati dai suoi 1 Copyright ©️ 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). stessi utenti per mezzo di hashtag e menzioni.
In questa ricerca mostrerò come un semplice traduttore può essere usato per portare a fattor comune stili, lessico, grammatica e altre caratteristiche che portano ognuno di noi ad essere unico nel modo di esprimersi. 1
Each of us has a unique way of writing. However, the fewer options we have to experience ourselves to express our concept, the more the necessary synthesis leads to the loss of precious information to accurately assess our real intentions.
Furthermore, the more the subject is debated, the more changes in style and tone occur. The conversation becomes full of irony or aggressive. Extrapolating a single line is dangerous without context. The same sentence can have different interpretations depending on the moment in which it is pronounced, the audience it is intended for, the place where you are, in the historical period in which it was composed.
My hypothesis is that we can translate all these
different styles into a single "language style" that
fully expresses the real intentions of the writer.
The challenge is to understand when a user has
expressed a comment in favor, against, or neutral
towards the Sardines' Italian political movement.
The research was carried out for the SardiStance
The two tasks are similar. In Task A, it is necessary to classify the stance of a tweet based only on the text of the tweet. Task A is divided into two subtasks: Constrained. It is allowed to use additional resources such as a Lexicon but no other resources (such as labeled tweets) to help the training process. Unconstrained. Where each resource used must be reported in the final report.
In Task B, you can use the context information provided by the post author. Additional information refers: to post statistics (favors, retweets, reply, source) to the author's information (number of posts, number of followers, emoji in the bio) to the author's circle of relationships (friends, replies, retweets, and quotes) The research focuses on Task A Constrained. Considering the constraints of Task A, it is not possible to access any additional information other than the text of the tweet, I concentrated on understanding how to clean it up.
the tweet ID the user ID the text the label
Against Favor Neutral / None To be sure to do not use any data except the text, the user id, useful for Task B, was discarded.
In order to validate my hypotheses, I used the
AlBERTo model, created from tweets,
The first one was to break up the composed hashtags into sentences and words.
For example, using capital letters as a separator: #IoStoConLeSardine has become "io sto con le sardine" ["I'm with sardines"]. #NessunoTocchiLeSardine has become "nessuno tocchi le sardine"["nobody touches the sardines"].
As a second step, I made sure to remove repeated vowels in a sentence, such as: "Svegliaaaa" to get the word "Sveglia" [Wake up!].
I also replaced the word sardines with "PartitoPoliticoS" ["PoliticalPartyS"] to prevent the entity from being mistaken for the fish that is its symbol. I did not remove any stop words because it is useful to create the translation system.
At this point, I made a copy of the dataset to translate it. I used the spaCy4 language functions of POS tagging, Dependency Parse, and Entity Recognition to have all the essential components of my translator.
The translator is a simple text representation. It is a matter of rewriting the sentence following the scheme: subject adjectives subjects verb in the infinitive form adjectives objects objects exclamations / other words At this stage, the words are not modified to make the sentence grammatically correct. Words are exchanged places, only the verb are modified to the
infinitive form. The entities of type person [PER] take precedence over others.
The translator concentrates its attention on the
aspect inside the sentences to be sure to do not
remove valid sentiment polarity words
Finally, I trained two models with the Ktrain framework. The model 1, which use the translated tweets, was submitted as ghostwriter19_Task_A_1_c. The model 2, trained with the only cleaned tweets, was submitted as ghostwriter19_Task_A_2_c. 2.1
The model will be evaluated with the F1-score. The main score is the average of the F1-score of the Favor tweets and the F1-score of the Against tweets.
When comparing the two models, the first result is that the translated tweets performed worse, albeit by a few percentage points (table 1).
ghostwriter19_Task_A_1_c ghostwriter19_Task_A_2_c
Analyzing the results of both the models in detail (table 2 and 3), we have that: ghostwriter19_Task_A_1_c
The problem is evident. Model 1 has a more challenging time distinguishing the favor tweets from neutral ones. The good news is that both the models overcame the estimated baseline. 2.2
Thinking that on Twitter the hashtags are also used for classification purposes, the operation that replaces them was modified. Now the hashtags are added at the end of the new tweets. Also, the mentions are considered and processed as hashtags (table 4).
ghostwriter19_Task_A_1_c ghostwriter19_Task_A_2_c
0.5822 Analyzing the results in detail (table 5), we can see that: ghostwriter19_Task_A_1_c
Neutral 0.70 The model gained two percentage points for both Against and Favor, compared with a one-point loss in Neutral. Unfortunately, it still remains two points below the model 2, with the only cleaned tweets. 2.3
Analyzing the new texts generated, I noticed that essential information was lost by putting all the verbs in the infinitive. If the verb was in the passive form, the subject and object of the sentence were reversed. At the same time, I noticed that very long tweets contained more than one sentence.
I modified the translator to consider passive and active verbs, swapping the sentence's subject and object if necessary. The hashtags inserted at the end of the tweet only left at the end of the new tweet generated (table 6).
ghostwriter19_Task_A_1_c ghostwriter19_Task_A_2_c
0.6306 Analyzing the results in detail (table 7), we can see that: ghostwriter19_Task_A_1_c
The model gained five percentage points for Against and Favor tweets, compared with a onepoint more loss for Neutral ones. Now the translation model is the best model.
Model 1 was ultimately 3 percentage points better than Model 2 with the Training dataset. The best performance of the model was also confirmed with Test datasets, with 2.5 percentage points of advantage. 3.1
The final results with the Test dataset are:
The model 1 is about 7.5% better than the baseline (table 8).
I remember that both models were trained with the autofit option, so without any particular study, to validate whether a "translation" of the original text could bring apparent advantages. 3.2
Although no context information was used, I still proposed the predictions for Task A to Task B. The final results with the Test dataset are:
ghostwriter19_Task_A_1_c ghostwriter19_Task_A_2_c
Even if model 1 was not able to reach the proposed baseline, the difference between the two systems is 0.4% (table 9). The detailed results of the models are showed in the tables 10 and 11. 3.3
Detailed results for Task A
Detailed results for Task B model 1_c 2_c 0.8106 0.8094 0.5012 0.4784 0.3810 0.3778
In a preliminary way, the final results demonstrate that it is possible to obtain an improvement of the predictions by reducing the differences of expression to a predetermined structure.
The system is, however, right now, more efficient
in terms of training times and final scores than
ensemble systems of Bi-LSTM, which were used
successfully up to 2 years ago
The next step is also to optimize the model's training to ascertain that the performance gain is maintained and in what percentage. At the same time, the translator can be improved by switching to a sequence-to-sequence system for a meaningful and efficient text representation that will include, among other things, the change of every words forms accordingly with the grammar and the original intention of the writers (Lewis et al., 2019).