Sentiment analysis is a class of text analysis methods in Natural Language Processing designed to automatically identify emotionally coloured vocabulary and emphatic evaluation of authors regarding objects in the text. Businesses are using such methods to analyze the competitive environment and monitor the customers' opinion. In this paper, we present Media Placement news analysis tool targeted at measurement of brand recognition and media coverage in online media.
The world economy is increasingly determined by industries with services and
intangible assets put in spot light. Nowadays, intangible assets such as brand
recognition, experience and tacit knowledge of employees, relationships and
informal commitments bring for most companies a larger share of their total value
than tangible assets such as equipment and infrastructure. Intangible assets and
their e ective management are the key to success in the long term, and many
research papers have been devoted to this subject, see e.g. [
Brand reputation of a company is a ected by various factors including the
product performance [
As a pilot project, there has been conducted automatic news analysis of Krasnodar region (Russia) local media. As a result of this research we have estimated brand reputation pro les of major corporations which are operating in southern Russia. 2
Our overview is based on paper [
All the above methods have been used for business purposes. We see the forth approach as the most e cient and promising. 3
Media Placement Startup based in Moscow, Russia, ventures to o er a complex sentiment analysis tool to big companies so that they get an idea of where they are in terms of brand recognition and media coverage compared with their competitors.
In this paper we present our pilot project, which is based on news data from Russian open new media sources such as RBC4, Krasnodarmedia5, Kommersant Yug, Delovaya Gazeta. 3.1
For company recognition we used a simple but e ective approach based on dictionaries and rules because there aren't a lot of ways to mention a company in the text compared to a more general problem called entity recognition with using more complicated methods such as neural networks.
Firstly, we collected a dictionary of company names. To do that, we parsed a website of top 500 Russian companies6. Then we applied our rules to detect potential company names, obtained the top of the most frequently run across company names in our data, excluding companies which are in the dictionary, veri ed them manually and added to the dictionary. We split our dictionary into two parts due to the fact that some companies have names which do not represent a unique word, i.e. a word has a meaning and used in context not a as company name, or some companies have the same names as another companies (but in a di erent domain), projects, funds and etc. First part is a dictionary 4 http://rbc.ru 5 https://krasnodarmedia.su/ 6 https://ru.investinrussia.com/russia-200 which contains \strong" names that are unique and used only as a company name, for instance, \Sberbank". The second part is a dictionary with \weak" names which will be used not only as a company name but also as a regular word or name of a project or fund etc, for example, \Saturn" or \Vozrozhdenie". We are not sure in names from the second dictionary, so we applied some veri cation rules which will be described later.
As mentioned above, we used simple rules to detect potential companies. We split rules into two types: \strong" rules which we are sure in, and \weak" ones which we aren't sure in and it's just a possible company's name to be veri ed manually. The strong rule description assumes it's a word in quotas which are followed by special word determined type of organisation, for instance \OAO", \PAO" and etc. It is very strict rule and we've never seen false positive detection, but it captures very limited number of companies. The second rule is weak and we determine this as words in quotas and there is some special word in a window of this words such as \company", \bank", \fabric" and etc. Verifying rules for a weak dictionary are almost the same: if the name is followed by one of the special word such as \OAO", or there is a special word in a window around such as \company" or \enterprise". Then, we introduced some minor additional rules, such as verifying that the rst letter is capitalized, a potential company's name (weak rule) is less than 5 words to avoid detecting some quotes which may include some special words around, if there are more open quotas than close (and obviously in this case some of them are nested), that rst close quota close two last open, checking that detected company names aren't nested and if they are, we'll remove the shortest.
As a result, with these two simple approaches we covered a big part of company names which would do for the MVP stage. Finally, we decided that existing datasets for NER consist a lot more wise de nition of organisation than just companies, so it's not suitable way to estimate perfomance of our algo. As a result, we marked our parsed data manually and estimated the algo. We got 87% of accuracy, 80% of recal and precision equals 93%. 3.2
As a rst step of analyzing news we picked binary classi cation for sentiment analysis. For that we used open dataset for sentiment analysis from linis source, trained model on this and then applied to our news data. Learning dataset includes 5 classes from negative to positive, we treated them as all negative (-2, -1 classes) as negative and the rest (including neutral) as a positive, because we have hypothethis that if company is mentioned even in a neutral context it's a good, because people will remember the company and when they want to buy some goods they will think about this company rstly neither about their competitors. Then we tried two approach to build nal model.
First approach is a simple baseline, we use TF-IDF encoding over normalized words (with pymorphy2) and logistic regression over the encoding. Quite a tricky task was to tune the parameters such as regularization, type of optimization, type of a kernel etc. We split learning dataset into train and test and estimated model quality on test part by AUC-ROC, then chose the best and the simplest approach: logistic regression with a linear kernel and next hyperparameters: penalty = l2, C = 1, solver = newton-cg, nal score of the model is 0.74. Then we applied the model on our news data and manually read some news and scored. Finally we got approximately 0.85 AUC-ROC score.
As for the second approach, we used the pretrained fastText model for word embedding and full text vectorization. To vectorize the text, we comupted coordinate-wise average, maximum and minimum of all words and concatenate this statistics into one vector. After that we apply logistic reggresion and chose hyperparameters as in previous approach. Our setting is logistic regression with penalty = l2 , C = 10, newton = cg optimizer. That brought about the improvment of our nal AUC-ROC score from 0.85 to 0.89. This approach perfoms much better and seems good enough to move further to do more comlicated analysis such as splitting sentiment into parts and analyse which of them if more improtant and etc.
Model
ROC-AUC Final ROC-AUC The presented project is currently under development. We also consider other areas of application for the techniques described above. One possible direction is analysis of medical records.
We would like to thank Dr. Oleg Lavrov for his participation and input, and very valuable comments on this work. 1.00 0.95 0.90 0.70 0.65
FastText Tf-Idf 0.0 0.2 0.4 Recall