The paper describes a combined approach to extraction of a domain-specific sentiment lexicon. At first, an initial version of a domainspecific lexicon is obtained by application of a supervised model. At the second stage, the ordered list of sentiment words is refined using the thesaurus informioant. This combined model is applied to several domains and at last the domain-specific sentiment lexicons are united to create an improved version of the Russian sentiment lexicon in the generalized domain of products.
1 Introduction At the second stage, the ordered list of sentiment words is refined using the thesaurus information, in our
Automatic sentiment analysis of texts is a fascta-se, newly published thesaurus of Russian language
developing technology in natural language processing. RuThes1. We trained a supervised model and tuned a
The task of automatic sentiment lexicon construction combined model in the movie domain. Then
and improvement is a basic tsak for sentiment analysis augmented model was utilized in four other domains.
of texts. There are fnreoely available sentiment Finally, extracted sentiment lexicons from five domains
lexicons for many languages or the quality of sucahre united to generate a high quality lexicon in the
lexicons is desired to be better. For example, in Russian general product domain for
only one automatically extracetd sentiment lexicon has (ProductSentiRus+).
been published [
Besides, sentiment analysis of domain-specific texts In Section 2 we review methods for generating requires adaptation of machine-learning models oserntiment lexicons. Section 3 briefly presents the sentiment lexicons to the target domain [6]. So, some structure of RuThes thesaurus, the Russian newly sentiment words can loss their polarity in specipfiucblished thesaurus intended for natural language domains. For example, such word asevil in the movie processing. Section 4 presents an approach for domain usually refers to the movie plot, but not a user extracting sentiment words in various domains. Section opinion. 5 describes the refinement of the lexicon in the general
Other words can obtain the sentiment polarity in a product domain. To evaluate the quality of the obtained specific domain. For exa,mpwleord киношный general resource extrinsically, we conduct (adjective to Russian wordкино (movie)) can have the experiments on the tweet subjectivity classification task. negative polarity with the meaning"far from the real life". Another example - word атмосферный (adjective 2 Related Work to word атмосфера (atmosphere)) has the positive polarity in art-related domains denoting "creation of a There are two main approaches to sentiment lexicon special mood or feeling" (asatmospheric in English) – extraction: corpus-based and dictionary-based methods. this is a relatively newsense of this word for Russian, of not described in Russian dictionaries.
this
words with each other [5, 9, 10], or appearance them in
specific collocations or lexico-syntactic patterns
Contemporary corpus-based approaches exploit a large
[
the
R reviews as in [6].
relations, which includes terms or hundreds of thousands of useerconomic, military, sports and other fields [7]. from political,
Dictionary-based methods utilize avatiolableWordNet-style resources through attachment electronic dictionaries and thesauri and usually begin several concepts. For example, in the current version of their work from a set of seed Inwor[d3s]. RuThes word пресный is attached to three concepts: SentiWordNet resource is described. It is the result of ПРЕСНАЯ ВОДА (fresh water); the automatic annotation of all the synsets of WordNet where each synset is associated to three numerical ПРЕСНЫЙ, БЕЗВКУСНЫЙ (tasteless, bland in scores that indicate how positive, negative, and neutral taste); the terms contained in the synset are. Different senses of ПРЕСНЫЙ (НЕИНТЕРЕСНЫЙ) (uninteresting). the same word may thus have different opinion-related properties.
sentiment words in a specific domain is described. The results of this algorithm are refined using the iterative procedure on the basis of RuThes thesaurus to obtain a high quality domain-specific sentiment lexicon.
In many studies domain-specific sentiment lexicons are created with corpus-based approaches using various types of propagation from a seed set of words, usually a Such a method is applied to four other domains general sentiment lexicon [6]. An important problem of without additional manual labeling and the results are such approaches is to determine an appropriate seedcombined in a sentiment lexicon in a generalized lexicon, which can depend on the domain. product domain ProductSentiRus+. to
In our study we create adomain-specific sentiment Table 1. Domain-specific collection statistics lexicon from medium-size datasets using multiple features of words and several collections without any Domain Reviews Descriptions co-occurrences between worsd. Then we improve an Movies 28, 773 17, 680 initial sentiment lexicon using sentiment labeling of the Books 23, 883 22, 321 thesaurus concepts in a specific domain practically Games 7, 928 1, 853 without pre-determined seed words. We use only two Digital Cameras 10, 208 920 fixed seed opinionated wordbsad, ( good), other Mobile Phones 30, 620 890 potential sentiment words are obtained automatically from a ranked list of a sentiment lexicon (words ordered 4.1 Extraction of domain-specific sentiment lexicon by the probability of their sentiment orientatbioans)ed on multiple features extracted from domain-specific collections.
3 RuThes Linguistic Ontology a corpus-based method utilizing a trained machinelearning model applied to several domain-specific text
In our study we use RuThes Thesaurus of Russian collections. language. RuThes is a linguistic ontology for natural The first domain-speccifi collection (with high language processing, i.e. an ontology, where cothnecentration of sentiment words) is a collection of user majority of concepts are introduced on the basis orefviews in the domain (review collection)with numeric actual language expressions. For a long time RuThes scores specified by their author.sIn these experiments has been manually developed within various NLP and collections were gathered from the neonliservices information-retrieval projects, and now it is available imhonet.ru and market.yandex.ru in five domains: for public use. The publicly available version of RuThes movies, books, computer games, mobile phones and contains around 100 thousand Russian words adnidgital cameras. The second domain-specific collection expressions [7]. (with low concentration of sentiment words) is a text
If compared to WordNet-style resources RuThes is collection of object descriptions (e.g. plots for movies).
organized as a united semantic net where different parts The overall collection statistics can be found in Table 1.
of speech (nouns, verbs, adjectives) can be text entries Another contrast corpus was a collection of two million
of the same concepts. Each concept has a uninqeuwe s documents. Such a collection is useful for correct
unambiguous name. Concepts can be connected with classification of general neutral words frequent in news.
several types of conceptual relations. In addition,
RuThes includes a lot of multiword expressions useful Using such collections the feature representation is
for applications and terms of so-called Sociopolitical calculated for each word. The set of features includes
domain – a broad domain of contemporary socthiael following feature types [
Books Games worTdos trwaiinthsuptheervisferdeqmueancchyineglreeaartneirng tahlagnoritthhmrese, aliln thCeDaimgietraals 85 92 65.8 66.3 amsosevsiesorrse.viewIf coltlheecrteion wwaesre laabeleddisamgareneumalelnyt by atbwoout tPMhheoobnieles 85 97 73.2 78.6 sentiment of a specific word, the collective judgment General after discussion was used as the final ground truth. As a Product 100 100 90.5 95.2 result of the assessment procedure the list of 4079Domain sentiment words was obtained. The best quality of classification using labeled data was shown by the ensemble of three classifiers: Logistic Regression, Words from the ranked sentiment list are quite LogitBoost and Random Forest from WdifEfeKreAnt relative to RuThes descriptions. Some words programming package. are not described in RuTh,ese.g. three of the most probable sentiment wordsin the movie domain are
The result of this corpus-based method is a ranked absent in RuThes, others are mentioned in list of domain-specific words ordered by the probability collections exactly in the same senses as described in of their sentiment orientation – fusrtehnetriment RuThes, the thirds (e.g.atmospheric) are described in weights. The algorithm boosts sentiment words to have RuThes but have an additional (or the other) sentiment high weights (to be closer to the beginning of the list) polarity. So we should try to correct the word order in and neutral words to have low weights. the sentiment list carefully applying
So in the movie domain in the list of more than 18 descriptions. thousand words the following words are located in the first positions:
Frequency-based: collection frequencyw,ith the model described in the previous subsection; document frequency, frequenyc of capitalized words, however, a similar input can be also generated with frequency of co-occurrence with polarity shifters (no, other methods. not), TFIDF;
high-opinionated position in the list.
The main idea of the lexicon refinement is to label conceptual subgraphs of the thesaurus network sentiment or neutral and use this labeling to reorder the initial sentiment list. This process in contrast to such a method as Label Propagation [8, 11s]hould be also regulated with previously obtained sentiment weights of 830th, words.
Evident sentiment adjectives of the movie domain where all words are orderedby their sentiment weights пресный and безвкусный (both are translated into (sw). Initially the algorithm forms two sets of thesaurus English as tasteles)s take even higher opinionatedconcepts using words from the both sides of the list WD: positions: 139th and 193th . Bu ttheir noun derivations Ls – concepts supposed to be opinionatedL,n – neutral пресность, безвкусие, безвкусность, and безвкусица concepts. With this aim the initial average sentiment are less successfПulр.есность, безвкусие, weights csw for all concepts containing words fromWD безвкусность, are absent from the list because of low are calculated. Then the algorithm adds toLs concepts frequency; безвкусица takes 1515th place in the list. So with the high average weight (csws > 0.85) and also two thesaurus-based improvements may be quite possible. pre-defined concepts, corresponding to senses of words
The obtained model was applied to four othbeard and good. domains (books, games, digital cameras, mobileConcepts with the low average weight (cswn < 0.05) phones) without any additional manual efforts. Thaere added to the set of neutral concLepn,ts which quality of extracted sentiment lexicons was measured formed without any pre-defined concepts. using precision measures and presented in the Baseline thresholds for csws and cswn are obtained from columns of Table 2. experiments. 4.2 Refinement of domain-sentiment sentiment lexicons using RuThes thesaurus
Further, every setLs (and Ln) is iteratively augmented with concepts using two conditions: the average sentiment weight threshold and the number of sentimdeinrtect thesaurus relations to the existing sets. Formally, foLrs and Ln are calculated as shown in Algorithm 1 of listtihneg. The algorithm uses also the following additional notation: Adj (L) is a set of direct-link neighbor concepts to set of concepts L;
nlink (C, L) is a function returning the number of direct thesaurus relations between concept C and set L.
corresponding to Ls concepts are modified multiplying them by factork1 (k1 > 1) and all words corresponding to Ln are multiplied by fackt2or (0 < k2 < 1). The resulting list is reordered by weight.
Low-frequent words (with the frequency less than 3) summed up using a formula intended to boost words of the source domain collection are absent in the initial that occur in many different domains and have high ranked sentiment list and therefore do not have anyweights in each of them. sentiment weights. The initial sentiment weights of such Thus, for combining multiple weighted word lists words are calculated as the average sentiment weights the following formula was used: fcorofenqcuecepontntsc,seypntiosnnytmthuesryno,r frreaolrmaeteadvcearlatcogu.elawteTedhigehftsrowomefingehoitgsthhebroo,rf mthoersee R(w) max( pdroDbd (w)) dD D1 1 posdd (w) , concepts in the labeling process.
5 Improvement of General Sentiment Lexicon Using RuThes Thesaurus
Integrating sentiment lexicons from various
productworodrsiented domains it is possible to create a general
sentiment lexicon in the broad domain of products and
by
services. Such a lexicon for Russian was described in
[
Input: concept list with sentiment weights csw Output: Ls, Ln
Ls = {Cbad, Cgood} Chigh, Chigh={Ci: csw(Ci)>0.85},
Ln = Ln Clow, Clow = {Ci: csw(Ci)<0.05} θ = 0.1, Nlink = 3, Ls_iter = Ls, Ln_iter = Ln while θ<0.6 for C Adj(Ls)
if nlink(C, Ls) > Nlink && csw(C)>0.7-θ
then Include(C,Ls); end for C Adj(Ln)
if nlink(C, Ln) > Nlink && csw(C)< θ end
then Include(C,Ln); if Ln == Ln_iter && Ls == Ls_iter
then Nlink = Nlink-1; if Nlink == 0
then θ = θ + 0.05, Nlink = 3;
Ln = Ln_iter, Ls = Ls_iter end where D – is the domain set with five domains,d is the sentiment word list for a particular domain and d is the total number of words in this list. Functionpsrobd (w) and posd (w) are the sentiment probability and position of the word thine lisdt . Precision@1000 of ProductSentiRus was eporrted as 90.5%. Similar combination of improved sentiment lexicons in the new resource (ProductSentiRus+) yields 95.2% in terms of Precision@1000 (Table 2).
in [10], which include two thousand tweets in Russian. We assumed that ProductSentiRus+ comprises sentiment units of Internet language. A tweet was classified as subjective if it contained at least one word from the lexicon. Table 3 demonstrates that such a generalized lexicon can be useful also in tweet subjectivity analysis.
After application of this algorithm in the domain our example wordпsресный, пресность, безвкусный, пресность, безвкусица have the following places in the generated sentiment lisпt:ресный – 81, пресность – 86, безвкусный – 115, безвкусность – 172, безвкусие – 173, безвкусица – 943. movie
extraction of domain-specific sentiment lexicons. At first, an initial version of a domain-specific lexicon is
The words related to the neutral sense of woorbdtained by application of a supervised model. At the пресный – ПРЕСНАЯ ВОДА (fresh water) preserved second stage, the ordered list of sentiment words is their very low positions in the sentimleisntt: вода refined using information described in RuThes (water) – 23059, айсберг (iceberg) – 26124.
P – –
R – – thesaurus of Russian language, which was late[l5y] Vasileios Hatzivassiloglou and Kathleen R published. McKeown. Predicting the semantic orientation of
This combined model is applied to several domains adjectives. In Proceedings of the 35th Annual and at last domain-specific sentiment lists are united to Meeting of the Association for Computational create a sentiment word listin the generalized domain Linguistics, 1997. P. 174–181. of products – ProductSentiRus+, which is an improved [6] Raymond Lau, Chun-Lam Lai, Peter Bruza, Kamversion of the only published Russian sentiment lexicon Fai Wong. Leveraging web 2.0 data for scalable and will be also publicly available. The proposed semi-supervised learning of domain-specific approach can be applied to other languages and can sentiment lexicons. Proceedings of the 20th ACM utilize other thesauri. international conference on Information and knowledge management. ACM. 2011.
Acknowledgments [7] Natalia Loukachevitch and Boris Dobrov. RuThes 07-0T0h6is82w. ork is partially supported by RFBR grant 14- PLrinocgeueisdtiincgOs notfoGlolgoybavlsW.RoursdsniaentCWoonrfderneentsc.eI.n2013.
[8] Delip Rao and Deepak Ravichandran. SemiReferences supervised polarity lexicon induction. In Proceedings of the 12th Conference of the European Chapter of the ACL, EACL-2009, 2009. P. 675– 682. [9] Leonid Velikovich, Sasha Blair-Goldensohn,
Kerry Hannan, and Ryan McDonald. The viability of web-derived polarity lexicons. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, 2010. P. 777–785. [10] Svitlana Volkova, Theresa Wilson, and David
Yarowsky. Exploring sentiment in social media: Bootstrapping subjectivity clues from multilingual twitter streams. In Proceedings of the 51st Annual Meeting of the Association of Computational
Linguistics (ACL13), 2013. P. 505–510. [11] Xiaojin Zhu and Zoubin Ghahramani. Learning from labeled and unlabeled data with label propagation. Technical Report CMU-CALD-02107, Carnegie Mellon University. 2002.