The presented student project outlines a natural language processing pipeline for brand metric comparison in the Twitter ecosystem. Sentiment calculation for an unlabeled data set is demonstrated and calibrated using the statistical Central Limit Theorem as a guidance to anchor the sentiment indicator in a homogeneous market. The process is evaluated by comparing the sentimental market performance of three leading German logistics companies. A support for the value of sentiment analysis for automated customer feedback analysis in real-time is concluded.
in the Twitter ecosystem. To achieve this objective, the hashtag space of
leading German logistics companies was analyzed and collected as an exemplary
use case. An analytics pipeline was constructed and applied to the dataset to
better understand the customer base and approximate overall consumer
sentiment towards the logistics brands, linking a scalar metric to consensus opinion
as outlined in [
The main contribution of this work aims to show how current ideas in natural
language processing may be applied to take advantage of automated social media
brand perception analysis. We would like to outline our course of action and the
thought process used in this applied research project as it may inspire other
research towards the stated problem case. For this purpose, two typical marketing
query questions were choosen as an example to demonstrate the approach:
1. How do leading German consumer logistics brands rank in Twitter user
approval?
2. How can Twitter meta information augment user approval analysis?
Related works predominantly make use of machine learning on labeled data with
the primary goal of method testing and evaluation [28][
The speci c characteristics of brand perception analysis allows for a third option. Munoz & Kumar [23] point out that perceptional metrics help to gauge the e ectiveness of brand-building activities across points of customer interaction. Brand pro ling can be achieved by xing an indicator within a metric of a market and then comparing this indicator to a companies brand and its competitors [23]. Therefore, the aim of the presented approach is to acquire such a polarity indicator. Tools, models and methods from both supervised and unsupervised natural language classi cation and processing become available with this modi ed problem de nition. The constructed data pipeline adheres to this premise and is presented in detail in the following paragraphs. 2.1
Data acquisition was implemented using the Tweepy1 library for the Python
programming language to interact with the commercially available Twitter API2
for developers. Tweepy was chosen for its native ability to handle rate limited free
access tokens and dynamic adjustment of tra c bandwidth. To stay within these
limits and to focus on customer opinion, only the top three logistics companies
for the courier, express and parcel services in Germany where selected, which as
of 2018 are DHL, Hermes and DPD [
First exploration and inspection of the dataset had shown that the Tweepy lter was not su cient to separate the communications concerning the three selected logistics brands in a satisfactory manner. The data contained tweets which did not directly relate to the domain of interest. It became clear that a more in-depth analysis of the communication topics was necessary to further sanitize the results. Tweet topic modeling through utilization of keyword annotations - better known as "hashtags" - was identi ed to hold valuable advantages for the solution of this problem [31][29]. Based on the work presented by Wang, Wei & Zhang [30], a graph model was de ned by the set of all hashtags H = fh0; h1; :::; hmg contained within the dataset, where each hashtag hi represents a node weighted by its global occurence count and is associated with a set of tweets Tk = f 0; 1; :::; ng in which it occurs. The set of edges E consists of a link between two hashtags if they co-occur in the same tweet. The weight of an edge eij between hi and hj is incremented for each co-occurence of hi and hj . The graph model HG= fH; E g was used to isolate the logistics subgraph of interest which reduced T , and thereby the rows of M, to only hold tweets relevant to the research. The "hashtags" column of M was transformed to HG and stored in GEXFformat3. This step made it possible to import the hashtag graph into existing graph analysis software4, providing access to pre-optimized methods. The Yifan
2 https://developer.twitter.com/ 3 https://networkx.github.io/documentation/stable/reference/
readwrite/gexf.html
4 https://gephi.org/
Hu multilevel layout algorithm was chosen for its speed and good quality on
large graphs [
To nally conclude ltering and feature construction, M was extended by the columns "is dhl", "is dpd", "is hermes", "hour" and "weekday". With these extra columns, searching and querying the dataset for the following steps is faster and more convenient. Finding tweets associated with speci c brands is done by simple boolean masking, resulting in desired subsets of M without having to repeatedly parse and search tweet content for each query. Aggregation over time windows is sped up by pre-splitting the complex timestamp format of the Twitter API.
After the outlined data sanitation treatment, actual text analysis was possible. 2.3
The Natural Language Tool Kit (NLTK) o ers the basic functionalities
necessary for natural language processing [
Since algorithms do not by default have any understanding of the emotional
impact of word semantics, sentiment analysis relies on human consensus opinion.
Databases with annotated word polarities between [
If the stemmer was not able to nd a morpheme in SentiWS, syntactical
alternative search is exhausted. Therefore, the actual meaning of the token can
be used to nd synonyms whose sentiment is known. Liebeck [
The lookup process was constructed as follows. Gensim is used to retrieve a
vector space embedding for the token, e.g. the word "house". As stated above, this
vector representation shares a topological vicinity with its contextual synonyms,
e.g "building" or "home". The aim is to nd a spatially close synonym which can
be associated with an entry in SentiWS. Starting from the "house" embedding,
its neighbouring entries are probed in order of increasing distance. For
example, "building" may highlight as the closest approximation to "house". The word
"building" is therefore chosen as an alternative candidate. This candidate is then
tested against SentiWS and if a polarity value could be retrieved, the candidate
is selected. In this case, "building" may not have been a valid alternative, is
rejected and the process continues with the next neighbour in increasing distance,
which is "house". The new candidate is tested in the same manner and can be
successfully matched with a sentiment, leading to its selection as a valid
alternative to "house". The algorithm encapsulates the same process a human would
employ in thinking about other phrasings of the same utterance. Figuratively
speaking, the amount of imagination necessary to come up with another phrase
is continually incremented until a suitable rephrasement is found. This can of
course lead to misleading levels of synonymity if done ad in nitum. Therefore,
it was decided to penalize the retrieved sentiment by the distance between the
original token vector and its alternative, similarly to Kim & Shin [
(1) (2) The complete algorithm to resolve the sentiment value for a token vector t can be de ned as follows: sentiment(t) =
max s2SentiWS\Word2Vec
SentiWS(s)
D(s; t) Note that this algorithm implicitly neutralizes alternatives if they are too broadly associated synonyms: The end behaviour of sentiment(t) ensures the absence of polarity distortion in synonym search.
All tokens in the data were labeled using this process. 2.5
After the tokens of M were given an emotional weight as outlined in Section 2.4
on a word level, further analysis on a sentence level can proceed. Fang & Zhan
[
As a last step before the culminating con ation of all individual token polarities per tweet, negations need to be handled as they signi cantly in uence the 7 https://github.com/ptnplanet/NLTK-Contributions/
tree/master/Classi erBasedGermanTagger
calculated emotion by their valency scopes [
After all negations were handled, it was nally possible to propose an estimated
polarity per tweet. Similarly to the work of Kumar & Sebastian [
Having calculated a value which one might consider "sentiment" is not enough for actual market analyses due to two reasons: 1. The scale - while argumentative coherent and grounded in the outlined rationale - can be understood as a valid indicator, it is still arbitrarily de ned. Its de nition is sound, but given that the scale is supposed to measure levels of human emotion, it needs validation. Such a test would require human evaluation, altering the problem to a supervised interpretation. 2. As Section 2 stated, Munoz & Kumar [23] emphasize indicator xation and anchoring within the metric of a market to achieve brand pro ling. Only then is empirical comparison to the calibrated indicator, and thereby competitive brands, feasible.
These two problems seemingly demand further research and evaluation.
Contradistinctively, it is argued that the combination of both allows for a use-case
speci c solution if the fundamental nature of the underlying data is exploited by
utilizing the broad scope of opinions being uttered on Twitter. This
characteristic permits the introduction of the established statistical Central Limit Theorem
(CLT). The CLT is the observation of the convergence behaviour of
probability distributions of an increasing number of one- or multi-dimensional random
variables to a normal distribution [
"#2018 OR #2019 OR #december OR #january" and is processed through the same pipeline as M, leading to a broad, dispersed set of tweets unrelated to any speci c topic. As these tweets cover a wide range of independent themes and conversational domains, it can be reasoned that the global population sentiment characteristics behave according to the CLT. This theory resembles the missing link between the "arbitrarily" constructed polarity estimation pipeline and actual market sentiment, resulting in the desired indicator described by Munoz & Kumar. Establishing the connection mathematically is possible in a multitude of ways, as long as the link adheres to the following formalism. Since the global sentiment distribution of the general dataset GT should at best follow the listed constraints, its histogram (GTpolarity) should resemble the normal distribution as close as possible. As such, the aim is to nd a projection of GTpolarity which minimizes the error between the histogram and the normal distribution. If such a projection is found, it acts as the calibration metric for the analytics pipeline. Thereupon, the calibrated projection can be used on the actual logistics dataset to infer class labels in relation to the opinion of the general population. If the distribution is discretized at the interquartile ranges (IQR), half of all opinions fall in the central area. These will be considered "neutral" and make up the overall majority. A quarter of all opinions fall left of the rst IQR marker and will be considered negatively extreme. Their class label is "negative". And lastly, the remaining datapoints fall beyond the third IQR marker and are hence labeled "positive" as they express positively extreme sentiment. This baseline polarity will be the ground truth reference and M can be labeled in the same way, using the absolute sentiment boundaries dictated by GT. Subsequently, sentiment analysis and classi cation are concluded and evaluation of logistics opinions is nally possible. 3
For evaluation, the questions put forward in Section 2 were answered using the constructed pipeline.
After discretization, the Twitter class label distribution is normally distributed and zero centered. The IQR markers force the calibration into the CLT assumption. Therefore, specialized tweet topics can be compared by calculating the relative distance between the class label tendencies. 3.1 "How do leading German consumer logistics brands rank in
For the complete logistics dataset M, the class labels deviate from the Twitter baseline GT. Neutral sentiment is 11.92% less present in tweets relating to logistics while positive and negative sentiment are 4.58% and 7.34% above baseline respectively. This observation of increased variance shows that users communicate with higher emotional tendencies towards the topic. It can be concluded that opinions regarding the logistics domain are mostly stated more vigorously. To reduce selection bias, the logistics brands must therefore be compared exclusively within their domain. Otherwise, their relative ranking would be distorted by the overall preconceived notions of opinion. Appendix Figure 1 (top) visualizes this distortion. On rst glance, all three brands perform with high emotional response, skewed towards negative feedback. This issue is the result of the distributional relationship to (GTpolarity). Drawing the conclusion that the three speci c brands perform bad on Twitter is not precise as the entire domain generally provokes the shown response. Due to this implication, a more accurate baseline indicator for performance ranking is the sentiment histogram of the logistics domain. All brands react di erently and more truthfully to the this metric and better conclusions can be drawn, as presented in Appendix Figure 1 (middle). Therein, it can be seen that the relative ranking is now increasingly expressive. DHL performs better than its competitors within the domain, having less negative class labels and more positive class labels than average. DPD and HERMES are negatively skewed beyond average expectation, performing worse than DHL. HERMES exclusively falls behind in both negative (more than average) and positive (less than average) opinions. 3.2 "How can Twitter meta information augment user approval analysis?" The presented comparison solely relies on single tweet content and thus individual sentiment. The Twitter API grants access to information beyond pure textual data. Correlating the meta data to polarity can yield clari ed insight. For example, one of the de ning functionalities of Twitter is the ability to like and/or share ("retweet") opinions of other users. The implications for sentiment statistics are pivotal. Nagarajan, Purohit & Sheth [24] observed di erent levels of endorsement by peer users depending on tweet positivity. Hence, weighting tweet sentiment by the amount of shared and approved peer a rmation links argumentative popularity to brand performance indication. Furthermore, if a large enough dataset is gathered, sentiment can be followed along the chain of retweets, forming an interesting graph traversal problem. It could be mapped out how positive and negative sentiment propagate through the Twitter ecosystem and how these multiplicative patterns di er for brands. For the presented project, the dataset is not large enough to reconstruct such patterns. Nonetheless, each entry of M does contain the integer counts of likes and retweets and these values hold analytic value. Reasoned by the arguments above, the two integers were summed per row and added as a new column labeled "Propagation". The new value resembles the amount of persons sharing the view being expressed in the tweet and can be used as a weight vector for sentiment aggregation. The results of Nagarajan, Purohit & Sheth were observable afterwards aswell. Our observations indicate that tweets in the logistics domain are generally shared less often than baseline, but if they are shared, they are more likely negative in contrast to GT. Due to this nding, the class distributions change signi cantly if polarity propagation is factored into relational brand metrics as shown in Appendix Figure 1 (bottom). User approval leads to considerable ampli cation of disparity. DPD and HERMES shift their distributions to pronounced neutrality. They both reduce negative sentiment slightly but also exceedingly decrease in positive sentiment by a large factor. In contrast, DHL overwhelmingly pro ts from the shared user opinions. Negative and neutral sentiment labels are shifted to an 11.04% increase in positive sentiment.
In addition to likes and retweets, the data also contains the timestamp at which a tweet was written. Especially in the logistics business, time plays an important role and should therefore be targeted analytically. In Appendix Figure 2, a heatmap of aggregated class labels per day and hour, averaged in mean rows and columns, is shown. For aggregation, the labels were interpreted as f 1; 0; 1g for fnegative; neutral; positiveg respectively. In GT, it can be observed that negative sentiment correlates with business hours. Furthermore, negativity peaks towards the end of the business week. Intuitively, non-delivery at the beginning of the weekend may induce disappointment as the customer would have to wait past the work-free days until the next business day. Such a hypothesis could be further evaluated if domain knowledge is introduced.
For the individual brands, di erent observations stand out: 1. DHL: Generally, DHL performs above average on Mondays. Sentiment then declines with progression of the week. Negativity peaks at the weekend. 2. HERMES: Best performance is expected on Tuesdays. Daily peak negativity shifts with weekly progression. Customers tend to express negative feedback incrementally in later hours, peaking at 20:00 Hours. The relation may be connected to the longer business hours employed by HERMES. 3. DPD: No obvious pattern is present except a sentiment low on Mondays 14:00 Hours. Otherwise, the data correlates to the baseline.
The underlying tweet texts and their themes where investigated at the emphasized days and hours but did not reveal any apparent common denominator explaining their occurrence in addition to there mere temporal correlation. These time distributions can serve more appropriate analyses for research with added knowledge of the internal structures of the di erent companies. Then, more precise assumptions can be made about the cause of the observed patterns. The di erent client agents were also investigated in relation to sentiment but did not show any discernible correlation. 4
The outlined process has shown how powerful Twitter can be for sentiment analysis in brand perception polling. Especially when meta data is introduced, insights far beyond classical polling techniques become apparent. The di erent logistics brands have shown distinguishable approval performance and the inclusion of Twitter meta data was bene cial to inquire into these variations. Acquiring a professional API access may be costly at rst, but the increase in data volume and quality can add more capabilities, such as instant metrics or even precise geolocation, a variable directly linkable to geographical key performance indicators in logistics. As such, the real-time data pipeline could be integrated into business intelligence and monitoring systems for anomaly detection and cause-e ect analysis.
Furthermore, natural language processing methods were employed to demonstrate their value for modern-day feedback evaluation. In-depth studies into the many di erent ways to approach language processing problems may highlight even more fruitful pipeline steps. Building a domain speci c language corpus should be the rst obstacle to take in that direction. The current lack thereof discouraged the usage of many techniques like supervised machine learning. The same holds for the limited size of the tested data set. If more comprehensive collection was possible, access to other ways of statistical description and model building opens up. Especially in regard to ner recognition of irony and sarcasm, further improvement of the pipeline is required. In its current form, sarcasm was not adequately recognized. Relative brand metric comparison is still valid, as all brands su ered from this de cit in the same way. Sarcasm seems to be inherently linked to strongly opinionated utterances, which is a reason why the proposed work ow did not rely on emoticon recognition as other works suggest for label inference. Data exploration has shown that emoticons played a predominant role in sarcasm emphasis. This observation may hold value for further research but discouraged their importance for the current student project.
Summarizing, the work supports the assumption that modern social media can have a vital contribution to fast re nement of a brands marketing mix for strategic realignment in real-time. This is a bene t to ensure positive reception of corporate philosophies directly as a reaction to automated analyses of innovative, digital consumer feedback channels, using contemporary research in natural language processing.
Fig. 2. Daily logistics sentiment. 18. Lo er, R., Wittern, H.: Markenwahrnehmung und marken-di erenzierung im zeitalter des web 2.0. In: Markendi erenzierung, pp. 359{375. Springer (2011) 19. Manning, C.D., Raghavan, P., Schutze, H.: Introduction to Information Retrieval, p. 32. Cambridge University Press, New York, NY, USA (2008) 20. Miller, G.A.: Wordnet: a lexical database for english. Communications of the ACM 38(11), 39{41 (1995) 21. Mohtarami, M., Amiri, H., Lan, M., Tran, T.P., Tan, C.L.: Sense sentiment similarity: An analysis. In: Proceedings of the Twenty-Sixth AAAI Conference on Arti cial Intelligence. pp. 1706{1712. AAAI'12, AAAI Press (2012), http://dl.acm.org/citation.cfm?id=2900929.2900970 22. Mueller, J., Stumme, G.: Predicting rising follower counts on twitter using pro le information. CoRR abs/1705.03214 (2017), http://arxiv.org/abs/1705.03214 23. Munoz, T., Kumar, S.: Brand metrics: Gauging and linking brands with business performance. Journal of Brand Management 11(5), 381{387 (2004) 24. Nagarajan, M., Purohit, H., Sheth, A.P.: A qualitative examination of topical tweet and retweet practices. In: ICWSM (2010) 25. Nicholls, C., Song, F.: Improving sentiment analysis with part-of-speech weighting.
vol. 3, pp. 1592 { 1597 (08 2009) 26. Pak, A., Paroubek, P.: Twitter as a corpus for sentiment analysis and opinion mining. In: LREC (2010) 27. Remus, R., Quastho , U., Heyer, G.: Sentiws - a publicly available germanlanguage resource for sentiment analysis. In: LREC (2010) 28. Schweidel, D.A., Moe, W.W., Boudreaux, C.: Social media intelligence: Measuring brand sentiment from online conversations (2011) 29. Steinskog, A., Therkelsen, J., Gamback, B.: Twitter topic modeling by tweet aggregation. In: Proceedings of the 21st Nordic Conference on Computational Linguistics. pp. 77{86. Association for Computational Linguistics (2017), http://aclweb.org/anthology/W17-0210 30. Wang, X., Wei, F., Liu, X., Zhou, M., Zhang, M.: Topic sentiment analysis in twitter: a graph-based hashtag sentiment classi cation approach. In: CIKM (2011) 31. Wang, Y., Liu, J., Qu, J., Huang, Y., Chen, J., Feng, X.: Hashtag graph based topic model for tweet mining. In: 2014 IEEE International Conference on Data Mining. pp. 1025{1030 (Dec 2014). https://doi.org/10.1109/ICDM.2014.60 32. Webber, F.D.S.: Semantic folding theory and its application in semantic ngerprinting. CoRR abs/1511.08855 (2015), http://arxiv.org/abs/1511.08855 33. Westpfahl, S., Schmidt, T., Jonietz, J., Borlinghaus, A.: Stts 2.0. guidelines fuer die annotation von pos-tags fuer transkripte gesprochener sprache in anlehnung an das stuttgart tuebingen tagset (stts) (2017)