Topics in a corpus evolve in time. Describing the way this evolution occurs helps us to understand the change in the prominence of concepts: one can gain intuition about what concepts become more important in a given topic, which substitute others, or which concepts become related. By de ning topics as weighted collections of concepts from a xed taxonomy, it is possible to know if said evolution occurs within a branch of taxonomy or if hitherto unknown semantic relationships are formed / dissolved with time. In this work, we analyze a corpus of nancial news and reveal the evolution of topics composed of concepts from the STW thesaurus. We show that using a thesaurus for building representations of documents is useful. Furthermore, the di erent abstraction levels encoded in a taxonomy are helpful for detecting di erent types of topics.
Of the many dimensions that can be ascribed to text corpora, the topics that they deal with is a very intuitive one for human readers. In a sense, topics constitute subgraphs of a \platonic knowledge graph", where only certain concepts and certain relations exist. Two qualities of topics are of particular interest: they re ect the intentions and context of the author of a text, and they are often treated in di erent texts by di erent authors. For these reasons, understanding their evolution in time can be treated as a proxy for studying the context of the authors.
In the case of corpora of news articles, understanding the evolution of topics can give insights into which entities become important in a given topic, or how they loose their importance. Furthermore, the detection of emerging and fading topics can be of interest for signalling major events.
In this work, we approach the study of topic evolution by using controlled, semantically enriched vocabularies. With our method, it is possible to describe the topics present in a certain time in terms of the topics present in the previous time points. With this in hand, the method is able to recover stable topics that are consistently dealt with in the news. Furthermore, detection of important events that shift the composition of topics is also possible. Finally, we perform an analysis of the topic-identi cation power of di erent levels of abstraction, as de ned by a thesaurus. 1.1
Topic discovery, also known as topic modelling, is the task of analyzing a corpus
and extracting from it clusters of terms that are semantically related.
When approached with statistical tools, the semantic relationship of the
discovered topics is deduced from their distributional properties: terms that co-occur
more often are deemed to be semantically related. Usually topic discovery is
approached by rst representing documents in terms of bags of words, n-grams[
In the above described scenario, the only semantic relations between the terms
that we have access to are those statistically discovered based on the
documentterm matrix. However, in many applications further semantic relations are known
between the terms. For example, if information about synonyms is known, topic
detection can be done by counting occurrences of synsets[
In this work, we aim at incorporating further semantic information into the topic discovery process by use of a thesaurus. A thesaurus is a controlled vocabulary whose concepts are organized according to their hypernym/hyponym relations. In e ect, it is a multihierarchical directed graph, where a node represents a concept and edges represent hypernym or hyponym relations. Each concept is assigned one or more labels: strings that can be matched against a document. 1.2
Studying topic evolution can be seen as a study of the history of ideas [
In this work, we present an intermediate approach. Topics are discovered independently for each corpus, and corpora in successive times are analyzed to determine in which ways did the topics transit into others or appeared de-novo. This second step allows us to describe the evolution of topics not just as the evolution of sets of co-occurring terms, but rather as the merging and splitting of existing topics. Hence, we are enabled to de ne the notion of persistent topic, i.e. topics that appear in several consecutive corpora. 2
The dataset we analyze is a nancial news data set. The news come from a single
source (Bloomberg news) and are made available as supporting data in[
The semantic relationships between concepts that we are using in this work
are those expressed by skos:narrower and skos:broader predicates in version
9.02 of the STW Thesaurus for Economics[
Entity extraction Entity extraction was performed using PoolParty Semantic Suite1. In brief, the texts are pre-processed in the same manner as the labels 1 poolparty.biz from the thesaurus, namely, stopwords are removed, tokens are lemmatized, ngrams up to 4-grams are constituted. Then, a matching is done to identify all the concepts appearing in the documents. Thus, for every concept c in the thesaurus and every document, we have computed nd(c), the number of times any label of c appears in document d. Finally, all the documents corresponding to the same week were put together into a single corpus, which we will denote by Cw, where w is the week number.
Representing documents For each document d in each corpus, two vector representations are computed.
The rst, which we call level 0 representation, contains information only about a subset of all concepts that have no narrower concept in the thesaurus. We call this set the set of leaves, and denote it by l1; l2; ::::; ; lm0 . The level 0 representation of a document d is then a m0-dimensional vector V0(d) whose i'th entry is given by V0(d)[i] = ndn(dli) , where nd is the number of tokens in document d. The second representation, called level 1 representation, contains only information of those concepts that are broader concepts of some leaf. If we denote that set by b1; b2; ::::; bm1 , then this representation consists of a m1 dimensional vector, whose i'th entry is given by
V1(d)[i] =
X
l2L(bi)
nd(l)
nd
(1)
where L(c) denotes the set of nodes which are narrower than concept c.
It must be noted that with both of these representations only the occurrences of
leaf concepts is considered. The di erence being that in level 1 representations
the occurrence of concepts that are narrower of one same concept are grouped
together. With these two representations de ned, we can represent each corpus
Cw by two matrices: A0(w) and A1(w), both of which have as many columns as
documents in corpus Cw, but with m0 and m1 rows respectively.
Detecting topics in a week For each week w, we compute a Non-Negative
Matrix Factorization (NMF)[
Detection of Topic Transitions For a representation q 2 f0; 1g and two consecutive weeks we get from the previous steps the matrices Tq(w) 2 R+mq k1 and Tq(w+1) 2 R+mq k2 , where k1 and k2 are the numbers of topics in di erent weeks. In order to detect transitions between topics in these two consecutive weeks, we solve the optimization problem of nding the matrix Mq(w) 2 [0; 1]k1 k2 that minimizes jjTq(w)Mq(w) Tq(w + 1)jj2. The resulting matrix Mq(w), which we call transition matrix, expresses each topic in week w + 1 as a linear combination of the topics in week w. The following points help in the interpretation of transition matrices: { If two topics merge into a new topic in the next week, then the new topics column will have large entries in the rows corresponding to the two previous topics. { A new topic in week w + 1 will correspond to a column whose entries are all small: it is not similar to any topic in the previous week { A topic whose concepts don't change between two consecutive weeks can be detected by a column and row that both have a single entry close to 1. { One can think of topic transition as the process of the topic from the previous week distributing its weight into the topics of the current week. The topic cannot give more weight than it has.
With a set of consecutive transition matrices Mq(w); Mq(w + 1); : : : ; Mq(w + g), it is possible to detect topics which remain stable for several weeks: they will be a sequence of indices t1; t2; ::::; tg) such that 1 Mq(w)[ti; ti+1] for i = 1:::g 1 and some small . We consider a topic to be a stable topic if the above condition holds for g >= 5 with = 0:2, i.e. if the topic keeps approximately with the same concept composition for at least 4 weeks. Figure 1 is an example of a set of transitions matrices that exhibit a stable topic. 4
We decomposed all corpora into topics leading to di erent number of topics per week (Fig 2). After computing the corresponding transition matrices we are able to detect the appearance of new topics as well as several stable topics. Among them, we can talk of persistent and ash topics.
50 s40 c i p to30 f o re20 b m uN10 0
Representation 0
Representation 1 total new s40 c i p to30 f o re20 b m u10 N 0 total new 0 50
100 150 Week number 200 250 0 50
100 150 Week number 200 250
Persistent topics are topics that the news source treats regularly. While interruptions in the data (weeks with few articles) sometimes fragmented these topics, in the sense of our de nition of stable topics, they were quick to recover. Several stable topics represented through their most important concepts are presented in Table 1. Furthermore, we were able to detect Flash topics, that is topics which relate to speci c, transient events. We found the following ash topics in level 0 representation: 1. the 2010 South Africa World Football Cup, 2. the 2010 artillery re exchange in the Korean peninsula. 3. the 2011 Drought, 4. the Arab Spring, 5. the Fukushima Daiichi nuclear disaster, The most frequently mentioned concepts in these topics can be seen in table 2. The evolution of topics can best be exempli ed by the Football World Cup example. In table 3, the change in the topic across weeks is shown. Notice how the number of countries decreases, and those which remain are also those which remained in the tournament. Let us recall that the tournament ended on the 11th of July.
Many of these topics were found also in the level 1 representation. In general, the level 1 representation yields longer stable topics, as can be seen on the length distributions of stable topics in Figure 3. It is worth mentioning, that a topic which is very fragmented (in time) in the level 0 representation is less so in the level 1 representation: that of the Japanese market. This suggests that looking at more abstract concepts can increase stability of topic detection. It is important that, while using the broader concept of a set of leaves increases stability, there is no evidence that similar topics are confounded by this process. This is an indication that topics are not necessarily matching branches of the taxonomy but, rather, are combinations of topics from across the thesaurus. For the same reason, topics in consecutive weeks which are not deemed persistent under the level 0 representation, become so in the level 1 representation. It is thus important to choose carefully the level of granularity of the concepts that will be used to annotate a corpus with the aim of persistent topic detection. It must also be noted that detecting topics based solely on the words (i.e. without a controlled vocabulary) does not provide this possibility.
Finally preliminary results show that, on average, concepts gained by a topic during its evolution are closer in the thesaurus to already existing topics that is expected at random. 5
We have presented a method that is able to detect both persistent and transient topics in news sources. Interestingly, we have shown that it is possible to detect such topics both when annotating documents only with leaves from a thesaurus, and when annotating them also with those concepts directly above leaves. Furthermore, we have shown that the relatively simple NMF method is able to detect stable topics, and that this stability can also be captured by our proposed method for computing topic transitions. By considering the news articles in each week as independent corpora, we are able to detect short-lived topics, that would otherwise be lost in a global topic modeling.
The fact that topics are detectable in both representations is an indication that topics are not easily confounded with each other when one considers more abstract categories. This is an interesting result, for it shows that the concepts comprising a topic are distributed widely enough across the thesaurus that abstracting them just one level still allows for their detection. In a sense, the way that the concepts have been organized in the thesaurus do not match the realworld occurring topics. We believe that this result can serve as a measure of the
Level 0 Representation
Level 1 Representation 50 40 30 20 10 50 40 30 20 10 0 5 15 25 35 45 55 65 75 85 95
Stable topic duration 0 5 15 25 35 45 55 65 75 85 95
Stable topic duration generality of a thesaurus, and its applicability to analyzing texts from various sources.
This work represents initial results in the analysis of topic transitions with the help of a thesaurus. In future work we intend to build on top of the current results and extend the methodology. In particular, the current results motivate us to: 1. Investigate and compare representation of topics in di erent levels. The preliminary observations suggest that several stable topics at level 0, in nonoverlapping weeks, could be merged into a single topic at level 1. 2. The topics at level 1 appear to be more stable. This could be useful in the case of limited data, when the detailed topic could fade away. 3. The di erent levels of representation could prove to be useful for people with di erent backgrounds. Namely, we expect that more detailed topics could be of interest for experts in the eld, whereas the more general topics could give a good overview for less experienced readers. 4. We aim at improving the transition computation with the help of taking the distances between concepts into account and employing methods similar to soft cosine similarity. 5. Finally, statistical analysis is required to con rm or observations that the concepts gained during topic evolution are more likely to be close, in the thesaurus, to concepts already in a topic.
Acknowledgements The work is partially supported by the PROFIT (http: //projectprofit.eu/) project. Part of the European Commission's H2020 Framework Programme, grant agreement no. 687895,