This paper presents an experiment of statistical word stemming based on axality measurements. These measurements quantify three characteristics of language. In this experiment we tested one strategy of stemming with three dierent sizes of training data. The developed stemmer was used by the automatic summarization system Cortex to preprocess input texts and produce readable summaries. All summaries were evaluated as part of the INEX 2012 Tweet Contextualization Track. We present the results of evaluation and a discussion about our stemming strategy.
The task proposed in the INEX 2012 Tweet Contextualization Track consists in obtaining some textual context from the English Wikipedia about the subject of a tweet. The nal contextualization of the tweet should take the form of a readable summary of 500 words. An amount of 1133 documents, contextualized tweets with text from Wikipedia from November 2011, were processed in order to obtained summaries. Bibliographic references an empty Wikipedia pages were omitted.
The evaluation of summaries was done by the INEX organizers taking into
account informativeness and readability. The former was obtained using
KullbackLeibler divergence with Dirichlet smoothing by comparing n-gram distributions.
The latter was accomplished by the participants in the track; they evaluated the
summaries taking into account syntax, anaphoric resolution and redundancy.
More details of the system of evaluation and the INEX 2012 Tweet
Contextualization Track could be found in [
For this track we developed a stemmer based on morphological segmentation. The stemmer was coupled with Cortex, an automatic summarization system, in order to generate the summaries. We tested three sizes of training corpora to determine the best option for statistical stemming for English.
The organization of this paper is as follows: in Section 2 we review some approaches of morphological segmentation; in Section 3 we present word stemming; in Section 4 we describe the axality measurements; Section 5 presents the stemming strategy; evaluation obtained in INEX track is expose in Section 6 and nally, in Section 7, we briey present our conclusions and future work. 2
The rst work for unsupervised discovery of morphological units of language
is due to Zellig Harris [
Now a day, one of the most utilized methods for unsupervised learning of
morphology is based on Minimum Description Length (MDL) approach. This
has been developed as a computational system called Linguistica [
The mission of preprocessing documents for tasks of NLP, such as Question Answering, Information Retrieval or Automatic Text Summarization, in agglutinative languages is more complex. This is due to the fact that agglutinative languages have numerous combinations of morphs rather than a simple prexstem-sux combination. A method of unsupervised morphological segmentation for these kinds of languages is called Morfessor [69]. 2 This approach uses MDL by Maximum a Posteriori framework. Also, it integrates a morphotactic analysis to represent each word by a Hidden Markov Model (HMM). We are not sure if this method has been used for word stemming. 3
The majority of NLP systems preprocesses documents in order to decrease the Vector Space Model representation. This is the case of Cortex, which will be explained below. A well-known strategy for that purpose is word stemming, i.e. 1 http://linguistica.uchicago.edu 2 http://www.cis.hut./projects/morpho/ truncating words by eliminating the inection. Also, it is possible to remove derivational axes.
The methods most widely used for word stemming are created by means
of hand-made rules, like [
In [
The axality measurements used to morphological segmentation were proposed
for Spanish in [
Entropy
As we said above, Harris’s approach revealed that uncertainty helps to
morphological segmentation. This uncertainty could be seen as the Shannon’s concept
of information content (entropy) [
H (ai;j :: Bi;j ) =
X p (bk;j ) log2 (p (bk;j )) (1) where k = 1; 2; 3; : : : jBi;j j and each bk;j 2 Bi;j . For our purpose we tested peaks of entropy from right to left in order to discover suxes. 4.2
Economy Principle
The Economy Principle could be understood as follows: fewer units at one level
of language are combined in order to create a great number of other units at the
next level. Taking advantage of this principle, we can dene a stem as a word
segment that belong to a big set of relatively infrequent units, and axes as word
segments that belong to a small set of frequent ones. In [
p Ki;j = 1 jAi;j j jBis;j j
p jAi;j j ;
Kis;j = 1 jBi;j j
jBis;j j p jAi;j j (2) where Ai;j is the set of segments which alternate with bi;j (ai;j 2 Ai;j ), and p Bi;j a set of segments which alternate with ai;j (bi;j 2 Bi;j ). Also, let Ai;j be the set of segments which are likely prexes, and Bis;j the set of segments which are likely suxes. 4.3
Numbers of Squares
Joseph Greenberg [
The axality of all possible segmentations within a word is estimated by an average of normalized values of the three explained measurements: AF n (sx) = cx=max ci + kx=max ki + hx=max hi 3 (3)
To calculate this axality, a training corpus of raw text is required. In this
track we use three dierent sizes of 100k, 200k, and 500k word tokens. With an
index of axality calculated for each possible word segment, it is possible to
choose a strategy for morphological segmentation; for example [
In this experiment we use a peak-valley strategy for segmentation. Given a set of axality indexes inside a word afik, let afik 1 < afik > afik+1 be a peak of axality from left to right, where k is the length of the word plus one (the ending of the word). The main disadvantage of this approach is that small peaks are taking into account generating oversegmentation.
Regarding stemming, we truncate words at most left peak of axality. For a language with scare morphology like English, we can imagine that a most right peak of axality could be sucient for stemming. However, in order to improve Cortex summarization, we decide to strongly conate words by a left-peak strategy. Next section explains CORTEX’s approach. 5.1
Cortex Summarizer As we mentioned before, Cortex is an automatic text summarizer system. A wide explanation of this summarizer could be found in [2529]. Here, we briey describe some relevant aspects. First, Cortex represents input documents in Vector Space Model. To do that, the documents should be preprocessed. Actually, we incorporate our stemmer in this step.
After preprocessing, a frequency matrix is generated representing the presence and absence of words (terms) in a sentence:
Finally, a decision algorithm combines those metrics to score sentences. Two averages are calculated, > 0:5, and < 0:5 ( = 0:5 is ignored): X =
X =1 k k>0:5 w w w w 0:5 ; X = 0:5 w w w w
(6) X =1 k k<0:5 The next expression is used to calculate the score of each sentence: If
X > X
! then = 0:5 +
P else = 0:5
P
Cortex sorts nal sentences by using ; = 1; ; P . Additionally, Cortex let us delimit a compression rate, which was xed at 500 words. 6
6.1
Design of Experiments We made use of three sizes of training corpora, 100K, 200K, and 500K word tokens, to test our stemmer. With these sizes we performed the three runs for INEX track. The assigned numbers of runs were 153 (100K), 154 (200K), and 155 (500K). The corpus for evaluation was the 1133 contextualized tweets with text from Wikipedia from November 2011. About training corpora, we selected 24 documents from the same contextualized tweets. 6.2
Results For informativeness, Cortex, coupled with our stemmer, obtained rank 12, 14, and 15. Average scores of informativeness are shown in Table 1. The best run in this evaluation was run 154 (200K). In this paper we reported an experiment using a stemmer based on morphological segmentation. We used axality measurements in order to segment words. This stemmer was coupled with Cortex, an automatic summarization system.
We suggested the next stemming strategy: given some peaks of axality of a word, we truncated at most left peak. Also, we tested three training corpus sizes to obtain statistical information for the axality indexes: 100K, 200K, and 500K word tokens. Our two goals were to know if our stemming strategy can produce readable summaries, and if dierent sizes of training corpora can improve the Cortex performance.
According to results of evaluation, our stemming strategy produces not only readable summaries but also competitive ones. That is, from an average of relevance, syntax, and structure (0.6148), run 155 obtained a rank 7 among 27 runs. What is more, concerning informativeness, run 154 obtained rank 12 among 33 participants.
Regarding corpus sizes, it is not clear what size is the best for English, between 200K and 500K word tokens. However, it is clear that increasing corpus size is a good strategy because 100K obtained the worst results. Additionally, a greater training corpus gives better position in the ranking, for example, from an average of relevance, syntax, and structure, run 155 (500K) obtained rank 7 and run 153 (100K) obtained rank 15.
In future experiments we will test dierent strategies for morphological segmentation and stemming. Additionally, we can test dierent stemming approaches, such as Porter’s stemmer. 28. Torres-Moreno, J.M., Velazquez-Moralez, P., Meunier, J.: CORTEX, un algorithme pour la condensation automatique de textes. In: ARCo. Volume 2. (2005) 365 29. Torres-Moreno, J.M., Velazquez-Morales, P., Meunier, J.: CondensØs de textes par des mØthodes numØriques. JADT 2 (2002) 723734 30. Torres-Moreno, J.M., St-Onge, P.L., Gagnon, M., El-BŁze, M., Bellot, P.: Automatic Summarization System coupled with a Question-Answering System (QAAS). CoRR abs/0905.2990 (2009)