In this paper, we attempt to extend existing Czech thesauri by using a word-formation network, DeriNet. Thesauri are an important resource for synonym retrieval / substitution generation but their lexical sparsity is an issue in Czech. We discuss the properties of existing thesauri and DeriNet and propose several ways of using DeriNet to extend the thesauri, such as deriving a synonym of an adverb from a synonym of corresponding adjective. We also evaluate some of our proposals.
A lot of effort has been invested in creating large
thesauri, of which the best known example is probably
WordNet [
However, those thesauri are heavily incomplete for
some languages, including the above-mentioned Czech
language. This incompletness presents a problem for
various NLP tasks, e.g. substitution generation as part of
lexical simplification (see [
On the other hand, for some languages (including Czech), a rich word-formation network is available. We propose using such network to extend existing thesauri, i.e. to discover synonymy relations between new pairs of words. Please note that while we target synonymy, as it is the only relation covered by all existing Czech thesauri, the approach would hold for any relation.
The rest of the paper is organized as follows: we briefly describe existing related work (section 2), present existing Czech thesauri (section 3) and describe the Czech wordformation network DeriNet (section 4). We then introduce several ways of combining DeriNet with thesauri to produce new relations (section 5) and evaluate the most promising of them (section 6).
Since thesauri are generally incomplete, there have been lots of attempts at extending them in an automated way.
These attempts have included aligning multilingual
resources (e.g. [
This paper is in its nature similar to a previous attempt
of extending Czech WordNet with derivational relations
[
We are aware of five notable Czech thesauri:
the most recent version of Czech WordNet [
ÚFAL thesaurus, a thesaurus developed at our department.
Both WordNet versions explicitly utilize synsets, each synset represents a meaning and lists literals (words or phrases) which can be used to express the meaning. Synsets might include a definition of the meaning but few have it filled.
The last three thesauri employ synsets implicitly, either by assigning a word with a set of sets of synonyms (as done in LibreOffice thesaurus and Wiktionary) or by listing sets of synonyms and including some words in more such sets (as done in our department thesaurus).
We perform our experiments both using each of the thesauri individually and using a concatenated thesaurus, i.e. an artificial thesaurus created by concatenating all synsets from each of the real thesaurus.
In our work, we do not make use of synsets. For each word, we merge its synonyms from all synsets and produce a set of its synonyms (despite the context, i.e. words which share the meaning at least in some contexts). This is partially to simplify the proof of concept, partially because senses in both WordNets are much more fine-grained than senses in other resources.
However, this step is in no way crucial. One could keep the synsets, and whenever we refer to retrieving synonym, they could first retrieve the synsets and only then retrieve the words (either from specific or all synsets). We actually expect to do this in our future work.
Some further statistics about the thesauri are provided in table 1. The concatenated line corresponds to concatenating all synsets. Please note that we only work with singleword expressions (as opposed to multi-word expressions). 4
DeriNet [
The directed edges then represent the fact that one word
is derived from the other one. The edges should be taken as
implicative, some derivations might not be captured in
DeriNet (yet). They are discovered using a variety of
methods, including manual deduction, rule-based automated
processes and machine learning; many of them were also
taken from the MorfFlex CZ morphological dictionary [
By the authors’ design decision, no word is allowed to have more than one parent, which simplifies the structure and could be justified by low occurence of compounds in Czech. Even though only one parent is allowed, recent versions of DeriNet allow for an indication of being a compound in part of speech specification.
The then current version of the network (1.7) contains
1; 027; 655 nodes, though only some of the nodes are
supported by corpus evidence (when compared to SYN v4
version of Czech National Corpus [
Of all nodes, 104; 563 (approx. 10 %) are isolated, i.e. they are not connected with any other node.
Except for the parent and part of speech, there is no further annotation, i.e. one cannot learn for example that the derived noun is agent noun of the base verb. DeriNet format is therefore farily simple: it gives node ID, its lemma and technical lemma (which contains some additional details such as sense disambiguation), its part of speech (perhaps with the above-mentioned indication of being a compound) and its parent’s ID (if the node has a parent). 5
We propose the following principle of discovering new word relations: 1. Find a non-root node A (i.e. a node which has a parent). 2. Get A’s parent, B. 3. Retrieve B’s synonyms using the existing thesauri. 4. Find all nodes C which correspond to the retrieved synonyms. 5. For each C, check if it has a child D which shares requested features with A.
This outline does not specify how to deal with the situation when more than one D exist (share given features with A) for single C. In our experiments, we opted for choosing neither (i.e. skipping the whole C subtree) but one could also develop strategies to select the best D or generate more pairs for A from single C.
It should be noted that due to this decision, discovering synonymous word pairs is not symmetric, that is, a word pair might be discovered when starting from one word, but not when starting with the other one.
We actually suggest further constraining all of A, B, C
and D to improve the reliability of the discovered relations,
i.e. by constraining their part of speech. While part of
speech is the only feature available in DeriNet itself, we
can use e.g. MorfFlex CZ dictionary or MorphoDiTa tool
for morphological analysis [
One could be tempted to only search for those nonroot nodes A which are not covered by any thesauri, the reasoning being that such nodes already have their synonyms in the thesauri. However, thesauri entries for individual words are often incomplete and the outlined process could still find new synonyms for node A, even if node A is present in a thesaurus. Furthemore, considering only nodes A which are not covered by thesauri could lead to a decrease in number of retrieved pairs after adding a new thesaurus as some nodes could be newly skipped. We therefore do not constrain node A on its presence/absence in the thesauri.
While we describe the process as deriving synonyms by using the synonymy relation of parent nodes, the parentchild relation is not crucial. One could reword the process e.g. with finding A’s child B and C’s parent D, or with finding A’s grand-parent B (and C’s grand-child D). However, by the nature of thesauri creation, we expect them to contain the base word rather than the derived word (though the direction of the derivation is sometimes ambigous). Longer distance relations, on the other hand, are more likely to introduce noise.
Having introduced the basic principle, we suggest specific approaches to relation derivation. We assume the access to richer morphological annotation, e.g. by using the MorphoDiTa tool. 5.1
Adverbs are often derived from adjectives and while adverb ratio in thesauri is close to corpus ratio (approx. 2:6 %-3:2 % of content words as measured using Czech National Corpus syn v4), we often ran into issues with them in our text simplification experiments.
We suggest constraining nodes A and D to be adverbs and nodes B and C to be adjectives. 5.2
In Czech, some words come in different forms for men (or males generally) and women. This in particular holds for roles in relationships and for agent nouns, e.g. there is ucˇitel ’teacher (man)’ and ucˇitelka ’teacher (woman)’.
Thesauri usually only cover the masculine variants, both because they are usually the default and because native speakers can infer the feminine variant (still, some language knowledge is required, e.g. there is ucˇitelka to ucˇitel but ministryneˇ to ministr ’minister’, not *ministrka).
We suggest constraining nodes A and D to be nouns having feminine gender and nodes B and C to be nouns having masculine gender. 5.3
Similarly to omitting feminine forms, thesauri generally do not cover possesive adjectives since they can be easilly inferred from the corresponding noun.
We suggest constraining nodes A and D to be possesive adjectives and nodes B and C to be nouns. 5.4
Thesauri differ in treating verb aspects, and often thesauri are not consistent even internally. Sometimes the verb of opposite aspect is listed as synonym, sometimes both aspects form their own synset, sometimes the other aspect is completely missing.
We suggest constraining nodes A and D to have matching aspect, nodes B and C to have matching aspect, and nodes A and B to have opposite aspect.
The issue with this suggestion is that aspect
information is neither present within MorfFlex CZ dictionary nor
provided by MorpohDiTa. We believe, however, that
annotation from Czech National Corpus (e.g. the
beforementioned version syn 4) [
We tried generating synonym pairs from adverbs using adjectives, feminine forms using masculine forms and possesive adjectives using nouns (see section 5 for more detail).
The generation procedure was carried out for each of the thesauri individually and also for the result of concatenating all the thesauri together.
Our results are reported in table 2. We report the number of obtained pairs for each of the thesauri as well as for the concatenated thesaurus. The reported numbers are after symmetrization, i.e. after expanding any pair A-D into both A-D and D-A. The actual numbers of discovered pairs are usually 1.6-1.9 times greater as most pairs (but not all of them) are discovered in both directions. These ratios seem to slightly correlate with the selected strategy (symmetrization is of greatest help when finding feminine variants).
When evaluating a specific thesaurus, we can discover a synonym pair which is actually present in some other thesaurus. Whenever this happens, we consider such pair
a confirmed one. We do not evaluate it further and expect that the pair is correctly derived and synonymous.
For each strategy, we sampled 100 non-confirmed pairs and asked 4 annotators to annotate them as either synonym, antonym or unrelated. The annotators were of varying gender and age, though all of them have obtained a university degree during their life. Asking annotators to distinguish antonyms from unrelated pairs was done based on our informal result analysis, which revealed antonym pairs do occur.
In 5 cases, the annotators admitted they did not know, in a few other, they noted they were not really sure. In all cases, a very infrequent word was involved. We treat I don’t know as unrelated when reporting precision and inter-annotator agreement. We treat answers marked with not sure in the same way as unmarked.
The inter-annotator agreement (Fleiss’ kappa) was 0.47 and it slightly varied over the strategies (0.47, 0.42 and 0.50, respectively). These numbers might seem low but it is important to keep in mind that most answers were synonym, hence this answer had a great probability, and therefore any disagreement on other answers had a big impact.
Examples of correctly discovered pairs (pairs annotated as synonyms by all four annotators) are given in table 3. There were 9 pairs marked as antonyms by all annotators (out of 28 marked as antonyms by at least one annotator), they are all listed in table 4. In all 9 cases, the pair was derived using a synonymy relation from LibreOffice thesaurus, when either two antonyms were suggested as synonyms to the same word, e.g. both tlouštík ’fatty’ and hubenˇour ’thin man’ to tlust’och’ ’fatty’, or when an antonym was suggested directly as e.g. inkompatibilní ’incompatible’ to kompatibilní ’compatible’. While these pairs are not synonymous, their existence should not be used to decline the principle. On the contrary, should the thesaurus pairs be correctly marked as antonyms, we would correctly derive antonymous pairs using our method.
Finally, there were 14 pairs annotated as unrelated by vodpoveˇdneˇ
hanebneˇ vyzveˇdacˇka cˇarodeˇjnice surovcu˚v maršálku˚v
spolehliveˇ bezcharakterneˇ špehounka divotvorkyneˇ krut’asu˚v maršálu˚v
dependably unscrupulously she-spy witch bully’s marshal’s all annotators. They are listed in table 5. In some cases (1, 4, 5), there is some evidence that the words can share a meaning but at least one of the words is associated with another meaning so strongly that the annotators probably did not realize the meaning could be the same.
Some pairs (2, 3, 10, 11, 13) come from a synset in thesauri, even though we could not find any other evidence that these pairs really could share the meaning.
Other pairs (6, 14) occur because of insufficiencies in the derivational process. While the base words are synonyms, the derived words are of distinct genders. These pairs could be prevented by constraining the suggested pairs more carefully.
Case 9 is quite similar. Both words stru˚jce ’creator’ and otec ’father’ could refer to a creator (author) and stru˚jkyneˇ ’she-creator’ is a feminine variant of stru˚jce. However, while the word otcˇina ’fatherland’ is directly derived from otec and is feminine, it does not in any way refer to shefather. This could be prevented with more detailed annotations in the word-formation network.
There are cases (7, 8) when, despite the principle proving good, derived words are not really perceived synonymous, even though the base words could be. For example, both words ku˚nˇ ’horse’ and osel ’donkey’ could be used to refer to a dumb person but their feminine variants are not used in that way (even though in theory they could be).
The last case, 12, is special in many ways. The word zárovenˇ ’at the same time, simultaneously’ is reported to be derived from word rovný ’straight’, which might seem surprising. The pair is further derived from thesauri pair pokrˇiveneˇ povšechneˇ ru˚zneˇ mladice tlust’oška živelneˇ jemneˇ inkompatibilneˇ bezcitneˇ distorted-ly in general diversely, differently young woman
fat woman elementally, unrestrainedly softly, lightly incompatibly heartlessly
rovno konkrétneˇ identicky starˇice hubenˇourka organizovaneˇ
pikantneˇ kompatibilneˇ vrˇele
straight specifically identically old woman thin woman organized-ly spicy, zesty compatibly heartily rovný ’straight’ – zakroucený ’tortuous’, which is rather antonymous.
We do not provide detailed report on pairs annotated differently by different annotators, though we have examined them too. In most cases, some evidence of shared meaning exist but some of the annotators did not consider the words synonymous.
Following from the above analysis, more than half of unrelated pairs is not less related that their base word counterparts. These pairs do not contradict our method, they only evidence the necessity of both checking thesauri quality and being careful about the synonymy itself as it is perceived differently by different people.
There are cases when our method fails to filter out nonsynonymous derived pairs. This could be improved both by better filtering during the inference process and by having better annotation in the word-formation network. 7
We have presented a method of deriving new synonym pairs using existing thesauri and word-formation network, we have suggested several strategies to do the actual derivation and we have evaluated some of them.
Our evaluation revealed that about half of derived synonym pairs are really perceived synonymous by all of our human annotators and around 80 % are perceived synonymous by at least two of them. The erroneous word pairs are caused by two distinct factors. First, there are errors in the thesauri synsets: unrelated, or even antonymous, words are occasionally marked as synonymous. Second, there are limitations of our method, where the derived words are not synonymous, despite being derived from synonymous base words.
Some of the limitations could be overcome by better filtering within our method or by more detailed annotations in the word-formation network. The latter has become available soon after we carried out our experiments, as DeriNet 2.0 has been released. This version has a more detailed annotation of both nodes (e.g. noun gender) and edges (the purpose of derivation is annotated, e.g. diminutivization), and we expect this version to be helpful in future experiments.
We also plan to try using Derivancze [
Overall, we consider our results good because they suggest that thesauri authors can focus on capturing the relations between the base words and NLP applications can still make good use of those thesauri even for derived words.
This work has been supported by the grant No. 1704218 of the Grant Agency of Charles University. It has been using language resources and tools stored and distributed by the LINDAT/CLARIN project of the Ministry of Education, Youth and Sports of the Czech Republic (project LM2015071). The research was also partially supported by SVV project number 260 453.