The University of Amsterdam at CLEF 2002

Abstract

This paper describes the official runs of our team for CLEF 2002. We took part in the monolingual tasks for each of the seven non-English languages for which CLEF provides document collections (Dutch, Finnish, French, German, Italian, Spanish, and Swedish). We also conducted our first experiments for the bilingual task (English to Dutch, and English to German), and took part in the GIRT and Amaryllis tasks. Finally, we experimented with the combination of runs.

Introduction

System Description

All submitted runs used FlexIR, an information retrieval system developed by the first author. The main goal underlying FlexIR’s design is to facilitate flexible experimentation with a wide variety of retrieval components and techniques. FlexIR is implemented in Perl; as it is built around the standard UNIX pipeline architecture, and supports many types of preprocessing, scoring, indexing, and retrieval tools, which proved to be a major asset for the wide variety of tasks in which we took part this year. 2.1

Approach

The retrieval model underlying FlexIR is the standard vector space model. All our official mono- and bilingual runs for CLEF 2002 used the Lnu.ltc weighting scheme [ 2 ] to compute the similarity between a query and a document. For the experiments on which we report in this note, we fixed slope at either 0.1 or 0.2; the pivot was set to the average number of unique words per document. 2.2

Morphological Normalization

Previous retrieval experiments [ 9 ] in English have not demonstrated that morphological normalization such as rule-based stemming [ 17 ] or lexical stemming [ 11 ] consistently yields significant improvements. As to the effect of stemming on retrieval performance for languages that are morphologically richer than English, such as Dutch, German, or Italian, in our experiments for CLEF 2001 we consistently found that morphological normalization does improve retrieval effectiveness [ 14 ].

Stemming/Lemmatizing. For this year’s monolingual experiments the aim was to improve our existing morphological analysis for languages that we had dealt with before (i.e, Dutch, German, and Italian), and to extend it to languages that we had not dealt with before (i.e., Finnish, French, Spanish, and Swedish). Where available we tried to use a lexical-based stemmer, or lemmatizer: for French, German, and Italian we used lemmatizers that are part of TreeTagger [ 19 ]. For Dutch we used a Porter stemmer developed within the Uplift project [ 21 ]; for Spanish we also used a version of Porter’s stemmer [ 3 ]. We did not have access to (linguistically informed) morphological normalization tools for Finnish or Swedish.

For the GIRT and Amaryllis task, we used TreeTagger for processing the main text. The keywords, i.e., GIRT’s controlled-terms and Amaryllis’ controlled vocabulary, were indexed as given, indexing the keywords or keyword-phrases as a single token.

Compound splitting. For Dutch and German, we applied a compound splitter to analyze complex words, such as, Autobahnraststa¨tte (English: highway restaurant), Menschenrechte (English: human rights), Friedensvertrag (English: peace agreement), etc. In addition to these noun-noun compounds, there are several other forms of compounding, including verb-noun (e.g., German: Tankstelle, English: gas station), verbverb (e.g., German: spazierengehen, English: taking a walk), noun-adjective (e.g., German: arbeitslos, English: unemployed), adjective-verb (e.g., German: sicherstellen, English: to secure); etc., see [ 6 ] for a more detailed overview. In last year’s participation we focused on noun-noun compound splitting, but this year we tried to cover the other forms for German as well. This resulted in a much larger compound dictionary for German. Whereas last year’s dictionary contained 108,489 entries, it grew up to 772,667 for this year’s participation. An entry in the compound dictionary consists of a complex word and its parts, where each part is lemmatized. See [ 14 ] for further details on the actual process of compound splitting.

For retrieval purposes, each document in the collection is analyzed and if a compound is identified, both the compound and all of its parts all of its parts are added to the document. Compounds occurring in a query are analyzed in a similar way: the parts are simply added to the query. Since we expand both the documents and the queries with compound parts, there is no need for compound formation [ 16 ]. Ngrams. To obtain a zero-knowledge language independent approach to morphological normalization, we implemented an ngram-based method in addition to a linguistically informed methods.

Avg. word length Ngram length

For each of the seven non-English languages in the monolingual task we determined the average word length, and set the ngram-length to be 1 less than the average word length, except for Finnish, where we set the ngram-length to be 2 less than the average word length; see Table 1 for the details. For each word we stored both the word itself and all possible ngrams that can be obtained from it without crossing word boundaries. For instance, the Dutch version of Topic 108 contains the phrase maatschappelijke gevolgen (English: societal consequences); using ngrams of length 5, this becomes: maatschappelijke maats aatsc atsch tscha schap chapp happe appel ppeli pelij elijk lijke gevolgen gevol evolg volge olgen 2.3

Blind Feedback

Blind feedback was applied to expand the original query with related terms. Term weights were recomputed by using the standard Rocchio method [ 18 ], where we considered the top 10 documents to be relevant and the bottom 500 documents to be non-relevant. We allowed at most 20 terms to be added to the original query. For Dutch and German, the added words are also decompounded, and the complex words and their parts are added to the query.

The text runs for the GIRT and Amaryllis tasks used blind feedback, while it was switched off for the keyword runs. To aid comparison with the monolingual runs, the same feedback settings were used. There is a remarkable difference in the effect of feedback: virtually no words are added for the GIRT and Amaryllis tasks. 2.4

Combined Runs

In addition to our morphological interests we also wanted to experiment with combinations of (what we believed to be) different kinds runs of runs in an attempt to determine their impact on retrieval effectiveness. More specifically, for each of the languages for which we had access to language specific morphological normalization tools (i.e., stemmers or lemmatizers), we created a base run using those tools. In addition, we used ngrams in the manner described above to create a second base run. We then combined these two base runs in the following manner. First, we normalized the retrieval status values (RSVs), since different runs may have radically different RSVs. For each run we re-ranked these values in [0:5; 1:0] using: RSVi0 = 0:5 + 0:5

RSVi

maxi mini mini and assigned all documents not occurring in the top 1000, the value 0:5; this is a variation of the Min Max Norm considered in [ 13 ]. Next, we assigned new weights to the documents using a linear interpolation factor λ representing the relative weight of a run:

RSVnew = λ RSV1 + (1 λ) RSV2: For λ = 0:5 this is similar to the simple (but effective) summation function used by Fox and Shaw [ 8 ], and later by Belkin et al. [ 1 ] and Lee [ 12, 13 ]. The interpolation factors λ were obtained from experiments on the CLEF 2000 and 2001 data sets (whenever available).

For the GIRT and Amaryllis task, we created alternative base runs based on the usage of the keywords in the collection, and combined these with the text-based runs. 3

Runs

We submitted a total of 27 runs: 10 for the monolingual task, 7 for the bilingual task, and 5 each for the GIRT and Amaryllis tasks. Below we discuss our runs in some detail.

Run UAmsC02DuDuNGiMO UAmsC02DuDuNGram UAmsC02FiFiNGram UAmsC02FrFrNGiMO UAmsC02GeGeLC2F UAmsC02GeGeNGiMO UAmsC02GeGeNGram UAmsC02ItItNGiMO UAmsC02SpSpNGiSt UAmsC02SwSwNGram

Type

Ngram/Morphological Ngram Ngram Ngram/Morphological Morphological Ngram/Morphological Ngram Ngram/Morphological Ngram/Morphological Ngram

Both topics and documents were stopped. First of all, for each language we used a stop phrase list containing phrases such as ‘Find documents that discuss . . . ’; stop phrases were automatically removed from the topics. We then stopped both topics and documents using the same stop word list. We determined the 400 most frequent words, then removed from this list content words that we felt might be important despite their high frequency. For instance, in most of the document collections terms such as ‘Europe’ and ‘dollar’ occur with high frequency. We did not use a stop ngram list, but in our ngram runs we first used a stop word list, and then ngram-ed the topics and documents. For the ngram runs we did not replace diacritic letters by their non-diacritic counterparts, for the morphological runs we did. 3.2

The Bilingual Task

We submitted a total of 7 bilingual runs, using English as the topic language, and Dutch and German as document languages.

Run UAmsC02EnDuMorph UAmsC02EnDuNGiMO UAmsC02EnDuNGram UAmsC02EnGeLC2F UAmsC02EnGeMOiMO UAmsC02EnGeNGiMO UAmsC02EnGeNGram

Topics

English English English English English English English

Documents

Dutch Dutch Dutch German German German German

Type

Morphological Ngram/Morphological Ngram Morphological 1 Morphological/Morphological 2 Ngram/Morphological 1 Ngram

Factor

– 0.71 – – 0.50 0.285 –

For the bilingual runs, we followed a dictionary-based approach. The translations of the words and phrases of the topic are simply added to the query in an unstructured way; see [ 15 ] for a more elaborated way of query formulation. The original queries are translated to Dutch using the Ergane dictionary [ 7 ], and to German using the Ding dictionary [ 5 ], version 1.1. The Ergane dictionary contains 15,103 English head words and 45,068 translation pairs in total. The Ding dictionary contains 103,041 English head words and 145,255 translation pairs in total.

Since the Ergane dictionary is rather small, we used a pattern-based approach to extend the translation dictionary with additional translation pairs. Table 4 shows some of the patterns. Notice that the vast majority of the words that match one or more of these patterns are words that are derived from Latin. If an English word was not in the Ergane dictionary each matching pattern was applied and all translations were added to the query. Of course, this rather ad-hoc approach to translation is far from perfect. For instance, privatization will be translated as privatisering (correct), by applying pattern (5), and privatisatie (incorrect), by applying pattern (6). Although this is unacceptable for machine translation applications, those erroneous translations have virtually no impact on retrieval effectiveness, because almost all of them are non-existing words that do not occur in the inverted index anyway.

Just like our Dutch and German monolingual runs, we prepared morphological and ngram-based runs, and combined these in order to improve effectiveness; see Table 3 for the details. 3.3

The GIRT and Amaryllis Tasks

As pointed out in Section 1, our strategy for the GIRT and Amaryllis tasks in CLEF 2002 was to compute the similarity of keywords based on their occurrence in the collection, and investigate whether the resulting keyword space can be used to improve retrieval effectiveness. We assumed that keywords that are frequently assigned to the same documents, will have similar meaning. We determined the number of occurrence of keywords and of co-occurrences of pairs of keywords used in the collection, and used these to define a distance metric. Specifically, we used the Jaccard similarity coefficient on the log of (co)occurrences, and used 1 minus the Jaccard score as a distance metric [ 10 ]. For creating manageable size vectors for each of the keywords, we reduced the matrix using metric multi-dimensional scaling techniques [ 4 ]. For all calculations we used the best approximation of the distance matrix on 10 dimensions. This resulted in a 10-dimensional vector for each of the 6745 keywords occurring in the GIRT collection. The Amaryllis collection uses a much richer set of 125360 keywords, which we reduced by selecting the most frequent ones; this resulted in vectors for 10274 keywords occurring 25 times in the collection. For our official CLEF runs we experimented with these keywords spaces for two specific purposes: keyword recovery and for document re-ranking.

We used the following strategy for determining vectors for the documents and for the topics: we took the top 10 documents from a base run (not using the keywords). For each of these documents we collected the keywords, and determined a document vector by taking the mean of the keyword vectors. Next, we determined a vector for the topic by taking the weighted mean of the vectors for the top 10 documents. For document re-ranking, we simply re-ranked the documents retrieved in the base run by the distance between the document and topic vectors. For keyword recovery, we considered the keywords used in the top 10 document, and selected the ten keywords that are closest to the topics vector. Table 5(a) shows the keywords recovered for GIRT topic 51.

Selbstbewußtsein familiale Sozialisation Junge Adoleszenz

Subkultur Erziehungsstil soziale Isolation Marginalita¨t Bewußtseinsbildung Puberta¨t (a) GIRT topic 51 (recovered) Concentration et toxicite´ des polluants Me´canisme de formation des polluants Re´duction de la pollution Choix du carburant Re´glage de la combustion Traitement des gaz d’e´chappement Le´gislation et re´glementation (b) Amaryllis topic 1 (monolingual, given)

Qualite´ air

Moteur diesel Trafic routier urbain Autobus Azote oxyde Exposition professionnelle Ve´hicule a` moteur Carburant diesel Inventaire source pollution Carburant remplacement (c) Amaryllis topic 1 (bilingual, recovered)

For the Amaryllis task, we can compare the provided topic-keywords in the narrative-field (shown in Table 5(b)), with the topic-keywords resulting from our automatic keyword recovery (shown in Table 5(c)). The recovered keywords are subsequently used in a keyword-only run.

For the GIRT task, we submitted three monolingual runs and two bilingual (English to German) runs. All our GIRT runs use the title and description fields of the topics. The morphological base run mimics the settings of our monolingual morphological base run for German. Based on the top 10 documents from the base run, we use the keyword space for recovering keywords for the topics as discussed above. The topic vector based on the top 10 documents of the base run is also used for re-ranking the documents retrieved in our base run. Experimentation on topics of CLEF 2000 and CLEF 2001 revealed that the keyword and re-rank runs perform worse than the base text run, yet a combination of the base run and either a keyword or a re-rank run helps to improve the performance.

Our runs for the bilingual GIRT task (English topics) used the translation method of the German bilingual task (using the ding dictionary) for translation of the title and description fields. For the rest, the bilingual tasks mimics the monolingual runs. We made a base morphological run, and recovered keywords for a keyword-only run, and a document re-ranking.

For the monolingual Amaryllis task, we submitted three monolingual runs and two bilingual (English to French) runs. Our morphological base run uses the same settings as the monolingual French run. For the keyword-only run, keywords were taken from the narrative fields of the topics. For the bilingual Amaryllis

Run UAmsC02GeGiTT UAmsC02GeGiTTiKW UAmsC02GeGiTTiRR UAmsC02EnGiTTiKW UAmsC02EnGiTTiRR UAmsC02FrAmTT UAmsC02FrAmKW UAmsC02FrAmTTiKW UAmsC02EnAmTTiKW UAmsC02EnAmTTiRR

Task

GIRT GIRT GIRT GIRT GIRT Amaryllis Amaryllis Amaryllis Amaryllis Amaryllis

Topics

German German German English English French French French English English

Documents

German German German German German French French French French French

Type

Morphological Morphological/Keyword Morphological/Re-rank Morphological/Keyword Morphological/Re-rank Morphological Keyword Morphological/Keyword Morphological/Keyword Morphological/Re-rank task, we used Systran [ 20 ] to translate the title and description fields of the English topics. We did not use the provided English keywords, nor the special dictionary provided. We made a morphological base run (similar to the monolingual task), and collected the keywords from the top 10 documents, which were then used for determining a document re-ranking and for keyword recovery. 4

Results

This section summarizes the results of our CLEF 2002 submissions. 4.1

Monolingual Results

We were somewhat surprised by the low scores of our morphological run for Dutch (0.3673) and of the ngram run for Italian (0.3672). The former is probably due to the fact that we used a reasonably crude stemmer, instead of a proper lemmatizer; the latter may be due to the fact that we did not replace diacritic characters by the corresponding non-diacritic letters. where Rcommon (Ncommon) is the number of common relevant (non-relevant) documents, and Ri (Ni) is the number of relevant (non-relevant) documents in runi. (A document is relevant if, and only if, it receives relevance score equal to 1 in the qrels provided by CLEF.) Table 8 shows the overlap coefficients for the base runs used to produce combined runs.

A few comments are in order. First, for French and Spanish the base runs are of similar (high) quality, but because the Noverlap coefficient is high, the combinations do not improve all that much. Furthermore, we conjecture that the reason for the limited gains of the combined runs over the best base runs for Dutch and Italian is due to the somewhat low quality of one of the base runs for these languages. Finally, the significant improvement obtained by combining the two German base runs may be explained as follows: both base runs are high quality runs, their Roverlap coefficient is high, and their Noverlap is fairly low — under these circumstances, Lee’s rationale predicts that the combined run is of high quality. 4.2

Bilingual Results

After we had received our results from CLEF, it emerged that one of the base runs submitted the for English to German task (UAmsC02EnGeLC2F) was not the correct one. As a consequence, the combinations in which this base run was used were also incorrect (UAmsC02EnGeNGiMO and UAmsC02EnGeMOiMO). The results and figures below have been obtained with the correct version of UAmsC02EnGeLC2F, using the qrels provided by CLEF.

To begin with, Table 9 shows our non-interpolated average precision scores for both bilingual sub task: English to Dutch and English to German.

Morphological 1

Morphological 2 Ngram Combined Ngram/Morp. 1 Combined Morph. 1/Morph. 2

English to Dutch

0.2576 – 0.2807 0.2933 (+4.5%) –

English to German

0.3363 0.3094 0.2614 0.3514 (+4.5%) 0.3451 (+2.6%)

For English to Dutch, we submitted one morphological run, where both stemming and compound splitting was applied. For English to German, we created two morphological runs, one with a large decompounding lexicon (Morphological 1), and one with last year’s settings, i.e., a smaller decompounding lexicon (Morphological 2). For both target languages we also submitted one n-gram run. In addition, we combined the n-gram run with the morphological run for both languages, and for German we also combined both morphological runs.

Table 10 shows the decrease in effectiveness compared to the best monolingual run for the respective target language.

Best monolingual Best bilingual

If we consider the decrease in retrieval effectiveness between monolingual and bilingual, we can observe a significant difference between Dutch and German. It is very likely that this is due to the difference in size between the translation dictionaries that were used to formulate the target queries: the Dutch translation dictionary contained 15,103 head words plus translation rules, whereas the German dictionary contained 103,041 head words; see Section 3.

As with the monolingual runs, we also analyzed the overlap coefficients for base runs that were combined; see Table 11. The gains in effectiveness of the combination over the best base runs is consistent with the coefficients, with comparable gains for the ngram/morphological combinations for Dutch and German; note that both have a fairly low Noverlap coefficient. The two (German) morphological runs share many non-relevant documents, and as consequence the combination of these two runs is less effective that the combination of the ngram run with the morphological 1 run.

Roverlap Noverlap

English to Dutch Ngram/Morphological 0.7737 0.2516

English to German

Ngram/Morphological 1 0.7898 0.3588

English to German

Morphological 1/Morphological 2 0.9338 0.5853

The results for the GIRT tasks are outright disappointing. Our morphological base run fails to life up to the performance of the corresponding monolingual German runs (average precision 0.1639 for GIRT versus 0.4476 for German). On our pre-submission experiments on the GIRT topics of CLEF 2000 and CLEF 2001, we also noticed a drop in performance, but far less dramatic as for the CLEF 2002 run (average precision around 0.31 for both runs versus 0.1639 this year). Still, the combination of the morphological run with either the keyword run or re-rank run improves retrieval effectiveness. For the English to German GIRT task, only the combination of the morphological and re-rank base runs improves compared to the base runs; this may be due to the extremely low precision at 10 of the bilingual base run (0.1417).

Our runs for Amaryllis are more in line with the results for the monolingual French task (average precision 0.2681 for the base run versus 0.4063 for French). The keyword-only run using the provided keywords even out-performs the morphological base run. The combination of the two runs leads to an impressive improvement of retrieval effectiveness (+26.7%). The English to French Amaryllis task performs fairly well compared to the monolingual Amaryllis task. The combination runs of the morphological base run with the recovered keywords, and of the morphological base run with the re-ranking show significant improvement.

Figure 3 contains precision-recall plots for the GIRT and Amaryllis tasks. In addition to the scores for our submitted runs, the figure also plots the scores for the base runs that were used to generate the combined runs.

Roverlap Noverlap

GIRT (mono) 0.4493 0.1031

GIRT (bi)

0.2984 0.0756

Amaryllis (mono)

0.6586 0.1236

Amaryllis (bi)

0.6506 0.1301

As above, we analyzed the overlap coefficients for base runs that were combined; see Table 13. As expected, gains in effectiveness are due to a high Roverlap coefficient combined with a relatively low Noverlap coefficient. This gives some rationale for why the combination of a baserun with a much lower quality run can improve retrieval effectiveness. It is interesting to note that the coefficients for the combined monolingual Amaryllis (using the provided keywords) are similar to those of the bilingual (using the recovered keywords). 5

Conclusions

The experiments on which we report in this note indicate a number of things. First, morphological normalization does improve retrieval effectiveness significantly, especially for languages such as Dutch and Morphological Keyword Rerank Morph./Keyword Morph./Rerank Morphological Keyword Morph./Keyword

German, that have a more complex morphology than English. We also showed that ngram-based retrieval can be a viable option in the absence of linguistic resources to support deep morphological normalization. Furthermore, combining runs provides a method that can consistently improve base runs, even high quality base runs; moreover, the interpolation factors required for the best gain in performance seem to be fairly robust across topics. Finally, our results for the bilingual task indicate that simple word/phrase translation, where all possible translations are used to formulate the target query in an unstructured way, leads to a significant decrease in effectiveness, when compared to the respective monolingual runs. Therefore, we are planning to investigate more restrictive ways of formulating target queries.

Acknowledgments

We want to thank Willem van Hage and Vera Hollink for their technical support, and Maarten Marx for useful discussions.

Jaap Kamps was supported by the Netherlands Organization for Scientific Research (NWO, grant # 400-20-036). Christof Monz was supported by the Physical Sciences Council with financial support from the Netherlands Organization for Scientific Research (NWO), project 612-13-001. Maarten de Rijke was supported by grants from the Netherlands Organization for Scientific Research (NWO), under project numbers 612-13-001, 365-20-005, 612.069.006, 612.000.106, 220-80-001, and 612.000.207.

References

[1] N.J. Belkin , P. Kantor , E.A. Fox , and J.A. Shaw . Combining evidence of multiple query representations for information retrieval . Information Processing & Management , 31 ( 3 ): 431 - 448 , 1995 .

[2] C. Buckley , A. Singhal , and M. Mitra . New retrieval approaches using SMART: TREC 4 . In D. Harman, editor, Proceedings of the Fourth Text REtrieval Conference (TREC-4) , pages 25 - 48 . NIST Special Publication 500- 236 , 1995 .

[3] CLEF resources at the University of Neuchaˆtel. http://www.unine.ch/info/clef.

[4] T.F. Cox and M.A.A. Cox . Multidimensional Scaling. Chapman & Hall, London UK , 1994 .

[5] Ding : A dictionary lookup program . http://www-user.tu-chemnitz.de/˜fri/ding/ .

[6] G. Drosdowski , editor. Duden: Grammatik der deutschen Gegenwartssprache. Dudenverlag, fourth edition , 1984 .

[7] Ergane : a free multi-lingual dictionary programme . http://download.travlang.com/Ergane/ frames-en.html .

[8] E.A. Fox and J.A. Shaw . Combination of multiple searches . In Proceedings TREC-2 , pages 243 - 252 , 1994 .

[9] W. Frakes . Stemming algorithms . In W. Frakes and R. Baeza-Yates, editors, Information Retrieval: Data Strcutures & Algorithms , pages 131 - 160 . Prentice Hall, 1992 .

[10] J. C. Gower and P. Legendre . Metric and euclidean properties of dissimilarity coefficients . Journal of Classification , 3 : 5 - 48 , 1986 .

[11] D. Harman . How effective is suffixing? Journal of the American Society for Information Science , 42 : 7 - 15 , 1991 .

[12] J.H. Lee . Analyses of multiple evidence combination . In Proceedings SIGIR'97 , pages 267 - 276 , 1997 .

[13] J.H. Lee . Combining multiple evidence from different relevant feedback networks . In Database Systems for Advanced Applications , pages 421 - 430 , 1997 .

[14] C. Monz and M. de Rijke . Shallow morphological analysis in monolingual information retrieval for Dutch, German and Italian . In C. Peters, M. Braschler , J. Gonzalo , and M. Kluck, editors, Proceedings CLEF 2001 , LNCS 2406 , pages 262 - 277 . Springer Verlag, 2002 .

[15] A. Pirkola , T. Hedlund , H. Keskustalo , and K. Ja ¨rvelin. Dictionary-based cross-language information retrieval: Problems, methods , and research findings. Information Retrieval , 4 ( 3 -4): 209 - 230 , 2001 .

[16] R. Pohlmann and W. Kraaij . Improving the precision of a text retrieval system with compound analysis . In J. Landsbergen, J. Odijk , K. van Deemter , and G. Veldhuijzen van Zanten, editors, Proceedings of the 7th Computational Linguistics in the Netherlands Meeting (CLIN 1996) , pages 115 - 129 , 1996 .

[17] M. Porter . An algorithm for suffix stripping . Program , 14 ( 3 ): 130 - 137 , 1980 .

[18] J. Rocchio . Relevance feedback in information retrieval . In G. Salton, editor, The SMART Retrieval System - Experiments in Automatic Document Processing. Prentice Hall , 1971 .

[19] H. Schmid . Probabilistic part-of-speech tagging using decision trees . In Proceedings of International Conference on New Methods in Language Processing , 1994 .

[20] Systran Online Translator . http://www.systransoft.com/.

[21] UPLIFT: Utrecht project: Linguistic information for free text retrieval . http://www-uilots.let. uu.nl/˜uplift/.