This paper describes an approach for paraphrase detection in Malayalam sentences developed as part of FIRE 2016 Shared Task on Paraphrase detection in Indian Languages. The task of paraphrasedetection is finding a sentence with the same meaning of another sentence expressed using same or different words. This detection is done by a semantic approach which is language dependent. Individual words, their root forms and synonyms are used in finding similarity between two given sentences. We present an algorithm for paraphrase identification which makes use of word similarity information derived fromCUSAT Malayalam WordNet Padasrinkala.. The approach is evaluated using the Malayalam corpus made available as part of of FIRE 2016 Shared Task on Paraphrase detection in Malayalam. • Computing methodologies~Natural language processing • Computing methodologies~Lexical semantics • Computing methodologies~Language resources • Computing methodologies~Information extraction matching;tokenization;POS Paraphrase detection; semantic tagging;lemmatization;corpus.
Paraphrase is defined as the reuse of text or its meaning in another sentence using the same or similar words or phrases. Paraphrase detection is used to determine whethertwo texts (sentences) of different lengths have the samemeaning. Such detection is used in various natural language applicationssuch as plagiarism detection, text summarisation, WSD, machine translation etc.Paraphrasing may be due to morphology based changes, lexicon-based changes, syntax-based changes, discourse-based changes, semantics-based changes etc. This approach to paraphrase detection comprises of pure lexical matching and also the similarity between sentences which use synonyms to convey the same meaning.
The outline for the rest of the paper is as follows. Section 2 describes some of the previous approaches to paraphrase identification and their limitations. The approach proposed here is described in Section 3. Section 4 gives a brief description of the Paraphrase Corpus which is used for evaluation. Section 5 presents the results of this evaluation. Conclusions and suggestions for future work are presented in Section 6.
Purely lexical based matching techniques for paraphrase detection
was used by
The proposed task at FIRE 2016 is focused on sentence level paraphrase identification for Indian languages (Malayalam). Sub Task 1: Given a pair of sentences from newspaper domain, the task is to classify them as paraphrases (P) or not paraphrases (NP). Sub Task 2: Given two sentences from newspaper domain, the task is to identify whether they are paraphrases (P), semiparaphrase (SP) or not paraphrases (NP).
Our proposed semantic approach foridentifying theparaphrases comprisesof three phases – matching identical tokens, matching lemmas and matching with synonyms replaced. Similaritycomparison is performed at the sentence level using the Jaccard, Containment, Overlap and Cosine similarity metrics and if thesimilarity score of a sentence pair is higher than a predetermined threshold, the pair ismarked as plagiarised.The steps are illustrated in Figure 1.
The two input sentences are broken down into individual words or tokens and compared for similarity. Given two sentences S1 and S2, thetokens produced from S1 will be {W1,W2. . .WN}, where N is the number of words in the sentence S1.
The individual words in the two input sentencesare reduced to their root form or lemmas using a suffix stripping algorithm.Lemmatization is the technique of transforming words into their dictionary base forms.
The inflected words for similarity analysis are converted to a valid root wordby means of suffix stripping along with some transformational rules. Each rule set consists of suffixes and their corresponding transformations that can generate the root word. This rule set is considers plurals and Vibhakthis in case of nouns and the different tense forms in case of verbs. Suffixes in Malayalam inflected word may range from a single character to a group of characters. So the algorithm starts stripping from the right side of the inflected word character wise. Each time a character which is a valid suffix in the rule set is stripped, corresponding transformations are done and the resulting word in checked in the dictionary. If it is found the algorithm terminates. Otherwise the procedure continues until a valid word is found. The root words are checked for correctness with the part of speech tag.These lemmas are then compared for similarity.
For the remaining lemmas that are not matched, substitute synonyms from the CUSAT Malayalam wordnetPADASRINKALA. An example is given below
The combined similarity obtained from direct word matches, lemma matches and synonym match produces a score between 0 and 1 that indicates the similarity between sentences S1 and S2. a) Jaccard Similarity
B b)
A A and B represent the sets of n-grams in the sentencesS1 and S2 respectively. The containmentmeasure calculates the intersecting n-grams but normalises them only with respectto the count of ngrams in the first sentence S1.
min( A B A and B are the unique n-grams contained in the sentence S1 and sentence S2 respectively. The intersecting n-grams of both sentences is dividedby the sentence with the smaller word count.
Scos ine ( A, B)
A B A B Sentences S1 and S2 are represented as vectors A and B respectively.
Consider the example sentence pairs പ്ദതിയുളട
കകരുണ്ം Direct match + lemma match + synonym match
There are no annotated corpora or benchmark data for paraphrases available for Indian languages till date..The data provided for this shared task have been splitinto two training sets containing 2500 and 3500 examples respectively and two test sets containing 900 pairs of sentences for task1 and 1400 pairs of sentences for task2. The training data-set -1 contains 1000 sentencepairs that have been marked by human judges as paraphrases and1500 sentencepairs that have been marked as not paraphrases. The training data-set -2 contains 1000 sentencepairs that have been marked as paraphrases , 1000 sentencepairs that have been marked as semi-paraphrases and 1500 sentencepairs that have been marked as not paraphrases.This train/test partitionhas been observed by all the approaches evaluatedhere.
The approach described in Section 3 was evaluatedagainst the Paraphrase Corpus.All synonyms of Malayalam WordNet were considered when finding the similaritybetween words. The training data was used to find the classificationthreshold (paraphrase/semi-paraphrase/not-paraphrase) for the two tasks. Considering the four similarity measures, the following observations are made.
Containment measure is useful in cases where thesuspicious text is shorter than the source text. Overlap measure is useful in cases where the size of suspicious and source text varies. Jaccard similarity values are less compared to the Cosine value. Hence only the Cosine value is considered for setting the threshold. Accuracy, precision, recall and F measurewere evaluated for the test corpus:These are defined as follows: accuraccy
TP TN
TP TN FP FN where TP are true positives, TN are true negatives,FN are false negatives and FP are false positives.
precision recall
TP TP FP
TP
TP FN F
Results for the semantic similarity approach on the test data areshown in Table3.
Table3. Results on test data Task Task-1 Task-2
This paper presented an approach to the problemof paraphrase detection in Malayalam language. Paraphrase has been identifiedbased on the tokens and its synonyms that are common thathas been taken as attribute for checking paraphrase. Thewords are checked against Malayalam Wordnet. Bycalculating the token matching ,lemma match and synonymtoken matching andfixing an appropriate threshold value, the given sentence can be classified as paraphrase, semi-paraphrase sentence or not paraphrase.
From the obtained values of Accuracy and F-measure, we consider combining the similarity approaches in future to improve the efficiency of the system. Also, the accuracy of this method can be further enhanced by including a spell-checker and correcting misspelled words before similarity checking.