This paper describes our approach on "Recognizing Reference Spans,Classifying Their Discourse Facets and Summarizing from Reference Text" as an attempt in the shared task on relationship mining and scienti c summarization of computational linguistics research papers at SIGIR 2017. The original reference is divided into sentences. The similarity between each of the sentences to the cited sentence in the citance is measured in using three standard measuring rules: a. Jaccards Coe cient (Trigram Model)(J(a, b)) b. Clough and Stevenson Coe ecient (Trigram Model) (C(a,b)) c. Model Probability Measure (Bigram Model) (P(a)) Thus for each sentence in the reference text we get an 3-tuple (J, C, P). Now this vector is fed to an Arti cial Neural Network to nd whether the sentence in the reference text is cited in the citance. Using the outputs of this Arti cial Neural Network we get the Candidate Sentences, the subset of which is the set of "Citation Sentences". Actually this Neural Network is used as an Filtering method.
The 3rd CL-SciSumm Shared task provides resources for scienti c paper summarization.An overview of the shared task,including speci c details on the dataset and subsequent analysis for each task.In this report we provide a short description of the methods we have used for the task 1A and 1B.
This system is a rule based implementation of Arti cial Neural Network. 3.1
If a Reference Text has no Candidate Sentence with score greater than 1.2, we say that the Reference text has not been cited at all.
Otherwise the sentence or the sentence segment with maximum score is declared to be the Citation Text.
The code to measure the Cosine Similarity Score is as follows: from k e r a s . p r e p r o c e s s i n g . t e x t import T o k e n i z e r , t e x t t o w o r d s e q u e n c e import math from s c i p y import s p a t i a l def c o s i n e s i m i l a r i t y ( t1 , t 2 ) : t e x t s =[ t1 , t 2 ] t k n z r=T o k e n i z e r ( l o w e r=True , s p l i t =" " ) t k n z r . f i t o n t e x t s ( t e x t s ) x=t k n z r . t e x t s t o m a t r i x ( t e x t s ) v1=x [ 0 ] v2=x [ 1 ] sumxx=0 sumxy=0 sumyy=0 f o r i in range ( len ( v1 ) ) : a=v1 [ i ] b=v2 [ i ] sumxx=sumxx+a a sumyy=sumyy+b b sumxy=sumxy+a b return ( f l o a t ( sumxy ) / f l o a t ( math . s q r t ( sumxx sumyy ))+1) 3.2
Task 1B considered as text classi cation problem.The ve discourse facets of a sentence in a reference paper are Aim,Methode,Implication,Result and Hypothesis.For this task we are following an unsupervised method.
For each bag vector measure cosine similarity between sentence vector and bag vector. The most similarity value corresponding to a bag vector will be assigned as class of cited text. Bag of Words: Each bag is a list of relevent words to a particuler class.Each bag is made based on unigram.For each class we have seperated the reference text from training data set.Then made a list of words from reference sentences and calculated their tf-idf score.Bag is constructed by taking words with highest tf-idf score. Bag Vectors: We used the pre-trained 200 dimensional GloVe(http://nlp.standford.edu/software/CRFNER.html) on Twitter data 2billion tweets(http://nlp.standford.edu.projects/glove/) to create the vectors of the reference text and the word bags.
The word bag vectors are created by taking the normalized summation of the vector of words in word bags which were present in the vocabulary of the pre-trained Glove model.Out of vocabulary words are assigned to null vector.
q~i = Nv1(qi) PjN=v1(qi) W~ij and W~ij = ~0 where,q~i = Topic vector of ith word bag,Nv(qi)= Number of words in qi present in vocabulary, W~ij = Vector of jth word in ith word bag. sentence vectors The sentence vectors are created by taking the normalized summation of the vectors of the words in the sentence,which were present in the vocabulary of the pre-trained GloVe model.In cases where the word was not a part of the model vicabulary,it was assigned to the null vector.
t~i = Nv1(ti) PjN=v1(ti) u~ij and u~ij = ~0 Where,t~i = Sentence vector of ith sentence,ti. Nv(ti)= Number of words in ti present in vocabulary. u~ij = Vector of jth word in ith sentence. Cosine Similarity We used cosine similarity measure to calculate the cosine similarity,S between the sentence vector and the topic vector.
S = cosine sim(t~i; q~j) =
t~iq~j jjt~ijjjjq~jjj
A high value of S denotes higher similarity between the sentence vector,t~i and the topic vector q~j and vice-versa. 3.3
From the outputs obtained from Task 1A and 1B, a community summary had to be formed, which is a structured extractive summary of the Reference Paper (RP) generated from the cited text spans of the RP.
Community summary The output of Task 1b, contained the cited text spans, along with their facets for each RP. The ve discourse facets of a sentence in a reference paper are Aim, Method, Implication, Result and Hypothesis. For each facet, duplicate entries and stop words were removed from the text spans. A similarity score was calculated between the text spans for each facet using the cosine similarity measure. If the cosine similarity score was high, then one out of the two sentence vectors were selected randomly as a probable candidate for summarization. It was analysed from the output that any sentence with word length less than three, contributed no meaning to the summary generated and hence were discraded. We have submitted two di erent runs for Task 1a and 1b and a single run for Task 2. Task 1a and 1b is scored by the overlap of text spans measured by number of sentences in the system output vs gold standard. Task 2 is scored using the ROUGE family of metrics between i) the system output and the gold standard summary fromt the reference spans ii) the system output and the asbtract of the reference paper. The performance of our system in Task 1 and 2 is shown below in Table 1 and Table 2 respectively. 5
In this paper,we presented a brief overview of our system to address automatic paper summarization in the Computational Linguistics domain.Recognizing the cited text spans and determining their discourse facets are very challenging task for the summarization of scienti c papers.We have observed, task 1A involves much more than similarity problem.More features that re ect the citation intentions should be explored. For task 1B,building word bags which contain all the topic words relevant to the facet showed better results than the rest.We could do that approach better using bigram or trigram. Task 2 evaluations show that more features have to be introduced in order to calculate the importance of the cited text spans. 6 Kokil Jaidka,Muthu Kumar Chandrasekaran,Sajal Rustagi and Min-Yen Kan(2016).Overview of the 2nd Computational Linguistics Scienti c Document Summarization Shared Task(CLSciSumm 2016),To appear in the proceedings of the Joint Workshop on Bibliometricenhanced Informatilon Retrival and Natural Language Processing for Digital Libraries(BIRNDL 2016),Newwork,New Jersey,USA.
Surojeet Dasgupta,Abhash Kumar,Dipankar Das,Sudip Kumar Naskar,Shivaji Bandyopadhyay.Word Embeddings for Information Extraction from Tweets.Microblog Track at Forum for Information Retrival Evaluation(FIRE)2016.