This paper describes a system for Indian Language Extractive Summarization which has been developed under the shared task ILSUM of FIRE 2022[1][2] Conference by SUMIL22 team. This system works by picking up sentences directly from the text using a ranking function to form the corresponding summary. In our approach, for each sentence, we first calculated various text features such as sentence position, sentence length, sentence similarity, frequent words, and sentence numbers. Then, these text features along with their optimized weights are used for ranking the sentences, and then the summary is generated by selecting top-ranked sentences. For optimizing the weights of the text features, we have used a population based meta-heuristic approach, Genetic Algorithm (GA). We submitted three runs and got an F-score of 0.3843 for ROUGE-1, 0.2584 for ROUGE-2, 0.1997 for ROUGE-3 and 0.2190 for ROUGE-4 in the best run.
Information has become the most important part of our life. People generally use a large number of sources from news articles to social media posts to know the information. Generating automatic summaries of larger texts will be useful for compressing the larger texts of information into smaller texts and also saves a lot of time. Our project focuses on automatic text summarization of news data. Text summarization is the creation of a shorter, precise, and more relevant summary of a larger text. Automatic summarization methods will be able to handle the ever-increasing amounts of internet text data, allowing users to find and consume important information more quickly.
Abstractive and extractive summarizations are the two broad types of automatic text summarization. Abstractive summarization methods attempt to create a summary by trying to interpret the text using advanced natural language methods to create a unique and more concise text of parts that may not show up in the original article, that symbolizes the most critical information from the original text, going to require paraphrasing of sentences and combining knowledge from the total article to generate summaries similar to what a human-written abstract does. An adequate abstractive summary is linguistically fluent and enwraps the core information of the input. On the other hand, extractive summarization methods create the summary by picking up sentences directly from the text, based on a ranking function to form a relevant summary. This method identifies essential sections of the text, cropping out and joining together parts of the content to produce an abridged version. We have used Extractive text summarization for English language in our approach.
The rest of the article is organized as follows. We review the state-of-the-art approaches related to extractive and abstractive document summarization in Section 2. Then, we presented our proposed approach in the Section 3. Section 4 discussed the results and analysis part of the work followed by conclusion and future scope in Section 5.
A brief literature survey[
Text Rank[
Further, a heuristic Approach for Telugu text summarization with improved sentence ranking
had been proposed by Mamidal et al.[
Preprocessing is the first process to be done in our method. It covers removing of stop words,
punctuations, unwanted data from the articles and doing sentence tokenization, word
tokenization. Stop words are the words that are commonly occurred and do not have a specific
significance meaning. Sentence tokenization divides the data into sentences and word
tokenization divides the data into words. Stop words removal will remove all the stop words from the
data after word tokenization. We have collected stop words for English language from Natural
Language Tool Kit (NLTK) library. We have used NLTK library for word tokenization and
for sentence tokenization, we used RegexpTokenizer from NLTK by writing our own regular
expression.
3.2. Text feature’s analysis
We have used various text features for scoring the sentences of each news articles. Let ,
represent the ℎ sentence of ℎ news article. And represent ℎ text feature. Then (, )
represent the score of ℎ text feature for a sentence , . The text features used in our approach
are as follows[
In a article, sentences present at the start and end are more relevant than those present in the
middle of the article[
2(, ) = 1 − |() − | , ||
(|, |) 1(, ) = ⃒⃒ 2 − ⃒⃒ 2 (1) (2) (3) (4)
Sentences which are like other sentences and have more common information are considered
more important. We calculate this feature score by taking the ratio of the intersection of words
in two sentences and the length of comparing sentence[
4(, ) = ∑︀|| ′=1,̸=′ ((, , ,′ )) ||
Words which are more frequent in the text are more important. Hence, the sentences containing
the frequent are also given more weightage than other sentences[
5(, ) = 5(, ) = ∑︀∈, ( ) | |
Sentences containing numerical data are regarded as more informational as numbers describe more analytical information.
6(, ) = ∑︀∈, ( ) || (5)
After computing the scores of each sentence for every text feature, a population based metaheuristic approach Genetic Algorithm (GA) assigns the weights to these text features.
Genetic Algorithm is a model or abstraction of evolution through natural selection based
on Charles Darwin’s theory of natural selection. It is a form of evolution that occurs on
computer[
• Each chromosome represents a potential solution. As a result, the population is made up of collection of chromosomes. • Each chromosome in the population is assigned a fitness value. As a result, higher fitness the better the solution is. • The best chromosomes out of the existing chromosomes in the population are used for reproducing the following generation ofsprings. • Because of crossover, the ofspring created will inherit characteristics from both parents.
A minor change in the structure of a chromosome will be called mutation.
The main advantages of GA over general optimization algorithms are, it supports multiple objective optimization, it is good for noisy environments and has the ability to deal with complex problems and parallelism. The basic steps present in GA are initialization, fitness evaluation, selection, crossover, mutation, and termination. The fitness function varies with respect to the problem. The steps to obtain the optimized weights of the text features are as follows (we consider our method as a maximization problem)
First, we generate random weights in (0,1) as the initial weights for the text features. These
are generated in the form of solutions = {1, 2, 3, ...., } [
Now, we have to evaluate the fitness of each solution. So, for calculating the fitness of each
solution, we have to rank the sentences in each article, with each element of the solution as the
weight of the corresponding text feature[
(, ) = ∑︁ , × ((, ))
=1
Where, , represents the ℎ dimension in the ℎ solution. (, ) represents the ℎ feature score for the ℎsentence in the ℎ article.
After ranking the sentences, select the top twenty percent ranked sentences in each article as the generated summary. We will extract the sentences for each article for every solution. Now, these extracted sentences are compared with the use of a fitness function.
Fitness Function: Here, we use ROUGE metrics ROUGE-1 and ROUGE-2 for calculating the similarity between the generated summary and the reference summary. For calculating the fitness of a solution, we have to calculate the sum of similarity between the generated summary and the reference summary for every article.
=1 = ∑︁ 1 [′ ′] + 2 [′ ′] 2 Where, e is the total number of articles present and 1 [′ ′] is the F1-score of the ROUGE-1 of ℎ article.
After calculating the fitness values of each solution, now we have to select the parents from the set of solutions to produce the ofsprings. We have to select half of the solutions for mating-pool. For selecting the parents, we have used elitist based roulette wheel selection. First, it selects the top 25 percent of the solutions as parents based on their fitness values. Then, it selects another 25 percent of the solutions from the remaining 75 percent solutions with the use of roulette wheel selection.
Crossover, which is also known as recombination is used for combining the information of two
solutions to produce a new ofspring. There are many diferent types of crossovers available.
But we have used simple single-point crossover. First, we select two parents from mating-pool
and select a crossover point. A new ofspring is produced by taking data before crossover point
from first parent and combining it with data after crossover point from second parent.[
Mutation is performed on the newly created ofsprings to bring some changes in them. It is used to bring diversity from one generation of solutions to the next generation solutions. We have (6) (7) applied mutation on two diferent points for each solution in the ofsprings. For each ofspring, we select a point in first half of the ofspring and select a random value in (-0.25,0.25) and add to the data at the selected point. Now, again for each ofspring, we select a point in second half of the ofspring and select a random value in (-0.25,0.25) and add to the data at the selected point. After performing mutation, all the new ofsprings and the parents in the mating-pool will be combined to form the solutions for next generation.
Once the number of predefined iterations are completed, the best solution having the highest iftness value will be selected as the final weights of the text features.
Once the optimal weights for the text features are obtained with the use of genetic algorithm, we generate the final summary. For producing the summary, we first score the sentences in each article with the optimized weights of the features. If the GA gives the optimal weights for text features at solution q = O, then the score of , is (, ) = ∑︁ =,=1 , × ((, )) (8) After ranking the sentences, the top twenty percent ranked sentences in each article are extracted as our summary.
In the English dataset, for each sentence in each of the articles, we calculate the values of all the text features. Then, we initialize 20 random individual population as the weights of the text features. Now, the GA is run for 200 generations. The optimized weights obtained using GA for the above described text features (in Section 3.3) are 0.86276778, 0.514368, 0.3003737, 0.49962974 and 0.01303291 respectively, which are shown in Figure-5.
The Recall-Oriented Understudy for Gisting Evaluation (ROUGE) scoring algorithm calculates the similarity between a text and a collection of reference articles. It determines the quality of the generated text. Rouge Score is the oficial metric for the ILSUM track.
By using these weights, we calculated the weighted average of the text feature scores of each sentence. We took the top twenty percentage ranked sentences in each article as the summary generated. The f-measure of ROUGE-1, ROUGE-2 and ROUGE-4 values of the summary generated and our rank for the English test dataset is shown in table-1. The detailed evaluation results of English test dataset containing F1-Score, Precision and Recall for ROUGE-1, ROUGE-2, ROUGE-3 and ROUGE-4 is shown in table-2. From the obtained evaluation results, it can be understood that our technique does not give great results as expected at least upto 50% F1-Score. This is mainly because we produce 20% of the sentences as our summary but the actual summary can be any number of sentences and also during the sentence tokenization, we used a regular expression which reduced the length of the sentences than expected as compared to expected sentences.
This work reports for the shared task of the Automatic Text Summarization for English Language Text -FIRE 2022. We have experimented with several types of algorithms for the shared task, including abstractive text summarization like BERT, BART, etc. For extractive text summarization, we have performed sentence ranking using text features. The text features are weighted using genetic algorithm, which gave us the better results. However the scores can be improved further by using slightly tweaking the genetic algorithm, by training for many more generations and by increasing the text features. In future, we can use this approach to summarize the documents written in other Indian languages such as Hindi, Bengali, Tamil, Telugu etc.
This work is supported by TIH, Indian Institute of Technology, Patna and in lined with it’s theme.