The task of Text Simplification (TS) aims at modifying textual content in order to improve its readability while preserving the meaning. Text Simplification can be categorized as Lexical Simplification, Syntactic Simplification, Explanation Generation, and Monolingual Machine Translation [1]. Lexical Simplification aims at identifying and replacing complex words (or phrases) by an easier-to-read alternative. The alternative can be a single word or a phrase. Syntactic Simplification (SS) simplifies the text by restructuring the words of the sentences, and/or rewriting it into smaller sentences. Explanation Generation systems take difficult concepts of a text and augment it with extra information in order to improve the context and user understanding, instead of just rewriting the input text. Monolingual Machine Translation based approaches treat the simplification problem as a translation problem from complex language to simpler language. This method usually simplifies both the lexicon and syntax of a text.
Long and syntactically complex sentences are difficult to understand for second language learners [2] and people with reading difficulties [3,4]. Syntactic Simplification typically involves sentence splitting, changing passive voice to active and resolving other ambiguities. The aim of the present work is to perform syntactic simplification of English sentences using their respective dependency tree structure. The proposed system focuses on rewriting along with splitting of sentences, if required, which contain passive voice, appositive, relative and conjoint clauses. It is considered lightweight in the sense that it relies only on the dependency structure of the sentence for syntactical transformations. It does not adopt more intensive simplification strategies, such as event-deletion, anaphora resolution, rhetorical structure theory that are being carried out in many other syntactic simplification systems as discussed in Section 2.
The rest of the paper is organised as follows. Existing related works are briefly described in Section 2. Section 3 presents the proposed approach. Experimental details are described in Section 4, and Section 5 presents the results. The paper is concluded in Section 6.
The earliest work on Syntactic Simplifcation by Chandrasekar et al. [5] developed rules for transformation of sentences containing relative clauses or appositions. It used Finite State Grammar to produce noun and verb groups, and a Supertagging model to produce dependency linkages via partial parsing. Chandrasekar et al. [6] automated the rule generation system for relative clauses using parallel data of 65 sentences. The transformation rules are identified using a tree-comparison algorithm, and they are generalized by replacing specific words by tags.
PSET developed by Caroll et al. [7] is the first assistive simplification system aimed for the benefit of people with aphasia. PSET applies both syntactic and lexical simplification on English newspaper texts as available on the Internet. Here, the sentences are converted into subject-verb-object order by replacing passive constructions with active ones. This was further improved using SYSTAR [8] to split compound sentences. It also converted seven passive clause types to active, and resolved and replaced eight frequently occurring anaphoric pronouns.
Previous research on text simplification did not account for the discourse level issues related of conjunction and anaphora that arise from applying syntactic transforms at the sentence level. In order to improve the cohesion in the simplified text, Siddharthan introduced a regeneration step in addition to transformation step [9,10]. It preserves conjunctive cohesion by using rhetorical structure theory for cue-word selection, sentence ordering and choice of determiner. Glavas et al. [11] performed simplification of sentences using an event-centered approach. Their system extracted events from a given sentence; and discarded text that did not belong to any of the factual events. However, it is an event-based simplification and conceptually different from lexical or syntactic simplification.
REGENT [12] applied transformation rules for syntactic simplification to typed dependency representations produced by the Stanford parser [13]. Simplification is performed for sentences containing coordination, subordination, apposition, passive constructions, and relative clauses. An overgenerate-and-rank approach was adopted by performing simplification on top-n parses of a sentence in order to reduce errors due to inaccurate parsing. The outputs were evaluated by determining the extent of simplification and level of accuracy in rule application.
Siddharthan et al. [14] developed a hybrid text simplification system using synchronous dependency grammars with hand-written and automatically harvested rules. This allowed for a linguistically sound treatment of complex sentences requiring reordering and morphological changes, such as conversion of passive voice to active.
Ferrés et al. [15] developed a hybrid text simplifier for English. Their syntactic simplification system used rule-based analysis and generation techniques based on PoS tags and dependency trees. Intrinsic evaluation on a self-collected dataset of 500 sentences suggested that correct simplifications were obtained for 74.2% sentences. In this system 66 rules have been used to simplify seven syntactic phenomena, namely appositive, relative, sentence coordination, coordinated correlatives, subordination, adverbial, and passive voice. Scarton et al. [16] developed a multilingual syntactic simplification tool (MUSST) using dependecy based rules. The rules simplify sentences containing conjoint clauses, relative clauses, appositive phrases and/or passive voice. The order of clauses were not taken into consideration when complex marker, such as although is replaced by a simpler marker but. An accuracy of 76% was reported for simplifying a set of 292 English sentences. However, system evaluations of this work for standard datasets, such as PWKP [17] and TurkCorpus [18] are not available.
Syntactic Simplification systems have also been developed for non-English languages, such as Spanish [19], Italian [20], Portuguese [21]. The SIMPLEXT system [19] utilizes the dependency structure of Spanish sentences to perform five types of syntactic transformation, such as separation of participial modifiers, splitting relative and coordinating clauses, reordering of quoted objects. ERNESTA [20] performs simplification of Italian sentences using an event-based approach and it focuses on anaphora resolution and syntactic simplification while removing unnecessary events. PORSIMPLES [21] is a rule-based simplification system for Brazilian Portuguese for simplifications of appositive, relative, coordinate and subordinate clauses. Additionally, sentence splitting and reordering of clauses were also performed.
More recently, the focus of researchers has shifted to Hybrid and Neural Text Simplification systems which perform both lexical and syntactic simplifications [22,23,24,25,26,27]. In the next section the proposed syntactic simplification algorithm, DEPSYM (DEpendency Parse based SYntactic siMplification) is discussed.
In the present work, rules based on dependency trees extracted from spaCy1 are utilized for simplification of appositive clauses, relative clauses, conjoint clauses, and passive-to-active conversion (See Table 1). Other dependency parsers for English (e.g. Stanford) may also be utilised for extraction of dependency trees but, since the proposed system is developed using Python, spaCy (version:2.2.4, en_core_web_sm) has been used for ease of implementation. It uses a non-monotonic arc-eager transition-system [28] for parsing sentences. Below we explain the different modules of the proposed approach.
In the first step, the appositive token, the one with appos tag, is extracted from the dependency tree of the sentence. If such a token is not found then token with amod tag is extracted. If neither of the two tags are found then appositive clause is not present in the sentence. The parent of the appositive token may be subject (with nsubj tag) or the object (with dobj/pobj tag). An in-order traversal in the sub-tree of the appositive token gives the appositive phrase. The noun phrase is determined by taking the left sub-tree traversal of the parent of the appositive token. The required auxiliary verb is determined using the tense of the root, and the singularity/plurality of the main subject. The appositive token and its sub-tree are deleted from the main parse tree. The first sentence is extracted by in-order traversal of the main tree. The second sentence is formed by combining the noun phrase, auxiliary verb, and the appositive phrase (See Figure 1).
The relative token, the one with relcl tag, is extracted from the dependency tree of the sentence. If such a token is not found then token with rcmod tag is extracted. If neither of the two tags are found then it is concluded that relative clause is not present in the sentence. The parent of the relative token gives the subject/object that the relative clause refers to. The left sub-tree of the parent gives the noun phrase. Inorder traversal of the relative token's sub-tree gives the relative clause. The relative token is then deleted from the main tree, and subsequent inorder traversal of the main tree gives the first sentence. The second sentence is formed by combining the noun phrase and the relative phrase. The noun phrase is not inserted within the relative phrase but the relative pronoun is retained in the relative phrase. The order of the two sentences is decided on the basis of whether the relative clause is attached to the subject or to the object (See Figure 2).
For illustration, the sentence Bob, who I live with in Delhi, is a nice man. is simplified as Bob who I live with in Delhi. Bob is a nice man., and the sentence Bob lives in Delhi, where his brother lives. is simplified as Bob lives in Delhi. Delhi is where his brother lives.
Conjoint clauses may contain a variety of conjunctions, such as though, although, when, because.
A sentence with a conjoint structure is typically made of two clauses 2 connected by a conjunction or an adverbial modifier. The dependency tags, namely conj, advcl, parataxis, and ccomp, connect the root of the sentence to the root of the second clause. The conjunction term or adverbial modifier is connected to the root of the second clause with a cc or advmod or mark tag. This token and root of the conjoint clause is removed from the main tree. If the conjoint clause has no subject then it is given the subject of the main clause (sentence). Inorder traversal of the main tree and the sub-tree of conjoint clause root gives the two separate sentences. The order of the two sentences depends on the type of conjunction and the position of the conjunction in the sentence. Conjunctions, such as although, whereas, however are replaced with but.
Conjunctions because and as are replaced with the conjunction so. Further, the order of the two related sentences (clauses) are reversed. For conjunctions before, after, once, since and when, the phrase this + auxiliary verb is inserted in the second sentence as shown in Figure 3.
Passive to active voice algorithm contains three modules, namely subject-object detection, verb tense transform, and pronoun form transform. To identify a passive sentence, the algorithm searches for a token with auxpass tag in the dependency tree. This token is a verb, but not the main verb, of the clause which contains the passive information. This is followed by a search for tokens with tag agent and nsubjpass. Absence of any of these results in no simplification. All 2 as observed in the development phase After that, agent and nsubjpass tokens are removed from the main tree.
The second module takes as input five parameters, namely the main verb (root), auxiliary verbs, auxpass token, subject token and subject POS tag. These are used to determine the tense of the sentence as described in Figure 4a, and also helps to decide the new form for the verb and the auxiliary verbs. Transformation of verb forms due to presence of pronouns and modal verbs are also performed. The third module is used to change the form of pronouns that are moving from subject to object or vice-versa. A dictionary containing common pronoun pairs, such as he: him, she: her, we: us, is used to carry out the transformation. Finally, the new sentence is constructed by interchanging the positions of subject-object in the tree and inserting the new verb. Other dependencies of the root are maintained in their positions; but any prepositional phrases to the right of root are moved to the end (See Figure 4b). Simplification of a complex sentence is performed recursively in the order appositive phrases, conjoint clauses, relative clauses and then passive voice. Finally, true casing3 is performed.
This section discusses experimental details about the datasets, baseline systems, automatic evaluation metrics and human evaluation procedure.
The proposed simplification algorithm has been evaluated on two commonly used test sets 4extracted from English Wikipedia (EW) and Simple English Wikipedia (SEW) namely, PWKP [17] and TurkCorpus [18].
• PWKP/WikiSmall: The test set for this dataset contains 100 sentences with only one simplification reference per original sentence. The references are extracted by aligning sentences of EW and SEW. • TurkCorpus (TC): The test set for this dataset contains 359 complex sentences. The following reference sets have been considered for this dataset:
-In MTurk [18], eight mannually written references focused on Lexical simplification are collected using Amazon Mechanical Turk for each sentence of TC . Here, the annotators were instructed to rewrite sentences by reducing the number of difficult words or idioms, but without deleting content or splitting the sentences. -HSplit [29] provides four reference simplifications for TC specifically for assessing sentence splitting. Two annotators were instructed to perform two tasks: (1) split the original sentence as much as possible, while preserving grammaticality, fluency and meaning, (2) split the sentence only when it simplifies the original sentence. -ASSET [30] contains ten references for TC focusing on several rewriting transformations, e.g. lexical simplification and reordering, sentence splitting, compression.
The following baselines 5 have been considered in the present work.
• Identity: This refers to the system where output is identical to the input.
• TSM: It aims at finding the best sequence of transformation of the constituency parse tree of the complex sentence to produce the target simplification [17]. It is trained on the train subset of PWKP. • PBSMT-R: It uses Phrase Based Statistical Machine translation along with a dissimilaritybased (measured using edit distance) reranking mechanism to chose among possible simplifications [31]. • HYBRID: It combines deep semantics and monolingual machine translation to derive simple sentences from complex sentences [23]. Semantic role information is used to determine events in the sentence for performing splitting and deletion. For substitution of complex words and sentence reordering a PBSMT-based model is used. • DSS: It is a sentence splitting algorithm which uses the UCCA semantic parser [32] to decompose a sentence to its main semantic constituents [26]. • YATS: It is a hybrid text simplification system which also uses a rule-based syntactic simplification system along with lexical simplification [15]. • DRESS: It uses a Reinforcement Learning (RL) architecture with standard attention-based encoder-decoder as an agent. The reward function uses a weighted sum of SARI [18], cosine similarity of input and output, and language model probability of output [24]. • ACCESS: It uses a discrete parametrization mechanism to improve performance of Seq2Seq models for the task of simplification. It adds control tokens corresponding to four attributes, namely amount of compression, amount of paraphrasing, lexical complexity and syntactical complexity [27].
For automatic evaluation of system outputs commonly used metrics, namely FKGL, BLEU and SARI have been utilized. Is the simplification easier to understand than the input? Structural Simplicity (StS) Is the output simpler than the input, ignoring the complexity of the words?
• Flesch-Kincaid Grade Level (FKGL) [33]: This metric corresponds to the grade level of a sentence. Lower FKGL implies simpler outputs and lower level of difficulty. • BiLingual Evaluation Understudy (BLEU) [34]: This is a precision-oriented metric used to measure the correctness of the generations by measuring n-grams overlaps between the generated sentences and (multiple) references. • System output Against References and Input sentence (SARI) [18]: This metric compares model-generated simplifications with both the input sentence and the gold references. It measures "how good" the words added, deleted, and kept by a simplification model are.
The above-mentioned metrics have been calculated using EASSE implementation [35]. Additionally, ten human annotators were asked to evaluate the system outputs on a 5-point Likert scale on the basis of Fluency, Adequacy, Simplicity, and Structural Simplicity as described in Table 2. Structural Simplicity is distinguished from Simplicity in order to ignore the effect of lexical simplification performed by the baseline systems. The annotators were non-native but fluent English speakers. Each annotator scored 260 system outputs6 correponding to 20 input sentences of each test dataset. YATS system was evaluated by two of the above annotators 7 . Fluency and Adequacy were measured using a 1 to 5 scale. Following previous works [26,25], a five point scale [-2, 2] is used to measure Simplicity and Structural Simplicity, where a 0 score indicates that the input and the output are equally complex. A score of 1 (-1) indicates that the output is slightly simple (complex) than input, and score of 2 (-2) indicates that the output is very simple (complex) than input. The advantage of using a -2 to 2 scale is that the sign of the score could indicate the efficiency of the simplification system. Negative scores indicate that the output is more complex than the input sentence.
The results for PWKP test set and TC test set are presented in Table 3 and Table 4, respectively.
The DEPSYM system8 proposed in this work performs well in terms of FKGL for both the test sets. The FKGL is reduced nearly by 50 percent with respect to the input sentences (Identity) for both the test sets for DEPSYM. However, since DEPSYM does not perform lexical simplifications or phrase deletions, BLEU and SARI scores are lower for PWKP, MTurk, and ASSET data. For TurkCorpus, BLEU scores for the DEPSYM system are consistent throughout the three reference sets. Moreover, it is higher than DSS and HYBRID systems. The BLEU and SARI scores obtained Human Evaluation indicates that in terms of Adequacy, the proposed system outperforms other baseline systems. With respect to Structural Simplicity DEPSYM performs the best for TurkCorpus, and for PWKP it came second highest. For Fluency and Simplicity, the values achieved by DEPSYM is highly competitive beating most of the baseline systems comfortably.
ACCESS and DRESS systems perform lexical substitution of the complex words, and thereby achieved higher scores with respect to simplicity. The outputs of DRESS are also rated as most fluent by the annotators. However, these systems perform poorly with respect to Adequacy as they often delete important phrases from the original sentence. Moreover, sometimes the lexical changes performed by these system lead to semantic errors in the output. For illustration, consider the simplified outputs for DEPSYM and baseline systems in Table 5. Here, HYBRID system omits important information, while other baseline systems replaces the word distributes incorrectly by collects or deals or sells. It can also be observed that the output obtained using the neural ACCESS system contain incorrect repetition of the phrase present location. The output of DSS system, which is developed for sentence splitting, contains an incomplete sentence 'was added in 1938-39'. The YATS system failed to simplify the conjoint clause present in the second example. In both the examples mentioned in Table 5, DEPSYM produces structurally simpler outputs with two or more smaller sentences.
Simplification of long and syntactically complex sentences is an important NLP task in order to facilitate accessibility to a larger set of audience. In the present work, syntactic simplification is performed by analyzing the dependency tree structure of a complex English sentence. The relationship between words in the dependency tree is utilised to split sentences containing any of appositive, relative or conjoint clauses. Furthermore, sentence rewriting from passive to active voice is also performed. Automatic and Human evaluations indicate that the proposed system produces structurally simpler outputs while preserving grammaticality and meaning. The proposed simplification system may be applicable to other languages having similar dependency structure as English. For languages where the dependency structures are different the rules will have to be fine-tuned in keeping with the linguistic properties of the language concerned.
The proposed DEPSYM algorithm uses the dependency parsing of the input sentence. Hence, parsing errors induced by SpaCy lead to some errors in simplification albeit inadvertently. In future, the passive to active voice conversion may be further improved to handle negative sentences. For illustration, He was not chased by the the police is incorrectly simplified as The police not chased him as it is unable to perform the transformation was not chased → did not chase. For sentences with relative clauses, the relative pronouns are retained in the simplified sentence. This can be improved by curating rules for insertion of the noun phrase within the relative phrase. For illustration, Bob, who I live with in Delhi, is a nice man. may be simplified as I live with Bob in Delhi. Bob is a nice man. In future we would also like to append a lexical simplification module with the current syntactic simplification algorithm.
This array distributes data across multiple disks, but the array is seen by the computer user and operating system as one single disk.
PBSMT-R This array distributes data across multiple disks, but the array is seen by the computer user and operating system as a single disk. HYBRID this array releases data across disks, but the array is seen by the user and computer operating system.
This array collects data across multiple disks, but the array is seen by the computer user and operating system. TSM This array sells data across multiple disks but the array is seen. The computer user and operating as one disk. YATS This array deals data across multiple disks, but the computer user and operating system sees the array as one single disk. DEPSYM This array distributes data across multiple disks. But, the computer user and operating system sees the array as one single disk.
The fourth ring is decorated with golden garlands and was added in 1938-39 when the column was moved to its present location.
PBSMT-R The fourth ring is decorated with golden garlands was added in 1938-39 when the column was moved to its present location.
The fourth ring is decorated with golden garlands. ACCESS The fourth ring was added in 1938 when the column was moved to its present location, and was added to its present location. DSS the fourth ring is decorated with golden garlands. was added in 1938-39. the column was moved to its present location. HYBRID The ring is decorated and was added the column was moved.
The fourth ring is decorated with golden Garlands. And the fourth ring was added in 1938 -39 when the column was moved to its present location. DEPSYM The fourth ring is decorated with golden garlands. And ring was added in 1938-39. This was when the column was moved to its present location.
The authors thank Daanish Bansal and Siddhant for helpful discussions. Raksha Agarwal acknowledges Council of Scientific and Industrial Research (CSIR), India for supporting the research under Grant no: SPM-06/086(0267)/2018-EMR-I. We thank Prof. Horacio Saggion for providing a working demo of YATS system. We also thank the reviewers for their valuable suggestions and constructive criticism.