The amount of information scientific society produces on a daily basis results in the necessity of researchers to have proper guidance in a digital space. The function of the virtual assistance is performed by various scientometric systems, research paper recommender systems (Haruna et al., 2017) and different kinds of search engines. (Shvets et al., 2015) summarizes the most common types of systems for scientometric analysis. The recent trend in scientific paper delivery is purpose specific webresources, blogs, and e-journals often coupled with email subscriptions. They often provide personalized recommendations based on users' behavior and preferences.
The recommendation usually has a form of imprint often limited only by a title (as in the case with email subscriptions associated with limited space and lack of time to attract people's attention). Eventually, the success of recommendation depends on the informativeness of the title of an article subject to user's intentions and acknowledgment with a certain scientific field. This denotes the necessity of finding a way of varying the title of the same paper for different categories of users.
The focus of this paper is in developing models for creating a variety of simplified versions of the titles of scientific articles which would be condensed and informative enough and at the same time would correspond to the original topic of a paper to maintain users' loyalty. We aim at supporting two scenarios of personalized simplification: the first ensuring narrow focus on specific scientific concepts for goal-oriented experts and the second providing a general overview for researchers working on the edge of a topic willing to expand their horizons. The second case should not be treated as a generation of clickbaits (catchy short misleading headings) that are to be blocked with the use of efficient machine learning approaches (Biyani et al., 2016).
There is a variety of algorithms that could be used for title simplification which is a rapidly growing research area (Bouayad-Agha et al., 2009;Saggion et al., 2015;Guo et al., 2018). As long as the defined task is similar to text compression and abstractive summarization we made a choice towards encoder-decoder neural architectures (Nallapati et.al, 2016, Nikolov et.al, 2018).
The remainder of the paper is structured as follows. In Section 2, we propose a method for scientific title diversification and simplification. Section 3 is devoted to describing the datasets used for training. Section 4 denotes the experiment setup. Section 5 provides results of numerical experiments. Section 6 is devoted to human evaluation. In Section 7, finally, we discuss results and outline future work.
Recent advances in natural machine translation (NMT) incite to solve the task in a supervised manner controlling the style of a title by conditioning training data. The method we propose comprises the following steps: a) selecting a subset from an abstract-to-title dataset to impose conditions that would force a model to generate hypotheses with desirable properties; b) training a sequence-to-sequence (seq2seq) model; c) applying a model to title-to-title generation; d) performing post-processing step to remove unnecessarily repeated tokens; e) filtering titles with improper structure. The remainder of this section describes each step in details.
To create titles of different styles for various categories of researchers several datasets should be used. The set of highly popular scientific titles may help to generate attractive headings for users with interests peripheral to the subject of a paper. The condition to have a multi-word noun phrase NPmw in a target text is to avoid producing overly shortened pointless titles. In case each training example contains a reference text Rt and a target text Tt that have similar NPmw (at least 2 common terms), a model might learn to preserve the most important concepts from original titles needed by experts. Figure 1 shows the training example with similar NPmw-s in Rt and Tt.
Input sequence (an abstract, lower case, tokenized, truncated)
Target sequence (a title, tokenized, lower case) effects of order of presentation on conditional reasoning the main goal of this research is to study whether or not the order of presentation of the premises in a logical argument form , such as a conditional reasoning task , could affect In particular, we used the realization in OpenNMT toolkit (Klein et al., 2018) enabling pointer that allows copying tokens from the reference text. The trained model is to be applied to new unseen titles, which are, in opposite to abstracts (cut-off after 50 tokens in our experiments), not truncated. Since the task differs from general NMT task and summarization task by the absence of need in tracking alignment, traditional coverage mechanism (Wu et al., 2016), that discourage repetitions, is not included not to impose potentially harmful restrictions and not to overcomplicate the model. Instead, we introduce the postprocessing step PS as follows. Firstly, each repetition of a term is removed leaving the only occurrence closest to the beginning of a text. Secondly, all the auxiliary tokens without required terms in between or after them are eliminated. In the end, we iteratively remove the last token in a text if it is an adjective or auxiliary token and, in addition, capitalize the title.
The last step consists in filtering improper titles, i.e., generated sequences that have less than two NPmw-s similar to some NPmw-s of the source title. In those use cases when even potentially pointless output is required, this step should be skipped.
We chose ResearchGate1 platform as a source of data. It has a recommender system and therefore openly counts the number of times a page with a paper was visited to provide reasonable recommendations that motivates authors to be more visible.
We selected 150K imprints of articles on various topics using a wide list of general scientific words (Osipov et al., 2014) as an entry point to the articles. Figure 2 To detect noun phrases we used Spacy chunker (Honnibal and Montani, 2017) that we elaborated for detecting complex phrases, which map single concepts (e.g., "vertex energy of a graph" that is a lexical variation of the concept "graph energy").
Selecting the first 75 characters of the reference text is generally used as a baseline in summarization tasks; cf., e.g., (Rush et al., 2015). We added subsequent cut-off after the last noun in a phrase. This improved baseline is referred to as MBase henceforth. Several seq2seq models (M1, M2, …) with the above-described architectures differed by a number of layers were applied to various datasets to bias the style of output text. They were then extended with post-processing PS (M1ps, M2ps, …) and filtering steps, which are novel for the best knowledge of the author; cf.
The most of the basic models performed reasonably: produced titles were in general shorter than original, multiple-word noun phrases from reference title covered a significant part of the generated title (NPdiff-p = 0.68 on average). However, some models, especially M5, introduced many repetitions (for all checkpoints): the BLEU value reflected it being equal to 0.18 for M5 while the average value for the rest of models was equal to 0.35. Since BLEU depends on a number of same word occurrences, the increase of it by 24% on average due to PS attests usefulness of the step (cf. Table 2). Filtering step allowed dropping less informative titles so that one can take advantage even of poor models reducing a risk to present misleading picking-eye headings or generic topics to an user (cf. Figure 3 for examples of generated texts).
The extension of basic models led to an increase of NPdiff-p by 9% and rouge-L-f by 11% on average. Table 3 gives an idea of the variation of titles of different models in style and in compression rate.
It is worth noting that 1-layer models M1 and M2 trained on conditioned datasets reached higher values for the majority of measures in comparison to models M3 and M4 fed with generic data. This highlights rationality in pre-directing the training.
For human evaluation, we selected five papers of the NLP research group (TALN UPF) with titles longer than 93 characters (10-18 words). Their authors who own Ph.D. degrees were asked to rank output titles for these papers including original title by preference on clicking if they saw a title briefly in a daily email digest. To face different decision criteria assessors worked with papers of their authorship (for simulating expert behavior) and with papers of their colleagues (expanding horizons use case). If some titles in a set were the same or assessors did not have any preference between two similar titles they were allowed to rank them equally. The top models sorted by the average rank and examples of titles from one set are listed in Table 4. Noted final increase of NPdiff-p and rouge-L-f indicates that common subsequences became longer in relation to the length of titles meaning that offered post-processing step with filtering plays an important role in forming a fluent text. At the same time, the output should not have been just one of the original subsequences, therefore, we did not aim at reaching too high precision values. Pure state-of-the-art seq2seq models without post-processing step got low ranks on human evaluation. The models M1ps and M2ps have a higher average rank of 6. Their titles are well-formed and represent a combination of original multi-word expressions (cf. Table 3 for relatively high scores of rouge-2-r), however, less corresponding to the topic that is partly reflected by comparatively lower values of rouge-L-p. The outputs of the models M3ps and M5ps were often preferred to original titles. Having 1.3 times shorter titles than M5ps, conditionally trained M6ps achieved almost the same average score. The baseline has the highest rank since it often better preserves the meaning although does not always form a complete phrase. The main drawback is that it usually only generalize a title to some extent (in case of well-turned subsequence) and miss details experts might need.
The close average ranks of models and rouge-L-f on the same level for all models denote an opportunity to overcome the general problem of lacking the variability in neural seq2seq generation. Different title styles give a possibility to reach a preferable trade-off between the conciseness of the title and its transparency.
For future work, we plan to gain value from methods of paraphrasing (Cao et al., 2017), advanced simplification (Zhang and Lapata, 2017; Štajner and Saggion, 2018) and surface realization for deep input representations (Belz et al., 2018) to obtain diverse semantically close outputs differ from text reformulated with mostly the same words. Fake-paper detecting (Byrne and Labbé, 2017) and assessing the quality of scientific texts (Shvets, 2015) will help to avoid training the models on misleading titles. Finally, pre-existing taxonomies (e.g., JEL codes in Economics, the ACM taxonomy in Computer Science, the Web of Science categories attached to journals), and meta information of papers such as authors' keywords or KeywordsPlus items inferred from the references cited (Garfield and Sher, 1993) are to be used for preselecting the most relevant concepts to bias the training.
The presented work was supported by the European Commission under the contract numbers H2020-700024-RIA, H2020-700475-IA, H2020-779962-RIA, H2020-786731-RIA, and H2020-825079-RIA and by the Russian Foundation for Basic Research under the contract number 18-37-00198. Many thanks to the four anonymous reviewers for their valuable comments, and to the five postdoctoral researchers for their high responsiveness in the evaluation and insightful feedback.