The increase in power and availability of Large Language Models (LLMs) since late 2022 led to increased concerns with their usage to automate academic paper mills. In turn, this poses a threat to bibliometrics-based technology monitoring and forecasting in rapidly moving fields. We propose to address this issue by leveraging semantic entity triplets. Specifically, we extract factual statements from scientific papers and represent them as (subject, predicate, object) triplets before validating the factual consistency of statements within and between scientific papers. This approach heavily penalizes blind usage of stochastic text generators such as LLMs while not penalizing authors who used LLMs solely to improve the readability of their paper. Here, we present a pipeline to extract such triplets and compare them. While our pipeline is promising and sensitive enough to detect inconsistencies between papers from diferent domains, the intra-paper entity reference resolution needs to be improved to ensure that triplets are more specific. We believe that our pipeline will be useful to the general research community working on the factual consistency of scientific texts.
For firms to make informed investment decisions, sound
forecasts on the development of technologies are necessary.
One prominent method for technology forecasting is
bibliometrics, which uses the information in scholarly books
and journals [
While scientific publications are thus increasingly more
important for technology forecasting, the quality of the
papers must be evaluated critically. The publish-or-perish
pressure led to a record growth in the number of
scientific publications per author, often with minimal peer
review [
Recently, there has been a growing interest in identifying
text generated by LLMs. As early as 2019, Zellers et al.
showed that a GPT2-like LLM Grover could detect its own
output. However, recent research suggests that, in general,
LLM detectors either do not work or are easy to evade [
Unfortunately, the situation is serious enough for some
of the most reputable providers of proxies of the impact
of scientific articles to have modified their algorithms to
only consider publications adhering to stringent criteria
[
Specifically, we represent facts as entity triplets of the form (subject, predicate, object) that are extracted from the claims of the paper. The entity triplet plays a crucial role as it serves as a proxy to understand the primary claims of the paper and subsequently validates factual consistency compared to other works in the domain. Our paper describes the workflow involved in entity triplet extraction and provides an overview of our initial findings regarding the efectiveness of the entity triplets and their relation to the number of clusters generated around the subject. The code of this project is available at https://github.com/technometrics-lab/ 0-Factual_Consistency_Through_Entity_Triplets, at commit c7b01e4.
Previous approaches for claim extraction can be categorized
into heuristics and machine learning methods. The
advantage of an approach based on heuristics is that no training
data is required and the computational cost tends to be low.
However, machine learning approaches can capture more
complex patterns, leading to the extraction of triplets of
higher quality. Such methods have been developed most
prominently in the biomedical domain. For example, Li et
al. (2021) use BiLSTMs to extract the factual statements
presented in papers [
The majority of existing triplet extraction models use
supervised training. Two notable examples are RECON and
sPERT, which require labeled training data [
Stanford OpenIE, which extractsrelational tuples without
the need to specify a schema in advance. However, it has
been shown that OpenIE tends to extract too aggressively,
resulting in the presence of non-useful relations [
The process of extracting informative triplets from raw PDFs consists of four main stages. First, PDFs are converted to text files, after which they are preprocessed to remove word breaks and citations, expand abbreviations and lemmatize words. Then, we extract the sentences from the paper that convey its core ideas, which we refer to as claims. From these claims, we can extract subject-predicate-object triplets. The last step is to process these triplets further so that they can be used in a comparative analysis. The entire pipeline is displayed in Figure 1. In the following subsections, we elaborate on each of the steps. While we focus on arXiv, the approach is generalizable to all scientific PDFs.
To convert the pdf to text we use the PyMuPDF library [
After preprocessing the text, we identify the sentences
that convey the authors’ claims. Specifically, we use the
ClaimDistiller framework developed by Wei et al. (2023)
[
Next, we want to reduce the claims to (subject,
predicate, object) triplets, analogous to the Resource Description
Framework (RDF) format commonly used in the
representation of OWL ontologies. We choose this representation, as it
will facilitate the comparison of claims across papers. We use
the Python library textacy, which is built on spaCy [
Our goal is to compare triplets across papers. Therefore, we further process the triplets so that we can pair triplets from diferent papers that refer to the same subject. The following steps are followed: 1. Lowercase all words in the triplet 2. Remove triplets where either the subject or object contains more than 6 words 3. Remove stopwords from the triplets based on the list included in NLTK 4. Remove any character that is not text 5. Lemmatize verbs and nouns in the triplet 6. Remove words containing less than 3 characters 7. Filter the triplets non-specific to scientific work by comparison with a general book corpus 8. Filter the triplets characteristic of scientific works in general by comparison with arXiv articles from diferent categories
In the second step, we choose this cutof, as we expect that phrases of over 6 words may contain nuances that cannot be captured in a simple subject-predicate-object relation. Words with less than 3 characters are removed, as we observed that such words were often noise. Moreover, as abbreviations are expanded we expect all informative terms to be at least of length 3.
We use the general-purpose Gutenberg book corpus to iflter the triplets that carry little information. We define the number of times term appears at least 5 times in a document in the book corpus and in the paper corpus as , and ,, respectively. We then assign a score to each term : = ⎪ ⎩∞ ⎧ ⎪−∞ ⎨( , ) − ( , ) , < 10 if , ≥ 10 and , > 0 if , = 0 and , ≥ 10
Terms that are not present in at least 10 papers, thus get a score of −∞ . If the term is present in at least 10 papers, the score increases when the frequency of the term in the book corpus is lower. We keep the triplets with subjects in the top 10% of the term scores.
In the last step, we aim to keep only the triplets that carry domain-specific information. Therefore, we sample a random subset of 1000 arXiv papers from December 2023 from diferent categories than our target papers. We then only keep the triplets with subjects present in a maximum of 15 papers.
As we use the extracted triplets to compare papers, it is
necessary to cluster them based on the subject and object.
Both SciBERT [
After clustering, we compare the triplets within the same
cluster based on the predicates. We take a first step in this
direction by analysing embedding inversions, as simple vector
arithmetic can provide valuable insights into word
relationships, such as negation or gender variants [
4.1. Data We consider two diferent datasets to evaluate our method. First, to leverage in-house expertise in the domain of computer science and natural language processing (NLP), we focus on publications relevant to the domain and retrieve data from the arXiv categories cs.AI, cs.CL and cs.LG. Specifically, we retrieve all papers from December 2023, which amounts to a total of 4225 research articles.
Second, for a quantitative analysis of the usefulness of
triplets for factual consistency evaluation, we consider two
surveys. To validate our approach in an independent
domain, we considered both a survey on LLMs and a survey
on Quantum Computing. Specifically, we analyse a survey
on LLMs by Zhao et al. (2019) [
After preprocessing the articles, we extract the triplets. For the triplet extraction, we used the hyperparameters displayed in Table 5 in the appendix. In total, 79,986 triplets were extracted from the research articles. We cluster these triplets based on both the subject and object embeddings, which resulted in 37,076 clusters. Figure 3 in the appendix shows the distribution of the number of triplets per cluster. This shows that most clusters contain less than 25 triplets, but that there are outliers that contain over 200 triplets.
For a more in-depth analysis, we consider the triplets extracted from the LLM and quantum computing surveys and their cited papers. In total, 1895 triplets are extracted from the 188 papers. We cluster these triplets based on the subjects so we can evaluate the diferences between the objects in a cluster. Figure 4 in the appendix shows that larger clusters often have triplets from multiple categories, whereas small clusters tend to have triplets from only one category.
Next, we make pairwise comparisons between the object embeddings within a cluster. Figure 2 shows the pairwise distances (L2 norm) between objects for each cluster size. We find that for clusters below size 8, the distance between objects from the LLM data and quantum data is larger than the distances of objects within a category. For larger cluster sizes, this efect disappears. This indicates that for smaller clusters with triplets from both categories, the objects are more diverse. Furthermore, we see that for clusters with triplets from one category, the distance between objects increases for larger sizes. This confirms that larger clusters are more domain-agnostic and contain more varied objects.
Table 2 shows manually selected clusters with sizes 2, 4 and 8. The column with mixed data clusters shows clusters that contain both triplets from the LLM data and the quantum data, where the triplets from the quantum data are displayed in italics. For smaller clusters, the triplets within a dataset tend to be similar and vary only slightly. In contrast, the objects difer more for the mixed data clusters as they are domain-specific. On the other hand, the results suggest that larger clusters contain more domain-agnostic subjects and objects, such as lab. Consequently, the distance between the objects from diferent datasets difers less than between objects from the same data. Further manual inspection supports this hypothesis with large clusters containing subjects such as appendix and conclusion.
Overall, we argue that it means that triplets as extracted by our pipelines can be used as proxies for factual consistency, but that additional refinement is needed to avoid extracting overly generic statements.
As a first step in evaluating the factual consistency between
papers, triplets in the same cluster are compared based
on the predicates. Specifically, two triplets are considered
consistent when the predicates are synonyms, hypernyms
or hyponyms. If the predicates are antonyms, the triplets
are considered inconsistent. The VerbOcean and WordNet
databases are used to label pairs of predicates [
To do a more qualitative assessment, we invert the diferences of the embeddings of predicates from the same cluster. Table 6 in the appendix shows a manual selection of 9 of these embedding inversions. The results show that an embedding inversion does not provide informative results in this context. In general, we do not find that there is a noticeable diference between embedding inversions of predicates that are consistent or inconsistent.
This paper presents an unsupervised method for the extraction of triplets from scientific work. Whereas previous methods either require labeled training data or the prespecification of entity relations, we allow for entity triplet extraction through domain specification.
The results show that the extracted triplets accurately reflect the domain from the corresponding scientific work. When we cluster triplets based on the subject, we find that smaller clusters tend to be domain-specific. In contrast, larger clusters are more generic and often contain triplets from diferent domains. We interpret it as extracted triplets being suitable for evaluating factual consistency, but requiring further refinement for a more specific extraction. We believe this is due to insuficient resolution of excessively general nouns (e.g. lab, conclusion). To compare the triplets, an embedding inversion was implemented on the diference of the verb embeddings for similar triplets. Our findings show that an embedding inversion does not allow us to discriminate between consistent and inconsistent triplets.
Our results suggest that the next steps for the usage of the extracted triplets for the development of LLM-resilient proxies should focus on better filtering of domain-agnostic subjects, for them to be informative about factual consistency. Then, a semantic network can be built based on the similarities between the triplets for the entirety of the scientific publications in a domain of interest. By leveraging this network, we can identify papers that are factually inconsistent or excessively consistent and use the remainder of the corpus for a bibliometric analysis.
During the preprocessing of papers, we expand abbreviations and map them to their long form. To compare the performance
of diferent abbreviation detection algorithms, we evaluate them on the paper Fundamentals of Generative Large Language
Models and Perspectives in Cyber-Defense, of which we have a thorough understanding [
Table 4 shows the performance of the Schwartz-Hearst [
Original triplets (example, illustrates, behavior), (example, mimic, behavior) (architecture, accomplishes, score), (architecture, achieves, score) (type rnns, perform, baseline model), (type rnns, outperform, baseline model) (image representation, extract, concept), (image representation, capture, concept) (language model, incurs, cost), (language model, slash, cost) (text, represents, knowledge), (text, requires, knowledge) (subgradient method, not ensure, convergence), (gradient algorithm, enjoy, convergence) not ensure
enjoy Verb 1 illustrates
Verb 2 mimic accomplishes
achieves performs
outperform extract incurs capture slash represents requires (knowledge transfer, demonstrates, improvement), (knowledge transfer, not maintain, improvement) demonstrates
not maintain
Top 10 embedding inversions ILLUSTRATES, ILLUSTRATING, schematically, EXEMPLARY, SUMMARIZES, DEPICTS, EMBODIMENT, ILLUSTRATED, ILLUSTRATIVE, DESCRIBES rigamarole, ERRAND, busywork, AFTERWORDS, canvasing, thigns, harrasing, forementioned, explaning, Busy-Work PERFORMS, PERFORMING, PERFORMED, PERFORM, SINGS, CONCERT, ACTs, SONG, RENDITION, PLAYS COMPLIANCE, COMPLY, INSUFFICIENT, IDENTIFIED, ENSURE, AUDIT, INDICATED, NON-COMPLIANCE, DETERMINES, IMPROPERLY EXTRACT, EXTRACTS, DECOCTION, TINCTURE, GINSENG, TURMERIC, Comfrey, KOLA, ALOE, Stevia INCURS,Accrues, ASCERTAINS, INCUR, INCURRING, incure, howsoever, INCURRED, internalizes, Indemnified REPRESENTS, REPRESENTED, REPRESENTING, RepresENT, ABSCISSA, symbolises, PERSONIFIES, DEPICTS, symbolised, Respresents DEMONSTRATES, demonstates, demostrates, EXEMPLIFIES, Dissects, Elucidates, DECONSTRUCTS, ILLUSTRATES, explicates, EXPLORES Distance