Understanding the evolution of research trends is critical for navigating rapidly developing scientific literature. Large Language Models (LLMs) ofer powerful tools for analysing scientific texts, enabling the extraction of key concepts and the construction of semantic networks. These capabilities can support the study of emerging ideas and research trends through graph-based representations. In this paper, we present a network-based text analysis framework designed to map the evolution of scientific knowledge. Our goal is to extract conceptual structures from research papers and construct graphs that represent both the occurrence of terms and their interrelationships. The integration of temporal information allows us to track the emergence and transformation of research themes.
https://www.cs.cas.cz/~petra (P. Vidnerová)
CEUR Workshop
ISSN1613-0073 the underlying content. By combining both qualitative insights and quantitative metrics, the framework ofers a comprehensive tool for analysing large volumes of text, making it applicable across a range of domains where text-based information plays a central role.
The designed paper networks enable better orientation among scientific papers as well as retrieval mechanisms delivering papers based on topic similarity.
In this section, we propose a methodology for analysing a collection of research papers using network text analysis. We begin by outlining the process of constructing the dataset, which includes assembling the papers and incorporating citation information. Next, we detail the procedure for selecting keywords and generating network graphs.
Dataset
We construct a dataset of research papers using the arXiv repository. Specifically, we use the arXiv API to download papers based on a search query related to the topic or field of interest.
In our case study, we retrieved 10,000 papers using the query ”neural architecture search.” For each paper, we obtained both the PDF and metadata, including the title, authors, abstract, and other relevant information.
However, not all papers retrieved through this method are truly relevant to the target topic—particularly when using a large value of . To address this, we apply a filtering step using an LLM. For each paper, we provide the abstract and a prompt asking whether the content is related to the target topic.
After filtering out irrelevant papers, we retained a final set of 2,423 arXiv papers that we believe to be related to Neural Architecture Search.
Citations
The second step involves retrieving citation information from the OpenAlex [
To account for this, we search OpenAlex for all papers with identical titles (ignoring diferences in punctuation and whitespace) and aggregate the citations from these matched entries. We then consider only those citations that originate from the set of arXiv papers compiled in the previous step. Keywords
To enable the analysis of topic trends, we associate each paper with a set of keywords. We consider two approaches for keyword generation. The first involves prompting a large language model (LLM) with a paper’s abstract and directly requesting a list of keywords. The second approach starts with a manually curated set of fixed keywords; for each abstract-keyword pair, the LLM is then asked whether the keyword is relevant to the given abstract.
The first approach can yield rich and diverse keywords, but it requires substantial postprocessing to unify synonyms and eliminate overly general terms. The second approach produces a clean, boolean keyword vector for each paper, which is easier to analyse, but it depends heavily on the quality and scope of the initial keyword set, which may be limited or biased.
In this study, we adopt a hybrid method. We begin by generating a set of keywords for each paper using the first approach. We then aggregate all generated keywords, select the most frequent ones, and iflter out overly general terms (e.g., ”neural network” in the context of neural architecture search). The resulting refined list serves as our fixed keyword set for further analysis.
Networks
A key advantage of network text analysis is its ability to reveal structural relationships between concepts, keywords, or entities within a large text corpus. This is achieved by constructing graphs—sets of vertices and edges , where each edge is a pair (, ) such that ∈ and ∈ . Vertices represent entities (e.g., keywords or papers), and edges represent the relationships between them.
In our study, we construct three types of networks: a keyword network, a paper network, and a citation network.
• In the keyword network, vertices correspond to individual keywords. An edge is created between two keywords if they co-occur in at least one paper. Each edge is weighted by the number of shared papers in which the two keywords appear. • In the paper network, vertices represent papers, and an edge is established between two papers if they share one or more keywords. The edge weight reflects the number of shared keywords. • In the citation network, vertices again represent papers, but edges are directed: an edge (, ) indicates that paper cites paper .
These networks serve both as analytical tools and visualisation aids, helping to map relationships and
navigate large collections of research papers. Additionally, we can compute various network properties
(such as centrality measures [
The first centrality measure we use is the PageRank [
Secondly, we will use the betweenness centrality [
As a case study, we chose the field of Neural Architecture Search (NAS) to showcase the capabilities of the proposed text analysis method.
NAS is a subfield of automated machine learning ( AutoML) focused on the automatic design of neural
network architectures. Traditional deep learning models require significant manual efort and expert
knowledge to design efective architectures, which may not generalise well across tasks or datasets.
NAS aims to automate this process by searching through a predefined space of possible architectures
to find models that achieve optimal performance for a given task. It typically involves three main
components: the search space (defining which architectures can be explored), the search strategy (how
architectures are sampled or generated), and the performance estimation strategy (how the quality of
each candidate architecture is evaluated) [
Over the past few years, NAS has seen rapid advances in both eficiency and efectiveness. Early
approaches, such as those based on reinforcement learning or evolutionary algorithms, were
computationally expensive, often requiring thousands of GPU hours. More recent methods—like diferentiable
NAS (e.g., DARTS [
In the next section, we describe our findings on the field of NAS.
We used a dataset of research papers retrieved from arXiv, following the procedure outlined in Section 2,
resulting in a collection of 2,423 papers. As the LLM, we used the tiger-gemma-9b-v3:fp16 model because
of its relatively small size and good performance. Visualisations were created using the graph-tool
library [
The temporal distribution of these papers is shown in Fig. 1. While the topic has appeared in the literature since the late 1990s, a significant increase in research activity began after 2017. This rapid growth was driven by the increased availability of high-performance hardware and the widespread adoption of deep learning techniques.
The following figure, Fig. 2, displays the proportion of papers associated with each keyword relative to the total number of papers. Based on these keyword counts only, we can make several observations.
First, the keywords include several objectives such as eficiency, accuracy, inference time or latency, robustness, sparsity, adversarial robustness, and energy eficiency. It is evident that the proportion of papers addressing secondary but still important objectives—like energy eficiency or robustness—is much smaller compared to those focusing on accuracy or overall eficiency. Although multi-objective optimisation is increasingly important today and solutions face multiple demands, many studies continue to concentrate solely on accuracy.
Objectives such as latency and energy eficiency correspond to hardware-aware NAS, which focuses on optimising neural architectures not only for performance but also for practical deployment constraints on specific hardware platforms. In Fig. 3, we can see the proportions of keywords over time, and we can see that the importance of hardware-aware NAS increased over time.
Another group of keywords relates to the optimisation techniques used in the NAS process. Notably, multi-objective optimisation plays a significant role, appearing in nearly 30% of the papers. The most prominent optimisation methods include evolutionary algorithms, diferentiable architecture search, reinforcement learning, and Bayesian optimisation.
Fig.4 shows the number of papers using diferent optimisation algorithms, with evolutionary algorithms and diferentiable architecture search clearly dominating, and Bayesian optimisation being the least common. However, the trend over time, illustrated in Fig.5, tells a diferent story: diferentiable architecture search methods have gradually emerged and become the dominant approach, whereas evolutionary algorithms were more prominent in the early years of the NAS field.
The next step is to move beyond individual keywords and examine the keyword graph. In Fig.6, keywords are represented as vertices, with the size of each circle proportional to the number of associated papers. The complete graph is shown in Fig.7 (left), while Fig. 7 (right) highlights the most significant edges connecting the keywords.
The strongest edges are: • latency - inference time • performance prediction - surrogate model • multi-objective optimization - performance prediction • model compression - multi-objective optimization • hyperparameter optimization - performance prediction • multi-objective optimization - inference time • hardware aware - multi-objective optimization
The strong connection between latency and inference time is expected, as the two terms are essentially synonymous. Similarly, the prominent edge between performance prediction and surrogate model highlights the central role of surrogate models in performance prediction tasks. Additionally, the strong link between hardware-aware NAS and multi-objective optimization reflects the inherently multi-objective nature of hardware-aware approaches.
A diferent type of graph is presented in Fig. 8, where each node represents a paper and edges indicate shared keywords. This graph includes only papers tagged with the ”evolutionary algorithms” keyword, and outlier nodes have been filtered out. The resulting clusters reveal groups of papers that share common concepts, potentially aiding in the identification of related work. Papers associated with the ”performance prediction” keyword are highlighted in red.
We reach a deeper level of analysis by incorporating citation data. Fig.9 (left) presents the citation
graph, excluding singleton nodes, while Fig.9 (right) displays papers with more than five citations,
visualized using the algorithm from [
The three most cited papers (taking into account only citation inside the dataset) are SNAS: Stochastic
Neural Architecture Search [
On the other hand, papers with most references are Weight-Sharing Neural Architecture Search: A
Battle to Shrink the Optimization Gap [
Citation counts correspond only to the number of incoming and outgoing edges in the citation graph. However, they do not capture the broader structure of the network. To address this limitation, we have used the two centrality measures – PageRank and betweenneess centrality–described in section 2.
The most influential papers according PageRank are:
• Designing Neural Network Architectures using Reinforcement Learning [
• DeepArchitect: Automatically Designing and Training Deep Architectures [
• Single Path One-Shot Neural Architecture Search with Uniform Sampling [
• Neural Architecture Search: A Survey [
In this paper, we introduced a network-based text analysis approach supported by large language models (LLMs) to explore and better understand the structure of scientific literature. By applying our methodology to the field of NAS, we demonstrated its ability to uncover key research trends, thematic clusters, and the evolution of methods over time. Through keyword co-occurrence networks, citation analysis, and centrality measures, our approach provided insights into dominant areas within the NAS domain.
The integration of LLMs enabled finer keyword extraction and classification, enhancing the quality of the resulting networks. Our hybrid method ofers a scalable and interpretable way to analyse complex scientific landscapes and can be readily adapted to other research domains.
Future work will focus on extending this methodology in several directions. One promising direction is the incorporation of full-text analysis, which would allow for a deeper semantic understanding beyond abstracts and keywords. However, the work done shows that regarding keywords and topics extractions, using abstracts only gives suficient results.
Additionally, applying dynamic network analysis could help capture how research topics and their interconnections evolve over time. Another direction involves refining the integration of LLMs—for instance, by using them not only for keyword extraction but also for automated topic labeling, trend detection, and hypothesis generation.
Future work will also explore the use of our approach for novelty detection, aiming to identify emerging topics, unconventional connections, or underexplored research directions within the scientific literature.
This work has been funded by a grant from the Programme Johannes Amos Comenius under the Ministry of Education, Youth and Sports of the Czech Republic ’Knowledge in the Age of Distrust’ (reg. no. CZ.02.01.01/00/23_025/0008711) and by the long-term strategic development financing of the Institute of Computer Science (RVO:67985807).
arXiv:1904.00420. [22] C. Sciuto, K. Yu, M. Jaggi, C. Musat, M. Salzmann, Evaluating the search phase of neural architecture search, CoRR abs/1902.08142 (2019). URL: http://arxiv.org/abs/1902.08142. arXiv:1902.08142. [23] L. Li, A. Talwalkar, Random search and reproducibility for neural architecture search, CoRR abs/1902.07638 (2019). URL: http://arxiv.org/abs/1902.07638. arXiv:1902.07638. [24] Z. Yang, Y. Wang, X. Chen, B. Shi, C. Xu, C. Xu, Q. Tian, C. Xu, CARS: continuous evolution for eficient neural architecture search, CoRR abs/1909.04977 (2019). URL: http://arxiv.org/abs/1909. 04977. arXiv:1909.04977.