Virtual particles are peculiar objects. They figure prominently in much of theoretical and experimental research in elementary particle physics. But exactly what they are is far from obvious. In particular, to what extent they should be considered “real” remains a matter of controversy in philosophy of science. Also their origin and development has only recently come into focus of scholarship in the history of science. In this study, we propose using the intriguing case of virtual particles to discuss the efÏcacy of Semantic Change Detection (SCD) based on contextualized word embeddings from a domain-adapted BERT model in studying specific scientific concepts. We find that the SCD metrics align well with qualitative research insights in the history and philosophy of science, as well as with the results obtained from Dependency Parsing to determine the frequency and connotations of the term “virtual”. Still, the metrics of SCD provide additional insights over and above the qualitative research and the Dependency Parsing. Among other things, the metrics suggest that the concept of the virtual particle became more stable after 1950 but at the same time also more polysemous. Semantic change detection, digital conceptual history, history and philosophy of science, virtual particle
Virtual particles have been important elements of particle physics since long. But despite their
widespread use, the term “virtual particle” holds diferent meanings and connotations within
today’s particle physics community, and its historical origins and development have remained
unclear. Virtual particles are peculiar objects which may be considered responsible for the
fundamental interactions of matter and radiation. In this sense, they have detectable and real
efects. However, they do not share the properties of real particles; for instance, the mass and
energy of a virtual particle does not stand in the same relation as would be the case with a
particle that is observed in the appropriate detectors. Virtual particles only ever occur in
intermediate, unobservable phases of decays or other processes involving elementary particles. The
precise meaning and interpretation of the term “virtual particle” has, therefore, been a topic for
philosophical debate [33]. Recent works by Ehberger [14] and Martinez [
To achieve this, we combine conceptual history with computational methods, an approach
also referred to asdigital Begrifsgeschichte [34]. First, we adapt our BERT model to the
domainspecific language of our large corpus of physics texts and extract contextualized word
embeddings for all occurrences of the term “virtual”. These word embeddings can then be used to
employ Semantic Change Detection (SCD), which aims to identify, interpret and assess shifts
in lexical meaning over time using computational techniques. SCD has emerged as a distinct
research field in recent years supported by multiple survey studies [e.g., 30, 32]. While most
studies focus on the technical implementation of SCD, there have also been calls for further
evaluation of the methods through in-depth case studies backed by qualitative analysis2[
Our dataset consists of a large number of scientific articles from eight journals of the
Physical Review-family. The corpus spans from the introduction of the concept of virtuality in
quantum physics in 1924 up to 2022, the latest complete year available for analysis, making it
well-suited for studying the history of the virtual particle. ThePR-journals are highly
influential in the field of physics [
The dataset’s substantial size, comprising nearly 700,000 articles, makes it well-suited for extensive analysis using computational methods. However, it also presents notable limitations, particularly concerning the early development of the concept. As a primarily US-based source written exclusively in English, significant developments from other regions are not captured. For instance, the center of the old quantum theory in the 1920s and early 1930s was in Central Europe, particularly in German-speaking countries, the Netherlands, and Denmark. Since they also published mostly in German journals, most of their works are not a direct part of this study. Another issue is the relatively small number of articles in the corpus published before 1950 (approximately 12,000 articles or just under 2 percent). For a more comprehensive analysis of the early phase of the concept using, it would be necessary to incorporate additional text sources.
Analyzing articles in the entire corpus using word embeddings is impractical due to scalability issues. Instead, we first identify articles potentially relevant to the concept of the virtual particle through a keyword search for “virtual” in the full texts, abstracts, and titles. Approximately half of the full texts are available as digitized and OCR-processed PDF files (331,210 entries before 2004), while the other half are in native digital XML format (329,880 entries from 2004 onwards). For processing the PDF-files we use GROBID 2, which allows parsing and restructuring of scientific publications in PDF format into uniformly TEI-formatted3 XML files. To catch common OCR-errors prevalent the PDF-extracted text data, we apply some basic cleaning steps like removing special characters etc. Subsequently, citations and mathematical formulas are also removed from the text. While the formulas used likely reflect significant developments in the conceptualization of the virtual particle, there are currently no established tools for the content analysis of mathematical formulas in the context of conceptual history and the history of science.4 Therefore, this work focuses on the analysis of linguistic text data.
To ensure the efÏcient use of the BERT model the texts are segmented into sentences. For
this task, we utilize the large language model of the Python natural language processing library
SciSpaCy5, which has been trained on a large corpus of scientific texts (albeit in bio-medicine),
making it suited for this purpose. We also use the model for dependency parsing, where a
sentence’s syntactic structure is created by identifying how words are grammatically related
through directed links. This is particularly helpful for analyzing adjectives like “virtual”, as it
allows for accurate identification of the associated nouns. We use these dependencies to evaluate
2GROBID stands for GeneRation Of BIbliographic Data h(ttps://grobid.readthedocs.io/en/latest)/.
3https://tei-c.org/
4We consider this an important open problem in semantic change detection in scientific texts. Also, it is hard
to estimate the impact of the omission of mathematical formulas. On the one hand, the symbols used in the
formulas are usually explained in the surrounding text (which we do take into account). On the other hand,
diferent mathematical formulas may describe diferent virtual entities (particles, states, processes etc.) without
clear indications of this in the surrounding text. Moreover, as one of the anonymous reviewers pointed out, the
frequency of formulas might have changed over time, which makes the omission potentially more or less impactful.
We did not control for this.
5https://allenai.github.io/scispacy/
and gain a deeper understanding of the observed semantic shifts. Following Laicher, Kurtyigit,
Schlechtweg, Kuhn, and Schulte im Walde [
For Semantic Change Detection using BERT[
We therefore employ the BERT-base-uncased mode9l, which features 12 attention layers
and a hidden layer size of 768, and apply domain-adaption on our “virtual”-corpus. We also
retrained and tested SciBERT [
The basic procedure of Semantic Change Detection (SCD) can be outlined as follows: Given a diachronic corpus of document s =
⋃=1
, where represents a subcorpus of documents at time within the overall investigation period[1, … , ] that contains the target word . The goal of SCD is to quantify the semantic shift for between two time-specific subcorpora and ′ or across the entire corpus. There are two ways a semantic shift can manifest: firstly, as a change in the dominant meaning of a term, or secondly, as a change in the degree of its polysemy. Both aspects will be analyzed in this study. Specifically, for our purposes the target word is “virtual”, the documents comprise all the full texts plus abstracts of thePR-corpus that contain “virtual”, and the time interval is one year.
The generalized work-flow required for performing contextualized SCD can be split into
three steps. In the first step ( Embedding), contextualized word embeddings are generated for
each occurrence of the target word in the corpus using a large language model like BERT. The
set of all these embeddings in the time-specific subcorpus is expressed asΦ
where , represents a contextualized word embedding in the subcorpus and denotes the
number of all occurrences of in it. In the second step (Aggregation) the embeddings of
= {
, , … ,
, },
8Recently, PhysBERT [
′ measured by directly comparing the degree of polysemy in the time-specific set of embeddings and Φ . Sense-based approaches, in contrast, attempt to first capture the diferent timespecific senses or meanings of the target word in specific meaning corresponds to a cluster of embeddings , using clustering methods. Each time
in the set of embeddingsΦ .
We apply two clustering methods to identify meaning clusters. InK-Means Clustering (KM), embeddings are organized into a predefined number of clusters by iteratively updating cluster centers until stable. Determining the optimal number of clusters is challenging; meaning clusters [25]. Therefore, we set the number of clusters to = 10 automated methods like the silhouette coefÏcient often fail to identify the actual number of based on qualitative assessment. AfÏnity propagation (AP)
identifies exemplars among data points and forms
clusters without the need to pre-specify their number by iteratively exchanging “messages”
between data points to determine the clusters. However, the number of clusters often correlates
with the number of input embeddings rather than actual meanings, potentially resulting in a
large number of clusters [
We apply two methods to quantify the temporal development of a term’s polysemy. The ifrst method is Shannon entropy (
), which utilizes the cluster distribution to describe
the degree of uncertainty in the distribution of embeddings across meaning clusters within a
given time period. Specifically, Shannon entropy quantifies the average amount of information
needed to assign a particular embedding, i.e., an occurrence of the term “virtual”, to a specific
cluster, i.e., a specific meaning of the term “virtual”. A higher value of ( )indicates a higher
degree of polysemy, as there is greater uncertainty or variability in the cluster membership of
the embeddings [
, ). AID is defined as follows:
The second method,Average Inner Distance (AID), utilizes the variance of the
contextualized word embeddingsΦ , reflecting the degree of polysemy of in . In this approach,
embeddings are not aggregated into meaning clusters or word prototypes. Instead, the average
distances between all possible pairs of embeddings within a single time period are calculated
[
1
To assess the shift in dominant meaning in a form-based manner, Cosine Similarity (CS) can
be used. CS measures the alignment between the vectors of two word prototype s and ′
by calculating the dot product of the vectors divided by the product of their norms (lengths).
CS values range between -1 and 1, where a high value indicates vector alignment and a low
value indicates opposition. We employ the variantInverted Cosine Similarity over Word
Prototypes (PRT), which, according to Kutuzov, Velldal, and Øvrelid 2[
PRT( , ′ ) =
CS( , ′ ), where CS( , ′ ) =
⋅ ′ ⋅ ‖ ‖ ‖ ′ ‖
The shift in dominant meaning can also be assessed using meaning clusters (sense-based) through the Jensen-Shannon Divergence (JSD). JSD, based on normalized Shannon entropy, measures the similarity between cluster distributions across diferent time periods. This method considers not only the variation in the size of the clusters but also how the size of specific clusters across the diferent time periods changes [ 17]. A high JSD value indicates significantly diferent cluster distributions, suggesting pronounced semantic shifts. Conversely, a low JSD value indicates relatively similar distributions, implying stability in the dominant meaning. JSD is defined as follows:
JSD( , ′ ) = ( 2 ( + ′ )) − 1 2
( ( ) − ( ′ )) ⋅ 1 1
Articles containing 'virtual' per year (~42k total)
Share of articles containing 'virtual' 1200 1000 tn 800 u o c lce 600 i t r A 400 200
The first result of our study is the descriptive analysis of the “virtual” corpus in regards to the temporal development of the term. Figure1 shows the number of published articles per year containing “virtual” for the entire corpus (left) and their proportion per journal (right). The dashed lines in the left figure indicate two key disciplinary diferentiations in the PR journals: the transition from Series II to PR A - D in 1970, and the introduction of new journals like PR - X (2011) and PRX - Quantum (2021). To focus on long-term trends, these newer journals are excluded from the analysis. The decline in articles after 2010 is thus an artefact of the dataset and does not reflect overall trends in PR publications or physics. Notably, there is a low number of articles in the early phase of the study period, with only 384 publications in our corpus containing “virtual” before 1950, especially sparse before 1930 and during the war years (1942-1945). The exact number of articles, “virtual”-embeddings and cleaned tokens per year for the early phase can be found in the appendix (table4). From 1950 onwards, the number of articles containing “virtual” increases steadily, with short periods of relative stagnation during the 1970s and 2010s, mirroring the broader increase in PR journal publications. Additional details on the total publication count per journal are available in the appendix (figure 4).
The average share of articles containing “virtual” across all journals, as depicted in the right
ifgure, is 6.04 percent over the entire period. In the pre-Feynman era (before 1950), this
percentage generally remains lower, except for two notable peaks. In 1937, there is a temporary
increase above 5 percent, driven by significant contributions from Bethe, Bacher, and Livingston
in RMP [
Zooming in on the individual journals or disciplines respectively, articles containing “virtual” are notably prevalent in PR - D (particle physics, field theory, gravitation, and cosmology) and PR - C (nuclear physics). Examination of arXiv classifications within PR - D reveals that nearly 90 percent of these articles fall under high-energy physics. The frequency of “virtual” inPR D peaks in the 1970s, 1990s and 2000s with drops in usage in between. Overall, it contributes approximately 27 percent of all articles containing the term “virtual” in the corpus, making it the largest source. Nuclear physics (PR - C) also features a significant percentage of articles containing “virtual”, comprising about 9 percent of the corpus. This aligns with recent research by Martinez on the origin of the notion of virtuality in modern physics27[]. The proportion of relevant articles inPR - C increases steadily until the mid-1990s, plateaus until around 2010, and shows a recent decline. The term is less prevalent in the remaining journals, which will not be discussed in detail here for the sake of brevity. A table showing the top 5 journal-specific dependencies of “virtual” can be found in the appendix (table3).
One key finding of our study is that the dominant meaning of “virtual” becomes more stable over time. Figure 2 presents the results of the SCD-calculations regarding the shifts in the dominant meaning throughout the entire investigation period. The left graph displays the PRTvalues for “virtual”, i.e., the inverted cosine similarity of the word prototypes for each year to preceding year. The right graph shows the JSD-values for both the K-Means-Clustering and the AP-Clustering. Due to the computational expense of AP-Clustering, we randomly sampled approximately 25 percent of all embeddings, ensuring a minimum of 400 embeddings per year, where available.
The resulting conceptual development of “virtual” can be divided into two distinct phases. 1.14 1.12
The first period, up until the 1950s, is characterized by pronounced fluctuations, indicating repeated conceptual reorientation during the early development of the concept, with no firmly established or dominant meaning. This trend can be seen in all three metrics, although the values for JSD on the basis of AP-Clustering stabilizes at around 0.4. Notably, peaks are observed in the late 1920s and early 1940s. Given the limited number of data points available for this period, it is important to emphasize that our results for this early period reflect general trends rather than individual peaks. To ensure the robustness of our results, we conduct permutation-based statistical tests, which are described in detail at the end of this chapter. From approximately 1950 onward, marking the beginning of the second phase, the dominant meaning begins to stabilize progressively, although a minor peak is observed in the early 1980s. This suggests the growing establishment of the concept of the virtual particle, following the outlined contributions of Feynman and Dyson. Additional details on the shifts in dominant meaning in the discipline-specific journals can be found in the appendix (figure 5), indicating that the peak in the 1980s is mainly caused by a change in dominant meaning inPR - C (nuclear physics). We plan to conduct further research into the cause of this and other peaks.
Our findings regarding the stabilization of the dominant meaning of “virtual” are also
supported by the time-specific dependencies, as shown in Table 2. From the 1920s to the 1940s,
“virtual” is most often associated with terms as diverse as “cathode”, “height”, “orbit”, “level”,
and “oscillator”. In the 1940s, “virtual quanta” came into use, prominently featured in
Feynman’s first diagrams [
The consistency of results across all three calculation methods, despite their diferent
approaches, also notable: The values of PRT strongly correlate with those of JSD (Pearson
coefÏcient for PRT and JSD - KM: 0.96, PRT and JSD - AP: 0.8), as well as the those of the two
JSD metrics (0.77). These high correlation values suggest that both clustering methods reliably
identify the various meanings of “virtual”, indicating stable and meaningful results. To further
ensure the robustness of our findings despite the relatively low frequency of “virtual” in the
early years, we employ permutation-based statistical tests for the PRT-metric, following the
approach outlined in Liu, Medlar, and Glowacka 2[
AID 1.0 0.8 yp o r t n
E 0.6 -on n n a h
S
0.4 ed
z
li
a
m
r
0.2 oN
Entropy (AP)
Entropy (K-Means) 0.0
assumptions; instead they generate the sampling distribution based on the available data itself.
This is achieved through the random and repeated rearrangement of the “virtual”-embeddings
across the two time periods by sampling without replacement and then recalculating the
SCDmetric for each permutation.12 Following Liu, Medlar, and Glowacka 2[
Degree of polysemy of 'virtual' over time 1920 1940 1960 1980 2000
The second key finding of our study is that the degree of polysemy of “virtual” increases. That means that while the most dominant use is that in association with the aforementioned concepts, its usage in diferent meanings is also expanding. Figure 3 presents the development of the degree of polysemy for “virtual” in the entirePR-corpus and over the entire investigation period. The left graph shows the AID-values, i.e., the average inner distances of all “virtual” embeddings in a given year. The values for the normalized Shannon-Entropy are displayed in the right graph, again for both the K-Means-Clustering and the AP-Clustering (with the same random sampling as described in Section4.2). 12We limit the number of permutations to a maximum of 100,000 per time interval, i.e. two subsequent years, to save computational resources.
Similar to the results regarding the dominant meaning, the degree of polysemy fluctuates significantly in the early phase of the concept. Notably, the values are particularly low in the mid to late 1920s and early 1940s. These results are expected given the limited number of articles during these periods, as a small number of embeddings implies a correspondingly low number of diferent meanings. From 1938 to 1940, however, the values for all calculation methods are particularly high. A clear explanation for this spike is not immediately apparent, as neither the examination of the dependencies nor the shift in dominant meaning during these years provide insight. The described peaks in PRT and JSD occur several years later. One possible explanation could be that few but very diferent embeddings cause the peak. While the correlation coefÏcients between the metrics are again high (0.64 for AID and Entropy (KM), 0.66 for AID and Entropy (AP), and 0.94 for Entropy (KM) and Entropy (AP)), suggesting stable results, we were, however, unable to identify a suitable method for statistical testing of polysemy. Further research and qualitative assessment of the relevant papers is required and planned. Consequently, our present analysis focuses, once again, on general trends rather than individual peaks.
From around 1950 or 1960, depending on the metric, the fluctuations become smaller and the degree of polysemy continues to steadily increase. Notably, there is a brief spike in the early 1980s in the AID-values and another sharp increase in the 1990s, followed by a relative stabilization in recent years. This increase in recent years is also reflected in the dependencies of “virtual” (table 2), with the most frequent usage contexts becoming more evenly distributed from the 1990s compared to earlier decades. This trend is supported by the introduction of the journal PR - E in 1993, which is characterized by distinct usage contexts difering from those of other journals (see table 3). The Shannon-Entropy based on both clustering methods remains consistently high, exceeding or maxing out at 0.8 from about the 1950s onward and reaching nearly maximum values around the 2000s in the case of K-Means. From 2010 onward, there is a small decrease in polysemy, possibly due to the second disciplinary diferentiation leading to a slightly less varied usage of the term across the remaining journals. The trends observed in discipline-specific journals generally align with the overall findings. The details can be found in the appendix (figure 7).
We have used a large number of contextualized word embeddings to employ various Semantic Change Detection metrics in order to trace the diachronic development of the concept of the virtual particle. Our findings show that the dominant meaning of “virtual” becomes more stable over time while at the same time its degree of polysemy is increasing. This development can be split into two periods: An initial phase characterized by repeated conceptual reorientation with no firmly established meaning yet, and a second phase marked by the growing consolidation of the dominant meaning in the sense of the virtual particle, following the seminal works of Richard Feynman and their reception around 1950. Simultaneously, the degree of polysemy steadily increases throughout almost the entire investigation period and only recently seems to stabilize at a high level.
While these two findings might seem contradictory at first, they can easily be reconciled. Simply put, the metrics for polysemy measure how spread out the word embeddings are in the vector space, while the metrics for dominant meaning measure where the relative majority of the embeddings lie and how this position changes from year to year. Our findings suggest that from the 1950s onward, the relative majority of the embeddings consistently centers around a usage in the sense of the virtual particle (especially virtual photons), while the overall usage of the term “virtual” diversifies, possibly due to its uses in diferent disciplines like those in PR - E.
We have combined our SCD-based approach with evaluation via Dependency Parsing as well as qualitative assessment of the results. We find that the observed semantic shifts are largely supported by recent work in the history of the virtual particle. This is particularly true for the first period of the conceptual development, whereas SCD can be employed in a more heuristic manner for the still relatively under-researched second phase. For instance, we identified a notable and unexpected shift in dominant meaning in the 1980s, primarily driven by articles in nuclear physics (PR - C). We plan to conduct further research into this peak as well as a more in-depth discussion of the relevance of our findings for the history and philosophy of physics.13 The complementary method of Dependency Parsing revealed that most of the semantic shifts coincide with significant changes in the most prominent dependencies at that time. While Dependency Parsing may have been particularly efective in our case because “virtual”, the focus of our study, is an adjective, it could prove to be a valuable and resourceefÏcient evaluation method for broader use in SCD research.
This work was supported by the DFG Research Unit “The Epistemology of the Large Hadron
Collider” (Grant FOR 2063). The members of the Unit provided valuable feedback at several
stages of this work. Special thanks go to Robert Harlander, Jean-Philippe Martinez, Rebecka
Mähring, Arno Simons and Friedrich Steinle as well as three anonymous reviewers for their
comments and helpful suggestions. The work is based on M.Z.’s MSc thesis, which has been
defended at the University of Leipzig (Computational Humanities Research Group), and was
supervised by A.W. and Andreas Niekler. We are also grateful to the American Physical Society
for granting us access to the relevant full texts and metadata.
13Many further preliminary results are contained in M.Z.’s master thesis on the topic 3[
M. Zichert. “Eine digitale Begrifsgeschichte des virtuellen Teilchens”. M.Sc. thesis. University of Leipzig, 2023.
Series II PR - A PR - B PR - C PR - D PR - E Letters
RMP
Published articles for PR-corpus per journal
D , C , B ,
A iIIrse f-PR e o foS rtta d S n
E
Year
Shifts in dominant meaning of 'virtual' for discipline-specific journals
PR - A PR - B PR - C PR - D
PR - E p-values (unadjusted) p-values (Benjamini-Hochberg) 2020 B. Tables