Interdisciplinary research is playing an important role in modern science. In recent years, a lot of studies have measured interdisciplinary knowledge flow based on the frequency of citations. However, this approach does not consider the content of knowledge carried in the citations. In this study, we attempt to investigate the content of knowledge flow towards an interdisciplinary field by analyzing the citation sentences (i.e., citances ) in the articles of the field. An emerging field, eHealth, is chosen in the case study. The associated knowledge phrases between citances and the references of the field are identified and categorized to analyze the content and categories of knowledge spread from the source disciplines to the field. The result shows that the ranks of disciplines by the frequency of associated phrases are consistent with the ranks by the frequency of in-text citations. Distribution of associated phrases over categories and disciplines is also analyzed. The associated phrases of research subject are the most, followed by entity. This study contributes to the understanding of content characteristics about interdisciplinary knowledge integration.
• Theory of computation ~ Semantics and reasoning ~ Program semantics ~ Categorical semantics; • Information systems ~ Information systems applications ~ Data mining; • Information systems ~ Information retrieval ~ Retrieval tasks and goals ~ Information extraction
Interdisciplinary research has become an important research
paradigm and many recent significant breakthroughs in science
are the fruits of interdisciplinary research. One fundamental
†Corresponding author.
feature of interdisciplinary research is the integration of
knowledge from multiple disciplines out of the field [
Recent studies have shifted to investigate interdisciplinary
knowledge flow from a finer-granular perspective by looking into
the content and contexts of citations. Citation contexts have
become more easily obtained in recent years, which embed the
syntactic (e.g., the location of section and rhetoric style) and
semantic (e.g., the meaning of citation content) information of
citations[
In this study, we attempt to explore the content of knowledge
integrated into an interdisciplinary field, eHealth, by analyzing the
citances. The field of eHealth is an emerging field, referring to all
aspects of the intersection of health care and the Internet[
Copyright 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). parsed from the XML files. Sentences were extracted by using the punctuations (periods, question marks, etc.) as sentence boundaries, then citances with in-text references were identified.
In total, 115,456 citances and 140,572 reference records were obtained.
To complete the abstracts of references, the reference records were fetched by searching PubMed for PubMed ID or Web of Science (WoS) for DOI. In total, the abstracts of 89,649 reference records were collected. 2.2
To explore the source of input knowledge, the references were then categorized into the 22 disciplines of Essential Science Indicators (ESI). We used the 2018 version of ESI journal list that covers 11,727 journals with full titles, abbreviated titles and their disciplines they belong to.
We designed a pipeline to determine the ESI disciplines of the references. First, 7,393 distinct journal titles were obtained from the 104,888 reference records with the citation type of ‘journal’ and with DOI/PubMed ID. We manually completed the full titles for the abbreviated journal titles that cannot be found in the ESI journal list but with more than 2 references. Next, we identified the disciplines of references by matching their journal titles with the journal titles in ESI. However, there were still 8,393 reference records without the ESI discipline information. Since the coverage of journals in ESI is not as broad as in WoS journal list, the WoS subject categories were then used to infer the ESI disciplines of the journal titles that were not matched directly. We designed a method to map the WoS subject categories into the ESI disciplines. We calculated the likelihood of a WoS subject category belonging to an ESI discipline through its journals whose ESI disciplines are known. The ESI discipline with the highest probability was then determined as the ESI discipline of the WoS category. If a journal has multiple WoS subject categories, we also chose the ESI discipline that has the highest probability with all the WoS categories.
Finally, approximately 94.09% of journal reference records (98,685) get the discipline information.
Citation contexts contain information about the cited articles
relevant to the citing papers[
To analyze the types of the knowledge that flows to the eHealth
field, we designed a classification framework of associated
knowledge phrases based on the previous studies [
Initializing knowledge classification framework. One author constructed a preliminary classification schema after reviewing the literature. Then the author randomly selected 100 knowledge phrases for trial annotation, organized the annotation details, and wrote an annotation specification document that provides detailed definition to each category with a few exemplar concepts.
Pre-annotation. Pre-annotation training was carried out for the two coders. Subsequently, two coders independently annotated 500 identical knowledge phrases randomly selected for pre-annotation. After labeling, we calculated the kappa statistics to assess the agreement of the two coders. The kappa was equal to 0.65, which was not as good as expected. Thus, two coders discussed the ambiguous cases with a professional in the eHealth field. We find some phrases may not make sense if they appear alone, but they are meaningful in the given context, therefore, there were many phrases that categorized into the research subject category or others category by different coders. After the discussion, two coders reached a consensus. included in the above categories
USA
Formal annotation. The two coders annotated all 24,132 unique phrases. During the annotation process, two coders maintained communication with the professional in the eHealth field to reach an agreement.
Our final framework contains seven categories, including research subject, theory, research methodology, technology, entity, data and others, which are defined in detail in Table 1. 3 3.1
We obtained 3,221 papers from the eHealth field with the publication year between 1999 and 2018. Some characteristics of our dataset for analysis are given in Table 2. In total, 115,456 citances and 98,685 reference records (55,744 distinct articles) with discipline information were extracted from our corpus. The 98,685 reference records were cited a total of 134,516 times (i.e., in-text references) in all citances. Roughly 90% of the reference records have abstracts.
To address our research question, we analyzed the distribution of
references over disciplines. Table 3 shows the number of unique
cited articles, CountOne citations, and in-text citations for the 22
disciplines. The CountOne citations were obtained by counting
each reference only once in a citing paper, whereas the in-text
citations count all the mentions of references in the paper[
In total, 215,138 associated knowledge phrases were extracted between the citances and the 123,206 in-text references with abstracts. Here, we only analyze 211,454 knowledge phrases associated with the top 10 disciplines (98.29% of all). Table 4 presents the frequency of associated knowledge phrases by discipline. It should be noted that only references with abstracts were used to extract associated knowledge phrases, therefore, the numbers of in-text citations in Table 4 are different from those in Table 3. Clinical Medicine contains the most associated knowledge phrases, followed by Social Sciences, General and Psychiatry/Psychology. The ranks of disciplines by the frequency of associated knowledge phrases are in harmony with the ranks by the frequency of in-text citations.
In addition, we calculated the knowledge density in the flow (i.e., the average number of phrases per citation) through dividing the frequency of phrases by the number of citations for each discipline. On average, every citation from the disciplines carried more than one associated knowledge phrase. The scores of knowledge density are slightly different between the 10 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
Social Sciences, General Psychiatry / Psychology Neuroscience & Behavior Multidisciplinary
Economics & Business Biology & Biochemistry Pharmacology & Toxicology Agricultural Sciences Engineering Molecular Biology & Genetics Mathematics Environment / Ecology Chemistry
Animal 9371 1914 1259 1153 839 693 632 567 546 303 254 181 181 80 51 46 27 26 5 2 47968 22530 15915 2414 2052 1660 1185 949 1041 710 839 357 323 271 216 91 53 47 30 26 6 2
In-text citations 66673 30196 21606 3152 2754 2278 1464 1222 1398 963 1145 441 425 312 249 94 44 38 36 15 9 2 disciplines. Pharmacology & Toxicology exceeds other source disciplines, with the most phrases per citation, while Computer Science contains the fewest phrases per citation. other disciplines. Computer Science has a higher proportion of technology phrases comparing with other disciplines. This could be explained by that Computer Science provides the study of eHealth with a lot of technique support, and many eHealth research problems are related to Computer Science. According to the annotation result, the number of associated knowledge phrases is shown for each category in Figure 1. The phrases in the category of research subject are the most, accounting for 43.8%. It shows that authors usually cite references related to their research subject. One noticeable thing is that there are many phrases in others, which is the second most. Such phrases often involve specific authors’ names, geolocations, specific projects, funding and some meaningless phrases. These phrases are not subdivided in our classification framework. In addition, the categories of entity and technology have more phrases than research methodology. This result may be due to the field of our corpus is medical-related, the research in which requires the use of many medical instruments, and the research entities it targets often varies in terms of research subjects (e.g., different diseases).
Figure 2 presents the number of associated knowledge phrases in different categories over the disciplines. The knowledge category distribution over different disciplines is significantly different (Pearson Chi Square test, p-value < 0.001). The top 3 disciplines, Clinical Medicine, Social Sciences, General, and Psychiatry/ Psychology, supply the most numbers of phrases in all categories. For each discipline, most of the associated knowledge phrases are research subjects.
In general, the distribution of associated knowledge phrases in each discipline over the categories are similar to the overall distribution in the entire dataset. However, a few exceptions are also observed. The proportion of theory phrases over all the phrases in Economics & Business are much higher than that in This study investigates the knowledge flow towards the interdisciplinary field of eHealth from the perspective of knowledge content. We extracted the knowledge phrases shared between the citances in the field with the references to represent knowledge content spread from source disciplines to the field. A classification framework was applied to annotate the identified knowledge phrases to explore the knowledge types of the phrases. The interdisciplinary features of eHealth are shown by analyzing the associated knowledge phrases.
The findings of this study could provide a few insightful implications on interdisciplinary knowledge integration. The result shows that the ranks of disciplines by the frequency of associated phrases are consistent with the ranks by the frequency of in-text citations. It means that to measure interdisciplinary knowledge flow, an indicator based on the frequency of shared phrases may produce similar results with the indicator using the frequency of references, in that the in-text references from different disciplines often carry similar amounts of phrases (Table 4). Associated phrases can indicate the spread content, which may be useful to generate knowledge map of interdisciplinary knowledge integration. However, they do not directly differentiate citations, thus, it is not enough to only consider phrase frequencies to measure interdisciplinary knowledge integration at the aspect of content.
The frequency distribution of knowledge phrases over the categories is heavily skewed. Except others, the most in-text references carry the phrases of research subject, followed by entity. The results show the distribution of different types of knowledge from the source disciplines. The types of knowledge phrases can be used as an important feature to differentiate references, for instance, the motivations of citations. The categories of knowledge will be helpful to understand the roles of source disciplines in the knowledge integration of an interdisciplinary field.
A few limitations can be identified as well. To obtain full text of research articles, we only chose the two open access journals to represent the field of eHealth, which may not cover all the articles of this field. The problem of data deficiency is common in fulltext based domain analysis. To identify the knowledge transferred from source disciplines to the interdisciplinary field, shared phrases are extracted by using simple text matching. However, synonyms are often used in citing others’ work, thus the coverage of the shared knowledge may be in short.
We also identified some directions of future research. We manually annotated the categories of associated phrases. To support the analysis on large scale datasets, automating the classification of spread knowledge is on great demand, which is a challenging task of our interest. This motivates us to design a more general classification framework to analyze the content of knowledge spread between disciplines. In addition, recent machine learning techniques will be applied to this task in our future study.
This study was funded by the National Natural Science Foundation of China (Grant No. 71804135) and Ministry of Education Humanities and Social Sciences project in China (Grant No.19YJC870018). We also thank Jing Tang for helping us with the data processing.