<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>A Scientometric Analysis of Publications Related to Predictive Medicine*</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Oleg Zolotarev</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          ,
          <addr-line>Maria Berberova</addr-line>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>FRC CSC of the Russian Academy of Sciences Moscow, Russia</institution>
          ,
          <addr-line>Moscow</addr-line>
          ,
          <country country="RU">Russia</country>
        </aff>
        <aff id="aff2">
          <label>2</label>
          <institution>Research Center for Physical and Technical Informatics</institution>
          ,
          <addr-line>Nizhny Novgorod</addr-line>
          ,
          <country country="RU">Russia</country>
        </aff>
        <aff id="aff3">
          <label>3</label>
          <institution>Russian New University.</institution>
          <addr-line>Moscow</addr-line>
          ,
          <country country="RU">Russia</country>
        </aff>
        <aff id="aff4">
          <label>4</label>
          <institution>School of Management, Hefei University of Technology</institution>
          ,
          <addr-line>Hefei</addr-line>
          ,
          <country country="CN">China</country>
        </aff>
      </contrib-group>
      <fpage>0000</fpage>
      <lpage>0001</lpage>
      <abstract>
        <p>Due to the increasing popularity of new research in medicine this study was conducted to determine recent research trends of predictive, preventive and personalized medicine (PPM). We identified the terms relevant to PPM using own search engine based on neural network processing in PubMed database. We extracted initially about 15000 articles. Then we carried out the statistical analysis for identifying research trends. The article presents the results of solving the problem of evaluating research topics at the level of thematic clusters in a separate subject area. An approach based on the analysis of article titles has been implemented. Identification of terms, connections between them and thematic clustering were carried out using the free software VOSViewer, which allows to extract terms in the form of noun phrases, as well as to cluster them.</p>
      </abstract>
      <kwd-group>
        <kwd>Predictive Medicine</kwd>
        <kwd>Preventive Medicine</kwd>
        <kwd>Personalized Medicine</kwd>
        <kwd>Biomedicine</kwd>
        <kwd>Trends</kwd>
        <kwd>Key Terms</kwd>
        <kwd>Machine Learning</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>2[0000-0003-3557-009X],
3[0000-0002-6357-7929],
Bibliometric mapping helps transform most of the metadata of publications into maps
or visualizations, from which useful information can be obtained through
post-processing.</p>
      <p>
        Bibliometry is one of statistical methods to analyze the mass of literature and to
reveal historical development [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ], as well as a scientific qualitative and quantitative
study of publications. Many authors used bibliometric in different areas of medicine,
such as ophthalmology [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ], rheumatology [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ], otolaryngology [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ], nephrology [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ],
geriatrics [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ], etc.
      </p>
      <p>
        For example, Zhu &amp; Guan (2013) [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ] and Sinkovics (2016) [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ] visualized keywords
to identify research topics or clusters in specific disciplines. Zhu &amp; Guan [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ] looked at
keywords and topic categories of publications as actors for mapping a keyword sharing
network and a topic sharing network and compared them with corresponding random
binary networks. Most of these studies focus on identifying major trends in the form of
the most cited studies or the most frequently used terms. While this is an excellent
method for identifying major research topics [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ], emerging or potentially interesting
topics may not be easy to spot.
      </p>
      <p>
        VOSviewer uses the VOS display technique (visualization of similarities) and is
freely available (www.vosviewer.com). Maps are generated from a sharing matrix. The
similarity matrix is calculated using the measure of the strength of the association [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ].
After transforming the data into visual form, VOSviewer offers two ways to display it:
network visualization and density visualization. The network visualization view
displays concepts based on their importance. The larger the label and circle, the more
important the concept. The color of the circle indicates which cluster the term belongs to.
Density visualization shows the importance of areas depending on the number of
connected elements.
      </p>
      <p>When creating maps based on a text corpus, the user can choose between binary and
full counts. When choosing a binary count, only the presence or absence of the term in
the document is considered. In the case of a complete count, all occurrences in the
document are considered.</p>
      <p>
        Content analysis is one of the areas of bibliometric analysis and it includes the
identification of trends based on words (Huffman et al., 2013 [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ]; Menendez-Manjon,
Moldenhauer, Wagener, &amp; Barcikowski, 2011 [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ]; Sooryamoorthy, 2010 [fourteen]).
      </p>
      <p>
        Gelman and Unwin [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ] warned against overuse of maps or renderings. They
recommended supplementing such maps with traditional graphs and tables to provide
additional evidence.
2
      </p>
    </sec>
    <sec id="sec-2">
      <title>Subjects and Methods</title>
      <p>The general process of visualization of scientific maps and their subsequent analysis
consisted of the following stages. As a principle for constructing the matrix, a network
based on jointly occurring keywords was chosen</p>
      <p>1. Obtaining a semi-structured amount of information from information sources
containing document annotations, as well as information about authors, their affiliation,
publication date, keywords, citation information (universal information retrieval
system Dimensions; specialized PubMed database).</p>
      <p>2. Pre-processing of data to improve the quality of the generated map (the formation
of groups depending on the period).</p>
      <p>3. Construction of scientific maps and their visualization.</p>
      <p>4. Analysis of scientific maps, identification of the most intensively developing
topics, analysis of temporal, statistical aspects.</p>
      <p>In this study, the PubMed database created by the National Center for Biotechnology
Information (NCBI) in the United States was identified as the first base for collecting</p>
      <p>
        A Scientometric Analysis of Publications Related to Predictive Medicine 3
information. The search for standard bibliometric processing was performed based on
the terms "predictive" and "personalize (s) e" "preventive" "medicine" with the logical
operator "AND". We measured the number of publications in the field of PPM
depending on the affiliation of the first author. The output of publications at the country level
was estimated using PubMed tools [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ].
      </p>
      <p>
        To isolate key terms using the Word2Wec method, about 15,000 articles (titles and
annotations) were selected from the PubMed database, containing the terms
"predictive" and "personalize (s) e" in their titles. To extract keywords from the headlines, we
used baseline Medline / PubMed database for all years. The annual baseline is released
in December of each year [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]. Key terms were extracted from the titles of articles. For
statistical processing, a set of programs in the Java language was developed.
      </p>
      <p>Algorithm for finding trends when processing a corpus of natural language texts:
1. Initially, an expert creates a dictionary of key terms, consisting of keywords and
phrases in a normalized form. This is a dictionary De or main dictionary.
2. Next, a temporary dictionary-template of new words and phrases is created Dn.
3. Then an array of statistics is created to analyze the neighborhood of keywords.</p>
      <p>S = { Dei{ Dnk, Rk}}
(1)
Dei is an element of key terms vocabulary,</p>
      <p>Dnk – new word (not a stop word or a word from the main dictionary De),
Rk – a frequency of occurrence of a new word Dnk in the vicinity of the key term Dei.
At first, its value is zero.</p>
      <p>4. If a key term is found, then an analysis of its neighborhood is performed. The
neighborhood of the word is determined by the size of the sliding window. The words
of the analyzed text are normalized. If a word (not a stop word) is found in the vicinity
of a keyword it is not included in the main dictionary, it is first checked whether the
array S contains a pair of values { Dei, { Dnk }}. If there is no such pair of values, then
it is added to the dictionary. Then the frequency value (Rk) for a given pair of values,
increases by one. Each new element Dnk can become either a key term or part of it.</p>
      <p>5. After processing the text corpus and filling in the S array, the most rated
candidates for key terms are selected. Minimum rating threshold (Rmin) the new term is
determined by the expert. нового термина определяет эксперт. All elements of the array
S with a rating less than Rmin are not considered.</p>
      <p>6. Next, candidates for a new key terms are generated. If there are several new words
for the same keyword with the same Rk, then there is a possibility that the new term
could be more than one word. The system generates a possible new multi word term.
At the same time, candidates for key terms are generated with already approved
elements (from the main dictionary De) and possible key terms found in the vicinity of the
keyword, but already without it.</p>
      <p>7. As a result of the analysis of the values of the array S and approval by the expert
new key terms are added into the dictionary. New key terms with the maximum rank
determine development trends in this field of medicine.</p>
      <p>The “Dimensions” scientific information database was used as the second database.
The search was carried out by keywords in the annotations "predictive preventive
personalized medicine" for 2008-2020.</p>
      <p>VOSviewer version 1.6.15 was used to construct and visualize scientific maps based
on the data obtained. The program allows you to carry out scientific mapping based on
scientometric analysis (frequency of co-occurrence of keywords).</p>
      <p>The construction of a graph of interconnected pairs of objects is based on
multidimension scaling (MDS), which is a means for visualizing pairwise connected graph
vertices displayed on a plane. The method is used when two or more dimensions need
to be examined for data analysis. MDS is used to construct bibliometric maps, which
can include either co-citation of sources or co-occurrence of terms (objects, authors,
documents, journals, keywords). The number of joint occurrences of elements within a
given neighbourhood i and j is denoted cij. ci is a frequency of element i.
If we have a set of n vertices, we want to build a graph of connected elements (for
example, vertices that indicate the joint appearance of elements, or joint citation of
documents), then sij will mean the strength of the associative connection between
vertices i and j (Van Eck &amp; Waltman, 2009) and represents.</p>
      <p>= ∑ ≠  
  =
2</p>
      <p>= 1 ∑</p>
      <p>2
They call this measure as proximity index. Otherwise, we can say that cij means the
total number of connections of the vertex i and m means the total number of connections
in the network.</p>
      <p>When constructing a graph, two types of similarity can be used: direct and indirect.
When using the indirect type of similarity to build a graph, direct comparison of
elements is used, when using the indirect type of similarity, vectors of elements are
compared. MDS allows to reduce the dimension of the graph. The method works in a similar
way as t-distributed Stochastic Neighbor Embedding (t-SNE).</p>
      <p>
        There are several options for implementing MDS, including Metric
multidimensional scaling (mMDS) [
        <xref ref-type="bibr" rid="ref28">28</xref>
        ], Non-metric multidimensional scaling (nMDS) [
        <xref ref-type="bibr" rid="ref29">29</xref>
        ],
Generalized multidimensional scaling (GMD) [
        <xref ref-type="bibr" rid="ref30">30</xref>
        ] and so on. The main advantage of this
classical approach is the integration into the VOSviewer software product. MDS
method implemented in several libraries (for Java, Python, Statistica).
3
3.1
      </p>
    </sec>
    <sec id="sec-3">
      <title>Results</title>
      <sec id="sec-3-1">
        <title>Statistical analysis</title>
        <p>We analyzed trends in the field of predictive and personalized medicine (PPM) for the
period from 1940 to 2020.</p>
        <p>To search in PubMed the keywords “predictive medicine”, “personalized medicine”,
“preventive medicine” were used. The collocation “preventive medicine” at first was
(2)
(3)
(4)</p>
        <p>A Scientometric Analysis of Publications Related to Predictive Medicine 5
appeared in 1857. The collocation “predictive medicine” at first was appeared in 1918.
The collocation “personalized medicine” at first was appeared in 1952.</p>
        <p>
          From the graph (see Fig. 1) we can see that predictive medicine, personalized
medicine and preventive medicine experienced ups and downs in popularity. There is the
increase of publications in these areas in last decade, apparently due to the activity of
the European Association of Predictive, Preventive and Personalized Medicine
(EPMA), which was established in 2008 [
          <xref ref-type="bibr" rid="ref17">17</xref>
          ].
        </p>
        <p>s 1200
n
ito 1000
a
c
i
lb 800
u
fpo 600
r
eb 400
m
iN 200
0
0 3 6 9 2 5 8 1 4 7 0 3 6 9 2 5 8 1 4 7 0 3 6 9
95 95 95 95 96 96 96 97 97 97 98 98 98 98 99 99 99 00 00 00 01 01 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 Y2ea20r 20
preventive medicine predictive medicine personalized medicine
Fig. 1. The number of PubMed publications in the fields of preventive, predictive and
personalized medicine from 1950 to 2020.
3.2</p>
      </sec>
      <sec id="sec-3-2">
        <title>Key term mapping with VOSviewer</title>
        <p>VOSviewer carries out the extraction of keywords in a matrix, the structure of which
forms in the form of a network by linking terms according to the calculated link
strength. Bond strength was defined as the total number of occurrences of a term in
pairs with other terms.</p>
        <p>The first step involved mapping terms extracted from an array of articles indexed in
Dimensions for the period 2008–2020. For further visualization, a list of 85 keywords
was formed, which included terms from article titles. In VOSviewer, the option "create
map from text corpus" was selected. The counting method was set to binary counting.
This means that each concept is counted only once per annotation, regardless of how
many times it actually appears in a given annotation. VOSviewer's algorithm
automatically excludes 40% of least significant terms. In total, 51 terms were used for mapping
out of 4979 terms (see Fig. 2).</p>
        <p>Stage 1 involved rendering a text corpus of 15,000 titles identified during the
preparation phase. These names were extracted from the PubMed database (see section 2).</p>
        <p>The purpose of this step is to create an overview of concepts within a given body of
literature.</p>
        <p>Stage 2 consisted of two parts. Part 1 was aimed at extracting key terms using special
software tools.
6 Aida Khakimova et al.</p>
        <p>The method is based on the sequential linear division of the sentence into various
phrases. The statistics of various options for parsing phrases in the corpus were
calculated using a floating "window" - a numeric hyperparameter that shows how many
words to the left and right of the central one is its environment. In this way, more than
9200 derivatives and phrases were extracted.</p>
        <p>Part 2 focused on mapping key terms with overlapping windows (see Fig. 3).
Part 3 focused on answering a research question about how software tools can help
identify interesting ideas.</p>
        <p>An additional visualization was performed in which the threshold for the minimum
number of occurrences was set to 5 and the number of concepts included in the
visualization was set to 123 (see Fig. 4). This step was done with a binary count. Non-specific
terms were excluded (Asia, October, USA, New York, 21st century, etc.).
The list of terms / concepts defined in VOSviewer can be used as an initial contextual
mapping template. Contextual analysis can also help to identify potential relationships
between two or more concepts.</p>
        <p>For example, let us select the first cluster. It includes the terms assessment, clinical,
discovery, evaluation, innovative approach, personalized medicine approach,
predictive, preventive, rare disease, translation, translational.</p>
        <p>We found these terms in our list of 9200 terms and phrases. The term «predictive»
is central to the cluster in terms of the number of links. We built a context map of terms
for this set of terms, considering the results of VOSviewer and our statistical processing
of the corpus of articles.
4</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>Results</title>
      <p>As discussed in the previous section, 123 concepts were included in the visualization
in Phase 1 of the analysis. The VOSviewer map gave 14 conceptual clusters. After
exploring the concepts in each cluster, we selected 1 cluster to build the contextual
conceptual mapping.</p>
      <sec id="sec-4-1">
        <title>Label</title>
      </sec>
      <sec id="sec-4-2">
        <title>Cluster</title>
      </sec>
      <sec id="sec-4-3">
        <title>Topic</title>
        <p>Cluster 1 focuses on topics related to innovative approaches in predictive
preventive personalized medicine (shown in table 1). Cluster 2 is devoted to topics
related to per- sonalized management and application of biomarkers. Cluster 3 can be
referred to as
Preventive Medicine and Cardiology Perspectives. Cluster 4 focuses on personalized
diagnosis and medicine in oncology. Cluster 5 brings together topics related to the
paradigm and strategy of personalized medicine. Cluster 6 deals with Omics in the
treatment of rheumatoid arthritis. Cluster 7 focuses on topics related to personalized
preventive medicine in cancer treatment. Cluster 8 includes issues of genetics and
genomics in the PPM. Clusters 9 to 12 cover topics related to PMP in the management of
chronic non-cancer diseases (multiple sclerosis, diabetes, Parkinson's disease COPD).
Clusters 13 and 14 focus on innovative PPM in oncology, including Non-small-cell
lung carcinoma (NSCLC). Data from 8 cluster have not included in the table 1.</p>
        <p>The above list of research areas may already be of interest from a scientific landscape
perspective. At this point, you can go back to the beginning of the process and develop
a search strategy based on one cluster.</p>
        <p>The starting point for contextual conceptual mapping (see Fig. 5) was a list of
concepts of the first cluster.</p>
        <p>The list was supplemented with additional concepts that proved to be important. The
question of whether a term or concept was considered potentially significant was
determined in terms of the frequency of its occurrence in titles and annotations (as
determined by frequency analysis).</p>
        <p>Figure 5 shows the results of the extended contextual analysis. This map can serve
as a starting point for a deeper review of the literature.
5</p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>Conclusion</title>
      <p>We have presented a method for analyzing publications and identifying trends in
predictive medicine. The proposed method allows to search for new terms in preventive
medicine based on the neighborhood approach.</p>
      <p>Experts took part in identifying new trends. The use of software to automatically
highlight trends significantly reduces the time it takes to generate new terms. To process
the corpus of selected articles, one expert would need several months, while using the
proposed software and considering the work of experts, this work was completed in 2
weeks.</p>
      <p>The modified word2vec method was used by the authors to highlight key terms and
to build forecasts. The diagrams are built using the MDS method. The search and
analysis of information was carried out in several electronic libraries using their analytical
mechanisms and visualization tools. A limitation of the current study is that the analysis
relied on information provided by the authors in the titles. Future research may include
full text analysis. Two contextual conceptual term maps were created in the prototype.
These maps were created to provide an overview of research topics and to identify
promising directions.</p>
    </sec>
    <sec id="sec-6">
      <title>Acknowledgments</title>
      <p>This work is supported by RFBR, grants 18-07-01111, 18-07-00909, 18-07-00225,
1907-00455 and 20-04-60185.</p>
      <p>References:</p>
      <p>A Scientometric Analysis of Publications Related to Predictive Medicine 11</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <surname>Hicks</surname>
            <given-names>D</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wouters</surname>
            <given-names>P</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Waltman</surname>
            <given-names>L</given-names>
          </string-name>
          ,
          <string-name>
            <surname>de Rijcke</surname>
            <given-names>S</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rafols</surname>
            <given-names>I.</given-names>
          </string-name>
          <article-title>The Leiden Manifesto for research metrics</article-title>
          .
          <source>Nature</source>
          <year>2015</year>
          ;
          <volume>520</volume>
          :
          <fpage>429</fpage>
          -
          <lpage>31</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <given-names>PubMed</given-names>
            <surname>Help</surname>
          </string-name>
          [Internet].
          <article-title>Bethesda (MD): National Center for Biotechnology Information</article-title>
          (US);
          <fpage>2005</fpage>
          -.
          <source>PubMed Help. [Updated 2019 Jul</source>
          <volume>25</volume>
          ]. Available from: https://www.ncbi.nlm.nih.gov/books/NBK3827/
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <surname>Young</surname>
            <given-names>H</given-names>
          </string-name>
          :
          <source>Glossary of Library and Information Science</source>
          .
          <year>1983</year>
          , Chicago: American Library Association Google Scholar.
          <source>Indian J Ophthalmol</source>
          .
          <year>2015</year>
          Jan;
          <volume>63</volume>
          (
          <issue>1</issue>
          ):
          <fpage>54</fpage>
          -
          <lpage>8</lpage>
          . doi:
          <volume>10</volume>
          .4103/
          <fpage>0301</fpage>
          -
          <lpage>4738</lpage>
          .
          <fpage>151471</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <surname>Mansour</surname>
            <given-names>AM</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mollayess</surname>
            <given-names>GE</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Habib</surname>
            <given-names>R</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Arabi</surname>
            <given-names>A</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Medawar</surname>
            <given-names>WA</given-names>
          </string-name>
          .
          <article-title>Bibliometric trends in ophthalmology 1997-2009</article-title>
          . Semin Arthritis Rheum.
          <year>2017</year>
          Jun;
          <volume>46</volume>
          (
          <issue>6</issue>
          ):
          <fpage>828</fpage>
          -
          <lpage>833</lpage>
          . doi:
          <volume>10</volume>
          .1016/j.semarthrit.
          <year>2016</year>
          .
          <volume>12</volume>
          .002.
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <surname>Redondo</surname>
            <given-names>M</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Leon</surname>
            <given-names>L</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Povedano</surname>
            <given-names>FJ</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Abasolo</surname>
            <given-names>L</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Perez-Nieto</surname>
            <given-names>MA</given-names>
          </string-name>
          ,
          <string-name>
            <surname>López-Muñoz F</surname>
          </string-name>
          .
          <article-title>A bibliometric study of the scientific publications on patient-reported outcomes in rheumatology</article-title>
          .
          <source>Clin Otolaryngol</source>
          .
          <year>2017</year>
          Dec;
          <volume>42</volume>
          (
          <issue>6</issue>
          ):
          <fpage>1338</fpage>
          -
          <lpage>1342</lpage>
          . doi:
          <volume>10</volume>
          .1111/coa.12910.
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <surname>Saunders</surname>
            <given-names>TFC</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rymer</surname>
            <given-names>BC</given-names>
          </string-name>
          ,
          <string-name>
            <surname>McNamara</surname>
            <given-names>KJ</given-names>
          </string-name>
          .
          <article-title>A global bibliometric analysis of otolaryngology: Head and neck surgery literature</article-title>
          .
          <source>G Ital Nefrol</source>
          .
          <year>2016</year>
          Nov-Dec;
          <volume>33</volume>
          (
          <issue>6</issue>
          ).
          <source>pii: gin/33.6</source>
          .10.
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <surname>Torrisi</surname>
            <given-names>AM</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Granata</surname>
            <given-names>A</given-names>
          </string-name>
          .
          <article-title>Bibliometric indicators of nephrology journals: strengths and weaknesses</article-title>
          .
          <source>[Article in Italian] Geriatr Gerontol Int</source>
          .
          <year>2017</year>
          Feb;
          <volume>17</volume>
          (
          <issue>2</issue>
          ):
          <fpage>357</fpage>
          -
          <lpage>360</lpage>
          . doi:
          <volume>10</volume>
          .1111/ggi.12880.
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <surname>Ang</surname>
            <given-names>HM</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kwan</surname>
            <given-names>YH</given-names>
          </string-name>
          .
          <article-title>Bibliometric analysis of journals in the field of geriatrics and gerontology</article-title>
          .
          <source>J Neuropsychiatry Clin Neurosci</source>
          .
          <year>2015</year>
          Fall;
          <volume>27</volume>
          (
          <issue>4</issue>
          ):
          <fpage>354</fpage>
          -
          <lpage>61</lpage>
          . doi:
          <volume>10</volume>
          .1176/appi.neuropsych.
          <volume>15010024</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <surname>Zhu</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Guan</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          (
          <year>2013</year>
          ).
          <article-title>A bibliometric study of service innovation research: based on complex network analysis</article-title>
          .
          <source>Scientometrics</source>
          ,
          <volume>94</volume>
          (
          <issue>3</issue>
          ),
          <fpage>1195</fpage>
          -
          <lpage>1216</lpage>
          . Retrieved from https://EconPapers.repec.org/RePEc:spr:scient:v:94:y:
          <source>2013:i:3:d:10</source>
          .1007_
          <fpage>s11192</fpage>
          -
          <fpage>012</fpage>
          -0888- 1.
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <surname>Sinkovics</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          (
          <year>2016</year>
          ).
          <article-title>Enhancing the foundations for theorising through bibliometric mapping</article-title>
          .
          <source>International Marketing Review</source>
          ,
          <volume>33</volume>
          (
          <issue>3</issue>
          ),
          <fpage>327</fpage>
          -
          <lpage>350</lpage>
          . https://doi.org/10.1108/IMR-10- 2014-0341
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11. van Eck,
          <string-name>
            <given-names>N.J.</given-names>
            and
            <surname>Waltman</surname>
          </string-name>
          ,
          <string-name>
            <surname>L.</surname>
          </string-name>
          (
          <year>2007</year>
          ), “
          <article-title>Bibliometric mapping of the computational intelligence field”</article-title>
          ,
          <source>International Journal of Uncertainty, Fuzziness and Knowledge-based Systems</source>
          , Vol.
          <volume>15</volume>
          No.
          <issue>5</issue>
          , pp.
          <fpage>625</fpage>
          -
          <lpage>645</lpage>
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <surname>Huffman</surname>
            ,
            <given-names>M. D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Baldridge</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bloomfield</surname>
            ,
            <given-names>G. S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Colantonio</surname>
            ,
            <given-names>L. D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Prabhakaran</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ajay</surname>
            ,
            <given-names>V. S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Prabhakaran</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          (
          <year>2013</year>
          ). Global Cardiovascular Research Output, Citations, and
          <string-name>
            <surname>Collaborations</surname>
            :
            <given-names>A</given-names>
          </string-name>
          <string-name>
            <surname>Time-Trend</surname>
          </string-name>
          ,
          <article-title>Bibliometric Analysis (</article-title>
          <year>1999</year>
          -
          <fpage>2008</fpage>
          ).
          <source>Plos One</source>
          ,
          <volume>8</volume>
          (
          <issue>12</issue>
          ), 7. doi:
          <volume>10</volume>
          .1371/journal.pone.0083440
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13.
          <string-name>
            <surname>Menendez-Manjon</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Moldenhauer</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wagener</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Barcikowski</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          (
          <year>2011</year>
          ).
          <article-title>Nanoenergy research trends: bibliometrical analysis of nanotechnology research in the energy sector</article-title>
          .
          <source>Journal of Nanoparticle Research</source>
          ,
          <volume>13</volume>
          (
          <issue>9</issue>
          ),
          <fpage>3911</fpage>
          -
          <lpage>3922</lpage>
          . doi:
          <volume>10</volume>
          .1007/s11051-011-0344- 9
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <string-name>
            <surname>Sooryamoorthy</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          (
          <year>2010</year>
          ).
          <article-title>Medical research in South Africa: a scientometric analysis of trends, patterns, produc-tivity and partnership</article-title>
          .
          <source>Scientometrics</source>
          ,
          <volume>84</volume>
          (
          <issue>3</issue>
          ),
          <fpage>863</fpage>
          -
          <lpage>885</lpage>
          . doi:
          <volume>10</volume>
          .1007/s11192-010-0169-9
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          15.
          <string-name>
            <surname>Gelman</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Unwin</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          (
          <year>2013</year>
          ).
          <article-title>Infovis and Statistical Graphics: Different Goals, Different Looks</article-title>
          .
          <source>Journal of Computational and Graphical Statistics</source>
          ,
          <volume>22</volume>
          (
          <issue>1</issue>
          ),
          <fpage>2</fpage>
          -
          <lpage>28</lpage>
          . https://doi.org/10.1080/10618600.
          <year>2012</year>
          .761137
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          16.
          <string-name>
            <surname>Yang</surname>
            <given-names>H</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lee</surname>
            <given-names>HJ</given-names>
          </string-name>
          .
          <article-title>Research Trend Visualization by MeSH Terms from PubMed</article-title>
          .
          <source>Int J Environ Res Public Health</source>
          <year>2018</year>
          ;
          <volume>15</volume>
          :
          <fpage>1113</fpage>
          -
          <lpage>27</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          17.
          <string-name>
            <surname>Jazayeri</surname>
            <given-names>SB</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Alavi</surname>
            <given-names>A</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rahimi-Movaghar</surname>
            <given-names>V</given-names>
          </string-name>
          .
          <article-title>Situation of medical sciences in 50 top countries from 1996 to 2010 - based on quality and quantity of publications</article-title>
          .
          <source>Acta Med Iran</source>
          <year>2012</year>
          ;
          <volume>50</volume>
          :
          <fpage>273</fpage>
          -
          <lpage>8</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          18.
          <string-name>
            <surname>Garfield</surname>
            <given-names>E.</given-names>
          </string-name>
          <article-title>Keywords plus - ISI's breakthrough retrieval method. 1. Expanding Your Searching Power on Current Contents on Diskette</article-title>
          .
          <source>Current Contents</source>
          <year>1990</year>
          ;
          <volume>32</volume>
          :
          <fpage>5</fpage>
          -
          <lpage>9</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          19.
          <string-name>
            <surname>Xu</surname>
            <given-names>Q</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Boggio</surname>
            <given-names>A</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ballabeni</surname>
            <given-names>A</given-names>
          </string-name>
          . Countries' Biomedical Publications and
          <string-name>
            <given-names>Attraction</given-names>
            <surname>Scores</surname>
          </string-name>
          .
          <article-title>A PubMed-based assessment [version 2; peer review: 2 approved]</article-title>
          .
          <source>F1000Research</source>
          <year>2015</year>
          ;
          <volume>3</volume>
          :
          <fpage>292</fpage>
          -
          <lpage>8</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          20.
          <string-name>
            <surname>Klimenko</surname>
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Khakimova</surname>
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Charnine</surname>
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zolotarev</surname>
            <given-names>O.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Merkureva</surname>
            <given-names>N.</given-names>
          </string-name>
          <article-title>Semantic approach to visualization of research front of scientific papers using web-based 3D graphic</article-title>
          .
          <source>В сборнике Proceedings of the 2018 International Conference Web 3D. The 23rd International Proceedings - Web3D 2018: 23rd International ACM Conference on 3D Web Technology 23, 3D Everywhere</source>
          .
          <year>2018</year>
          . С.
          <year>a20</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          21.
          <string-name>
            <surname>Golubnitschaja</surname>
            , Olga &amp; Kinkorova, Judita &amp; Costigliola,
            <given-names>Vincenzo.</given-names>
          </string-name>
          (
          <year>2014</year>
          ).
          <article-title>Predictive, Preventive and Personalised Medicine as the hardcore of 'Horizon 2020': EPMA position paper</article-title>
          .
          <source>The EPMA journal. 5. 6. 10.1186/1878-5085-5-6.</source>
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          22.
          <article-title>Geographic directory «About countries»</article-title>
          . http://ostranah.ru/_lists/population.ph.
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          23.
          <string-name>
            <surname>Wikipedia</surname>
          </string-name>
          .
          <article-title>Lists of countries by GDP</article-title>
          . https://en.wikipedia.org/wiki/List_
          <article-title>of_countries_by_GDP_(nominal).</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          24.
          <string-name>
            <surname>Chen</surname>
            <given-names>C</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wang</surname>
            <given-names>Z</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Li</surname>
            <given-names>W</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sun</surname>
            <given-names>X</given-names>
          </string-name>
          .
          <article-title>Modeling Scientific Influence for Research Trending Topic Prediction</article-title>
          .
          <source>Thirty-Second AAAI Conference on Artificial Intelligence</source>
          ,
          <year>2018</year>
          . https://aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/16721.
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          25.
          <string-name>
            <given-names>A.</given-names>
            <surname>Kh. Khakimova</surname>
          </string-name>
          ,
          <string-name>
            <given-names>O.V.</given-names>
            <surname>Zolotarev</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.A.</given-names>
            <surname>Berberova</surname>
          </string-name>
          .
          <article-title>Visualization of bibliometric networks of scientific publications on the study of the human factor in the operation of nuclear power plants based on the bibliographic database Dimensions</article-title>
          .
          <source>Scientific Visualization</source>
          ,
          <year>2020</year>
          , volume
          <volume>12</volume>
          , number 2, pages
          <fpage>127</fpage>
          -
          <lpage>138</lpage>
          , DOI: 10.26583/sv.12.2.10, E-ISSN:
          <fpage>2079</fpage>
          -
          <lpage>3537</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          26.
          <string-name>
            <surname>Zolotarev</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          ;
          <string-name>
            <surname>Solomentsev</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          ;
          <string-name>
            <surname>Khakimova</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ;
          <string-name>
            <surname>Charnine</surname>
            <given-names>M.</given-names>
          </string-name>
          <article-title>Identification of semantic patterns in full-text documents using neural network methods</article-title>
          .
          <source>In Proceedings of the 29th International Conference on Computer Graphics and Vision</source>
          . Graphicon-
          <year>2019</year>
          .
          <year>2019</year>
          . Available online: http://ceur-ws.
          <source>org/</source>
          Vol-
          <volume>2485</volume>
          /paper64.pdf.
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          27.
          <string-name>
            <surname>Klimenko</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ; Charnine,
          <string-name>
            <given-names>M.</given-names>
            ;
            <surname>Zolotarev</surname>
          </string-name>
          ,
          <string-name>
            <given-names>O.</given-names>
            ;
            <surname>Merkureva</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            ;
            <surname>Khakimova</surname>
          </string-name>
          ,
          <string-name>
            <surname>A.</surname>
          </string-name>
          <article-title>Semantic Approach to Visualization of Research Front of Scientific Papers Using Web-Based 3d Graphic</article-title>
          .
          <source>In Proceedings of the 23rd International ACM Conference on 3D Web Technology</source>
          .
          <year>2018</year>
          , 1-
          <fpage>6</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          28.
          <string-name>
            <surname>Kruskal</surname>
            ,
            <given-names>J. B.</given-names>
          </string-name>
          (
          <year>1964</year>
          ),
          <article-title>"Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis"</article-title>
          ,
          <source>Psychometrika</source>
          ,
          <volume>29</volume>
          (
          <issue>1</issue>
          ):
          <fpage>1</fpage>
          -
          <lpage>27</lpage>
          , doi:10.1007/BF02289565.
        </mixed-citation>
      </ref>
      <ref id="ref29">
        <mixed-citation>
          29.
          <string-name>
            <surname>Bagdonavicius</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kruopis</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nikulin</surname>
            ,
            <given-names>M.S.</given-names>
          </string-name>
          (
          <year>2011</year>
          ).
          <article-title>"Non-parametric tests for complete data"</article-title>
          ,
          <source>ISTE &amp; WILEY: London &amp; Hoboken. ISBN 978-1-84821-269-5.</source>
        </mixed-citation>
      </ref>
      <ref id="ref30">
        <mixed-citation>
          30.
          <string-name>
            <surname>Bronstein</surname>
            <given-names>AM</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bronstein</surname>
            <given-names>MM</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kimmel</surname>
            <given-names>R</given-names>
          </string-name>
          (
          <year>January 2006</year>
          ).
          <article-title>"Generalized multidimensional scaling: a framework for isometry-invariant partial surface matching"</article-title>
          .
          <source>Proc. Natl. Acad. Sci</source>
          . U.S.A.
          <volume>103</volume>
          (
          <issue>5</issue>
          ):
          <fpage>1168</fpage>
          -
          <lpage>72</lpage>
          . Bibcode:
          <year>2006PNAS</year>
          .
          <fpage>103</fpage>
          .1168B. doi:
          <volume>10</volume>
          .1073/pnas.0508601103.
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>