Scientific research allows for increasing the knowledge of reality and the phenomena that manifest and evolve in it. Today, research is considered a professional competence, which is mainly developed in university institutions. Day by day, the so-called information society is flooded with research works of all types and qualities. At the same time, there is an increase in the number of annual publications that analyze data and measure intellectual works and their contribution to the scientific heritage of each discipline. A series of indicators are generated based on various parameters such as authors, type of document, language, and keywords, among others. The visualization of these numerous indicators, which vary over time, can be achieved efectively through the use of bibliometric software, which is also renewed in its functionality and features. The following is a study, assisted by systematic and bibliometric review techniques, with the aim of identifying the most widely used bibliometric software and those emerging that serve as a guide and good practice in new bibliometric research to obtain notable and relevant knowledge. The applications "VOSviewer", "CiteSpace" and "Bibliometrix" stand out and tools such as "Scopus API R code", "Covidence", "HistCite", "BICOMB", "EndNote X9.6 Statistical", "Ucinet 6.0", "Tools for Innovation Monitoring", "Profiles Research Networking Software", among others, emerge.
The number of scientific works published today is very large; it is impossible to read all these
contributions, even if they are classified as a science or activity. From this point of view, it
is dificult for a single person, without methods, techniques, and tools, to judge whether the
scientific activity of a country, an institution, or an expert is relevant. In the face of this
dificulty, bibliometrics emerges. A bibliometric approach provides, through a detailed analysis
of published articles, a qualitative and quantitative result of the contributions made to the
scientific community [
In bibliometrics, a wide variety of measures are used to evaluate the content of articles and
the performance of their authors. Researchers, through bibliometrics, try to measure, among
others, the evolution of a scientific domain, the impact of scholarly publications, authorship
patterns, and the process of scientific knowledge production [
In addition to the above, bibliometric analysis is a meta-analytical tool used to measure or
access the characteristics of any variable, whether there is an efect, and whether it is positive
or negative [
It is important that research metrics are defined in an eficient, transparent, and solid way;
demonstrating the true impact and influence of the research. Diferent methods have been
developed to seek answers to research questions, using collaborative networks, thematic research
groups, patterns of historical evolution, and trends in the topics addressed [
Research designs for bibliometrics are structured primarily with the identification of suitable
bibliographic databases, the development of search criteria, and the selection of informatics
tools for analysis [
Bibliometrics can assess publications and development trends in a scientific field, and reveal
tentative key research directions, by analyzing databases and publication characteristics [
In the academic and scientific environment, it is a requirement of science managers and those
who define policies to increase bibliometric studies to support the decisions of the developed
research [
In view of the above and considering the importance of computer tools, which are used in diferent bibliometric studies, the following is a systematic review and a meta-analysis assisted with bibliometric methods, to collect and summarize evidence on the most used contemporary bibliometric software, in the evaluation of scientific production, from diferent perspectives that guide researchers in the generation of remarkable and relevant scientific knowledge.
The aim of this systematic review was to identify and evaluate the main software used in bibliometric studies and other emerging tools. To this end, two groups of scientific articles were reviewed; the first group included research that used globally recognized bibliometric software, and the second group, also bibliometric studies, did not use any of the bibliometric software identified as the most representative, but worked with other software.
For the literature review, the PRISMA (Preferred Reporting Items for Systematic reviews
and Meta-Analyses) statement was used, published in 2009 and updated in 2020 [
Some guidelines were defined for the extraction of the documents to be examined. The main inclusion and exclusion criterion was given by a list of the most used software in bibliometric studies. For this, two contemporary research papers were previously searched and selected, which list the main bibliometric software.
From the research works conducted [
Of the software listed in the research [
Other inclusion criteria determined were that bibliometric studies published between 2018 and 2022 had to be evaluated, and the type of document chosen was an article; in addition, they had to have been written in English, and be freely available for reading, without the need for payment.
The Scopus database was used, which contains abstracts and citations of multidisciplinary
scientific literature from peer-reviewed journals and web sources. This database ensures that
only the highest quality data are indexed through rigorous content selection [
Considering the eligibility criteria, two search strings were structured for the Scopus database, which are presented below, resulting in 1,246 and 210 documents, respectively.
String 1: TITLE-ABS- KEY ( ( ( "Bibexcel" OR "Bibliometrix" OR "BiblioTools" OR "CATAR" OR "CiteSpace" OR "CitNetExporer" OR "CRExplorer" OR "Headstart" OR "Publish or Perish" OR "RobotReviewer" OR "SciMAT" OR "Scopus API R code" OR "VOSviewer" ) AND ( ( "bibliometric" OR "scientometric" ) ( "Bibexcel" OR "Bibliometrix" OR "BiblioTools" OR "CATAR" OR "CiteSpace" OR "CitNetExporer" OR "CRExplorer" OR "Headstart" OR "Publish or Perish" OR "RobotReviewer" OR "SciMAT" OR "Scopus API R code" OR "VOSviewer" ) AND ( "bibliometric" OR "scientometric" ) ) AND ( ( LIMIT- TO ( OA , "all" ) ) AND ( LIMIT-TO ( PUBYEAR , 2023 ) OR LIMIT-TO ( PUBYEAR , 2022 ) OR LIMIT-TO ( PUBYEAR , 2021 ) OR LIMIT-TO ( PUBYEAR , 2020 ) OR LIMIT-TO ( PUBYEAR , 2019 ) OR LIMIT-TO ( PUBYEAR , 2018 ) ) AND ( LIMIT-TO ( DOCTYPE , "ar" ) ) AND ( LIMIT-TO ( LANGUAGE , "English" ) ) ).
String 2: ( TITLE-ABS-KEY ( ( ( "bibliometric" OR "scientometric" ) ) AND TITLE-ABS-KEY ( ( ( "software" ) ) AND NOT TITLE-ABS- KEY ( ( ( "Bibexcel" OR "Bibliometrix" OR "BiblioTools" OR "CATAR" OR "CiteSpace" OR "CitNetExporer" OR "CRExplorer" OR "Headstart" OR "Publish or Perish" OR "RobotReviewer" OR "SciMAT" OR "Scopus API R code" OR "VOSviewer" OR "VOSviewer" ) ) ) AND ( LIMIT-TO ( OA , "all" ) ) AND ( LIMIT-TO ( PUBYEAR , 2023 ) OR LIMIT-TO ( PUBYEAR , 2022 ) OR LIMIT-TO ( PUBYEAR , 2021 ) OR LIMIT-TO ( PUBYEAR , 2020 ) OR LIMIT-TO ( PUBYEAR , 2019 ) OR LIMIT-TO ( PUBYEAR , 2018 ) ) AND ( LIMIT-TO ( DOCTYPE , "ar" ) ) AND ( LIMIT-TO ( LANGUAGE , "English" ) ) ).
Once each search string was executed in Scopus, all results were selected. Immediately, for the extraction process, from the same database, the option "CSV export" and "Export document settings" were used. All the information fields provided by the tool were selected. The records were exported to "RIS", "CSV" and "BibTex" formats for further processing.
To make a detailed recognition of the data extracted from Scopus, bibliometric techniques were used, supported by functions of the Bibliometrix library, which are executed on the R Studio programming tool.
In addition, using the R Studio platform and the R language, algorithms were constructed and applied to the two groups of records (1246 and 210 documents) to process and evaluate the data and summarize them graphically.
A meta-analysis was performed with the review; this study made it possible to achieve the statistical synthesis of the data extracted from both groups of scientific document data from Scopus.
In order to know where bibliometric studies tend to go within the diferent fields of application,
the fields of education and training of the reference system of the International Standard
Classification of Education (ISCED) [
In view of the criteria described in [
Subsequently, with the 210 records of scientific articles, corresponding to the second group of data extracted, a new group of data was constructed, through a bibliometric process, with the fifty most cited documents. All of them were reviewed by two of the authors of this study. The review was carried out through a content analysis, emphasizing the summary and methodological aspects that were applied to identify and find any emerging software used in bibliometric studies.
The results obtained from the analysis of the data extracted after applying the two structured search strings with the inclusion and exclusion criteria defined by the authors are presented below. The first group of data was used to identify the software currently most used in bibliometric studies; and the second group was used to try to identify other bibliometric software.
A total of 1246 documents were extracted from Scopus between 2018 and 2022. All these studies considered the following software: "Bibexcel", "Bibliometrix", "BiblioTools", "CATAR", "CiteSpace", "CitNetExporer", "CRExplorer", "Headstart", "Publish or Perish", "RobotReviewer", "SciMAT", "Scopus API R code", and "VOSviewer". Various bibliometric techniques were applied to the bibliographic information extracted from the database to determine the visibility of scientific production using bibliometric software and the intensity of its use.
China stands out in scientific production considering the use of software and bibliometric techniques. With a much lower number than this country, the following countries stand out: Spain, Malaysia, India, Brazil, Italy, Turkey, Iran, Colombia, and United Kingdom. Of this leading group, only United Kingdom has a very good international collaboration, with an index of 77%. Turkey’s production has been developed only with internal collaboration. In the rest of the countries, although there is both local and international collaboration, when evaluating the contribution in scientific productions, all of them are consolidated mainly with collaboration within the country itself, see Table 1.
CHINA SPAIN MALAYSIA INDIA BRAZIL ITALY TURKEY IRAN COLOMBIA UNITED KINGDOM
Articles
Despite the fact that China has a total number of citations much higher than the second most cited country, and by logic a wide gap with the rest of the countries, China has a lower average number of citations of articles, lower than the rest of the leading countries in the group of scientific production evaluated, see Table 2.
On evaluating the bibliographic records, it was also found that the authors of the scientific production studied have 1793 diferent afiliations. The following universities stand out: "Central South University", "Sichuan University", "An-Najah National University", "Beijing University of Chinese Medicine" and "Southeast University". See Table 3.
SCP
MCP 424 53 22 19 14 14 21 19 15 4 86 17 12 13 10 10 0 1 3 14
0,01370614 0,010416667 0,010142544 0,009046053 0,008497807 0,008223684 0,007675439 0,00685307 0,00685307 0,006030702 759 414 356 303 268 221 174 141 132 131 CENTRAL SOUTH UNIVERSITY SICHUAN UNIVERSITY AN-NAJAH NATIONAL UNIVERSITY BEIJING UNIVERSITY OF CHINESE MEDICINE SOUTHEAST UNIVERSITY ESPOL POLYTECHNIC UNIVERSITY KING ABDULAZIZ UNIVERSITY AN-NAJAH NATIONAL UNIVERSITY HOSPITAL LANZHOU UNIVERSITY
UNIVERSITY OF GRANADA
Of the bibliometric studies evaluated, the following keywords stand out, reported directly by the authors: "BIBLIOMETRIC ANALYSIS", "VOSVIEWER", "CITESPACE", "BIBLIOMETRICS" and "BIBLIOMETRIC". Among the keywords associated with the Scopus database itself, the following stand out: "BIBLIOMETRICS", "HUMAN", "ARTICLE", "UNITED STATES" and "HUMANS". See Table 4.
In the abstracts of the 1246 documents, we proceeded to search for the presence of each of the previously selected software, as part of the applied method. As a result of the search, it was found that "VOSviewer" was the most used bibliometric software in this first group of evaluated scientific papers; far behind, and with significant distances, were "CiteSpace" and "Bibliometrix". See Figure 2.
The fields with the greatest presence in the abstracts of the 1246 papers evaluated were: "Health" in 245 documents, "Environment" in 190, "Medicine" in 143, "Education" in 105, and "Materials" in 85. With an incidence of less than 50 papers, the fields were: "Biology", "Economics", "Services", "Chemistry", "Statistics", "Agriculture", "Law", "Psychology", "Arts", "Sports", "Biochemistry", "Languages", "Environmental sciences", "Pharmacy", "Physics", "Mathematics", "Forestry", "Veterinary", "Humanities", "Fisheries", "Welfare", "Information and Communication Technologies", "Earth sciences", "Arts and humanities", "Food processing", "Textiles", "Nursing and midwifery", "Therapy and rehabilitation", "Occupational health and safety", "Physical sciences". Figure 3 shows a simple and quick representation of the most relevant fields identified.
The second group of bibliographic records was extracted from the Scopus database. As a main feature of the search string applied, prior to the extraction, it is noted that previously identified software were excluded, i.e.: "Bibexcel", "Bibliometrix", "BiblioTools", "CATAR", "CiteSpace", "CitNetExporer", "CRExplorer", "Headstart", "Publish or Perish", "RobotReviewer", "SciMAT", "Scopus API R code", and "VOSviewer".
The extraction generated 210 bibliographic records between 2018 and 2022. The growth rate of this type of scientific production was 14.42%. The records report the involvement of 878 authors with afiliation to 476 institutions, from 36 diferent countries.
Although the general opinion held among bibliometric experts seems to be that citations
represent a good, but not perfect, a measure of impact, the use of citation indicators has increased
[
A careful content analysis was carried out, taking into account mainly what was stated in the summary and the methodological procedure declared, for each of the selected articles. This review was carried out with the purpose of recognizing other tools or applications, in addition to those already identified, that the authors rely on to improve bibliometric studies.
The articles were evaluated, and as a result, it can be pointed out that 82% of the sample corresponds to research strictly related to bibliometric studies. In addition, 56.09% use software other than those initially identified in this study. Other relevant and useful software for bibliometric analysis and visualization of results emerged, characterized by being open source, free or proprietary.
The use of statistical analysis software was found, such as: "Microsoft Excel" (proprietary) and "XLSTAT" (proprietary), "R" (free), "GraphPad Prism" (proprietary), and "Addinsoft" (proprietary). In other cases, some data mining applications, such as "QDA Miner software" (proprietary), "Wordstat software" (proprietary), and "Python’s Pandas library" (free). In addition, we found software specifically cataloged in bibliometrics, such as "Covidence" (proprietary), "HistCite" (proprietary), "BICOMB" (free), "EndNote X9.6 Statistical" (proprietary), "Ucinet 6.0" (proprietary), "Tools for Innovation Monitoring (TIM)" (proprietary), "Profiles Research Networking Software" (free). Finally, among the visualization programs for bibliomentry, the following stand out: "gCLUTO" (free) and "Pajek software" (free).
In addition, a number of variations in the way the names of the tools are spelled were detected,
including "VOSviewer", "BICOMB" and "CitNetExplorer", which could limit the results of the
identification and selection of suitable software applications for the purposes of other studies.
does not apply
QDA Miner software
Provalis Research
pAuPsIs RESTful of Sco- [
Spectrometry-Based Proteomics Evidences 139 Scientific Development Of Smart Farming Technologies
And Their Application In Brazil
The Four Dimensions Of Social Network Analysis: An
Overview Of Research Methods, Applications, And
Software Tools
A Bibliometric History Of The Journal Of Psychology
Between 1936 And 2015
Artificial Intelligence In Health Care: Bibliometric
Analysis
Current Status And Challenges Of Additive Manufac- does not apply
turing In Orthopaedics: An Overview
3D Printed Medical Parts With Diferent Materials
Using Additive Manufacturing
Open-Design: A State Of The Art Review
Sustainable Approach Of Using Sugarcane Bagasse Ash
In Cement-Based Composites: A Systematic Review
Classification Of Forecasting Methods In Production
Engineering
The State Of Exosomes Research: A Global Visualized
Analysis
R Core Team 2015
Bibliometric Analysis Of Tumor Immunotherapy Stud- BICOMB; gCLUTO
ies
How New Concepts Become Universal Scientific
Approaches: Insights From Citation Network Analysis Of
Agent-Based Complex Systems Science
Corporate Social Responsibility And Supply Chain Man- VOSviewer
agement: Framing And Pushing Forward The Debate
R basado en Diderot
[
A Large-Scale Analysis Of Bioinformatics Code On
Github
Citation Network Analysis Of The Novel Coronavirus
Disease 2019 (Covid-19)
A Bibliometric-Based Technique To Identify Emerging
Photovoltaic Technologies In A Comparative
Assessment With Expert Review
Twenty-Year Span Of Global Coronavirus Research
Trends: A Bibliometric Analysis
The Relationship Between Innovation And Sustainabil- VOSviewer
ity: A Bibliometric Review Of The Literature
Software Review: COCI, The Opencitations Index Of
Crossref Open Doi-To-Doi Citations
Identification Of Recent Trends In Research On Vitamin
D: A Quantitative And Co-Word Analysis
A Bibliometric And Topic Analysis On Future
Competences At Smart Factories
A Comprehensive Overview Of Geopolymer Compos- VOSviewer
ites: A Bibliometric Analysis And Literature Review
EndNote X9.6
Statistical; SPSS; CiteSpace; [
Index
BICOMB; gCLUTO
wWaorredstat Provalis soft- [
Twenty-Five Years Of The Information Systems Journal: VOSviewer
A Bibliometric And Ontological Overview
Bibliometric Analysis Of Global Research On The Re- HistCite; CiteSpace
habilitation Of Spinal Cord Injury In The Past Two
Decades
Scientific Mapping Of Industry 4.0 Research: A
Bibliometric Analysis
Measuring The Impact Of Pharmacoepidemiologic Re- Almetrics
search Using Altmetrics: A Case Study Of A Cnodes
Drug-Safety Article
Trends And Visibility Of "Digital Health" As A Keyword
In Articles By Jmir Publications In The New Millennium:
Bibliographic-Bibliometric Analysis
The Most-Cited Authors Who Published Papers In Jmir
Mhealth And Uhealth Using The Authorship-Weighted
Scheme: Bibliometric Analysis
Profiles Research
Networking Software
Pajek software
50 Years Of International Journal Of Systems Science: CiteSpace; VOSviewer [
Business Capabilities For Industrial Firms: A Bibliomet- CitNetExplorer ric Analysis Of Research Difusion And Impact Within And Beyond Industrial Marketing Management Evlncrnas 2.0: An Updated Database Of Manually Cu- does not apply rated Functional Long Non-Coding Rnas Validated By Low-Throughput Experiments Documenting Contributions To Scholarly Articles Us- R library tenzing ing Credit And Tenzing Customer Relationship Management (CRM): A Biblio- does not apply metric Analysis
TC: Total citations Ref.: Reference
This type of study provides information on applications, technological tools, or software that can be used by researchers to improve the eficiency of bibliometric research. It supports the development of bibliometric research, which can be applied to any discipline or science, in the sense that it can be developed through a wide variety of tools, for bibliometric evaluation and mapping in any selected field of research.
In the first analysis, a total of 27 software tools used in bibliometric studies were identified. This software were evaluated, and due to their characteristics of being actively developed, free of charge, non-commercial, or open source, the following applications were selected for this study: "Bibexcel", "Bibliometrix", "BiblioTools", "CATAR", "CiteSpace", "CitNetExporer", "CRExplorer", "Headstart", "Publish or Perish", "RobotReviewer", "SciMAT", "Scopus API R code", "VOSviewer".
Two sets of bibliographic records were extracted from the Scopus database; a total of 1,246 and 210 items were selected respectively. The first was the result of a search that expressly included the names of the previously identified software; the second was based on the search for papers that had considered software-based techniques and bibliometrics, but excluded the listed software, for the recognition of other bibliometric application alternatives.
From the bibliometric evaluation of the first group of data, it is worth highlighting the high annual increase rate of scientific production, which was 64.71%, a sign of the importance of this topic in diferent fields.
As a fact of regional interest, the Latin American countries of Colombia, Ecuador, and Brazil stand out in their scientific production, in the first group of evaluated works related to bibliometrics. The first country for the amount of scientific production and the next two for the number of citations registered.
The following institutions stand out for the number of articles published: Central South University, Sichuan University, An-Najah National University.
The software with the greatest presence in scientific production related to bibliometrics are "VOSviewer", "CiteSpace" and "Bibliometrix"; when evaluating their frequency, it should be noted that the distance separating each of the three tools is significant.
The fields detected with the greatest presence in the abstracts evaluated were: "Health", "Environment", "Medicine", "Education" and "Materials".
Through the content analysis method, fifty articles were evaluated, selected by their number of citations, from the second group of bibliographic records taken from Scopus; this in order to identify emerging software related to bibliometric techniques. It was found that more than half of the documents reviewed used software other than those already recognized.
In addition to the software classified as bibliometric or visualizers of bibliometric results, those that allow statistical analysis or data mining were identified. Statistical programs such as: "Microsoft Excel" and "XLSTAT, R", "GraphPad Prism" and "Addinsoft" are used. In data mining: "QDA Miner software", "Wordstat software" and "Python’s Pandas library". Specific in bibliometrics: "Covidence", "HistCite", "BICOMB", "EndNote X9.6 Statistical", "Ucinet 6.0", "Tools for Innovation Monitoring" and "Profiles Research Networking Software". As visualization software for bibliomentrics, "gCLUTO" and "Pajek" were found.
Information technologies are evolving exponentially, and new software alternatives are emerging every day in all areas. It is important to periodically evaluate the scientific production of bibliometric type, to know in a permanent way, the new software that supports this type of activity. In addition, in future research, it would be advisable to review and describe the functional scope of bibliometric software in order to classify and evaluate it.