=Paper=
{{Paper
|id=Vol-3691/paper55
|storemode=property
|title=MDSS: Methodology for the Development of Scientometric Studies
|pdfUrl=https://ceur-ws.org/Vol-3691/paper55.pdf
|volume=Vol-3691
|authors=Emigdio Alfaro
|dblpUrl=https://dblp.org/rec/conf/cisetc/Alfaro23
}}
==MDSS: Methodology for the Development of Scientometric Studies==
https://ceur-ws.org/Vol-3691/paper55.pdf
MDSS: Methodology for the Development of Scientometric
Studies
Emigdio Alfaro1
1 Universidad Tecnológica del Perú, Av. El Sol 235, San Juan de Lurigancho, Lima, Perú
Abstract
The problem of scientific research was the lack of a methodology that guides the development of
scientometric studies, which would limit the development of this type of study. Therefore, the purpose
of the study was to elaborate a methodology for the development of scientometric studies for various
sciences. The study methodology was developed through a qualitative and narrative topical research
design and was applied to the development of scientometric studies of environmental engineering. As a
result of the application of the methodology, two scientometric studies have been published in Scopus
and Web of Science journals (Q2 and Q1 [17th place in the Soil Science category]) and an article is under
evaluation in an indexed journal. The MDSS methodology contains the details for the following
processes: (a) to propose the idea of the scientometric study, (b) to generate the information search
strings and their results, (c) to check the quality of the information search results, (d) to structure the
information through bibliometric applications, and (e) to complete the sections of the article. The
application of the methodology developed in this study to other sciences is recommended.
Keywords
scientometric study, bibliometric study, science trends, method, methodology1
1. Introduction
Scientific research must be original and provide knowledge contribution and without these
characteristics they cannot be accepted in high-level scientific journals, especially in journals
indexed in Scopus and Web of Science in their respective quartiles [1]. Scientometric studies are
also important sources of ideas for the development of scientific research, because they allow to
know the trends of scientific research on the topic of the researcher's specialty [2].
A systematic review attempts to collect all relevant evidence that fits pre-specified eligibility
criteria to answer a specific research question; Furthermore, a systematic review uses explicit
and systematic methods to minimize bias in the identification, selection, synthesis and summary
of studies, presenting reliable results from which conclusions can be drawn and decisions made
[4]. The key characteristics of a systematic review are: (a) a set of clearly stated objectives with
an explicit and reproducible methodology; (b) a systematic search attempting to identify all
studies that would meet the eligibility criteria; (c) an assessment of the validity of the results of
the included studies (e.g., assessment of risk of bias and confidence in cumulative estimates); and
(d) systematic presentation and synthesis of the characteristics and findings of the included
studies (p. 3) [3]. Furthermore, the systematic review becomes a meta-analysis when it allows
the collection and combination of quantitative data from various studies along with their
respective statistical analysis [4].
Scientometric studies are different from systematic reviews. Scientometrics studies the
quantitative aspects of science, including: (a) The quantitative growth of science, (b) the
development of disciplines and subdisciplines, (c) the relationship between science and
CISETC 2023: International Congress on Education and Technology in Sciences 2023, December 04–06, 2023,
Zacatecas, Mexico
emigdio.alfaro@gmail.com (E. Alfaro)
0000-0002-0309-9195 (E. Alfaro)
© 2023 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)
CEUR
ht
tp:
//
ceur
-ws
.or
g
Works
hop I
SSN1613-
0073
Pr
oceedi
ngs
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
technology, (d) the obsolescence of scientific paradigms, (e) the structure of communication
between scientists, (f) the productivity and creativity of researchers, and (g) the relationships
between scientific development and economic growth [5].
As can be seen, scientometric studies are quantitative, while systematic reviews are
qualitative. Scientometric studies seek to make known the evolution and future trends of a
particular topic of a science, in addition to providing the associated bibliometric information,
unlike systematic reviews that seek to know the studies carried out and published in greater
depth, with the purpose to obtain much more valuable conclusions from a global analysis of the
literature on the specific topic based on specific analysis criteria. Furthermore, scientometric
studies are also different from meta-analyses, because meta-analyses collect quantitative
information from the contents of published articles to process them statistically; however,
scientometric studies process bibliometric information from published articles, but not
information about their contents.
Various scientometric studies were found in the literature review; but, none with a generic
methodology for the preparation of scientometric studies for the various sciences. One study was
found with a flexible method to improve the quality of data retrieved for scientometric studies,
which the authors called systematic scientometric reviews [6]. As previously explained,
systematic reviews and meta-analyses are different from scientometric studies. It should be
noted that scientometric studies collect information from the Scopus and Web of Science
databases separately, because the structures of the information classified by Scopus and Web of
Science are not exactly the same because they are competing indexes in the scientific information
market.
2. Methodology
This study had a qualitative and narrative topical design. The processes of the Methodology for
the Development of Scientiometric Studies (MDSS) are in figure 1.
1. To propose the 2. To generate
3. To check the quality of
idea of the information search strings
information search results
scientometric study. and their results.
5. To complete the 4. To structure
sections of the information through
article. bibliometric applications.
Figure 1: Processes of the Methodology MDSS.
2.1 To propose the idea of the scientometric study
The steps of the formulation of the research idea are the following:
A. To search scientometric studies related to the topic under study.
B. To identify not analyzed aspects within previous scientometric studies.
C. To evaluate the way in which information about not analyzed aspects within previous
scientometric studies will be collected.
D. To evaluate technologies or methodologies to perform additional analyses that have not
been covered.
E. To evaluate the originality of the study and the potential knowledge contribution. If the
topic of the scientometric study proposed is original and is expected to provide knowledge;
then, continue with the following steps.
2.2 To generate information search strings and their results
The steps for generating information search strings and their results are the following:
A. To generate the search string in Scopus. The steps are the following:
a. To go to the link: https://www.scopus.com/search/form.uri?display=basic#basic
b. To enter the keywords for the search.
c. To select the search options, considering: types of publications, years of publications,
subject areas, languages, etc.
d. To get the search string text under the “Search history” option.
e. To export the generated information. This export can be done in the following formats:
Mendeley, ExLibris, RIS, CSV, BibTeX and plain text. The use of the RIS format is
suggested.
B. To generate the search string in Web of Science. The steps are the following:
a. To enter the Web of Science with institutional access.
b. To enter the keywords for the search.
c. To select the search options, considering: types of publications, years of publications,
subject areas, languages, etc.
d. To obtain the text of the search string with the “Copy query link” option.
e. To export the information generated by clicking on the “Export” option. This export
can be done in the following formats: EndNote online, EndNote deskop, Add to my
researcher profile, plain text, RefWorks, RIS, BibTeX, Excel, Tab delimited file,
Printable HTML File, InCites, Email, Fast 5000, etc.
2.3 To check the quality of information search results
The steps for checking the quality of the information search results are as follows:
A. To review the summaries of 10% of the total articles, considering the years of publications
during the evaluated time period.
B. If there are articles that are not related to the subject of the specialty; then, repeat the steps
in section 2.2, excluding the keywords associated with the different result.
2.4 To structure information through bibliometric applications
The suggested structure for the “Results and discussion” section (for Scopus and Web of
Science, in separate tables and figures of the bibliometric applications) is as follows: (a) types and
quantities of publications, (b) publication trends per year, (c) subject areas, (d) most cited
journals, (e) most cited authors, (f) most cited articles, (g) most used keywords and co-
occurrence, (h) countries with the highest production and collaborations, (i) institutions with
greater production and collaborations, (j) authors with greater production and collaborations,
(k) most used technologies, methods or materials, and (l) evolution of technologies, methods or
materials.
2.5 To complete the sections of the article
The contents to be completed in the sections of the article are the following:
A. Introduction: importance of the study, knowledge contribution, problematic reality,
background (previous studies), theoretical bases (related theories) and conceptual
framework.
B. Methodology: sources of information (used academic databases), search string developed
in Scopus, search string developed in Web of Science, and detailed description of the
subsections of the “Results and discussion” section.
C. Results and discussion: processed information from Scopus and Web of Science in tables
and figures with comments that present it before each table or figure, together with
comparisons of similarities and differences of the results of the current scientometric
study with explanations about the reasons for the similarities and differences. All this
content must be structured in the sections described in the methodology previously (to
see step 2.4).
D. Conclusions: synthesis of the reasons expressed in the discussions along with the
knowledge gaps that have not been covered in scientific research about the topic of the
scientometric study being carried out.
E. Recommendations: recommendations for future research that include how and why the
knowledge gaps described in the conclusions should be covered, along with suggestions
for systematic reviews or meta-analyses on some specific topics that would be interesting
for the human knowledge contribution, considering the opinion of the researchers.
3. Results
As a result of the application of the methodology, two scientometric studies have been published
in Scopus and Web of Science journals, which are mentioned in table 1.
Table 1
Articles based on scientometric studies that have been developed with MDSS
Article Year Journal Scimago Scimago Category
Journal Journal
Ranking’s Ranking’s
Quartile Position
Scientometric study of 2023 Applied Soil Q1 17 Soil Science
treatment technologies of soil Ecology
pollution: Present and future
challenges [7]
Scientometric study of 2021 Cogent Q2 157 Chemical
drinking water treatments Engineering Engineering
technologies: Present and (Miscellaneous)
future challenges [8]
An additional study titled: “Scientometric study on air quality: Trends and challenges” has also
been developed, which is being evaluated in the journal Atmospheric Environment.
4. Discussion
The extensive and diverse scientific literature provides scientometric studies of various
knowledge areas; however, only one study has been found that provides a methodology related
to the methodology developed in this research [5]. This study was focused on improving the
search chain of a scientometric study, without offering the specific steps for the preparation of a
scientometric study that allows it to be applicable to the various areas of knowledge, nor
presenting the specific structures of the content of each one of the sections of the article, as has
been achieved with this research [5].
The results of the study show that the MDSS methodology allowed the achievement of two high-
level scientific publications (Q1 and Q2 in the Scimago Journal Ranking) and is expected to
continue allowing the achievement of a greater number of scientometric studies, given that the
five processes developed would allow them to guide their developments in detail. The application
of the methodology developed in this study is recommended for other sciences in addition to
environmental engineering, considering that its steps are generic and could be applied without
major adaptations to the various areas of human knowledge.
Acknowledgements
The author thanks to Universidad Tecnológica del Perú for the support to this study.
References
[1] Alfaro, Emigdio. ”Criterios para la selección de revistas científicas para el envío de
potenciales publicaciones" [Criteria for the selection of scientific journals for the sending of
potential publications]. Segundo Congreso Internacional de Editores e Investigadores de
Revistas Científicas (2021). URL: https://www.youtube.com/watch?v=h7x2z6spZCs&t=281s
[2] Alfaro, Emigdio. ”Metodología para la identificación de temas de investigación científica
publicables" [Methodology for the identification of publishable scientific research topics].
Primer Congreso Internacional de Editores e Investigadores de Revistas Científicas (2020).
URL: https://www.youtube.com/watch?v=X7UiRMIYDMk&t=186s
[3] Moher, David, Shamseer, Larissa, Clarke, Mike, Ghersi, Davina, Liberati, Alessandro,
Petticrew, Mark, Shekelle, Paul, Stewart, Lesley. Preferred reporting ítems for systematic
review and meta-analysis protocols (PRISMA-P) 2015 statement. Systematic Reviews (2015)
4(1): 1-9. http://10.1186/2046-4053-4-1
[4] Ferreira, Ignacio, Urrutia, Gerard y Alonso, Pablo. Revisiones sistemáticas y metaanálisis:
bases conceptuales e interpretación [Systematic Reviews and Meta Analyses: conceptual
basis and interpretation]. Revista Española de Cardiología (2011) 64(8): 688-696.
[5] Araújo Ruiz, Juan A, & Arencibia Jorge, Ricardo. Informetría, bibliometría y cienciometría:
aspectos teórico-prácticos [Informetrics, Bibliometrics, and Scientometrics: theoretical-
practical aspects]. ACIMED (2002) 10(4): 5-6.
http://scielo.sld.cu/scielo.php?script=sci_arttext&pid=S1024-
94352002000400004&lng=es&tlng=pt.
[6] Cheng, Chaomeo, Song, Min. Visualizing a field of research: A methodology of systematic
scientometric reviews. Plos One (2019) 14(10): e0223994.
https://doi.org/10.1371/journal.pone.0223994
[7] Valdiviezo, Lorgio, Castañeda, Carlos, Cabello, Rita, García, Fausto., Munive, Rubén, & Alfaro,
Emigdio. Scientometric study of treatment technologies of soil pollution: Present and future
challenges. Applied Soil Ecology (2023) 182: 104695.
[8] Valdiviezo, Lorgio, García, Fausto, Cabello, Rita, Castañeda, Carlos, & Alfaro, Emigdio
Scientometric study of drinking water treatments technologies: Present and future
challenges. Cogent Engineering (2021) 8(1): 1929046, DOI:
10.1080/23311916.2021.1929046