MDSS: Methodology for the Development of Scientometric Studies Emigdio Alfaro1 1 Universidad Tecnológica del Perú, Av. El Sol 235, San Juan de Lurigancho, Lima, Perú Abstract The problem of scientific research was the lack of a methodology that guides the development of scientometric studies, which would limit the development of this type of study. Therefore, the purpose of the study was to elaborate a methodology for the development of scientometric studies for various sciences. The study methodology was developed through a qualitative and narrative topical research design and was applied to the development of scientometric studies of environmental engineering. As a result of the application of the methodology, two scientometric studies have been published in Scopus and Web of Science journals (Q2 and Q1 [17th place in the Soil Science category]) and an article is under evaluation in an indexed journal. The MDSS methodology contains the details for the following processes: (a) to propose the idea of the scientometric study, (b) to generate the information search strings and their results, (c) to check the quality of the information search results, (d) to structure the information through bibliometric applications, and (e) to complete the sections of the article. The application of the methodology developed in this study to other sciences is recommended. Keywords scientometric study, bibliometric study, science trends, method, methodology1 1. Introduction Scientific research must be original and provide knowledge contribution and without these characteristics they cannot be accepted in high-level scientific journals, especially in journals indexed in Scopus and Web of Science in their respective quartiles [1]. Scientometric studies are also important sources of ideas for the development of scientific research, because they allow to know the trends of scientific research on the topic of the researcher's specialty [2]. A systematic review attempts to collect all relevant evidence that fits pre-specified eligibility criteria to answer a specific research question; Furthermore, a systematic review uses explicit and systematic methods to minimize bias in the identification, selection, synthesis and summary of studies, presenting reliable results from which conclusions can be drawn and decisions made [4]. The key characteristics of a systematic review are: (a) a set of clearly stated objectives with an explicit and reproducible methodology; (b) a systematic search attempting to identify all studies that would meet the eligibility criteria; (c) an assessment of the validity of the results of the included studies (e.g., assessment of risk of bias and confidence in cumulative estimates); and (d) systematic presentation and synthesis of the characteristics and findings of the included studies (p. 3) [3]. Furthermore, the systematic review becomes a meta-analysis when it allows the collection and combination of quantitative data from various studies along with their respective statistical analysis [4]. Scientometric studies are different from systematic reviews. Scientometrics studies the quantitative aspects of science, including: (a) The quantitative growth of science, (b) the development of disciplines and subdisciplines, (c) the relationship between science and CISETC 2023: International Congress on Education and Technology in Sciences 2023, December 04–06, 2023, Zacatecas, Mexico emigdio.alfaro@gmail.com (E. Alfaro) 0000-0002-0309-9195 (E. Alfaro) © 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) CEUR ht tp: // ceur -ws .or g Works hop I SSN1613- 0073 Pr oceedi ngs CEUR ceur-ws.org Workshop ISSN 1613-0073 Proceedings technology, (d) the obsolescence of scientific paradigms, (e) the structure of communication between scientists, (f) the productivity and creativity of researchers, and (g) the relationships between scientific development and economic growth [5]. As can be seen, scientometric studies are quantitative, while systematic reviews are qualitative. Scientometric studies seek to make known the evolution and future trends of a particular topic of a science, in addition to providing the associated bibliometric information, unlike systematic reviews that seek to know the studies carried out and published in greater depth, with the purpose to obtain much more valuable conclusions from a global analysis of the literature on the specific topic based on specific analysis criteria. Furthermore, scientometric studies are also different from meta-analyses, because meta-analyses collect quantitative information from the contents of published articles to process them statistically; however, scientometric studies process bibliometric information from published articles, but not information about their contents. Various scientometric studies were found in the literature review; but, none with a generic methodology for the preparation of scientometric studies for the various sciences. One study was found with a flexible method to improve the quality of data retrieved for scientometric studies, which the authors called systematic scientometric reviews [6]. As previously explained, systematic reviews and meta-analyses are different from scientometric studies. It should be noted that scientometric studies collect information from the Scopus and Web of Science databases separately, because the structures of the information classified by Scopus and Web of Science are not exactly the same because they are competing indexes in the scientific information market. 2. Methodology This study had a qualitative and narrative topical design. The processes of the Methodology for the Development of Scientiometric Studies (MDSS) are in figure 1. 1. To propose the 2. To generate 3. To check the quality of idea of the information search strings information search results scientometric study. and their results. 5. To complete the 4. To structure sections of the information through article. bibliometric applications. Figure 1: Processes of the Methodology MDSS. 2.1 To propose the idea of the scientometric study The steps of the formulation of the research idea are the following: A. To search scientometric studies related to the topic under study. B. To identify not analyzed aspects within previous scientometric studies. C. To evaluate the way in which information about not analyzed aspects within previous scientometric studies will be collected. D. To evaluate technologies or methodologies to perform additional analyses that have not been covered. E. To evaluate the originality of the study and the potential knowledge contribution. If the topic of the scientometric study proposed is original and is expected to provide knowledge; then, continue with the following steps. 2.2 To generate information search strings and their results The steps for generating information search strings and their results are the following: A. To generate the search string in Scopus. The steps are the following: a. To go to the link: https://www.scopus.com/search/form.uri?display=basic#basic b. To enter the keywords for the search. c. To select the search options, considering: types of publications, years of publications, subject areas, languages, etc. d. To get the search string text under the “Search history” option. e. To export the generated information. This export can be done in the following formats: Mendeley, ExLibris, RIS, CSV, BibTeX and plain text. The use of the RIS format is suggested. B. To generate the search string in Web of Science. The steps are the following: a. To enter the Web of Science with institutional access. b. To enter the keywords for the search. c. To select the search options, considering: types of publications, years of publications, subject areas, languages, etc. d. To obtain the text of the search string with the “Copy query link” option. e. To export the information generated by clicking on the “Export” option. This export can be done in the following formats: EndNote online, EndNote deskop, Add to my researcher profile, plain text, RefWorks, RIS, BibTeX, Excel, Tab delimited file, Printable HTML File, InCites, Email, Fast 5000, etc. 2.3 To check the quality of information search results The steps for checking the quality of the information search results are as follows: A. To review the summaries of 10% of the total articles, considering the years of publications during the evaluated time period. B. If there are articles that are not related to the subject of the specialty; then, repeat the steps in section 2.2, excluding the keywords associated with the different result. 2.4 To structure information through bibliometric applications The suggested structure for the “Results and discussion” section (for Scopus and Web of Science, in separate tables and figures of the bibliometric applications) is as follows: (a) types and quantities of publications, (b) publication trends per year, (c) subject areas, (d) most cited journals, (e) most cited authors, (f) most cited articles, (g) most used keywords and co- occurrence, (h) countries with the highest production and collaborations, (i) institutions with greater production and collaborations, (j) authors with greater production and collaborations, (k) most used technologies, methods or materials, and (l) evolution of technologies, methods or materials. 2.5 To complete the sections of the article The contents to be completed in the sections of the article are the following: A. Introduction: importance of the study, knowledge contribution, problematic reality, background (previous studies), theoretical bases (related theories) and conceptual framework. B. Methodology: sources of information (used academic databases), search string developed in Scopus, search string developed in Web of Science, and detailed description of the subsections of the “Results and discussion” section. C. Results and discussion: processed information from Scopus and Web of Science in tables and figures with comments that present it before each table or figure, together with comparisons of similarities and differences of the results of the current scientometric study with explanations about the reasons for the similarities and differences. All this content must be structured in the sections described in the methodology previously (to see step 2.4). D. Conclusions: synthesis of the reasons expressed in the discussions along with the knowledge gaps that have not been covered in scientific research about the topic of the scientometric study being carried out. E. Recommendations: recommendations for future research that include how and why the knowledge gaps described in the conclusions should be covered, along with suggestions for systematic reviews or meta-analyses on some specific topics that would be interesting for the human knowledge contribution, considering the opinion of the researchers. 3. Results As a result of the application of the methodology, two scientometric studies have been published in Scopus and Web of Science journals, which are mentioned in table 1. Table 1 Articles based on scientometric studies that have been developed with MDSS Article Year Journal Scimago Scimago Category Journal Journal Ranking’s Ranking’s Quartile Position Scientometric study of 2023 Applied Soil Q1 17 Soil Science treatment technologies of soil Ecology pollution: Present and future challenges [7] Scientometric study of 2021 Cogent Q2 157 Chemical drinking water treatments Engineering Engineering technologies: Present and (Miscellaneous) future challenges [8] An additional study titled: “Scientometric study on air quality: Trends and challenges” has also been developed, which is being evaluated in the journal Atmospheric Environment. 4. Discussion The extensive and diverse scientific literature provides scientometric studies of various knowledge areas; however, only one study has been found that provides a methodology related to the methodology developed in this research [5]. This study was focused on improving the search chain of a scientometric study, without offering the specific steps for the preparation of a scientometric study that allows it to be applicable to the various areas of knowledge, nor presenting the specific structures of the content of each one of the sections of the article, as has been achieved with this research [5]. The results of the study show that the MDSS methodology allowed the achievement of two high- level scientific publications (Q1 and Q2 in the Scimago Journal Ranking) and is expected to continue allowing the achievement of a greater number of scientometric studies, given that the five processes developed would allow them to guide their developments in detail. The application of the methodology developed in this study is recommended for other sciences in addition to environmental engineering, considering that its steps are generic and could be applied without major adaptations to the various areas of human knowledge. Acknowledgements The author thanks to Universidad Tecnológica del Perú for the support to this study. References [1] Alfaro, Emigdio. ”Criterios para la selección de revistas científicas para el envío de potenciales publicaciones" [Criteria for the selection of scientific journals for the sending of potential publications]. Segundo Congreso Internacional de Editores e Investigadores de Revistas Científicas (2021). URL: https://www.youtube.com/watch?v=h7x2z6spZCs&t=281s [2] Alfaro, Emigdio. ”Metodología para la identificación de temas de investigación científica publicables" [Methodology for the identification of publishable scientific research topics]. Primer Congreso Internacional de Editores e Investigadores de Revistas Científicas (2020). URL: https://www.youtube.com/watch?v=X7UiRMIYDMk&t=186s [3] Moher, David, Shamseer, Larissa, Clarke, Mike, Ghersi, Davina, Liberati, Alessandro, Petticrew, Mark, Shekelle, Paul, Stewart, Lesley. Preferred reporting ítems for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Systematic Reviews (2015) 4(1): 1-9. http://10.1186/2046-4053-4-1 [4] Ferreira, Ignacio, Urrutia, Gerard y Alonso, Pablo. Revisiones sistemáticas y metaanálisis: bases conceptuales e interpretación [Systematic Reviews and Meta Analyses: conceptual basis and interpretation]. Revista Española de Cardiología (2011) 64(8): 688-696. [5] Araújo Ruiz, Juan A, & Arencibia Jorge, Ricardo. Informetría, bibliometría y cienciometría: aspectos teórico-prácticos [Informetrics, Bibliometrics, and Scientometrics: theoretical- practical aspects]. ACIMED (2002) 10(4): 5-6. http://scielo.sld.cu/scielo.php?script=sci_arttext&pid=S1024- 94352002000400004&lng=es&tlng=pt. [6] Cheng, Chaomeo, Song, Min. Visualizing a field of research: A methodology of systematic scientometric reviews. Plos One (2019) 14(10): e0223994. https://doi.org/10.1371/journal.pone.0223994 [7] Valdiviezo, Lorgio, Castañeda, Carlos, Cabello, Rita, García, Fausto., Munive, Rubén, & Alfaro, Emigdio. Scientometric study of treatment technologies of soil pollution: Present and future challenges. Applied Soil Ecology (2023) 182: 104695. [8] Valdiviezo, Lorgio, García, Fausto, Cabello, Rita, Castañeda, Carlos, & Alfaro, Emigdio Scientometric study of drinking water treatments technologies: Present and future challenges. Cogent Engineering (2021) 8(1): 1929046, DOI: 10.1080/23311916.2021.1929046