LIA Method for the Application of Microsoft Excel in Data Tabulation in Systematic Reviews Guillermo S. Miñan1, Joandri A. Moreno1 and Ximena D. Fernández1 1 Universidad Tecnológica del Perú, Km 424 Panamericana Norte - Calle 56 S/N, Chimbote, Perú Abstract The objective of this research was to design an office tool for data tabulation in systematic reviews with a quantitative approach. Therefore, in the methodological aspect, the research was applied, with a qualitative approach and a descriptive scope. A research/action design was used, which is a type of qualitative research that aims to solve everyday and immediate problems, and improve concrete practices, in this specific case, data management in systematic reviews. The results described the use of tables, dynamic tables, forms, code in Visual Basic, among other aspects. In that sense, a tool was designed in Microsoft Excel that was able to record research, process it orderly and extract information through graphs and tables for their respective bibliometric and content analysis. Finally, it was concluded that a computer solution based on Microsoft Excel is viable for the development of systematic reviews due to its low cost, flexibility and robust capacity for handling large amounts of data. Keywords Visual Basic, Systematic Review, Microsoft Excel, Macros1 1. Introduction Nowadays, it is well known that digitalization is allowing all information to be found with the help of technology, which in turn causes the need for modern tools that increase the efficiency of various processes or tasks. This need is present in systematic literature reviews. A systematic review is the explicit assessment of the literature, derived from a clear research question, along with a critical analysis according to different tools and a qualitative summary of the evidence [1]. In 1965, Derek estimated that after 30 or 40 papers were published in a specialty, a review was necessary and, in that sense, in 1986 Garfield examined the percentage of existing review articles with respect to the total number of articles indexed in the Science Citation Index (SCI) which was approximately 4.8 (about 30,000 review articles out of the 625,432 articles included in the index that year); while in the Chemical Abstracts (CA), it was 10.4 (approximately 40,000 out of 385,000) [2]. These approaches are analyzed through the arrival of the current world and its globalization process that have generated the increasing and constant emergence of new information, reflected in multiple articles and publications [3]; however, the number of systematic reviews published in Spanish journals has increased in recent years. In a quick attempt to support this statement, it has been estimated that there would be about 750 systematic reviews or meta-analyses published to date [4]. Systematic reviews are an integrative, observational, retrospective, secondary study, in which studies that examine the same question are combined. In turn, within the systematic review there are two forms: “quantitative or meta-analysis” and “qualitative or overview”. The differences are fundamentally given by the use of statistical methods, which allows the combination and quantitative analysis of the results obtained in each study that is CISETC 2023: International Congress on Education and Technology in Sciences, December 04–06, 2023, Zacatecas, Mexico C20342@utp.edu.pe (G. Miñan-Olivos); u22219738@utp.edu.pe (J. Moreno-Ramos); u22322252@utp.edu.pe (X. Fernández-Cruz); 0000-0001-9523-8043 (G. Miñan-Olivos); 0000-0002-3848-5782 (J. Moreno-Ramos); 0000-0003-1797-703X (X. Fernández-Cruz); © 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) CEUR ht tp: // ceur -ws .or g Works hop I SSN1613- 0073 Pr oceedi ngs CEUR ceur-ws.org Workshop ISSN 1613-0073 Proceedings carried out. The term meta-analysis was introduced by Glass in 1976, who defined it as follows: a statistical analysis of a large series of analyses of results from individual studies with the aim of integrating their findings. Although sometimes the two terms are used interchangeably, a systematic review is not the same as a meta-analysis [5]. In that sense, a systematic review will not always have a meta-analysis, in many cases it will depend on the information collected or on the specialty on which it is developed. The realization of high-quality systematic reviews is not easy and, sometimes, they can be difficult to interpret, for this reason certain recommendations should be taken into account to avoid errors that affect the search for relevant studies for a systematic review. Therefore, it is very useful to use in this phase a software for the automated management of bibliographic citations such as, for example, ProCite that has a wide variety of input and output formats [6]. There are also methodological search filters, previously tested to locate the relevant literature within a database. To date, many search filters have been developed, operated and evaluated to identify studies and reviews in online databases. In this regard, in the field of medicine, the Prognosis group has identified prognostic and search strategies as a methodological priority for the development of this type of systematic reviews [7]. However, users suggest that various improvements in the models of production of Cochrane systematic reviews could come from improving clarity of roles and expectations, ensuring continuity and consistency of contributions, allowing active management of the review to create capacity and share information among authors and review groups [8]. On the other hand, when starting a systematic review, it is advisable to find out if there is any regional or specialized library in the subject, as well as specific databases that are relevant for your review topic. As its name indicates, specialized databases in subjects index studies on a particular topic [9]. That is, they provide a complete picture of the totality of the evidence on a given topic [10]. Systematic reviews of quality often have greater power and less bias than the individual studies they include, and the careful combination of treatment effects can provide the most accurate overall assessment of an intervention [11]. In that sense, in the field of Psychology, the arrival of Evidence-Based Psychology (EBP) has become a methodological tool by combining the best evidence with psychological practice. EBP advocates that professional practice be based on the best evidence obtained from psychological research [12]. On the other hand, there is also the PRISMA method, which is a checklist of requirements that a systematic review and meta- analysis of clinical trials must meet to present the information. It was published in 2009 with the aim of helping the authors of this type of documents to justify their research, to explain what they did and what they found. Likewise, it should be taken into account that the realization of a rigorous systematic review requires careful scientific consideration in the search, as well as considerable attention in its methodology. Systematic reviews usually, but not necessarily, must apply meta-analysis to examine a research question, which generally provide a higher degree of precision compared to other types of research because the estimation of the combination of studies is based on a larger sample than any of the individuals [13]. In accordance with what has been mentioned, there are currently different methodological tools for the elaboration of systematic reviews. Generally, the available softwares are focused on the bibliographic management of the studies, but they do not generate the transformation of the available data into tables or figures that in turn allow an adequate bibliometric, content or traceability analysis of the information. At the other extreme, there is a great variety of computer packages focused on a specialized statistical analysis (meta-analysis) but not on the bibliographic management. There is also an availability of computer programs applied for the bibliometric analysis; however, these programs focus on the analysis of the information without the application of the inclusion and exclusion criteria that are considered in a systematic review. As a consequence, there is a need for a computer tool that allows to record the information of the studies analyzed in a systematic review for their subsequent tabulation based on inclusion and exclusion criteria, with the purpose of automating the generation of tables and figures that present the bibliometric and content information in a logical and concise way. These tables and figures should be a primordial part of the results of a systematic review and their automation should facilitate the writing of this type of articles. In that sense, Excel offers the ability to create pivot tables that allow users to analyze large data sets efficiently. Therefore, it could be a very useful tool for conducting a systematic literature review, especially for those who are just beginning to explore and know the possibilities of data analysis. The use of Microsoft Excel as an office tool for systematic reviews would present various benefits in the processing of the data. First, Excel has a large storage capacity, that is, a large amount of bibliographic information could be included without any limitation. Likewise, Excel offers the possibility of mechanizing the entry of information through tables, macros or filters which in turn would facilitate the labeling of key concepts and the organized classification of the information. In the same way, Excel’s pivot tables and graphs offer an efficient alternative for the presentation of results. Based on the above, the following research question was raised: Is it possible to apply Microsoft Excel for data tabulation in systematic reviews with a quantitative approach? 2. Methodology The research had a qualitative approach by considering in detail each of the characteristics and qualities of a tool in Microsoft Excel for data tabulation in a systematic review; however, no indicators, data or quantitative values have been presented that empirically demonstrate a hypothesis, on the contrary, it has been possible to articulate results based on schemes or figures that answer the research question about the feasibility of designing an office tool for data tabulation in systematic reviews with a quantitative approach. Therefore, the study presented a descriptive scope, as it described the use of tables, pivot tables, forms, code in Visual Basic, among other aspects, to obtain a spreadsheet capable of processing all the necessary information for a systematic review. In addition, the design of the study was of the research/action type, as it focused on a specific diagnosis to propose the solution to a research problem associated with information management in the specific processes of collection, tabulation and transformation [14]. The tool designed was named as: “LIA Method for data tabulation in systematic reviews using Microsoft Excel”. Regarding the population, it is defined as a set of units, usually, people, objects, transactions or events; on which there is an interest of study. In that sense, the population of the study was structured considering all the processes of a systematic review. However, it is not possible to design an office tool that can be applicable to all the processes of a systematic review, considering the limitations of the study; that is why a sample was used, which is a portion or part of the population of interest. The sample was represented by the processes of collecting studies, tabulating bibliometric data and classifying findings or empirical results [15]. In the same way, due to the fact that the sample was not selected under any statistical method, the sampling used was a non-probabilistic sampling by convenience; that is, the sample was chosen arbitrarily [16]. 3. Results The results of the study show each of the stages executed during the development of the office tool for the support of the construction of systematic literature reviews (SLR). In Fig. 1, the tables designed for the functioning of the spreadsheet can be observed. 3.1. Tool design in Microsoft Excel The table 1 had the function of collecting primary information that would later feed tables 2, 3 and 4. All the tabulated information would allow generating graphs and dynamic tables. Table 1 contained descriptive information of each of the articles consulted, such as the name of the authors, research title and its respective URL (link). Likewise, it allowed verifying the compliance of inclusion and exclusion criteria which can be modified by each researcher or according to the methodology of the study. Likewise, Table 2 was created, also called Bibliometrics, with the purpose of extracting the complete information of the included articles such as the year of publication, the country, keywords, name of the journal, the database, the type of document and the main affiliation. All this information would generate tables and graphs of bibliometrics. In the case of Table 3, the tools used in the included articles are identified, which in turn are classified by groups or types. This information would generate a dynamic table with a count of the studies according to the classification made. The same procedure is carried out in Table 4 but identifying and classifying the findings of the included studies, specifically the effects obtained on a variable. These findings or effects are also grouped by types. The name of the authors and the research title of each study are automatically transferred by macros and formulas generating a relationship between the tables for the generation of queries and reports. Figure 1: Relationship of Tables in Microsoft Excel for the tabulation of data in Systematic Reviews 3.2. Entering information for systematic review Corresponding to Fig. 2, the functioning of Table 1 is referred to. As can be seen, there are 6 columns that are distributed as follows: Authors, research title, URL, inclusion/exclusion criteria, article status and alternatives for excluded studies. First, the names of the authors of each article will be placed, then the research titles follow; consequently, the URL or link of the database where these are located will be placed. Next, the 10 criteria that will allow the inclusion and exclusion of the articles that will be used in the systematic review are evaluated. On the other hand, the status and alternatives will depend on the compliance of criteria. If the 10 criteria are met, it means that the article will have a status of “Included” and in the alternative there is a blank space. If the criteria are partially met, the status of the study will be “Excluded”, while the alternative will show a note: “The study cannot be included in the systematic review, but it could be included in another section”; this section could be part of the introduction of the article. Finally, if the compliance of criteria is less than 4, the status of the research will be “Excluded” and in the alternative column only a blank space will appear. The importance of this table is to separate the included articles that meet all the criteria, the studies that could be used in the introduction and those that are totally excluded. This process can be done collaboratively and then join the work of different members of the research team. Likewise, the criteria can be customized according to each study. Figure 2: Table for selection of articles according to inclusion and exclusion criteria For the case of Table 2, the name of the authors and the research title of each included study are extracted. For this, macros are applied. The objective of this table is to collect all the bibliometric information of the articles included in the systematic review. It is distributed in 10 columns: Authors, Research Title, Year, Country, Language, Keywords, Journal, Database, Document Type and the Main Affiliation of each of the studies. As can be seen, the first 2 columns are columns previously registered in Table 1. As shown in Fig. 3, there is a button called “update input data” that has the function of copying all the articles included according to Table 1. This task is done at the beginning and then the bibliometric information is registered. It should be noted that a study can have several keywords. In that sense, the same row of the study should be copied and pasted, but only modifying the keyword. In this way, the dynamic tables will work correctly when generating reports or queries. Figure 3: Bibliometric table compiling information from the articles included in the systematic review 3.3. Bibliometric Reports As a result of Table 2, all the bibliometric information of each of the included studies is collected. That is why different Dynamic Tables and Graphs are obtained to show information about: Authors, Research Title, Year, Country, Language, Keywords, Journal, Database, Document Type and Main Affiliation. For example, in Fig. 4 you can see the dynamic table that consolidates the authors of the included articles and the research title. On the other hand, in Fig. 5 you can see each of the automatic graphs generated by the spreadsheet on the basic bibliometric information of the articles included in the systematic review. Figure 4: Pivot table report to consolidate information from the articles included in the sys- tematic review Italia 1 5 Sudáfrica 1 4 Siria 1 20% 3 España 1 Estados Unidos 1 80% 2 2 2 Perú 2 1 1 Rumania 2 China 6 2013 2015 2016 2017 2019 2020 2021 2022 0 1 2 3 4 5 6 7 Español Inglés Fig. 5.1 Articles published by year Fig. 5.2 Articles by country of publication Fig. 5.3 Articles by language Big Data 19 E3S Web of Conferences 2 1 Innovation Strategy 8 Journal of Innovation Management 1 Scopus 13 Company 1 7 Sustainability 1 Quality improvement 6 1 Wireless Communications and Mobile… 1 Competitiveness 4 1 DOAJ 4 Value of information 3 Ingeniería Industrial 1 1 Industria 4.0 3 Discrete Dynamics in Nature and Society 1 Sustainability 3 1 Dialnet 2 Quality & quantity 1 Information System 3 1 Revista de Administração… 1 Business Intelligence 2 1 Industria alimentaria 2 Indonesian Journal of Electrical… 1 Aerospace Database 1 1 Retail 1 Open Computer Science 1 Fig. 5.4 Keywords of included articles Fig. 5.5 Articles by Scientific Journals Fig. 5.6 Databases of included articles Figure 5: Graphical bibliometric reports according to the articles included in the systematic review 3.4. Entry of thematic or content information On the other hand, Table 3 is divided into 6 columns that correspond to: Authors, Research Title, Tool Type, Tool, Page and Description of the use of the tool, as shown in Fig. 6. Likewise, Table 2 was complemented by Table 1 in relation to the authors and research titles. This table has been designed with the purpose of identifying tools associated with the research variable. For example, if a review is done on Big Data, a list of tools for its application could be relevant. In addition, to ensure a correct tabulation, the type of tool that serves as a label to group the data must be considered. Also, the page number where the indicated information is located must be identified and a detailed description of it must be provided. In this way, an adequate traceability of the information is ensured. Figure 6: Table to collect information on the type of tools and the tools used according to the articles included in the systematic review In the same way, for the elaboration of Table 4, the same procedure is applied as in Table 3, but this time focusing on the findings or results of the study. A finding or result is considered as the effect obtained on the variable studied in a given population. In Fig. 7 it can be seen that the following columns are considered: Authors, Research Title, Type of Effect, Specific Effect, Page and Description of the effect. Like the previous tables, this one was complemented by Table 1. To ensure a correct tabulation, the type of effect that serves as a label to group the data must also be considered. Likewise, it is necessary to identify the page number where the indicated information is located and provide a detailed description of it; in this way, an adequate traceability of the information is ensured. Figure 7: Table to compile information on the type of effects and the specific effect identified according to the articles included in the systematic review 3.5. Content or thematic reports For each table, a dynamic table is obtained as a report. Therefore, all the information obtained and classified in Table 3 allows obtaining a dynamic table with the count of articles for each tool identified. Each type of tool that is considered works as a label that groups the data. If the label is unique, no division is presented, as can be seen in Fig. 8. In the same way, all the information obtained and classified in Table 4 allows obtaining a dynamic table with the count of articles by effect identified. Each label allows grouping the data in an organized way, as can be seen in Fig. 9. Figure 8: Pivot table report on the tools used according to the articles included in the systematic review Figure 9: Pivot table report on the findings/results in the articles included in the systematic review 3.6. Traceability of findings or results The designed tool allows the traceability of the data tabulated in the dynamic tables (Fig. 10). Therefore, at the time of citation, you can double-click on the count and a report is generated only of the studies included in that count. In the same way, a teacher or principal investigator can use that option to verify or correct the recorded information. Figure 10: Traceability report on findings/results identified in the articles included in the systematic review 4. Discussion There are various computing alternatives focused on systematic reviews. Health studies [18] tend to emphasize specific tools like Covidence and Rayyan, designed for efficient study selection. Conversely, tools in software engineering and social sciences [17][19][21] often emphasize text mining and online collaboration, reflecting these fields' unique needs. This implies that health prioritizes efficiency in managing large data volumes, whereas social sciences and engineering value flexibility and collaboration. Covidence and Rayyan are prominent in the health field, offering functionalities tailored to systematic reviews. Covidence simplifies study selection, title and abstract review, data extraction, and bias risk assessment. It also enables efficient teamwork [18]. Rayyan offers efficient reference management, advanced filtering in title and abstract review, and a blind review mode to minimize bias [18]. These capabilities are vital in health, where managing large data volumes with precision and efficiency is crucial. The 2015 [17] and 2014 [20] studies identified tools with functionalities like multiuser support and data extraction, valuable across disciplines. However, despite their effectiveness in systematic reviews, Covidence and Rayyan have limitations affecting their research applicability. Covidence's cost can be prohibitive for individual researchers or budget limited institutions. Its interface, while generally intuitive, may pose a learning curve due to multiple functionalities. Though Covidence allows data extraction customization, it may lack the flexibility of an Excel spreadsheet for projects with unique data requirements. Its integration with other databases or reference management tools can be limiting. Being web based, it also requires a constant internet connection, a potential issue in areas with limited connectivity. Rayyan, ideal for initial study selection, might lack tools for complex analyses. Its interface may seem less intuitive than others, potentially affecting initial efficiency. The support and documentation for Rayyan could be insufficient for new or less experienced users. Like Covidence, Rayyan's internet dependency could be problematic in areas with limited access. These limitations are crucial when selecting research tools, especially in resource limited contexts or for highly specific research needs. Choosing between Covidence, Rayyan, and tools like Excel depends on the project's specific needs, available resources, and user familiarity with these platforms. Another vital aspect is managing findings or results. Covidence focuses on study selection, title and abstract review, data extraction, and bias risk assessment. It allows efficient data organization from selected studies. However, its detailed table synthesis capability might be limited compared to a customized Excel spreadsheet. Covidence ensures data traceability, indicating the origin of each data point. Rayyan mainly aids in study selection and review. While it enables study tagging and filtering, its detailed data extraction and table synthesis functionalities are limited. Rayyan is useful for initial reference management and team collaboration but falls short in data synthesis depth compared to Excel. In this regard, Excel offers more flexibility and customization for detailed findings extraction, table grouping, information synthesis, and tracing each finding to its original study. Furthermore, while Covidence and Rayyan excel in data management and analysis for systematic reviews, they aren't specifically designed for bibliometric table or chart generation. These platforms focus on study selection, data extraction, and bias analysis, unlike the proposed spreadsheet, which manages information and generates bibliometric tables and figures. Collaboration is essential across disciplines. Tools like SLRTool [20] facilitate team collaboration in various research areas. However, Excel can also be effective for small teams or less complex social science and engineering projects. Systematic review tools in health, social sciences, and engineering differ in approaches and functionalities, reflecting each field's needs. While health emphasizes data volume management efficiency, social sciences and engineering prioritize flexibility and collaboration. Excel, adaptable and accessible, can be valuable in social sciences and engineering, especially for specific requirement projects or those with limited budgets. 5. Conclusions In conclusion, systematic reviews are an important tool for scientific research. This research model provides a systematic and objective method to synthesize the evidence from the scientific literature on a specific topic. Likewise, these must follow a set of established guidelines to ensure that they are reliable and reproducible. Therefore, the methods used must be transparent, so that readers can evaluate their validity. In addition, it should be noted that this helps researchers a lot, either to identify knowledge gaps or to evaluate the existing evidence. Therefore, the construction of tools to assist in systematic reviews is essential, either for the collection of information or for data analysis. This will facilitate the construction of the systematic review, as well as its structure and writing, being done in a more dynamic way and in less time than working conventionally. Therefore, an alternative as technological support for the elaboration of systematic reviews would be Microsoft Excel. This represents the opportunity to work orderly, separating the information in several stages: from the selection of the articles to the tools and effects that can be identified in the various studies. The power of Excel should be taken advantage of to work with a large amount of information and collaborate with other researchers, keeping the consolidation of the data in a single file. Likewise, Excel has powerful tools equivalent to complex databases, such as dynamic tables or graph generation, especially if you want to maintain a quantitative approach through data counting and bibliometrics of the selected studies. Finally, the proposed alternative does not replace any existing methodological procedure, but it does strengthen the data tabulation process in view of the scarcity of computer solutions that achieve the same objective. References [1] H. García-Perdomo, “Conceptos fundamentales de las revisiones sistemáticas/metaanálisis,” Urología Colombiana, 24 (1), 28-34, 2015. http://dx.doi.org/10.1016/j.uroco.2015.03.005 [2] M. Cué and J. Oramas, “Síntesis de información y artículos de revisión,” Revista Cubana de Información en Ciencias de la Salud, 17 (2), 1-12, 2007. http://scielo.sld.cu/scielo.php?pid=S1024- 94352008000200007&script=sci_arttext&tlng=en [3] B. Moreno, M. Muñoz, J. Cuellar, S. Domancic and J. Villanueva, “Revisiones Sistemáticas: definición y nociones básicas,” Revista clínica de periodoncia, implantología y rehabilitación oral, 11 (3), 184-186, 2018. http://dx.doi.org/10.4067/S0719-01072018000300184 [4] G. Urrutia and X. Bonfill, “Revisiones sistemáticas: una herramienta clave para la toma de decisiones clínicas y sanitarias,” Revista Española de Salud Pública, 88 (1), 1-3, 2014. https://dx.doi.org/10.4321/S1135-57272014000100001 [5] O. Beltrán, “Revisiones sistemáticas de la literatura,” Revista colombiana de gastroenterología, vol. 20 (1), 60-69, 2005. https://www.redalyc.org/pdf/3377/337729264009.pdf [6] I. Ferreira, G. Urrútia and P. Alonso-Coello, “Revisiones sistemáticas y metaanálisis: bases conceptuales e interpretación,” Revista Española de Cardiología, 64 (8), 688-696, 2011. https://doi.org/10.1016/j.recesp.2011.03.029 [7] N. Corp, J. Jordan, J. Hayden, E. Irvin, R. Parker, A. Smith and D. van der Windt, “Protocol: A systematic review of studies developing and/or evaluating search strategies to identify prognosis studies,” Systematic Reviews, 6 (1),1-5, 2017. https://doi.org/10.1186/s13643- 017-0482-y [8] T. Turner, […], C. Watts, “Producing Cochrane systematic reviews-a qualitative study of current approaches and opportunities for innovation and improvement,” Systematic Reviews, vol. 6 (1), 1-11, 2017. https://doi.org/10.1186/s13643-017-0542-3 [9] S. Briscoe, “Errors to avoid when searching for studies for systematic reviews: A guide for nurse researchers,” International Journal of Older People Nursing, 18 (3), 1-7, 2023. https://doi.org/10.1111/opn.12533 [10] C. Wiysonge, “The importance of systematic reviews in radiology,” South African Journal of Radiology, 18 (1), 1-2, 2014. http://dx.doi.org/10.4102/sajr.v18i1.609 [11] S. Freeman, H. Williams and R. Dellavalle, “The increasing importance of systematic reviews in clinical dermatology research and publication,” Journal of Investigative Dermatology, 126 (11), 2357-2360, 2006. https://doi.org/10.1038/sj.jid.5700560 [12] J. Sánchez and J. Botella, “Revisiones sistemáticas y metaanálisis: Herramientas para la práctica profesional,” Papeles del psicólogo, 31 (1), 7-17, 2010. https://www.redalyc.org/pdf/778/77812441002.pdf [13] B. Chrcanovic, “Importance of Systematic Reviews in Implant Dentistry,” International Journal of Prosthodontics and Restorative Dentistry, 9 (2), 36-37, 2019. https://www.ijoprd.com/doi/pdf/10.5005/jp-journals-10019-1234 [14] R. Hernández y C. Mendoza, Metodología de la investigación: Las rutas cuantitativa, cualitativa y mixta, 1era ed., Ciudad de México: McGraw Hill Interamericana Editores, 2018 [15] L. Mucha-Hospinal, R. Chamorro-Mejía, M. Oseda-Lazo and R. Alania-Contreras, “Evaluación de procedimientos empleados para determinar la población y muestra en trabajos de investigación de posgrado,” Revista Desafíos, 12 (1), 50-57, 2021. http://revistas.udh.edu.pe/index.php/udh/article/view/253e [16] O. Hernández, “Aproximación a los distintos tipos de muestreo no probabilístico que existen,” Revista Cubana de Medicina General Integral, 37 (3), 1-3, 2021. http://scielo.sld.cu/scielo.php?pid=S0864-21252021000300002&script=sci_arttext [17] Marshall, C., Brereton, P., & Kitchenham, B. (2015). Tools to support systematic reviews in software engineering: A cross-domain survey using semi-structured interviews. Proceedings of the 19th International Conference on Evaluation and Assessment in Software Engineering, Article No.: 26. https://doi.org/10.1145/2745802.2745827 [18] Marshall, I. J., Noel-Storr, A., Kuiper, J., Thomas, J., & Wallace, B. C. (2020). Software tools to support title and abstract screening for systematic reviews in healthcare: An evaluation. Medical Informatics and Decision Making, 20(1), 1-10. https://doi.org/10.1186/s12911- 020-1023-5 [19] Alkadhi, R., & Maalej, W. (2020). Social network sites and requirements engineering: A systematic literature review. Software: Practice and Experience, 50(7), 1255-1279. https://doi.org/10.1002/smr.2332 [20] Santos, R. M., & do Carmo, R. M. (2014). Slrtool: A Tool to Support Collaborative Systematic Literature Reviews. Proceedings of the 9th International Conference on Software Engineering and Applications, 440-447. https://doi.org/10.5220/0004972204400447 [21] Gren, L. (2018). On Psychometric Instruments in Software Engineering Research: An Ongoing Study. Proceedings of the 11th International Workshop on Cooperative and Human Aspects of Software Engineering, 48-51. https://doi.org/10.17771/pucrio.dimcc.60876