One of the indicators of the influence and competitiveness of university is participation in international and local rankings. In the methodologies of the most authoritative rankings, the quality and transparency of Universities is assessed by indicators of the university teachers' publishing activity. The article analyzes the experience of designing systems that analyses university teachers research activities, tools and components of evaluating the efectiveness of research, designed a structural and functional model of rating system for the analysis the university teachers' research activities taking into account research and digital competencies. The developed model provides performance of the basic functions and allows for systematic monitoring of openness, transparency, eficiency of the research component of university teachers' professional activity. The model was based on key indicators for evaluating the efectiveness of research - citation indicators of the three most important scientometric databases Scopus, Web of Science, Google Scholar. The connection between the indicators presented in the model forms a portrait of the university teachers' scientific activity, gives an overall assessment of productivity, influence and contribution to the research direction of the university as a whole. The article describes 1 stage of implementation of the presented model by developing a “Rating of Transparency of Structural Units”, the efectiveness of experimental ranking. The study has established the positive impact of the implementation of the rating system, identified the main activities to increase the visibility, presence, dissemination of research results, the systematic implementation of which contributes to the optimal representation of the scientist in the rating evaluation of the research component of the university teachers professional activities, improving the digital competence of teachers and positively afects the quality indicators of the university scientific work in local and international rankings, as the existence of the system and the formation of ratings is an incentive for university teachers to present and disseminate their own publishing activities in the international online scientific community.
The main priority of each university is to provide quality educational services, increase
competitiveness in the educational market and to attract the most promising students. Participation
in international and domestic rankings is an important tool for measuring the
competitiveness of Universities. In the methodologies of the most authoritative rankings, the quality and
transparency of Universities are assessed by indicators of publishing activity of researchers.
Indicators of the scientific citations’ quality according to Google Scholar Citations and citations
in influential scientific journals are taken into account in international rankings – “Academic
ranking of world universities” (ARWU), “QS Top University Ranking”, webometric ranking of
world universities “Webometrics”, “Transparent ranking: Top Universities by Google Scholar
Citations” and Ukrainian – “University ranking by Scopus indicators”, “Top-200 Ukraine”,
“Consolidated rating of higher education institutions of Ukraine”, “Bibliometrics of Ukrainian science”
[
The purpose of the study is to build a structural and functional model of a rating system for the analysis the research component of university teachers’ activities based on key indicators of research and digital competencies.
Delgado López-Cózar et al. [
Theoretical bases of rating assessment of teachers’ activity of the Ukrainian Universities
were studied by Dzoba [
Issues of assessing the quality of research results were studied by Kostenko et al. [11], Shynenko et al. [12], Gasparyan et al. [13].
Hohunskyi et al. [14] determined that the use of profiles by scientists in Google Scholar, ORCID, Mendeley, Academia, ResearchGate increases the visibility of publications in the information space and has a positive efect on increasing citation rates.
In [11] the information-analytical system “Bibliometrics of the Ukrainian Science”, developed by Social communication research center, which is the register of bibliometric profiles of researchers and research teams in the most authoritative scientometric databases, as Scopus, Web of Science and Google Scholar. The system allows to analyze the local scientific potential, to carry out a comparative analysis of the efectiveness of Ukrainian scientists in the fields of science, cities, afiliation, agencies, etc. by constructing rating tables on the value of h-indices in Scopus, Web of Science or Google Scholar databases [15]. Open profiles of a scientist in scientometric databases are a passport of a scientist in the European space of higher education, which reflects his authority and influence on the development of a particular field.
Spivakovsky et al. [16] describes the experience of designing an information system to analyze the research activities of university teachers, tested on the basis of Kherson State University and Kherson State Maritime Academy. The system implements tools for constructing teachers’ ratings by the main indicators of scientific citation in the databases Scopus, Web of Science, Google Scholar, Semantic Scholar, Tutor Network. Ratings are built in the context of structural units, departments and scientific journals. The rating of departments is based on the maximum value of citation profiles of all employees in the department in these databases [17].
The project Open Ukrainian Citation Index (OUCI), developed by the State Scientific and Technical Library of Ukraine, was studied by Nazarovets [18]. OUCI is a search engine and database of scientific citations that contains meta-data of all publications that use Cited-by service from Crossref and support the Initiative for Open Citations. The OUCI database contains metadata of scientific publications that receive DOI from Crossref and takes into account citations between publications. OUCI is designed to simplify the search and analysis of research results and expand the presentation and visibility of Ukrainian scientific publications in international search engines, such as Dimensions, Lens.org, 1findr, Scilit, etc [ 19]. The implemented analytical tool allows to assess the state and dynamics of the development of scientific potential in terms of fields of knowledge, years, to select the appropriate scientific journal for publication.
A common practice among universities is the implementation of rating systems to assess the professional activity of university teachers, as part of the system of quality assurance in education, one of the areas of which is research. Borys Grinchenko Kyiv University has introduced the “E-portfolio” system, which reflects the detailed picture of the teacher’s activity with certain quantitative and qualitative indicators of activity. The system allows not only to create an e-portfolio of the teacher, but also to form rating tables of indicators for assessing the main activities of each university teacher [20].
Borys Grinchenko Kyiv University has introduced the Corporate Standard of Research Activity of Employees and the Corporate Standard of Digital Competence of Teachers, which present and correlate the performance indicators of university teachers’ research activity and their corresponding indicators of the levels of formation of teachers’ digital competence (figure 1).
One of the indicators of university teacher’s compliance with the Corporate Standard of Research Activities of the staf of Borys Grinchenko Kyiv University [ 21] is the indicator of scientific recognition, which is determined by the values of citation indices in the scientometric databases Scopus, Web of Science, Google Scholar. The key to optimal presentation of research results in scientometric databases is a suficient level of digital competence, which lies in the ability to efectively use open digital systems in own research, create, update, supplement the articles of the scientist profile in scientometric databases, distribute research results in scientific • Actualization of scientist profiles in scien
tometric databases; • Dissemination of research results; • Systematic use of digital tools in conducting and disseminating research results. web-communities and social networks [22] (table 1).
Analyzing the experience of developing rating systems for the analysis the activities of university teachers, it was determined that the most common quantitative indicators of evaluation of research activities are publications in journals included in influential scientometric databases; values of citation indices in scientometric databases – Scopus, Web of Science, Google Scholar, etc. [20].
The most important scientometric databases that demonstrate the efectiveness of the university teachers’ research activity are Scopus, Web of Science and Google Scholar, in which the values of bibliographic citation are used as quantitative indicators of evaluation and analysis. At the same time, it is important for university teachers to comply with the Corporate Standard of Research Activity and the Corporate Standard of Digital Competence, as the level of research and digital competence directly afects the quality of research and therefore the university’s ranking in international and local rankings. That is why the basis of the model of the rating system for the analysis the research activity of the university teachers are these three scientometric databases.
The main principles in developing the methodology of the rating system for the analysis the research activity of the university teachers were objectivity, measurability and transparency. That is why as indicators of the efectiveness of university teachers’ research activities were selected indicators of citing scientific publications in open teachers’ profiles in Scopus, Web of Science and Google Scholar that is easy to get and check – the number of bibliographic references, the h-index and i10-index.
The rating system for the analysis the research activity of the university teachers performs the following functions: • informational – the formation of a single information base of indicators of research efectiveness of university teachers for quality assessment; • visualization – presentation of information in a visualized form to improve perception, reporting; • motivational – the formation of an atmosphere of healthy competition, activation of the scientific potential of the university; • competence – encouraging teachers to create, update, actualization information in their own profiles in scientometric databases; • analytical – analysis of the current state of the research component of the departments and the university as a whole, identification of factors influencing the efectiveness of teachers’ research activities, identification and analysis of relationships; • prognostic – forecasting of perspective trajectories of the university development, system measures development for increasing the indicators of eficiency of the university teachers’ research activity; • managerial – information basis for making managerial decisions.
The implementation of certain functions is possible in the presence of the following structural components of the rating system for the analysis the research activity of the university teachers.
The administrative component provides for the delimitation of access rights to the rating system – administrator, teacher, guest. Administrator rights include the start of rating formation, marking of incorrectly filled profiles, export of reports. The rating is available to university teachers and unauthorized users only for review and analysis.
Organizational component. For each lecturer of the Borys Grinchenko Kyiv University, a profile in the “E-portfolio” system is automatically generated, from which data on afiliation and scientific citation are obtained.
The technological component (figure 2) includes obtaining scientometric indicators from Scopus databases, Web of Science and the Google Scholar system and teachers’ personal data. The “E-portfolio” system of the Borys Grinchenko Kyiv University automatically synchronizes the values of bibliographic references, h-index and i10-index for the last 5 years from the bibliographic profile of the scientist in the Google Scholar system, a link to which the teacher can add to his portfolio.
Scientometric indicators from the Scopus and Web of Science profiles are entered the “Database of activity registers”, the data from which are automatically synchronized in the teachers’ profile in the “E-portfolio” system. The relationship between the systems is implemented through APIs. The values of scientometric indicators from the Scopus, Web of Science and Google Scholar databases available in the profile are obtained during the formation of the “Rating of Transparency of Structural Units”, which is one of the results of the designed model.
Efective component – based on the collected data on the afiliation and scientometric indicators of teachers are formed rating tables of research efectiveness of the university as a whole and in terms of departments and departments, based on which the ability to export reports.
Communication of all components is provided in the Borys Grinchenko Kyiv University Digital Campus.
Thus, the structural and functional model of the rating system system based on competencies for the analysis the research activity of the University teachers will have the following form (figure 3):
Based on the presented model, a rating was developed, which is one of the indicators of system performance. As a result of experimental implementation, a “Rating of Transparency of Structural Units” by citation indicators in Google Scholar was developed, which ranks teachers according to their indicators of publishing activity (figure 4).
The proposed model of the rating system was implemented as a web-oriented client-server architecture. PHP programming language using the MVC architectural model was used in the development of the server software. The web user interface is implemented using HTML, CSS and JavaScript.
In the context of chairs, a list of teachers is displayed, ranked in descending order of the value of the total number of bibliographic references according to the Google Scholar (figure 5). The detailed table of research performance indicators contains the values of the total number of bibliographic references, h-index, i10-index and the link to the profile in the Google Scholar.
Creating a database of scientific profiles of researchers in Google Scholar allowed to implement the designation of incorrectly configured profiles in one place.
For further analysis of rating positions and indicators of the university teachers’ research activities, construction of visualized reports on the efectiveness of research activities, it is planned to implement the 2nd stage of research – expanding the rating for other indicators presented in the model – the number of publications and citation indices in Scopus and Web of Science databases, establishing relationships to increase the university teachers’ publishing activity.
The research took place in several stages: preparatory, design, experimental (implementation).
At the preparatory level, the theoretical foundations of rating of evaluation the university teachers’ activity were investigated. At the design stage, a structural and functional model of the rating system for the analysis the research activity of the university teachers was designed, “Rating of Transparency of Structural Units” by citation indicators in Google Scholar was developed and a preliminary rating was built. For two months, the administration and university teachers had the opportunity to review and analyze the data of the previous rating, after which an experimental rating was built.
The analysis of preliminary and experimental ratings (figures 6, 7), built with an interval of 2 months, confirmed the positive dynamics of research performance. The availability of the system and the formation of ratings is an incentive for teachers to update their e-portfolio profiles, create profiles in scientometric databases, in the absence, update information in scientometric profiles, including Google Scholar, supplement indexed publications, dissemination of research results, their discussion in the international online scientific community, thus developing skills in using digital technologies in organizing research and disseminating results.
According to the results of the study, it was found that the optimal presentation of the results of university teachers’ research activities in ranking is facilitated by: • implementation of open access, transparency, openness of research results; • preference for publications, assigning articles a unique DOI identifier, which facilitates publication identification and provides a permanent publication link, regardless of changes in the publication web address; • creating profiles and periodically adding new scientific developments to scientometric and bibliometric databases such as Google Scholar, ResearchGate, Mendeley, and the like;
• use of ORCID scientist identifier to correctly identify the afiliation of the article to the author’s profile; • implementation of dissemination activities to increase visibility and presence in the international online scientific community.
Currently, work is underway to refine and expand the rating for other indicators of Scopus, Web of Science, presented in the model to obtain a full rating system; systems of analytics and statistics with the formation of dynamic visualization of the efectiveness of the scientific potential of the university.
For optimal presentation of the university’s activity in international and Ukrainian rankings, ensuring the competitiveness of the university, it is important to systematically monitor the openness, transparency of the efectiveness of the university teachers’ research activities. An efective tool for monitoring is the development and implementation of a rating system for research analysis.
The system is based on the three most important scientometric databases – Scopus, Web of Science, Google Scholar and provides the main functions – information, visualization, motivation, competence, analytical, prognostic and managerial. The rating system is implemented as a web-oriented client-server architecture. One of the components implemented in the system at the 1st stage of the study is “Rating of Transparency of Structural Units” by citation indicators in Google Scholar. According to the results of the study, recommendations were developed to improve the quality of research activities. The formation of the ranking has confirmed the importance of designing a system to analyze research activities, improve the quality of scientific research, disseminate research results in the international educational space, to analyze teachers’ own professional development, monitor the scientific growth of university teaching staf and obtain visualized analyses in real time. The prospects for further research is considered in the adding other research activity indicators to the ranking, realisation of real-time updates to the rating system, integration of modern business intelligence systems to build visualised statistics to analyse the efectiveness of the research component of the university’s activities.
The systematic implementation of measures to ensure open access to own research results, to increase the visibility, transparency and dissemination of the results of research activities in the Internet space contributes to the optimal representation of the scientist’s activities in the rating evaluation of the research component of the professional activities of university teachers, improving the professional competence of teachers and positively influencing on the indicators of quality of university’s scientific work in local and international rankings. National Technical University of Oil and Gas (Series: Economics and Management in the Oil and Gas Industry) 1(19) (2019) 84–95. URL: https://eung.nung.edu.ua/index.php/ecom/ article/view/229. doi:10.31471/2409-0948-2019-1(19)-84-95. [9] I. Reheilo, Theoretical bases of estimation and development of scientific and pedagogical workers of universities in the context of realization of scientific activity, in: Y. Skyba (Ed.), Theoretical bases of development of scientific and pedagogical potential of universities of Ukraine in the context of expansion of institutional autonomy: preprint (analytical materials) (Part I), IVO NAPS of Ukraine, Kyiv, 2020, pp. 72–103. doi:10.31874/978-617-7486-40-3-2020. [10] O. Yaroshenko, Theoretical bases of rating as a tool for evaluating the activities of scientific and pedagogical staf of ukrainian universities, in: Y. Skyba (Ed.), Theoretical bases of development of scientific and pedagogical potential of universities of Ukraine in the context of expansion of institutional autonomy: preprint (analytical materials) (Part I), IVO NAPS of Ukraine, Kyiv, 2020, pp. 57—-71. doi:10.31874/978-617-7486-40-3-2020. [11] L. Kostenko, T. Symonenko, O. Zhabin, Project "Bibliometrics of Ukrainian science": idea, realization, plans, Bulletin of the Book Chamber 5 (2019) 30–33. [12] M. A. Shynenko, A. V. Kilchenko, S. M. Tukalo, Application of scientific metric indicators to evaluate the performance of scientific and pedagogical research, in: Reporting scientificpractical conference of the Institute of Information technologies and learning tools of the National Academy of Pedagogical Sciences of Ukraine, Kyiv, Ukraine, 2021, pp. 89–93. [13] A. Y. Gasparyan, M. Yessirkepov, A. Duisenova, V. I. Trukhachev, E. I. Kostyukova, G. D.
Kitas, Researcher and author impact metrics: Variety, value, and context, Journal of Korean Medical Science 33 (2018) e139. doi:10.3346/jkms.2018.33.e139. [14] V. D. Hohunskyi, V. O. Yakovenko, T. O. Liashchenko, T. V. Otradska, General mechanisms of citation system of scientific articles, Bulletin of NTU "KhPI 1173 (2016) 14–18. URL: http://dspace.opu.ua/jspui/handle/123456789/1774. [15] Bibliometrics of Ukrainian science, 2022. URL: http://nbuviap.gov.ua/bpnu/. [16] A. Spivakovsky, M. Vinnyk, M. Poltoratskiy, Y. Tarasich, Y. Spivakovska, G. Gardner, K. Panova, Information system of scientific activity indicators of scientific organizations: Development status and prospects, CEUR Workshop Proceedings 2393 (2019) 220–228.
URL: http://ceur-ws.org/Vol-2393/paper_260.pdf. [17] Rating of departments of Kherson State University, 2021. URL: http://publication.kspu.
edu/rating/departments/. [18] S. Nazarovets, Open Ukrainian Citation Index (OUCI) project: idea, working principle and development prospects, Intellectual Property in Ukraine 3 (2019) 10–13. URL: http: //eprints.rclis.org/34365/. [19] Open Ukrainian Citation Index, 2022. URL: https://ouci.dntb.gov.ua/. [20] N. Morze, O. Buinytska, Rating indicators system of evaluation teacher’s activities in modern universities, Scientific journal of National Pedagogical Dragomanov University 26 (2017) 34–44. URL: http://enpuir.npu.edu.ua/handle/123456789/19082. [21] Corporate Standard of Research Activity of the staf of Borys Grinchenko Kyiv University, 2018. URL: https://kubg.edu.ua/images/stories/Departaments/vdd/documenty/rozdil_8/ nakaz_212_02.04.2018.pdf. [22] Corporate Standard of Digital Competence of Teachers of Borys Grinchenko Kyiv University, 2021. URL: https://kubg.edu.ua/images/stories/Departaments/vdd/nakaz__421_11.06. 21.pdf. [23] V. Smirnova, Research on open digital information systems for analysis of research activity results of higher education institutions academic staf, Open educational e-environment of modern university 9 (2020) 134–144. doi:10.28925/2414-0325.2020.9.11. [24] L. Luparenko, The use of electronic open journal systems in scientific and pedagogic research: Results of experiment, CEUR Workshop Proceedings 2732 (2020) 1113–1128.
URL: http://ceur-ws.org/Vol-2732/20201113.pdf. [25] A. Martín-Martín, E. Orduna-Malea, E. Delgado López-Cózar, Coverage of highly-cited documents in Google Scholar, Web of Science, and Scopus: a multidisciplinary comparison, Scientometrics 116 (2018) 2175–2188. doi:10.1007/s11192-018-2820-9.