Nowadays ICT are one of main ways to arrange and create effective tools for organizing the interaction and processing large amount of information. In our opinion, information of university's scientific activity should be presented in the rating form, which gives an opportunity to analyze development in different directions and changes. The key idea of the article is presenting of our experience in developing rating system for universities based on scientist's scientometric indices. The system provides open data from different scientometric systems, such as Google Scholar, Scopus, Web of Science and Semantic Scholar. The possibility of automatical construction of a rating of scientists, research groups and organizations (as well as their structural subdivisions) is described.
The criteria for evaluating the effectiveness of fundamental scientific research can be divided into qualitative and quantitative indicators. Today in Ukraine, in the process of evaluating of the effectiveness of science, scientometric indicators are being used increasingly both to evaluate the work of an individual scientist, research teams, organizations and scientific publications. For example, it is necessary to indicate the number of publications and the number of citations over the past five years, as well as the Hirsch index in the tender documentation for submitting applications for grants for research activities, for the passing a competition for teaching positions in an educational institution, for drawing up reports of the scientific activities of departments and faculties, for reports on the work of graduate students, for passing accreditation of specialties, departments, etc.
And this information must be described for each database, such as Web of Science, Scopus, Google Scholar.
Scientometric indicators are also used to assess the quality of scientific journals. In this case, the indicators of the editorial board, and, in fact, published publications and issues, their openness and accessibility to the scientific community are taken into account.
Each of the indices has its own criteria for the selection of scientific publications and the inclusion of publications in the database, its specific calculation of scientometric parameters. If the rules for selecting journals in WoS and Scopus indices are now the most stringent, then Google Scholar’s system, on the contrary, is the most extensive system, that collecting all scientific publications on the Internet, including open access repositories, personal pages of scientists and university electronic libraries. And if we take into account and analyze many other scientometric systems, we get a whole set of various indicators and only after their general analysis we'll can to give a correct assessment of scientific activity. Therefore, the problem of building consolidated ratings is remains relevant.
As shows a practice, the collection and analysis of this kind of information is a very hard and lengthy process that needs automation and optimization. Previously, the authors of the article had already considered the requirements for the system of scientific activity indicators of scientific organizations, and also described the basic functionality of the first version of such a system.
This article describes the new features of the system, its current state and the results of its use in higher education institutions. 1
A big number of the recently created scientometric services allow assessing the relevance of the research results by a scientist, the number of his publications, citations, etc. The most outstanding services with rapidly growing impact are Google Scholar, Scopus, Orcid, Academia.edu, Research Gate, Mendeley, arXiv.org, cs2n, Epernicus, Myexperiment, Network.nature, Sciencecommunity.
The main interesting information systems that run on the activities of scientists, scientific groups, publishers, etc..., are:
Bibliometrics of Ukrainian Science [
The system “Bibliometrics of Ukrainian Science” is representation of information of Ukraine scientists’ profiles who provided information about their publication in the Internet; national component of the project Ranking of Scientists (Cybermetrics Lab).
Scopus. Scopus system is designed to maintain efficient workflow of researchers,
helping them to: find new articles from the area of their specialization; find
information about the author; analyze the publication activity in the subject area; track
citation; view the h-index; identify the most cited articles and authors; assess the
relevance of the study [
Google Scholar. Google Scholar is freely accessible search system, which indexed
the full text of the scientific publications all formats and discipline [
Web of Science. Web of Science – International established database of Scientific
Citation, it is presented by company Thomson Reuters. In addition to search of
scientific publications, Web of Science establishes a reference link between the specific
research using the cited materials and thematic links between articles established
reputable researchers working in this field [
But all of them can’t to resolve such problems, as ─ the absence of a clear mechanism of evaluation of personal contribution of the scientist in the scientific work of his organization, ─ the incomprehension of the construction of university decisions related to scientometric.
In addition, today, the analysis of the scientific indicators of scientists’ group, organization and its department, is carried out manually. The only option of its partial automation is rating the organization's profile in Google Scholar.
In addition, this article is a continuation of the previous works of the authors [
The main task of building our system is the realization of the possibility of automatic processing of scientometric and bibliometric indicators of scientific organizations on the basis of analysis of scientific profiles of scientometric databases and systems, including automatic search and its analysis.
The high-level system architecture is shown in Fig.1.
As you can see in Fig.1, the main components of the system are: Parser, Module of analytics, JSONE module, system Web-site, databases.
Parser is using for search, receiving and transfer the open information of scientometric indicators of authors and journals provided by Scopus, Web of Science, Google Scholar, Semantic Scholar and Tutor Network. For interaction with Scopus and Google Scholar parser uses xpath queries, and API is used for getting information from Web of Science and Tutor Network.
All data received by parser is stored in the system database. DB of system is distributed by the data storage. Individual entities of DB are database of scientometric indicators of researcher and scientific publications.
Information processing is realized by performing a set of predefined SQL queries.
Module of analytics (based in R language) provides an opportunity to get diagrams present data showing relation between different scientometric indicators, as the value of h-index in Scopus and the number of papers in Scopus or Google Scholar [11].
Tools and technologies thus were used for developing of the system were described
more detail in previous articles [
In the current version of the system we determined next forms of presentation of the results of analyzes of scientist’s activities indicators (Fig.2): ─ profiles of the scientists of the university with generalized information of scientometric indicators for each database, table of scientists articles and their availability in the scientometric systems; ─ the rating list of all the scientists of organization; ─ the rating list of all the scientists of organization’s structural subdivision (faculty or department); ─ the rating list of all scientific journals of organization; ─ the general scientometric information about the university.
It is possible to display ranking lists of scientists according to data from such systems as Scopus, Web of Science, Google Scholar, Semantic Scholar, and also sorted data by the increasing (descending) of such indicators as – number of publications, citation, h-index.
It should be noted the presence of color indicators of publication activity (displayed as a line of a certain color near the name of the scientist): ─ blue ― the number of documents is over 10; ─ green ― the number of documents is in the range of 5 to 10 (inclusive); ─ yellow ― the number of documents is in the range from 1 to 4 (inclusive); ─ red ― no documents aren’t included in such database. Ratings of scientific collectives, organisations and their departments (Fig.4)
Also, like as in the previous version, in this version of the system is the opportunities to present data by such categories as students, assistants, teachers, professors, and etc. (Fig.5).
Personal page of the scientist provides the following information: ─ author’s general information (full name, name of the institution, faculty and department); ─ information of author’s scientometric indicators for Scopus, Google Scholar, Web of Science and Semantic Scholar (for each separately), such as numbers of articles, citation, h-index; ─ special diagrams that represents the dynamics of the growth of scientist’s number of documents, h-index and citation index for each of the scientometric systems (Fig.6); ─ the list of scientist’s articles with marks that show their availability in the scientometric systems (Fig.7). ─ the diagram of scientists coauthors (Fig.8)
As you can see in Fig.8, the line that connects the scientist with his coauthor has a certain color:
Blue – if scientists have only one general article, Green – if the number of such articles are from two to five, Violet – if more than 5 publications was written in co-authorship.
The number of general publications is indicated in brackets after the co-author's name.Also we can build the semantic network of partners of organization (Fig. 9):
Today we have developed and implemented new version of information system of scientific activity indicators of scientific organizations. The main capabilities of it are: ─ automatic work with scientometric databases (algorithm of automatically search for links to profiles of Ukrainian scientists, algorithm of automatic distribution profiles of scientists on the name of the organization in which they work); ─ automatic analyze of getting information; ─ automatic generation of ratings of scientific organizations, their departments, scientists, scientific journals, etc; ─ ability to send messages to e-mail scientists about changes of academic indexes.