New approaches to formalization and determination of quantitative indicators characterizing some aspects of scientometric data are proposed. The concepts of a scientific action (наукового заходу) and a scientific event (наукової події) are introduced. An overview of the main indicators of a scientific event is offered. The mathematical model of the scientific event and the formalization of its main components are presented. Peculiarities of the policy of conducting a scientific action are studied. Special attention is paid to indicators of geographical representation, as one of the most important characteristics that represent the level of internationality of a scientific event in quantitative terms. The results of a computational experiment conducted on real data taken from the CEUR-WS open access database are presented. It is proposed to consider the indicators of the geographical representation of the members of the Program Committee and the geographical diversity of the affiliation of the authors of the materials of the scientific event as indicators of its internationality. Thus, a method of determining quantitative indicators of the level of internationality of a scientific event was proposed and the effectiveness of this method was tested on real data. Scientometrics, scientific space, policy of conducting a scientific action, scientific event,
geographical representation, heuristics
Scientific activity in the modern world is directed and stimulated by various organizational methods and is formalized by applying criteria that are generally accepted in the scientific environment. In many cases, approaches to formalization are too schematic and set only the contours of requirements that can be used later for classifications, comparisons, and determination of the significance of events that occur in the scientific space.
The
most popular international scientometric systems are Scopus, Web of Science (WoS), ResearchGate databases. At the same time, the most authoritative international scientometric databases for a considerable period of time are Scopus and Web of Science. For the correct and constructive use of information from these databases, new approaches should be developed, and researchers should focus on further formalization and analysis of scientometric data. Therefore, publication placement in scientometric databases that accumulate publications of a high level of material submission and significant scientific achievements, in particular, Web of Science, Scopus, Google Scholar, etc., are often a quantitative benchmark of the scientific significance of publications.
The scientific space is characterized by the complexity of measuring results, a wide range of approaches to comparing the activities of scientists and scientific teams [1, 2]. In this regard,
2023 Copyright for this paper by its authors. generally accepted agreements regarding scientometric bases, metrication of scientific activity, criteria of publication activity, citations of scientific articles, indices of influence, quantitative indicators of productivity, impact factors of periodicals, etc. are being introduced. All these innovations are due to the fact that determining the intellectual and scientific level of any scientific event in modern large-scale information flows is an important and urgent scientific problem [3, 4].
Evaluation of the scientific activity of scientists and scientific teams is one of the most urgent problems, which has been developing since ancient times in parallel with the development of scientific research.
Information technology is characterized by the creation of new knowledge based on the analysis and generalization of existing data. This is especially relevant for weakly structured and unstructured subject areas. That is, the result of the application of information technology is the formalization of weakly structured information and its transformation into structured information. In addition, it is also important to visualize the various stages of decision-making.
In the modern information society, one of the most important forms of activity is the production of scientific information. Therefore, the development of substantiated scientific approaches to the definition and application of criteria for the quality of scientific activity is an urgent problem today. In addition, in modern conditions, the problem of quantitative assessment of the results of scientific activity with the help of scientometric indicators has become especially relevant, due to the fact that such indicators, together with a wide range of expert opinions, are used at all stages of the science management process.
It should be noted that it is important not only to fix, but also to use quantitative scientometric criteria for evaluating the effectiveness of scientific work when conducting various hackathons, competitions, large-scale international actions, etc.
2.2.
The development of scientific information is an important indicator of the level of development of a modern state in the global information society. The contribution of each nation to the world's information resources increasingly determines the place of each nation and state in modern civilization. Scientific activity is an indirect, but often a direct indicator of the progressive development of the state.
Scientific space is a network of cognitive processes within which science functions [8]. It also includes scientific infrastructure: scientific institutions, scientific communications, processes of training scientists for professional activities, etc. [1, 9]. A scientific event is a component of the scientific space, which, in turn, is a complex, weakly structured system, which should be investigated using the methods of decision-making theory [10].
2.3.
The quality of scientific activity is an important criterion for the success of countries in the modern world [11]. In the scientific and management circles of various countries of the world, the importance attached to the development of criteria for evaluating scientific activity is regularly growing. This aspect becomes especially important in the conditions of informatization and global competitive challenges in all spheres of social activity. Today, the world scientific community has considerable experience in developing methods for evaluating the effectiveness of scientific activity [12]. Modern scientometric approaches are successfully used for effective development of domestic science. At the same time, additional scientometric indicators should be created for some areas of scientific activity [1, 12].
Globalization of society, expansion of international cooperation in various areas of scientific activity, adequate assessment of the effectiveness of scientists requires the establishment of uniform rules for calculating the impact of scientific research on the growth of scientific knowledge [13, 14]. This will also contribute to the motivation of effective scientific research in the field of innovation and increase the competitiveness of scientific activity.
At the same time, in modern scientific research, the trends of defining target criteria related to the satisfaction of specific interests of innovative development are increasing [15, 16]. When determining the effectiveness of scientific activity and making a decision about the level of scientific results, various indexes and ratings that reasonably determine the level of collective scientific results in quantitative measurement scales can be an effective evaluation tool.
2.4.
The high quality of conducting scientific actions (SA), which ensure a high level of reliability of
scientific information, is a necessary condition for the development of modern science and advanced
technologies [1, 17]. Adequate evaluation, especially quantitative evaluation of any scientific event (SE)
is also important and promising. This will contribute to the possibility of comparing scientific events
with each other, comparing the level or significance of periodic SE in different time periods [
All this requires the introduction of a mathematical apparatus for an adequate comparison of SE in
the scientific space of Ukraine. This is due, in particular, to the fact that the current stage of the
development of human civilization is defined as the transition to a knowledge society and is
characterized by qualitatively new requirements for the development of science [
For the publication of SA materials and their indexing in Scopus and/or WoS, the fulfillment of a
number of formal requirements is an important, and in many cases necessary, condition. One aspect of
SA formalization is the clear and unambiguous representation of SA policy. The policy of organizing
and conducting SA is characterized by many important directions [
This paper reviews and develops some approaches to quantitatively express the last two characteristics of SA. Mathematical models will be built, additional heuristics will be introduced.
A necessary condition for the inclusion of materials submitted for participation in the SA is to ensure the necessary condition of scientific integrity and high quality of the SA. For transformation of SA into SE, an additional study of the "sufficiency" of SE in the context of the interests of the scientific institution should be conducted.
Today, there exist a great variety of SA. There are scientific congress, symposium, scientific
seminar, forum, scientific or scientific-practical conference, round table, school, interdisciplinary or
international seminar, etc. [
Similar to the criteria applied to scientific journals that can be indexed in recognized international scientometric databases, we will propose criteria for editorial evaluation of the SE level.
It is logical to highlight the mandatory conditions for determining the quality of a scientific event, i.e. the necessary conditions without which the scientific action cannot be considered successfully conducted and thus turn into a scientific event.
Let's list the main necessary conditions, that is, a priori requirements, without which there is no chance to consider the international SA as having been successfully carried out, and thus to turn it into an international SE.
First, it is the presence of a website. The website must be created and maintained at a professional level. The site should present accessible details about SA, access to full-text SE materials, etc.
Secondly, an important condition for conducting SA is the functionality of the website. That is, the accuracy and reliability of information on the website. The navigation system should provide easy access to content, detailed information about the editorial board, instructions for authors, review procedures, etc.
Thirdly, the existence of a review policy is a necessary condition for conducting SA. That is, an easily accessible, transparent review policy and editorial control of all published content should be developed and implemented. Research materials intended to be published as SE results must be peerreviewed by independent external experts.
events, and the set of indices of these SE is denoted by I 1,..., n:
Let it is given some finite set A of events that should be compared with each other or determine their integral performance (efficiency, importance, significance, quality, etc.). This set consists of n ai A, i I.
As noted in [1], the problem of determining quantitative values of SE indicators that characterize the quality of SE implementation can be divided into several stages. It should be noted that the absolute necessity of passing the stages that transform SA into SE significantly affects the level of integral quality of SE, i.e. the necessary conditions for conducting SA are ensured. The main stages of structuring the problem of determining quantitative indicators that characterize SE are: • a priori determination of the policy of organization and conduct of SE, as well as approval of the policy of review of submitted materials;
• making a decision by the Program Committee regarding the preparation and publication of SE materials, which are of the highest quality among all submitted materials;
• preparation for making a decision on acceptance of a specific article for publication in the collection of SE materials;
• determination of quantitative SE performance indicators: geographical representation of the Program Committee and geographical diversity of SE participants.
Heuristics H1. We will assume that the level of SE internationality of type (1) can be quantitatively measured based on the analysis of the geographical representation of the members of the Program Committee and the geographical diversity of the authors of the materials submitted for publication.
To prepare a mathematical model for determining the level of SE internationality, we will consider a set of k SE indicators ij , j J 1,..., k, i I . (1) (2)
Indicators of the form (2) can be used to determine the quantitative indicators of SE performance from the set (1) and applied to compare the performance of different SE.
3.1.
Approaches to determining the quality of SE can be similar to determining the impact factor of scientific journals. But among SE in the form of conferences, symposia, etc., there is significantly more diversity. In particular, they are not regular and periodic: usually one-time or annual.
At the same time, we note that the publication of the next issue of the scientific journal is also SE. And to this SE, quality assessment approaches can be proposed in this work. The prerequisit e for using the described approach is to provide preliminary text recognition, that will allow filling the database with data necessary for analysis, in particular: • surnames of authors of articles; • affiliation of authors; • search for authors affiliated with the organization among cited literature; • determination of indexing in Scopus and/or WoS of references presented in the article, etc.
Today, there are at least three different approaches to evaluating the effectiveness of scientific activity. Tghere are expert approach, statistical approach and combined approach that based on the first two.
The basis of the expert approach is the subjective perception by experts in a specific field of science of the quality of work and it has significant disadvantages such as the influence of the human factor, large labor costs, significant time costs, and high cost.
The statistical approach is based on numerical evaluations of indicators that characterize the results of SE and represent various aspects of scientific activity in quantitative terms. Such evaluations are resault of analysis of certain aspects of the researcher's or institution's work.
The combined approach uses the best features of both of the above approaches and can be successfully applied to solve the problem of quantitative assessment of the quality and effectiveness of SE. 3.2.
When studying the quality of SE, various methods of formalization and calculation of quantitative indicators of SE can be successfully applied: • SE ranking in the collective determination of SE quality; • clustering of SA participants, authors of materials, members of the Program Committee, countries with which SE participants are affiliated, etc.; • rating of articles submitted for publication; • calculation of weight coefficients of SE indicators, submitted materials, etc.; • construction of membership functions for determining fuzzy indicators of the quality of materials submitted for publication.
When using expert technologies, an important role is assigned to the development and substantiation of models and computational methods for solving decision-making problems, problems accompanying the processes of creating the appropriate algorithmic support. Special attention is paid to ways and methods of obtaining, processing, analyzing and interpreting expert information. In addition, the analysis of the effectiveness of models and methods used in decision-making processes and based on "soft" computing paradigms is carried out.
In such situations, the basis for decision-making processes are expert technologies, as a set of models, methods and tools that, with the help of certain transformations, make it possible to achieve the desired result. Since a significant number of problems that require expert intervention in the process of solving are difficult to formalize and weakly structured, in such problems "soft computing" technologies are reasonably used to solve the problem of clustering and restore data gaps.
A generally accepted methodology used in the creation and research of complex socio-economic and
technical systems is system analysis [
Decision-making problems are accompanied by two defining aspects: the need to take into account subjective influences and the use of mathematical formalisms in their formulation and solution. For mathematicians, solving a problem begins with building a model of the phenomenon being studied. Psychological research is limited to experiments and the isolated application of certain statisti cal methods. Persons responsible for decision-making justifiably respond to such disparate approaches when modeling practical situations with distrust of "isolated" scientific developments and use a spontaneous-volitional approach.
The processes of obtaining, formalizing and transmitting expert information, in addition to objective difficulties arising from the researched scientific direction itself, are also accompanied by difficulties of a subjective nature that arise during the organization of an expert survey, coordination of the actions of a team of experts, mutual relations of a group of researchers, coordination of knowledge obtained from various experts, etc.
Today, natural language processing technologies are a powerful tool that can be successfully applied to solving the problem that is the purpose of writing this work namely determining the quality of SE in quantitative terms. Textual data provides a wide range of possibilities for analysis, especially in the earlier stages of development, namely: • determination of the main features of the submitted materials; • automatic determination of authors' affiliation; • automatic determination of the countries with which members of the Program Committee are affiliated;
• determination of other indicators that can be taken into account when determining the geographical diversity of SE.
Building a mathematical model of SE will allow automating a wide range of problems and can be used for: • comparing different SE with each other; • ranking of the set of SE in order to identify the quality indicators of these events; • determining the dynamics of increasing the representation of scientific events; • for the prospective development of scientific institutions; • attracting new regions and popularizing the scientific action; • additional opportunities that contribute to international recognition.
The main indicators of SE, which quantitatively characterize the international aspect of the SE, are: i the number of foreigners represented in the Program Committee of SE and the geographical 11 representation of the SE, which are measured using relevant heuristics;
1i2 to detail the previous indicator, in turn, the geographical representation of the SE can be objectified by dividing the Program Committee members according to their belonging to different groups of countries, for example, post-Soviet countries, Eastern Europe, Western Europe, Canada, USA, Japan, China, etc.; 1i3 indicators of the DBLP indices of the members of the Program Committee;
i indicators of DBLP indices of authors or co-authors of materials of SA submitted for 14 publication; 1i5 the level of plagiarism of submitted materials; 1i6 generalized assessment of other significant indicators.
Definition 1. The scientific community or country whose representatives predominate in the preparation of the collection of materials, both among the members of the Program Committee of SA and among the authors of the submitted materials, will be called the title.
Note 1. For some SA, the title scientific community may not be the only one. That is, a characteristic feature of such SA is the presence of representatives of two or more countries among the organizers and/or participants of the SA.
Note 2. For some SA, there may not be a title scientific community unless a pronounced majority is found among participants or organizers.
Let us denote by T the set of title scientific communities in a specific SA, T 1,...,t 0 , t 0 the number of such scientific communities.
Heuristics H2. We will consider that the title scientific community does not exist, if representatives of any country do not represent more than 25% of the SA organizers and/or participants. In this case, the set of title scientific communities in a particular SA is empty: T .
To refine the H2 heuristic, a coefficient of geographical diversity of the provided materials can be introduced, such as the coefficient of dispersion of shares in stock market research. This aspect of SA analysis needs additional development in order to study the composition of co-authors, clustering of articles by foreign authors, etc. countries; different countries.
Additional features that allow to determine the title host country of SA: 1i7 venue of the SA; 1i8 the largest number of SA participants among representatives of all countries; 1i9 the largest number of representatives in the Program Committee among representatives of all 1iа the largest number of articles submitted for publication among authors affiliated with 4.1.
It should be noted that ensuring the necessary conditions for conducting SA should be comprehensive, all details and tasks should be provided in order to prevent a situation that could hinder success. In particular, abuse or neglect of publication ethics is unacceptable and is certainly also a necessary condition for the transformation of SA into SE.
Necessary conditions for determining the impact (significance, importance, influence, authority) of SE can be considered, for example, standard and directive advices and requirements of CEUR. In addition, the fulfillment of the necessary conditions for conducting SA involves the fulfillment of standard conditions in the form of requirements for the members of the Program Committee and for the authors of the submitted materials:
When determining the indicators of the participants of the SA, the Program Committee, the
Organization Committee, we should take into account, in particular, their representation in the DBLP
database - the computer science bibliography site at the University of Trier in Germany [
The main parameters of a scientific action, the definition of which can be easily automated, and through which the integral quality of a scientific event can be reasonably calculated are: i the number of foreigners among the authors; 21 2i2 the number of countries represented by the authors; 2i3 the average value of the authors' DBLP index; 2i4 the maximum value of the authors' DBLP index.
The number of foreign authors is an ambiguous indicator. In this case there is a problem of determining how to understand the concept of foreign authorship. It is also advisable to analyze the differentiation of countries and regions, to take into account individual authorship of articles or cooperation between representatives of different countries, etc.
Important features of SA are those that take into account different aspects and features of SA: 3i 1 scientific significance of SA materials; 3i 2 authority of SA: citations of SA members in Scopus and/or WoS, authority of the editorial board and members of the Program Committee of SA, number of publications in the DBLP database, etc.; 3i 3 the number of years during which the SA is organized and transforms to a significant SE; 3i 4 languages of publication of SA materials that contribute to its international recognition; 3i 5 the quality of the site of SA, the quality of the design of SA materials; 3i 6 generalized assessment of other indicators important for international recognition. Additional features, the analysis of which can affect the quality factor of conducting SA: 3i 7 authors' Hirsch indices; 3i 8 the authors' availability of papers in journals that are assigned to quartiles Q1-Q2; 3i 9 the quality of the English language of the submitted materials; 3i а citations of domestic authors and self-citations (should not exceed 20%). 4.2.
The policy of conducting a scientific action can be structured or weakly structured. In particular,
one of the important element of the policy of conducting a scientific action is the drop-out rate of
articles submitted for consideration [
The policy of conducting SA is determined by a significant number of features: 4i1 the value of the expert indicator of the quality of the editorial policy; 4i2 expert assessment of the value of the review policy indicator; 4i3 geographical representation of the Program Committee; 4i4 geographic diversity of SA participants; 4i5 the history and regularity of the SA.
At the same time, SE organizers can be guided by different a priori settings. In particular, the Program Committee may accept a policy of rejection of papers based on some heuristics.
Heuristics H3. The basis for making a decision on the acceptance or rejection of submitted materials for participation in the SA may be an expert determination of the a priori rejection percentage of the materials submitted for the SA: 40, 45, 50, etc.
According to the policy of each individual scientometric database or other resource, the fulfillment of a priori constraints is a necessary condition for transformation SA into SE. Numerical values of some indicators can be as reliable indicators of the scientific level of SA in the complex.
Heuristics H4. To improve the quality of the materials selected for publication, different levels of review criteria can be applied and a structured review of submitted materials can be provided.
Based on the analysis of the SA history, numerical estimates can be obtained. For this, some more heuristics should be introduced.
Heuristics H5. The influence of the history of conducting SA is determined by the formula Z1 n1 1, (3) where n1 the number of years during which the SA has been organized and conducted, 1 some empirical value that reflects the importance of conducting this SA.
A more important event is not only the fact of conducting SA, but also the provision of indexing of published materials of SA by international scientometric databases.
Heuristics H6. The influence of the history of publication and indexing in the international scientometric databases Scopus and WoS SA is determined by the formula
Z2 n2 2 , (4) where n2 the number of years during which the indexing of the published materials of the SE is carried out by international scientometric databases, n2 some empirical value reflecting the importance of indexing the published materials of this SE.
It is obvious that for the indicators involved in formulas (3) and (4), the ratios are valid 1 2.
Heuristics H7. The history of conducting SA, publication of SE materials and their indexing by international databases Scopus and WoS can be described by the formula
Z3 1Z1, Z2 , (5)
where Z1, Z2 the values of variables calculated according to formulas (3)-(4), and 1 some
analytical functional dependence introduced empirically [
When modeling the quality of conducting SA [
The above parameters can be formalized, for example, by assigning points, by classification, by applying a membership function, etc.
Let us introduce some basic concepts related to this aspect of formalization.
To determine the quality of the selection of materials for publication based on the results of SA, we will introduce several notations: 6i 1 the total number of scientific materials submitted for participation in i th SA, i I ; 6i 2 the total number of authors who submitted their scientific works for participation in i th SA, i I ; i I ; participation in i th SA, i I ; 6i 3 the total number of pages of scientific works submitted by the authors of scientific works for 6i 4 the number of scientific works accepted for publication in the materials of the i th SA, 6i 5 the number of authors whose scientific works have been accepted for publication in the materials of the i th SA, i I ;
6i 6 the number of pages of scientific works that are accepted for publication in the materials of i th SA, i I .
Based on the entered indicators, we formulate some heuristics.
Heuristics H8. The quality of the selection of materials for publication based on SA results is defined as the ratio of scientific works accepted for publication in SA materials to the total number of scientific works sent for participation in SA:
Z 4i 6i 4 / 6i 1, i I. (6)
But the selection quality indicator can be determined not only by formula (6) taking into account the H8 heuristic. It is logical to consider other approaches to determining the selectivity indicator, that is, the quantitative characteristic of the quality of material selection.
Heuristics H9. The quality of the selection of materials for publication based on the results of the SA is defined as the ratio of the number of authors whose scientific works were accepted for publication in the materials of the i th SA, i I , to the total number of authors who submitted their works for participation in this SA:
Z 5i 6i 5 / 6i 2 , i I.
It is also possible to propose an exotic measure of selectivity, which only indirectly indicates about the quality of the selection of materials, but at the same time has the right to exist and in some special situations can be applied to determine the level of selectivity of SA. (7)
Heuristics H10. The quality of the selection of materials for publication based on SA results is defined as the ratio of the number of pages in scientific works accepted for publication in SA materials to the total number of pages in all scientific works sent for participation in SA:
Z 6i 6i 6 / 6i 3 , i I. (8)
Selection quality indicators determined by formulas (6)-(8) can be used to determine the integral indicator
Z7i 2 z 4i , z 5i , z 6i , i I , (9) where 2 some empirically introduced analytical functional dependence. 4.4.
When calculating the quantitative values [
To introduce measurement metrics and perform a comparative analysis of indicators typical for
some current SA [
Let's list some of the most important features that significantly affect the decision to accept for publication the materials submitted for consideration.
9i 1 assessment of scientific content: materials submitted for participation in SA must contain original scientific materials and have a high academic level of research;
9i 2 assessment of the clarity and comprehensibility of the language of presentation of submitted materials: headings, annotations, main text, formalization of literature;
9i 3 assessment of compliance with the periodicity and provision of the appropriate amount of materials, i.e., the existence of a history of conducting SE;
9i 4 assessment of publication ethics: clear and understandable ethical requirements for authors and materials submitted for publication; 9i 6 assessment of author affiliation, including country and institutional affiliation. 4.7.
An important stage of evaluation of submitted scientific materials can be structured using different measurement scales. These estimates can be presented in the form of values of a number of additional characteristics: ai1 affiliation of all members of the Program Committee and their geographical diversity; ai2 evidence of peer review of submitted materials, not just peer review statements; ai3 the relevance of the content, i.e. its relevance to the name of the SE and the declared problem of the SE;
ai4 compliance with the standards of the scientific community, coordination of the editorial policy is consistent with recognized modern practices;
ai4 geographical distribution and information about the authors of the submitted scientific materials.
4.8.
The measure of importance and scientific weight of SA, which turns it into SE, in particular, is the level of influence of SE on the scientific space. This direction of assessment can be structured by determining the values of several characteristics: bi1 comparative analysis of citations: number and sources of citations of SE materials; bi 2 analysis of authors' citations: history of authors' publication activity; bi 3 content relevance: the content of SE materials should be relevant, interesting, important and valuable to the scientific audience.
A necessary condition for wide recognition and popularization of SA, its transformation into SE is
internationality. This is an ambiguous concept that can have quantitative dimensions. This article
proposes approaches to the formalization of this concept, a toolkit for assessing the level of
internationality, and approaches to comparing different SE based on the open publications in CEUR.
To quantify the level of internationality, both rough express estimates and detailed refined estimates
can be used [
One of the main aspects of such an assessment can be geography, as an indicator of SE internationality. Let's formulate this thesis in the form of heuristics.
Heuristics H11. Geographical diversity is an indirect indicator of SE internationality.
At the same time, geographical representation can use for further calculation of the validity of quantitative indicators.
Different aspects of selectivity can be investigated based on the analysis of information about scientific activities in different sources:
• CEUR-WS – service of free online publication of collections of scientific conferences and provision of open access to them; • EasyChair program – a web system of software for managing scientific conferences; • MS CMT program – universal and scalable conference management system; • other open tools for supporting SA and transformation them into SE. 5.1.
In order to determine the geographical diversity of the members of the Program Committee of SA, we will introduce several notations: ci1 the total number of members of the Program Committee of the i th SA, i I ; ci 2 the total number of scientific organizations to which members of the Program Committee of the i th SA, i I , belong;
ci 3 representation of the countries of the world to which the members of the Program Committee belong - the number of represented countries, i I ; ci 4 the relative number of foreign members of the Program Committee of SA, i I ; ci 5 geography of countries represented by members of the Program Committee, i I .
Heuristics H12. The presence of one title scientific community sets the initial internationality rating of SE at level 1.
Heuristics H13. The presence of n title scientific communities sets the initial rating of the internationality of SE at the level n .
It should be noted that to determine the indicator of geographical diversity of the Program Committee, an express analysis of this value can be applied, as well as several approaches aimed at an in-depth analysis of this indicator.
Clustering of countries can be carried out as a tool for additional detailing and validity of the metric level of internationality of the event. In particular, the determination of quantitative estimates can be carried out taking into account clusters: Near Abroad, Eastern Europe, Asia, Western Europe, North America, etc.
For more detailed quantitative assessments, an analysis based on scientific organizations can be applied. To structure this problem, it is necessary to add classification and introduce classification criteria.
Expert evaluation of points that reflect the importance of the clusters to which the countries represented by the members of the Program Committee belong.
Let us denote by K the set of members of the Program Committee of the current SA, K 1,..., k 0 , k 0 the number of members of the Program Committee.
The following formula can be used to numerically estimate the geographical diversity of the Program Committee:
Z8 n 3 , (10) lL: lT where n the number of title scientific communities in the Program Committee, determined using heuristics H2, H12, H13, n 1 ;
3 some empirical value that reflects the opinion of experts about the importance of having foreign members of the Program Committee, in addition to representatives of the title scientific community or title scientific communities of several countries;
T the set of title scientific communities in a particular SA, T 1 5.2.
The aggregated (generalized, resulting, integral) indicator of internationality can be constructed in different ways, which are described by different formulas. Some logical and obvious heuristics can be introduced to investigate the level of internationality of SA and to determine quantitative estimates of geographic diversity.
To determine the geographical diversity of authors of articles submitted for publication in materials of SA, we will introduce several notations: di1 the total number of articles submitted to SA; di 2 the number of articles submitted by foreign co-authors; di 3 the number of articles authored only by foreign authors; di 4 the total number of authors who participated in SA; di 5 the number of authors of the title scientific community who participated in SA; di 6 the number of foreign authors who participated in SA; di 7 representation of countries of the world among authors of SA - the number of represented countries, i I ; di 8 the relative number of foreign authors of materials of SA, i I ; di 9 a detailed analysis of the geography of the countries represented among the authors.
The following heuristics can be used for express analysis of authors who participated in SA and materials submitted to SA. Accordingly, when applying express analysis, various formulas can be applied, the relevance of which is determined by the needs of express analysis.
Heuristics H14. The ratio of the number of articles with foreign authors/co-authors to the total number of articles: (11) (12) (13) (14) (15)
Heuristics H15. The ratio of the number of articles with foreign authors/co-authors to the number of articles by the authors of the title host country:
Heuristics H16. The ratio of the number of foreign authors/co-authors to the total number of authors:
Heuristics E17. The ratio of the number of foreign authors/co-authors to the number of authors from the host country:
Z 9i di 2 di 3 / di1.
Z1i0 di 3 / di1.
Z1i1 di 6 / di 4 .
Z1i2 di 5 / di 4 .
Z1i3 di 6 / di 4 .
Heuristics H18. The ratio of the number of countries represented in the conference to the nominal number of countries:
For a detailed analysis of the collective of authors who participated in SA, we will introduce additional notations.
Heuristics H19. One foreign co-author adds value 4 , 4 0, to the overall geographic diversity ranking of SE.
Heuristics H20. Each foreign co-author adds an adjustment coefficient 4 to the overall geographic diversity rating of SE.
If the internationality of SE is determined taking into account the clustering of countries, the value of the coefficient 4 may be different for representatives of different clusters.
Heuristics H21. Foreign authors without co-authors representing the title scientific community add a coefficient value 5 to the overall internationality rating of SE.
Heuristics H22. Each foreign co-author without representatives of the title community adds an adjustment coefficient 5 to the overall geographic diversity rating of SE.
Thus, a formula for a detailed definition of the geographical diversity index of authors of SE can be proposed
Z14 n 4 5 , (16) lL: lL: lT1 lT2 where n the number of title scientific communities among authors is determined using heuristics H2, H15, H21, H22, n 1 ; 4 some empirical value introduced taking into account heuristics H19, H20; 5 some empirical value introduced taking into account heuristics H21, H22; T a set of indexes of articles, the co-authors of which, in addition to representatives of the title 1 scientific community, are also foreign co-authors;
T2 set of indexes of articles co-authored only by foreign authors or co-authors.
At the final stage, the index of internationality of SA is determined. This indicator is a funct ion whose arguments are the geographic diversity indicator of Program Committee and the geographic diversity indicator of authors of SA.
We will preliminarily enter the expert values of the relative weighting factor of the importance of geographical diversity of the Program Committee 6 0, and the relative weighting factor of the importance of geographical diversity of the authors 7 0 , taking into account the requirement of normality 6 7 1.
The internationality index of SA will be determined by the formula
Z15 3 6Z8 , 7 Z14 , (17) where 3 some function from the products 6Z8 and 7 Z14 , entered expertly, which reflects the level of influence of these arguments on the level of internationality of SA, for example, additive convolution; 6 , 7 relevant weighting factors;
Z8 , Z14 respectively, the index of geographical diversity of Program Committee members, determined by formula (10) and the index of geographical diversity of authors of SE, determined by formula (16).
In order to check the possibilities and features of the application of the developed method, the authors conducted a computational experiment. For this purpose, the materials of some scientific projects in which the authors participated were selected in the CEUR archive. At the first stage, indices of geographical diversity of Program Committees were determined by applying formula (10) using the value 3 0,3 . These data are presented in Table 1.
After that, the authors of the materials of these scientific events were monitored and their results are summarized in Table 2. The indices of geographical diversity of the authors of the scientific event were calculated taking into account heuristics E14 and E15, for which the such values were selected: index of internationality when applying formula (17) using the values of the weighting factors 6 0,6 and 7 0,4 .
SE number according to CEUR numbering
The results of the computational experiment were presented to more than ten experts. All experts confirmed that their subjective feelings about the level of internationality of events coincided with the calculated experimental values at least at the level of ordinal scales. Thus, the results of the expert survey testify to the acceptability of the proposed approach to determining the level of internationality of the SE and to the promise of further research in this direction.
In most cases, only in-depth unbiased professional expertise can provide a comprehensive objective assessment of researchers' scientific results: scientometric indicators in this case is only as an indirect tool for metrication and support for experts' decision-making regarding the importance and quality of SE.
It is obvious that the determination of quantitative indicators of SE and the transfer of qualitative
forms of measurement to quantitative ones do not contribute to an automatic increase in the quality of
SE, even with significant positive dynamics of quantitative indicators [
The geographical diversity of SE can be perceived as an indicator of the globalization of the world, the expansion and deepening of international scientific activity. It should contribute to the improvement of the quality of scientific research, the modernization of research areas through direct interaction between different scientific teams, the positive influence of synergistic effects, etc.