In the scientific work, a module is proposed that allows to detect incorrect and irrelevant information based on the content analysis of web resources in the intellectualized academic staff performance appraisal system. The methods and means of detecting irrelevant information based on existing user needs are analyzed. This module allows to analyze the information filled in by the user and make an automatic analysis of its correctness. Web system, performance appraisal system, academic staff, scientific and pedagogical workers, rating, correctness of information, higher education institution.
Improving the work of scientific and pedagogical workers is important for a higher education
institution. This will allow: to improve the educational process (if scientific workers work efficiently,
the educational process becomes more qualitative, which leads to better training of students and an
increase in the university's rating); attract funding (additional funds are attracted to educational
institutions that have highly qualified scientific workers); increase competitiveness (universities
constantly compete with each other for students, funding and other resources, accordingly, qualified
employees increase their competitiveness, which increases the number of students and the attractiveness
of the university for employers); career development of employees (conducting scientific and research
development of employees) [
To improve the work of scientific and pedagogical workers, a number of measures can be taken, for
example:
knowledge;
and increasing the rating of the university;
organization of education and trainings that will help improve their professional skills and
involvement in scientific and research work, which contributes to the development of science
academic staff support and motivation (financial support, prizes and awards);
introduction of new technologies into the educational process [
The evaluation of the effectiveness of the academic staff activities can be carried out on the basis of various criteria, depending on the specific situation and requirements of the university, for example: the educational process (assessment of the quality of learning, degree of education for classes); scientific work (assessment of the level of research work, number of publications, participation in scientific conferences, projects);
organizational activity (assessment of participation in the work of university management bodies, performance of tasks assigned to the employee, organization of conferences, seminars);
2023 Copyright for this paper by its authors. CEUR
ceur-ws.org cooperation with other scientists and universities (level of cooperation with colleagues, involvement in joint projects, cooperation with other universities).
It is important to take into account that the evaluation should be objective and based on criteria that
were established in advance. In addition, it should be remembered that the evaluation should not be the
only criterion for determining the effectiveness of the work of a scientific and pedagogical worker [
The most popular method of evaluating the performance of employees in many universities is rating
systems [
In many universities, the rating system includes the opportunity for employees to independently contribute their achievements. For example, a scientific worker can add his publications in scientific journals, feedback from students, participation in projects, conferences, seminars and other information about his activities. These informative materials can be included in his personal profile in the rating system. However, it is important that the information provided by the employee is objective and supported by relevant documents. Also, it is important to consider that the rating system should be comprehensive and evaluate the employee's work according to various aspects of the activity, and not only based on what the employee independently enters into the system.
Verification of the correctness of the information provided by the employee is very important, as it helps to ensure the objectivity of the evaluation of his work. In addition, inaccurate information can lead to incorrect conclusions about the employee's performance, which can negatively affect his career and reputation, and cause the expenditure of time and resources to verify and correct data. Therefore, checking the information for correctness is necessary to ensure the accuracy and objectivity of the rating system and to ensure a fair assessment of the employee's work.
Currently, not all universities in Ukraine have their own academic staff performance appraisal system, but some of them already do so. Here are some examples of universities that have their own systems: National University "Kyiv-Mohyla Academy", National Technical University of Ukraine "Ihor Sikorsky Kyiv Polytechnic Institute", Taras Shevchenko Kyiv National University, Ivan Franko Lviv National University, Odesa National University named after I. I. Mechnikov, West Ukrainian National University (WUNU), and others.
If we compare different rating systems, they provide a variety of functions, among which we can highlight: evaluation of academic staff activities according to various parameters (such as learning quality, scientific activity, professional development, etc.); stimulating to improve the activities (through ratings); determination of priority areas of development (analysis of the results of ratings and statistics allows to determine the priority areas of development of academic staff and their weak areas); monitoring the quality of education and making the necessary adjustments; stimulating the academic staff to professional development and continuous improvement of their work; show the rating of a specific users (users can view the rating of academic staff at the university according to various criteria); provide performance reports (number of publications in scientific journals, number of citations, etc.).
An important function for such systems is the verification of user documents, namely the process
of checking the correctness of the data entered by the employee. The rating system used by WUNU
[
Considering the architecture of the rating system at WUNU, it was decided to use a client-server architecture. It is the most common and is based on the separation of functionality between the client and the server, and allows creating software systems with distributed data: provides fast and reliable data exchange between the client and the server, allows the use of different programming languages and technologies, provides a convenient user interface and high performance, security and availability of the system.
Accordingly, there is no binding to a specific programming language or DBMS. The database in the
specified system is implemented using MySQL - the most famous open relational DBMS, which is used
in most large organizations for data storage and management. The Laravel framework was used to write
the module. It is used to develop web applications and APIs in the PHP programming language (which
will need to write a data validation module), supports MVC (an architectural pattern for software
design), and has a large number of components and libraries that allow to extend its functionality and
support the best development practices. Controllers are responsible for processing requests from users
and calling the appropriate methods and models to retrieve and store data. View files are responsible
for displaying data on the client side and interacting with the user. MVC in Laravel allows to divide the
logic of the application into components and make the code more clean and organized [
In the WUNU rating system, users can fill out annual reports on their scientific activities. The report is populated using various criteria, which are created by the system administrators in accordance with the annual rating regulation. Some criteria allow users to enter data about their articles, publications, research papers, and more. 3.2.
First, it need to collect data and carry out their initial verification [
Understanding how the resource page is built, now it is necessary to perform parsing of sites (web pages), that is, sequential syntactic analysis of information. Web pages themselves may have a hierarchical data set structured in some format.
Web parsing is used to automatically collect data from web pages is an important step. This process
includes parsing the HTML code of a web page. We get the main context and main attributes (headers,
titles, <body>, <label>, <context>, eth) of the page using HTML parsing and MeaningCloud Text
Analytics Api). Then we extracting the necessary information and saving it in a convenient format
(CSV) for later use, for example, in a database [
In order to display the received data in the correct form, it is necessary to process them and highlight
the main information, which will be enough to save short tags on request. For this, the CSV format was
used to save intermediate and final data. It is easily opened through Excel, and easily converted into
SQL queries for further work. Collected digital data may be analyzed using mathematical models [
A more complex process is identifying incorrect and outdated information based on content analysis of web resources. In the WUNU rating system, each user has a certain set of tags that associate and verify him. These can be mandatory tags (which the user must fill in on their own), general tags (which the user can fill in at will), optional external tags (which are entered by their heads of departments, or senior academic staff of the university), and specific tags for each answer (entered data) of the user in the report (entered by the user). Having a complete database (a certain set of correct data/tags) about the user, a method of detecting incorrect and outdated information on web resources was implemented by comparing it with its own database.
The schematic structure of the process of preparation for the procedure of identifying irrelevant and unreliable information on web resources is shown in (Figure 2), and involves the following actions: collecting content/information, most often by downloading a web page and processing it. the received data in HTML format is transformed into the specified format. conversion of the obtained results into a higher format convenient for users and employees and saving in a database. comparison of received data with existing data in the database and evaluation of information.
It may also be the case that the existing database is not filled, or there is no information on a specific user (for comparison), so it is necessary to analyze the information based on the content of external web resources. One way to check the data is to check other sources that can confirm the input information. This can help determine how reliable and correct the information received is. And for this, search web browsers are used, which make it possible to search for various types of content on the Internet (for example, web pages). Search web browsers are powered by various search algorithms that typically use keywords or phrases to find relevant content. In addition, they use other methods, such as ranking results by relevance, analyzing context, and using other metrics to provide quality search results to users. This way of searching for data can yield a certain number of web resources (which may contain content similar to the content analyzed by the informational web resource). If there are multiple sources of data (pages) that are obtained as a result of the search web browsers and they provide information about whether a particular entry is correct or not, a majority vote method can be applied to determine which information is correct. To do this, it need to compare the results of each data source and determine which value occurs most often. If more than half of the sources confirm the authenticity of the entry, then it can be assumed that this value is correct. It can use the majority method, that is, the majority of search results must confirm the information in order to recognize it as valid. The implementation scheme of the "majority voting" method with the received n-results of web resources is shown on (Figure 3).
After the implementation of this module for detecting incorrect and outdated user information, the general structure of the process of checking and approving the report will look like this (Fig. 4).
In this work, a module for detecting incorrect and irrelevant information based on content analysis of web resources is proposed for an intellectualized academic staff performance appraisal system. Validation of data entered by academic staff is important to ensure the accuracy and reliability of data used to make decisions in the field of management of a higher education institution. For example, if the data on a academic staff’s scientific publications or his academic achievements are not reliable, this may lead to incorrect determination of his rating, incorrect distribution of financial resources, as well as incorrect decision-making regarding his career development.
In addition, data validation is an important element of data quality control and protection against possible errors or carelessness in their entry. This can help prevent inaccurate, false, or incomplete information from being entered, which could spoil the results of data analysis and lead to negative consequences for the university and its employees.
The novelty of the work consists in the integration of the module for checking the correctness and conformity of the data entered by the user (answers to certain criteria) into the existing system of rating users of the university, taking into account the features of this rating system (the form of filling out the report, the format of the answers, the user's sphere of activity).
This module is being prepared for integration into the WUNU rating system. Currently, the module is at the final stage of implementation. Testing took place on a local machine with personal test data. A part of the data of the production base, for the previous rating year, is being prepared, and now criteria and tags are added to the necessary entities so that the module can be tested on real data. For the university, this new module is an improvement of the current rating system, which expands its work and the functionality of the system. Therefore, this module will improve the rating system of the university, automate part of the process of checking user reports and simplify the work of department heads and reviewers, accordingly, not only the efficiency of the employees will increase, but also the efficiency of the activity and the image of the university.