This paper is devoted to the question of current labor market needs modeling for automated workshop control in knowledge-intensive areas. On the basis of the ontological model of labor market needs, the content of the workshop is systematically processed for certain pro les in knowledge-intensive areas. An approach to the e-workshop project control based on the pedagogical design model ADDIE is described. The approach is implemented on the basis of an information system that takes into account the regular updating of the technological and tool base in professional areas. The paper proposes a method for forming and modifying the list of main learning tasks - the kernel of the workshop, based on the analysis of labor market needs. This method involves systematic monitoring of vacancies in this pro le, processing their descriptions in order to highlight the requirements of employers. The methodology is implemented in the framework of the information system for workshops designing. Vacancy data is obtained from the API hh.ru, lists of the applicants requirements and responsibilities are extracted from their descriptions. The resulting lists of requirements are processed using natural language processing and cluster analysis technologies.
During the period of high competition in the labor market, the main task of educational institutions is to prepare graduates who are competitive in the labor market. Constant updating of the goals, learning outcomes and content of educational programs becomes not just a requirement of regulations and educational standards, but a necessary condition for ensuring high-quality training of students. At the level of Federal state standards, measures are being taken to strengthen the in uence of employers on the learning process in higher education institutions: involving them in the development of educational standards, creating basic departments at enterprises, and ensuring the practical orientation of learning process [1{4].
Such actions are particularly relevant when it comes to training specialists in knowledge-intensive areas. A feature of knowledge-intensive areas is the constant updating of the technological and tool base, and, as a result, constantly changing the requirements of employers.
This article is devoted to the formation and further systematic revision of the workshop kernel for knowledge-intensive areas. The workshop kernel in the context of this paper is a list of the main educational tasks that correspond to the labor functions (professional tasks that are constantly faced by active specialists in the labor market) and the competencies of the graduate. The method of forming the workshop kernel is proposed on the basis of an information system that constantly monitors and analyzes the requirements of the professional area professional standards and vacancies descriptions in online-recruitment systems. 2
At the moment, the direction related to information support for the
implementation of the educational process is actively developing. Modeling of the curriculum
based on the ontological approach is described in the paper [
In order to monitor the connection between the needs of employers and the
implementation of the educational process, concepts of various intelligent
systems are emerging. The following papers describe an automated system for
monitoring and analyzing the compliance of personnel needs based on the
nomenclature of universities [
It should be noted that the normative documents of the educational and professional areas are inert in relation to the rapidly changing requirements for the practical part implementation of the educational process. Therefore, the main drawback of systems that focus on the normative documents is that all recommendations are made for educational programs, but no speci c recommendations are given for the implementation of these. Such recommendations must necessarily include the tools and technologies updates in the professional areas, if the purpose of training is to train highly quali ed specialists of this pro le. 3
This article describes an approach to solving this problem, namely the system
of automated workshop design and workshop project control [
The approach describes the process of creating and modifying the workshop content for the education pro le of students. The model of this process is shown in Figure 1.
P(alissttyoeftakapersr'osrknfswee)slosrikosnhaolp leLtbaaisarsntskioesnfg T(hliisstyoeftakapresr'sorknfswee)solsrkiosnhaolp oAepnxianoplifyeosrntiss (lisWt Soorfiegkmtssnathaiisfimoikcnpasal)tnekeceaerrnneinlg mOsnouatdbrooeejfleoalicngtgy (WmLaaoibrnkeosatsuhantrios-mdikpnsaatpe)tuernxosijleiitaycrty Modeling
of education process Educational standards
Required skills Ontology modification
Content modification
Disc.i.p.line N Discipline 2
Disciplinel 1 Workshop prloject for each descipline
Points estimate
Workshop implementation Professional standard Selection of the list
of work tasks Selection of key skills from employers Labonueremdasrket
Teacher
Recommendations for improving the workshop
Results Analysis
Learning outcomes 1. Formation the workshop kernel for a current education pro le.
During the rst design, the information system forms the the workshop kernel
list of the main learning tasks for a speci c pro le of graduates. The main
learning tasks at this stage are the adaptation of labor tasks that the student
must be ready to solve in order to be competitive in the labor market. The
collection of labor tasks is carried out by analyzing professional standards,
normative documentation of the pro le and corresponding vacancies from
online-recruitment systems. The obtained data are processed using semantic
analysis technologies [
The resulting list of learning tasks for the workshop kernel is passed to experts to assess their signi cance and complexity (quali cation level). The information system o ers learning tasks to the workshop kernel with an assessment of signi cance above the threshold set by the workshop developer. This stages of interaction with experts is repeated every year to update the list of tasks, assess their signi cance and technologies for solving them. 3. Formation of the workshop project with calculation of labor-intensity characteristics of learning tasks and division of these tasks into auxiliary ones. The labor-intensity of learning tasks in the workshop project is determined in accordance with the weight coe cient of each learning task. The weight coe cient is calculated using the formula 1
Wi =
Pn i=1 Pm j=1 Ci Pm j=1 Ci
Kj
Kj (1) where Wi { the weight coe cient of the i-th learning task, Ci { the expert evaluation of the i-th factor of the learning task in points (from 1 to 10), Kj { con dence level of the j-th expert, n { number of practical tasks of the workshop, m { number of evaluation factors.
According to the value of the weight coe cient, each main learning task is assigned a share of the total labor intensity of the workshop. 4. Formation of disciplines workshops projects of the education pro le on the basis of educational process modeling.
At the previous stage, the main learning tasks are also divided into blocks of sub-tasks and analyzed for duplication { if the main learning task i contains one or more identical sub-tasks with the previous (within the educational process) learning task i 1, then these sub-tasks are excluded from the preparatory set i. This allows you to reduce the "preparatory" trajectory of the main educational task without losing the quality of students learning. After that, the learning tasks are arranged in the optimal order for studying and distributed among the disciplines, forming the educational trajectory of the pro le. 5. Processing the learning outcomes of the disciplines workshops and making recommendations for modifying the workshop content.
Based on the learning outcomes, the information system generates recommendations for modifying the electronic workshop content based on the results of its passage. According to the described approach, the implementation of the main learning tasks and sub-tasks is carried out using data obtained at the stages of formation and expert evaluation of the workshop kernel. Technologies and tools selected according to the labor market needs and recommended by experts are included in the educational process. Since we are talking about training specialists in knowledge-intensive areas where popular technologies and tools are constantly changing, constant monitoring of the labor market and an annual survey of specialists allow us to timely update the workshop content implementation. In this work, we have implemented an approach to ontological modeling of labor market needs based on the results of systematic monitoring of these online-recruitment systems. 3.1
Modeling the current labor market needs within a speci c educational pro le In order to clearly de ne the format, purpose of the extracted data and how to analyze it, it was decided to describe the ontological model of the professional area for the speci c educational pro le. Ontology in the context of this paper will be understood as a speci c knowledge base [16{18] used for solving learning tasks.
OP A = (EP A; RP A); (2) where EP A is a set of entities of the professional area, RP A is a set of relationships between entities in a professional area.
The set of EP A = V; L; S; Req entities is divided into four main subsets: { sub-set V (vacancies) - a subset of professions described by professional standards (PS) and vacancies. Each profession is characterized by appropriate professional standards and a speci c set of vacancies; { sub-set L (labor tasks) entities that describe labor tasks from professional standards or from vacancies descriptions; { sub-set S (skills) entities includes knowledge and skill requirements. These skills are described in the PS in relation to each labor task. The student must master these skills in order to be a sought-after specialist; { sub-set Req (requirements) - this set includes the employers requirements technologies and tools knowledge, or so-called key skills from the vacancies description that the applicant must have in order to solve labor tasks. For modeling knowledge-intensive areas this subset is key in the formation of actual content for practical learning of students in the educational pro le.
Figure 2 shows a generalized model for matching professional standards entities and vacancies description entities in online-recruitment systems.
Figure 3 shows an ontological model of the professional area of a certain educational pro le, formed on the basis of an analysis of the labor market needs and the competencies of the graduate.
These models show the main data that should be processed and compared in the process of forming the actual workshop kernel of a certain educational pro le - labor tasks and skills of professional standards, requirements, labor tasks and key skills from employers.
Profession (Prof. standart) - name - possible posts: list attach
1...*
Post - name influence 1...*
Labor function - name - labor tasks: list
1...* determine
1...*
Skills - competencies: list
Profession - name - type attach 1...*
Vacancy - name - requirements: list determine
1...*
Labor tasks confirm
1...1 confirm
- name 1...* - key skills: list confirm 1...*
Method for forming the electronic workshop kernel based on the analysis of current labor market needs The proposed method is based on semantic analysis of current requirements of employers for speci c projects (tasks) in knowledge-intensive areas. Based on the results of this analysis, the main labor tasks that a graduate of the pro le must be able to solve in order to be competitive in the labor market are highlighted. The collection of labor tasks is carried out using the analysis of vacancies corresponding to the pro le from online-recruitment systems. The required skills and responsibilities of applicants are highlighted from the vacancies description. The data obtained are processed using semantic analysis technologies, clustered by topic. The most popular labor tasks and technologies (tools) that need to be solved are highlighted. If the workshop design is not the rst iteration, then the list of learning tasks from the previous learning period can also be used as a basis.
This approach assumes the following steps: 1. determining the current labor market needs based on the analysis of onlinerecruitment systems; 2. formation (addition) of an ontological model of the professional area to identify new trends in the labor market; 3. formation of the actual workshop kernel and recommendations for the implementation of the workshop content in this educational pro le in accordance with the professional area needs. 3.3
Determining the current labor market needs based on the analysis of online-recruitment systems Professional standards are basic normative documents that de ne the legally xed requirements of the professional area. The main drawback of such documents is their inertia { the term for changing such documents can be 5-10 years. In addition, professional standards re ect skills, actions, and functions in general, without delving into speci c tools and technologies. However, the demand of the labor market for specialists of a certain pro le and the requirements for the technological and tool base change faster than professional standards. In order to track these changes in a timely manner, you need to contact online-recruitment systems.
The texts of vacancies descriptions in such online-recruitment systems have a similar structure { each vacancy contains a block describing the duties and requirements for the applicant. Depending on the employer, the di erence may be that the data may be indicated by di erent words, such as "Requirements" "Required" "It is necessary to...". The scheme of the algorithm for extracting and processing information from the vacancies description is shown in Figure 4.
As part of the information system, the analysis of vacancies received using the services API hh.ru is currently implemented. First, the workshop developer selects the required professional pro le. The system sends a request to the API and receives a list of vacancies for this pro le. Since this list o ers vacancies descriptions in an abbreviated format, vacancies identi cation number is extracted from this list. Next, the algorithm sends a request to get a full description of each of the vacancies by its ID. Data is provided in json format, and descriptions
Selection of requirements from the "description" block and preprocessing the received data.
Preprocessing the "key_skills" block data
clusters. Processing and sorting data by the number of cluster
elements
are contained in the description block. A parser was implemented in Python, extracts the "responsibilities" and "requirements" blocks and makes a list of Them for further processing. In addition, the description contains the key skills section, which describes the tools and technologies that the applicant must possess for the position.
At the processing stage, punctuation marks (commas, dots, and semicolons)
were removed. Abbreviations like "SW" were replaced with "software".
Normalization was carried out-bringing words to the initial form. In this task, we
decided to use stemmer, because due to the speci c terminology of vacancies
descriptions and the speci cs of the Russian language, the exclusion of endings
does not distort the meaning of the phrase. This system implements the Porter
stemmer [
Next, the TF-IDF weight matrix is created. We consider each selected request record as a separate document. The matrix is compiled using T dfVectorizer (sklearn package), the block of stop words is taken from the ntlk case and supplemented in accordance with the speci cs of the professional area. DBSCAN, MiniBatchKMeans, and agglomerative clustering algorithms were applied to the resulting matrix.
The choice of the DBSCAN algorithm is due to the fact that with this spatial clustering algorithm, you dont need to specify the number of clusters in advance it is di cult to predict the number of thematic clusters in the variety of employers requirements. In addition, the list of requirements often contains requirements of the type "well understand what it is about". These records may be considered noise and are not included in any of the clusters. Clusters of the following types are highlighted: corresponding to the subject area and noise requirements (the latter are removed from the list of requirements). Table 1 shows examples of such clusters.
Cluster content [user support, cost management, PC repair, responsibility, competent speech', . . . ] [ms sql development experience from 2 years, experience with industrial databases: oracle ms sql postgresql, experience with ms sql server from 1 year, experience with ms sql server (distributed databases), practical experience with ms sql server 2008/2016, experience with ms sql server pro ler, experience with ms sql server / oracle database, . . . ]
When selecting the clustering algorithms MiniBatchKMeans and agglomerative clustering, you must specify the number of clusters. It is based on the number of main labor tasks from the professional standards for this pro le, which has been doubled (since about 50 % of the requirements in the vacancy are insigni cant for the formation of this pro les workshop). This increase allows you to take into account " noise" and allocate additional clusters for storing such records. 3.4
Formation of an ontological model of the professional area At this stage, cluster data is converted to ontological entities and described using OWL. An example of an intelligence map of an ontological model fragment for Cluster1 from Table 1 is shown in Figure 5.
Embedded System
Databases
Design Work ms sql ms sql server profiler
postgresql ms sql server 2008/2016
Fig. 5. Fragment of the ontological model mind map for Cluster1
The generated ontological model is o ered to the developer for analysis and selection of the nal formulation of the main learning tasks and tools and technologies for their solution. This set of learning tasks will be submitted for further evaluation and modi cation to a group of experts from employers. 4
The proposed approach and implemented tools were used to form the workshop kernel on the pro le \Embedded and cyber-physical systems". As a basic data set for forming a kernel fragment dedicated to embedded system software, requirements and key skills for software - vacancies of 6 di erent specializations were taken and analyzed - 11183 vacancies and 121698 requirements. The total execution time of the algorithm with data loading and processing was 31.5 minutes.
Visualization of the DBSCAN and MiniBatchKMeans clustering algorithms is shown in Figure 6 and 7.
The DBSCAN algorithm showed the best results when setting the distance between the eps clusters=0.001. With these parameters, about 270 clusters are allocated that contain data that is signi cant for this pro le. Using agglomerative clustering 100 signi cant clusters and 96 clusters of the MiniBatchKMeans algorithm were identi ed.
Further, the clusters were correlated with the labor tasks of professional standards, and they were chosen as close as possible to the formulations of labor tasks. Since it is di cult to assess the semantic proximity of texts in Russian, a \manual" assessment of the semantic proximity of the selected clusters and the main labor tasks was carried out. According to the results of this assessment, the DBSCAN clustering algorithm showed better results - the selected clusters were more \thematic" and contained fewer insigni cant requirements. 5
The article describes an approach to creating the electronic workshop content of a certain education pro le for knowledge-intensive areas. These areas have annualy strong changes in the issues of technologies used and requirements for the skills of industry specialists. An ontological approach to modeling the actual labor market needs based on a systematic analysis of online-recruitment systems is proposed.
Ontological model of the professional area have been developed that take into account: the relationship between professional standards and labor market requirements, and the competence model of the graduate. This model show the main data that must be processed and compared in the process of forming the actual workshop kernel for a speci c educational pro le - labor tasks and key skills from employers. As part of the paper, the analysis of vacancies obtained using the services API hh.ru was implemented. For the pro le "Embedded and cyber-physical systems" 11183 vacancies and 121698 requirements were analyzed for the workshop kernel segment of the embedded systems software. According to the results of the expert evaluation, the DBSCAN clustering algorithm showed better results - the selected clusters were more " thematic" and contained fewer insigni cant requirements.
An ontological model of the professional area of the pro le was developed based on the results of the analysis of vacancies descriptions, and a list of current main learning tasks and tools for solving them was de ned in the kernel block of the discipline workshop \Embedded systems software" for transfer to expertsemployers for evaluation.
Based on the described approach, a software package for extracting and processing current labor market needs from vacancies descriptions of onlinerecruitment systems has been developed, and a mechanism for forming and supplementing an ontological model of the professional area has been implemented.
As a direction for further research, it was decided to compare the cluster analysis of vacancies descriptions and classify them using neural network models.