The main aim of this research is to increase project effectiveness in the ICT domain. In order to achieve this goal, it was decided to focus on a process of team formation, since a strong team is undoubtedly one of the most significant components of a successful project. To build a stronger and potentially more effective team from a wide range of candidates with different skills and knowledge, it is vital to determine the most eligible ones. Therefore, it is necessary to assess available candidates and to make this process effective, it has to be formalized and then optimized. To perform a fairly objective assessment of a candidate for a role in a project an approach using a comparator identification method is proposed to increase the effectiveness of the whole process. The European e-Competence Framework and ICT Professionals' Role Profiles documents are used to support this approach, and the appropriate software tool is designed to implement its main functionality.
maximum 5) and includes pieces of knowledge and skills needed for this competence
[
Comparator identification method is a special type of inverse identification. Input
can be presented as a set of signals of any nature, output is 0 or 1. The idea is to
determine whether input signals are in a particular relation, which is predetermined [
As it is proposed in the integrated knowledge-based methodological framework for
staff-training in IT-companies presented in [
The paper is organized as follows. Section 2 presents a short review of related work. It includes an overlook of several existing methods that can be applied to solve the problem, and a couple of software tools that use these methods. Section 3 provides a description of the proposed approach based on the comparator identification method with several detailed examples. Section 4 introduces an idea of what a software tool using proposed approach can seem like with its architecture design and user interface. Section 5 concludes the paper with a brief summary and an outlook on future work. 2 2.1
Some Existing Methods in the Domain of ICT Professionals’ Skills Assessment Nowadays there are a couple of approaches that allow to determine one’s ICT role suitability, but they are not agile and too subjective. That makes them ineligible for determination whether a specialist is suitable for a role or not in real projects. Consequently, the problem of role eligibility assessment is not formalized and does not have a specific solution, although several methods can be partially adapted for it.
Questionnaire method. Questionnaire method is an assessment method of a professional competence of a person in a chosen field based on self-rating [11]. Competence is assessed with competence index C, which is calculated by the formula: = + ∙ 100%
2 – Overall index, = 0.1 ∙ – Self-rating given on scale 1 to 10.
– Aggregated index, calculated by a summarization of scores, obtained from a reference table on scale “Low”, “Medium”, “High”.
All importance indices ( / ,[ / /ℎ ℎ] − ) for each knowledge and skill are predetermined. Knowledge/skill rating equals ∗ . Respondent chooses between options low/medium/high. After that, all ratings are summed and is obtained [11, 12].
Simple equation method. The problem of the definition whether an employee matches an ICT Role can be solved by using a simple equation [13]. In this case, the equation can be presented: = ∑ ( competences levels, which makes it less attractive for wide usage. Moreover, this method provides assessment only for competence levels and does not aggregate them into a role suitability level, which, in turn, makes it incomplete and usable only paired with another method that would calculate role suitability level based on competence levels.
Simple equation method does not consider specifics of each knowledge and skills competence pieces which significantly decreases its objectivity and makes its application in the real world questionable.
Methods based on analytical model are quite objective and accurate, but they tend to be very complex. Moreover, their adaptation to ECF and ICT PRP is either very difficult or even impossible which makes them inapplicable to our domain.
Methods based on fuzzy logic can be adapted to ECF and ICT PRP and provide result that is accurate enough. However, they are quite subjective since specialists’ skills are assessed in fuzzy numbers by other people. 2.2
Available Tools for ICT- Competence Assessment Support CEPIS e-Competence Benchmark. CEPIS has developed a free online tool for ICT professionals to assess their professional skills, based on European Competence Framework [14]. The tool allows respondents to check which of the ICT professional profiles matches them the best.
The assessment tool is presented by a questionnaire, where respondents select their own level of competence for each of them. Available level options are “None”, “Knowledge”, “Experience”, “Knowledge and Experience”. If the respondents select “Experience” or “Knowledge and Experience”, their choice corresponds to competence level (Dimension 3 of ECF).
The algorithm is based on a simple equation method. At first, Proficiency Index is computed for each of the 36 competences identified in the ECF, based on the respondent’s answers. The index (expressed in percentage), represents the degree of proficiency for each competence with respect to the ECF. These scores are then compared with what is required for each of 23 ICT profiles. Finally, the result for each profile is given in a Proximity Index, expressed in percentage. This index indicates a role suitability degree [14].
EXIN e-Competence Assessment. EXIN has developed an online tool based on ECF and ICT Professional Profiles similar to CEPIS’s [15]. It allows ICT professionals to find professional profiles, which match their skills best.
Respondents answer a questionnaire, where they select a competence level (Dimension 3 of ECF) and an extend level (“General”, “Partial” or “Superficial”) for each of 36 competences.
The algorithm is based on a simple equation method and consists of two main steps. At first, a level of competence proficiency (expressed in percentage) for each of 36 competences is computed by multiplying competence level by it extension level. Then, these scores are compared with what is required for each profile of 23 ICT profiles. A result is represented in Proximity Index, which indicates a role suitability degree [15]. 2.3
The Proposed Method After the described above analysis of significant disadvantages in existing methods, it becomes obvious that it is necessary to develop a new approach that would eliminate those serious flaws in order to provide an optimal solution for the problem. The approach should be based on a comparator identification method, which should significantly increase objectivity and allow deep connection to ECF and ICT PRP.
Comparator identification method is a method of indirect identification, which uses
predicate logic for calculation. This method takes any types of data (signals) for input,
but output is always binary (true/false or 0/1). Basically, this method allows to
determine whether objects are in a particular relation or not [
Method's feature of taking any data type for input allows full compatibility with ECF and ICT PRP, therefore, this method can be easily adapted to our domain. Binary output, in turn, allows to obtain a definite answer, which is in our case, “whether a specialist is eligible for a particular role in project or not?”.
The following Table 1 gives the result of the methods comparison.
A comparator identification method provides a differentiated result in a binary form. Although it is not as accurate as in other methods, it meets the requirement of determination whether a specialist is suitable for a role in a project.
The main advantage of a proposed solution over other methods is high objectiveness and full compatibility with ECF and ICT PRP. High objectivity level is achieved by using predicate logic to strictly determine results for all possible cases. Flexibility of predicate logic, in turn, allows to fully adapt method to ECF and ICT PRP. All that makes the assessment process formalized and applicable in real projects.
Workflow description. An applicant for a role in a project is interviewed for knowledge and skills contained in the ECF competence description for each competence included in the Professional Role profile description. Based on the answers, a level for each competence and later – a role suitability degree – are determined using comparator identification method.
Algorithm. On the first step, respondent selects answers knows/doesn’t know for each piece of knowledge and has/does not have for each skill (the fourth dimension) for each competence, needed for the role.
Then, the level of each competence is determined by the following predicate : ( 1, … , , 1, … , ) = , = 1, – Number of competences needed for the role – Pieces of knowledge – Skills – Proficiency level of ith competence
The predicate includes only levels, available for this particular competence. The predicate has the following internal structure, where = or = : = =
… ( 1, … , , 1, … , ) ( 1, … , , 1, … , ) – Minimum possible proficiency level of competence – Maximum possible proficiency level of competence The predicate is solved form bottom to top: if = 1, then = , if 0, we go the predicate above and repeat algorithm until then the respondent does not have this particular competence. is reached. If
= = 0,
Finally, after all competence levels are determined a role relevance degree is calculated by the following predicate:
( 1, … , ) = − (1 0)
The result is obtained in the form suits ( = 1) or does not suit ( = 0). 3.2
Calculation of the Test - Examples Example 1: Quality Assurance Manager. Let us suppose there are candidates for a vacant role of quality assurance manager in a new project, and they need testing in order to determine whether they suit this role (see Fig. 1). The role includes four competences. On the first step, our candidate will be tested to determine his proficiency levels for each competence. The determination of the competence level will be performed based on the answers in yes/no form for each piece of knowledge and each skill of the 4th dimension of the competence. The correctness of the results is supposed to be checked by company’s technical specialists.
The first competence is D.2 - ICT Quality Strategy Development (see Fig. 2).
The second competence is E.3 - Risk Management (see [
The following Table 2 presents answers for each competence. The equation for determination of D.2 competence proficiency level is following: 4 = 2 3( 1 2 ∨ 1 3 ∨ 1 2) {
5 = 1 2 3 1 2 3 The result for D.2 competence is: 1(̅̅1̅, 2, ̅̅3̅, 1, 2, 3)= 4 The equation for determination of E.3 competence proficiency level is following: 3 = 2 1 2 { 4 = 1 2 1 2( 3 ∨ 4) 5 = 1 2 3 1 2 3 4 The result for E.3 competence is: 2(̅̅1̅, 2, ̅̅3̅, 1, 2, 3, ̅4)= 3 The equation for determination of E.5 competence proficiency level is following: The result for E.5 competence is: 3( 1, 2, ̅̅3̅, 4, 5, 6, 1, 2, 3)= 4 The equation for determination of E.6 competence proficiency level is following: 2 = 2 2 5 { 3 = 1 2 2 5( 3 ∨ 4)
4 = 1 2 2 3 4( 3 ∨ 1) The result for E.6 competence is: 4( 1, 2, ̅̅3̅, 1, 2, 3, 4, 5)= 4 After each competence level is calculated, it is possible to determine whether the candidate is eligible for the role. The following Table 3 presents the calculated competence levels.
The final result equals: ( 14, 23, 34, 44) = 1 (The candidate fits the role).
Example 2: System Analyst Role. Let us suppose there is a candidate for a vacant role
of system analyst in a new project, and this contender needs testing in order to
determine whether he/she suits this role (see [
The first competence is A.5 - Architecture Design (see [
The following Table 4 presents answers for each competence. The equation for determination of A.5 competence proficiency level is following: The result for A.5 competence is: 1( 1, 2, ̅̅3̅, ̅̅4̅, ̅̅5̅, 1, ̅̅2̅, 3, ̅4, 5)= 3 The equation for determination of B.5 competence proficiency level is following: The result for B.5 competence is: 2( 1, 2, 3, 4, 1, 2, 3, 4)= 3 The equation for determination of B.6 competence proficiency level is following:
The equation for determination of E.5 competence proficiency level is following: The result for E.5 competence is: 4( 1, 2, 3, ̅̅4̅, ̅̅5̅, 6, 1, 2, 3)= 3 After each competence level is calculated, it is possible to determine whether the candidate is eligible for the role. The following Table 5 presents the calculated competences levels.
The final result equals: ( 13, 23, 34, 43)= 0 (The contender does not fit the role).
Development of Software Tool for ICT- Professionals’ Skills Assessment
The software system is designed for ICT companies with a medium-to-large number of employees. Its mission is to automate the process of team formation, which should increase an overall project success.
The system should provide a functionality for project data management for company directors, team management for project managers and personal data management for all employees. Moreover, it should provide an opportunity for project managers to send employees project participation invitations and an opportunity for employees to apply for projects. Finally, one of the main of its features is an opportunity to test personnel for role eligibility to determine the best candidates and form an optimal team squad.
The defined functional requirements are shown on Fig. 3 in a form of a use case UML diagram. Conceptual data model is given in the form of UML class diagram (see Fig. 4). “Competence” entity represents a competence from the ECF with all its attributes. “RoleProfile” entity represents a role profile from the ICT PRP with all its attributes. “RoleCompetence” entity represents a particular competence with its required level, included into a particular role.
These three entities are used to store ECF and ICT PRP documents in the database.
Database stores project managers (“Project Manager” entity) separately from all the other employees (“Employee”) because of their relation to projects (“Project” entity). Projects can have any amount of employees of any specializations (“specialization” field in “Employee” entity), but one and only one project manager.
“Project team” entity represents a group of employees that work on a particular project or several projects. All employees in a project team are assigned specific roles (“Role” entity), which they have in this particular project team.
“Test” entity represents a test taken by a particular employee for a particular role. “Role” entity represents a role of a particular employee in a particular project team.
Several factors determine optimal architecture. The first factor is target platforms (mobile devices in our case). The second factor is database and server provider (Firebase by Google in our case). It was motivated by high accessibility and the fact that Firebase provides both server and database management system.
Considering these two factors, Rich Mobile Application architecture was chosen (see Fig. 5). On the basis of designed architecture and with regards to functional requirements, a software tool can be built. The prototype of such tool is shown in the next subsection. According to the functional requirements (see Fig. 3), the appropriate database model (Fig. 4) and the chosen architecture (see Fig. 5) as a prototype of a software tool was developed. Especially, on Fig. 6 and 7 the user’s interface fragment is shown. Fig. 6 demonstrates options to analyze the project characteristics (Fig 6,A) and look through team’s requirements (Fig 6,B). Fig. 7 demonstrates options to choose the appropriate workers from the list (Fig 7,A), and finally observe result of their skills testing with the ability to approve one of them for each project role (Fig. 7,B).
Fig. 6, 7 demonstrate the next case: There is a project (named “Example”) for which a project team is needed. Currently, a team has a business analyst, a system architect, a tester and two developers approved. The next step is to approve an employee for a role of quality assurance manager. There are 3 candidates for this role with only two of them being suitable for it. A person responsible for a team squad formation chooses one of them and approves him/her for a role.
Usually, this process requires more people and time, as it is needed to form a pool of potential candidates, test every one of them, decide if they are eligible for the role, choose the best candidate and finally, inform everyone about the result. The suggested software tool automates this process so the choosing the best candidate is the only step that has to be performed manually. The experimental usage of the developed approach in the practice of candidates evaluating for an employment in the IT-company “Academy – Smart” LTD, Kharkiv [22] showed the processing time for appropriate data was reduced about 22%. 5
This paper includes the overview of some existing methods, which can be used to perform an assessment of a specialist’s suitability for a particular role in ICT competences domain. Their weak points were determined and a new approach was proposed in order to eliminate them. Our approach is fully compatible with ECF and PRP documents and it is based on the comparator identification method that helps to increase the objectivity of the assessment result, because it calculates the quantitative parameters for this purpose. The experimental usage of this method has shown its feasibility in some real test – cases of IT-staff’s assessment.
Next steps to be done is a modification of our approach to increase its objectivity, in order to obtain more precise results that will help to make better decisions about potential team squad. In our future efforts, we also would like to elaborate a comprehensive methodology to test our approach not for selection of prospective candidates only, but with respect to a possible software product quality improvement in a target ITcompany. 10. Tkachuk, M. V., Sokol, V. E., Bilova, M. O., Kosmachev O. S.: Classification, typical functionality and application peculiarities of learning management systems and training management systems at IT-companies. In: Modern Informational Systems, Vol. 2 (4). pp.87–95 (2018) 11. Grabovetsky, B.S.: Economic forecasting and programming. Vinnitsa, VDTU (2001) 12. Kovalenko, I.I.: Expert assessments in innovative projects control. Nikolaev, NUK (2007) 13. CEPIS, e-Competence in Europe - Analysing Europe’s Gaps and Mismatches for a Stronger
ICT Profession, European Report (2014) 14. CEPIS e-Competence Benchmark // www.cepisecompetencebenchmark.org. 15. EXIN e-Competence Quality // http://www.e-competence-quality.com. 16. Zakarian, A., Kusiak, A.: Forming teams: An analytical approach. In: IIE Transactions, Vol.
31, pp 85–97 (1999) 17. Campelo, M., Figueiredo, T., Silva, A.: The sociotechnical teams formation problem: a mathematical optimization approach. In: Annals of Operations Research, Vol. 286, Issue 1–2, pp 201–216 (2018) 18. Chen, S.-J., Lin, L.: Modeling team member characteristics for the formation of a multifunctional team in concurrent engineering. In: IEEE Transactions on Engineering Management, Vol. 51 (2), pp. 111–124 (2004) 19. Fitzpatrick, E. L., Askin, R. G.: Forming effective worker teams with multi-functional skill requirements. In: Computers and Industrial Engineering, Vol. 48 (3), pp. 593–608 (2005) 20. Tseng, T. L, Huang, C. C., Chu, H. W., Gung, R. R.: Novel approach to multi-functional project team formation. In: International Journal of Project Management, Vol. 22 (2), pp. 147–159 (2004) 21. Karsak, E. E.: A fuzzy multiple objective programming approach for personnel selection. In: Proceedings of the 2000 IEEE International Conference on Systems, Man, and Cybernetics, Nashville, TN, USA, Vol. 3, pp. 2007–2012 (2000) 22. Official Web-site of the “Academy - Smart” IT-company // https://academysmart.com.ua/