Recruitment is a long, complex, and crucial process for every company. Matching candidates to job ofers is time-consuming and requires a broad knowledge of the diferent domains, in particular the relevant skills and qualifications and their relationships. By comparing the required skills with candidates' profiles, recruiters can identify potential collaborators for a given position. To support recruiters' work, we propose recommendation techniques that help the identification of suitable candidates. While job ofers contain required skills for a position, the recruiter's preferences are not explicit: for example, some skills might be more important than others etc. These preferences are very hard to elicit, and they even might depend on the particular job ofer. We propose conversational recommendation techniques that can support recruiters' work and recommend candidates to a given job ofer, based on relevant skills, that can provide an explanation for the recommendation, so that the recruiter has specific information as to why a candidate is recommended. Interaction with the system that can reveal more details about preferences is possible. In this way, a new set of recommendations can be obtaines or he can reinitialize the recommendation procedure, if preferred. Our system is evaluated using a real Resume/Job Ofer dataset from a company database. Performance of generated recommendations is compared to a reference Deep-Learning based matching system trained on the same dataset.
for developers experimented with the “express” development framework, one might also consider candidates Recruitment is a long, complex, and crucial process for ev- with similar competencies or experiences. For example, ery company. Recruiters and human resources experts of- a candidate who knows well the “hapi” framework (that ten need to evaluate a large number of candidates. There is a diferent framework for the same task) could be well are many tools to support the work of recruiters that suited for the job. Identifying such highly competent have been in use for decades [1]. These tools can help candidates requires a broad knowledge of the technical with various tasks, including information extraction or terms. Moreover these technologies evolve quickly. This matching candidates to job ofers. means that it is not easy for the recruiters to understand
Our work also focuses on this matching task and we the relation between these skills. Clearly, other factors propose a recommender system for recruiters that can than skills are highly relevant for the recruitment (such as support their work through recommendations. We do geographic proximity or other factors), and our methods not aim to replace the human agent in the recruitment can be extended to deal with these aspects as well. process, on the contrary we would like to empower the Our recommender system can help recruiters to ideninvolved persons. In large companies, the number of tify potential candidates and is capable to suggest canpotential candidates for a position can be overwhelming. didates beyond a simple keyword match, based on the In particular, we focus on reducing the number of candi- relatedness of the necessary skills and the candidate’s dates a recruiter has to look at using skill-based matching competencies. There are a number of challenges involved of job ofers and candidates. Understanding how vari- in designing and realising such a recommender system. ous skills listed in job ofers or resumes relate to each A job ofer in general should express all the requirements other is often not self-evident. For example, for a “back- for the position in question, however in reality it conend developer” position where the company searches tains only partial information. For example, some skills might be more important than others. In other cases, the recruiters can have specific politics: they might prefer someone with a broad range of skills, while in other cases, one specific skill is crucial for the position, so they seek someone who has this skill. These preference structures are not known in general. They might even be diferent for each position, or each company. It is dificult to elicit these preferences, as they might not even be clear to the recruiter: he might realize it as he consults the candidates resumes. To overcome these challenges, our system fol- geographic location, education, etc. The Job Ofer inform lows the conversational recommendation paradigm [2]. us on the skills required for the position, and the Resume We have developed techniques that can propose a candi- on the skills possessed by the candidate. We assume that dates based on some initial assumptions. The user can Resumes and Job ofers are semi-structured, that is we later refine the recommendations through interactions can identify skill labels in these documents. A skill label with the system or restart the recommendation process represent a skill and is composed of a few words. Howif he prefers so. ever, diferent labels can refer to the same skill depending
In our application context, it is particularly important on the vocabulary used by the person which redacted the that recommendations be explainable and configurable: label, for example, "back-end development" and "servera recruiter should be able to understand why a candi- side development" refer to the same skill, but expressed date has been recommended, that is, what is the relation in a diferent vocabulary. between the candidate skills and the job ofer. A Job Ofer in our model is a document containing a Our contributions in this work are the following: set of labels of the necessary skills = {1, 2, . . . , }.
A Resume is also a document containing a set of labels, • We propose a recommender system for resumes but they represent the skills possessed by the candidate to job ofers, based on a domain-specific ontology. = {1, 2, . . . , }. Those labels can’t be compared • We generate explanations for the generated rec- directly, because they are redacted by diferent persons, ommendations. hence use diferent vocabulary. A function to translate • We propose a conversational recommendation those labels to the appropriate skill in a unified referential strategy to update recommendations through in- is required () → , ∈ . For this referential, we teractions with the user. propose to use an ontology. An Ontology is a hierarchical • We evaluate our methods based on a real word structure which takes the form of a Directed Acyclic dataset and compare to a full deep learning-based Graph that represent a set of concepts and their relations. recommendation method. In our work we use a skill ontology, in which nodes correspond to skills and edges denote a “is more specific
The rest of the paper is organised as follows. We for- skill than” relation = {, }. mulate the job candidate recommendation problem in Section 2. We present our conversational recommender system for job recruitment in Section 3. We present the 3. Conversational experimental evaluation of our approach in Section 4, in- recommendation of job cluding a comparison to a deep learning-based matching candidates system. We discuss related works in Section 5. Section 6 concludes the paper.
Our goal is to recommend candidates for a specific job ofer to support recruiters’ work. In our recommendation model, each recommendation is accompanied by an explanation and complementary information (see Section 3.2). The user can observe these pieces of information and can decide to change certain parameters of the recommendation process and obtain a new, more refined, set of recommendations. This process can be repeated as many time as necessary, resulting in a conversation between the system and the user that should lead to a few good recommendations.
In the following we describe the data model used to generate job recommendations. Positions are described in “job ofers”. We recommend job candidates, who are represented by their “resume”. In this work we focus on recommendations based on skills only. The model could be extended to take into account other aspects, such as Section 3.1 presents an ontology-based recommendation approach, that can produce interpretable job candidate recommendations, which represents the basis of our conversational recommendation approach. This method generate recommendations to identify potentially relevant candidates. For each recommendation explanations are generated that we describe more in detail in Section3.2. Section 3.3 discusses how the user can interact with the system and further refine the recommendations.
candidate’s resume and the job ofer. In order to deal with the linguistic diferences between the various terms that denote skills we rely on a widely adopted ontology of skills: ESCO (There are a number of alternatives which could also be used, including the American ontology O*NET, or company specific proprietary skill catalogs).
Our recommendation technique can identify a set of candidates who are likely to suit for a given job ofer. The recommendations are generated if a user -who is the agent in charge of the recruitment- is engaged in searching for a candidate. We obtain recommendations for a given job ofer in the following way: Level Accuracy
• Skill matching. We first match skill labels of the resume and the ofer to nodes of the ontology.
This is a preparatory step, that we execute only evaluation (we input this the larger amount of data used once, we do not need to repeat this step for each for training). We used every label in French, English, recommendation. Spanish, German, and Italian to do so. Despite using • Skill similarity. We estimate the similarity of multiple languages, we made sure that every label would individual skills in the job ofer and resume using only appear once in the dataset for a given skill to prevent the ontology. the same label to appear in multiple subsets (i.e. train, • Aggregation. We aggregate individual skill simi- dev, or test) because it is the same for multiple languages larities (between the skills in the ofer and resume) (e.g. SQL would appear in every language, it is never depending on the search politic of the user. translated). • Generating recommendations. We generate For each label from the ontology, the model was trained candidate recommendations for the job ofer, to- to predict the skill it comes from, as well as each of its gether with an explanation. ancestors up to the top of the ontology. This ensured strong accuracy for skills placed higher in the hierarchy 1. 3.1.1. Skill matching
people, hence use diferent vocabularies. To deal with this problem we use an ontology and map each skill in the Resume and in the Job ofer to the corresponding skill in the ontology. To realize this task, we use deep learning-based matching techniques.
We trained a classifier on top of a multilingual version of BERT because we can find documents in French and English in our dataset, and the multilingual model showcased stronger performances during experimental
3.1.2. Skill similarity Skills from the ofer are requirements and those from the resume can match/fill those requirements. They can be related to each other using a similarity function based on the ontology. One could use diferent similarity function. We have chosen a similarity function that works particularly well for skills: we would like that the similarity function respects the structure of the ontology. 3.1.4. Generating recommendations We have decided to use the similarity function that was proposed by Panagiotis et al. [3] for comparing skills in a hierarchical structure (Equation 1) : Usually a recommendation consists in presenting a user with an item he could be interested in. In the case of recruitment the item isn’t for the user (i.e. the recruiter), (, ) = − ℎ((, )) , (1) bpuretsfeonrtathjoebufsreormwaitchuastloismt oerf.cSaondgiidveanteasnthoafetrc,owueldwmoualtdch the ofer. Candidates are recommended based on their where and are skills from the ontology, is score. the maximal depth of the ontology, and (, ) their The most straightforward way to make recommendalowest common ancestor. tions given an ofer and a set of resumes, is to rank the resumes based on their matching score with the ofer 3.1.3. Aggregating skill similarity scores according to our model, and recommend the top-k to the We are reducing Resumes and Job Ofers to a list of skills recruiter. On our 6181 ofers and resumes, with = 10, which can be compared using their relationships in an this would lead to an average score for the top-k of 0.98 ontology. We want a scoring function that measure how which ensure high precision, but only 2204 resume would much a Resume fits to a Job Ofer using only their skills ever be recommended meaning that almost two third of and the ontology, and keep it simple enough that it can our consultants would never be attributed a job. be easily explained. The higher the score, the higher the An other approach would be to probabilistically recmatch. Also, the score should be easy to interpret for ommend a resume based on its score. The probability to a human: 1 is a perfect match, and 0 is no match at all. recommend a resume is its score divided by the sum A resume scoring higher than another resume for the of every score: () = ∑︀ ∈ . This method reduce same ofer is to be interpreted as a better match. Also, the average score of the recommendation to 0.74 but (1, 1) > (2, 2) means the resume 1 is increase the catalog coverage to almost 100% (6124 rea better match for job ofer 1 than 2 is for 2. sumes are now recommended). We could also use the
Using this measure we can associate each skill from the exponential of the score instead of the score itself; this ofer with its most similar counterpart from the resume. leads to an average score of 0.71 and 6179 resumes being From there we can design multiple possible aggregation recommended. politics to match ofers and resumes that can correspond to diferent preferences or recruitment strategies:
Average politic:
3.2. Explanation denotes the job ofer , the resume, the skills set of , and the similarity between two skills (equation 1).
1. Average: compute the average of the best associ- By using a manually designed scoring function and a ation for each skill of the Ofer (this is the politic human-understandable representation of skills (skill onused by default). tology), we can generate an explanation for each rec2. Min: only consider the association with the low- ommendation. An example of such explanation can be est similarity. seen in Figure 1. The resume match the job ofer at 69% 3. Max: only consider the association with the high- because its skills are close to those contained in the job est similarity. ofer: SQL is close to Mongo, Python to Javascript, Flask to Node, and Flask is also not too far away from Angular.
If the user wants to know why a skill is matched to an(, ) = ∑︀∈ max|∈| (, ) (2) boethtwereeonnethwe mithtoa sceeertwaihnersecoarree,thhee ccaonrrecslipcoknodnintghneolidneks located in the ontology and how they are linked.
An explanation can have diferent forms, graphical or textual, and pursue diferent aims [ 4]. Law et al. [5] (, ) = min max (, ) (3) compared a graphical explanation to a textual one, and ∈ ∈ if the first was preferred by users, the last one yielded the best accuracy in terms of decision making. In our case and given the use of an ontology, we can generate (, ) = max max (, ) (4) an explanation by making sentences of the form "1 is ∈ ∈ to 2 because both are (1, 2)". If skills 1 and 2 would be matched together, would be an adjective selected based on the similarity between the skills, and (1, 2) their lowest common ancestor. For example, the link between Mongo and SQL from Figure 1 give: SQL is close to Mongo because both are database (, ), and we considered the last expemanagement systems. 1 rience of a resume as a job ofer.
The explanation enables also to refine the search and We evaluate our system as a Job Recommender System guide the user to identify her (or her company’s) pref- and as a Conversational Recommender System through erences. In this way, the user can fine-tune the query the following aspects: and obtain a new set of recommendations. To allow this ifne-tuning, we propose a slight variation from our initial • Raw accuracy compared to Deep Learning on a scoring function where a weight is associated to each binary classification task. skill from the Ofer. is a weighting function that asso- • Weighting techniques, to understand the impact ciates to each skills of the job ofer a weight between 0 the user can have. and 1. Those weights can be seen in the Figure 1. • Utility/Eficiency of its conversational components. ∑︀∈ ∈ (, ) × ()
∑︀∈ () (, , ) =
(5) following task: given a resume and a job, the system
If the recruiter notice a lot of resumes are recom- must predict if they constitute a match or not. For this mended because they contain a skill he doesn’t find that evaluation we can rely on a variety of classification metimportant, he can lower the weight attributed to the skill, rics: accuracy, precision, recall, and F1-Score. A high or even remove it. On the opposite, if he notice a very precision would greatly improve the eficiency of the reimportant skill is always missing from recommended cruiters (less irrelevant resumes to look at), and a high resumes, he can increase its weight. recall would reduce the risk of missing a good resume.
For example, if a recruiter is looking for an "Angular" Also too low a recall would raise ethical concerns, some developer and gets a lot of recommendations that do not resume might never be recommended, hence preventing contain the "Angular" skill, but javascript and other web their owners from getting a job. frameworks (e.g. Vue) he can increase the weight of this We’re going to compare our ontology-based recom"Angular" skill to get more recommendations containing mender system to a system entirely based on Deep Learnit. ing.
To evaluate our model, we have access to a database of 343.802 resumes from the ALTEN company. It uses a specific format called "Technical Files", that is a structured resume which contains a list of professional experiences that include a job title, a description and a set of skills. We have a total of 249,721 experiences, less than the number of resumes. This means that some resumes are incomplete and do not contain any experiences. After preprocessing we keep only resumes that contain at least 2 experiences, so we’re left with 30,907 resumes and 154,033 experiences. Most of these resumes are in French, but we also have some in English. We generated a resume/job matching dataset by creating pairs
use, there is currently not any mention of Mongo, Angular, Flask or Node. Though we can find SQL, JavaScript and Python along other IT technologies like CMS (e.g. Wordpress). ESCO is being extended, and also one can use a company specific enriched version that would contain all relevant skills and competences. 4.3.1. Dataset for binary classification We constructed a new evaluation dataset from our dataset introduced in section 4.1. The original includes a number of resumes matched with an appropriate job. For this evaluation we also need pairs of resumes with jobs that do not match. We duplicated every pairs and replaced the job with an experience randomly selected from another resume. In this way, we obtained an equal amount of good and bad pairs of resume and job. This procedure is depicted in Figure 2.
This methodology has a few shortcomings: • If an Experience is part of a Resume, this does not mean the person was a good fit to the job (stafing errors can occur). • Taking an Experience at random for bad pair may sometimes land a good match, especially since all jobs are from the same company, and as such in in similar fields. However, the fact that ALTEN employ a wide variety of profiles can mitigate this efect.
• a Train set composed of 49,452 pairs, on which we trained a Deep Learning baseline and the Linear Regression on top of the score from the Ontologybased model. • a Dev set composed of 6,181 pairs, used to decide when we must stop training the Deep Learning model and to fine-tune the Ontology-based model. • a Test set composed of 6,181 pairs, to make a ifnal evaluation of the models and compare them between each other.
Metric Accuracy Precision Recall F1-Score
Full Deep Learning 83.03% 86.96% 77.92% 82.20%
Ontology-based 62.76% 61.29% 70.32% 65.50% 4.3.3. Results for binary classification 4.3.2. Deep Learning Recommender System We use as a baseline a full deep learning recommender system that we trained using BERT [6]. We use the Transfomers library from Hugging Face [7] for implementation. The pairs were fed to the model using the following structure : <cls>Resume<sep>job Ofer<sep>. <cls> is the classification token, it indicates the position at which the model makes its prediction (i.e. does the resume and the job ofer match). The resume and job ofer in the input are the list of skills they contain separated by a comma followed by a space (", "). <sep> is the separator token, it indicates the end of a document (resume/job 4.4. Weighting evaluation ofer).
We used the BertForSequenceClassification model from We test the impact of the weighting paradigm introduced the Transformers library which use a cross entropy loss in section 3.3 on the accuracy by randomly generating when the number of labels is 1. It was trained with a a thousand weighting for each pair, and measuring the batch size of 16 until the accuracy on the dev set stopped portion of those weightings that led to a good prediction. showing any improvement for 3 consecutive epochs. The In the case in which the expert always finds the perfect best model (3 epochs prior) was kept. We used the op- configuration regarding the importance of each skills in timization function AdamW [8] with an initial learning the job ofer, the accuracy may rise up to 87%. Obviously rate of 5e-5. it is unlikely even an expert would always find the one in a thousand best weighting, but we could easily expect We now have 2 Job Recommender Systems which can be compared on the same dataset. The results in table 2 are in favor of Deep Learning with a strong accuracy of 83.03% higher than the Ontology-based one, which is only 62.76%. This result is not surprising as the ontology based similarity distances cannot compete with deep learning based ones. However, as we show below, one can obtain much better figures through conversational interactions.
the ofer is the inverse of its frequency among recommended resumes (equation 6). • Increase the weight of a skill: for each skill in the correct resume and in the ofer, the probability to increase its weight, is also the inverse of its frequency among recommended resumes (equation 6). • Decrease the weight of a skill: for each skill in the ofer but not in the correct resume, the probability to decrease its weight is its frequency among recommended resumes (equation 7). /() =
|| | ∈ ∧ ∈ | + 1
(6) () = | ∈ ∧ ∈ | (7)
||
At each step/turn, we only apply the action with the highest probability.
We also evaluated our conversational recommendation w.r.t. the following metrics: success rate, low rank rate, average number of turns, see Sun et al. [9]. For a list of 100 ofers and 100 resumes, and = 5 recommendations per step/turn, we achieve a 74% success rate, 8.69 average turns/steps, and 26% low rank rate.
him to do better than average. Empirical testing would be required to evaluate how and with which accuracy a recruiter would find a good weighting.
We take a subset of 100 pairs of ofers with the right resume from the test set, and for each ofer we recommend resumes, the set of recommended resumes is denoted by . If the correct resume is recommended it’s a success, otherwise we take a step to fine-tune the recommendation until it is recommended or we reached the maximum number of steps (100).
In the absence of real users, We adopted heuristic techniques to try to mimic human reasoning. The actions that can be taken are the following:
way:
There are a number of systems that try to support the
recruitment process through recommendations. Surveys
of these methods include [
More recent proposals rely on bilateral preference tional recommendation technique is a suitable approach. matching methods [13], [18] or on deep learning [19], [20], [21]. A major shortcoming of machine learning and 5.2. Conversational recommendation deep learning based techniques in this context is the lack of interpretability. This poses even an ethical problem Conversational recommender systems (CRS) [2] rely on in this context. Even if recruitment decisions are finally interactions with the users to compute recommendations. made by humans, recommendations that influence the This exchange with the user can take the form of a conprocess are not interpretable and they are not transparent. versation in natural language, but isn’t limited to this The usage of ontologies inside Recommender Systems modality. For example, Yo et al. [26] use visual dialogues isn’t new, they can be used to overcome the cold-start in the conversation. CRSs often realize the following the problem and improve accuracy, by displaying the profile following phases: Request, Recommend, Explain, and of a user in a human-readable format he can easily edit Respond. In our work we adopt the conversational recto fine-tune his recommendations [22]. ommendation process. In our context, the Request cor
The article [23] surveys explainable recommendation responds to the situation where the recruiter starts the techniques [23]. One can distinguish model-intrinsic search for candidates and asks for potential resumes. He methods and model-agnostic methods. For model-intrinsic can obtain recommendations, together with complementechniques the decision mechanism is transparent, one tary information that serves as explanation as to why can derive explanations directly for the model decisions, a given candidate was recommended. Based on these while for model-agnostic (or post-hoc) explanations the pieces of information the user can respond, refine or recommendation decision is black box, and explanations restart the search for candidates. The excellent survey try to give some insights event if they are not directly paper [2] highlights a number of application areas of CRS, derived from the model. Our method is model-intrinsic, but to the best of our knowledge they have not yet been we obtain explanations directly from our recommenda- applied to the recruitment domain. tion model. This choice has a price: one could potentially obtain more accurate recommendations with the help 6. Discussion and Conclusion of deep learning based methods, but then we could not have transparent model-intrinsic explanations. An in- We have proposed a job candidate recommendation techterpretable job-person recommendation, with post-hoc nique based on conversational recommendation princiexplanations was presented in [24]. ples. Our system helps identifying suitable candidates,
Recommender systems (for movies, products, services, based on the skills in the resumes and their relation to
etc.) try to exploit and reconstruct the user’s prefer- the required skills in the job ofer. Our recommendation
ences, based on some signals or patterns in the data. Yan is completely transparent and for each recommendation,
et al. [