Recent advances in large language models (LLMs) have enabled a wide range of applications across diferent domains, including human resources (HR). A key challenge in this space is identifying relevant skills for a given job title. This paper details our approach developed for the TalentCLEF 2025 shared task on job-skill matching. We explore various AI-based strategies, with a significant focus on leveraging LLM-driven data augmentation to enhance model training. Our methodology evaluates cross-encoder models and sentence similarity architectures utilizing diverse pretrained models, analyzing their trade-ofs in mean Average Precision (mAP) and computational eficiency. Our final system achieved a mAP of 0.345 on the test set. We also discuss insights from contrasting experiments, including those less successful, to highlight modeling limitations and ofer recommendations for future improvements in this domain. This work contributes to the TalentCLEF 2025 shared task's aim to eficiently align occupational titles with relevant professional skills.
Job-skill matching is a fundamental task in modern human resources (HR) and talent management. With job postings often attracting thousands of applicants, there is a growing demand for automated systems that can eficiently and accurately match candidates to job requirements. A key component of this automation is identifying and ranking the most relevant skills for a given job title, a challenge actively being addressed by research and shared tasks in the field. Recent advances in natural language processing (NLP), particularly large language models (LLMs), have opened new avenues for building intelligent HR systems. These technologies have the potential to power applications such as skill-based candidate ranking, personalized up-skilling recommendations, and early detection of emerging skill gaps. However, deploying such systems in real-world settings involves critical challenges related to scalability, semantic ambiguity, and fairness, which this work aims to explore.
The Job Title–Based Skill Prediction task, a part of the TalentCLEF 2025 lab within the Conference and
Labs of the Evaluation Forum (CLEF) [
This task utilizes a dataset derived from the ESCO (European Skills, Competences, Qualifications,
and Occupations) database [
• Skill names file: Provides multiple names and aliases for each skill ID.
Each job and skill ID corresponds to a unique URL in the ESCO taxonomy. For validation and testing, the data is structured as follows: • A skills corpus file: Lists all candidate skill entries (with their names and aliases). • A job queries file: Provides a list of job titles for which skills need to be ranked. These job titles were specifically selected by domain experts, which may result in a data distribution diferent from the training set. • A ground truth file: Contains the relevance labels, represented as pairs of job and skill indices that reference their respective positions in the queries and corpus files.
This formulation simulates a real-world retrieval scenario where a system must match a job title query against a predefined skills corpus to identify the most relevant skills.
This task presents several key challenges. First, scalability is a major concern: matching hundreds of job titles with hundreds of candidate skills quickly becomes computationally expensive, particularly when using cross-encoder models that require evaluating every possible pair, resulting in quadratic complexity. Second, accessibility must be considered. High-performing solutions should ideally run on afordable hardware to ensure practical adoption in real-world HR settings. Third, semantic ambiguity and lack of context make it dificult for models to make accurate predictions based only on short job titles or skill names. This issue afects both humans and machines. Finally, while LLM augmentation can efectively provide additional context, its deployment is often constrained by significant computational costs due to the need for high-end GPUs or reliance on expensive APIs. Designing a solution that balances contextual reasoning and computational eficiency is therefore non-trivial.
In this study, we investigate both cross-encoder and bi-encoder (sentence similarity) approaches for the job–skill matching task, evaluating their trade-ofs in performance and eficiency. To address the lack of input context, we experiment with augmenting job descriptions using lightweight LLMs that do not require high-end hardware. Our experiments include successful and unsuccessful attempts, with all results documented to inform future work in this area.1
Encoder models are central to job–skill matching tasks, as they transform textual input, such as job titles and skills, into dense vector representations that capture semantic similarity. In our experiments, we evaluate several pretrained encoder models, each with diferent domain characteristics and training strategies.
Zhang et al. fine-tuned XLM-RoBERTa [
Wang et al. introduced the E5 model family [
Reimers and Gurevych developed JobBERT [
LLMs can complement encoder-based systems by generating additional context or augmenting inputs
with richer descriptions. While many LLMs are resource-intensive, recent open-weight models have
improved accessibility. In our experiments, we use LLaMA 3.1 8B [
The first approach centered on a cross-encoder architecture. In this setup, a job title and a candidate skill were concatenated, typically separated by a special [SEP] token, and fed as a single sequence into a pretrained transformer encoder. For extracting a final representation for classification, we experimented with two common strategies: using the final hidden state of the [CLS] token, and applying mean pooling over the last hidden layer’s outputs (respecting attention masks).
This pooled representation was then passed through a linear projection layer followed by a sigmoid activation to predict a relevance score between 0 and 1. The core idea was that the cross-attention mechanisms within the encoder would allow for deep contextual interaction between the job and skill, enabling the model to learn diferent nuanced relevance signals. The intended advantage for this design was better modeling of nuanced job-skill interactions. However, while conceptually powerful, this approach sufered critically from computational ineficiency. Evaluating each job-skill pair independently resulted in quadratic inference complexity (( · ) where J is the number of jobs and S is the number of skills). With even a moderately sized test set, this led to prohibitive processing times. Despite showing some promise in capturing relevance, achieving an initial mean Average Precision (mAP) of approximately (0.23-0.26) after 1-5 epochs of fine-tuning with cross-entropy loss across diferent encoders (ESCOXLM-R, E5-Large, and E5-Instruct), its lack of scalability made it impractical as a standalone solution for the full task.
Our key takeaway from this experiment was that while the cross-encoder approach efectively captures relevance through deep job-skill interactions through attention mechanisms, its significant computational expense at both training and inference makes it more suitable for re-ranking a smaller set of candidates rather than for initial large-scale retrieval. Due to time and resource limitations, we decided not to move forward with this approach.
The second exploratory direction utilized a bi-encoder architecture, focusing on sentence similarity. Here, job titles and skills were encoded into dense vector embeddings independently using a shared pretrained encoder. The relevance score for a job-skill pair was then computed using the cosine similarity between their respective embeddings.
The intended advantage for this approach was its high computational eficiency, as job and skill embeddings could be pre-computed and stored, allowing for fast nearest-neighbor searches or similarity calculations. Although significantly more eficient, this approach often struggled with the lack of explicit interaction context. When relying solely on the often concise raw job titles and skill names, the encoders found it challenging to disambiguate meanings or infer relevance accurately without the direct comparison ofered by cross-attention. This was reflected in its performance, resulting in a lower mAP of approximately (0.13-0.22) across diferent encoders (ESCOXLM-R, E5-Large, E5-Instruct). It is important to note that these bi-encoder results reflect zero-shot performance using the base pretrained encoders without any task-specific fine-tuning at this exploratory stage.
Our key takeaways from this experiment are that bi-encoders are eficient but require either highly efective base encoders adept at capturing nuanced semantics from short texts or, more critically, richer input representations that provide more context for the individual job and skill texts.
These initial explorations highlighted a key trade-of: the deeper contextual understanding of crossencoders came at a steep computational price, while the eficiency of bi-encoders was often undermined by a deficit in contextual awareness when using their base pretrained representations and basic textual inputs. This understanding directly motivated our subsequent eforts to develop a more balanced and efective methodology, focusing on enhancing the input to bi-encoder systems, as detailed in the following section.
To address the context limitations of the bi-encoder design and improve its performance, we experimented with several approaches to enrich job and skill representations, as detailed below.
ESCO’s taxonomy database contains not only information about jobs, skills, and their relations, but also detailed descriptions for each job and skill entry. These descriptions represent an excellent opportunity for contextual enrichment, being reliable, accurate, and directly relevant to the entities the encoder processes.
However, this approach quickly encounters a challenge with the validation and test sets. While the skills in these sets are from ESCO (and thus have descriptions), the job titles are not directly from ESCO. Instead, they are selected by domain experts and lack a direct mapping or URL to an equivalent ESCO occupation (even if such an equivalent might conceptually exist).
To overcome this, our strategy involved training models with the available ESCO descriptions and then, for validation and testing, attempting to find the closest matching job title from the training set (which has ESCO descriptions) and using its description as a proxy.
We used two distinct methods to find these close matches. First, we tried Python’s get_close_matches function from the Diflib library. This function performs sequence-based text matching, identifying the closest n matches for a given string (a validation/test job title) from a list of strings (training set job titles) based on a similarity cutof score. This approach, while efective, was relatively slow but could be performed ofline and prepared in advance.
The second method utilized JobBERT. This was a conceptually strong approach, as JobBERT is specifically pretrained for job title similarity. We first generated embeddings for all job titles from both the training set and the validation/test sets using JobBERT, then for each validation/test job title, we identified the most similar training set job title via cosine similarity ranking. This method also benefited from being significantly faster due to eficient batch processing and parallelization capabilities for embedding generation.
We evaluated this proxy-description augmentation technique with various encoders. A key observation across both Diflib and JobBERT-based matching methods was that the model had the highest score when using a very high similarity cutof (around 0.97). This strict cutof meant that approximately 95% of the validation set job titles did not receive a proxy description from the training set, as no suficiently close match could be identified. However, using descriptions only for the matched 5% still outperformed using no descriptions at all or using a lower cutof (which would introduce more, potentially less accurate, descriptions). This finding strongly signaled that while accurate descriptions were crucial, having no description was preferable to having an inaccurate one. This insight provided a strong motivation for exploring LLM-based description generation, as discussed next.
To generate informative descriptions, particularly for potentially ambiguous job titles, and ensure system accessibility, we needed a high-performing LLM capable of running on a single consumer-grade GPU. We opted to use LLaMA 3.1 8B instruct. We engineered the prompt with the following objectives: conciseness (to fit encoder context windows), clarity (to avoid superfluous tokens or hallucinations), inclusion of 2-3 relevant skills (to directly enhance job-skill similarity signals), direct output of only the description (omitting conversational preambles), and avoidance of redundant information already present in the job title. We worked with the following prompt:
You are an expert HR assistant specialized in writing job descriptions. Generate a highly concise, professional, and factual job description based ONLY on the provided job title. Focus on typical key responsibilities. Crucially, explicitly list 2-3 common, essential skills associated with the role within the description, often towards the end. Do not add information not directly implied by the job title. The description must be brief, ideally 2-3 sentences maximum, to fit within processing limits. Output only the description text, without any preamble or introductory phrases.
We generated descriptions for all job titles in the training, validation, and test sets. For the training set, this was done despite the availability of oficial ESCO descriptions to ensure consistency in the style and focus of descriptions across all data splits; LLM-generated descriptions might emphasize diferent aspects than the ESCO originals, and training on a consistent distribution was deemed important. Furthermore, we generated a unique description for each job title alias, even those mapping to the same underlying ESCO job ID. This was to prevent the encoder from potentially learning to disregard the specific alias text and overly rely on identical descriptions if they were shared.
Our exploration into enriching context, initially with oficial ESCO descriptions and then more efectively with LLM-generated content, aimed to provide richer inputs for the encoders. We used LLMs to generate descriptions for jobs for training, validation, and test sets. For skills, we used the available ESCO descriptions as they were available across training, validation, and test sets. The final input text to the encoders would be: • For jobs: "Job: (job title) [SEP] Description: (LLM generated description)" • For skills: "Skill: (skill name) [SEP] Description: (oficial ESCO description)"
For E5-Instruct, we include an instruction prompt to make use of the model’s instruction tuning. The input was adjusted to the following: • For jobs: "Instruct: Given a job title and its description, retrieve relevant skills based on their descriptions. Query: Job: (job title) [SEP] Description: (LLM generated description)" • For skills: "Skill: (skill name) [SEP] Description: (oficial ESCO description)"
Our final system uses a bi-encoder architecture. Pretrained transformer models serve as the backbone to independently encode job and skill textual representations. These encoders are fine-tuned using a contrastive learning approach. At inference, the relevance of a skill to a job is determined by the cosine similarity between their respective vector embeddings, allowing for eficient ranking. This architecture was selected to balance predictive performance with the computational eficiency required for potentially large-scale matching, informed by our initial explorations in Section 3.1.
The textual inputs for jobs and skills were combined with their descriptions to provide rich contextual information to the encoders, as detailed in Section 3.2.3.
We evaluated three distinct pretrained transformer models as the backbone for our bi-encoder: ESCOXLM-R, E5-Large, and E5-Instruct. These models were fully fine-tuned. For each input text, the dense vector embedding was derived by mean pooling over the last hidden layer’s token embeddings, respecting attention masks.
The bi-encoder models were fine-tuned using Multiple Negatives Ranking Loss (MNRL) following
Henderson et al. [
A comprehensive list of hyperparameters can be found in Table 1. We used the AdamW optimizer
[
During inference, all job texts and skill texts along with their generated descriptions are first encoded using the fine-tuned bi-encoder to generate their respective dense vector embeddings. For a given query job, its embedding is compared against all skill embeddings in the target corpus using cosine similarity. Skills are then ranked for that job in descending order of their similarity scores.
Our best model achieved a mAP score of 0.345 on the oficial TalentCLEF test set. In this section, we present diferent encoder models’ results on the validation set.
Our experiments reveal several key insights into building efective job-skill matching systems. The ifnal fine-tuned bi-encoder architecture, particularly when using E5-Instruct with LLM-generated job descriptions, demonstrated strong performance, achieving 0.313 mAP on our validation set and generalizing well to the oficial TalentCLEF test set with a mAP of 0.345.
The E5 model family, especially E5-Instruct, consistently outperformed ESCOXLM-R in our finetuned setups (Table 2). The E5 models’ robust performance, even with minimal context (as seen in the "No Descriptions" fine-tuned column), suggests their large-scale diverse pretraining provides a strong foundation. The instruction tuning of E5-Instruct likely further aided its ability to focus on the retrieval task when presented with augmented inputs. The comparatively lower scores for ESCOXLM-R, despite its ESCO-specific pretraining, were notable. The largest performance gap between ESCOproxied descriptions and LLM-generated descriptions was observed for ESCOXLM-R (an increase from 0.2689 to 0.298 mAP). This could imply that ESCOXLM-R is highly sensitive to the style and ESCOXLM-R
E5-Large
E5-Instruct 5.2. Discussion completeness of contextual information; when most validation job titles lacked a high-quality ESCO proxy description, its performance sufered more acutely than the E5 models. Conversely, it benefited more when comprehensive LLM descriptions were provided for all titles. This suggests that while domain pretraining is valuable, adaptability to varied descriptive styles, or a consistent supply of rich context, is equally crucial.
An important finding is the significant improvement from zero-shot performance (e.g., 0.2197 for E5-Instruct) to fine-tuned performance with LLM descriptions (0.313 for E5-Instruct). This suggests that while pretrained encoders have general semantic understanding, task-specific fine-tuning is essential for optimal retrieval. The LLM-generated descriptions, which explicitly included 2-3 relevant skills as per our prompt, likely created stronger semantic anchors for the bi-encoder, directly facilitating better job-skill similarity assessments. The guarantee of having a description for every job title via LLM generation, regardless of its presence in ESCO, was a key advantage over the proxy-based ESCO description approach. Our final fine-tuned bi-encoder system also surpassed the performance of our initial cross-encoder explorations (0.23-0.26 mAP), demonstrating a more scalable and ultimately more efective solution.
Despite these results, limitations exist. Our choice of LLaMA 3.1 8B for generating job descriptions balanced performance with accessibility; larger LLMs might ofer further improvements at a higher computational cost. The prompt guiding this LLaMA 3.1 8B generation process, though iterated upon, could be further refined. Similarly, the instruction prompt prepended to the input for E5-Instruct, while simple and efective, was not exhaustively optimized and could be a subject for further experimentation. Additionally, our hyperparameter search for fine-tuning, while guided by common practices, was not exhaustive due to resource constraints.
In this work, we explored various approaches for the TalentCLEF job-skill matching task, systematically evaluating diferent system architectures, encoder models, input augmentation strategies, and finetuning techniques. Our findings highlight the significant impact of rich contextual input (particularly through LLM-generated job descriptions) and task-specific fine-tuning (using MNRL) on the performance of bi-encoder models. Our most efective system utilized a fine-tuned E5-Instruct bi-encoder, augmented with LLM-generated job descriptions and oficial ESCO skill descriptions. This approach achieved a mAP of 0.345 on the oficial TalentCLEF test set, demonstrating a robust and scalable solution.
For future research, several avenues warrant exploration. Investigating the use of larger or more specialized LLMs for job description generation, as well as experimenting with LLM-generated descriptions for skills, could further enhance input representations. Additionally, more extensive hyperparameter tuning for the fine-tuning process and dedicated experimentation with more sophisticated instruction prompts for E5-Instruct may unlock additional performance. Finally, exploring a two-stage hybrid architecture, where our eficient fine-tuned bi-encoder retrieves an initial set of candidates subsequently re-ranked by a more computationally intensive cross-encoder, could ofer a compelling balance of eficiency and accuracy. These directions could build upon the insights gained in this study to further advance the state-of-the-art in job-skill matching.
During the preparation of this work, the author used Gemini 2.5 Pro in order to: Paraphrase and reword, Grammar and spelling check. After using this tool, the author reviewed and edited the content as needed and takes full responsibility for the publication’s content.