This paper describes the participation of the Unidad Docente de Informática Industrial at the Universidad Politécnica de Madrid (UDII-UPM) in TalentCLEF 2025, a competitive evaluation campaign focused on Human Resources. We addressed Task A: Multilingual Job Title Matching, which aims to develop systems capable of identifying and ranking job titles that are most similar to a given query title. Our approach is based on the fusion of semantic representations generated by two diferent multilingual embedding models. First, we compute pairwise similarities between query and candidate job titles. Then, we apply a weighted combination of the resulting similarity scores to produce a final ranked list of job titles for each query. The proposed method is simple, language-adaptable and does not require fine-tuning. Oficial results confirm that our system is competitive although there is still room for improvement in the ranking quality and model alignment across languages.
In this section, we give an overview of the Job Title Matching dataset, the methods used, how the run delivered was set up, and the evaluation metric used in the lab.
The corpus [
The validation data are divided into four languages, the three in the training data and Chinese. This set contains: i) a file that contains the job title used as a query and the associated id ( queries); ii) the job title in the corpus and its id (corpus_elements) and iii) the relationships between the query and the corpus elements (qrels). The statistics of the validation part of the dataset are shown in Table 2.
The test data has four languages, as the validation set, and the provided files: i) contains the job title used as a query and the associated id (queries) and ii) contains the job title in the corpus and its id ( corpus_elements). The statistics of the test part of the dataset are shown in Table 3.
This study uses two learning algorithms as a starting point for future work. In particular, the following
algorithms have been used:
• Masked and Permuted Language Modeling (MPNET) [
These models were used via HuggingFace2
Our proposal consists of the weighted combination of the similarities produced by the models mentioned in section 2.2. To achieve this, each model individually generates embeddings for the provided texts. Then, a similarity matrix is computed between the query texts and the corpus elements, and these similarities are fused according to the following equation: _ = × _ + (1 − ) × _ (1) where, are the fusion weights, empirically determined for each language, described in Table 4; similarity_mpnet is the similarity between queries and corpus texts using mpnet embeddings and similarity_distiluse is the similarity between queries and corpus texts using distiluse embeddings.
Finally, the output ranking of documents is generated by sorting them in descending order of similarity.
To evaluate our approach during the development phase, we used the oficial evaluation script provided by the organizers.3 The final performance was assessed using the Mean Average Precision (MAP) metric [6].
2https://huggingface.co/ 3https://github.com/TalentCLEF/talentclef25_evaluation_script
Based on the results, our system consistently obtains better results than the baseline; however, these results are far from the system that achieved the best results. This may happen because our system does not perform a training step prior to the generation of the embeddings so the methods used here are not adapted to the task. Also, as can be seen from the results, the language with which we have encountered the most problems is German, which may be due to the fact that it is the language with the most number of queries and corpus texts. If we observed the rest of the oficial results provided in Table 5 the other systems also had problems with this language.
In this work udii-upm team provides information of its participation at TalentCLEF 2025, Task AMultilingual Job Title Matching. The system presented in this task provides a good starting point for future developments which includes: i) train the models with the training data provided in the corpus; ii) explore in depth the problems presented in the German language in order to improve the results and iii) to explore of monolingual models instead of multilingual ones.
During the preparation of this work, the authors used GPT-4o mini in order to: Grammar and spelling check. [6] H. Schütze, C. D. Manning, P. Raghavan, Introduction to information retrieval, volume 39, Cambridge University Press Cambridge, 2008.