Finding the most relevant person for a job proposal in real time is challenging, especially when resumes are long, structured, and multilingual. In this paper, we propose a re-ranking model based on a new generation of late cross-attention architecture, that decomposes both resumes and project briefs to eficiently handle long-context inputs with minimal computational overhead. To mitigate historical data biases, we use a generative large language model (LLM) as a teacher, generating fine-grained, semantically grounded supervision. This signal is distilled into our student model via an enriched distillation loss function. The resulting model produces skill-fit scores that enable consistent and interpretable person-job matching. Experiments on relevance, ranking, and calibration metrics demonstrate that our approach outperforms state-of-the-art baselines.
The application of machine learning to Human Resources
(HR) data has led to significant advances in tasks such as
career path prediction [
A common approach for such systems is a two-stage
pipeline consisting of a retrieval and a ranking phase. Our
work focuses on the latter, where creating an efective and
scalable model presents several challenges. First, freelancer
profiles and project briefs are often long, structured
documents written in multiple languages. While lexical
matching methods fall short in capturing deep semantic meaning,
many transformer-based models that excel at this are
limited to short input lengths or require significant
computational resources not suitable for real-time inference. Second,
the performance of supervised learning-to-rank models is
highly dependent on the quality of training data. Real-world
recommender systems often produce sparse, biased
interaction data [
Beyond the challenges of training on biased data, neural
ranking models present another critical limitation in HR
contexts: their outputs frequently lack the global score
calibration that is essential for interpretability and consistent,
query-invariant comparisons. A promising strategy to
instill these desired properties while maintaining eficiency
is knowledge distillation [
To address all of these challenges, we propose a novel framework that distills the semantic reasoning of a generative Large Language Model (LLM) into a lightweight, eficient, and calibrated ranking model. We use the LLM as a "teacher" to generate fine-grained and semantically grounded relevance scores, creating a robust supervisory signal that overcomes the limitations of biased historical data. Our contributions are twofold: • A Distillation Framework for a Calibrated and Interpretable Semantic Score: We introduce a distillation framework to produce a relevance score that is both interpretable and semantically calibrated. First, to overcome the limitations of biased and sparse historical interaction data, our method uses a generative LLM to provide a semantically calibrated relevancy score. Second, we combine a distillation loss sensitive to both ranking and score magnitude with direct score supervision to improve calibration. Overall, This ensures that the final score produced by our model has a consistent meaning across diferent freelancer-project pairs, making it suitable for ranking and for use in downstream business applications. • A Lightweight, Long-Context Reranking Architecture: We propose an eficient student model that processes long-form profiles and project briefs by decomposing them into structured utterances. Its cross-attention comparison block, inspired by late interaction mechanisms, efectively captures finegrained semantic alignment while remaining computationally inexpensive for real-time inference.
The remainder of this paper is organized as follows. Section 2 refines our problem statement and reviews related work on HR recommendation systems and semantic ranking algorithms. Section 3 presents in depth our dataset creation and ranking algorithm alongside the training objective. Section 4 describes the experimental setup and results. We conclude in Section 5 with future directions.
In HR applications, ranking or re-ranking is used to refine
candidate-job matches after an initial retrieval stage,
aiming to surface the most relevant candidates eficiently. One
approach to candidate-job ranking uses single-stream
comparison models, from Recurrent Neural Networks (RNNs)
with attention [
Driven by real-time constraints in deployment, bipartite
architectures (bi-encoders) became popular. These encode
candidates and jobs separately before applying a similarity
function. Early implementations relied on convolutional
encoders and contrastive training [
To go beyond independent encoding, graph-based
methods capture richer structural relationships. Some use
external knowledge graphs to learn job and candidate
embeddings [
Recent attention-based comparison models reintroduce
interaction layers, similar to earlier single-stream designs, to
capture fine-grained alignments between job requirements
and candidate profiles, improving ranking accuracy [
Generative large language models (LLMs) have recently
emerged as competitive zero-shot rerankers, for large
multilingual documents, by leveraging their capacity to
reason over document-query pairs in natural language.
Unlike dense retrieval or cross-encoders trained on annotated
datasets, generative LLMs can predict ranking permutations
directly [
Distillation from Generative LLMs. To address the
impracticality of using directly generative LLMs in production,
they can serve as teachers in a distillation setup—generating
either synthetic training data (e.g., queries or candidate
items [
Ranking Calibration. In ranking systems, score
calibration is critical when downstream tasks rely not just on
item orderings but also on the absolute values of predicted
scores—for example, in multi-stage retrieval, risk-aware
decisions, or interpretability of results. However, most ranking
losses do not enforce calibration, often producing scores
that lack a consistent scale [
Let P and F denote the sets of all projects and freelancers, respectively. Each project ∈ P is represented by a brief document , and each freelancer ∈ F by a profile . These documents are composed of structured textual sections denoted ,, where ∈ {P, F} and ∈ refers to a document-specific section type. Thus, each document is defined as:
= {, | ∈ }.
Our goal is to obtain a model with parameters that
estimates a continuous skill fit relevance score , ∈ [
( , ) = ,.
This score is intended to reflect the skill-fit between a freelancer and a project, enabling skills based ranking such that: ∀, ′ ∈ F, , > ′, ⇒ ≻ ′.
Beyond ranking accuracy, we aim for the scores to be interpretable and comparable across diferent project–freelancer pairs, i.e., semantically calibrated. This enables to use them as meaningful features for downstream applications such as explanation, policy decisions, or business-level ranking.
Let ℛ be a fixed reference set of matching interpretations
(e.g., “unqualified,” “partial match,” “strong match” , ...), and let
: [
This ensures that identical scores carry consistent meaning across contexts, facilitating explanations and cross-project comparability.
To obtain such calibrated scores, we assume the existence of a model ′ that assigns a relevance category ∈ ℛ to any freelancer–project pair (, ):
′ (, ) = .
Conceptually, an approximate inverse mapping ˜− 1 :
ℛ → [
˜− 1() = E∼ (|()=)[].
In results, a teacher model teacher can be defined as : teacher(, ) = ˜− 1( ′ (, )), which outputs semantically calibrated scores grounded in the interpretation space ℛ.
To make inference scalable, we train a compact student model student that mimics teacher: student(, ) ≈ teacher(, ). (2)
This distillation process, illustrated in Figure 1, enables us to preserve the hypothetic semantic calibration and interpretability of teacher-generated scores in a lightweight model suitable for real-time deployment.
Having outlined our general approach and the necessity of semantic calibration, we now detail our methodology for obtaining calibrated training data.
In the absence of high-quality, semantically calibrated
data, we propose leveraging generative large language
models (LLMs) as teachers to generate skill-fit supervision
signals. Indeed, as stated in the introduction (Section 1),
our historical data is mostly sparse, noisy, biased and
uncalibrated. In contrast, a generative LLM can evaluate
tasks within a defined context, allowing us to create more
ifne-grained and semantically accurate supervision signals,
denoted as , ∈ [
To generate these scores, the following context is defined
in the model’s prompt:
" You are an objective assistant in a freelancer-job
matching platform. Given a job description and several freelancer
profiles, evaluate each freelancer’s suitability for the job.
Provide a concise reasoning and a score between 0 and 1 for each
freelancer. "
Within this context, the model is instructed to return both
a score and a reasoning justification which has been shown
to provide better results for complex tasks [
To ensure semantic calibration, the prompt also contains a predefined matching interpretation set ℛ, aligning each score with a relevance category : Listing 1: Reference list within the teacher model’s prompt 0.0: No relevant skills or experience. Completely unable to perform the job. 0.2: Minor relevance. Few matching skills or limited experience.
High chance they will be unable to perform the job. 0.4: Moderate match. Some relevant skills or experience. Would probably not be able to do the job. 0.6: Good match. Mostly relevant skills and experience. Can perform with some ramp−up. 0.8: Strong match. Highly relevant skills and experience. Ready to perform well. 1.0: Perfect match. Skills and experience fully aligned with job needs. Expert on the topic.
By inserting this predefined mapping directly into the prompt design, the LLM is able to select a score that best represents the skill-fit between the freelancer and the job, according to the defined semantic categories.
To promote better instruction understanding and score consistency, the LLM is prompted with twelve freelancer profiles per project, with at least one " unsuitable" and one "perfect". While each candidate is scored independently, batching them in the same prompt encourages the model to improve the score quality, without compromising semantic interpretability.
While this approach allows to obtain the expected semantically calibrated relevancy score, it remains computationally and environmentally expensive for large-scale inference. To overcome such limitations, a lightweight student model can be designed to approximate the LLM’s output, as detailed in the following sections.
To retain the semantic abilities of the teacher model within a more compact one, the proposed architecture for the student model, student, illustrated in Figure 1, is composed of two main components: 1. Document Encoders: One encoder branch for each document type (i.e., project briefs and freelancer profiles), as shown in Figure 2. 2. Comparison Block: An attention-based comparison module, detailed in Figure 3.
The following sections describe each of these components in detail.
To encode documents, we build upon our previous work on
retrieval models [
Unlike our previous token-level model, each section , is segmented into minimal textual units that are referred to as utterances
, = {,, | = 1 · · · ,}.
This strategy aligns with the intended use of sentence-BERT
models [
Utterances are defined diferently depending on the section type: • For paragraph-based sections (e.g., descriptions), utterances correspond to individual sentences. • For tag-based sections, each tag is treated as a separate utterance.
• Titles are encoded as single-utterance sequences.
Each utterance ,, is processed by the pre-trained sentence encoder backbone and enriched with a learned categorical encoding categorical specific to section type . The resulting vector is passed through a linear layer: ,, = · (︀ Backbone(,,) + categorical ︀) + . (4) This categorical encoding helps preserve structural information. While the sentence encoder backbone remains frozen, both the categorical encoding and the projection layer are trained, efectively adapting the general-purpose encoder to our domain-specific skill matching task.
Thus, each document ( or ), is ultimately represented by a sequence of utterance embeddings: = {, | = 1 · · · = {, | = 1 · · · ∑︁ ,}, ∈ ∑︁ ,}, ∈ (5) where , and , denote the number of utterances per section and document.
This utterance-based encoding strategy significantly reduces the computational cost of processing long documents. Since most of the computational burden lies in encoding, the backbone’s utterance embeddings can be pre-computed and cached to accelerate training. In production, can be computed and stored in advance, leaving only the project’s utterances to be encoded at inference time, along with the ifnal comparison block, presented in the next section.
Inspired by the late interaction mechanism [
First, context-aware embeddings context and context are derived using cross-attention. Then, similarity distributions , and , are computed between the original embeddings and their respective context vectors.
To compute context , multi-head attention from the brief to the profile embeddings is applied:
MultiHead(, , ) = head1 ‖ · · · ‖ head
(6) = context , where each attention head is defined as: head = softmax ︃( ( )⊤ )︃ √ . (7) This results in a new sequence context of the same length as , where each embedding context, reflects the best-matching combination of profile content for the corresponding brief utterance ,.
To assess how well this profile context aligns with the original brief, their pairwise cosine similarities are computed:
⎧ ⃒⃒ , = ⎨sim(,, context, ) ⃒⃒⃒ = 1 · · · ⎩ ⃒ ⎫ ∑︁ ,⎬ . (8) ∈ ⎭ This results in a similarity distribution with one score per brief utterance.
Symmetrically, the process is reversed to account for mutual interest. First, the profile is attended to the brief : MultiHead( , , ) = context , (9) then the corresponding similarity distribution is computed: ⎧ ⃒⃒ , = ⎨sim(,, context, ) ⃒⃒ = 1 · · · ⎩ ⃒⃒ ⎫ ∑︁ ,⎬ . (10) ∈ ⎭
These distributions, , and ,, provide a detailed view of skill alignment: how well a profile matches the most relevant parts of a brief, and vice versa. They form the basis for the final scoring step, described in the following section.
Since the similarity distributions , and , vary in length, ∑︀∈ , and ∑︀∈ , respectively, the original late interaction mechanism aggregates them using only a sum over brief-wise similarities to produce a score.
Instead, more expressive statistical pooling operations are computed over both distributions to produce fixed-size feature vectors more suitable for scoring. Specifically, we extract descriptive statistics from both , and ,, defined as desc() = [︀ min(), max(), (), (), 1(), 2()︀] , where denotes the mean, the standard deviation, 1 the skewness, and 2 the kurtosis (see Appendix A). These richer features are hypothesized to better capture the interaction dynamics between the documents’ utterances.
Finally, these descriptive statistics are concatenated with the averaged pooled embeddings from the two branches, and , as well as the averaged context embeddings context and context, forming the input to a multi-layer perceptron (MLP):
MLP ︃( desc(, ) ‖ ‖ context )︃ ‖ desc(,) ‖ ‖ context = , .
(11)
Adding the averaged pooled embeddings enriches the similarity distributions with contextual information from the documents.
Only the projection layers of the branches, the two multihead attention modules, and the MLP require training. These components are trained to approximate the semantically calibrated scores produced by the LLM teacher model via distillation, as described in the next section.
It is often possible to derive a binary relevance label from historical interaction data. We denote this indicator function as: , = {︃1 if freelancer is relevant to project ,
0 otherwise.
Let I be the set of all project–freelancer pairs for which such a label is available:
I = {(, ) ∈ P × F | ∃! , }, and let ⊂ I be the subset used for training.
When using these historical relevance labels, a standard
approach is to frame the problem as binary classification.
Hence, a model (, ) can be trained using common
classification losses such as Binary Cross-Entropy, Focal
Loss, or Asymmetric Loss [
In contrast, ranking distillation considers supervision from a teacher model that outputs continuous-valued scores rather than binary labels. One straightforward approach is to treat the teacher’s score (cf. eq.3) as a regression target for the student model score, (cf. eq. 11), which can be optimized using a mean squared error (MSE) objective:
To validate the use of teacher model scores as ground truth, we compared relevancy metrics across three settings: the teacher model, the student model trained on historical data, and the same student further trained on teachergenerated scores.
As shown in Table 1, knowledge distillation from Gemini-2.0-flash (cf. section 4 for details) improves the model’s ability to reject non-relevant candidates compared to training on historical binary labels. While this comes with a slight decrease in recall, the ranking quality remains comparable, suggesting that supervision via soft scores enhances discriminative capacity without sacrificing relevance. Furthermore, a qualitative evaluation, based on expert curation, supported the relevance and consistency of the generated scores. Overall, this setup provides access to ifner-grained signals, such as ranking quality and model interpretability, that are not directly measurable from historical binary labels alone.
However, the MSE formulation treats each (, ) pair
independently, ignoring the relative ordering between
candidates, which is central to ranking tasks. This point-wise
distillation approach may therefore be suboptimal for
ranking supervision [
To overcome the limitations of treating interactions independently, the relative ordering between two freelancers and ′ competing for the same project can be distilled. Specifically, the score diference given by the teacher model: ∆ ,,′ = , − ,′ ,
is used to train the student model to replicate this margin: ∆ ,,′ = , − ,′ .
This approach can be implemented using the Margin MSE
loss [
=′ = 1
∑︁ (,)∈ (,′)∈ 1[≻ ′] ︀( ∆ ,,′ − ∆ ,,′ ︀) 2 .
(13)
In the later, the indicator function ensures that only pairs where is relevant and ′ is not (for the same project) are included, and n is the number of such pairs. This formulation encourages the student model to preserve the relative ordering ≻ ′ induced by the ground truth.
An extension [
∑︁ (︀ ∆ ,,′ − ∆ ,,′ ︀) 2 . (14) (,)∈ (,′)∈ This loss encourages the student model to preserve both the ordering and the magnitude diferences between the teacher’s predictions. However, it does not enforce alignment of the absolute score values themselves.
To address this, we propose coupling the margin-based objective with a pointwise MSE regression loss (Eq. 12) computed on the teacher’s scores. This yields our combined loss: ℒCMMD() = ℒmargin_mse() + ℒMSE(), (15) which we refer to as the Calibrated Margin MSE Distillation (CMMD) loss. Empirically, this combination yields improved performance by aligning both relative and absolute semantics of the teacher’s signal
Another training strategy employs a more natural
listwise objective. List-wise losses are well-aligned with
ranking problems which aim to order a set of candidates
for a given project. Early methods such as ListNet [
In the CLID framework, the scores from both teacher and student models are normalized across the candidate set (i.e. all freelancer profiles associated with a given project ): ˆ, = ∑︀ , (,′)∈ ,′
and ˆ, = ∑︀
, (,′)∈ ,′ .
A cross-entropy loss is then applied to align these two score distributions: ℒCLID() = − 1
∑︁ || (,)∈ ˆ, log(︀ ˆ, ︀) (16)
CLID can be interpreted as aligning the probabilities of each freelancer being ranked above all others within the candidate set . These probabilities are derived from the normalized scores:
P( ≻ { ′}) = ˆ, and
P( ≻ { ′}) = ˆ, .
Alternative normalization strategies (such as computing
the probability of outranking only lower-scored candidates,
i.e., P( ≻ { ′ | , > ,′ }), as in ListMLE [
The following section presents our experiments, which assess the efectiveness of the proposed distillation strategy and student model architecture in generating semantically calibrated similarity scores between freelancers and project briefs suitable for large-scale deployment
First, we introduce the teacher model used to generate the semantically calibrated ground-truth scores, before defining our student’s model settings. Then, we compare our approach against a state-of-the-art re-ranking model and a small generative language model. For both baselines, we present results using publicly available pre-trained checkpoints, with and without fine-tuning for our specific use case.
employed the generative LLM Gemini-2.0-flash , supporting up to 1M input tokens. It provides structured responses with reduced latency, balancing performance and computational eficiency compared to Gemini-1.5 and Gemini-2.0-pro. Additionally, Gemini-2.0-flash ofers multilingual capabilities.
Arctic Embed [
baseline, we evaluate Qwen3 [
1B-parameter Gemma3 model [
Our corpus includes project briefs created between January 1, 2023 and April 15, 2024, along with historical versions of freelancer profiles that either applied or were rejected due to lacking skills. Profile representations are recomputed to reflect their state at the time of interaction. 1https://huggingface.co/Snowflake/snowflake-arctic-embed-xs 2https://huggingface.co/Qwen/Qwen3-Reranker-0.6B 3https://huggingface.co/google/gemma-3-1b-it
For evaluation, we reserve projects from January 1 to May 1, 2024, yielding a test set of 85K interactions between 8K projects and 78K profile versions. The training set contains 585K interactions across 55K projects and 520K profile versions.
To mitigate presentation bias from training solely on historical interactions, we augment the dataset with two types of negative examples. For average matches, additional freelancers are scored with Gemini 2.0 and those with a score between 0.4 and 0.6 are retained, adding 265K interactions involving 256K additional profiles. For unsuitable matches, profiles not having any job category in common with the project are randomly sampled at batch time.
All supervision signals, both for training and evaluation, are derived from Gemini 2.0, as described in Section 3.1. These scores are used as ground-truth relevance labels in our experiments.
Our Architecture. We train our architecture using three supervision strategies: point-wise, pair-wise, and list-wise. For clarity, we report only the best-performing objective for each.
Point-wise training uses the standard mean squared error loss (MSE), as defined in Eq. 12, denoted ℒMSE.
Pair-wise training employs our combined margin-based distillation loss ℒCMMD (Eq. 15), which outperforms previous formulations (Eq. 13, Eq. 14).
List-wise training uses a combination of calibration distillation loss (Eq. 16) and ℒMSE, mentioned as ℒ+CMLISDE in the results section.
Point-wise batches were sampled independently. For
pairwise and list-wise, batches included one freelancer per
discrete teacher score (e.g., 0.0, 0.2, . . . ) per project, plus two
synthetic unsuitable profiles. Models were trained for 50
epochs with batches of 64 projects ( 320 profiles). The frozen
encoder allowed precomputing embeddings, reducing
training time to 10 hours on an NVIDIA RTX A1000. Learning
rate followed linear decay starting at 0.001:
Qwen3 and Gemma3. Both were fine-tuned using ℒMSE
on 5,000 interactions for 10 epochs. Gemma3 was also
trained with next-token prediction to match Gemini 2.0’s
format. We used parameter-eficient quantized fine-tuning
(e.g., QLoRA [
Qwen3 was used in a standard BERT-style cross-encoder setup, where the brief and profile were concatenated before being fed into the model. Gemma3 followed the Gemini distillation format, encoding the (freelancer, project) context and criteria set. Profiles were truncated to 2,000 tokens to fit memory, prioritizing recent relevant experience.
To evaluate model performances, three categories of metrics are used: (i) relevancy metrics that evaluate the models’ ability to discriminate between relevant and non-relevant candidates; (ii) ranking metrics, that assess the correctness of candidate ordering; and (iii) calibration metrics that evaluate how well the predicted scores aligns with the ground-truth, indicating calibration quality.
Relevancy metrics. We define a freelancer as relevant if > 0.5, and non-relevant otherwise. Based on this definition, we compute: • Recall (Rec.), measuring the proportion of relevant freelancers correctly identified. • Specificity (Spec.) , assessing the ability to correctly reject non-relevant freelancers. • R-Precision (- ), which is the precision at per project, with the number of relevant freelancers. • Non-Relevant False Omission Rate ( ¯-), an inverse analogue of - that captures how many of the bottom-ranked freelancers are non-relevant. • Mean Average Precision (mAP), a standard metric assessing relevancy across ranks.
Ranking metrics. To evaluate the quality of the ranking itself (independent of relevance thresholds), we report: • Mean Reciprocal Rank (MRR), which considers the position of the first relevant freelancer. • Normalized Discounted Cumulative Gain (NDCG), which accounts for the order of all relevant items, assigning higher importance to those ranked higher.
Calibration metrics. Assuming the teacher model (Gemini 2.0) provides semantically calibrated scores, we assess how well the predicted score distributions fit the teacher’s. We measure the distance between the predicted and target score distributions using: • Mean Absolute Error (MAE), • Diference in Means ( ∆ mean), • Diference in Interquartile Ranges ( ∆ IQR), • Wasserstein Distance, measuring the minimal cost of transforming the predicted distribution into the ground-truth one.
Unlike the other metrics, which are normalized between 0 and 1 (with higher being better), lower values indicate better performance for these calibration metrics.
In terms of relevance evaluation, we observe a trade-of between discarding non-relevant candidates and preserving relevant ones. Qwen3 performs well in zero-shot, particularly on specificity and R-Precision (R-P), but fine-tuning appears to harm its generalization. In contrast, regressionbased fine-tuning significantly boosts Gemma3’s performance, especially in recall, R-P, and ¯-. Our model is competitive, being second on key relevance metrics and performing well in specificity.
Looking at ranking metrics such as mean average precision (mAP), mean reciprocal rank (MRR), and normalized discounted cumulative gain (NDCG), our method achieves consistently strong performance. When trained with ℒCMMD, it consistently achieves top performance across all metrics. Gemma3 slightly outperforms on NDCG (0.975 vs. 0.973), while our model with CLID loss is a close second in overall Qwen (0.6B) Gemma (1B) ranking quality.
Calibration analysis based on Figure 4 reveals that Qwen3 often produces extreme scores (close to 0 or 1), suggesting poor calibration despite good binary discrimination. Fine-tuning helps mitigate this but does not fully resolve the issue. Gemma3 in zero-shot generates a wide range of hallucinated scores, which are corrected with fine-tuning. However, next-token tuning introduces discretization that appears misaligned with Gemini 2.0’s scoring, and regression reduces the expressiveness of the scores. Overall on this aspect, Our method provides the best alignment with Gemini 2.0 scores, both visually and based on evaluation metrics.
In terms of eficiency, our model is highly scalable, processing 1,000 profile-brief pairs in under one minute, or just 287 milliseconds when using precomputed profile embeddings. In comparison, Qwen3 requires 7 minutes and Gemma3 about 13 minutes. This substantial speed advantage makes our approach more practical for real-time or large-scale deployments, including CPU-only environments.
To assess potential bias, we conducted a simple
genderbased fairness analysis. The test set was split by gender
declared by freelancers when creating their profiles, and
recall was computed for each gender group. The diference
in recall between women and men was 0.005 using historical
labels and 0.010 using Gemini 2.0 scores, suggesting our
model does not discriminate relevant freelancers based
on their gender. We acknowledge this is a preliminary
assessment and that a deeper fairness analysis [
Lastly, our model demonstrates good robustness to out-ofdistribution samples (Appendix C), while retaining semantic alignment from the frozen backbone (Appendix B). However, performance on synthetic average-match cases indicates room for improvement, particularly in distinguishing nonrelevant profiles.
This paper presents a lightweight model for long-context multilingual reranking of project–freelancer pairs, leveraging a distillation framework to produce semantically calibrated and interpretable scores. Our two-step architecture, comprising two encoding branch followed by a comparison block, outperforms both zero-shot and fine-tuned baselines on relevance, ranking, and calibration metrics, demonstrating its efectiveness for skill relevance assessment. Furthermore, the proposed utterance-based encoding strategy significantly reduces computational complexity, enabling eficient processing of long documents. The ability to precompute freelancer profile embeddings further supports low-latency inference, making the model well-suited for real-time deployment in production environments.
Future work will focus on refining key components of the distillation framework. In particular, careful attention should be paid to the construction of training and evaluation datasets. A dedicated test set derived from historical data with high-quality labels is essential to better evaluate calibration and interpretability. This may require debiasing the data and conducting a label annotation campaign to introduce finer-grained, calibrated labels.
In addition, deeper analysis of potential biases, especially under production conditions, is essential. We also plan to extend the comparison of the teacher’s scores with expert judgments to better assess its own calibration. Since our supervision relies on synthetic labels, we must remain cautious about inherited biases and explore strategies to monitor and mitigate them. Long-term robustness will require handling potential drift of the teacher model, for example through periodic re-calibration or self-distillation. Finally, future research should investigate how this model can be integrated into downstream systems, for example as a feature within existing ranking algorithms or as a tool to improve transparency and interpretability in user-facing applications. Indeed, an important next step will be controlled online experiments to assess business impact.
During the preparation of this work, the authors used chatGPT 3.5 and Gemini-2.5-flash for grammar and spelling check. After using these tools, the authors reviewed and edited the content as needed and take full responsibility for the publication’s content.
French English Spanish German Dutch Impact of out-of-distribution examples on model robustness. The test set is enriched with synthetic average or unsuitable matches. Red highlights show degraded performance, green indicates improvement or consistency. , and ,, we extract a set of descriptive statistics from each distribution, in order to get fixed size features more suitable for scoring. These descriptive statistics are defined as the following for , : desc(, ) = ⎨
, = ⎸⎷ ⎪⎪ ∈, min ( ) max ( ) ⎪⎪ ∈, ⎪ ⎪ ⎪⎪, = ⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ 1 1 ⎪⎪⎪ |, | ∈, ⎪⎩ |, | ∈, ∑︁ ∑︁ ∑︁
⎯ ⎸ ∈, |, |
∑︁ ∈, ( − , )2
|, | ︃( − , ︃( − , , , )︃3 )︃4
Minimum Maximum Mean Standard deviation Skewness Kurtosis
(17) desc(,).
The same statistics are computed based on , to obtain
Table 3 reports the performance of our model (Section 3.2), trained with the ℒCMMD loss (Eq. 15) and evaluated using the metrics from Section 4.4, on test set splits by brief language. Rows are ordered by language frequency. This analysis assesses the model’s multilingual robustness, despite relying solely on a multilingual backbone without additional language-specific training.
French, being the most represented language, yields “average” performance across most metrics. This confirms that the model does not overfit to the dominant language in a way that degrades generalisation.
Interestingly, Dutch achieves the strongest results across most metrics, despite having the smallest support. This can be explained by the recent introduction of Dutch on the platform, which involved more manual curation, human refinement, and assisted onboarding. We hypothetize that These factors have led to cleaner training signals.
German shows excellent recall but poor specificity and ranking metrics, suggesting that the model tends to overestimate relevance in that language. This may reflect weaker signal quality or domain mismatch.
English underperforms on multiple fronts, particularly in calibration and ranking. As English is used in many diferent regions and contexts, the briefs likely exhibit higher lexical and stylistic variability, which may introduce noise during training.
Overall, the model shows strong generalization across languages, including underrepresented ones. This highlights the robustness of the approach, even in the absence of multilingual-specific objectives or balancing strategies. However, performance gaps observed in certain languages, particularly German and English, suggest that enhancing the quality of training signals in these languages could further improve results.
Table 4 reports the performance evaluation results for our proposed model (Section 3.2), trained with the ℒCMMD loss (Eq. 15) and evaluated using the metrics defined in Section 4.4. The original test set (first row) is enriched with out-of-distribution samples: average matches in the second row, and unsuitable matches in the last row. The synthetic average and unsuitable samples are constructed using the same heuristics as those used during training.
The borderline average matches pose a challenge: the model’s recall and precision (R-P) decrease, indicating dificulty in confidently labeling them as relevant. However, specificity and ¯ − improve, meaning the model successfully avoids over-recommending these borderline candidates. This behavior leads to a slightly worse mAP, likely because some relevant items are misclassified as irrelevant.
The model remains robust to "unsuitable" matches, with unchanged recall and significant gains in specificity. Calibration metrics are stable or slightly improved, indicating good semantic separation. However, NDCG shows a small drop, suggesting some unsuitable candidates may still rank above borderline ones, an aspect for further exploration.
In conclusion, the model demonstrates strong robustness to unknown interactions. It conservatively handles average cases and reliably down ranks unsuitable ones. This behavior is desirable in production, where pushing weak or irrelevant recommendations should be avoided, supporting its deployment in real-world scenarios. Nonetheless, additional investigation is needed to mitigate undesirable behaviors, such as the misclassification of relevant candidates or the overranking of unsuitable ones.