Finding the perfect match between a job proposal and a set of freelancers is not an easy task to perform at scale, especially in multiple languages. In this paper, we propose a novel neural retriever architecture that tackles this problem in a multilingual setting. Our method encodes project descriptions and freelancer profiles by leveraging pre-trained multilingual language models. The latter are used as backbone for a custom transformer architecture that aims to keep the structure of the profiles and project. This model is trained with a contrastive loss on historical data. Thanks to several experiments, we show that this approach efectively captures skill matching similarity and facilitates eficient matching, outperforming traditional methods.
With more than 700,000 registered freelancers, Malt is the
leading freelancing platform in Europe. On the platform,
users have the option to search for freelancers on their
own or to post a project that will be handled fully by our
recommender system. Like other players in the human
resources field [
At the time, this system sufered from three limitations. First, we could not scale well, as each new project needed to be scored against each freelancer to build the global ranking. Second, the system only relied on partial information contained within the rich freelancer profiles. For example, job title and skills were taken into account but long text sections such as profile description or experience description were under-exploited. Finally, as Malt is active across Europe, language management was painful since skill matching model were monolingual and one had to be developed and maintained for each language supported. Plus, the cross-lingual matching was badly handled. In summary, we needed a multilingual approach that could scale and use richer information than the legacy system.
Our legacy system had just the filtering and ranking
phases. To map to a traditional recommender architecture
that can scale, we decided to add a retrieving phase to
generate relevant candidates. Traditionally, two approaches
are used [
Indeed, in our case, there are fundamental diferences in the nature and form of information disclosed by freelancers and project proposals. Freelancers and companies may employ diferent vocabularies, potentially due to varying levels of expertise. Finally, projects typically require a subset of a freelancer’s skills, resulting in less specific information being communicated.
Therefore, our goal is to build upon existing work, such
as SentenceBERT [
The proposed approach, while similar to traditional job
board methods, adopts a reversed recommendation
strategy: it recommends freelancers to employers rather than
job posts to candidates. Despite this perspective shift we
can benefit from current approaches in the human resources
(HR) domain. Indeed, in recent years, available data in HR
have enabled to enhance person-job fit algorithms. By
improving how candidates are matched with job roles, these
advancements have made hiring processes more eficient
and efective, ensuring that the right talent meets the right
opportunity. Various approaches, including content-based
ifltering [
Due to their principal modality, documents containing
text, most of these methods utilize embeddings of input
data to measure similarity, a critical aspect of evaluating
a candidate’s fit for a particular role. The richness of
embeddings is explored in multiple studies [
Approaches vary on embedded textual information types
and techniques used, such as Word2Vec and Doc2Vec
[
The aforementioned models transform textual data
through multiple non-linear transformations, resulting in
more abstract representations. They create a framework for
manipulating the representational space, referred to as the
latent space. The emerging field of representation learning
[
Regarding retrieval process within HR applications,
natural clustering representation is highly desirable for similar
entities. Most approaches rely on contrastive representation
learning [
To improve contrastive learning, researchers have
focused on enhancing settings, relational data, and loss
functions. For settings, models like CrosCLR [
The following section describes our proposition of a model handling documents structures in diferent languages: a neural encoder based on contrastive learning using InfoNCE.
In the following, we denote project proposals as for all project ∈ P, where P is the set of all possible projects. Similarly, freelancers’ profile are denoted as for all freelancer ∈ F, where F is the set of all possible freelancers. This section aims to describe two-tower encoder models, denoted as and , which encode and enable the retrieval of freelancers based on submitted projects. Generic documents (i.e. either a proposal or a profile) are denoted as , for all projects or freelancers ∈ {F ∪ P}. Each document consists of diferent sections, denoted as ,, for all ∈ section and ∈ {F ∪ P}. Here, represents a type among the section diferent section types. Each section is plain text, and thus, each document is a set of section texts, denoted as:
= {, | ∈ section}.
The role of the models is to encode these documents into embedding vectors. Specifically, we have: ( ) = and () = .
However, these embeddings must lie in the same representation space, where the distance between them is semantically meaningful in terms of skill matching. Projecting a project into this space allows for retrieval by proximity. To achieve this, we leverage past interactions between freelancers and projects using a contrastive loss function. As illustrated in Fig. 1, for past interactions, we consider a recommended freelancer as a negative match for a project if either the company or the freelancer himself declares that the freelancer does not have the skills. Such cases are considered negative project-freelancer pairs. If the freelancer replies that they are interested in the project, then the pair is considered positive. If no feedback is provided, the pair is discarded from the dataset.
In the subsequent sections, we first present the architecture before describing the loss used for training the models and obtaining a semantically meaningful representation space.
For both proposals and profiles document types, the same architecture is used for both models ( , ) of the two tower approach. Therefore, the two models ( ) = ( , ) and () = ( , ), depend on the trained weights and respectively. For a generic document type , whether a profile or a proposal, the following detailed architecture is designed to process the input, resulting in its corresponding embedding vector (, ) = . This embedding vector will subsequently be utilized for retrieval purposes.
The proposed architecture is designed to process any document type consisting of text, with the objective of leveraging its inherent structure, while keeping language alignment. It is depicted in Fig. 2 and can be summarized as follows : A pretrained multilingual backbone is used to capture local context, specifically at the section level, while handling diferent languages, and is described in section 3.1.1 ( Section encoding on Fig. 2). A positional encoding is added for the model to carry the indication of the section type, as explained in section 3.1.2 (Categorical encoding on Fig. 2). In complement, a global context processing at the document level is introduced through a dedicated transformer head, which is detailed in section 3.1.3 (Document context on Fig. 2). Both components ensure a comprehensive understanding of local and global contexts within the document, while the use of a frozen pre-trained multilingual backbone ensure language alignment. Finally, the section 3.1.4 explains how embedded sections are balanced for obtaining the final document representation (Documents embedding on Fig. 2).
By considering a document as a sequence of sections, and each section as a sequence of tokens, it becomes possible to process these sections independently using state-of-the-art pre-trained language models.
To encode the sequence of tokens with a length of
within a section , = { | = 1, . . . , }, we utilize a
sequence-to-sequence transformer [
The choice of a multilingual similarity backbone has been
made considering the need for a latent space that is both
organized based on semantic similarity [
Hence, using a model without a projection layer after pooling will allow semantic similarity and language alignment to be directly reflected in the token embeddings. In other words, the token embeddings token,, ∈ token, of a section , will inherently maintain semantic alignment across diferent languages. Therefore, we assume that using such a model as a sequence-to-sequence backbone (i.e. with no ifnal projection layer), omitting the final pooling layer and freezing its weights, will result in the conservation of the semantic space organization and the alignment across languages.
The following sections describe the adaptation of this backbone and the specification of the whole architecture to the freelancer-project domain.
By concatenating the encoded token sequences from each section, token, , we can construct a single sequence for the entire document, resulting in t′oken .
However, concatenating the sequences leads to the loss of information regarding the original section from which each token is derived. While each token in the newly formed sequence retains its semantic meaning, section-level context, and positional information, it no longer carries any indication of its section type.
To alleviate this issue, we propose the use of categorical encoding to incorporate section type information at the token level, similar to how positional encoding provides positional information to the model. To achieve this, we utilize learned embeddings categorical for each section type label ∈ section, and add these learned weights to the token embeddings. This process is formalized as follows: t′oken =
{token,, + categorical | = 1...} ⨀︁ ∈section = {′token,, | = 1 . . . }, (2) where the length of the sequence is the sum of the lengths of all sections in the document, given by = ∑︀∈section , and ⊙ denotes the concatenation operation.
At this stage, each token represents section-level context
and remains language-agnostic due to the use of the frozen
backbone model. To incorporate document-level context,
we process the concatenated sequence t′oken using a
BERTlike sequence-to-sequence transformer [
Given the frozen backbone and the previous processing steps, this context integration is grounded in semantic meaning, section type, and aligned across languages. For each document , a new sequence t′o′ken is obtained from t′oken , such a:
Head(t′oken ) = t′o′ken, = {′′token, | = 1 . . . }. (3) This ensures that each token’s representation is both comprehensive and contextually aware, reflecting the document as a whole.
Finally, empirical tests have demonstrated that incorporating a skip connection layer between the output of the frozen backbone embeddings token, for all ∈ section and the embeddings produced by the head t′o′ken, facilitated the training of the head. We hypothesize that this improvement arises from the head’s ability to better adapt to the pretrained backbone by learning the diference between the generic domain and our specific domain.
Once the embedded tokens t′o′ken ∈ t′o′ken are obtained for a document , the final goal is to derive a single representation vector, denoted as . To achieve this, a final pooling layer is applied.
Our experiments indicate that using a weighted average pooling method yields better results than simple average pooling, which assigns equal importance to each token in the document sequence. This improvement is due to the fact that diferent sections within a document may vary significantly in length and relevance, making some tokens, such as those from a lengthy description, less significant than others, like those from a job title. To mitigate this issue, we set the pooling weights inversely proportional to the section length. Taking into account the skip connection, the ifnal representation vector is computed as follows: = |∑sec︁tion| ∑︁ t′o′ken,+(∑︀=0 ) =1 =1 .|section| + token,, . (4) This operation efectively performs pooling in two stages: ifrst within each section, and then across all sections, resulting in a more balanced and representative final document representation vector.
For structuring the latent space based on similarity between profiles and project proposals, a contrastive learning paradigm is adopted. Hence, a two-tower approach is used, with one tower for profiles and the other for project proposals. Finally, a contrastive loss function is applied for optimization.
Those kinds of loss leverage the relationship between positive and negative pairs of elements. During training, the model can receive input in the form of pairs corresponding to an anchor and a positive or an anchor and a negative element ; triplets corresponding to an anchor, a positive, and a negative element, or n-uplets with multiple positives and negatives elements.
Implementing the loss function for n-uplets allows for a more flexible and generalized approach better aligned with (2, 2+, 2− ), (1, 1, 1− ), (2, 2+, 2− )
The training objectives in a contrastive setting aim to
optimize the similarity or distance between document
embeddings by maximizing alignment for positive relations and
minimizing it for negative ones. This framework can be
applied to both supervised and unsupervised contexts.
triplet loss is formulated as:
Among these methods, the triplet loss [
− log ∑︁ ′∈
∑︀ ′′∈∪′ exp(.′ / ) exp(.′′ / ) (6) with a temperature parameter and = Both aforementioned losses can be generalized to handle n-uplets. It is constructed by considering the relationships between multiple elements, as detailed in the following |1| . sections.
The relationships between elements can be modeled as a bipartite graph, as shown in Fig.
In this figure, the nodes represent the entities—freelancers denoted as 1, 1+, 1− , 2, 2+, 2− , and projects 1, 2—while the edges depict similarity relations based on historical data, such as skill matches (+) or contrasting (-). In this setting, the adjacency matrix correspond to a useful representation of the project proposal-profile relationship for tensor and matrix computation. To computationally iflter or weight document pairs in the loss function, we represent a positive skill match with 1, a negative match with − 1, and an unknown relation with 0. This matrix can be preprocessed based on skill set agreements in historical data, as described in Section 3.
Furthermore, to help structure the latent space, a freelancerto-freelancer skill similarity relation using a project as a pivot can be computed. Specifically, if two freelancers are both a positive skill match for a project, it indicates that they share a subset of skills that qualify them to undertake the project. Conversely, if one freelancer is a positive match while the other is a negative match, it suggests that the required skills are possessed by one but not the other. The model is then expected to learn these similarities and diferences.
In this setting, when training a model at the batch level, portion of the adjacency matrix can be extracted : a subset of projects ⊂ selected freelancers ⊂
P and the corresponding set of
F can be sampled, and the relevant project-freelancer = ∈,∈ , (7) for computational purposes.
Using this submatrix, we can compute the freelancer similarity relation and, by extension, derive the transi{− 1, 0, 1}| |×| | in the following manner : tive freelancer-to-freelancer adjacency matrix freelancer ∈ freelancer = ∈,∈ 2.( > ′)∈,′∈ .[1 − ( < 0)∈,∈ 2] (8) This operation retains only the upper triangular part (( > ′) ∈ , ′ ∈ ) and ensures that two freelancers who are negatively related to a project do not become positively related (1 − ( < 0) ∈ , ∈ 2).
During our experiments, we did not resort to a project-toproject similarity relation. Such a relation is more challenging to determine and is not directly obtainable from our current settings. Indeed, a freelancer being capable of solving two diferent projects does not necessarily imply that the projects are similar, as they could require diferent sets of the freelancer’s skills. In contrast, freelancer relationships can be directly derived and even supplemented from various data sources.
While learning a latent space organization in a contrastive setting, negative examples are crucial as they serve as counterexamples, helping to refine both broad scenarios and edge cases. However, if we rely solely on the skill match relation in historical data, we are limited to addressing only edge cases. In practice, freelancers are contacted based on their perceived relevance to a project, and if negative feedback is received, it indicates that the project is not suitable for the freelancer at a more granular level (i.e. on finer details, not visible at first glance).
To introduce more trivial counterexamples, which we refer to as weak negatives, we utilize a categorical feature of the profiles. Specifically, when creating a profile, a freelancer ∈ must select a job category ∈ profile . By defining an arbitrary negative similarity relation between two freelancers, [, ′] = − 1, when they belong to diferent categories ̸= ′ , we can incorporate weak negative examples since these freelancers are expected to have diferent skill sets. The transitive freelancer to freelancer adjacency then becomes: freelancer =(project-freelancer 2
+ ∈,′∈ ) · ( > ′)∈,′∈ . (9)
These negative examples are intended to assist in the initial training iterations, aid convergence, and improve the organization of the latent space at a coarse granularity level.
We can reformulate the diferent contrastive loss functions using the previously mentioned adjacency matrix within our framework. Our goal is to obtain a loss function, ℒ(, , ), that takes a set of proposals , a set of freelancers , and an adjacency matrix as input.
First, we introduce a triplet loss that is calculated for all possible document triplets, based on the distances between two sets of entities, D and D’. Afterward, we filter the results using the adjacency matrix to retain only the valid triplets. This loss function is referred to as ℒA-triplets and is computed as follows: ℒA-triplets(, ′, ) = ∑︁ ∑︁ ∑︁
ℒtriplet(, ′, ′′) ∈ ′∈′ ′′∈′ .1,′ >0,,′′ <0.
For tensor manipulation, the triplet loss ℒA-triplets can be eficiently computed by first calculating the loss for all possible document triplets, producing a tensor with the shape R||×| ′|×| ′|. This tensor is then filtered using an element-wise product with a mask tensor. The mask tensor is derived from the relevant submatrix of the adjacency matrix between the relations and ′, and it retains only the valid triplets specifically, where the second dimension of the tensor represents the positive example relative to the anchor, and the third dimension represents the negative example relative to the anchor.
The final triplet loss used in our experimentation is computed by summing two A-triplets. In the first case, the set of project proposals ⊂ P is considered as the anchor, and in the second case, the set of profiles ⊂ F serves as the anchor. In both scenarios, the positive and negative elements are drawn from the set of profiles ⊂ F. The resulting loss is defined as the following : ℒdual A-triplets(, , ) =ℒA-triplets(, , project-freelancer) + ℒA-triplets(, , freelancer).
In our case, we want the InfoNCE loss to be fully supervised, meaning we want to control exactly which document are negative in the loss. To do so, we can leverage the adjacency matrix:
1 ℒA-InfoNCE(,′,)= |>0| . ∑︀∈ ∑︀′∈′ − log((,′,)), (12) (10) (11) with (, ′, ) = ∑︀
exp(.′ / ).1,′ >0 ′′∈∪′ exp(.′′ / ).1,′ ̸=0 . (13) From this implementation, the tensor computation can be done in two part, first computing all dot products between all possible pairs then applying a masked softmax and filtering on positive pairs. Then consider both projects ⊂ P freelancers ⊂ F pairs and freelancer ⊂ F only pairs and adding the weak negatives in computed from categories, we get: ℒdual A-InfoNCE(,,)=ℒA-InfoNCE(, , project-freelancer) + ℒA-InfoNCE(, , freelancer + ∈,′∈ ) (14) We note that for the A-InfoNCE the cosine similarity is better suited than a distance function to reflect on semantic similarity, as it implies the maximization of a cartesian product.
With our experiment, we want to evaluate our choices on how to leverage both pretrained models and our document data, while also comparing our approach to a reference model and models based on state-of-the-art approaches. The diferent model evolution tested here are to reflect on the impact of modifying the weights of pretrained models in the semantic organization and the language alignment of the latent space, but also the influence of our solutions to exploit document content and structure.
In the following, we compare our approach presented in
section 3 against one reference model and several models and
architectures we tested while building our final approach:
• The first model, used as reference, is PaLM 2,
specifically its Gecko version1. This is a closed commercial
model from Google, based on a decoder-style
transformer architecture with over 10 billion parameters.
It is fifty to one hundred times bigger than the other
models evaluated here.
• The second model, later referred to as S-BERT, is
a multilingual model from the Sentence BERT[
Some models were evaluated out-of-the-box in a zero-shot manner, while others were trained on our dataset, as described in the following sections.
To avoid data contamination, the data used for the following experiments and testing were split in a temporal manner. Specifically, the test set consists of ten weeks of past interactions starting from March 2023, comprising approximately 61k interactions, 4k project proposals, and 32k freelancer profiles. For the training and validation data, which were split in an 80/20 ratio, all interactions that occurred prior to the aforementioned date were considered. Furthermore, all profiles were recomputed to reflect their state at the time of the interactions.
Concerning the freelancer profiles, various declared information are used as sections, including a free-text job title and description, a job family and category selected from our taxonomy, and a set of skills, which may be provided in free text or selected from those previously defined by other freelancers. For the project proposals, the retained information includes the mission title, its associated job title and description as free-text, a job family and category from our taxonomy, as well as mandatory and bonus skills chosen from the list of freelancer-declared skills, as illustrated in 1.
We consider empty sections, referring to sections not iflled by the freelancers or the companies, as sequences containing only the tokens [CLS] and [END]. This data modeling approach ofers two key benefits: (1) it enables the use of static computational graphs for optimization, and (2) it provides the models with explicit information about the emptiness of the section.
In this experiment, Gecko and Section-S-SBERT were evaluated in a zero-shot setting, while all other models were trained on our dataset, either with full retraining or with a frozen backbone. Specifically, we evaluated both S-BERT and Section-S-BERT with fully retrained backbones, starting from their initial pretrained states. Additionally, we trained a Section-S-BERT with a projection layer and our ifnal proposed approach, both with their pretrained backbone weights frozen. In the case of Section-S-BERT with a projection layer, it should be noted that only the projection layer was trained.
Regarding the loss functions, almost all models were
optimized using a triplet loss [
Each of the aforementioned models aims at encoding projects and freelancers for retrieval within the proposal step of a recommender system. However, evaluating them in the same manner as we would evaluate the full pipeline, using ranking or retrieval evaluation metrics, is not perfectly suited. Moreover, the proposal-profile relevancy, in the context of document retrieval, is only partially available in our dataset, as the relevancy of all possible proposal-profile pairs remains unknown. Finally, evaluating solely based on past interactions may not provide a complete picture of the intended performance of a retrieval system. Indeed, historical data is biased by previous processes and lacks information about preposterous project-freelancer pairs.
To better reflect the future usage of the models, we chose to evaluate them in three diferent settings: • Supervised, using only historical interactions and the interacted profiles, • Unsupervised, by simulating a retrieval with knearest-neighbor (k-NN) using all test profiles, • Weakly supervised, using the simulated retrieval enriched with the historical interactions.
The cosine similarity or Euclidean distance was used to compute the predicted proposal-profile score when the training loss was InfoNCE or the triplet loss, respectively. Both were converted to scores or distances according to the requirements of the metrics and the k-NN algorithm.
Considering the supervised experiments, we tested the recall metric using score comparison, denoted as (· ), to determine true and false positives. This metric, which we will refer later to as "valid score recall," is computed as follows: recallsingle(, +, − ) =
∑︁ +∈+ 1 [︀ s(, + ) > s(, − ), ∀− ∈ − ]︀ |+| (15) where is a project proposal, corresponds to its encoding, + ∈ + the relative positive freelancers, − ∈ − represents its negative freelancers, and + , − are their respective embedding vectors.
In this setting, we also compare the average of positive score against the average score of negatives, with the following : recallall(, +, − ) =
⎡⎡ 1 ⎣⎣
∑︁ +∈+ s(, + ) |+| ⎤ ⎦ > ⎣ ⎡
∑︁ − ∈− s(, − ) |− | ⎤⎤ ⎦⎦ .
(16) Inverting the comparison and using a distance function allows for the computation of the metric when the triplet loss was used.
For the unsupervised setting, a retrieval is first simulated using a k-NN algorithm for each project on all freelancers in the test set. Then, the proportion of freelancers who 0.437 0.477 have declared the same category as the one in the project proposal is computed.
We introduce the A-overlap metric to quantify lexical similarity between freelancers and project proposals using exact lexical matching on a predefined set of terms:
A-overlap(; , ) = ∑︁ | ∩ | , ∈ | | · | | (17) where denotes either the set of skills or of categories . The overlap score is calculated for each freelancer, and the results are averaged across all retrieved freelancers . Finally, for the weakly supervised setting, both the simulated retrieval and the historical interaction data are combined. This allows us to compute the proportion of retrieved positives, such as : retrieved-positves(, , +) = | ∩ +| , |+| and retrieved negatives : retrieved-negatives(, , − ) = | ∩ − | , |− | (18) (19) among the retrieved freelancers .
The following results, reported in table 1, consist of the metrics defined previously, computed across all models described in Section 4.1. Each metric represents the average value computed over all project proposals in the test dataset. Gecko and Section-S-BERT, corresponding to the first and fourth lines of Table 1 are evaluated in a zero-shot manner, and provide strong results for the unsupervised metrics related to category and skill overlap. These results reflect the efectiveness of the semantic-aware pretraining and the models’ ability to organize documents in the latent space based on their content. However, due to the lack of finetuning on our dataset, the supervised valid score recall metric reveals these models’ inability to accurately structure the latent space according to our data domain and historical interactions.
Adapting the models to our domain through full retraining yields improved results, as shown in the second and third rows of the table. However, the low values obtained for the weakly-supervised and unsupervised metrics suggest a degradation in the semantic space organization established during the pretraining phase.
The results reported in the last three rows of the table, corresponding to models with frozen backbones, appear to ofer the best of both worlds. These models successfully retain the benefits of the semantic pretraining of the backbones while adapting to our domain. This is evidenced by the improved supervised metrics compared to the fully retrained versions, without significantly degrading the unsupervised metrics.
Adding synthetic weak negatives in the training improves the unsupervised metrics, as reported in the last two rows of the table. We hypothesize that this improvement is due to the backbones being adapted to both edge cases and broader scenarios using more extensive data. In contrast, the other models adapt to our domain by focusing exclusively on edge cases during training.
Finally, our latest approach appears to be the most efective in leveraging content. Not only does it avoid degrading the unsupervised metrics, but it also achieves the best overall results in this regard. It seems to adapt the backbone to edge cases based on historical data while simultaneously adapting it to our domain, being 50 times smaller than PALM 2. It is important to note that our approach enables the retrieval of more positives than negatives in the historical data, although the number of negatives remains substantial. This should be carefully considered when designing the entire pipeline to ensure proper filtering of these freelancers. In addition to the aforementioned quantitative metrics, we present the freelancers’ latent space organization for our trained approach in Fig. 4. A two-dimensional projection of the freelancers’ density is illustrated both by job family in the left of the Figure, and by category in its right, specifically for the web, graphic and design job family. When focusing on the density distributions by family, denser spots can be observed. Indeed, some families exhibit a single dense spot, while others display multiple dense areas.
When exploring the latent space representing the job categories, it can be observed that the previously mentioned denser spots correspond to regions of space reserved for categories within the families, even though some categories may have multiple dense spots. We hypothesize that this may be due to the presence of sub-categories and diferent typologies of freelancers.
This latent space can be leveraged not only for retrieval, but also to gain a deeper understanding of our market and freelancers. In particular, it could be used to drive the evolution of our job category taxonomy. Overall, our model appears to efectively organize freelancers based on semantic relationships.
In conclusion of these experiments, we report metrics based on the language alignment aspect of our approach, as shown in Table 2. These results highlight the model’s ability to maintain pretrained language alignment while simultaneously adapting the backbone to our domain. This result is particularly notable given that our dataset contains approximately ten times more French profiles than profiles in other languages. Hence, achieving efective language alignment solely based on this data is challenging without the use of pretrained backbones, highlighting the good performances of our approach.
As outlined in Section 1, a legacy recommender system with one filtering phase and one ranking phase was already in place. This section details the integration of the newly developed retriever within the existing infrastructure and examines the impact of its deployment in our production environment.
As illustrated in Fig. 1, we implemented a conventional architecture for deploying our neural retriever. The embeddings for all freelancer profiles are precomputed and stored in a vector database. For this purpose, we selected Qdrant4, 4https://qdrant.tech/ due to its high performance and advanced filtering capabilities, including geographical filtering. The vector database is updated daily to incorporate new and updated profiles.
At inference time, upon receiving a new project proposal, its embedding is computed on the fly. Then, we use that embedding to perform an approximate nearest neighbors search taking into account hard filters to retrieve a set of candidates for the projects. These candidates are subsequently passed to the legacy ranking system for final evaluation. 5.2. A/B test To ensure that the new retrieval phase did not negatively impact conversion rates, an A/B test was conducted over the course of November 2023. As anticipated, there was a significant improvement in response times, with the 95th percentile latency decreasing from tens of seconds (and occasionally exceeding a minute) to a maximum of 3 seconds. More unexpectedly, a 5.63% improvement in conversion for efective matches ( i.e., cases where both the freelancer and client confirm the match) was observed. We hypothesize that the new retriever, with its more advanced technology, efectively eliminates suboptimal candidates that might have been selected by the legacy ranker.
This research aimed to develop two encoder models that generate embeddings for freelancer profiles and project proposals, enabling eficient vector-based retrieval. The models were designed to handle multilingual data, exploit document structure and content, and scale efectively as part of a candidate proposal system.
We proposed and validated an architecture and training approach, which outperformed other methods we tested. When deployed in production, it improved latency without any matching performance loss, leading even to an improved conversion rate.
Future work could explore improving data preparation for contrastive learning, such as creating better positive and negative pairs. Additionally, using continuous values in the adjacency matrix, rather than binary edges, could enhance learning semantic similarity, particularly in synthetic relationships between freelancers. Combining lexical and semantic retrieval methods, such as integrating BM25 scores, also presents a potential area for improvement. While our models are efective for recommendations, they are less suited for search tasks where the input are just short queries instead of complete project proposals. A possible direction for future research could involve a three-tower architecture, with one tower dedicated to handling search queries.
Finally, an important area of research within human resources technology involves studying biases induced by such models. That topic is deeply important to Malt and for the AI Act. That is why we plan on investigating and mitigating potential biases in our retriever models. This would ensure fairness and transparency in the recommendations, while exploring techniques to identify and correct any unintended biases that may arise during training and deployment.