Humor-sensitive information retrieval presents unique challenges, particularly the need to understand wordplay and implicit cultural references. This paper presents our work on the JOKER 2024 track at CLEF. The first task will explore the use of TF-IDF (Term Frequency-Inverse Document Frequency) weighting combined with logistic regression to improve the eficiency of retrieving relevant humorous texts. The second task will focus on classifying humour texts based on genres and humor techniques. The architecture combines stacking and weighted voting approaches to optimize classification performance. The goal is to leverage the strengths of diferent base models to enhance classification accuracy. The third task will be to translate texts from English into French. Ensuring the preservation of meaning while maintaining fluency and contextual appropriateness is particularly dificult. To address this issue, we employ an approach based on the MarianMTModel, a neural machine translation model designed for translating text between various languages.
1.1. TF-IDF TF-IDF is a commonly used weighting technique to evaluate the importance of a word in a document relative to a corpus. Term Frequency (TF) measures how often a term appears in a document, while Inverse Document Frequency (IDF) measures the importance of this term in the entire corpus. The
TF-IDF formula is given by: documents containing term .
TF-IDF(, ) = TF(, ) × IDF() IDF() = log ︂(
|{ ∈ : ∈ }| ︂) where TF(, ) is the frequency of term in document and IDF() is calculated as follows: with being the total number of documents in the corpus and |{ ∈ : ∈ }| the number of Logistic regression is a classification model that can predict the probability that a document is relevant to a given query. The logistic function is defined by: ( = 1|) =
1 1 + − (· ) where is the feature vector (here, the TF-IDF values of the terms) and is the weight vector learned during model training.
are as follows:
For Task 1 of JOKER 2024, the goal is to retrieve humorous texts relevant to a specific query. The steps • Collect a corpus of humorous and non-humorous texts. Each text is labeled; 0 for non-joke and 1 for joke. • Pre-processing: The data is cleaned by removing special characters, stop words, etc. • Vectorization: We vectorized the documents using the TF-IDF technique (Tfidf Vectorizer). • Training: The logistic regression model was trained to distinguish jokes from other texts. • Once the training is complete, we used the model to filter the documents and retain only the jokes. the jokes.
similarity. 1.3.2. Step 2: Retrieving Relevant Jokes • TF-IDF Calculation: We apply TF-IDF vectorization to each identified joke to obtain a numerical representation based on term importance. • Creating the TF-IDF Matrix: We construct a TF-IDF matrix where each row represents a joke and each column represents a term, with the TF-IDF values corresponding to the term weights in • Query Comparison and Retrieving Relevant Jokes: TF-IDF values are used to calculate the cosine similarity between the queries and jokes. We retrieve the most relevant jokes based on The following figure show the pipeline diagram for joke retrieval.
The model architecture is illustrated in Figure 2. It consists of two main components: a Stacking
Classifier [
The Stacking Classifier r combines several basic classifiers (decision trees, random forestsand gradient boostingand uses logistic regression as the final classifier. Each basic classifier is trained on the input data and its predictions are then used as features to train the final classifier. This approach makes it possible to capture complex patterns by combining the strengths of each base classifier. • DecisionTreeClassifier : used for its simplicity and ability to handle nonlinear interactions between features. • RandomForestClassifier : combines multiple decision trees to improve robustness and accuracy. • GradientBoostingClassifier : uses a boosting approach to correct the errors of previous classiifers and enhance overall performance.
• LogisticRegression: used as the final classifier to combine the predictions of the base classifiers.
The Voting Classifier combines the predictions of the Stacking Classifier and an SVC (Support Vector Classifier) after text vectorization by a Tfidf Vectorizer . The final predictions are obtained by a weighted voting, where the Stacking Classifier and SVC contribute to the final decisions according to their performance on diferent classes.
• Tfidf Vectorizer : Used to transform texts into feature vectors based on term frequency. • SVC: Used for its high performance in text classification tasks.
Model performance is assessed using the confusion matrix and classification metrics. Figure 3 shows the confusion matrix for model predictions. Table 1 shows the main performance metrics, including precision, recall, and F1-score for each class. 3. Task3 : Translation of puns from English to French
’MarianMTModel’[
• Decoder : It uses these contextual representations to generate the target text word by word.
MarianMT models are trained on large multilingual corpus. They are able to translate between a large
number of language pairs thanks to training on high-quality aligned data. Being pre-trained on generic
data, we used fine-tuning to improve performance on our data.[
Fine-tuning is a process in machine learning where a model pre-trained on a large corpus of data is re-trained on specific data for a given task. It allows the model to retain the general knowledge acquired during the initial training while learning the specifics of the new task.
The Fine-tuning procedure is described as follows: • Loading the Pre-trained Model • Preparation of specific data : tokenisation, alignment, etc. • Re-training : training the model on specific data using appropriate hyperparameters • Rating and adjustments : Evaluate performance on a validation set and adjust hyperparameters if necessary.
For task 3 of JOKER 2024, the aim is to translate texts from English into French while preserving the meaning. The various steps are as follows:
Step 1: Library import • train_test_split from sklearn.model_selection • pandas: for handling dataframes • json: for data management • datasets: for loading data sets • transformers: for the translation model ’MarianMTModel’ Step 2: Loading data The data is then loaded and converted into dataframes.
Step 3: Preparing the training data The data is pre-processed and divided into training and validation sets Step 4: Loading the Model and Tokenizer The MarianMT model and tokenizer are loaded from Helsinki-NLP Step 5: Data pre-processing Tokenisation of texts Step 6: Model configuration and training The training arguments are defined and the model is trained using Seq2SeqTrainer Step 7: Saving the Model and Tokenizer
For humour information retrieval, we have worked on a lightweight method, using TF-IDF and logistic regression, which gives acceptable results for humour text identification and extraction. Future work will focus on the integration of larger models to further improve the performance and accuracy of humorous text retrieval tasks.
For humour classification based on genre and technique, we have proposed an architecture that combines the advantages of stacking and voting approaches to improve the performance of humour text classification. By exploiting the strengths of the diferent classifiers, this approach provides a better understanding of the diversity of humour texts, genres and techniques, resulting in a more accurate and robust classification.
As part of the Translation of puns from English into French, we used the MarianMTM model for JOKER Task 3, combined with fine-tuning, ofers state-of-the-art machine translation performance thanks to the Transformer architecture, while minimising the time and resources required. It is easy to use and can be adapted to diferent languages. Although very powerful, the model can nevertheless encounter dificulties with very subtle nuances or complex cultural references.