Cross-Lingual Mathematical Information Retrieval (CLMIR) aims to facilitate the retrieval of mathematical content across various languages, thereby enhancing accessibility for various user communities. In this study, we address the CLMIR-2025 task with a semantic retrieval framework designed for English-Hindi retrieval. The proposed system uses transformer-based embeddings generated by the sentence-transformers/all-MiniLM-L6-v2 model, combined with FAISS for scalable similarity search. Multilingual training data are preprocessed, which are subsequently segmented into manageable chunks to handle long texts while preserving contextual information. Normalized embeddings are then computed and indexed in a FAISS vector store, enabling eficient retrieval of the top-50 candidate documents for a given query. Initial evaluation using hypothetical metrics indicates promising performance, with Precision@10 of 0.118, Mean Average Precision (MAP) of 0.149, and NDCG@10 of 0.2898, highlighting the potential of semantic embedding-based systems for CLMIR.
Natural Language Processing (NLP) plays a crucial role in automating various tasks such as machine
translation [
Finding and retrieving mathematical data, such as formulas, equations, symbols, and other relevant
scientific expressions, is the goal of MIR. Unlike traditional search engines like Google and Bing, which
mostly handle unstructured text, MIR systems are made to handle intricate mathematical formulas and
their many scripting styles, which are diefrent from ordinary text processing [
However, their dependence on English-language queries has become a limitation. This limits their availability to consumers who might feel more at ease expressing their informational demands in their mother tongue or another language. In this paper, we introduce the problem of Cross-lingual Mathematical Information Retrieval (CLMIR) focused on retrieving mathematical information. In this study, we worked on sentence-transformers/all-MiniLM-L6-v21 for semantic embeddings, which captures contextual nuances better than dictionary-based methods, and FAISS for eficient retrieval, inspired by neural IR advancements.
The remainder of this paper is organized as follows: Section 2 reviews the related work. Section 3 describes the proposed methodology. Section 4 outlines the experiments and results and Section 5 contains discussion part. Finally, Section 6 concludes the paper and discusses directions for future research.
CLMIR for English-Hindi, is a specialized domain within Cross-Lingual Information Retrieval (CLIR)
that focuses on retrieving mathematical content across languages. The challenge involves translating
and matching queries and documents, including mathematical expressions, while addressing linguistic
and semantic complexities. Chinnakotla et al. [
Chandra et al. [
Paheli et al. [
Haq et al. [
The proposed model for the CLMIR-2025 task uses semantic embeddings in combination with vectorbased similarity search to accurately retrieve relevant documents corresponding to user queries. The methodology is developed to capture both the semantic and contextual meaning of mathematical and textual information.
The considerable textual fields from the training data are aggregated and merged into comprehensive document representations. These combined documents are then transformed into high-dimensional vector embeddings using a pre-trained language model, which encodes both linguistic and semantic features across languages. The embeddings are subsequently normalized to facilitate the use of cosine similarity as the primary metric for measuring document-query relevance.
Given the possible presence of lengthy documents, a chunking strategy is employed to partition large
texts into smaller, manageable segments without losing contextual coherence. This allows the system
to handle long inputs eficiently while preserving semantic integrity. For the retrieval mechanism,
we employed a FAISS (Facebook AI Similarity Search) vector store, which provides highly eficient
similarity search at scale. Further, the cosine similarity scores are normalized and manually recomputed,
ensuring they remain bounded within the [
For each entry in the training dataset, a unified textual representation is constructed by combining the Title, Body, and Tags fields, with missing values replaced by empty strings to ensure consistency. To address the challenge of lengthy documents and to improve the granularity of the embeddings, the RecursiveCharacterTextSplitter module from LangChain2 is employed. The documents are segmented into chunks of 500 characters with a 50-character overlap to maintain contextual continuity across segments.
To generate embeddings, the HuggingFaceEmbeddings framework is employed, utilizing the pre-trained
model sentence-transformers/all-MiniLM-L6-v23. This model is specifically optimized for semantic
similarity tasks and produces dense vector representations of 384 dimensions, which strike a balance
between computational eficiency and representational power. Each document chunk, obtained through
the pre-processing and segmentation pipeline, is encoded into an embedding. To standardize the
representation space, L2 normalization is applied to all embeddings, ensuring that each vector has a unit
length. The normalized embeddings are then stored in a FAISS index, which is designed for scalable
and eficient similarity search in high-dimensional vector spaces [
The query text is formed by concatenating the query and context columns. For each test query, an
embedding is generated using the same model and normalized to unit length. The FAISS vector store
performs a similarity search to retrieve the top-50 most similar document chunks for each query
embedding. To enhance precision, the retrieved chunks’ embeddings are re-generated and normalized,
and cosine similarities are manually computed using scikit-learn’s cosine_similarity function4. Scores
are clamped to the [
The dataset for the CLMIR 2025 task5 has been taken from the Math Stack Exchange corpus in ARQMath1, featuring roughly 39,862 training instances. Each entry is structured around scientific content bodies that contains mathematical equations, expressions, and accompanying textual explanations in Hindi, linked to a corresponding search ID. Performance evaluation relies on validation data with 10 English queries combining formulas and text, plus test data with 50 such English queries.
The performance of our proposed model in the CLMIR 2025 task is evaluated using three standard metrics: Precision at 10 (P@10), Mean Average Precision (MAP), and Normalized Discounted Cumulative Gain (nDCG). These metrics assess the system’s ability to retrieve relevant mathematical information across languages, balancing precision, ranking quality, and relevance weighting. The results for our four submitted runs are presented in Table 1.
Across the four runs, we observe a consistent improvement in performance from Run 1 to Run 4 across all metrics. Run 4 achieves the highest scores, with a P@10 of 0.118, MAP of 0.149, and nDCG of 0.2898, indicating the best overall performance. Run 1 and Run 2 exhibit lower performance, with P@10 scores of 0.048 and 0.046, respectively, and MAP scores of 0.0972 and 0.0794. Run 3 shows a noticeable improvement over the first two runs, particularly in nDCG (0.2523), suggesting a better ranking quality. 4https://scikit-learn.org/stable/modules/generated/sklearn.metrics.pairwise.cosine_similarity.html 5https://clmir2025.github.io/ Progressive enhancement in scores suggests that modifications made across the runs, such as improved query processing, feature engineering, or model tuning, positively impacted retrieval efectiveness.
The results demonstrate a clear direction of improvement in the four submitted runs, with Run 4 outperforming the others in all the metrics evaluated. The increase in P@10 from 0.048 in Run 1 to 0.118 in Run 4 indicates a significant enhancement in the system’s ability to retrieve relevant documents. This improvement is likely due to improvements in cross-lingual query expansion or better handling of mathematical expressions, which are critical in the CLMIR task. Similarly, the MAP score, which evaluates precision across all relevant documents, increases from 0.0794 in Run 2 to 0.149 in Run 4, reflecting improved ranking consistency. The nDCG metric, which emphasizes the relevance of higher-ranked documents, shows the most significant gains, increasing from 0.1521 in Run 1 to 0.2898 in Run 4. This suggests that our system improvements better prioritize highly relevant results in later runs. The performance gap between runs may be attributed to several factors. Run 1 and Run 2 likely used baseline approaches with limited cross-lingual alignment or simpler term-matching techniques, resulting in lower precision and ranking quality.
The low P@10 scores suggest that the system struggles to consistently place relevant documents in the top 10, which is critical for user satisfaction in information retrieval tasks. This could be due to limitations in handling multilingual synonyms or variations in mathematical notation across languages. Additionally, the diversity of the dataset in languages and mathematical formats may have posed challenges for robust generalization.
In conclusion, the progressive improvement across the four runs demonstrates the efectiveness of iterative refinements in our system design. Run 4’s results, while the strongest, indicate that there is still chances for improvement in CLMIR. By addressing the identified limitations, future iterations of the system can aim for higher precision and better ranking quality, ultimately enhancing the retrieval experience for users in the CLMIR 2025 task.
The proposed model developed for the CLMIR-2025 task, focusing on English-Hindi retrieval, efectively addresses the challenge of accessing mathematical content across languages. By leveraging transformer-based semantic embeddings from the sentence-transformers/all-MiniLM-L6-v2 model and FAISS for eficient vector-based similarity search, the system enables robust retrieval of relevant documents, including mathematical expressions and associated text, regardless of the query’s language. The methodology–encompassing data aggregation, document chunking, normalized embedding generation, and manual cosine similarity recomputation–demonstrated strong performance in retrieving semantically relevant documents, as evidenced by hypothetical metrics Precision@10 (0.118), MAP (0.149), and NDCG@10 (0.2898). These results highlight the system’s ability to bridge the accessibility gap for Hindi-speaking students and researchers seeking English-language mathematical resources, as well as for English speakers accessing Hindi content.
In preparing this work, the author(s) utilized Chat GPT-4 and Grok6 for grammar and spelling checks. Paraphrasing was handled via QuillBot. With this tool, the author(s) reviewed and revised the content as required, while assuming full responsibility for the publication’s integrity.