In recent years, fuzzy joins have emerged as a critical tool in data integration, data cleaning, and similarity-based retrieval tasks, especially when dealing with noisy or heterogeneous data. Although fuzzy joins have been widely explored for relational data, existing approaches typically focus on textual similarity, matching records based on certain text values and text distances. Apart from relational data, syntactic similarity of XML and JSON documents has also been investigated, where the matching documents are compared over their hierarchical structure. These methods often struggle when confronted with variations in element names, formats, or hierarchical layouts. In contrast, the semantic similarity of semi-structured data, such as JSON documents, remains underexplored, despite its importance in capturing the true meaning conveyed by document values. This paper addresses the problem of performing fuzzy joins between JSON documents by comparing their semantic content rather than their structural form. We propose and describe several methods based on document embeddings, aiming to capture the meaning of a document from its values and contextual relationships. The proposed techniques ofer a promising foundation for semantic joining tasks in semi-structured data environments.
The growing volume and heterogeneity of semi-structured data, particularly JSON (JavaScript Object Notation) documents, poses significant challenges for data integration, data cleaning, similarity-based information retrieval and analysis tasks. In particular, organizations need more and more to integrate, deduplicate, and correlate JSON documents that describe the same real-world entities or events, originating from diferent sources and production pipelines. Such documents are rarely uniform. They may use diferent key names (e.g., author vs. creator), diferent value conventions and diferent nesting schemes, resulting a variety of diferences in both structure and content. As a result, conventional equality joins, which rely on strict equality between attributes, fail to identify such correspondences. The problem becomes more challenging when the volume of the input datasets is large.
Fuzzy joins address this limitation by replacing strict equality with a similarity measure; i.e., returning
pairs of documents whose similarity exceeds a predefined threshold [
In this work, we present a distributed evaluation method for computing fuzzy joins over large collections of JSON data, handling similarity over both content and structure. The proposed approach uses methods for converting JSON documents first to text and then to document embeddings, aiming to preserve both content and structure through descriptive representations of the information encoded in the JSON documents. In particular, the main contributions of this paper are summarized as follows: (1) we propose a novel approach for evaluating fuzzy joins over collections of JSON documents, by using a two-step transformation of each document to a document embedding that captures the meaning of the information encoded in the JSON documents, (2) present five diferent transformation approaches of a document to its text-representation, where the four of them use specialized prompts posed on an LLM, and (3) present scalable MapReduce algorithms for computing the fuzzy joins over large collections of generated embeddings, avoiding shufling the JSON documents during join computation.
This paper focuses on JSON (JavaScript Object Notation) [
To process large JSON collections, we use a distributed processing framework. Although Apache
Spark [
Large Language Models (LLMs) [
In this section, we investigate the problem of distributed computation of a fuzzy join over two JSON collections using semantic similarity. In general, a fuzzy join is a join operation applied over two collections 1, 2 of records, and its result is a new collection containing the pairs of records from 1, 2 that are similar according to a given similarity measure and a threshold. In our setting, the joining collections include JSON documents. Unlike prior fuzzy join methods that focus on textual similarity over selected values or on structural similarity, we assess JSON-record similarity by jointly considering content (keys and values) and structure. For example, consider the JSON collections 1 = {1} and 2 = {2, 3}, where 1, 2, 3 are illustrated in Figure 1. Intuitively, although 1, 3 share the same structure, and 1, 2 have diferent structures and vocabularies, we would like the joining result to return only the pair (1, 2); since only this pair describes the same real-world entity (i.e., the documents are semantically highly-similar). It is easy to understand that traditional fuzzy joining approaches relying on structural similarity fail to capture the underlying semantic meaning of these documents.
To address this limitation, our approach, named FLESHJ-Join1, focuses on transforming each JSON document in the joining collections into a dense vector representation (embedding) that encodes its semantic content and structure, and then to join collections by comparing them using vector-space similarity measure [35] (e.g., cosine similarity, (⃗1, ⃗2) = ⃗⃗1· ⃗2⃗ ). Figure 2 illustrates an ‖1‖·‖ 2‖ overview of the proposed semantic fuzzy-join framework, which is organised into an ofline phase and an online phase. In the ofline phase, for each JSON document included in each collection, we initially generate a text-representation that preserves the JSON semantics (content and structure), and then encode this representation into a fixed-dimensional vector space. The collections 1 and 2 include the text-representations of the documents in 1 and 2, and the collections ℰ1 and ℰ2 contain the corresponding embeddings, respectively. In the online phase, the precomputed embeddings in ℰ1 and ℰ2 are fed into a fuzzy-join execution module, which computes semantic similarity scores (e.g., cosine similarity) between vectors and identifies pairs whose similarity exceeds a given threshold.
Formally, consider two collections of JSON documents 1 and 2, so that each document ∈ 1 ∪2 is associated with a unique identifier (). Suppose two transformations 1, 2 transforming a JSON document to one or more text representations, and a text to an embedding, respectively. The FLESHJJoin is defined as follows, where is a given similarity threshold: 1 ⋊⋉≥ 2 = {[1, 2] | 1 ∈ 1, 2 ∈ 2, (2(1(1)), 2(1(2))) ≥ }. Our objective is to eficiently compute 1 ⋊⋉≥ 2 by maximizing the semantic accuracy (assessed using standard information retrieval metrics, such as precision, recall and 1 score, over a gold-standard set of matching pairs); i.e., if is the desired output, we want (1 ⋊⋉≥ 2) − ≃ ∅ , while minimizing both false positives and false negatives.
The primary challenge lies in designing scalable and efective methods for representing JSON documents through natural-language text. In this context, the first step consists in transforming each JSON document ∈ 1 ∪ 2 into a natural-language text-representation 1( ) ∈ , using a transformation function 1 : 1 ∪ 2 → 1 ∪ 2. The goal of this transformation is twofold: (i) to preserve the semantic content of the original document (i.e., what real-world entity or concept it describes), and (ii) to encode enough information about the document’s structure so that produced document embeddings can capture similarities that depend on both content and structure.
To generate the representative text from a certain JSON document, we consider two main methods. The first method simply generates the text by following a certain text-generation rules. The second one uses an LLM to generate this text-representation. In both cases, such a transformation is applied independently to each document, which makes it suitable for large-scale, parallel processing frameworks (e.g., Spark, or MapReduce).
In the first approach, we initially model each JSON document as a tree ( ) = (, , , ℓ) rooted at a special document node representing the entire JSON document, so that for each key and primitive value of the document, a tree node is considered. In addition, for each object element in a JSON array at position , we create an item node labeled as "Item ". Edges are defined as parent–child links from each enclosing JSON construct to its component: from an object (or the document root) to its key nodes, from an array node (typically a key node) to each "Item " node, and from each enclosing key/item node to its primitive value node. To construct such a tree, we follow a top-down approach, starting from the document node. Note that the nested arrays of primitive values are encoded using their flattened form; e.g., the array [[””, ””], ””] is similarly treated as the array [””, ””, ””]. For example, in Figure 1, the tree-representation of document 1 is the tree (1).
Based on the tree representation ( ) of a JSON document , we define text-generation rules that convert each node and its children into short declarative sentences. Traversing ( ) from the root, for each internal node we generate "[n] has fields 1, . . . , " (or "document is described by 1, . . . , " for the root), where 1, . . . , are the children of . For nodes with leaf children, we generate "[n] has values: 1, . . . , ", and when there is a single leaf value we use "The value of [n] is [v]". For example, for (1) we obtain: "document is described by . has fields 1, 2. 1 has ifelds , . The value of is ℎ. The value of is . 2 has fields , . The value of is . The value of is ". Since multiple nodes might have the same key, we could use the reachable (root-to-node) path of each node from the root, instead of its single label.
The second approach uses an LLM to convert each JSON document into a concise text-representation that captures both structure and content, with the goal of expressing the document’s meaning. We study four zero-shot, role-based prompts (Figure 3) that enforce faithfulness (no hallucinated keys/values) and a standardised, fluent output for FLESHJ-Join: Prompt 1 produces a compact paragraph reflecting important keys, values, and nesting; Prompt 2 generates a higher-level semantic summary (type of information, main entities, and relationships) while avoiding technical format details; Prompt 3 follows a similar style but restricts the output to information explicitly present in the JSON; and Prompt 4 uses process-guided constraints to generate a few simple, consistent sentences, improving embedding stability and semantic distinctiveness. These prompts define JSON-to-text transformations ( 1), span the trade-of between structural fidelity (Prompts 1/3) and higher-level semantics/readability (Prompts 2/4) and can be selected or combined per document. Prompt 1 and Prompt 3 are more tightly coupled to the original structure and values, whereas Prompt 2 and Prompt 4 focus on higher-level semantics and human interpretability. In our framework, any of these prompts (or combinations of them) can be used to generate text-representations of a JSON document. For example, the text-representation of 1 depicted in Figure 1 is "This document describes two actors. One actor is Matthew, whose role is a scientist. The other actor is Julie, whose role is an artist. It identifies each actor by name and role." , which has been generated using Prompt 4. As you can see both the structure and content of 1 are captured in this text.
This section presents distributed evaluation techniques for fuzzy joins over embeddings. Once the
embedding collections ℰ1, ℰ2 have been properly constructed, with ℰ = {(( ), ⃗) | ∈ , ⃗ =
2(1( ))} for ∈ {1, 2}, FLESHJ-Join aims to find all pairs (1, 2) such that (⃗1, ⃗2) ≥ , where
⃗ represents ∈ . A naïve all-pairs evaluation requires (|ℰ1||ℰ2|) comparisons and is infeasible
for large JSON collections; moreover, in distributed settings we must also limit communication cost
and balance work. Using MapReduce notation, we outline blocking strategies (e.g., LSH) to reduce
communication and computation, without aiming for an exhaustive optimization study [
Let us initially model the input as a unified multiset of pairs < ( ), (⃗ , ) >, where ∈ {1, 2}. A generic MapReduce job for producing candidate pairs can be written as:
Map < ( ), (⃗ , ) >→−− {
< , (( ), ⃗ , ) >| ∈ ( )}
Reduce < , {(( ), ⃗ , )} →−− >−{− ((1), (2)) | 1 ∈ 1, 2 ∈ 2, (⃗1, ⃗2) ≥ }, where ( ) is a set of blocking keys determining replication and communication cost. Each reducer receives embeddings sharing a key and evaluates a local join between the subsets from 1 and 2.
We consider two representative options for computing ( ). (1) Matrix bucketing (baseline): embeddings from ℰ1 and ℰ2 are hashed to rows and columns of an × matrix, and each reducer processes a cell bucket. This guarantees full coverage but can incur high replication [38, 39]. If is the number of reducers, it is recommended to select ≈ √2. (2) LSH blocking: for cosine similarity, let ℎℓ (ℓ = 1, . . . , ) be independent LSH functions [40, 41, 42] and define ( ) = {ℎℓ(⃗ ) | ℓ = 1, . . . , }. The map function emits one pair per hash value, while each reducer compares only vectors co-assigned to the same bucket, followed by optional duplicate elimination across buckets. Matrix bucketing is deterministic but expensive in communication, while LSH trades deterministic coverage for probabilistic recall with reduced shufle; increasing improves recall at the cost of additional replication.
This section presents a preliminary evaluation focusing on the generation of text-representations from JSON documents. To evaluate our approach, since no publicly available dataset is available, we generated a dataset 2 of ∼ 200K JSON documents using ChatGPT3, ensuring that ≈
5% of the documents form pairs with at least 80% similarity in content and/or structure (including similar content under diferent structures, and similar structure with diferent content). From this dataset, we evaluated 31 documents (9 matching pairs with similarity ≥ 80%), manu- PPrroommpptt1 Pre4c7i%sion R8e9c%all Ac9cu5%racy ally validated the similarity estimates (and com- Prompt 2 13% 100% 74% puting precision, recall, and accuracy). For each Prompt 3 82% 100% 99% prompt, we generated text representations (Meta Prompt 4 64% 100% 98% Llama 3.3-70B-Instruct), embedded them (all- Table 1: Accuracy, precision, and recall. MiniLM-L6-v2), and compared the 220 crosscollection pairs with cosine similarity; Table 1 reports precision, recall, and accuracy. Three prompts achieve accuracy ≥ 95%; Prompt 3 yields the best precision/recall trade-of, with Prompt 4 close. Metrics are computed over pairs with similarity ≥ 80%, where a produced pair is counted as correct if it is estimated to have similarity ≥ 80% and its computed cosine similarity is also ≥ 80%.
Existing work on fuzzy joins in distributed environments has largely focused on structural or tokenbased similarity measures [43, 44, 45, 38, 46], often combined with optimization techniques such as filtering, prefix indexing or partition-based pruning. Approaches leveraging MapReduce and its derivatives [38, 47, 48] have enabled large-scale execution, with improvements including Bloom filterbased pruning, ball hashing and substring splitting [47] to reduce candidate comparisons. While these methods are efective for structure-oriented matching, for example when using Hamming or edit distance on flattened keys, they struggle to capture cases where two JSON documents are structurally diferent but semantically equivalent. In the context of assessing structural similarity in semi-structured data (e.g., JSON documents), it is worth mentioning JSONVec [49] and ORIGAMI [50], both of which propose methods for encoding JSON structures into embeddings. Recent advances in semantic similarity, such as sentence embeddings and large language models (LLMs), ofer new opportunities, but integrating them eficiently into a distributed fuzzy join pipeline introduces new computational and communication bottlenecks. Moreover, research on semantic similarity in order to create clusters of files has been done [51, 52, 53, 54], and shows that semantic similarity between JSON files is already useful for clustering; we therefore investigate how to repurpose these semantic signals to perform fuzzy joins with higher recall and precision. Computing such a join is non-trivial, especially when performed at scale. In addition to the inherent complexity of comparing high-dimensional semantic embeddings, the integration of language models into a distributed computation pipeline introduces additional challenges, including network overhead, fault tolerance, load balancing, and the serialization of model calls across the cluster. These factors make the deployment of fuzzy join operations in real-world distributed systems particularly demanding.
In this work, we presented a distributed semantic fuzzy join for JSON collections that (1) textualizes each document to preserve content and structure (rule- or prompt-based), (2) encodes the text as embeddings, and (3) applies a scalable similarity-join pipeline over vectors. Future work will expand evaluation on larger datasets, study prompt/embedding choices, investigate extensions of JSONVec/ORIGAMI [49, 50] and study distributed cost optimizations (e.g., batching/caching model calls).
During the preparation of this work, the authors used GPT in order to: Grammar and spelling check. After using these tool/service, the authors reviewed and edited the content as needed and take full responsibility for the publication’s content. arXiv:2302.13971 (2023). [26] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, I. Polosukhin,
Attention is all you need, Advances in neural information processing systems 30 (2017). [27] P. Liu, W. Yuan, J. Fu, Z. Jiang, H. Hayashi, G. Neubig, Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing, ACM computing surveys 55 (2023) 1–35. [28] J. Wei, M. Bosma, V. Y. Zhao, K. Guu, A. W. Yu, B. Lester, N. Du, A. M. Dai, Q. V. Le, Finetuned language models are zero-shot learners, arXiv preprint arXiv:2109.01652 (2021). [29] J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou, et al., Chain-of-thought prompting elicits reasoning in large language models, Advances in neural information processing systems 35 (2022) 24824–24837. [30] T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, J. Dean, Distributed representations of words and phrases and their compositionality, Advances in neural information processing systems 26 (2013). [31] Z. Lin, M. Feng, C. N. d. Santos, M. Yu, B. Xiang, B. Zhou, Y. Bengio, A structured self-attentive sentence embedding, arXiv preprint arXiv:1703.03130 (2017). [32] N. Reimers, I. Gurevych, Sentence-bert: Sentence embeddings using siamese bert-networks, arXiv preprint arXiv:1908.10084 (2019). [33] Y. Bengio, R. Ducharme, P. Vincent, C. Jauvin, A neural probabilistic language model, Journal of machine learning research 3 (2003) 1137–1155. [34] J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, Bert: Pre-training of deep bidirectional transformers for language understanding, in: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), 2019, pp. 4171–4186. [35] W. H. Gomaa, A. A. Fahmy, et al., A survey of text similarity approaches, international journal of
Computer Applications 68 (2013) 13–18. [36] B. Yang, H. J. Kim, J. Shim, D. Lee, S.-g. Lee, Fast and scalable vector similarity joins with mapreduce,
Journal of Intelligent Information Systems 46 (2016) 473–497. [37] R. Baraglia, G. D. F. Morales, C. Lucchese, Document similarity self-join with mapreduce, in: 2010
IEEE International Conference on data mining, IEEE, 2010, pp. 731–736. [38] R. Uhartegaray, L. D’Orazio, M. Damigos, E. Kalogeros, Scalable computation of fuzzy joins over large collections of json data, IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) (2023). [39] A. Okcan, M. Riedewald, Processing theta-joins using mapreduce, in: Proceedings of the 2011
ACM SIGMOD International Conference on Management of data, 2011, pp. 949–960. [40] M. S. Charikar, Similarity estimation techniques from rounding algorithms, in: Proceedings of the thiry-fourth annual ACM symposium on Theory of computing, 2002, pp. 380–388. [41] A. Andoni, P. Indyk, T. Laarhoven, I. Razenshteyn, L. Schmidt, Practical and optimal lsh for angular distance, Advances in neural information processing systems 28 (2015). [42] X. Hu, Y. Tao, K. Yi, Output-optimal parallel algorithms for similarity joins, in: Proceedings of the
ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems, 2017, pp. 79–90. [43] T. Hütter, N. Augsten, C. M. Kirsch, M. J. Carey, C. Li, JEDI: These aren’t the JSON documents you’re looking for..., SIGMOD ’22 (2022). [44] A. Sevim, A. Eldawy, E. P. Carman, M. J. Carey, V. J. Tsotras, Fudj: Flexible user-defined distributed joins, IEEE International Conference on Fuzzy Systems (2024). [45] T. Mizokami, S. Bou, T. Amagasa, Subtree similarity search based on structure and text, in: Big
Data Analytics and Knowledge Discovery, 2024. [46] M. Yu, G. Li, D. Deng, J. Feng, String similarity search and join : A survey, Frontiers of Computer
Science (2015). [47] T.-T.-Q. Tran, T.-C. Phan, A. Laurent, L. D’Orazio, Optimization for large-scale fuzzy joins using fuzzy filters in mapreduce, IEEE International Conference on Fuzzy Systems (2020). [48] A. Drissi, L. d’Orazio, M. Damigos, Scalable structural similarity analysis of json documents using mapreduce, IEEE International Conference on Fuzzy Systems, Jul 2025 (2025). [49] W. Woof, K. Chen, A framework for end-to-end learning on semantic tree-structured data, arXiv preprint arXiv:2002.05707 (2020). [50] T. Rückstieß, A. Huang, R. Vujanic, ORIGAMI: A generative transformer architecture for predictions from semi-structured data, arXiv preprint arXiv:2412.17348 (2024). [51] U. P. Dharmaraj, P. S. Thilagam, JSON document clustering based on schema embeddings, Journal of information science (2022). [52] U. P. Dharmaraj, P. S. Thilagam, JSON document clustering based on structural similarity and semantic fusion, in: Proceedings of International Conference on Computational Intelligence and Data Engineering, 2023. [53] D. Chandrasekaran, V. Mago, Evolution of semantic similarity - a survey, ACM Computing Surveys (2021). [54] D. R. Santana, P. H. S. Lima, L. A. Ribeiro, EM-Join: Eficient entity matching using embeddingbased similarity join, in: Proceedings of the International Conference on Enterprise Information Systems, 2025.