Tabular data is widely available on the web and in private data lakes run by commercial companies or research institutes. However, data that is essential for a specific task at hand is often scattered throughout numerous tables in these data lakes. Accessing this data requires retrieving the relevant information for the task. One approach to retrieve this data is through table augmentation. Table augmentation adds an additional attribute to a query table and populates the values of that attribute with data from the data lake. My research focuses on evaluating methods for augmenting a table with an additional attribute.
these data lakes, such as Google’s Dataset Search, which relies on a keyword search that exploits a table’s metadata to find relevant tables [ 1]. Recognising the heterogeneity and scarcity of metadata, table augmentation pipelines explore data-driven search beyond keyword search [2, 3]. For table augmentation, a user provides an initial query table. This query table can be augmented by adding new columns, new rows, and completing cells with relevant data from a data lake [4]. In my research, I focus on augmenting a query table with a new column that is populated with content from a table corpus also known as augmentation by attribute name [2]. The query table and column header are user-defined. Figure 1 shows an example. Cell completion is closely related but is not the Published in the Workshop Proceedings of the EDBT/ICDT 2023 Joint 1http://webdatacommons.org/structureddata/schemaorgtables/
tributions. The paper is organised as follows. Section 2 discusses related work. My general work plan is given in Section 3. Section 4 introduces work that has already been done and Section 5 concludes my research plans.
of data sources for tabular data as it occurs in real-world data lakes. For the empirical evaluation, I will define a set of benchmark query tables including ground truth information for each of the datasets if the necessary re- 3. Work Plan sources are not provided in the related work [3]. Through the benchmark query tables, all methods are compared This section presents my work plan. To provide a comon a common basis. By profiling the datasets and sys- mon ground for evaluating table augmentation methods, tematically analysing the errors made by the evaluated I present table corpora that difer in size, source, and methods, the strengths and weaknesses of the methods content representing real-world data lakes. I then disin terms of pipeline accuracy and pipeline runtime can cuss the table augmentation methods for Data Search, be identified. Information Extraction and Data Fusion I will experiment
With my PhD research, I will make the following con- with. tributions: • Introduction of neural methods for data search, information extraction and data fusion in the con- I evaluate table augmentation methods on the large-scale text of table augmentation. table corpora WDC Table Corpus and Web Tables. Through • Introduction of benchmark query tables for the the diversity of the table corpora, I aim to represent the table corpora schema.org table corpus and Web heterogeneity of tabular data present in real-world data Tables. lakes. For this evaluation, the table corpora are profiled • Evaluation of new methods, existing methods to identify dataset-specific challenges and a set of benchand complete pipelines for table augmentation mark query tables will be defined in order to compare on benchmark query tables in terms of accuracy the table augmentation methods on a common basis. and runtime to systematically identify strengths and weaknesses.
3.1. Table Corpora for Evaluation for tasks such as data search and entity matching. Entity matching aims to identify records in two datasets that describe the same real-world entity [8]. To reduce these runtimes, entity resolution pipelines consist of two parts: a blocker, which applies a a computationally cheap method to select candidate pairs of records, and and a matcher, which then extracts matching pairs from this set using more set using more expensive methods [8]. Both blocking [9, 10] and matching [11, 12, 13] have recently been successfully tackled with deep learning approaches.
My research focuses on experimenting with neural and symbolic methods that find a matching record for each record in the query table. Another approach to finding matching records in the data lake is to search for joinable tables based on an explicitly mentioned column [14]. If a joinable table contains the target attribute value being searched for, the query table is populated accordingly.
WDC table corpus2 consists of 4.2 million relational tables generated by extracting schema.org3 annotations from the Common Crawl and grouping the annotations by class and host. All tables in this corpus share a common schema. By removing schema matching a focus can be put on other tasks of the table augmentation pipeline.
WebTables and Dresden Web Tables contain 59M to 90M relational tables extracted HTML tables in the Common Crawl [5, 6]. The heterogeneity of tables and their usage in related work make the table corpora interesting for my research [4].
2http://webdatacommons.org/structureddata/schemaorgtables/ 3https://schema.org/ 3.2. Table Augmentation pipeline
corpus, table augmentation pipelines have to deal with data search, schema matching, information extraction and data fusion [4]. Table 1 shows my timeline to work on the specific tasks.
Data Fusion. Data search delivers lists of matching records for the query table records. Augmenting a record based on matching records can lead to data conflicts. Data fusion tries to solve these conflicts. Classic conflict resolution methods are instance- or metadata-based. Examples of instance-based methods are majority vote or averaging conflicting values [ 8]. Source quality [21] and minimal set coverage [22] are metadata-based and exploit provenance information to resolve conflicts. I plan to compare the classic methods to generative LLMs [23] and retrievalaugmented generative LLMs [24]. Generative LLMs memorize knowledge from a data lake during training and directly predict values of the target attribute [23]. For the retrieval-augmented generative LLMs, the retrieved conflicting records are added to the context of the generative LLMs, which resolve the conflict by predicting the values of the target attribute [24]. Both approaches have been successfully applied to related NLP tasks [23, 24] and are promising for data fusion, too.
Data Search. Data search and more specifically record My initial experiments deal with entity resolution pipelines search aims to find records in a data lake, which match that aim to identify records across two datasets that dethe records of a query table. Two records match if they scribe the same real-world entity [8]. Since comparing all describe the same real-world entity. Record search meth- record pairs between two datasets can be computationods exploit schema, context and content of a query table ally expensive, entity resolution is usually tackled by a to find matching records in a table corpus [ 4, 15]. Exist- blocking and a matching step. Blocking applies a compuing symbolic methods use approaches like calculating the tationally cheap method to remove non-matching record edit-distance of table headers and the Jaccard similarity of pairs and produce a smaller set of candidate record pairs two records [3] or measuring the similarity through the reducing the workload of the matcher. During matching vector product of TF/IDF-weighted term vectors to find a more expensive pair-wise matcher generates a final matching records [2]. Existing neural methods embed set of matching record pairs. In the context of these extable records into a high-dimensional space and apply periments, I propose SC-Block, a supervised contrastive nearest neighbour search to retrieve matching records for blocking method which combines supervised contrastive query table records [9, 10]. My initial experiments as pre- learning for positioning records in an embedding space sented in Section 4 deal with entity matching pipelines. and nearest neighbour search for candidate set buildInformation Extraction. Information extraction is only ing. In addition to pairs completeness and candidate set relevant if the content of the target attribute is not ex- size, I report F1 scores and runtimes of complete entity plicitly shared in the table corpus, but has to be extracted resolution pipelines. I do this to evaluate SC-Block’s imfrom another attribute. For example, if a user searches pact on complete entity resolution pipelines. SC-Block is for the colours of a set of products, this information may benchmarked against eight state-of-the-art blockers and be contained in the attribute description and needs to combined with four state-of-the-art matchers. On three be extracted before the target attribute colour can be product-matching datasets from related work [9, 10], SCpopulated in the query table. I will experiment with ap- Block creates the smallest candidate sets and pipelines proaches that finetune large language models (LLM) to with SC-Block run 1.5 to 2 times faster compared to the extract attribute-value pairs [16] and in-context learning benchmarked blockers without afecting the F1 score of where a LLM predicts the attribute-values pairs based the pipeline. These datasets are rather small, which might on a context augmented with a few examples [17]. If lead to runtime efects resulting from a large vocabulary the extracted attributes do not match the target attribute, size being overlooked. In order to measure runtimes schema matching might be helpful to match extracted in a more challenging setting, I introduce a new benchattributes to the target attribute. In this context, I will mark dataset featuring a large vocabulary of terms used test label-based, instance-based, structure-based, or com- within entity descriptions. On this large-scale benchbined methods for schema matching [18, 19, 20]. mark dataset, pipelines utilizing SC-Block and the bestperforming matcher execute 8 times faster than pipelines utilizing the second best-performing BM25 blocker with the same matcher reducing the runtime from 2.5 hours to 18 minutes, clearly compensating for the 5 minutes that are required for training SC-Block. These results are promising for table augmentation because they show how the symbolic data search in related work [2, 3] can be accelerated on large-scale datasets.
PhD. I will present new neural methods within table augmentation pipelines. The methods will be benchmarked against existing methods on two sets of benchmark query tables defined to compare complete table augmentation pipelines and methods for specific table augmentation tasks on a common basis. The goal of complete pipelines is to augment a query table with high accuracy. In addition, I will compare the runtimes of complete pipelines and evaluate how new methods for data search and information extraction can reduce the overall runtime without negatively afecting the accuracy of the pipeline. By proifling the benchmark query tables and their corresponding table corpora, I will identify the strengths and weaknesses of both complete table augmentation pipelines and methods for specific pipeline tasks. duplicate detection, Springer, Berlin, Heidelberg, 2012. [9] S. Thirumuruganathan, H. Li, N. Tang, Deep learning for blocking in entity matching: a design space exploration, VLDB 2021 (2021) 2459–2472. [10] R. Wang, Y. Li, J. Wang, Sudowoodo: Contrastive
Self-supervised Learning for Multi-purpose Data Integration and Preparation, 2022. ArXiv:2207.04122. [11] Y. Li, J. Li, Y. Suhara, A. Doan, W.-C. Tan, Deep Entity Matching with Pre-Trained Language Models,
VLDB 2020 14 (2020) 50–60. ArXiv: 2004.00584. [12] U. Brunner, K. Stockinger, Entity matching with transformer architectures - a step forward in data integration, in: EDBT 2020, 2020. [13] R. Peeters, C. Bizer, Dual-objective fine-tuning of
BERT for entity matching, in: VLDB 2021, volume 14 10, New York, NY, 2021, pp. 1913–1921. [14] E. Zhu, D. Deng, F. Nargesian, R. J. Miller, JOSIE:
Overlap Set Similarity Search for Finding Joinable Tables in Data Lakes, SIGMOD ’19, New York, NY,
USA, 2019, pp. 847–864. [15] N. Barlaug, J. A. Gulla, Neural Networks for Entity Matching: A Survey, ACM TKDD 15 (2021) 52:1–52:37. [16] X. Zhang, C. Zhang, X. Li, OA-Mine: Open-World
Attribute Mining for E-Commerce Products with Weak Supervision, in: WWW ’22, ACM, Virtual
Event, Lyon France, 2022, pp. 3153–3161. [17] Q. Dong, L. Li, D. Dai, A Survey on In-context Learn
ing, 2023. ArXiv:2301.00234 [cs]. [18] E. Rahm, P. A. Bernstein, A survey of approaches [1] N. Noy, M. Burgess, D. Brickley, Google Dataset to automatic schema matching, VLDB ’01 (2001) Search: Building a search engine for datasets in an 334–350.
open Web ecosystem, in: WebConf ’19, 2019. [19] C. Koutras, G. Siachamis, A. Ionescu, Valentine: [2] M. Yakout, K. Ganjam, K. Chakrabarti, InfoGather: Evaluating Matching Techniques for Dataset Disentity augmentation and attribute discovery by covery, in: ICDE ’21, 2021, pp. 468–479. holistic matching with web tables, SIGMOD ’12, [20] R. Shraga, A. Gal, H. Roitman, ADnEV: cross2012, pp. 97–108. domain schema matching using deep similarity ma[3] O. Lehmberg, D. Ritze, P. Ristoski, The Mannheim trix adjustment and evaluation, VLDB ’20 (2020) Search Join Engine, Journal of Web Semantics 35 1401–1415.
(2015) 159–166. [21] X. L. Dong, E. Gabrilovich, K. Murphy, Knowledge[4] S. Zhang, K. Balog, Web Table Extraction, Retrieval, based trust: estimating the trustworthiness of web and Augmentation: A Survey, ACM Trans. Intell. sources, VLDB ’15 (2015) 938–949.
Syst. Technol. 11 (2020) 1–35. [22] J. Eberius, M. Thiele, K. Braunschweig, Top-k entity [5] O. Lehmberg, D. Ritze, R. Meusel, A Large Public augmentation using consistent set covering, in: Corpus of Web Tables containing Time and Context SSDBM ’15, Association for Computing Machinery, Metadata, in: WWW ’16, 2016, pp. 75–76. New York, NY, USA, 2015, pp. 1–12. [6] J. Eberius, K. Braunschweig, M. Hentsch, Building [23] C. Rafel, N. Shazeer, A. Roberts, Exploring the the Dresden Web Table Corpus: A Classification Limits of Transfer Learning with a Unified Text-toApproach, in: BDC ’15, 2015, pp. 41–50. Text Transformer, Machine Learning Research 21 [7] M. J. Cafarella, A. Halevy, N. Khoussainova, Data (2020) 1–67.
integration for the relational web, VLDB ’09 2 (2009) [24] P. Lewis, E. Perez, A. Piktus, Retrieval-Augmented 1090–1101. Generation for Knowledge-Intensive NLP Tasks, in: [8] P. Christen, Data matching : concepts and tech- NeurIPS 2020, volume 33, 2020, pp. 9459–9474. niques for record linkage, entity resolution, and