A wealth of knowledge is contained in tabular data, and there are a vast number of e orts to model and capture this knowledge. Unfortunately, these e orts have disparate inputs, outputs, and goals hindering research progress and making table understanding tools di cult to use in practice. In this paper, we propose a table understanding framework that formalizes the problem of understanding tabular data into three distinct subtasks: cell classi cation, block detection, and relation prediction. We introduce a common API for table understanding systems that supports a host of existing approaches and allows easy development of new approaches. Our framework supports approaches that range from heuristic rules to probabilistic models, allows outputs that span simple, correlational tuples to sophisticated, semantic knowledge graphs, and provides tools for visualizing model outputs and transforming complex tabular data into attened relational dataframes. .
Copyright © 2019 for this paper by its authors. Use permitted under Creative
Commons License Attribution 4.0 International (CC BY 4.0)
We formulate the table understanding problem based on the exhaustive e orts
on building theoretical models of tabular data [
Relationships between cells can be expressed at a simple, correlational level (identifying a tuple of associated values) or at a deep, semantic level (producing a knowledge graph with ontological mappings of entities, types and properties). We believe there is a need to support the full spectrum of table understanding methods. For tables with complex layouts and specialized domains, correlational associations may be immensely valuable. For conventional layouts with mainstream entities, a more knowledge-driven output may be desired.
Our table understanding framework provides the overall architecture and interface to support the full spectrum of table understanding approaches, from structural and syntactic to fully semantic systems. We organize this framework three primitive operations, illustrated in Figure 1: cell classi cation, block detection, and relation prediction. These three tasks can be viewed as answering three basic questions: Cell Classi cation: What type of data does this cell contain? Block Detection: What logical groupings of cells are present? Layout Relationships: How are groups of cells related to each other? 2.1
The goal of cell classi cation is to assign a label, (l(cj; k)), to each cell, CC(T ) !
l1;1 : : : l1c;K : : : lJc;1 : : : lJc;K . The domain of cell labels can be customized based
c
on the complexity of the table understanding system, for example producing
simple datatype labels (string, integer, oat, datetime) for a syntactic system,
more functional labels (metadata, header, attribute, value) based on table
structure [
A Common Framework for Developing Table Understanding Models
Block detection identi es a region composed of individual cells that share a common functional role in a table, de ned as a block (Bi). Blocks can be de ned hierarchically, such that a single large block can be composed of several smaller blocks (and regions), some of which may be further subdivided. Thus a block can be de ned as either a rectangular region of cells, or the union of a set of sub-blocks, Bi , [Bi1; : : : Bikj fca;b : : : cx;yg]. Block detection systems identify a set of blocks in a table and assign a label to each block, BD(T ) ! B1; l1b : : : Bs; lsb . Similar to cell labels, block labels can also be de ned at several levels, ranging from syntactic (headers, notes, attributes, values) to semantic (entities from a particular domain). 2.3
The nal task in table understanding is determining the relational structure between blocks. Relationships can take many forms, but common relationships include subset relationships (e.g., a block of year attributes may be related to a block of month attributes because the months are temporal subsets of the years) and attribute-value relationships (e.g., a temperature measurement may have attributes of the year and month of measurement). Each relationship can be speci ed as a labeled, directed edge between blocks, RP(T ) ! [hBs; Bt; l1ri : : : hBu; Bv; lnri]. As with other subtasks, the label space can be dened at di ering levels of granularity, from basic subset, indexing, and attribute relationships to ontologically meaningful properties (e.g., age, location, source). 3 Using the table understanding formalism introduced in the previous section, we have developed a table understanding framework that provides common abstractions and tools for all three table understanding subtasks. We summarize and illustrate the key features of our framework below.
Implementation: The table understanding framework is implemented as a set of Python APIs and accompanying documentation.
Common Representations: The framework de nes a common representation for tabular data that can be loaded as CSV or Excel-style formats, and supports translation tools for Web tables.
APIs: In addition, the framework de nes appropriate abstract classes for each table understanding subtask (as shown in Figure 2). We also provide an elegant method to supply custom labels for each task, so that structural and semantic modeling approaches can reuse the same abstract classes. Label outputs are speci ed as probability distributions over label classes to support machine learning models that produce scored outputs.
Reference Implementations: The table understanding framework supports several reference implementations, including baseline models that demonstrate simple, functional outputs and more sophisticated CRF-based cell classi cation and relation prediction models and a decision-tree based block detection algorithm.
Tools: The framework supports several tools to allow developers to visualize and use the outputs of table understanding models. One such output, shown in Figure 3 is a colorized version of the table that shows cell types and block boundaries. Additional rows are added at the bottom of the sheet to summarize block relations. Another such output is a attened dataframe representation (with a single record per row) generated by using relational mappings between blocks. Open Source Release: Our table understanding framework API is available at https://github.com/usc-isi-i2/isi-table-understanding