In recent years, the integrity of data used in machine learning pipelines has become increasingly critical, as even state-of-the-art models are constrained by the quality of their input. Among the various threats to data reliability, data smells-subtle and often semantic anomalies-pose unique challenges due to their nuanced and context-sensitive nature. This paper presents an innovative approach to detect semantic data smells using BERT, a transformer-based language model originally designed to understand natural language. We focus specifically on two underexplored categories: Split Value Smells and Multiple Value Smells, which respectively reflect improper data fragmentation and conflation. To facilitate model training, we constructed a large and heterogeneous corpus comprising synthetic and real datasets, utilizing data augmentation techniques via Faker and SDV. We trained two binary classifiers using fine-tuned BERT models, achieving high performance (F1 scores of 0.88 and 0.98) in detecting these smells. Our findings demonstrate that transformer-based models are not only efective in capturing structural patterns in tabular data but also capable of generalizing across diverse semantic anomalies. This work establishes a foundation for the broader application of language models in data quality assurance, opening new avenues for semantic-level data cleaning automation.
As machine learning (ML)-enabled systems evolve from experimental prototypes to deployed
applications, ensuring their reliability requires attention to all stages of the ML lifecycle, from data preparation
to model deployment and monitoring [
In this preliminary work, our goal is to investigate initial opportunities for modeling and identifying subtle semantic anomalies in tabular data. To start exploring these capabilities, we focused on two simple data quality issues related to the distribution of information among the columns of a dataset: Split Value Smells and Multiple Value Smells. By transforming tabular data into textual representations,
CEUR Workshop
ISSN1613-0073 we leveraged the ability of BERT to capture semantic patterns and irregularities. Our approach supports automated semantic data quality checks that can be integrated into MLOps pipelines, reducing manual overhead, and improving the robustness of ML-enabled systems over time.
To train and evaluate our models, we created a synthetic dataset using a combination of the Faker
library [
• We build two meta-datasets to explore further research on semantic data smells.
In this section, we outline key data quality issues relevant to ML-enabled systems, establishing the broader context in which these issues arise. We then review related work on the detection of such issues, with a particular focus on recent advances in data smell identification.
In MLOps pipelines, data quality plays a central role in ensuring long-term performance, reliability,
and maintainability of ML-enabled systems. Unlike traditional software systems, where source code is
the primary artifact, ML-enabled systems are highly dependent on data, an inherently dynamic and
evolving asset. Renggli et al. [
In addition to classical data quality issues, researchers have introduced the concept of data smells,
recurring patterns in data that suggest latent quality problems. Foidl et al. [
The former refers to cases where a single piece of information is unnecessarily distributed across multiple columns, for example, splitting a date of birth into separate month, day and year components, as illustrated in Table 1. Such fragmentation can hinder downstream processing and reduce data usability. The latter involves instances where multiple distinct pieces of information are inappropriately merged into a single column, for example, by storing both an email address and a phone number in the same field, as shown in Table 2. This practice complicates data interpretation and parsing. Both smells represent semantic anomalies that go beyond syntactic validation and require contextual understanding to detect and resolve efectively. These semantic issues are particularly problematic in production-grade ML systems, where they can silently propagate through automated workflows, resulting in weak models and unreliable predictions. However, such techniques often fail to capture more complex violations of the data’s semantic intent, such as improperly merged or split values. Overcoming these limitations requires more adaptive and context-aware strategies that go beyond rule-based checks and can understand the structure and meaning of the data within its application context. … → → → → → →
Recent studies have increasingly focused on formalizing and detecting data smells, recurring patterns
in data that may indicate latent quality issues. Foidl et al. [
In parallel, recent advances in transformer-based models—particularly BERT and its derivatives—have
shown strong potential for processing tabular and hybrid (text-tabular) data. These models have been
successfully adapted to capture the structure and semantics of tables in diverse tasks. For example,
TaBERT [
While existing work on data quality predominantly addresses structural and syntactic issues, this work explores the capabilities of transformer-based models of detecting semantic data smells that require semantic understanding.
The goal of this study is to explore whether BERT can support automated detection of semantic smells in tabular data sets. We focus our investigation on two key categories: Split Value Smells, which occur when a single logical entity (e.g., a person’s full name or an address) is improperly fragmented across multiple columns; and Multiple Value Smells, which result from the incorrect merging of distinct values, such as a phone number and an email, into a single cell.
This study is carried out from a dual perspective. On the one hand, our aim is to contribute to the
body of knowledge by understanding the applicability of language models in semantic data quality
tasks, thereby addressing the interests of researchers in empirical data-centric ML. On the other hand,
we respond to the practical needs of developers and data engineers, who routinely face the challenge of
detecting and correcting these types of anomaly at scale. In terms of reporting, we follow the guidelines
of Wohlin et al. [
To conduct our study, we constructed a synthetic dataset of labeled tabular entries. These were designed to represent realistic examples of both clean and anomalous data.
Where possible, we introduced smells based on known real-world issues observed in open datasets, supplemented by manually engineered cases to ensure balanced coverage across scenarios.
All records were serialized in a column-based format suitable for the input structure of BERT, and each instance was annotated with a binary label corresponding to the presence of a smell, allowing us to train and evaluate the models under consistent conditions. in more detail in the following sections.
Model Fine-Tuning
Kaggle Data Augmentation Faker SDV 20 Datasets for Smells Split Value Multiple
Smell Value Smell 4040000,0k0 0 rroowwssppeerr dsamtaesellt Split Value
Smell Multiple Value Smell
BERT
Analysis of results
Bert Fine-Tuning Split Value Multiple
Smell Value Smell
Research Questions SDpelSittmeVcetalillnuge VDaMleuuteeltcSitpmilneegll
Due to the limited availability of well-defined semantic data smells datasets, we constructed two
comprehensive meta-datasets1 for detecting Split Value Smells and Multiple Value Smells, respectively [
To enhance diversity and quantity, we expanded the dataset using Synthetic Data Vault (SDV) [
Data Smell Injection. We injected two data smells: Split Value Smells and Multiple Value Smells. On the one hand, we programmatically fragmented single logical values (e.g., full addresses or dates) into multiple columns. On the other hand, we concatenated logically distinct values into single fields, such as merging “email” and “phone number” into a single string-separated column.
Tables 1 and 2 show examples of these transformations. In total, we generated 40 base datasets (20
per smell type), each consisting of 5,000 rows. These were then aggregated into two meta-datasets
with approximately 400,000 labeled records. Due to the injection strategy, the class distribution in each
raw meta-dataset was naturally imbalanced. For example, in the case of Split Value Smells, the smelly
examples were overrepresented because each split introduced multiple rows. Conversely, Multiple Value
Smells tended to be underrepresented due to the merging of values into fewer fields. In addition, row
deduplication is performed across all data sets to ensure diversity. Finally, an undersampling strategy
was applied to mitigate learning bias. The final version used for training and evaluation contained a
balanced 50/50 distribution of positive and negative labels. Specifically, we applied stratified sampling
to preserve variation between diferent base datasets and structural patterns. After this step, two
meta-datasets, each containing 5,000 rows, are created.
1A meta-dataset refers to a collection of datasets, where each entry captures general information about an individual dataset.
2https://www.kaggle.com/
3SDV diagnostic tool: https://docs.sdv.dev/sdv/single-table-data/evaluation/diagnostic
Preprocessing and Labeling. Each row in the meta-dataset was labeled as “smelly” or “not smelly”
based on whether it contained a known transformation. Additionally, to prepare the data for model
training with BERT, we transformed each row into a structured text format (e.g., Feature: Value),
allowing the model to treat tabular records as natural language sequences, as recommended in the recent
literature [
To efectively detect semantic data smells within tabular data, we adopted a fine-tuning approach on
a pretrained BERT model. This decision leverages the strong contextualization capabilities of BERT,
which, although initially designed for natural language processing tasks, has recently demonstrated
high efectiveness in structured data domains [
Preprocessing and Input Representation. The fine-tuning process began transforming tabular
records into text sequences. Each row was serialized into a single textual string by concatenating
feature names with their corresponding values (e.g., Feature: Value). This format ensures that the
BERT tokenizer can efectively segment and encode the data while preserving the structure of the
original features. All inputs were tokenized using the BERT base uncased tokenizer and padded or
truncated to a maximum sequence length of 128 tokens. Given the binary nature of our task (smelly
vs. not smelly), each data sample was associated with a binary label. The input sequences were then
divided into training (80%) and validation (20%) sets using stratified sampling to maintain label balance.
Model Architecture and Training Setup. We used the bert-base-uncased model from the
Hugging Face Transformers library.4 A classification head—consisting of a dropout layer followed by a
dense layer—was added on top of the final [CLS] token representation. This setup enables binary
classification by mapping the contextualized embedding to a single probability score. The training was
carried out using AdamW optimizer [
To support replicability and facilitate future research, we have made all materials, including scripts and
datasets, publicly available in a permanent online appendix [
This section reports on the findings of the study. To improve clarity, we discuss each RQ separately.
As shown in Table 3a, the model achieved an accuracy of 88.99%, with perfect precision (1.000) and an F1 score of 0.8763. Although these results confirm the ability of the model to detect Split Value Smells with high reliability, the recall was 0.7799, indicating that some true positives were missed.
Table 3b shows the confusion matrix for the Split Value Smells model. In particular, the model did not produce false positives, resulting in perfect precision. However, it did not detect 11% of smelly instances, which were mistakenly classified as clean (false negatives), highlighting a common trade-of in binary classification: the model favors precision over recall. Such behavior is particularly desirable 4Available at: https://huggingface.co/google-bert/bert-base-uncased detrimental if undetected anomalies significantly impact downstream tasks. when trained on relatively limited data.
One notable challenge lies in the semantic variability of Split Value Smells. Unlike syntactic anomalies, these smells are often context-dependent. For example, splitting an address into “Street”, “City”, and “Zip Code” might be acceptable in some domains but smelly in others if those components are only meaningful when considered as a whole. This ambiguity limits the model’s ability to generalize across domains and may explain the relatively lower recall. Moreover, diferent instances of splitting (e.g., full name into first, middle, and last names) lack a consistent pattern, which increases intra-class variability and training noise. The zero false-positive rate of the model demonstrates that it serves as a reliable iflter for high-confidence cases. In practice, this makes it well suited for human-in-the-loop data quality workflows, where automatic detection aids but does not replace manual review. A flagged instance can be trusted to be problematic, while borderline or context-sensitive cases may still require user judgment.
In summary, BERT can efectively detect Split Value Smell, ofering high precision and generalizability despite the challenges posed by semantic ambiguity. Future improvements could involve integrating auxiliary metadata to capture implicit semantic groupings better.
ILST
RQ1 — Summary of the Results.
BERT efectively detects Split Value Smells with high precision and generalizability. Although the recall is modest due to the semantic variability of smells, the model serves as a strong, highconfidence filter in human-in-the-loop data cleaning workflows.
As shown in Table 4a, the BERT-based model for detecting Multiple Value Smells achieved exceptional results. The model reported an overall accuracy of 98.00%, with perfect precision (1.000) and a remarkably high recall of 0.9599, which led to an F1 score of 0.9795. These metrics indicate that the model not only correctly classifies true positives but also maintains a near-zero false negative rate, substantially outperforming the counterpart model developed for Split Value Smells. precision makes the model both sensitive and specific, a highly desirable trait in anomaly detection. from the training accuracy, implying that the model avoids overfitting and maintains robustness.
The superior performance of the Multiple Value model can be attributed to the more uniform representation of the smells. Unlike Split Value Smells, which vary significantly in their structural form, Multiple Value Smells often exhibit consistent patterns that are more easily captured by the token-based architecture of BERT. These regularities provide stronger learning signals, enabling the model to distinguish between smelly instances and clean data with high reliability. From a practical perspective, the high recall achieved by the model ensures that very few smells are overlooked, making it suitable for autonomous data cleaning tools or alert systems in large-scale data pipelines. Furthermore, perfect precision allows its output to be used confidently without requiring downstream verification, minimizing the risk of incorrect flagging.
In conclusion, BERT is not only capable of detecting Multiple Value Smells but does so with near-optimal performance. The consistent structure of these smells aligns well with BERT’s contextual encoding capabilities, enhancing the utility of the model in detecting semantic level anomalies in tabular data.
ILST
RQ2 — Summary of the Results.
BERT achieves near-perfect detection of Multiple Value Smells, with 98% accuracy, perfect precision, and very high recall. The model is efectively generalized and benefits from the uniformity of the smell structure, making it suitable for reliable automated anomaly detection in tabular data.
The results obtained in this study have important implications for both the development of data quality systems in practice and future research on semantic anomaly detection. By demonstrating that BERT can be fine-tuned to detect data anomalies with high performance, this work opens new perspectives on how language models can be leveraged to address long-standing issues in structured data validation. lFask
From an applied standpoint, our findings suggest that transformer-based models are not only theoretically interesting but also practically helpful in improving data quality workflows. The model trained to detect Multiple Value Smells achieved near-perfect performance, with high recall and perfect precision, meaning that data engineers and analysts can rely on it to flag problematic entries with minimal risk of false positives automatically.
Equally important, the Split Value Smells detector, despite a lower recall, also showed perfect precision, indicating that its predictions are highly reliable when a smell is detected. The model could thus serve as a high-confidence filter embedded in human-in-the-loop workflows, where semantic anomalies are surfaced to data curators or engineers for review.
Furthermore, the text-based input representation—where each row is serialized into a “natural language-like” form—lowers the barrier to integration. It allows this kind of model to be embedded in tools without deep coupling to schema logic or data format assumptions, making deployment across diverse data platforms more feasible.
For researchers, this study highlights the underexplored potential of using language models for structured data quality tasks. Although most prior work in NLP and tabular learning focuses on downstream prediction or representation learning, our results demonstrate that pre-trained models, such as BERT, can be adapted to detect subtle structural inconsistencies typically overlooked by both traditional rule-based systems and data-centric AI methods.
In addition, the successful use of synthetic smells, systematically injected into otherwise clean data, demonstrates a promising way to create large-scale training datasets when labeled examples are scarce. This result could be a useful methodological direction for further research on other types of semantic anomalies, including those related to believability, completeness, or cross-column dependencies. iLghtbul
RWENCH RWENCH
BERT can act as a high-precision semantic smell detector, enabling safe automation in data quality workflows.
The approach fits well within human-in-the-loop pipelines due to its low false-positive rate. This work supports a novel line of research using language models to detect contextual inconsistencies in structured data.
Synthetic smell injection is a scalable solution to train data in semantic validation tasks.
Although the results presented are encouraging, several threats to validity must be acknowledged. Internal Validity. A potential threat to internal validity concerns the synthetic nature of the datasets used. Although we used data generation techniques that simulate realistic tabular structures and semantics, the smells were injected programmatically using deterministic rules. This injection may not fully capture the complexity and variability of smells that arise in organically grown data sets. Furthermore, the use of undersampling to achieve class balance may afect the natural distribution of “smelly” vs. “clean” instances.
External Validity. The external validity of our findings may be limited due to the reliance on synthetic and semi-synthetic datasets. Although the data sets were designed to reflect common patterns in realworld data, we did not perform experiments on large-scale industry data sets or diverse domains such as ifnance, healthcare, or e-commerce. As such, the generalizability of the trained models across diferent contexts remains to be evaluated.
Construct Validity. We defined Split and Multiple Value Smells based on logical criteria and formalized these into transformation rules. However, these definitions may not encompass all the nuances recognized by domain experts. Additionally, converting tabular rows into text sequences for BERT input involves abstraction choices that may omit structural cues present in multi-column formats. Conclusion Validity. The conclusions drawn from this study are based on standard classification metrics, including accuracy, precision, recall, and F1 score. Although these metrics provide a reliable indication of model performance, a further evaluation of operational pipelines or user-facing tools is necessary to assess practical efectiveness. Moreover, the limited number of training epochs and the small amount of available GPU resources could constrain the models’ full learning potential.
This paper investigated the applicability of transformer-based language models, specifically BERT, for the automated detection of semantic data smells in tabular datasets. Our goal was to assess the extent to which BERT can be fine-tuned to identify structural anomalies that require contextual understanding—namely, Split Value Smells and Multiple Value Smells.
Our results revealed that BERT performs well in both scenarios. The model trained for Split Value Smells achieved high precision and generalizability, despite the intrinsic variability of how such smells manifest in real-world schemas. In contrast, the model trained to detect Multiple Value Smells achieved near-perfect performance, supported by the regularity of the merged value patterns. These outcomes suggest that pre-trained language models can capture subtle semantic cues within structured data when appropriately adapted.
At the same time, our findings highlight that semantic variability and feature inconsistency can impact model recall, particularly with fragmented values, underscoring the need for more comprehensive data representations and possibly multi-column modeling strategies to leverage the potential of BERT in this domain fully.
Our future research agenda includes extending this work to additional categories of semantic smells and exploring cross-dataset generalization. We also plan to integrate metadata-aware modeling and assess the efectiveness of alternative transformer architectures optimized for tabular data. Ultimately, our goal is to support the development of intelligent data validation tools that can operate at the semantic level across diverse domains and formats.
This work has been partially supported by (1) the QUAL-AI national research projects funded by the EU - NGEU and the MUR under the PRIN 2022 program (Contracts 2022B3BP5S), and (2) the project “FAIR” (PE0000013). Additionally, the work is an extended version of Nicolò Gallotta’s Bachelor’s thesis, developed at the University of Salerno in 2024. We gratefully acknowledge his valuable contribution to the early stages of this research.
During the preparation of this work, the authors used ChatGPT-4o and Grammarly to check grammar and spelling.