The early detection and classification of mechanical faults in rotating machinery is essential for predictive maintenance. We propose a novel neuro symbolic framework that integrates a one-dimensional Transformer encoder with Logic Tensor Networks and a dynamic rule generation module. The Transformer extracts temporal and spectral features from raw vibration segments via multi-head attention, while logic rules enforce label consistency and similarity constraints that adapt to evolving cluster patterns. We tested our approach on two benchmark datasets: on the OEDI recorded by using SpectraQuest's Gearbox Fault Diagnostics Simulator, where it achieves an F1 score of 0.992, and on the nine-class UoC gear fault data it reaches 0.899 versus 0.756 for a Transformer alone, thus delivering accurate and interpretable fault classification.
Rotating machines are used in many industrial settings, and
hidden faults have been shown to cause costly shutdowns or
safety risks [
Deep end-to-end models. Convolutional, recurrent and
autoencoder networks now learn features directly from raw
signals, outperforming classical methods on CWRU, IMS and
MFPT benchmarks and adapting through augmentation or
domain adversaries [
Neuro-symbolic promise and objective. Integrating
neural representation learning with symbolic reasoning
can inject domain rules, boost data eficiency, and produce
human-readable explanations [
Transformer architectures have recently become a backbone
for machinery diagnosis. The Time-Series Transformer
(TST) lifted CWRU accuracy to 99.1%, four points above
CNN/LSTM baselines [
Eforts to inject expert knowledge have given rise to several neuro-symbolic approaches. LTNs embedded in LSTM backbones enhance generalisation when labels are limited [25]. DeepProbLog combines neural perception with probabilistic logic programming, and forces models to predict intermediate engineering quantities, enabling subsequent auditability [26].
Despite these advances, Transformer-based models rarely encode formal knowledge, whereas LTN systems rely on legacy feature extractors. To date, no study has combined a Transformer backbone with a neuro-symbolic reasoning layer. Our work unifies self-attention representation learning and diferentiable first-order logic in a single end-to-end framework aimed at improving accuracy, data eficiency and transparency.
We fuse first-order reasoning with a lightweight 1-D Transformer to classify sample vibration segments(best results with 20) under explicit consistency constraints. Symbolic rules are encoded with Logic Tensor Networks (LTNs) [27] their penalisation signals update the network, letting prior knowledge shape the learned features.
An LTN grounds every formula in a truth degree () ∈
[
We adopt the aggregated -mean error (ApME), ME(1:) = 1 − ︁( 1 ∑︁(1 − ) )︁ 1/
, with =2; low-valued clauses thus receive larger gradients.
Each class is a fuzzy predicate (x) = ˆ, i.e. the network softmax output. For a labelled segment x with ground truth we impose (x),
¬(x) ( ̸= ), using product - and -norms plus the Goguen implication; run-time overhead is negligible. (1) (2) (3)
Each vibration segment is a length-20 time series x ∈ R1×20 with a categorical label ∈ {1, . . . , }, where is the number of classes in the current dataset. The proposed model attaches a lightweight one-dimensional Transformer to the LTN layer, so that symbolic constraints are applied directly to the learned representation.
The raw signal is split into three overlapping patches of ten samples (stride = 5). A 1×10 convolution projects each patch to a 64-dimensional token u. A learnable class token zcls is prepended to these tokens and sinusoidal positional encodings are added, yielding the input sequence Z0 = [︀ zcls ‖ u1 ‖ u2 ‖ u3]︀ . Six pre-norm Transformer layers, each with eight attention heads, refine the sequence. With only four tokens (one CLS + three patches) the memory footprint remains low while self-attention still captures long-range dependencies. The updated CLS token provides a global summary, whereas the three patch tokens are average– and max-pooled, concatenated, and fed to a two-layer MLP that outputs class probabilities pˆ(x) = (ˆ1, . . . , ˆ ). These probabilities ground the LTN predicates (x) = ˆ, allowing the logic rules to influence the entire encoder.
To regularise the embedding space, similarity rules are refreshed after every training epoch. For each class we run -means on the current embeddings and retain two centroids { ,1, ,2}, suficient to distinguish the low- and high-load regimes observed in practice. Proximity of x to centroid is measured by a Gaussian kernel ,(x) = exp[︁− 12 ⃦⃦ f (x) − ⃦ ]︁ ,⃦ 2 , which yields the soft implication ,(x) ⇒ (x). (4) Rule (4) encourages any sample that lies close to a class centroid to be assigned that class, yet it never conflicts with the primary label rules in (3). (5) (6) Training objective. Let { }=1 be the truth values of all instantiated rules. We define the satisfiability aggregation SatAgg = 1 − ⎯ ⎸⎸ 1 ∑︁(︀ 1
⎷ =1 − ︀) 2 , which attains its maximum value of 1 when every rule is fully satisfied. The overall loss is then ℒ = 1 − SatAgg + ‖Θ‖ 2 2 , where Θ comprises all learnable parameters and = 10 −3 is the weight-decay coeficient.
All experiments ran on a 16 GiB Linux workstation with an AMD EPYC CPU. Datasets are UoC gearbox [28] (nine fault modes, 20 kHz): HEA (healthy), CTF (chippedtooth PGB), MTF (missing-tooth PGB), RCF (root-crack PGB), SWF (surface-wear PGB), BWF (ball-wear bearing), CWF (composite-wear races), IRF (inner-race bearing), ORF (outer-race bearing); and data from OEDI [29] recorded by using SpectraQuest’s Gearbox Fault Diagnostics Simulator, refered to as SpectraSimulator (healthy, broken-tooth).
Pre-processing Continuous vibration signals are segmented into windows of 20 samples with a stride of 10 inputs). Signals are standardised; faulty (yielding 1 × 20 frames are discarded. Class balance is enforced by uniform sampling. Splits use a stratified 80/20 train–test ratio, and results are averaged over two folds.
Training Details Training uses Adam with a learning rate of 10−4
and ℓ2 weight-decay 10−3 . When training the vanilla variant, training has been kept identical.
UoC.
Our hybrid model attains 89.9 % accuracy and macroF1 = 0.900 as in Table 1, outperforming the vanilla Transformer with an F1 = 0.756. Gains are largest for data-sparse classes such as HEA and CTF, highlighting the benefit of logical regularisation.
Results comparison on the UoC dataset: Transformer–LTN vs. vanilla Transformer (the transformer itself has been kept identi
0.830 0.895 0.921 1.000 0.928 0.860 0.906 0.847
We introduced a compact neuro-symbolic pipeline that unites a 1-D Transformer with LTNs, enabling end-to-end learning under first-order constraints. Experiments on real and simulated gear faults show that logical supervision raises both accuracy and class balance without extra computation, while every prediction remains traceable to interpretable rules.
This research was supported by the project “Romanian Hub for Artificial Intelligence - HRIA”, Smart Growth, Digitization and Financial Instruments Program, 2021-2027, MySMIS no. 351416 and partially supported by the European Union, via the oc1-2024-TES-01-19 sub-grant issued and implemented by the ENFIELD project, under the grant agreement No 101120657.
During the preparation of this work, the authors used Writefull for Overleaf to perform grammar and spelling checks. No figures were generated by AI. The authors reviewed and edited all suggestions and take full responsibility for the publication’s content.