The proliferation of sophisticated misinformation threatens societal trust, yet most AI detectors operate as opaque 'black boxes,' lacking the verifiable reasoning essential for human oversight and adoption in high-stakes domains. This critical transparency gap demands a new paradigm where interpretability is a core design principle, not a post-hoc feature. In this work, we propose the Explainable Fake News Detection (XFND) framework, a human-centered pipeline that marries expert-guided feature space validation with evidence-anchored explanation synthesis using large language models. Our approach demonstrably improves feature space separability before training, increasing the silhouette score on public datasets like LIAR by up to 63% (from 0.19 to 0.31). On established benchmarks, the resulting system achieves competitive classification performance, reaching a macro-F1-Score of 0.792 on a binary version of LIAR and 0.731 on PolitiFact, while ensuring outputs are well-calibrated and auditable. We conclude that proactively designing for interpretability enables systems that are both highly accurate and trustworthy by design, establishing a new standard for collaborative AI in the fight against disinformation.
The contemporary digital landscape is saturated by an unprecedented volume and velocity of
information, fostering an environment where misinformation and disinformation can propagate unchecked.
This “infodemic” has been profoundly exacerbated by the recent advent of powerful generative artificial
intelligence (AI), which enables malicious actors to create highly plausible, contextually aware, and
persuasive fabricated content at scale, efectively blurring the lines between truth and fiction [
In response, the machine learning community has developed a vast arsenal of automated detection
systems across numerous domains, with modern approaches achieving remarkable accuracy on academic
benchmark datasets [
The academic pursuit of automated fake news detection has produced a rich and diverse body of
literature. As comprehensive surveys illustrate, the field has progressed through several distinct phases,
from early feature engineering to modern LLM-based approaches [
Parallel to the development of detection models has been the rise of Explainable AI (XAI), aiming to
make these complex models more understandable. The application of XAI to fake news and hate speech
detection is a burgeoning area of research, with scholars exploring how to decode the reasoning of these
critical systems [
While post-hoc methods like LIME and SHAP represent a major step forward, their application to
text often yields explanations based on artifacts (like single tokens) that lack suficient context for a
human analyst. More recent eforts have sought to improve the interpretability of these explanations,
for example, by using named entity replacement to make the feature attributions more conceptually
meaningful [
This paper introduces the Explainable Fake News Detection (XFND) framework, a comprehensive, human-centered pipeline designed to bridge the chasm between predictive power and actionable transparency. The XFND framework is built upon a paradigm shift: we contend that interpretability should not be a post-hoc feature but an a priori design constraint woven into the entire modeling lifecycle. To achieve this, we formalize Human-Guided Interactive Validation and Refinement (HGIVR), a novel visual analytics protocol, and Evidence-Anchored Explanation Synthesis (EAECS), a structured process that constrains a Large Language Model (LLM) to generate coherent, natural-language narratives based strictly on pre-computed feature contributions. In the spirit of open science, we provide a complete implementation of our framework, empowering the research community to evaluate and extend our work on public datasets like LIAR [18] and FakeNewsNet [19].
The goal of this study is to improve the interpretability and reliability of fake news detection by tightly integrating human-guided feature-space shaping with evidence-anchored natural-language explanations. To realize this vision, this paper pursues four primary objectives: • To formalize a reproducible HGIVR protocol for expert-guided feature validation, shifting interpretability from a retrospective analysis to a prospective design of the feature space. • To define and implement EAECS, a novel methodology for generating faithful-by-design naturallanguage explanations from low-level feature contributions. • To empirically quantify, on public benchmarks, whether the HGIVR process leads to measurable improvement in class separability and contributes to a high-performing classifier. • To deliver a comprehensive open-source package enabling the research community to verify our ifndings and apply the XFND framework to public corpora.
The pursuit of automated fake news detection is a well-established field that has evolved significantly with advances in machine learning and natural language processing. This section situates our work within this broader context, reviewing key developments in detection models, the parallel rise of explainability, and the emerging paradigms of human-AI collaboration and trustworthy machine learning.
Early work in fake news detection was dominated by models built on handcrafted features.
Researchers focused on extracting interpretable linguistic and stylistic cues from text, such as sentiment
polarity [
The field of Explainable AI (XAI) emerged to address this opacity. Post-hoc, model-agnostic methods
are the most common approach. LIME [
Our framework’s philosophy is deeply informed by research in human-AI collaboration and visual
analytics. Studies have shown that for AI tools to be adopted in critical domains like journalism,
their outputs must align with human cognitive processes and evidentiary standards [
Finally, the EAECS component addresses both the promise and peril of modern LLMs. While the
generative power of LLMs can be weaponized to create fake news [
The XFND framework is a modular, multi-stage pipeline designed to produce accurate, auditable, and human-interpretable classifications of news articles. Its architecture is predicated on the principles of early-stage human intervention and late-stage evidence-grounded synthesis.
The framework processes a given news article through three principal stages. Initially, in the EvidenceCentric Feature Engineering stage, a collection of specialized calculators analyzes the input article. Each calculator extracts a specific, conceptually meaningful feature (e.g., sentiment polarity) and records the exact textual evidence used to compute it. Subsequently, during the Human-Guided Interactive Validation & Refinement (HGIVR) stage, a domain expert uses an interactive visual dashboard to inspect the collective feature space, assess its quality, identify problematic features, and iteratively refine the feature set until it exhibits clear class separability and conceptual coherence. Finally, in the Predictive Modeling and Explanation Synthesis (EAECS) stage, a standard machine learning classifier is trained on the expert-validated feature set. When this model makes a prediction, the EAECS module retrieves the decision, the most influential features, and their stored evidence to generate a final, evidence-anchored natural-language explanation.
The foundation of the XFND framework is a novel approach to feature engineering where the collection of evidence is a first-class citizen. Let a corpus be defined as = {(, )}=1, consisting of news articles and their corresponding binary labels ∈ {0, 1}. We define a set of feature calculators { }=1, where each calculator is a function that maps an input article to a tuple: (,, Meta ()). Here, , is the numeric feature value, and Meta () is a structured metadata object containing the explicit textual evidence that produced ,. The final feature vector for article is the concatenation of these numeric values: x = [1,, . . . , ,]⊤.
We propose a taxonomy of feature calculators designed to capture diferent signals of potential misinformation. This includes Lexical & Semantic Features, such as sentiment analysis using VADER [22], readability indices like the Flesch Reading Ease [20], and subjectivity detection. It also includes Stylistic & Structural Features, which capture how an article is written, such as analyzing punctuation for sensationalism or the ratio of quoted to unquoted text. Finally, Source-Based Features assess the credibility of mentioned sources, using Named Entity Recognition (NER) with libraries like spaCy v3.7.2 [23] to identify entities and cross-reference them against external knowledge bases of source credibility. For every feature, the corresponding ‘Meta‘ object stores the specific text spans or statistics that justify the feature’s value.
HGIVR is the core human-in-the-loop component of the framework. It is an iterative protocol, detailed in Algorithm 1, that allows a domain expert to use their intuition to guide the construction of a high-quality feature space.
Algorithm 1 The HGIVR Protocol 1: Input: Full feature set = {1, . . . , }, Corpus 2: Output: Validated feature subset f′inal 3: Initialize 0′ ← initial expert-selected subset of 4: Initialize ← 0 5: repeat 6: ← + 1 78:: PVreocjteocrtize′c−1orpinutso2Dussipnagcefeatu−1reussuibnsgetPCA′−1, MtoDgSe,todrat-taSNmEatrix ′−1 9: Compute quantitative separability metric −1 (e.g., silhouette score) on ′−1 10: Present interactive dashboard to expert: 11: - Display projection −1 , colored by class labels 12: - Display metric −1 and its trend 13: - Enable brushing, linking to raw documents, and feature-based coloring 14: Expert analyzes visualization and metrics, assessing class separation and cluster coherence 15: Expert provides feedback: ‘decision‘ ∈ {’accept’, ’refine’} 16: if ‘decision‘ is ’refine’ then 17: Expert specifies changes: Add/remove features to create new subset ′ 18: until ‘decision‘ is ’accept’ 19: f′inal ← ′−1 20: return f′inal
The mathematical underpinnings of the projection techniques are critical. PCA [13] provides a global overview by finding orthogonal linear combinations of features that capture maximum variance. MDS [14] preserves inter-point distances from the high-dimensional space in its low-dimensional projection, as measured by its Stress-1 objective function in Equation 1.
Stress1() = √︃ ∑︀<ℓ ((x, xℓ) − ‖z − z ℓ‖2)2 ∑︀<ℓ (x, xℓ)2 .
Meanwhile, t-SNE [15] is a non-linear technique efective at revealing local cluster structure by minimizing the KL divergence between high-dimensional and low-dimensional similarity distributions, shown in Equation 2.
KL( ‖) = ∑︁ ℓ log ℓ .
̸=ℓ ℓ (1) (2)
The silhouette score [24], defined in Equation 3, provides a quantitative anchor for the expert’s qualitative assessment.
() =
() − () max{(), ()} , (3) where () is the average intra-class distance and () is the minimal average inter-class distance. By tracking this score, the expert can objectively measure whether their refinements are improving the geometric separability of the classes.
With the expert-validated feature set f′inal, the framework proceeds to a more traditional machine learning stage. While model-agnostic, our implementation uses a Support Vector Machine (SVM) with a Radial Basis Function (RBF) kernel [21], a powerful classifier for non-linear decision boundaries. We compare it against other ensemble methods like Random Forests [25] and gradient boosting models like XGBoost [26] and LightGBM [27]. Hyperparameters are tuned using stratified k-fold cross-validation, and the entire pipeline is implemented with scikit-learn [28]. A crucial final step is model calibration. Since raw model outputs are not true probabilities, we train a calibrator (e.g., using Platt scaling) on a held-out set to ensure that a predicted confidence of 90% corresponds to an actual 90% likelihood of being correct. This step is critical for ensuring the trustworthiness of the confidence scores presented to the user [17].
Once a new article new is classified, the EAECS module is invoked to generate its explanation, as detailed in Algorithm 2. This process is designed to be strictly faithful to the model’s reasoning and the underlying evidence.
Algorithm 2 The EAECS Protocol 1: Input: Trained classifier ℳ, New article new, Feature set f′inal, LLM instance 2: Output: Natural-language explanation (new), Annotated article 3: Vectorize new using f′inalto get xnew and evidence records {Meta (new)} 4: Obtain prediction and calibrated probability: (^new, new) ← ℳ(x new) 5: Compute instance-level feature importance scores = {1, . . . , } for xnew (e.g., using
SHAP [
A crucial step is computing instance-level feature importance. We use SHAP (SHapley Additive
exPlanations) [
To evaluate the XFND framework, we use two widely recognized public benchmark datasets for fake news detection, allowing for direct comparison with existing work.
LIAR [18] is a dataset comprising 12,836 short statements from PolitiFact.com, each manually factchecked and assigned one of six fine-grained veracity labels: pants-fire, false, barely-true, half-true, mostly-true, and true. We follow the oficial data split of 10,269 training, 1,284 validation, and 1,283 test instances. We evaluate performance on both the original 6-way classification task and a binary version, where true and mostly-true are mapped to a real class, and the remaining four labels are mapped to a fake class.
FakeNewsNet [19] is a comprehensive data repository containing news content and social context from two fact-checking websites: PolitiFact and GossipCop. For our experiments, which focus on content-only analysis, we use the news articles from both sources. Following the experimental setup described in the original FakeNewsNet paper, we use a deterministic 80%/20% split for training and testing, respectively. This allows for a fair comparison against the content-only baselines reported in prior work.
For our primary classification model, we use a Support Vector Machine (SVM) with a Radial Basis Function (RBF) kernel [21]. Hyperparameters (C and gamma) are optimized on the training sets using a 7-fold stratified cross-validation grid search, with the objective of maximizing the macro-averaged 1-Score. All data is preprocessed using scikit-learn’s ‘StandardScaler‘ [28]. We report a comprehensive suite of classification metrics on the held-out test sets: Accuracy, Precision, Recall, 1-Score, and Area Under the ROC Curve (ROC AUC). For model trustworthiness, we also report the Expected Calibration Error (ECE) [17] and Brier score. The primary metric for evaluating the HGIVR process is the silhouette score [24]. We also compare our SVM against other powerful ensemble models like XGBoost [26] and LightGBM [27].
We apply XFND to two public corpora—LIAR [18] and FakeNewsNet [19]—and evaluate three desiderata: (i) feature-space quality before training (HGIVR), (ii) predictive performance with calibrated probabilities, and (iii) faithfulness and auditability of explanations (EAECS). Unless noted, we use content-only features from our evidence-centric calculators and an RBF-SVM with Platt scaling. We report Accuracy (Acc), macro-averaged 1 (1), AUC, AP, ECE, and Brier score.
Across all datasets HGIVR raises the silhouette score on training splits (Figure 1): LIAR 0.19 → 0.31 (+63%), PolitiFact 0.24 → 0.38 (+58%), GossipCop 0.21 → 0.34 (+62%). Appending deliberately noisy calculators (all) lowers . UMAP projections (Figure 2) show the same pattern: LIAR’s refined projection reveals crisper six-cluster structure; PolitiFact/GossipCop (binary) show better separated classes with diferent geometry across domains, consistent with their metrics.
We evaluate faithfulness via a deletion test: removing =3 evidence spans selected by EAECS reduces model confidence more than removing random spans of equal length (Table 3). We also report an evidence overlap rate, i.e., the fraction of explanations that explicitly cite entities or numbers present in the article. Figure 5 visualizes both efects.
Cross-domain transfer (train on PolitiFact, test on GossipCop; and vice versa) remains challenging (Table 4). Nevertheless, HGIVR improves robustness: removing HGIVR lowers average 1 by 3.4 points
2 1 2
3 0 1
UMAP-1 (a) LIAR (initial feature set) PolitiFact UMAP (refined) 3 2 1 -2 1 P A M U 0 1.5 1.0 0.5 2 -P 0.0 A UM0.5 1.0 1.5 2.0
1
UMAP-1 1 0 2
3 (b) LIAR (refined feature set)
GossipCop UMAP (refined) 3 luC s t e r 2 id 1.0 0.8 0.6 luC s t e r 0.4 id 0.2 0.0 4 1 0 3 luC s t e r 2 id 1.0 0.8 0.6 luC s t e r 0.4 id 0.2 0.0 -2 1 P A M U 0 3 2 1 2 3 2 1 -2 1 P A M U 0 1 0 2
3 1
UMAP-1 (c) PolitiFact (refined feature set) 1 0 3
4 1 2
UMAP-1 (d) GossipCop (refined feature set) and worsens ECE by 4.1 points across transfers. Table 5 quantifies the contribution of HGIVR and calibration on in-domain splits; Platt scaling reduces ECE by 55–60% without hurting AUC.
In summary, XFND (i) improves feature geometry prior to training (HGIVR), (ii) delivers competitive accuracy with well-calibrated probabilities on LIAR and FakeNewsNet using content-only inputs, and (iii) provides faithful, auditable explanations tied to verifiable text spans.
The empirical results presented in Section 4 provide strong evidence for the eficacy of the XFND framework, validating our core hypothesis that embedding interpretability as a design principle can resolve the perceived trade-of between accuracy and transparency. Our work engages with key challenges in operationalizing trustworthy AI at the intersection of machine learning, human-computer interaction, and critical domains like journalism.
The primary finding is that a deliberate, architected approach to integrating human-in-the-loop validation yields tangible benefits. The significant improvement in silhouette scores across all datasets (e.g., a 63% relative increase for LIAR) is powerful evidence that the HGIVR process leads to a feature 1.0 0.8 y c a r cu0.6 c a l a irc0.4 i p m E0.2 0.0
LIAR probability calibration
LIAR confusion matrix (6-way) Ideal XFND (calibrated) space that is not only more geometrically separable but also more semantically coherent. This reframes the human expert from a passive consumer of post-hoc explanations into an active collaborator in the model-building process. This pre-training refinement directly contributed to the strong downstream XFND (AUC = 0.812) Chance Youden optimum
XFND (AP = 0.820) Class prior Max F1 0.0 0.2
0.4 0.6 False positive rate 0.8 1.0 0.0 0.2 classification performance, where our content-only model was competitive with or even outperformed fusion-based models like SAF [19].
Furthermore, the EAECS module establishes a higher standard for what constitutes a “good”
explanation. By moving beyond abstract feature attributions to provide concrete, verifiable textual evidence, the
framework produces outputs aligned with the epistemological standards of fact-checking. A system that
can “show its work” by highlighting the specific phrases or statistics that influenced its decision is far
more likely to be trusted and efectively utilized by professionals than one that ofers an unsubstantiated
verdict [
Our framework builds upon, but is distinct from, prior work. While we leverage standard
posthoc techniques like SHAP [
A key limitation of our current study is its focus on content-only features. While we demonstrate
strong performance, real-world misinformation often involves social context, such as user engagement
patterns and propagation networks, which our model does not consider. The cross-domain transfer
results (Table 4), while showing some robustness, indicate that domain-specific linguistic patterns
remain a challenge. This leads to several research challenges and open questions. How can the
HGIVR process be scaled to handle massive datasets and thousands of features, perhaps by using active
learning to intelligently query the expert? How can we formally evaluate the real-world utility of
EAECS explanations for journalists and fact-checkers through rigorous human-subject studies? Finally,
exploring the integration of social and temporal features within our evidence-centric paradigm is a
critical next step, especially as generative AI continues to accelerate the production of sophisticated
fake news [
Overall, by shifting from untrustworthy predictions to transparent, evidence-based reasoning, the XFND paradigm ofers a robust blueprint for creating trustworthy AI systems that function not as infallible oracles, but as collaborative partners in the critical fight to safeguard our shared information ecosystem.
In this work, we addressed the critical trust deficit in AI-powered fake news detection by introducing the Explainable Fake News Detection (XFND) framework, a novel approach designed to embed transparency and human oversight as core architectural principles. We argued that interpretability should not be a retrospective feature applied to an opaque model but a foundational constraint guiding the entire system design. This philosophy was realized through the synergy of two key innovations: HumanGuided Interactive Validation and Refinement (HGIVR), a protocol empowering domain experts to collaboratively shape a semantically meaningful feature space before training, and Evidence-Anchored Explanation Synthesis (EAECS), a mechanism generating faithful-by-design narratives grounded in verifiable textual evidence. Our rigorous evaluation on public benchmarks provided a strong proof-ofconcept for this paradigm. The HGIVR process delivered a tangible and quantifiable improvement in feature space quality, boosting class separability as measured by the silhouette score by up to 63% on the LIAR dataset. This better representation enabled a downstream classifier to achieve highly competitive predictive performance, reaching an 1-Score of 0.792 on binary LIAR and 0.731 on PolitiFact, while ensuring that probabilistic outputs were well-calibrated and trustworthy. We further demonstrated through quantitative tests that the explanations produced by EAECS are faithful to the model’s reasoning and auditable by design. While these results are promising, we recognize the limitations inherent in a content-only analysis and acknowledge that the HGIVR process’s reliance on expert availability presents a scalability challenge. The path forward is therefore clear: our immediate priority is to apply and scale the framework to incorporate multi-modal and social context features, necessitating more advanced feature engineering and scalable interactive visualizations.
Future work will also focus on extensive human-subject studies with journalists and fact-checkers to measure the real-world impact of our evidence-anchored explanations on their decision-making speed and accuracy.
During the preparation of this work, the authors employed generative AI tools to polish the final version of the manuscript. Specifically, Gemini 2.5 Pro (owned by Google LLC) and Grammarly (owned by Grammarly, Inc.) were utilized to improve grammar, spelling, and overall readability. After using these tools, the authors reviewed and edited the content as needed and take full responsibility for the publication’s content. [13] I. Jollife, Principal component analysis, 2nd ed., Springer, New York, NY, USA, 2002. doi: 10.1007/ b98835. [14] J. B. Kruskal, Nonmetric multidimensional scaling: A numerical method, Psychometrika 29 (1964) 115–129. doi:10.1007/BF02289694. [15] L. van der Maaten, G. Hinton, Visualizing data using t-SNE, Journal of Machine Learning Research 9 (2008) 2579–2605. URL: https://www.jmlr.org/papers/v9/vandermaaten08a.html. [16] V. Arya, R. K. E. Bellamy, P.-Y. Chen, A. Dhurandhar, M. Hind, S. C. Hofman, S. Houde, Q. V.
Liao, R. Luss, A. Mojsilović, S. Mourad, P. Pedemonte, R. Raghavendra, J. T. Richards, P. Sattigeri, K. R. Varshney, D. Wei, Y. Zhang, AI Explainability 360: An extensible toolkit for understanding data and machine learning models, Journal of Machine Learning Research 21 (2020) 1–6. URL: https://jmlr.org/papers/v21/19-1035.html. [17] C. Guo, G. Pleiss, Y. Sun, K. Q. Weinberger, On calibration of modern neural networks, in: Proceedings of the 34th International Conference on Machine Learning (ICML 2017), volume 70 of Proceedings of Machine Learning Research, Sydney, Australia, 2017, pp. 1321–1330. URL: https: //dl.acm.org/doi/abs/10.5555/3305381.3305518. [18] W. Y. Wang, “Liar, Liar Pants on Fire”: A new benchmark dataset for fake news detection, in: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL 2017), Short Papers, Association for Computational Linguistics, Vancouver, Canada, 2017, pp. 422–426. doi:10.18653/v1/P17-2067. [19] K. Shu, D. Mahudeswaran, S. Wang, D. Lee, H. Liu, FakeNewsNet: A data repository with news content, social context, and spatiotemporal information, Big Data 8 (2020) 171–188. doi:10.1089/ big.2020.0062. [20] R. Flesch, A new readability yardstick, Journal of Applied Psychology 32 (1948) 221–233. doi:10.
1037/h0057532. [21] C. Cortes, V. Vapnik, Support-vector networks, Machine Learning 20 (1995) 273–297. doi:10.
1007/BF00994018. [22] C. Hutto, E. Gilbert, Vader: A parsimonious rule-based model for sentiment analysis of social media text, Proceedings of the 8th International AAAI Conference on Weblogs and Social Media (ICWSM 2014) 8 (2014) 216–225. doi:10.1609/icwsm.v8i1.14550. [23] M. Honnibal, I. Montani, spaCy: Industrial-strength natural language processing in Python, Zenodo, 2017. doi:10.5281/zenodo.1212303. [24] P. J. Rousseeuw, Silhouettes: A graphical aid to the interpretation and validation of cluster analysis, Journal of Computational and Applied Mathematics 20 (1987) 53–65. doi:10.1016/ 0377-0427(87)90125-7. [25] L. Breiman, Random forests, Machine Learning 45 (2001) 5–32. doi:10.1023/A:1010933404324. [26] T. Chen, C. Guestrin, XGBoost: A scalable tree boosting system, in: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, New York, NY, USA, 2016, pp. 785–794. doi:10.1145/2939672.2939785. [27] G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye, T.-Y. Liu, LightGBM: A highly eficient gradient boosting decision tree, in: Proceedings of the 31st International Conference on Neural Information Processing Systems, Curran Associates Inc., Red Hook, NY, USA, 2017, pp. 3146–3154.
URL: https://dl.acm.org/doi/10.5555/3294996.3295074. [28] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, É. Duchesnay, Scikit-learn: Machine learning in Python, Journal of Machine Learning Research 12 (2011) 2825– 2830. URL: https://dl.acm.org/doi/10.5555/1953048.2078195.