We investigate how large language models (LLMs), when paired with personalized knowledge graphs (PKGs), can power decentralized recommendation systems. This work lays the groundwork for more intelligent and user-aligned personal digital assistants that respect user autonomy and data sovereignty. A key focus of our research is exploring how LLMs can be fine-tuned in federated settings to balance personalization with privacy. To this end, we evaluate several fine-tuning methods and compare their performance to select the best one. Early results indicate that LLMs fine-tuned to use PKGs can outperform symbolic and embedding-based KGC models (e.g., KBGAT, HAKE) in both centralized and federated contexts, and that fine-tuning with Kahneman-Tversky Optimization (KTO) is more resilient to lopsided data distributions.
CEUR
ceur-ws.org
Ford’s work, the system recommends another of his movies, like ”What Lies Beneath (2000)”. If the user accepts this suggestion, their PKG is updated to reflect this new preference. Later, when the user’s device participates in a round of federated learning, the model is fine-tuned locally on this new interaction. The resulting model updates are sent to a central server for aggregation. Because the user’s specific preferences never leave their device, their privacy is preserved, while the aggregated global model still learns general patterns, such as the connection between diferent Harrison Ford movies, benefiting all users.
Because PKGs are interpretable and adaptable, users can both understand and influence how their data informs recommendations. The system can be flexibly tuned to specific recommendation contexts. They can also be designed to deliberately disrupt PIEs by adapting preference patterns within the PKG.
This problem has broad societal relevance, impacting virtually all users of modern digital platforms. For individuals, it concerns the fundamental right to privacy, transparency, and control over personal data. Furthermore, exploring a variety of content enriches a user’s experience. For organizations, especially smaller companies and startups, it presents a barrier to entry due to the infrastructure requirements and legal risks of handling sensitive user data.
Who will benefit and who should care? • Users gain enhanced privacy, transparency, and agency over how they are profiled and marketed to. • Companies and organizations benefit from reduced infrastructure costs and data liability, making advanced recommendation systems more accessible to smaller players. • Researchers interested in privacy, personalization, fairness, and human-computer interaction gain a novel paradigm for user modeling and recommendation.
Solving this problem could democratize access to recommendation technology while restoring trust
in digital systems. It fosters user empowerment through interpretable data structures and safeguards
personal data by eliminating the need for centralized storage. From a societal perspective, it combats
the dangers of algorithmic echo chambers by enabling users to actively shape their digital experiences.
From a research perspective, it introduces a hybrid model that combines language-based reasoning
with structured, user-owned knowledge, contributing to the evolution of federated and interpretable
AI. This approach aligns with recent perspectives that unifying KGs and LLMs can combine their
respective strengths: the factual knowledge and interpretability from KGs with the advanced reasoning
and language abilities of LLMs [
Prior work has explored key components of our proposed system—namely, knowledge graph completion (KGC), FL, and recommendation using LLMs, but no existing approach integrates these elements into a unified, interpretable, privacy-preserving, and scalable recommendation framework.
In the space of centralized KGC, models such as KBGAT [
To test the potential of fine-tuning LLMs with FL, Ye et al. [
LLMs have also been tested for their ability to perform KGC tasks. For example, Meyer et al. [
Eforts to mitigate PIEs include the Dual Echo Chamber framework [
Despite their strengths, existing KGC models like KBGAT [
While OpenFedLLM [
While Fede [
Regarding LLM-based solutions, existing work does not integrate federated training or PKGs. Most rely on static prompting rather than fine-tuning, and do not explore dynamic, user-guided recommendations or PIE mitigation strategies.
Eforts to explore outside PIEs also sufer limitations. The Dual Echo Chamber framework [
• Federated, PKG-based Recommendation: We integrate federated learning with personalized
KGs, allowing users to retain control over their data while still benefiting from high-quality,
LLM-driven recommendations. Additionally, we employ diferential privacy to ensure the user’s
data remains private [14].
• LLM Fine-Tuning with KTO: We leverage KTO to fine-tune the LLM for knowledge graph
tasks, enabling more efective and context-aware recommendations [
This combination of techniques results in a system that is privacy-preserving, interpretable, scalable, and user-controllable, qualities that are not achieved in existing work.
These prior eforts underscore the growing interest in privacy, personalization, and interpretability in recommendation systems. They highlight both the promise and limitations of existing technologies. Specifically, they show: (i) The potential of LLMs in semantic reasoning tasks involving KGs. (ii) The practical challenges of deploying centralized models at scale. (iii) The importance of user-guided and interpretable systems in exploring new content. Together, these works provide valuable benchmarks and conceptual foundations upon which we can build a more robust and user-centered solution.
We aim to investigate how LLMs can be integrated into a privacy-preserving, decentralized recommendation system grounded in user-controlled knowledge representations. Specifically, our work explores the following research questions: RQ1: How can PKGs be used to fine-tune an LLM to improve recommendation accuracy and personalization compared to standard knowledge-aware recommenders? (Core technical question focusing on PKG+LLM eficacy) RQ2: Which fine-tuning strategies are most efective for LLMs in a federated learning setting, particularly in the presence of heterogeneous data and privacy constraints? RQ3: How does knowledge graph completion via LLM, with PKG data, impact recommendation quality, such as coverage of long-tail items and novelty?
These questions leverage Semantic Web technologies to structure and store PKGs in interoperable, machine-readable formats. By grounding user data in open standards (e.g., JSON-LD), stored within Solid pods, we aim to ensure that users retain full control over their data while still enabling high-performance model inference on the client side.
H1: PKGs will enable an LLM to generate more accurate recommendations than an LLM without KG ifne-tuning, due to the injection of structured personal knowledge.
H2: LLMs fine-tuned with KTO are more performant and resilient to lopsided data in a federated
setting, even with diferential privacy applied, than those fine-tuned via other human feedback-based
ifne-tuning methods, such as Direct Preference Optimization (DPO) [
H3: If a user’s PKG is adjusted, an LLM fine-tuned for KGC can be led to make recommendations outside the PIE without retraining more efectively than if the LLM was simply asked to avoid the PIE with the unadjusted PKG.
Figure 1 illustrates our framework design. Figure 1a shows how clients perform local training on personalized KGs and communicate model updates to a central server for aggregation. Before sending the updates, diferential privacy is applied to ensure the user’s data cannot be reverse engineered from the updates. Figure 1b shows how, at inference time, the LLM receives a user’s PKG and natural language query, and generates KG completions. However, if the user indicates they want to avoid the PIE in the request, the PKG is first adjusted to represent a user with PIE-avoiding preferences.
Table 1 contains our previous work [16] where we showed that an LLM fine-tuned, via FL, with KTO outperformed DPO, even when the distribution of the data was randomized. We fine-tuned alpha7B [17] with chatbot arena data [18] in a federated setting using both DPO and KTO, with and without randomizing the data (KTOR and KTOO respectively). Our findings demonstrate that KTO consistently outperforms DPO across all aggregation methods and evaluation benchmarks. Notably, KTO achieves higher scores even when the training data is randomized (KTOR), highlighting its robustness to data distribution variance. The evaluated benchmarks include: Legend ?i = Weight updates at client i ?Agg = Aggregated weight updates DPi = Differential privacy applied at client i
LLM
PKG Derive KG Completion
Response ?N
DPN
Fine-Tuning Client N ?1
DP1
PKG User Request
Federated Aggregation Server
?Agg LLM PKG ?2
DP2 Yes Avoid PIE? No
Knowledge Graph Update
Rule Adaptation Adapted PKG
PKG LLM
PKG • MT-Bench-1: Measures one-turn conversational performance [18]. • Vicuna: Assesses instruction-following capabilities [19]. • AdvBench: Evaluates model safety and adversarial robustness [20].
• MMLU: Measures factual knowledge and reasoning ability [21].
The MT-Bench-1, Vicuna, and AdvBench results have been published in our prior work [16], while the MMLU results are newer and not yet published.
Table 2 reports performance on our custom Movie KG dataset, derived from the Recommendation Dialogues dataset [22, 23]. We took dialogues between users recommending movies to build PKGs that encode each user’s preferences. In the table, we refer to our model as PKGLLM (centralized) and FedPKGLLM (federated) with (noSyn) indicating that no synthetic data was used in fine-tuning.
We evaluated LLMs fine-tuned with KTO against strong symbolic and embedding-based KGC baselines: KBGAT and HAKE. Results show that our model, PKGLLM, which is a KTO-tuned Qwen3-0.6B model [24], achieves significantly better performance in both centralized and federated configurations, across all metrics: Precision, Recall, Hits@K, and MRR. This suggests that LLMs trained with KTO can outperform specialized KGC models on link prediction tasks grounded in personalized, dialogue-derived KGs. These results strongly support H1 and H2, indicating that LLMs fine-tuned via KTO can serve as efective and robust KGC engines, particularly in decentralized, privacy-sensitive settings.
Our evaluation strategy centers on systematically testing each hypothesis and research question through comparative experiments and controlled ablation studies. For RQ1 and RQ2, we rely on benchmark datasets and model performance metrics as outlined in the preliminary results. These results already support H1 and part of H2, showing that LLMs fine-tuned via KTO outperform DPO-based models in a federated setting across a variety of benchmarks, and an LLM fine-tuned to make recommendations using PKGs outperforms KBGAT and HAKE. For further testing, we will continue to compare against baseline models.
To evaluate the other half of H2 (regarding the efect of diferential privacy), we will fine-tune Alpaca7B
using both KTO and DPO under varying levels of privacy budgets. We select Alpaca7B to maintain
comparability with the OpenFedLLM [
To assess H3 and investigate the impact of KG adaptation on recommendations that venture outside a user’s PIE, we will conduct experiments using three strategies:
2. Recommendations generated by prompting the model to avoid the PIE without PKG adaptation. 3. Recommendations generated after adapting the PKG to avoid the PIE.
We will compare the precision, relevance, and diversity of recommendations with each strategy to determine the eficacy of PKG adaptation as a strategy for PIE avoidance.
For hypotheses related to diferential privacy ( H2), we will use the same benchmarks as in the preliminary results: MT-Bench-1, Vicuna, AdvBench, and MMLU. These metrics provide comprehensive coverage of conversational quality, instruction following, safety, and factual knowledge.
For evaluating PIE avoidance (H3), we will categorize recommendations into three buckets: • Invalid Recommendation: The recommendation fails to satisfy the user’s explicit query constraints • In-PIE Recommendation: The recommendation is valid but reinforces the user’s existing PIE, ofering little novelty. • Out-PIE Recommendation: The recommendation is valid and successfully steers the user away from their established PIE.
Consider two illustrative cases: (1) a user seeking a cute animal video but aiming to avoid cat videos, having been overexposed to them; and (2) a user searching for Italian food recipes while wanting to avoid tomatoes due to over-personalized exposure. Table 3 presents representative examples of each recommendation type for these PIE contexts.
After running query tests, the number of recommendations in each buckets for diferent strategies will allow us to quantify the tradeof between respecting user’s stated preferences and introducing novel, out-of-distribution content in recommendations. This would allow us to assess each strategy’s efectiveness in balancing personalization and recommendation diversity.
A key limitation of our approach lies in the scalability of large language models in federated learning environments. While our experiments show that relatively small models like Qwen3-0.6B can still yield strong performance, larger models could achieve even better results in both reasoning and generalization. However, using larger models significantly narrows the pool of eligible client devices in FL, as they demand greater computational resources and memory. This presents a tradeof between model capacity and practical deployability across a diverse, decentralized user base.
Our immediate next steps are focused on two fronts. First, we will implement and rigorously evaluate the KG adaptation strategy to test its efectiveness in enabling LLMs to recommend outside a user’s PIE without compromising relevance. Second, we will conduct a thorough diferential privacy analysis, comparing the performance resilience of KTO and DPO across varying privacy budgets. These experiments will help validate H2 and H3 and further assess the broader potential of our KGC model to operate efectively in decentralized, privacy-preserving, and user-adaptive settings.
During the preparation of this work, the authors used ChatGPT, Gemini and Grammarly in order to rephrase some of the sentences and also to fix grammar and spelling issues. After using these tools and services, the authors reviewed and edited the content as needed and take full responsibility for the publication’s content. [14] C. Dwork, F. McSherry, K. Nissim, A. Smith, Calibrating noise to sensitivity in private data analysis, in: Theory of Cryptography: Third Theory of Cryptography Conference, TCC 2006, New York, NY, USA, March 4-7, 2006. Proceedings 3, Springer, 2006, pp. 265–284. [15] A. Saeidi, S. Verma, M. N. Uddin, C. Baral, Insights into alignment: Evaluating dpo and its variants across multiple tasks, arXiv preprint arXiv:2404.14723 (2024). [16] F. Spadea, O. Seneviratne, Federated fine-tuning of large language models: Kahneman-Tversky vs. direct preference optimization, arXiv preprint arXiv:2502.14187 (2025). [17] R. Taori, I. Gulrajani, T. Zhang, Y. Dubois, X. Li, C. Guestrin, P. Liang, T. B. Hashimoto, Stanford alpaca: An instructionfollowing LLaMA model, https://github.com/tatsu-lab/stanford_alpaca, 2023. [18] L. Zheng, W.-L. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. Xing, et al., Judging llm-as-a-judge with mt-bench and chatbot arena, Advances in Neural Information Processing Systems 36 (2023) 46595–46623. [19] W.-L. Chiang, Z. Li, Z. Lin, Y. Sheng, Z. Wu, H. Zhang, L. Zheng, S. Zhuang, Y. Zhuang, J. E. Gonzalez, et al., Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, See https://vicuna. lmsys. org (accessed 14 April 2023) 2 (2023) 6. [20] A. Zou, Z. Wang, N. Carlini, M. Nasr, J. Z. Kolter, M. Fredrikson, Universal and transferable adversarial attacks on aligned language models, 2023. URL: https://arxiv.org/abs/2307.15043. arXiv:2307.15043. [21] D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, J. Steinhardt, Measuring massive multitask language understanding, 2020. [22] R. Li, S. Ebrahimi Kahou, H. Schulz, V. Michalski, L. Charlin, C. Pal, Towards deep conversational recommendations,
Advances in neural information processing systems 31 (2018). [23] Hugging Face, ReDial dataset, https://huggingface.co/datasets/community-datasets/re_dial, 2024. [24] Hugging Face, Qwen3-0.6b, https://huggingface.co/Qwen/Qwen3-0.6B, 2024.