Transparent Recommender Systems, especially those based on machine learning, could convey the reasoning behind their predictions of great importance. In the literature, many Recommender Systems based on Machine Learning have proven their strength. However, focusing on improving accuracy in these systems means more complex algorithms (black-box models), making the reasons behind the prediction difficult to understand. Explainability and efficiency should not be at odds with each other. The few existing neural networks-based recommendation algorithms suffer from data sparsity problems caused by the lack of rated items, or user social information is not exploited. Recently, Knowledge Graphs have been explored for that purpose. However, still, manual efforts are required for feature extraction. Graph Embeddingbased algorithms are promising solutions that use the Graph Theory properties to learn user and item feature vectors for generating the recommendation. The goal of this paper is to outline the research questions set out in my Ph.D. thesis related to the transparency and trust in a recommendation by providing an intuitive explanation and to put forward the preliminary ideas to tackle the challenges.
Recommender Systems (RSs) change the way we interact with many services. Instead of providing static data, they bring interactive experience and an option to leave your feedback. The purpose of RSs is to provide users with personalized, sorted, aggregated, and highlighted information based on user behavior, ratings, preferences, as well as other relevant features generated by Social Networks (SNs) depending on the type of system. Social trust relationships deriving from SNs are significant to qualify this information. Hence, merging RSs and SNs with explanations could enhance the accuracy of recommendations and improve the overall user experience. ? Copyright c 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
Nowadays, there is a vast amount of web content created by users, utilized to provide recommendations. At the same time, SNs have become popular with over a billion accounts shared across hundreds of networks. Often a user cannot choose among the variety of the given options by checking each of them (e.g. movies in an online cinema, consumer goods in an online store, or any other content). In case there is a lot of content in service, a user faces the problem of informational overload. Moreover, traditional RS algorithms such as Collaborative Filtering (CF) or Matrix Factorization (MF) so far are not able to provide efficient, satisfactory recommendations due to the various parameters and the complex relationships between them.
In response to the challenges associated with processing large amounts of diverse data, deep learning (DL) techniques, extremely popular in recent years, are used as a way to solve this problem in RSs. Thanks to nonlinear functions such as Rectified Linear Units (ReLu) [10] or Logistic Sigmoid [10], algorithms can learn the complex and complicated patterns of interaction between users and items. Besides, DL enables model learning based on more heterogeneous information in the form of text, graphs, images, or audio signals. It allows providing additional data on user behaviors and their preferences, and as a result, models can become enhanced.
Even though the extensive use of deep learning algorithms in RSs has led these recent years to the increase of their results quality by generating more accurate recommendations, they are becoming progressively less easy to understand for humans. One of the most challenging aspects are to understand why appropriate items are recommended to specific users and how the filtered information reaches the right individual. Finding an answer can help the improvement of transparency, effectiveness, persuasiveness, and trustworthiness.
In this paper, I present my research directions on a Transparent RS as a part of my 1st year Ph.D. studies. Social Network Analysis (SNA), and Graph Theory (GT), including Knowledge Graphs (KGs) seem to be relevant research directions into the Explainable Artificial Intelligence (XAI) due to shared underlying graph technologies. However, understanding mathematical calculations and decisions in a black box neural network is another challenge. Therefore, the use of logic and reasoning (Symbolic AI) issues also seems promising. Moreover, hash function algorithms are promising research areas to implement novel RS method due to the possibility of discovering patterns in graphs. The assumptions that combinations of these areas (SNs, KGs, Symbolic AI) could enhance the explanations and be helpful to reach the level of self-explanatory Artificial Intelligence. These fields share many key features such as topology build on graph structures, relations, and ontologies. Until recently, however, these research areas have progressed independently. Their combination could help to explain the operations that occur in deep networks that also are based on graph structure. I believe that graph elements consisting of nodes (objects) and edges (relations) could present a calculation path passing through successive neural network layers and decisions made at individual stages.
The rest of the paper is organized as follows: First, I give an overview of the problem of search in. Section 3 describes the first ideas and proposed contributions. Finally, Section 4 presents my conclusions and future directions. 2
So far the main work on the RS enhancement focused on the explainable recommendations. This section presents the recent research in the Explainable Recommender System (XRS) field. 2.1
In past works, explanations are generated from different sources and presented in different styles. They are mainly based on relevant nearest-neighbor users or items (close to CF techniques) [9], and item features that were similar to the target interest of the user (close to the content-based methods) [15]. Researchers realized some more advanced approaches based on textual information (generally produced using Natural Language Processing (NLP) methods) [8], visual explanation based on images [7], and social explanation based on the target user social relations [18].
In my work, I am more interested in social explanations for the benefits they provide thanks to the quantity of information generated by Online Social Networks (OSNs), and their graphical representation. 2.2
The explaining recommendation approaches depend on the type of RSs and algorithms. Nowadays, there are many different research directions on XRSs. Several popular model-based explainable approaches use traditional MF, graphs, and deep learning paradigms. Matrix and Tensor factorization approaches [5, 21] rely on the latent factor learning. However, the factors are hidden, and the meaning of each factor is unknown, which makes explaining the recommendations challenging. In many cases data can be represented as graphs, including social networks. The explanations in this case can be generated by the neighbors who have similar preferences [18]. In recent years, as deep learning techniques have become popular thanks to their effectiveness, their use in RSs can only be beneficial. Despite the challenge of the neural network explanation, many approaches have covered a wide range of deep learning techniques such as Convolutional Neural Networks [19], Recurrent Neural Networks [11], and Attention Mechanism [6]. 2.3
Many recent works have focused on XRSs using a variety of methods. Some of them put emphasis on the result explanations while others have rather addressed the algorithms’ explanations. The first findings from the literature review are that there is a great deal of confusion between explanation and interpretation. In addition, as you can notice, most of the explainable recommendation research are based on unstructured data, such as texts or images. However, recent works has explored the use of KG, which is a more structured information. In this context, two types of works are distinguished: meta-path-based methods [23] and embedding learning-based algorithms [20]. If the former requires manual efforts, the latter, which use the KG structure to learn users and feature vectors, seems promising. Nevertheless, the results are still not explained satisfactorily.
Research directions on explanations focus on deep architectures in RSs. Although the use of deep models brings better results than traditional methods, research into new methods for RSs should be continued, using innovative approaches.
As a result, in this paper, the following issues will be investigated: (i) Clarifying the main differences between the terms Explanation and Interpretation; (ii) First ideas toward a Knowledge graph-based Explainable Recommender System; (iii) First ideas toward a novel RS algorithm. 3
Thus far, my work has led to a multitude of ideas and projects that are in progress. Research directions that highlight my goals within 1st-year Ph.D. work described in the following subsections. 3.1
The research problem of XRSs began with understanding what is an explanation.
It turns out that this is not an easy task, and this term is used interchangeably
with interpretation. My first research work focused on pointing out the
difference between these terms. Recent approaches were presented in surveys showing
some research directions on XAI [
The figure shows the main idea of obtaining explanations. The first module - learning process explanations (see Fig. 1a) could be based on KGs described in the following subsection (see Section 3.2). The assumption is that it would still be understandable only to a group of engineers or experts who could assess the transparency of the system based on the obtain graph explanation. The interpretation on the other hand - second module (see Fig. 1b) could be based on the first part and recommendations understandable by the end-user, showing what you can do with this information.
I believe that the first step toward an explainable system should focus on explaining the algorithm itself. Indeed, understanding the learning model could ensure that the system would be fully transparent. On the other hand, the output interpretation will not make the system transparent without an explainable system core process. I make the assumption that explaining the learning algorithm could enhance the output interpretation, including recommendations . 3.2
Many significant real-world datasets are represented as graphs or networks: SNs or KGs. Until recently, however, very little attention was paid to the generalization of neural network models to such ordered datasets. Today it is one of the main directions of research into recommender systems. Plentiful studies used deep learning algorithms with promising results in Graph Neural Networks (GNNs) technology, including Graph Convolutional Networks (GCNs). To date, several interesting RSs, described in [12, 13], have been developed using graphs and deep networks. However, current methods still do not solve the problem of transparency. They usually use more complex methods consisting of combined deep learning algorithms, hard to explain. It seems that the research direction should seek a compromise between the effectiveness and transparency in RSs.
In response to these challenges, the first concepts of using KGs appeared as a
technology to help explaining DL algorithms in RSs. It seems to be another
contribution towards generalizing neural network models into such ordered datasets.
To date, the first interesting papers showed the idea of using KGs as a helpful
technology in explaining the black box in deep architectures [
Within my contribution, the main idea is to represent each neuron calculation on a KG representation. To date, the mapping between a neural network and a KG representation seems to be promising research direction due to the graph topology that exists in both representations, although not eliminating some the major limitations related to the scalability problem. My prior research work on embedding neural network calculations on graphs showed a problem with the result interpretation.
The main challenge of the research problem is twofold: to find a way to tag the vector representation on a graph so that it would be understandable to humans and to discover appropriate relations connecting the vectors. 3.3
Research works on XRSs mainly focus on the challenges of the deep network.
I believe that alternative research directions should be sought. As one of the
alternatives, I designed a novel approach for RSs based on graph structures.
Patterns discovery in graphs could be an alternative approach to GCNs by hash
function implementation while allowing the interpretation of recommendations.
GCNs tried to discover nodes and edges on graph networks using convolution
procedures. According to my idea, achieving the same goal could be possible
using a hash function, making the system more transparent with less complex
calculations. Recent papers described recognizing frequent graph patterns in
RDF (Resource Description Framework) [
Figure 2 shows the evolution of the algorithm on the example user 1 (node u1) and user 2 (node u2). The graphs illustrate a user (orange node) with items (yellow nodes). Predicates take weighted parameters a, b, c, d. Each edge (relation user and item) has a weight determining the item rating by the user. Simply put, a scale assumes 0 to 5, where the highest rate equals the highest user satisfaction. Graph similarity narrows to node (user) similarity using vector space. At the reception of the first graph (Graph 1 ), the algorithm checks one by one the presence of its predicates a,b,c,d, generating Predicates Hash-table. Giving 1011 as a bit-vector value, where 0 means no edge (relations), then is used as a pattern searching similar graphs. The final bit vector is cleaned of indexes with a value 0. After recognizing graph similarity, node i3 from the second graph (Graph 2 ) is a new recommendation to the user in the first graph. The rating prediction for a new item could be calculated by using similarity measures or averaging ratings. 4
As I delve into my research work, I expect my understanding to grow and meld into a unified knowledge about a transparent RS with enhanced recommendations and interpretations.
The facets discussed here will link underlying realms of my research (i.e., GT and Knowledge Reasoning), but it is still in progress. Through continued research within symbolic AI, XAI, and GT, I hope to establish a unique solution that remains broad and yet powerful.
Acknowledgments Prior research was supported in part by Efrei Paris. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the Efrei Paris. I would also like to thank Amel Bouzeghoub, Katarzyna Węgrzyn-Wolska, Yessin Neggaz and Hanen Ochi for their contributions to this line of work. 5. Chen X., Qin Z., Zhang Y. & Xu T.: Learning to rank features for recommendation over multiple categories. In: Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp.305–314.
ACM, Pisa, Italy (2016). 6. Chen C., Zhang M., Liu Y. & Ma S.: Neural Attentional Rating Regression with Review-level Explanations. In: Proceedings of the 2018 World Wide Web Conference (WWW ’18). International World Wide Web Conferences Steering Committee, pp.1583–1592. WWW, Republic and Canton of Geneva, Switzerland (2018). 7. Chen X., Chen H., Xu H., Zhang Y., Cao Y., Qin Z. & Zha H.: Personalized Fashion Recommendation with Visual Explanations based on Multimodal Attention Network: Towards Visually Explainable Recommendation. In: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 765–774. ACM, New York, NY, USA (2019). 8. Chen Z., Wang X., Xie X., Wu T., Bu G., Wang Y. & Chen E.: Co-attentive Multitask Learning for Explainable Recommendation. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence, pp. 2137–2143. AAAI Press (2019). 9. Cleger-Tamayo S., Fernandez-Luna J. M. & Huete J. F.: Explaining Neighborhoodbased Recommendations. In: Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1063–1064.
ACM, New York, NY, USA (2012). 10. Di W., Bhardwaj A. & Wei J.: Deep Learning Essentials. Packt Publishing (2018). 11. Donkers T., Loepp B. & Ziegler J.: Sequential User-based Recurrent Neural Network Recommendations. In: Proceedings of the 11th ACM Conference on Recommender Systems, pp.152–160. ACM, Como, Italy (2017). 12. Fan W., Ma Y., Li Q., He Y., Zhao E., Tang J. & Yin D.: Graph Neural Networks for Social Recommendation. In: Proceedings of 28th International Conference on World Wide Web, pp. 417–426. ACM, San Francisco, USA (2019). 13. Fan W., Ma Y., Yin D., Wang J., Tang J. & Li Q.: Deep Social Collaborative Filtering. In: Proceedings of 13th ACM Conference on Recommender Systems, pp. 305–313. ACM, Copenhagen, Denmark (2019). 14. Gilpin L. H., Bau D., Yuan B. Z., Bajwa A., Specter M. & Kagal L.: Explaining Explanations: An Overview of Interpretability of Machine Learning. In: Proceedings of 5th IEEE International Conference on Data Science and Advanced Analytics, pp. 80–89. IEEE, Turin, Italy (2018). 15. Hou Y., Yang N., Wu Y. & Philip S. Y.: Explainable Recommendation with Fusion of Aspect Information. World Wide Web, 22(1), 221–240 (2019). 16. Lecue F.: On the Role of Knowledge Graphs in Explainable AI. In Semantic Web Journal (Forth-coming). http://www.semantic-web-journal.net/content/roleknowledge-graphs-explainable-ai-0 Last accessed 19 May 2020. 17. Lin Y., Ren P., Chen Z., Ren Z., Ma J. & de Rijke M.: Explainable Outfit Recommendation with Joint Outfit Matching and Comment Generation. ArXiv preprint arXiv:1806.08977 (2019). 18. Park H., Jeon H., Kim J., Ahn B. & Kang U.: Uniwalk: Explainable and Accurate Recommendation for Rating and Network Data. ArXiv preprint arXiv:1710.07134 (2018). 19. Tang, J. & Wang K.: Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding. In: Proceedings of the 11th ACM International Conference on Web Search and Data Mining, pp. 565–573. ACM, New York, NY, USA (2018). 20. Wu T., Khan A., Gao H. & Li C.: Efficiently Embedding Dynamic Knowledge
Graphs. ArXiv preprint arXiv:1910.06708 (2019). 21. Zhang Y., Lai G., Zhang M., Zhang Y., Liu Y. & Ma S.: Explicit Factor Models for Explainable Recommendation based on Phrase-level Sentiment Analysis. In: Proceedings of the 37th International ACM SIGIR Conference on Research & Development in Information Retrieval, pp.83—92. ACM, New York, NY, USA (2014). 22. Zhang Y. & Chen X. : Explainable Recommendation: A Survey and New Perspectives. arXiv preprint arXiv:1804.11192 (2018). 23. Zheng Y., Shi C., Cao X., Li X. & Wu B.: A Meta Path based Method for Entity Set Expansion in Knowledge Graph. In IEEE Transactions on Big Data, (2018). https://doi.org/10.1109/TBDATA.2018.2805366