An Agent-based Architecture to Recommend Educational Video

Abstract

-Agent-based recommender systems are tools able to assist users' choices with suggestions coming closest to their orientations. In this context, it is relevant to identify those users that are the most similar to the target user in order to require them suitable suggestions. However, particularly when we deal with video contents for e-Learning, it should be appropriate also to consider (i) recommendations coming from those students resulted the most effective in suggesting video and (ii) the effects of the device currently exploited. To address such issues in a multimedia scenario, we propose a multi-agent trust based recommender architecture, called ELSA, appositely conceived to this aim. Some preliminary performed simulations permitted to evaluate our proposal with respect to the other considered agentbased RSs.

-

Index Terms—Device adaptivity, e-Learning, Multimedia, Recommender system, Trust system

I. INTRODUCTION

An increasing number of video contents (VC) is available on the Web and many recommender systems (RS) support users by suggesting them those VCs that better meet their orientations on the basis of their past interests (Contentbased RSs [ 1 ]) and/or those of similar people (Collaborative Filtering RSs [ 2 ]) or a their combination (Hybrid RSs [ 3 ]).

A current trend in collaborative filtering (CF) processes is to consider both the similarity with respect to the characteristics of the device used in accessing VCs [ 4 ] (e.g. interface features, storage and computational capabilities, bandwidth and connectivity cost) and the effectiveness in providing suggestions. For instance, (i) a suggested video could not be accessible on a mobile device unable to correctly displays it and/or if its download is expansive in time or money or (ii) the opinions provided by a user, apparently similar to the target user, could result totally ineffective or even misleading.

To face such issues in a VCs e-Learning scenario, we propose a distributed multi-agent RS architecture, called ELearning Student Assistant (ELSA), to support students with personalized suggestions about VCs which considers both the device currently exploited by a user and the information derived by a trust system [ 5 ]–[ 7 ] in order to choose the most effective users in generating CF suggestions. In particular, ELSA adopts the Trust-Reliability-Reputation (TRR) model [ 8 ] which dynamically merges reliability (a measure of the trust directly perceived by an agent with respect to another) and reputation (a measure derived by the opinions of the other agents) into a global trust measure on the basis of the number of interactions occurred between the two agents. This characteristic appears important being the trustworthiness of an agent determinable only after a sufficient number of observations, and this leads to the necessity of dynamically changing the importance of the reliability vs the reputation.

In the ELSA architecture (Figure 1) each device hosts a device agent which monitors the student’s behavior on that device to build his/her local profile. An assistant agent periodically collects such profiles to build the student’s global profile and, based on it, associates the student with one/more partitions. Each partition groups similar students and it is managed by a partition agent that off-line precomputes personalized suggestions for its members by considering the exploited device and effectiveness (based on the adopted trust model) in providing opinions in the CF process. Note that assistant and partition agents exploit the cloud technology. Finally, each VC site is supported by a site agent that builds personalized site presentations [ 9 ]–[ 12 ] for each its student with the VCs suggested by the partition agents which him/her belongs to.

We remark as in computing suggestions the partition agents exploit information periodically sent them by the assistant agents to take into account changes occurring in students’ interests. This implies that partitions are periodically recomputed, allowing the system to adapt itself to the evolution of students’ behaviors, while the VC sites have not the computational costs for continuously performing onerous computations. Some simulations performed to evaluate ELSA by comparing it with other agent-based RSs shown an improvement in terms of performances.

clouds assistant agent

partition agents device agents site agent device agent

Each ELSA Web site is an XML site which presents unvisited VCs potentially interesting to each visiting student. To this aim, each VC on the platform is associated with a unique identifier code and with a category c (e.g. Java, XML, etc.) belonging to a catalogue C, common to the ELSA agents, realized as an XML-Schema document where each element represents a category c 2 C. Besides, for each student a global profile is built and updated to represent videos and categories of interest and a measure of their relevance for him/her.

In the ELSA architecture (Figure 2) each device agent is associated with a student’s device, assistant and partition agents live on the cloud and each site agent exploits computational and storing resources of a Web site server. When the student s uses the device D to visit a Web site W , the associated device agent d: (i) monitors his/her Web activities to build and update his/her Local Profile; (ii) sends its Working Profile, storing constraints in accessing VCs on D, to the site agent w associated with W . Besides, each device agent periodically sends its Local Profile to the student’s assistant agent that builds and updates his/her Global Profile to compute a global measure of relevance for videos and categories visited by s in order to associate s with one or more partitions, each one managed by a partition agent which in turn: (i) stores all the Global Profiles of the affiliated students and of the ELSA sites; (ii) for its affiliate students determines similarities and trust levels, in terms of effectiveness in providing suggestions, to pre-compute content-based (CB) and CF suggestions about potentially attractive unvisited VCs: (iii) sends the precomputed suggestions, compatible with the Working Profile of the device exploited by s in visiting W , to a site agent which arranges them on-the-fly in a presentation for s.

A. The Device Agent

Each device agent d builds and updates its Device Profile consisting of Working (W P ) and Local (LP ) Profiles. The Working Profile stores the following data and data structures: m, n : numbers of categories and videos for category that s desires to be suggested in a Web presentation on D; Constraint List (CL) : list of restrictions in accessing videos due to device hardware/software limitations; Agent List (AL) : list of the student’s assistant agent and of all his/her partition agents identifiers.

The Local Profile LP stores the student’s sessions history in accessing VCs on D and includes the Accessed Video Set (AV Set) and the Interest Weight Set (IW Set). AV Set is a set of tuples ⟨id; LVid; LAVid; ϕid⟩, each one associated with a video v, where id is the identifier code (unique in ELSA) of v, LVid is the number of accesses to v performed by s on D with LAVid (i.e. Last Access Video) its last access date and ϕid 2 [0; 1] a score provided by s to measure his/her appreciation about v, where 1/0 means the maximum/minimum satisfaction.

After the first access of s to v, his/her device agent computes ϕ as the ratio between the time spent by s on v with respect to its whole time length. It will be the measure of the satisfaction degree of s in absence of his/her explicit evaluation, always in the range [0; 1]. For each further access of s to v then he/she could (i) confirm the current ϕ value, (ii) provide a new ϕ value or (iii) accept a new device agent evaluation of ϕ.

IW Setd is a set of tuples ⟨c; LCc; LACc; LIWc⟩, each one associated with a category c 2 C, where LCid is the number of accesses a video belonging to c performed by s on D with LACc (i.e. Last Access Category) its last access date and LIWc (i.e. Local Interest Weight) a measure of his/her interest in c on D, that we suppose to be null after 180 days. In other words, the first time in a day that c is visited, LIW is updated as LIWc = LIWc + 1 based on its past value weighted by and a contribute for the current access set to 1. The coefficient (Figure 3) decreases the past LIWc based on its age (measured in days) and is a coefficient experimentally set to 0:445. More formally:

= ( 180 (curren1t80date LACc) ) 1

The device agent periodically sends its profile to the assistant agent of s and its profile W P to the site agents of the visited ELSA Web sites (see below). Periodically, the device agent prunes its profile LP from aged information.

B. The Assistant agent

The assistant agent a of s runs on the cloud to build a global representation of activities and interests of s (by using the information received from his/her device agents) to associate s with one/more partitions (Figure 1). The assistant agent profiles consists of the Assistant Working (AW P ) and the Global (GP ) Profiles. The first stores the following information: N D, N P : number of device agents of s and number of partitions which s desires to be affiliated; k : a real coefficient, arbitrarily set by s, ranging to [0::1], and weighting similarity vs trustworthiness; z : number of agents that s desires to exploit in computing collaborative filtering recommendations.

The Global Profile (GP ) includes the Global Accessed Video Set (GAV Set) and the Global Interest Weight Set (GIW Set). GAW Set represents each video v accessed by s with a tuple ⟨id; GVid; GLAVid; id⟩, where: (i) id is the identifier code (unique in ELSA) of v; (ii) GVid is the overall number of accesses to v performed by s; (iii) GLAVid (i.e. Global Last Access Video) is the most recent date among the accesses to v performed by s; (iv) id is the average rating among all the ϕid values stored in his/her Local Profiles. In GIW Set each category c 2 C is associated with a tuple ⟨c; GCc; GLACc; GIWc; ⟩, where: (i) GCc is the overall number of accesses to c performed by s; (ii) GLACc (i.e Global Last Access Category) is the most recent date among the accesses to c performed by s; (iii) GIWc is the Global Interest Weight computed as GIWc = G1Cc ∑iN=D1 LCc LIWc;i.

Note that GIWc implicitly considers device and connection cost in the choices of s. In particular, the connection cost is an indirect indicator for the relevance of c [ 4 ]. Periodically a updates the global profile GP of s by using updated local profiles and sends its GIW Set 2 GP to each partition agent in ELSA to obtain a similarity measure of s with their affiliated. In turn a will affiliate s with the first k partitions having the higher similarity measures, where their partition agents will generate personalized suggestions for s. This approach appears as a reasonable compromise, although it could not consider the last orientations of s.

C. The Partition Agent

A partition P , associated with a partition agent p, affiliates students similar for interests and all the ELSA site agents. The profile of p consists of the Site Catalogue Set (SCS), the Global Profile Set (GP S) and the Site Interest Set (SIS). The first includes the catalogues of all the ELSA Web sites. The Global Profile Set stores the global profiles of all the students of P . The Interest Site Set stores for each site W and in a data section SISW , a list SISW [s; D] for each student s visited W by using D. The elements of SISW [s; D] are pairs (c; SIWc), where c is a category present in W and considered interesting by s, and SIWc is its Site Interest Weight, an interest measure of s in c computed on the site side. These information are sent by the site agent w to p at the end of each Web session of s.

The partition agent p computes the similarity measures each time it is required by an assistant agent to evaluate the similarity of its student s with those affiliated with P based on their Global Interest Weight Sets (in order to consider the affiliation of s with P ). The average similarity measure sims;P 2 [0::1] between s and each other student t 2 P is the mean of all the Jaccard similarity measures sims;t [ 13 ], a real number ranging in [0::1] computed as the ratio between the number of their common categories which s and t are interested to and their total number (data stored into the s and t profiles) as sims;t = jGIW Ss \ GIW Stj/jGIW Ss [ GIW Stj. In computing sims;t we consider a student as interested in c only if its GIWc value is greater than a system threshold J . Therefore, the average similarity sims;P is computed as sims;P = (∑t2P sims;t)/jNP j, where NP is the number of students of P , and sent to as to consider the affiliation of s, (this implies that s will send its Global Profile GPs to p).

When s visits the site W by using the device D, the associated device agent d sends its W P to the site agent w that forwards it to the partition agents of s. To generate CB suggestions, each partition agent of s builds a list (CBs) storing the ns;D most popular VCs of W , unvisited from s and compatible with W P , for each one of the ms;D most interesting categories into GPs. To compute CF suggestions each partition agent uses opinions coming from those students resulting mostly similar to s (also for the device exploited in visiting W ) and trusted in P for providing fruitful suggestions. To this aim, a partition agent compares the profile of s stored in SISS [s; D] 2 SISW with each profile DSISW [t; D] of each its student t that visited W by using the same type of device in order to compute a similarity measure SIM (s; t; D), based on all the shared categories, c 2 C as SIM (s; t; D) = ∑c2SISW [s;t] ∩ SISW [s;D] jSIWc;s SIWc;tj. All these measures are normalized among them to assign to each agent t a score t by taking into account similarity and trust as t = k SIM (s; t; d) + (1 k) s;t, where s;t is the trust of t perceived by s (see Section III). Then the partition agent inserts in the list CFs the most popular ns;D videos stored in the site W , unvisited from s and compatible with W P , for each one of the ms;D most interesting categories presents into their profiles from the zs agents having the highest score.

D. The Site Agent

The site agent w is associated with the Web site W and its profile only consists of the catalogue of the categories present on the site. When the student s is visiting W by using the device D, the associated device agent d sends its current Working Profile W P to the agent w that in turn forwards it to all the partition agents of s for receiving their CB and CF suggestions lists, conform to W P , they pre-computed for s. From such lists the site agent selects the first most relevant n VCs for each one of the m categories. The site agent uses the selected recommended VCs to build on-the-fly a personalized site presentation for s When a student’s Web session ends, the site agent informs the partition agents of s about his/her choices in order to update the students’ trust values.

III. THE TRUST COMPUTATION

This trust model, derived by [ 8 ], provides each student of P with the trustworthiness of the other students of P in suggesting VCs associated with a category c 2 C. In particular, for each category c and for each student s, to know the trust values of each other student t 2 P (i.e. sc;t) on c, a linear system of N S equations in N S variables (where N S is the Number of Students of P ), that admits only one solution, has to be solved. With respect to the category c and the students s and t then each equation [ 8 ], [ 14 ] has the form: RS2 MWSuggest ELSA RS2 MWSuggest ELSA

TABLE I SETTING OF THE ELSA PARAMETERS OF THE DEVICE AGENTS. device which combines reliability and reputation measures weighted by sc;t 2 [0::1], that depends on the number of interactions occurring between s and t with respect to c (i.e. ics;t) and is computed as sc;t = ics;t=N , if ics;t < H or 1 otherwise, where the integer H is a system threshold. In other words, sc;t increases with the direct knowledge that s has of t about c. The first contribution in Eq. 1 is the reliability (i.e. cs;t 2 [0::1]) based on the level of knowledge that s has of t about the category c, where 0=1 means that t is unreliable/reliable. It is computed as the number of suggestions that s accepted from t, normalized by the total number of recommendations provided by t to s. The second contribution, ranging in [0::1], is the reputation of t that s computes by requiring an “opinion” on the capability of t to provide good suggestions in the category c to each student j ̸= t 2 P . This opinion is the “trust” that j has in t, weighted by the trust measure sc;j that s has of j In this way, the reputation is the weighted mean of all the trust measures jc;t that each student j (with j ̸= s; t) assigns to t.

IV. EXPERIMENTS

We performed some simulations to compare ELSA with RS2 [ 15 ] and MWSuggest [ 4 ] by using 20 XML Web sites, each one including in average 50 VCs associated with categories belonging to C. The RSs have been implemented in JADE [ 16 ] and JADE/LEAP [ 17 ] for the devices having limited resources. The ELSA and MWSuggest device agents (associated with a desktop PC, a tablet and a cellphone) have set their parameters as in Table I. Note that k is set to 0:5 and gives the same relevance to similarity and trust.

We simulated students visiting the Web sites and stored in a file (exploited as test-set) the first simulated 200 choices of each one on the first 10 Web site for different VCs as a list of tuple (consisting of source (s) and destination (d) links and the timestamp (t) of this choice) and other 200 tuple for the other 10 sites used to evaluate the RSs. The experiment involved three sets of 200, 400 and 800 simulated students. For each set, when a student s is visiting a Web page, each partition agent (for each one of the 3 RSs) provides a set R(p) of suggestions. Each element r 2 R(p) is a link to a VC and (i) if r is accepted then it is considered as a true positive and inserted in a set T Ps of all the true positives generated for s, (ii) if it is unaccepted then it is a false positive and inserted in a set F Ps, otherwise (iii) if a choice of s not belong to R(p) then it is considered as a false negative and inserted in a set F Ns. To measure RSs performances we computed the 1 0,9 0,8 in 0,7 o isc 0,6 e rP 0,5 e rga 0,4 e vA 0,3 0,2 0,1 0 0,9 0,8 0,7 lla 0,6 ceR 0,5 e rga 0,4 e vA 0,3 0,2 0,1 0

S200

S800 S400

Set standard measures Precision s ( s = jT Psj jT Ps [ F Psj) and Recall s ( s = jT Psj jT Ps [ F Nsj) for the set of produced suggestions for s that can be interpreted as the probabilities that a suggested link is considered as relevant by s and that a relevant link is recommended, respectively.

The Average Precision and the Average Recall of each RS, defined as the average of the and values, have been computed for the all tested RSs and student sets and show as ELSA is better than the other tested RSs on both such measures that increase when the size of the agent population is larger. The maximum advantage in term of precision (resp. recall) with respect to the second best performer (MWSuggest) is equal to 10,25 % (resp. 12,5 %) for a size of 200 users, and becomes equal to about 16 % (resp. 19 %) for a size of 800 users. If the advantage of ELSA with respect to RS2 might be attributed to consider the exploited device in generating the suggestions, the better performances with respect to MW-Suggest are surely due to the use of the trust model in computing the CF suggestions (indeed in terms of performances these two systems perform in an identical way with respect to the CB component of the suggestions).

V. RELATED WORK

e-Learning RSs [ 18 ], [ 19 ] guide students by suggesting educational resources [ 20 ] based on their profiles [ 21 ], [ 22 ]. In this context, CB suggestions are in line with users’ past interests [ 22 ], but suffer for attribute selection [ 23 ], overspecialization and inability to consider unknown items, while CF [ 24 ] approaches have high computational costs due to high data dimensionality and sparsity [ 25 ]. By combining CB and CF techniques most effective (hybrid) suggestions can be computed [ 3 ], [ 26 ]. For instance, hybrid e-Learning systems are presented in [ 27 ], which explores the impact of using a massive repository of educational indexed resources, personal data derived from students’ actions and more combinations of CB and CF recommenders and in [ 28 ] where group collaboration is supported, independently of time and space distance, and hybrid suggestions consider students and learning resources profiles, metadata, structural and semantic filtering criteria.

Nowadays, students access Web resources every time and everywhere by using different type of device and, therefore, RSs should suggest resources (i) natively compatible or (ii) adaptable to the device. Moreover, suggestions can be based on (i) a unique global student’s profile which takes into account learner’s activities performed on each his/her device or (ii) different learner’s profiles, one for each his/her device. ELENA [ 29 ] is a full distributed RS where a personal agent recommends resources to a student based on the information stored in his/her profile, only referred to the exploited device, and in the those of other students sharing the same interests and device. An interface based on a traffic light metaphor brings out the recommended resources. In ISABEL [ 30 ], a distributed multiagent learning system, each student’s device is monitored by an agent when the student accesses e-Learning Web sites, each one associated with a teacher agent. Each student is associated with one or more tutor agents providing personalized suggestions for him/her, also considering the device, that the teacher agent shows in a personalized presentation compatible with the student’s devices. Also in [ 31 ] are considered the opportunities provided by mobile devices to delivery personalized contents (adapted by algorithms designed to this purpose) compatible with learner’s preference on that device, device capabilities and contextual environment.

Human interactions widely exploit the concept of trust to know the trustworthiness of own counterparts with respect to same held skills or in order to avoid deceptions [ 32 ]– [ 35 ]. Therefore trust plays a role similar to a social control to determine the best subjects to interact with, particularly important in virtual environments which encourage possible malicious behaviors [ 36 ], [ 37 ]. In particular, information derived by direct experiences (i.e., reliability) can be used to trust others, but they usually only exist for a narrow set of users and/or for a small number of times. As a consequence, a direct and reliable opinion about someone could be impossible to have, therefore to trust potential partners the opinions provided by other users (i.e. reputation) have to be considered and the reputation accuracy increases as much as their number increases [ 36 ]. Reliability and reputation information are often combined together in order to obtain a single synthetic trust measure [ 38 ]–[ 43 ] also by considering a multidimensional approach [ 40 ], [ 44 ]–[ 47 ].

With respect to a RS, trust can be assumed as the perception that the source is competent or, conversely, that a learning resource is valid and interesting. In other words, it is the perceived skill of the recommender to offer the right suggestions. In [ 48 ] video on demand are recommended by adopting CB and CF techniques, but this later exploits only expert users selected by a trust system. In [ 49 ] the idea of trustworthiness is associated to both learning resources (described by common ontologies) and peers in a P2P e-learning scenario. Trust relationships among peers allow to select which ones of them can be considered more authoritative in answering a query within a given topic, whereas trust about learning resources allows the most reliable resources to be selected.

Learning Networks are open infrastructure to provide teachers and learning objects. However, due to the available number of learning resources teachers are supported in finding the most suitable for them by RSs. In this context, to make accurate recommendations by solving the problems due to sparsity of educational datasets, in [ 50 ] it is proposed to adopt trust information obtained by monitoring the teachers’ activities. Finally, EVA [ 51 ] is a framework of learning recommender agents migrating among users based on a cloning mechanism. Each agent stores in its profile the knowledge learned from all its past and current owners. A reputation system, inspired to genealogical criterion, helps the system to select the most trusted agents in the community to be cloned and migrated among users’ to provide suggestions.

VI. CONCLUSIONS

We presented ELSA, a fully distributed hybrid RS agent architecture appositely designed to provide students with personalized suggestions on potentially interesting VCs by also taking into account the device currently exploited. Besides, in the generation of CF suggestions, ELSA considers not only those users that are the most similar to the current user but also those that in the past are resulted as the most effective in provide suggestions on the basis of a trust system integrated in ELSA. Some simulated evaluations have shown promising performances of the ELSA platform with respect to some other tested RSs proposed in the literature. As our ongoing research we are planning to test ELSA with real users in the next future.

References

[1] P. Lops , M. Gemmis , and G. Semeraro, “ Content-based recommender systems: State of the art and trends,” in Recommender Systems Handbook . Springer, 2011 , pp. 73 - 105 .

[2] J. Breese , D. Heckerman , and C. Kadie , “ Empirical analysis of predictive algorithms for collaborative filtering ,” in Proceedings 14th International Conference on Uncertainty in Artificial Intelligence . Morgan Kaufmann, 1998 , pp. 43 - 52 .

[3] R. Burke ., “ Hybrid recommender systems: Survey and experiments,” User Modeling and User-Adaptivity Interaction , vol. 12 , no. 4 , pp. 331 - 370 , 2002 .

[4] D. Rosaci and G. M. L. Sarne ´, “ Recommending multimedia web services in a multi-device environment , ” Information Systems , vol. 38 , no. 2 , pp. 198 - 212 , 2013 .

[5] J. Sabater and C. Sierra , “ Review on computational trust and reputation models , ” Artificial Intelligence Review , vol. 24 , no. 1 , pp. 33 - 60 , 2005 .

[6] D. Sarvapali , S. Ramchurn , and N. Jennings , “ Trust in multi-agent systems,” The Knowledge Engineering Review , vol. 19 , pp. 1 - 25 , 2004 .

[7] S. Na , K. Choi , and D. Shin , “ Reputation-based service discovery in multi-agents systems ,” in Proceedings IEEE International Workshop on Semantic Computing and Applications . Springer-Verlag, 2010 , pp. 326 - 339 .

[8] D. Rosaci , G. Sarne`, and S. Garruzzo , “ Integrating trust measures in multiagent systems ,” International Journal of Intelligent Systems , vol. 27 , no. 1 , pp. 1 - 15 , 2012 .

[9] C. Anderson , P. Domingos , and D. Weld , “ Adaptive web navigation for wireless devices ,” in Proceedings 17th International Joint Con. on Artificial Intelligence . Morgan Kaufmann, 2001 , pp. 879 - 884 .

[10] P. De Meo , D. Rosaci , G. M. L. Sarne ´, G. Terracina, and D. Ursino , “ An xml-based adaptive multi-agent system for handling e-commerce activities ,” in Proceedings 1st International Conference ICWS-Europe 2003 , ser . LNCS , vol. 2853 . Springer-Verlag, 2003 , pp. 152 - 166 .

[11] S. Macskassy , A. Dayanik , and H. Hirsh, “ Information valets for intelligent information access ,” in Proceedings AAAI Spring Symposium Series on Adaptive User Interfaces. AAAI , 2000 .

[12] D. Rosaci and G. M. L. Sarne´, “A Multi-Agent Recommender System for Supporting Device Adaptivity in e-Commerce,” Journal of Intelligent Information System , vol. 38 , no. 2 , pp. 393 - 418 , 2012 .

[13] G. Greenstette , Explorations in Authomatic Thesaurus Construction. Kluwer Academic Pub., 1994 .

[14] F. Buccafurri , D. Rosaci , G. M. L. Sarne ´, and L. Palopoli , “ Modeling cooperation in multi-agent communities , ” Cognitive Systems Research , vol. 5 , no. 3 , pp. 171 - 190 , 2004 .

[15] K. Che , Y. Tsung-Hsien , and L. Wei-Po , “ Personalized multimedia recommendation with social tags and context awareness ,” in Proceedings World Congress on Engineering 2011 , vol. 2 . Springer, 2011 , pp. 6 - 11 .

[16] JADE URL , “http://www.jade.tilab.org,” 2012 .

[17] Caire G. , LEAP 3 .0: User Guide , TLAB.

[18] H. Drachsler , H. Hummel , and R. Koper , “ Personal recommender systems for learners in lifelong learning networks: the requirements, techniques and model,” Internatinal Journal of Learning Technology , vol. 3 , no. 4 , pp. 404 - 423 , 2008 .

[19] O. Santos and J. Boticario , Educational Recommender Systems and Technologies: Practices and Challenges. Information Sc. Ref ., 2012 .

[20] P. Brusilovsky and C. Peylo , “ Adaptive and intelligent web-based educational systems ,” International Journal of Artificial Intelligence in Education , vol. 13 , no. 2 , pp. 159 - 172 , 2003 .

[21] H. Liu and V. Keselj , “ Combined mining of web server logs and web contents for classifying user navigation patterns and predicting users' future requests,” Data Knowledge Engineering , vol. 61 , no. 2 , pp. 304 - 330 , 2007 .

[22] F. Wang and H. Shao , “ Effective personalized recommendation based on time-framed navigation clustering and association mining , ” Expert Systems with Applications , vol. 27 , pp. 365 - 377 , 2004 .

[23] K. Cheung , J. Kwok , M. Law , and K. Tsui , “ Mining customer product ratings for personalized marketing,” Decision Support Systems , vol. 35 , pp. 231 - 243 , May 2003 .

[24] U. Shardanand and P. Maes , “ Social information filtering: Algorithms for automating ”word of mouth”,” in Proceedings Conference on Human Factors in Computing Systems. ACM/Addison-Wesley , 1995 , pp. 210 - 217 .

[25] S. Weng and M. Liu, “ Feature-based recommendations for one-to-one marketing , ” Expert Systems with Applications , vol. 26 , pp. 493 - 508 , 2004 .

[26] J. Herlocker , J. Konstan , and J. Riedl , “ Explaining collaborative filtering recommendations ,” in ACM Conference on Computer Supported Cooperative Work. ACM , 12 2000 , pp. 241 - 250 .

[27] M. Khribi , M. Jemni , and O. Nasraoui , “ Automatic recommendations for e-learning personalization based on web usage mining techniques and information retrieval,” in Advanced Learning Technologies , 2008 . ICALT' 08 . 8th IEEE International Conference on. IEEE, 2008 , pp. 241 - 245 .

[28] A. Carbonaro , R. Ferrini , and M. Zamboni , “ Considering semantic abilities to improve a web-based distance learning system , ” in ACM Interantional Workshop on Combining Intelligent and Adaptive Hypermedia Methods/Techniques in Web-based Education Systems , 2005 .

[29] P. Dolog , N. Henze , W. Nejdl , and M. Sintek , “ Personalization in distributed e-learning environments ,” in Proceedings of the 13th International World Wide Web Conference on Alternate track P & P. ACM , 2004 , pp. 170 - 179 .

[30] D. Rosaci and G. M. L. Sarne ´, “ Efficient Personalization of e-Learning Activities Using a Multi-Device Decentralized Recommender System,” Computational Intelligence , vol. 26 , no. 2 , pp. 121 - 141 , 2010 .

[31] X. Zhao , F. Anma , T. Ninomiya , and T. Okamoto, “ Personalized adaptive content system for context-aware mobile learning ,” International Journal of Computer Science and Network Security , vol. 8 , no. 8 , pp. 153 - 161 , 2008 .

[32] C. Corritore , B. Kracher , and S. Wiedenbeck , “ On-line trust: concepts, evolving themes, a model ,” International Journal of Human-Computer Studies , vol. 58 , no. 6 , pp. 737 - 758 , 2003 .

[33] J. Sabater-Mir and M. Paolucci , “ On representation and aggregation of social evaluations in computational trust and reputation models ,” International Journal of Approximate Reasoning , vol. 46 , no. 3 , pp. 458 - 483 , 2007 .

[34] S. Ramchurn , D. Huynh , and N. Jennings , “ Trust in multi-agent systems,” Knowledge Engeenering Review , vol. 19 , no. 1 , pp. 1 - 25 , 2004 .

[35] P. Resnick , R. Zeckhauser , E. Friedman, and K. Kuwabara , “ Reputation systems,” Commununication of ACM , vol. 43 , no. 12 , pp. 45 - 48 , 2000 .

[36] A. Birk , “ Boosting cooperation by evolving trust , ” Applied Artificial Intelligence , vol. 14 , no. 8 , pp. 769 - 784 , 2000 .

[37] G. Lax and G. M. L. Sarne ´, “ Celltrust: a reputation model for c2c commerce ,” Electronic Commerce Research , vol. 8 , no. 4 , pp. 193 - 216 , 2006 .

[38] Aberer K. and Despetovic Z. , “Managing Trust in peer-2 -peer Information Systems,” in Proceedings 10th International Conference on Information and Knowledge Management. ACM , 2001 , pp. 310 - 317 .

[39] M. G o´mez , J. Carb o´, and C. Benac-Earle , “ An anticipatory trust model for open distributed systems,” in Anticipatory Behavior in Adaptive Learning Systems, ser . LNAI , vol. 4250 . Springer-Verlag, 2007 , pp. 307 - 324 .

[40] T. Huynh , N. Jennings , and N. Shadbolt , “ An integrated trust and reputation model for open multi-agent system,” Autonmous Agent and Multi Agent Systems , vol. 13 , 2006 .

[41] S. Kamvar , M. Schlosser , and H. Garcia-Molina , “ The eigentrust algorithm for reputation management in P2P networks,” in Proceedings 12th Interantional Conference on World Wide Web. ACM Press , 2003 , pp. 640 - 651 .

[42] P. Melville , R. Mooney , and R. Nagarajan , “ Content-boosted collaborative filtering for improved recommendations ,” in Proceedings 18th National Conference on AI. AAAI , 2002 , pp. 187 - 192 .

[43] L. Xiong and L. Liu, “ Supporting reputation-based trust for peer-to-peer electronic communities,” IEEE Trans. on Knowledge and Data Eng ., vol. 16 , no. 7 , pp. 843 - 857 , 2004 .

[44] J. Sabater and C. Sierra , “Reputation in gregarious societies,” in Proceedings 5th International Conference on Autonomous Agents. ACM , 2001 , pp. 194 - 195 .

[45] D. Jia , F. Zhang , and S. Liu, “ A robust collaborative filtering recommendation algorithm based on multidimensional trust model , ” Journal of Software , vol. 8 , no. 1 , pp. 11 - 18 , 2013 .

[46] D. Rosaci and G. M. L. Sarne ´, “ Rebecca: A trust-based filtering to improve recommendations for b2c e-commerce,” in Intelligent Distributed Computing VII . Springer, 2014 , pp. 31 - 36 .

[47] G. Wang and J. Wu , “Multi-dimensional evidence-based trust management with multi-trusted paths,” Future Generation Computer Systems , vol. 27 , no. 5 , pp. 529 - 538 , 2011 .

[48] J. Cho , K. Kwon , and Y. Park , “ Collaborative filtering using dual information sources , ” IEEE Intell. Sys. , vol. 22 , no. 3 , pp. 30 - 38 , 2007 .

[49] V. Carchiolo , D. Correnti , A. Longheu , M. Malgeri , and G. Mangioni, “ Exploiting trust into e-learning: adding reliability to learning paths ,” International Journal of Technology Enhanced Learning , vol. 1 , no. 4 , pp. 253 - 265 , 2009 .

[50] S. Fazeli , H. Drachsler , F. Brouns , and P. Sloep , “ A trust-based social recommender for teachers ,” from: http://hdl.handle.net/ 1820 /4428, 2012 .

[51] D. Rosaci and G. M. L. Sarne ´, “ Cloning mechanisms to improve agent performances , ” Journal of Network and Computer Applications , vol. 36 , no. 1 , pp. 402 - 408 , 2012 .