Recommendations play a very important role for revealing related topics addressed in the wikis beyond the currently viewed page. In this paper, we extend KiWi, a semantic wiki with three di erent recommendation approaches. The rst approach is implemented as a traditional tag-based retrieval, the second takes into account external factors such as tag popularity, tag representativeness and the a nity between user and tag and the third approach recommends pages in grouped by tag. The experiment evaluates the wiki performance in di erent scenarios regarding the amount of pages, tags and users. The results provide insights for the e cient widget allocation and performance management.
Wiki is a collaborative knowledge space that can be edited by anybody who is
granted permission [
In this work, we extend the KiWi system with three tag-based recommender
approaches, which suggest links to wiki pages based on the similarity of their
tags. The rst approach recommends pages which share tags, the second
approach takes into account external factors such as tag popularity, tag
representativeness and the a nity between user and tag and nally the third approach
groups recommendations by tags. The performance of the approaches is
compared in di erent scenarios, which varies in terms of amount of pages, users
and tags. The outcome from this analysis provides insights for widget allocation
(where the recommendations are placed) and subsequent performance
optimization. Our development is placed at KiWi [
The paper is organized as follows: In Section 2 we discuss related work. In Section 3, a motivation scenario is presented. Section 4 introduces the recommendation approaches. Section 5 presents the experimental evaluation and results. A discussion about the results from the previous section is presented in Section 6 and nally in Section 7, we conclude the study and also point to future works. 2
A number of semantic wiki applications have been explored over the last years
and most of them utilize annotations to contextualize the content presentation
and improve the navigation throughout all existing pages. SemperWiki is a
semantic personal wiki developed for the Gnome desktop in which users can edit
and annotate pages semantically [
Equally to KiWi, OntoWiki interface is surrounded by widgets that provide
meta-information from semantic annotations and navigation support. Although
OntoWiki does not use tags for processing recommendations, it contains a
particular widget for related pages categorized by the Most Popular and Most Active
[
In general, the semantic wiki applications analyzed utilize annotations in
the pages for navigation, rendering and search purposes. However, we observe
that navigation is still centered on dynamically generated lists of related content
with no or few personalized information. Personalization will drive the system
features in accordance with individual tastes and preferences [
In general, tags are assigned to Web resources in order to conceptualize,
categorize and organize them in a way that users can be reminded later about the
tagged content [
Figure 1 shows the current development in KiWi system in which tags assigned to the currently viewed page are located on the bottom widget on the left side (1). The recommendation widgets are highlighted on the right side: the widget number (2) contains the standard recommendation, the widget number (3) contains the multifactor recommendation and the widget number (4) contains the recommendations grouped by tags. The recommendations expose a variety of options for a user to visit just on a single click. If this activity is designed well, then the choice is easy, and the user keeps interacting with the system by visiting the related pages or adding new content. Although each recommendation approach has its own particularities (See in Section 4), very hardly the three solutions will run in parallel in real life scenario because they occupy much space in the user's interface and occasionally issue the same information (at the same time). In addition of being useless from the usability perspective, to have all three widgets running together compromise the system performance at all. As known, wiki is a collaborative space utilized for multiple interactions and any performance concern is always advisable. Based on these premises, the performance analysis is undertaken in order to nd out widget combinations so that the overall performance is optimized and users take advantage of better widget arrangements. 3.1
In this work we are mapping semantics of user activities based on tagging activity. Using uncontrolled tags, users are able to annotate pages without any restriction constrained by ontology vocabularies. In this sense, users are free to express their feelings about the page as they like on any purpose. The outcome of this tagging activity is a relation between user and a page through a tag property, as seen in the Figure 2.
From the tag-based relationships, we are deriving personalized recommendations by computing similarities between tags, however, in later stages, other relevant information can be derived and utilized to annotate the wiki pages using RDF properties such as ont:mostFrequentTag, ont:userMostInterested and ont:mostSimilarPage. These properties would create a semantic network between content items in KiWi, and also could be utilized for other personalization goals such as group formation, semantic search and creation of link structures. 4
This section depicts the standard, multifactor and recommendation grouped by tags addressed in this work.
Standard Tag-based Recommendation. In this approach, all pages that share tags with the currently viewed page are recommended. In this standard approach no further similarity processing is carried out therefore the list of recommendation is not ranked. The advantage of this approach is the performance since the recommendations relies simply on a data retrieval task. On the other hand, a single tag shared by pages may not be su cient means to determine a similarity between pages. This approach however cannot be discarded without analyzing its applicability in the di erent possible KiWi scenarios. Figure 3 shows standard recommendations in KiWi with their respective authors.
Multifactor Recommendation. The multifactor recommendation approach computes similarity between pages considering multiple factors. The recommendations rely on calculus of cosine similarity, tag popularity, tag representativeness and a nity user-tag. We utilize a cosine similarity measure between tag vectors to calculate basic similarity of the pages. We measure tag popularity as a count of occurrences of a certain tag in total number of wiki pages. The term frequency measure is used to compute tag representativeness for a certain wiki page. The tag a nity between a user and a tag is calculated as a count of how many times the user utilized the tag at di erent web pages. We propose a formulae which consider all these factors in a normalized way and gives a ranking of pages for particular user.
We de ne a page score as:
n n P s = X weight(T agi)+X representativness(T agi), where n is the total numi=1 i=1 ber of existing tags in the repository.
We de ne the tag user a nity as: Af f inity(u;t) = cardfp 2 P ages j (u; t; p) 2 P; P U T P g=cardft 2 T j (t; u) 2 Pu; Pu U T g, where t is a particular tag, u particular user, U is a set of users, P set of pages and T set of tags.
Finally, similarity is computed as: Similarity(Pi;Pii) = [P sPi + P sPii cosine similarity(Pi; Pii)] Af f inity(u;t).
Informally, each one of the factors in the above formulas is calculated as
follows:
i. Cosine Similarity | Our tag similarity is a variant on the classical cosine
similarity from the text mining and information retrieval [
Figure 4 shows the same recommendations as Figure 3 however sorted di erently due to the quality factors calculus. In terms of performance, the multifactor approach is worse than the standard one however the ranked list provides credibility to the recommendations. Pages ranked higher are personalized to user's tastes and closer to the content discussed in the currently viewed page. Although the recommendations are more e ective than the standard approach, its applicability also depends on further performance analysis in di erent KiWi scenarios. Recommendations Grouped by Tags. In this approach, the recommendations are grouped by the tags which are assigned to the currently viewed page. Similarly to the standard approach, no further similarity processing is undertaken and the list of recommendation is not ranked. On the other hand, the user can go directly to the recommended wiki page just following the tag he/she is interested. The tagbased distribution explicitly provides a justi cation why the recommendations were generated and assist users to nd related speci c wiki pages.
The disadvantage however is the possibility of existing duplicated recommendations since two di erent pages can share two distinct tags as well. Figure 5 shows two tags xml and Semantic Web with their respective linking recommendations. The duplication problem is outlined since both tags recommend a link to the RDF wiki page. The performance analysis therefore will answer whether this duplication problem a ects the performance to the point of discarding the applicability of this approach. The experimental evaluation assessed the performance of the recommender approaches by simulating a mix of scenarios regarding the amount of pages, users and tags. In addition, we discussed which widgets of recommendations should be displayed or suppressed in accordance with the performance ndings. Although having in mind that standard approach is theoretically the most advisable approach in terms of performance, we evaluated when and in which conditions the other approaches become suitable and necessarily required. The goal therefore is to propose insights for widget adaptation aligned with running scenarios without compromising the system performance.
The proposed scenarios were created aiming at simulating realistic usage of KiWi. Nevertheless there is no "pattern" or "standard" about wiki activity. This is more about politics from where the system is deployed, maturity of whom is using it and time of activity. Although understanding that building wiki scenarios is a little subjective, we proposed three growing scenarios that are likely envisioned in KiWi life cycle as described in Table 1.
The variables addressed by each scenario are: { Amount of Pages { each page has a set of tags that are compared for processing the recommendations. Therefore the more pages exist, the more time will be spent to calculate the similarity between the pages. { Amount of Tags { the similarity of the pages is given by their tags. The whole set of tags of each page must be compared to verify which ones are similar. In particular, the amount of tags of an user impacts directly in the time for the computation of the a nity between user and tag. { Amount of Users { the more users KiWi contains, the more are the chances of tagging activity. As a consequence, more time will be necessary to process the personalized recommendations.
These variables were chosen justi ed because they are row material for calculating the recommendations. This process is time consuming and invariably a ects the system performance however it does not mean that other factors such as page size should not be considered. Finally, the scope of this work only comprises these three variables. 5.1
Initially, the KiWi system was populated with pages and tags using a random generator to produce su cient amount of content. The users were created manually since they were only 15 at most. The content of the pages and tags were extracted from Web sites on the Internet and local documents. Similarly, we utilized a random generator to assign tags to wiki pages tagged by a particular user. In this study, it is not important to speculate about the random function we have utilized as we look at overall performance of the system and not the method for distributing the content. We needed just to generate adequate number of satisfactorily di erent pages and su ciently di erent assignment of tags to them. To each scenario created, we collected the response time necessary to load the recommendations. We repeated this procedure for 10 times in order to have a more real and democratic results. In KiWi, the recommendations are processed every time a page is called, then we tested the performance after the user login, accessing a page from page link and from the own recommendations. The tests ran in machine equipped with processor Intel (R) Core (TM) 2 Duo CPU T7500 @2.20Ghz.
Expected Results Our assumption is that at some point due to the high amount of recommendations, the standard approach become ine ective although the performance continues better than the other approaches. The quality achieved with the multifactor recommendation will be needed even though the performance is decreased. Moreover, we believe the group recommendation is always useful due to its facility of identifying similar pages by tags however its performance may discourage its adoption due to the high number of duplicated pages. 5.2
The rst scenario was setup with 20 pages, 5 users and 225 tags. Figure 6 shows 10 time stamps collected from KiWi system to calculate the recommendations for the three approaches. The average column from Figure 6 shows that standard recommendations were computed in 28 ms; multifactor recommendation in 77 ms and recommendations grouped by tags in 29.1 ms.
The standard and grouped approaches had better performance than the multifactor approach. While standard and grouped approaches achieved rates quite close to one another (about 28ms), the multifactor approach spent 175% more time than both approaches to calculate the recommendations. As already known, the multifactor recommendation generates a ranked list of recommendations more personalized and reliable. The point to be assessed therefore is whether the ranking compensates this time necessary for ranking the recommendations. In this particular case where only 20 wiki pages are considered, the multifactor approach may be ignored depending on the visibility of the recommended pages to the users. If the widgets of standard and grouped recommendations are able to expose the whole recommendations with easy access, they will be continuously exposed and hardly forgotten. In this sense no ranked recommendation is necessary. 5.3
The second scenario was setup with 50 pages, 10 users and 500 tags. Figure shows 10 time stamps collected from Kiwi system to calculate the recommendations for the three approaches. The average column from Figure 7 shows that standard recommendations were computed in 97 ms; multifactor recommendations in 121 ms and grouped recommendations in 45.3 ms.
The grouped approach achieved the best performance followed by standard and nally by the multifactor approach. The multifactor approach lasted approximately 25% more than the standard approach to generate the recommendations. Comparing to the rst scenario, it is observed a considerable approximation between standard and multifactor approach (from 125% to 25%). The issue to be discussed therefore is whether the standard recommendation approach is still desired due to the low performance variation between the standard and multifactor approach. In this scenario, 50 wiki pages are addressed and the recommendation widgets tend to enlarge with the increase of recommendations. In this case, therefore, a ranking approach becomes useful since the most similar pages are placed at the top of the list of recommendations. In addition, very hardly the recommendation widget will provide an ample visibility of whole set of recommendations. In this sense, the 24 additional milliseconds to generate the multifactor recommendations compensate the probability of having ine ective recommendations. The high performance of the group based approach can be justi ed by the low amount of duplicate recommendation. Furthermore, it is important to observe that although the amount of pages of this scenario is the double comparing to the rst one, the current performance was decreased of only 55.6% (or 16.2 ms) from the rst measure. This approach therefore becomes a strong candidate to be utilized in this particular scenario even in combination with the multifactor recommendations.
The third scenario was setup with 100 pages, 15 users and 700 tags. Figure 8 shows 10 times stamps collected from KiWi system to calculate the recommendations for the three approaches. The average column from Figure 8 shows that the standard recommendations were computed in 98.1 ms; multifactor recommendation in 145 ms and grouped recommendations in 53.1 ms.
Similarly to the second scenario, the grouped approach achieved the best performance followed by standard and nally by multifactor recommendation. In this turn the standard approach was approximately 50% faster than the multifactor approach. This is a considerable di erence that stands face-to-face two goals: performance and e ectiveness. On one side, KiWi process lots of recommendations very quickly however unsorted, on the other hand the same recommendations last at least 50% more but they are ranked. A qualitative experiment in which the users show their satisfaction in terms of performance and e ectiveness would indicate in which direction to go. A possible solution however is to combine the multifactor approach with the fastest group recommendation, which attenuates the overall loss of performance for this scenario. The group approach performance decreased only 17% from the second scenario (from 45.3 ms to 53.1 ms) and still provide a special distribution of the recommendations. 5.5
The performance of the approaches in each scenario analyzed are presented in the Figure 9.
In the rst scenario, the multifactor approach was considerably more expensive (in terms of performance) than the others. This signi cant cost for generating recommendations discourages its adoption. In the second scenario, the standard approach reduces signi cantly its performance, which encourages the use of the multifactor approach. From the rst to the second scenario, the group approach maintains a satisfactory level of performance. In the third scenario, standard and grouped approaches keep their performance approximately equal to the previous scenario, whereas multifactor approach presents a signi cant loss of performance. 6
The performance outcomes from the scenarios analyzed allow us to suggest intelligent widget allocations without compromising the system performance. For the rst scenario, the standard and group approaches are the most advisable to run together spending in total about 57.1 ms to calculate the recommendations. For the second scenario, we suggest the multifactor and group approaches running in parallel spending in total 166.3 ms and for the third scenario, due to the need of quality, again the multifactor and group approaches are the most advisable spending together about 198.1 ms to calculate the recommendations. Figure 10 shows the performance with the suggested combination to each scenario analyzed.
Fig. 10. Suggested Performance Graph
According to Figure 10, the performance for the rst scenario is 57.1 ms, which is signi cantly low since two approaches of recommendation are being computed. In the second scenario, it is observed an expressive loss of performance (from 57.1ms to 166.3 ms) mainly due to the utilization of the multifactor approach. In the third scenario however the loss of performance was 31.8 ms, which is much less than the loss of performance from the rst to the second scenario (109.2 ms). In fact, the multifactor approach utilized in both second and third scenarios reduced the performance, however, we assure that the best recommendations will be placed in the top of the list of recommendations. In general, the overall result obtained shows the tendency of the performance whenever the amount of user, page and tag grows. Although the more scenarios are advisable for con rming this tendency, this preliminary outcome can be employed for predicting the system performance in emergent scenarios.
Other advantage from the results obtained, besides the widget allocation, is that they show a tendency of the performance whenever the amount of user, page and tag grows. The achieved numbers can be utilized for predicting the system performance in emergent scenarios. On the other hand, the amount of the scenarios analyzed is still low to con rm this tendency. More experiments, with a bigger setup and with multiple users using the system at the same time would provide a more realistic feedback about the performance. 7
This paper analyzes the performance of three tag-based recommender approaches for a semantic wiki. Three di erent scenarios were assessed varying in terms of number of pages, amount of tags and users. To each scenario assembled, it was analyzed which recommendation approaches could be more appropriate taking account the system performance and user needs. The results showed that the combination between standard and group approach is feasible for scenarios up to 20 pages, which are constantly accessed. For scenarios with 50 and 100 pages with more than 10 users, the multifactor and group approaches are more advisable in spite of being more expensive in terms of performance. The grouped recommendation approach is always adequate since it provides justi cation for recommendation and visual support for navigating among the recommendations.
As future works, rst, pre computation of some factors in multifactor recommendation will be studied to further increase performance of recommendation computing. Furthermore, semantic relatedness between tags must be considered since current recommendations in KiWi only consider the tag syntax to identify similarities. The next step therefore is to combine the tag algorithms with some reasoning on the annotations to provide more e cient recommendations. Another direction is to annotate tags with their role (or purpose) in order to formalize the relationship between tags and pages. The semantic recommendation will likely be more e cient by capturing precise needs of the users expressed by the annotations. 8
The research leading to these results is part of the project "KiWi - Knowledge in a Wiki" and has received funding from the European Communitys Seventh Framework Programme (FP7/2007-2013) under grant agreement No. 211932.