Web spamming describes behavior that attempts to deceive search engine's ranking algorithms. TrustRank is a recent algorithm that can combat web spam by propagating trust among web pages. However, TrustRank propagates trust among web pages based on the number of outgoing links, which is also how PageRank propagates authority scores among Web pages. This type of propagation may be suited for propagating authority, but it is not optimal for calculating trust scores for demoting spam sites. In this paper, we propose several alternative methods to propagate trust on the web. With experiments on a real web data set, we show that these methods can greatly decrease the number of web spam sites within the top portion of the trust ranking. In addition, we investigate the possibility of propagating distrust among web pages. Experiments show that combining trust and distrust values can demote more spam sites than the sole use of trust values.
H.3.3 [Information Storage and Retrieval]: Information Search and Retrieval
In today's Web, a link between two pages can be considered to be an implicit conveyance of trust from the source page to the target page. In this case, trust implies that the author of the source page believes that the target page provides some content value.
With the increasing commercial interest of being ranked
high in search engine results, content providers resort to
techniques that manipulate these results. This behavior is
usually termed Web spam, or search engine spam. Many
kinds of spam have been discovered [
Trust can be used to combat Web spam. Gyongyi et
al. [
TrustRank shows that the idea of propagating trust from a set of highly trusted seed sites helps a great deal in the demotion of Web spam. But TrustRank is just one implementation of this idea. This approach makes certain assumptions with regard to how trust is propagated from a parent page to a child page. For example, the authors claim that the possibility of a page pointing to a spam page increases with the number of links the pointing page has. Because of this, they proposed the idea that the trust score of a parent page be equally split amongst its children pages.
This assumption is open to argument. Why should two equally trusted pages propagate di erent trust scores to their children just because one made more recommendations than the other? Also, with respect to the accumulation of trust scores from multiple parents, TrustRank puts forth just one solution, that of simple summation. Clearly, there are other alternatives.
A natural extension of the idea of the conveyance of trust between links is that of the conveyance of distrust. Here, distrust has a di erent meaning to that in the context of social networks. In social networks, distrust between two nodes A and B usually means that A shows distrust explicitly to B. In contrast, in our system, distrust is a penalty awarded to the source page for linking to an untrustworthy page. Hence, this distrust is an indication that we don't trust some web pages, not an indication that one page doesn't trust another page on the web. Actually, the trust score of a page can also be interpreted as how much we trust this page.
In general, spam pages can be considered to be one type of untrustworthy pages. To elaborate on this idea, consider that a page links to another page and hence according to the above de nition of trust, this page expresses trust towards the target page. But if this target page is known to be a spam page, then clearly the trust judgment of the source page is not valid. The source page needs to be penalized for trusting an untrustworthy page. It is likely that the source page itself is a spam page, or is a page that we believe should not be ranked highly for its negligence in linking to an untrustworthy page.
In this paper, we explore the di erent issues present in the problem of propagating trust on the Web. We also study the application of propagating distrust on the Web. Additionally, we present techniques to combine trust and distrust scores to improve the overall performance in demoting Web spam.
The rest of this paper is organized as follows: the background and related work will be introduced in Section 2 and Section 3 respectively. The motivation of this work will be introduced in Section 4. The details of our technique are given in Section 5. The experiments and results will be shown in Section 7. We nish with discussion and conclusion in Sections 8 and 9.
The web can be represented by a directed graph, given web pages as the nodes and hyperlinks among web pages as the directed links among the nodes. The adjacency matrix M of the web graph is: M [i; j] equals 1 if there is a hyperlink from page i to page j, or 0 otherwise. Suppose we use I(i) to represent the in-degree of node i and O(i) as the out-degree of node i, the de nition of the transition matrix T is: and the de nition of the reverse transition matrix R is: T [i; j] = M [j; i]=O(j) R[i; j] = M [i; j]=I(j) (1) (2) 2.2
Gyongyi et al. [
In many SEO discussion boards, participants discuss the
latest ranking and spam- nding techniques employed by
commercial search engines. One approach, called
BadRank1, is believed by some to be used by a commercial
engine to combat link farms.2 BadRank is based on
propagating negative value among pages. The idea of BadRank
1One description of BadRank can be found at [
While the idea of a focused or custom PageRank vector
has existed from the beginning [
Jeh and Widom [
Several researchers have done some work to combat
different kind of Web spam. Fetterly et al. propose using
statistical analysis to detect spam [
Benczur et al. proposed SpamRank in [
In prior work, we [
Guha et al. [
Massa and Hayes [
Ziegler and Lausen [
Gray et al. [
The original TrustRank paper proposed that trust should be reduced as we move further and further away from the seed set of trusted pages. To achieve this attenuation of trust, the authors propose two techniques, trust dampening and trust splitting. With trust dampening, a page gets the trust score of its parent page dampened by a factor less than 1. With trust splitting, a parent's trust score is equally divided amongst its children. A child's overall trust score is given by the sum of the shares of the trust scores obtained from its parents.
In the case of trust splitting, we raise a question: Given
two equally trusted friends, why should the
recommendations made by one friend be weighted less than the other,
simply because the rst made more recommendations? A
similar argument has been made by Guha [
It is observed that a spam page often points to other spam
pages for the purposes of boosting their PageRank value
and manipulating search engine results [
Hence, given a set of labeled spam seed set, we can
propagate distrust from this set to the pages that point to
members of this set. The idea is that a page pointing to a spam
page is likely to be spam itself. But sometimes, good pages
may unintentionally point to spam pages. In this case, these
pages are penalized for not being careful with regard to
creating or maintaining links (as suggested by [
In doing so, each page on the Web is assigned two scores, a trust score and a distrust score. In the combined model, a link on the Web can then propagate these two scores. As shown in Figure 1, suppose there is a link from Page A to
Page B, then trust is propagated from Page A to Page B, while distrust is propagated from Page B to Page A.
We explore di erent techniques for the handling of propagation of trust and distrust from the respective seed sets to other pages on the Web. 5.
In this section, we present details of our ideas on propagating trust and distrust among web pages. 5.1
TrustRank propagates trust among web pages in the same manner as the PageRank algorithm propagates authority among web pages. The basic idea is that during each iteration, a parent's trust score is divided by the number of its outgoing links and each of its children gets an equal share. Then a child's overall trust score is the sum of the shares from all its parents.
Two key steps in the technique described above may be explored. One is, for each parent, how to divide its score amongst its children; we name this the \splitting" step. The other is, for each child, how to calculate the overall scores given the shares from all its parents; we name this the \accumulation" step.
For the splitting step, we study three choices:
Equal Splitting: a node i with O(i) outgoing links and trust score T R(i) will give d TOR((ii)) to each child. d is a constant with 0 < d < 1; Constant Splitting: a node i with trust score T R(i) will give d T R(i) to each child; Logarithm Splitting: a node i with O(i) outgoing links and trust score T R(i) will give d logT(1R+(Oi)(i)) to each child.
We term d to be the decay factor, which determines how much of the parents' score is propagated to its children. In fact, if d equals 1, then the above \Equal Splitting" is the same as the method used in TrustRank. As discussed in the Section 4, why should equally trusted pages propagate di erent trust scores just because they have di erent number of children? With \Constant Splitting", each parent will give a constant portion of its trust value to all of its children irrespective of the number of its children. Thus for a child, if two of its parents have identical trust values but di erent number of children, then the child will get the same value from both of these parents. The third choice, \Logarithm Splitting" does not eliminate the e ect of the number of children that a page has but can decrease it.
Since \Equal Splitting" is the choice already being employed in TrustRank, we will focus on \Constant Splitting" and \Logarithm Splitting" in our experiments.
For the accumulation step, we study three choices.
Simple Summation: Sum the trust values from each parent.
Maximum Share: Use the maximum of the trust values sent by the parents.
Maximum Parent: Sum the trust values in such a way as to never exceed the trust score of the mosttrusted parent.
The rst choice is the same as in PageRank and TrustRank; using the sum of trust scores from all parents as the child's trust score. For \Maximum Share", the maximum value among the trust values inherited from all the parents is used as the child's trust score. For \Maximum Parent", rst the sum of trust values from each parent is calculated and this sum is compared with the largest trust score among each of its parents, the smaller of these two values is used as the child's trust score.
By using the above choices, the equation for calculating trust score is di erent from Equation 3. For example, if using \Constant Splitting" and \Simple Summation", the equation will become: where t is the trust score vector, is the jump probability, d is the constant discussed in the above splitting choices, M is the web matrix shown in Section 2.1 and s is the normalized trust score vector for the seed set. 5.2
The trust score of a page is an indication of how trustworthy the page is on the Web. In the case of web spam, the trust score can be seen as a measure of the likelihood that a page is not a spam page.
Similarly, we introduce the concept of distrust to penalize the pages that point to untrustworthy pages. Now, it is possible that pages unintentionally point to spam pages. In these cases, we argue that the (otherwise good) page should be penalized to some extent for not being careful in its linking behavior.
Distrust propagation makes sense when spam sites are used as the distrusted seed set and distrust is propagated from a child to its parent. So, based on this idea, one link can represent two propagation processes, i.e., the trust score is propagated from the parent to the children while the distrust score is propagated from the children to the parent.
In this technique, some known spam pages are selected as the distrusted seeds and assigned some initial distrust scores. During each iteration, the distrust score is propagated from children pages to parent pages iteratively. After convergence, a higher distrust score indicates that this page is more likely to be a spam page.
A direct method of calculating distrust score for each page is to follow the same idea as TrustRank. The calculation can be represented by Equation 5.
n = (1 )
R n + r (5) where n is the distrust score vector, is the jump probability, R is the reverse transition matrix shown in Equation 2 and r is the normalized distrust score vector for the distrusted seed set. Before calculation, n is initialized with the value of r.
However, as discussed in Section 5.1, the propagation mechanism of TrustRank may not be optimal to propagate trust or distrust for the purpose of demoting spam pages. We propose that the same choices to propagate trust, discussed in Section 5.1, can be taken to propagate distrust.
Suppose we use DIS T R(i) to represent the distrust score for node i. For the splitting step, we have three choices: Equal Splitting: a node i with I(i) incoming links and DIS T R(i) will give dD DISI(Ti)R(i) to each parent. where 0 < dD < 1; Constant Splitting: a node i with DIS T R(i) will give dD DIS T R(i) to each parent; Logarithm Splitting: a node i with I(i) incoming links and DIS T R(i) will give dD lDogI(S1+TIR((ii))) to each parent.
The \Equal Splitting" choice is quite similar to that in the case of trust propagation in TrustRank. Intuitively, this kind of splitting may raise problems when the purpose of propagating distrust is to demote spam. For a simple example, by \Equal Splitting", a spam site with more parents will propagate smaller distrust to its parents, while spam sites with fewer parents will propagate bigger distrust to its parents. Obviously, this policy supports popular spam sites and this is clearly not desirable for the purpose of demoting spam. In comparison, \Constant Splitting" and \Logarithm Splitting" present better choices.
For the accumulation step, we also have three choices: Simple Summation: Sum the distrust values from each child.
Maximum Share: Use the maximum of the distrust values sent by the children; Maximum Parent: Sum the distrust values in such a way as to never exceed the distrust score of the mostdistrusted child.
Di erent choices will employ di erent equations during the calculation. For example, if using \Constant Splitting" and \Simple Summation", the equation of calculating distrust score is: n = (1 ) dD
M n + r (6) where n is the distrust score vector, is the jump probability, d is the constant discussed in the above splitting choices, M is the web matrix shown in Section 2.1 and r is the normalized distrust score vector for the distrusted seed set. 5.3
On propagating trust and distrust to the pages on the web, each page will be assigned two scores, a trust score and a distrust score. Then comes the question of combining them to generate a uni ed ranking of pages that is indicative of their trustworthiness.
Our goal of propagating trust and distrust is to demote spam sites in the ranking. Since the trust score is an indication of how unlikely it is that the page is a spam page, while the distrust score is an indication of how likely it is that the page is a spam page, a direct solution is to simply calculate the di erence of these two scores and use this value to represent the overall trustworthiness of the Web page.
Additionally, we may apply several methods for the combination. For example, we may give di erent weights when calculating the sum. Suppose we use T otal(i) to represent the di erence of trust and distrust score for page i. Then we can apply the following formula:
T otal(i) =
T R(i)
DIS T R(i) (7) where and (0 < < 1, 0 < < 1) are two coe cients to give di erent weights to trust and distrust scores in this formula.
The data set used in our experiments is courtesy of
search.ch search engine [
In order to generate a trusted seed set, we extract all the
URLs listed within the search.ch topic directory [
For the generation of a distrusted seed set, we use the labeled spam list which contains 3; 589 sites or domains. In our experiments, we use only a portion of this list as the distrusted seed set with the rest being used to evaluate the performance.
We test all the ideas we propose in Section 5 by using the search.ch data set. Since the goal of this paper is to investigate how di erent mechanisms of propagating trust and distrust can help to demote top ranking spam sites, we will focus on the ranking positions of the labeled 3; 589 spam sites.
We rst calculate the PageRank value for each site based on the search.ch site graph. These sites are then ranked in a descending order of their PageRank values. Based on this ranking, we divide these sites among 20 buckets, with each bucket containing sites with the sum of their PageRank values equal to 1=20th of the sum of the PageRank values of all sites.
We then calculate the TrustRank score for each site based
on the site graph, to generate a ranking of sites sorted in
the descending order of these scores. As in the case of the
TrustRank paper [
TrustRank is shown in Figure 2. It is clear that TrustRank is good at demoting spam sites compared to PageRank.
In this paper, we use the number of spam sites within the
top 10 buckets as the metric for measuring the performance
of algorithms. This choice of choosing the top 10 buckets was
arbitrary as in the case of [
The results of this metric for the PageRank and TrustRank algorithms are shown in Table 1. These results will be used as the baseline results. We can see that PageRank ranks 90 spam sites within the top ten buckets, while TrustRank ranks only 58 spam sites. 7.1
In TrustRank, the jump probability in Equation 3 is usually assigned a value of 0:15. We measure the performance of TrustRank with di erent values of this jump factor.
Since we use all the URLs listed in dir.search.ch as the trusted seed set, it is quite possible that some spam sites get included in this set too. On checking, we nd that 35 labeled spam sites are within the trusted seed set. It is worthwhile to drop these spam sites from the seed set. We run TrustRank again with di erent jump probabilities after dropping these 35 labeled spam sites from the seed set.
The results with both the original seed set and the cleaned seed set are shown in Figure 3. We observe that larger jump probabilities decrease the number of spam sites from top ranking positions. Since a larger jump probability means that smaller trust values are propagated from a parent to its children, the results show that for the purpose of demoting spam sites, in TrustRank, a better approach is of relatively little trust propagation. We also observe that the dropping of spam sites from the seed set results in fewer spam sites within the top ten buckets.
Algorithm PageRank TrustRank
No. of Spam sites in top 10 buckets 90 58
As introduced in Section 5, we explore two choices in the splitting step: \Constant Splitting" (d T R(i)) and \Loga
T R(i) rithm Splitting" (d log(1+O(i)) ), while we have three choices in the accumulation step: \Simple Summation", \Maximum Share" and \Maximum Parent".
The number of di erent combinations of the above choices is six. For each combination we try using di erent values of d ranging from 0:1 to 0:9. The results of these six combinations with di erent values of d are shown in Table 2.
From the results in Table 2, we can tell that \Simple Summation" always generates the worst performance, which is worse than TrustRank and even PageRank. A lot of spam sites are raised in the ranking. Intuitively, this \Simple Summation" will boost the rankings of sites with multiple parents. In general, it is likely a spam site that has a large number of incoming links will be able to accumulate a fairly large value of trust. Hence, spam sites may be bene ted by this \Simple Summation" method.
We also observe that, in most cases, both \Maximum Share" and \Maximum Parent" methods generate much better performance than TrustRank and the \Simple Summation" method. With regard to the splitting methods, we observe that in most cases, \Logarithm Splitting" performs better than \Constant Splitting".
The results clearly demonstrate that for the purpose of demoting web spam, propagating trust based on the idea of \Equal Splitting" and \Simple Summation" which is used by TrustRank, is not the optimal solution.
Gyongyi et al. [
It is worth mentioning that by introducing the above ideas of splitting and accumulating trust, we notice, in some cases, long ties in the trust scores. For example, the top several thousand of sites may have identical trust scores. This is di erent from the values by PageRank or TrustRank. We think this tie is still reasonable as long as few spam sites are in the ties close to the top. Since there are 3; 823 sites in the top ten buckets by PageRank, we consider the ties that have rankings around this position still within top ten buckets, thus all the spam sites before or within this tie will still be counted within top ten buckets.
Actually, we nd that for most cases, these ties can help to demote more spam sites. But some small d may cause a strong tie with more than 10; 000 sites and thus raise the number of spam sites within top ten buckets. One example is that there are 372 spam sites within top ten buckets when combining \Maximum Parent" and \Logarithm Splitting" with d set to 0:1. 7.3
Trust can be propagated from trusted seed set to the
children pages iteratively. Similarly, distrust can be propagated
from a distrusted seed set to the parent pages iteratively.
While our distrusted seed set was provided to us, in general
a search engine will maintain a spamming blacklist, using
both manual and automatic methods (perhaps, e.g., [
In order to investigate whether introducing distrust can help to improve the performance in demoting spam sites, we randomly select a portion of labeled spam sites as the distrusted seed set and calculate distrust values for each site. The ranking positions of the remaining spam sites will be used to evaluate the performance. 7.3.1
As described, there are several di erent choices of propagating distrust among web pages, we rst use the method shown in Equation 5.
We randomly select 200 spam sites from the 3; 589 labeled spam sites as the distrusted seed set to calculate distrust score. Then we calculate the sum of this distrust score and the trust score generated by TrustRank. By using the sum for ranking, we count the number of spam sites (m) in the top ten buckets as in the case of previous experiments.
But we can not compare the above number m directly with the results shown in Table 1. The reason is that some top ranked spam sites may have been selected in the distrusted seed and they will get demoted as an e ect of their selection, not as an e ect of our algorithm. Thus, in order to be fair, we need to count the number of spam sites (n) that are in the top ten buckets by TrustRank which are also in the distrusted seed set. Only when the sum of m and n is smaller than 58, which is listed in Table 1, we can claim that the performance is better than that of TrustRank.
Also the random selection of distrusted seeds may still not be representative of the 3; 589 spam sites. In order to neutralize this bias, we repeated the above seed selection ve times for calculating distrust scores. Then we use the average results as the nal results for the distrusted seed set with 200 seeds. On average, there are 54 spam sites still in the top ten buckets and 4 spam sites are in the distrusted seed set. The sum of 54 and 4 equals the number of spam sites, which is 58, in top ten TrustRank buckets; this shows that using TrustRank's mechanism (Equation 5) to propagate distrust is not helpful in demoting top ranked spam sites.
In order to verify whether introducing more distrusted seeds with this basic distrust propagation is useful, we generated distrusted seed sets of sizes ranging from 200 to 1; 000. Similarly, for each seed set size, we repeated this generation ve times. The average results are shown in Table 4. The results show that no matter how many seeds are selected for the distrusted seed set, the sum of the second element and third element in Table 4 is always around 60. Since this sum is quite close to the 58 spam sites in Table 1, we believe that using the same mechanism as TrustRank to propagate distrust can not help to demote top ranked spam sites. 7.3.2
Since we have shown that propagating distrust by using the TrustRank mechanism may not be helpful, the next obvious step is to investigate whether the choices of propagating trust can also be applied for propagating distrust in order to demote top ranked spam sites.
Similar to the methods used for generating results in Table 2, we applied the six combinations of di erent choices for the splitting step and accumulation steps to the propagation of distrust. In order to evaluate the performance, for each combination, we calculate the sum of the distrust value and TrustRank value for each site. Then this sum is used for ranking. Since the TrustRank value is unchanged for Number of seeds 200 400 600 800 1000
No. of Spam sites No. of Spam in top 10 buckets sites in seed set 54 4 55 5 49 12 48 13 45 16 each di erent combination, we can see how di erent choices of propagating distrust can a ect the overall performance and thus we can tell which choice is better for propagating distrust. For simplicity, we only choose 200 spam sites to generate the distrusted seed set once. Results of six di erent combinations with di erent d values are shown in Table 3.
From the results in Table 3, we can see that some choices can help to demote more spam sites than others. For example, the combination of \Logarithm Splitting" and \Maximum Share" with d set to 0:7 or 0:9. 7.4
In the above experiments, we use the sum of the trust and distrust values as the nal value for ranking. As discussed in Section 5, we may use di erent weights to combine trust and distrust values.
In practice, we did the following experiment to show how the combination of trust and distrust values can a ect performance.
To calculate trust score, we select the choice that can generate best performance in Table 2, i.e., using \Maximum Share" for accumulation and \Logarithm Splitting" for splitting while with d set to 0:3.
To calculate distrust score, we select the choice that can generate best performance in Table 3, i.e., using \Maximum Share" for accumulation and \Logarithm Splitting" for splitting with dD set to 0:9.
For combining trust and distrust values, we follow the Equation 7, with equals 1 . Test with di erent values of .
We test with di erent numbers of distrusted seeds.
The results for these experiments are shown in Figure 4. There are three lines in the gure. Each represents the results by using 200, 400, 600 spam sites as distrusted seed respectively. From these results, we can tell that an increase in the size of the distrusted seed set will result in an increase in performance.
Compared with the baseline results in Figure 1, more than 80% of spam sites disappear from the top ten buckets. This veri es our hypothesis that using di erent trust propagation methods together with distrust propagation can help to demote spam sites e ectively.
Actually, the results in Figure 4 are not our best results. During our experiments, we nd that by using \Constant Splitting" and \Maximum Parent" for trust propagation, \Logarithm Splitting" and \Maximum Share" for distrust propagation with d, dD and as 0:1, we can remove all the spam sites from the top ten buckets. We believe that there may be several other combinations that generate optimal results. However, due to resource constraints, we have not enumerated every such combination. 7.5
Since the trust or distrust scores are propagated from limited number of seed pages, it is quite possible that only a part of the whole web graph can be touched by this propagation. In other words, some pages will have zero values after the algorithm is employed. We are not in a position to make trust judgments with regard to these pages. It is highly desirable to have a well performing algorithm that with a limited seed set enables us to make trust judgments about a large fraction of web pages.
Intuitively, di erent values for in Equation 3 or d in \Constant Splitting" and \Logarithm Splitting" will determine how far trust and distrust are propagated. In TrustRank, smaller means that more trust will be propagated to children pages in each iteration; thus more pages may have nonzero value after 20 iterations. In order to show this, for the same experiment shown in Figure 3, we check what percentage of sites have nonzero values according to di erent values of . The results are shown in Table 6.
If more sites have nonzero values by using di erent choices, then we can claim that the trust scores are propagated further by these choices. Since the results obtained by using \Maximum Share" and \Maximum Parent" in Table 2 are better than TrustRank, we check the percentage of pages with nonzero values for these choices. The results are shown in Table 5.
The results in Table 5 show larger numbers when compared to the results in Table 6. This demonstrates that our choices can a ect more pages as well as generate better performance in the demotion of top ranking spam sites.
In this paper, we investigate the possibility of using different choices to propagate trust and distrust for ranking Web pages or sites. We only focus on the demotion of spam sites. In the future, we intend to study how the propagation of trust or distrust can help raise high quality sites in the ranking positions.
We show that mechanisms such as \Logarithm Splitting" or \Maximum Share" for propagating trust and distrust can do better than TrustRank in demoting top ranked spam sites. We intend to explore other choices that can help improve the performance.
In our paper, we combine trust and distrust scores only at the nal step. It is possible that this combination can be done during the calculation of trust and distrust scores. We aim to study the di erent choices that may be taken into this combination.
Ranking algorithms such as PageRank are used by several popular search engines for ranking Web pages to given queries. The concept of authority and trustworthiness are not identical|PageRank gives an authority value for each page, while propagating trust from seed sets tells how trustworthy a page on the web is as a source of ranking information. In this paper we have only explored the value of trust propagation for spam demotion; ultimately the goal, however, is to improve the quality of search results. We plan to investigate combinations of trust and distrust with authority to measure the e ect on search results ranking (quality of results).
All of our experiments are based on the search.ch data
set. This data set may have special characteristics di erent
from the whole web. We need to test the ideas presented
here on a larger data set, such as the WebBase [
In this paper, we show that propagating trust based on the number of outgoing links is not optimal in demoting top ranked spam sites. Instead, we demonstrate that using different choices such as \Constant Splitting" or \Logarithm Splitting" in the splitting step and \Maximum Share" or \Maximum Parent" in the accumulation step for propagating trust can help to demote top ranked spam sites as well as increase the range of trust propagation.
Additionally, by introducing the concept of propagating distrust among Web pages or sites, we show that the performance of demoting top ranked spam sites can be further improved.
This work was supported in part by the National Science Foundation under award IIS-0328825. We are grateful to Urban Muller for helpful discussions and for providing access to the search.ch dataset.