This paper describes the participation of the Loz team in the Multilingual Web Person Name Disambiguation task of IberEval 2017. The solutions consist of diferent variants of the traditional hierarchical agglomerative clustering algorithm. Tha analysis of results gives information about the relative e ectiveness of considering di erent feature projections (word presence, term frequency and if.idf).
Users search information on the Web using search engines, and frequently the
need of information is about people. As di erent people share the name, the
problem of disambiguating people names consists in grouping the results of a
web search engine according to the di erent individuals they refer to. This
problem has been studied extensively, but usually in a monolingual context, with all
the web pages results written in the same language. Previously, WePS (Web
People Search) evaluation campaigns proposed this task in a web searching scenario
providing several corpora for evaluating the results of their participants,
particularly WePS-1 [
The rest of the paper is organized as follows: Section 2 describes our proposed methods to disambiguate person names. The results obtained are presented in Section 4 and their analysis and discussion are in Section 5. Finally, Section 6 draws the conclusions of the work. 2.1
In a rst step we transform each document into a vector of values which is used as input for the hierarchical agglomerative clustering algorithm. To this aim each document is divided into tokens by just splitting the text by spaces. After this, each token is transformed into a lowercase representation in order to avoid ambiguity and decrease the number of words in the dictionary for the vector representation. Finally, all words with a frequency of 1 were removed. Due to computational constraints, for each entity only the 2000 most frequent words in the dictionary generated in the previous step were selected. In order to transform each document into a vector of values we have proposed three state of the art approaches based on term frequency.
{ Presence: for each document the resulting vector contains a value of 0 if the document does not contain the word of the dictionary in the position i of the vector, and a value of 1 if the document contains the word. { Frequency: for each document the resulting vector contains a value of 0 if the document does not contain the word of the dictionary in the position i of the vector, and the frequency of the word in the document if the document contains the word. { tf/idf: for each document the resulting vector contains a value of 0 if the document does not contain the word of the dictionary in the position i of the vector, and the tf/idf value for each word if the document contains the word. The if.idf weight is computed regarding the full set of documents associated to the corresponding person name. 2.2
Using the training data set, we have compared the clustering results using euclidean distance vs. cosine similarity. The second one is the most recommended approach in the literature. The advantage of using angular distances (cosine) is that frequent words in documents are not overweighted. However, our preliminary experiments over the training data set show that the euclidean distance gives better results. It can be due to the fact that term weighting in the representation step is computed over the person name collection of document. Therefore, the normalization of frequencies in the person name domain is implicitely solved in the previous step. 2.3
Finally, we have implemented the traditional Hierarchical Aglommerative clustering algorithm. HAC required to de ne the criterion to determine what pair of clusters are joined in each step (linkage) and in what moment the clustering process stops. We have experimented with single linkage (minimum distance between items from both clusters), and complete linkage (maximum distance between items from both clusters). However, the results showed that single linkage is more suitable. This could be due to the heterogeneity of documents that refer to the same person.
The most basic alternatives for the stopping criterion is a certain proximity and stating a prede ned amount of nal clusters (k value). We have used the second approach, testing the results with k=5 and k=15. 3
We have submitted 5 runs, with di erent linkages, stopping criteria and feature projections as we described in the following table:
Run Run 1: Run 2: Run 3: Run 4: Run 5: k (stop criterion) Feature Weighting k=5 Word frequency (tf) k=15 Word presence k=15 tf.idf k=5 Word presence k=5 tf.idf
In addition, the organization of the task provided two baselines: { One-in-one. The baseline method where every Web page is assigned to a di erent cluster. { All-in-one. The baseline method where all Web pages are assigned to a single cluster. 4
This section reports the results of the experiments that we have performed to
disambiguate person names. The metrics for evaluating the results are:
Reliability (R), Sensibility (S) and their harmonic mean F0:5(R; S) [
Table 1 shows the results achieved by our methods considering in the evaluation only related web pages, and Table 2 shows the results considering all web pages. Interestingly, for both k=5 and k=15, we have found the same relative behavior of feature projections (presence, term frequency, tf.idf). The presence of words outperforms term frequency. This result suggests considering the frequency of terms in documents can overscore high frequent terms, even when the cosine distance mtigate this e ect. These improvements across feature projections are robust across component evaluation metrics. Both Reliability and Sensitivity are improved simultaneously.
The second remarkable conclusion is that considering ve clusters as stop criterion instead of 15, increases substantially the Sensitivity (recall) at the cost of a relatively small decrease in reliability. However, there is a clear trade o anyway, so the solutions are not comparable and could be highly a ected by the weight of each metrics in F. 6
This paper describes the participation of the Loz team in the Multilingual Web Person Name Disambiguation task of IberEval 2017. The solutions consist of diferent variants of the traditional hierarchical agglomerative clustering algorithm. The results have reported information about the relative e ectiveness of considering di erent feature projections (word presence, term frequency and if.idf). In addition, they have corroborated the sensitivity of the k value (stop criterion) to the trade of between recall and precision oriented evaluation metrics.