Patent search systems allow complex queries to be formulated by combining di erent search terms using boolean and other operators such as proximity, wildcards, etc. in order to nd relevant patents. This widely adopted approach is based on exact match, making it difcult to e ciently identify and analyze relevant patents, as the search terms often do not match the terminology used by the inventors. Another problem concerns the large number of relevant hits due to weekly and monthly updates of patent applications and grants. Although some semantic search systems for patents based on latent semantic analysis have been implemented as black-box systems in the past, word embeddings that have been successfully applied to generate semantic representations of text have rarely been employed and evaluated for a (large) patent corpus. The work described here aims to evaluate semantic representations for patent data via a pre-trained general model in comparison to an adapted word embedding model created from a patent corpus in order to contribute to a multitude of semantic analysis tasks for patents such as similarity search, content analysis, entity linking etc..
Patents are regarded as an important source allowing companies to de ne new
business strategies and support high-level decision making processes. With
increasing complexity and volumes of patent data enhanced search systems and
novel methods for analyzing patents [
In [
Dataset. In order to create a domain-speci c word embedding model, a subset of patent documents from the WIPO (World Intelectual Property Organization) and the EPO (European Patent O ce) patent databases has been sampled by ltering for speci c patent classi cation codes as shown in Table 1. In total Search Fields Used IPC, CPC Codes IPC, CPC G06* AND NOT G06M* OR H04L0009* OR
H04W0012-00 OR H04H0060-23 OR G11* 410.607 patent documents have been retrieved and the English patent description texts were used to generate the patent embedding model, furtheron referred to as IT corpus.
Model Creation. To create a word2vec model, the Gensim library was applied using default parameters. The model creation and document vector representation were based on the description text of the patent documents. For comparison the pre-trained Google word2vec model has been used as a baseline.
Get Your Hands Dirty: Evaluating Word2Vec Models for Patent Data
The evaluation3 of both models for the task of semantic similarity of patent documents was performed based on a randomly selected set of 25 documents (14 WIPO and 11 EPO) from the IT corpus. For each document the 10 most similar patent documents were determined based on the vector representation of the description text. To obtain document vectors, word vectors for the patent description text were averaged. The cosine similarity was used to compute the similarity among document vectors. As to our best knowledge there exists no ground-truth for this task, a qualitative evaluation was carried out with patent experts. In the evaluation they assesed the relevance of the top 10 similar patents according to the following 3-level scale: 0: irrelevant patent, 1: related patent, 2: similar patent. 4.2
Google Model Patent Model irrelevant related similar irrelevant related
similar 0:6 0:5 0:4 0:3 0:2 0:1 0 1 0:6 0:5 0:4 0:3 0:2 0:1 0 1 2 3 4 5 6 7 8
In order to compare the results of both models we analyzed the cumulated and normalized scores (Fig. 1 and Fig. 2) in dependence of the rank for the top 10 documents. Looking at the similarity, we see better average score for the Patent Model with increasing rank, while the relatedness shows a di erent picture. Here, the higher ranks of the Google Model4 show better results, while with increasing rank the relatedness score meet at the same level. It can also be
4 https://code.google.com/archive/p/word2vec/ observed that the average score for the irrelevant documents rise with increasing rank in the Google Model, while in the Patent Model the score stabilize around 40%. We assume that for the similarity the representation of domain-speci c words in the customized model is more sophisticated compared to the Google model, which was trained on general (news) data. In contrast, the patent model was trained only with a much smaller number of domain-speci c patent data not being able to cover all aspects of relatedness appropriately.
In this paper, we have evaluated semantic representations of patent texts via word embeddings created from distinct corpora. We compared the results for the two regarded distinct word2vec models for the tasks of semantic similarity and relatedness. The achieved results showed that the mean average scores for the domain speci c word embedding model are much higher in comparison to the general Google word2vec model, while the relatedness aspect must be evaluated in additional experiments employing a more ne-grained scoring scheme for the experts. In future work, we aim to enhance our approach by considering a more sophisticated pre-processing also taking into account the inherent structure of a patent document. In this work, the evaluation was performed based on the patent description text only, while in future we also want to analyze how abstracts and claims of the patent text can be exploited for di erent patent analysis use cases in di erent selected domains such as life science, engineering, etc.