Patentability and novelty search is an essential part of any patent application. It ensures that the idea that should be patented has not already been registered anywhere else in the world. However, this task is complicated by the large number of documents and the fact that they are written in many different languages. In this paper we survey four approaches that will help to automate the task and share the insights we have gained through our participation in the CLEF-IP Workshop 2013.
The level of innovation is one of the principal measures that determines if an idea can be patented or not. In order to estimate it, an exhaustive search in over 80 million patents from more than 100 patent authorities has to be performed. To date, this is still usually carried out with the help of keyword searches. That this strategy does not work is proven by the fact that 54% of the 2.5 million annual patent applications are rejected. The main problem is that the language used in patents is often subject specific as well as inaccurate and misleading. The reason for this is that the applicants are usually not interested in making their ideas public and therefore try to disguise them as good as possible. Another problem is that patent documents may be written in many different languages, which makes it even harder to find similar ideas.
For our participation in the patentability and novelty search task of the CLEF-IP workshop, which was to automatically find all documents and their respective passages that describe concepts strongly related to those explained in the source document, we explored the applicability of four approaches that will be explained in the following chapter. For the CLEF-IP patentability or novelty search task all participants were provided with a corpus divided into two sets of patent documents: the first set Dpat contains 3.118.088 patent documents (2.680.604 from the EPO and 437.484 from the WIPO) and the second set Dapp 210 patent documents (belonging to 69 patents). Furthermore a set of 149 topics T was given. Each ti 2 T was defined as a subset from the claims of one patent document di 2 Dapp. The basic task was, given a topic ti, find those passages pn in dpat 2 Dpat that are semantically related.
The results were then evaluated on document as well as on passage level. On
document level the Patent Retrieval Evaluation Score (PRES) [
In order to obtain the most relevant documents and passages from Dpat for ti, the retrieval process was divided into two stages: candidate-retrieval and detailed comparison. During the candidate-retrieval the most relevant documents from Dpat were selected and subsequently a detailed comparison was performed to determine the most relevant passages.
In order to reduce the space of possible candidates and improved retrieval quality, we only considered patents from Dpat, which shared at least one International Patent Classi cation (IPC) with the patent containing ti. We furthermore used abstracts, claims and descriptions from all patent family members, since the text contained in ti was too short.
As the amount of patents in the workshop task, as well as in a real-world
scenario is limited, in three of the four approaches the candidate-retrieval was
performed by calculating the text similarity between ti and all possible candidate
documents from Dpat. The detailed comparison at passage level was executed
subsequently between ti and the top 100 candidates from Dpat using the same
similarity measure. As text similarity measure, we evaluated 3 approaches which
were all based on the Vector-Space Model (VSM) and the cosine similarity:
(i) Word uni-grams (ii) Character tri-grams, (iii) Cross-Language Explicit
Semantic Analysis (CL-ESA) . The first approach was based on the idea of simply
comparing the used vocabulary, ignoring the fact that some documents are
written in other languages. This approach was considered the best approximation
for a keyword-search, a strategy commonly used by humans to generate a patent
search-report. The second approach was based on the findings that two text
documents written in different european languages have a strong character N -gram
overlap [
In comparison to the first three approaches, the fourth was based on a heuristic candidate retrieval. Therefore, we generated a set of queries for each source document and executed them on a search engine, which had all documents from Dpat indexed. For the top candidate documents we then executed a detailed comparison based again on word uni-grams.
The results of each submitted run, are presented in Table 1. They show, that
MAP 0.30 0.26 0.23 0.21 0.38 0.37 0.31 0.29 the simplest approach (cosine similarity between vectors of word uni-grams) outperforms any other approach. This can be explained by the fact that it guarantees a higher ranking for documents with similar vocabulary. The same documents will get a higher score in case of an intelligent keyword-search as it is typically executed by humans. This also explains the good results of the heuristic candidate retrieval, as it aims to imitate humans behaviour too. That the approach based on character tri-grams did not bring the expected advantage is due to the fact that documents of the same language still share a lot more character n-grams than semantically related documents in different languages. The interpretation of the results obtained using CL-ESA may require further investigation. They show that this approach is, out of the four, the worse approximation for human search behaviour. Nevertheless, they do not reflect necessarily a bad performance. Preliminary investigations of the results have shown that CL-ESA assigned a higher rank to documents which seem to be highly related and which did not appear in the result-set of simple keyword-searches and neither in the patent search report. Hence, in order to obtain a better idea of the result-quality, it would be necessary to manually judge the relatedness of the top-ranked results.