Our group participated in the CLEF-IP 2013 Passage Retrieval starting from Claims task. We focus on formulating representative queries from various metadata that is embedded in a patent document. We then submit the queries to a state-of-the-art search engine to perform document level retrieval. For passage level retrieval, we implement a TF-IDF algorithm that calculates the sum of query keywords' TFIDF scores. We submitted six runs, which tested di erent uses of the metadata and di erent retrieval algorithms. We nd that carefully constructed structured queries from titles and terms with mid-range IDF values are e ective for patent prior art retrieval.
A patent is a set of legal documents authorized by a government's patent o ce. It is used to grant exclusive rights for exploitation of the invention for a span of time, usually 20 years. A Patent Application document is written by the patent applicant to describe the background and the description of the invention, and to declare a set of claims. The claims are usually drafted with the help of a patent attorney, and are used to specify what exactly the patent should protect.
The novelty search, also called Prior Art Search, is the procedure that patent
examiners search for existing patent documents, which are called prior arts, to
prove that the all or part of the claims in a newly led patent application
document are not novel and hence can be rejected. CLEF-IP 2013 Passage Retrieval
starting from Claims task exactly captures the procedure of the novelty search.
Given one or a few claims, the participants are asked to retrieve relevant patent
documents in the collection and mark out the relevant passages. The following
is an example of a CLEF-IP query:
<t le>EP-1752179-A2.xml</t le>
<title>Needle guard clip with stylus</title>
<abstract>A needle guard (10) includes a clip (12) with a canting wall (16) to grip
the needle shaft (56) and a distal wall (22) to block the tip (58) thereof, wherein
the canting and distal walls may be interconnected by an angled strut (24,26). . .
</abstract>
<tclaims>/patent-document/claims/claim[
A safety catheter device comprising a catheter hub and a catheter tube extending therefrom, a needle having a needle shaft terminating in a sharp tip, . . . </claim> <claim num="8">
A needle protector device comprising a housing adapted to slidably receive a needle therethrough, a clip positioned in the housing and having a rst wall with an aperture adapted to slidably receive a needle shaft of the needle. . . </claim>
In the example, the participants are asked to search the given patent corpus and to retrieve prior arts for the patent le EP-1752179-A2.xml with relevant passages being marked out. These retrieved documents and passages should be evidence to help patent examiners reject the 1st and the 8th claims in the patent application.
The data collection [
Dataset Preparation and Document Indexing
We adopt the Lemur Search Engine1 to build index and retrieve the patent
documents. Speci cally, we use Lemur to build inverted index for each word
in the CLEF-IP collection except the stopwords. We stem the terms using the
Krovetz stemmer [
The Lemur Search Engine implements many retrieval algorithms, including
the vector space model, Language Modeling, and Okapi BM25 [
The language modeling with Dirichlet smoothing can be shown as in the following scoring formula:
tft;d + P (tjMC )
P (tjd) = Pt02V tft0;d + where tft;d means term t's term frequency in document d, MC is the corpus model, V is the Vocabulary. In order to get the best value for parameter , we xed the input query keywords, and switched between f3000, 3500, 3700, 3800, 3900, 3950, 4000, 4050, 4100, 4200, 4300, 4500, 5000, 5500g. Our experiments shows that 4050 is the best value.
Okapi BM25 follows the scoring formula as below:
X (log N t2Q
dft + 0:5 dft + 0:5 ) (k1 + 1)tft
(k3 + 1)qtft k1((1
b) + b avdgocdolcenlen ) + tft k3 + qtft where Q is the query keywords set, N is the number of documents in the corpus, dft is term t's document frequency, tft is term t's term frequency in document d, doc len is the length of document d as the number of terms, avg doc len is the average length of a document, qtft is term t's term frequency in query set Q. There are 3 parameters (k1, b ,k3) in Okapi's scoring formula. In order to evaluate the best value set for (k1, b ,k3), we xed b=0.75, k3=7, and varied k1 through 1.0 to 10. We perform parameter tuning to b and k3. The experiments show that the best value set for (k1, b ,k3) is (8.0, 0.85, 1000). 3
In this section, we present several approaches to formulate queries from patent documents.
(1) (2) Extracting claim texts We directly extract task claims out of patent documents and use these claims as query keywords to retrieve documents. For patent EP-1752179-A2, we get query:
A safety catheter device comprising a catheter hub and a catheter tube extending therefrom, a needle having a needle shaft terminating in a sharp tip, . . . A needle protector device comprising a housing adapted to slidably receive a needle therethrough, a clip positioned in the housing and having a rst wall with an aperture adapted to slidably receive a needle shaft of the needle. . .
Extracting hyphenating phrases We believe that hyphenating phrases like \water- bed' are usually representative words, hence we propose to form queries by extracting hyphenating phrases from the claims.
Extracting titles We believe that a document's title usually summarizes a document well. In this approach, we use patent title to generate queries. Some patent application documents may have multiple titles; each title is written in English, German, or French. We refer the title written in English as the English Title, and the title which shares the consistent language with the patent application document as the Consistent Title. For example, patent EP-0195350-A2 is written in German and has three titles:
The rst title is called the English Title and the second title is called the Consistent Title since it matches with the language document of the application. For a patent document written in English like EP-1752179- A2 (see Section 1), its English Title is also the Consistent Title.
Through experiments on last year's data, we nd out that if we retrieve results using the Language Modeling approach, we get better results when we use the Consistent Title; on the other hand, if we retrieve results using the Okapi BM25 Model, we get better results when we add both the Consistent Title and the English Title into the query keywords.
IDF ltering Each of the previous three approaches alone or their combinations
can provide us with a set of query keywords. Even though we remove stopwords
out of the query keywords, there are still many terms like \water" which are not
stopwords but are also common in the document collection; hence they are not
representative. In this approach, we propose that using IDF (inverse document
frequency) [
We propose a two layer ltering strategy. We believe that hyphenating phrases and terms extracted from title are better words than words extracted from claims. Based on this assumption, we split query keywords into two sets. One set is called the Standard Query, it contains terms extracted from claims. The other set is called the Re ned Query, it contains hyphenating phrases and terms extracted from titles. We conduct a stricter ltering strategy on the Standard Query and a looser one on the Re ned Query. That is when we lter the Standard Query, we set the IDF's lower bound to be 0.7 and the upper bound to be 3.2, while when we lter the Re ned Query, we set the best IDF lower bound to be 0.65 and the upper bound to be 3.2. These values are decided by a series of experiments based on CLEF-IP 2012 Passage Retrieval starting from Claims training tasks and testing tasks.
POS tagging Another way to lter query keywords is using POS [
The runs that we submitted to this year's CLEF-IP Prior Art Retrieval Task used a combination of some or all of the approaches that we list in Section 3. These runs are: 1. Run ID: OnlyClaimLM
- extract the task claims to be query keywords 2. Run ID: coOnlyTtlLM
- use patent application's Consistent Title as query keywords 3. Run ID: HypCoTtlNoIdfUpperBoundLM - extract claims, hyphenating phrases and the Consistent Title to form query keyword set - lter the Original Query with IDF lower bound 0.7 and the Re ned
Query with IDF lower bound 0.65, no IDF upper bound is set - POS tagging query keywords and only leave nouns, adjectives and verbs 4. Run ID: HypCoTtlWithIdfUpperBoundLM - the same as <HypCoTtlNoIdfUpperBoundLM>, but also lter the
Standard Query and the Re ned Query using IDF upper bound 3.2 5. Run ID: HypDuTtlNoIdfUpperBoundBM - the same as <HypCoTtlNoIdfUpperBoundLM>, but di erent in 2 ways: add not only the Consistent Title but also the English title into the Re ned Query use Okapi Model instead of Language Model 6. Run ID: HypDuTtlWithIdfUpperBoundBM - the same as <HypDuTtlNoIdfUpperBoundBM>, but also lter the
Original Query and the Re ned Query using IDF upper bound 3.2
Each patent document has a kind code. The code can be A1, A2, A3, . . . or B1, B2, . . . The A* kind codes means that patent documents are published during the patent application phase, while the B* kind codes means that patent documents are published during the granting phase. In our runs, we ltered the retrieval list to make sure that only A1, A2 and A* type patent documents will show in the nal result list.
The top run in the English sub task comes from our submission. Table 4 lists the o cial evaluation results from CLEF-IP 2013. The results show that our approaches of generating highly representative queries are e ective. The results also show that the Okapi BM25 retrieval model outperforms the Language Modeling retrieval model in the Novelty Search Task.
Our experiments show that in the Novelty Search Task, using long queries helps to nd more relevant patent documents than using short queries, e.g. only using document titles. Hence although titles and hyphenating phrases are good resources, we also added claim texts as one resource to extract quali ed query keywords. But very long queries always contain noise, in our best run, our strategy to balance this is to control the query keywords in less than 20 words. Speci cally, after ltering quali ed terms from titles, hyphenating phrases and claims texts by using IDF and POS tagging, if the query keywords coming from titles and hyphenating phrases are less than 20 words, we ranked query keywords from claims texts by IDF score and added the top ranked terms into the query keywords set until it contains 20 words.
All the approaches we talked above are about document level retrieval. At passage retrieval level, in all runs, we used the sum of query keywords' T F IDF score to rank passages, where TF is the query keyword's term frequency in a passage and IDF is its corpus inverse document frequency. Only the top 10 ranked passages will be returned. Our approaches have good passage MAP and Precision scores which proves that our approach is e ective.
In summary, our approach is highly e ective in nding Patent Prior Arts written in English, as well as e ective in nding Patent Prior Arts written in German or French. 5
CLEF-IP 2013 Passage Retrieval starting from Claims task precisely captures the procedure of the Prior Art Search. Participants are given one or a few claims, and are asked to retrieve relevant patent documents in the collection and mark out the relevant passages.
In this paper we present an integrated process of Prior Art Search.We focus on formulating representative queries from various metadata that is embedded in a patent document. We then submit the queries to the Lemur search engine to perform document level retrieval. We mainly used two retrieval algorithms, Language Modeling and Okapi BM25. We tuned the parameters for CLEF-IP 2012 dataset. We believe that the parameter setting we list in Section 2 are also suitable for other Patent collections. Moreover, in the paper, we present six approaches to formulate queries from patent documents. The experimental results from CLEF-IP 2012 and 2013 both support that our approaches are e ective to identify representative query keywords.