Practitioners in the patent domain require high recall search solutions with precise results to be found in a large search space. Traditional search solutions focus on retrieving semantically similar documents, however we reason that the diferent topics in a patent document should be taken into account for search. In this paper we present PatentExplorer, an in-use system for patent search, which empowers users to explore diferent topics of semantically similar patents and refine the search by filtering by these topics. PatentExplorer uses similarity search to first retrieve patents for a list of patent IDs or given patent text and then ofers the ability to refine the search results by their diferent topics using topic models trained on the domains in which our users are active.
The ever-increasing volume [
A real time database system configured to store database content. . . such that the replicas of each partition are contained on diferent physical storage units. . . wherein the system provides an interface for user searches for documents types including video, audio. . . To provide an efective patent search tool under these conditions we present PatentExplorer, an in-use system for patent search, which empowers the users to explore diferent topics in search results and refine the results by their topics. PatentExplorer uses similarity search for first stage retrieval and domain-specific topic modelling for refinement of the search results. We propose topic modelling for search refinement because it is typical that a patent document will deal with multiple related but orthogonal subjects. For a particular information need, some but not all of these will be relevant. Therefore we combine a document level analysis (similarity) with a sub-document level analysis (topic models) for patent search. The intention is that the user can retrieve a large set of semantically related patents and inspect the topic distributions of the most similar ones. In order to refine the results the user can apply filters on specific topics, thereby increasing the task-specific relevance of the most highly ranked results.
This paper presents the design and user interface of the in-use web application which implements this idea as well as the technical description. The system has been designed with a particular user persona in mind. The intended user is a patent search professional, and therefore is familiar with patent search tools and also has deep knowledge of existing patent search methodologies, such as boolean retrieval and category filtering, as well as having broad technical knowledge of the relevant industrial domains. 2
In this section we give some background about related work on patent search tools, furthermore we introduce the methods for similarity search and topic models which we employ in PatentExplorer. 2.1
Patent search holds several domain-specific challenges for
information retrieval [
Methods 2.2.1 Similarity search. Similarity search is a method for retrieval where for a given query document, a ranked list of semantically relevant documents is computed, as shown in Figure 2. The general approach is to first embed the query document into a vector representation which encodes its semantics. This representation is then compared to the equivalent representations for each of the known documents in the search index. The results are then sorted by similarity score and the highest ranking results are presented to the user. The similarity function is usually cosine similarity.
The crucial step is to find an embedding which computes a
suitable document representation. Diferent representations have been
used in previous research, for instance tf-idf weighted sparse
representations, latent semantic indexing, or contextualised document
embeddings, for instance computed by a BERT model [
Despite the semantic richness of contextualised document
embeddings, sparse representations have been found to be competitive
in large scale retrieval scenarios [
We consider two algorithms for topic modelling in this work,
latent Dirichlet allocation (LDA) [
In this section we first show the user interface of PatentExplorer and give some implementation details about the architecture, the data and the similarity and topic models being employed in PatentExplorer. 3.1
The user interaction begins with the submission of a list of patent IDs (accession numbers) or the text of a patent, as shown in Figure 3. The system retrieves the text of the patents given in the list of patent IDs and creates a local copy of the text content of each of the patents. How many of the patents in the list are found in the index is indicated with "Dataset contains - documents". The user can then submit the "Dataset" to the system to retrieve similar documents based on the similarity search.
For each of the similar documents, the system also computes their topic distribution. The distribution is displayed along with the accession number and similarity score between the query patent and each similar patent, as shown in Figure 4. The most highly weighted words for each topic, drawn from the matrix , are displayed by hovering over the bars. The figure also shows the filter function which the system provides to re-rank the search results according to their topics. Both positive and negative filters can be applied. Positive filters lead to matching documents being lifted to the top of the ranked list, negative filters lead to the matching documents being discarded from the results set. For both filter types, a list of topics can be specified in the text field on the left hand side, as well as two slider values. The two slider values restrict when the iflter will match: a document matches if at least one of the chosen topics has a weight in the topic distribution of that document of at least “min probability”. The default value is 0.1. With a max rank of , the filter will also only match if the chosen topic is among the most highly weighted topics in the distribution for that document. So if the query document is the example in Figure 1, the user could inspect the topic distribution to find, for instance, the topic concerning physical storage, and apply a negative filter to remove it, leaving those results which have more to do with user search. Finally, when the user is finished applying filters to the search results, the results set can be downloaded as tabular data, preserving the filtered order and including similarity scores. 3.2
Technical implementation 3.2.1 Data. To prepare the components of our system we collected two overlapping data sets. The source is a commercially provided database of patent abstracts in which patents from patent ofices worldwide have been translated into a consistent, English-language form. We chose this data source in order to achieve maximum uniformity of the input data, however PatentExplorer makes no strong assumptions about the content of the documents, and would also work on publicly available patent data. The Our-Portfolio data set contains the patents whose assignee is our company or its subsidiaries. It contains 73k documents. We filtered this data set to only contain patents filed since 2010, resulting in a set of 36k documents. The All-Patents data set is the collection of all patents published between 2014 and 2020, which contains approximately 15 million documents. For both data sets we extract the title and abstract of the patents. 3.2.2 Architecture. The architecture of the system is shown in Figure 5. The two main components are the similarity search and the topic model. Each component ofers an API with one function: “get-similar-ids” and “get-topic-distribution”, respectively. The “getsimilar-ids” function receives one or more patent IDs and retrieves the most similar documents from the search index, defined as the cosine similarity between their representations. This is equivalent to finding the nearest neighbours of the query document in the representation space. The “get-topic-distribution” receives a single patent ID and computes the topic distribution for that document from the previously trained topic model. The search index and the topic model are static resources which are not changed during run time. Both components retrieve the patent document content from the database “Patent documents” directly as required, so that the user must only supply document IDs. 3.2.3 Training the topic model. The topic model is trained on the Our-Portfolio data set. The documents were preprocessed to remove approximately 50 patent-specific stop words, such as “invention” or “apparatus”, as well as usual English stop words. We performed stemming and then extracted all n-grams for = 1, 2, 3, 4 to construct the term-document matrix. We discarded words which occurred in fewer than 10 documents or in more than 40% of the documents.
In preliminary experiments we used a coherence metric to
investigate the optimal parameters for the topic model. In recent
years, several approaches to measure coherence have been
developed based on distributional properties of word pairs over a set of
words [
We use the coherence score as proposed by Aletras and
Stevenson [
To investigate which parametrisation of topic modelling works
best for patent text we took a sample of 513k English-language
patents from those published in 2010. We removed duplicates and
documents which were either very long or very short, leaving a set
of approximately 255k documents. As we show in Table 1, both LDA
and NMF exhibit similar performance on this data set, as measured
by , with NMF discovering marginally better topics. We find
upon manual inspection that NMF is more robust across a wide
range of number of topics. We therefore choose NMF to implement
the system. We finally use NMF with 75 topics to train the topic
model for the system on the Our-Portfolio data set.
3.2.4 Compiling the search index. To compile the search index we
must first compute an embedding for each document in the search
space. We use latent semantic indexing (LSI) to compute the
document vectors, which is the result of tf-idf vectorisation followed
by SVD compression [
To implement the lookup of documents given a query document we use Annoy1, a library which provides approximate nearest neighbour search. Each document embedding is normalised before insertion so that the cosine similarity can be computed with the dot product function. The similarity component of the system provides an endpoint which returns the IDs of the most similar documents for some query document and some .
In this paper we present PatentExplorer, an in-use system for patent search. PatentExplorer gives users the ability to retrieve similar patents given a list of patent IDs or the patent text and refine their search results depending on the diferent topics of the patents. The topic models are tailored to the domain-specific topics of a company operating in the technical domain.
Tailoring the search representation and topic models to our domains turned out in initial user testing to ofer mixed results. Feedback from patent search experts indicates that while the system can deliver relevant results within our domains, outside of these domains it can return results with few or no relevant documents among the ten highest ranked results. While building and testing our system we have found that the requirements of patent search use cases place high demands on the accuracy of dedicated search tools. In order to reduce the latency of the similarity search to an acceptable level we were forced to simplify the similarity computation, using a compressed tf-idf representation where a contextualised document embedding may well have produced better results. It is also crucial to provide full coverage: The dataset of patents which the system contains goes back to 2014, however for prior art searches, all previously published patents should be discoverable. Finally the need to update the search index continuously leads to considerable recurring computational load and data management tasks—this is not yet provided for.
Our future work to improve the system will include expanding the system architecture to eficiently handle a larger number of documents in the search space. In the longer term we intend to investigate introducing more appropriate document representations to be used in the search index, for instance by using a large language model such as BERT, or by learning the representations via a supervised auxiliary task.