=Paper= {{Paper |id=Vol-2957/paper7 |storemode=property |title=Old is Not Always Gold: Early Identification of Milestone Patents Employing Network Flow Metrics |pdfUrl=https://ceur-ws.org/Vol-2957/paper7.pdf |volume=Vol-2957 |authors=Manajit Chakraborty,Fabio Crestani |dblpUrl=https://dblp.org/rec/conf/swisstext/ChakrabortyC21 }} ==Old is Not Always Gold: Early Identification of Milestone Patents Employing Network Flow Metrics== https://ceur-ws.org/Vol-2957/paper7.pdf

Old is Not Always Gold: Early Identification of Milestone Patents
Employing Network Flow Metrics

Manajit Chakraborty Fabio Crestani
Faculty of Informatics Faculty of Informatics
Università della Svizzera italiana Università della Svizzera italiana
Lugano, Switzerland Lugano, Switzerland
chakrm@usi.ch fabio.crestani@usi.ch

Abstract often varies depending on the nature of the tech-
nology. Previous research has already endorsed
Bibliometrics has been employed previ- technological forecasting1 as an integral element
ously with patents for technological fore- to stay ahead of the curve for corporations and
casting. The primary challenge that tech- governments (Campbell, 1983). Acs et al. (2002)
nological forecasting faces is early-stage suggested that patents provide a fairly reliable mea-
identification of technologies with the po- sure of innovative activity. Identifying important
tential to have a significant impact on the patents, observing their change of importance as
socio-economic landscape. With this aim, captured by the variation of citation measures and
we carry out an exploratory study using analyzing them can lend us new insights as to how
various network-based metrics on patent innovation evolves over a period of time. This
citation network to identify patents which could be beneficial for innovators and companies
are possible candidates for major influence who are actively involved in producing patents. It
in the immediate future. To effectively would facilitate them to take stock of the innova-
uncover these patents shortly after they tion quotient of a particular technological area and
are issued, we need to go beyond raw ci- help measure its growth and potential over a certain
tation counts and take into account both period of time.
the citation network topology and tempo-
In this paper, we aim to identify influential
ral information. We posit that, as with
patents from different technological areas from
scholarly citations, not all patent citations
patent citation network using network flow algo-
carry equal importance with age. This in-
rithms. Identifying top patents from any particular
formation is captured by dynamic network
category can help companies interested in patenting
flow metrics that take the effect of time
to glean an overview of the important innovations
on the citations into account. Identifying
in their field of concern. It can also benefit govern-
top patents can aid in re-ranking of search
ments in deciding various policies such as funding
results in a patent search. We carry out our
to technological areas that have shown promise
experiments on two standard collections
over the last few years. We argue that while cita-
of patents and present some insightful re-
tion count may help us identify important patents,
sults and observations based on rigorous
it tends to favour patents that have been filed or
analysis.
granted long ago, thus providing it a longer citation
1 Introduction accumulation period. While PageRank helps to
mitigate the situation to a certain extent by consid-
Patent citations, namely references to prior patent ering the whole network instead of simple citation
documents and the state-of-the-art included therein, count, PageRank too has been known to be biased
and their frequency are also often used as indica- against recent network nodes. CiteRank (Walker
tors for the technological and commercial value et al., 2007) introduces exponential penalization of
of a patent and to identify “key” patents, which old nodes, thus modelling the node score such that
it captures the future citation count gain. However,
Copyright © 2021 for this paper by its authors. Use permitted
under Creative Commons License Attribution 4.0 Interna-
1
tional (CC BY 4.0) https://hbr.org/1967/03/technological-forecasting
due to CiteRank’s known limitations, we propose a tions reflect the fact that either a new technology
new model called Time-Attentive Ranking, which builds on an existing one or that they serve a sim-
helps to capture the temporal changes and their ilar purpose. As a result, chains of citations al-
effect on certain nodes. We carry out our experi- low us to trace the technological evolution, and
ments on two different datasets to determine the hence patent centrality in the citation network can
efficacy and effectiveness of our method against be used to score patents. In our preliminary citation
baselines both qualitatively and quantitatively. We analysis, we have adopted a couple of PageRank
then carry out a comparison of the top-N ranked list based approaches along with other citation metrics.
of patents provided by three algorithms using Rank PageRank (Bedau et al., 2011; Bruck et al., 2016)
Biased Overlap (Webber et al., 2010) and against and similar eigenvector-based metrics (Doira and
a list of significant patents by Strumsky and Lobo Banerjee, 2015) has been computed on patent cita-
(2015), to point out the relative changes. We posit tion networks earlier. Mariani et al. (2016) argued
that top-ranked patents or the ranking criteria for on similar lines in the case of scholarly articles and
the same could be employed for a ranking based proposed a re-scaled version of PageRank that dis-
patent retrieval method as have been exploited by counts citations for old papers based on age. We
Xue and Croft (2009) and Liao and Veeramacha- build upon this notion and perform a thorough anal-
neni (2010). Our experiments are two-pronged – ysis of patent citation network in sub-categories
first, we study the effect of the network metrics on and sectors and in the presence (or absence) of
European patents from the MAREC dataset and patent content by employing a proposed network
secondly, we employ an adapted version of a deep flow algorithm.
learning model that infuses both textual content
and network flow metrics on USPTO patents in 3 Methodology
order to spot influential patents and validate our We employ three different network-based patent-
hypothesis. level metrics for comparison: PageRank scores P ,
CiteRank score C and our proposed Time-Attentive
2 Related Works Rank score T .
The notion of quantitative evaluation of scientific 3.1 PageRank
and technological impact builds on the basic idea PageRank (Brin and Page, 1998) is a link analysis
that the scientific merits of papers (Radicchi et al., algorithm and it assigns a numerical weighting to
2008), scholars (Egghe, 2006), journals (Bollen each element of a hyperlinked set of documents,
et al., 2006), universities (Molinari and Molinari, such as the World Wide Web, with the purpose of
2008) and countries (Cimini et al., 2014) can be “measuring” its relative importance within the set.
gauged by metrics based on the received citations. The algorithm may be applied to any collection of
Bibliometrics has been employed in a variety of entities with reciprocal quotations and references.
scenarios to measure and analyze citations since The numerical weight that it assigns to any given el-
they provide a rich source of information. Sci- ement E is referred to as the PageRank of E. PageR-
entific papers and scholarly articles have been in- ank normalizes the number of links on a document
vestigated using various bibliometric tools, espe- by not counting each of them as equal. PageRank
cially citations for a long period (Narin et al., 1976; can be defined as follows (Equation 1):
Bakkalbasi et al., 2006). One of the early studies to
measure the technological impact based on patent X pnj X pnj 1 − α
citations was done by Karki (1997). He proposed a Pin+1 = α. Aij +α. +
kjout N N
host of technological indicators based on citations j:kjout >0 out
j:kj =0
among patents. (1)
where kjo ut =
P
Carpenter et al. (1981) and Fontana et al. (2013) l Alj is the outdegree of node
compared patents associated with inventions that j, α is the teleportation parameter, and n is the
received a prize and patents from a control group, iteration number. The PageRank score Pi of node i
finding again evidence that “important” patents are can be interpreted as the average fraction of time
more cited (the mean number of citations received spent on node i by a random walker who with
was found to be about 50% higher for important probability α follows the network’s links and with
patents). As argued by (Jaffe et al., 2000), cita- probability 1 − α teleports to a random node. We
consider α = 0.5 throughout this paper since it is
the accepted choice for citation networks (Chen 
N −ni , if p cites p and
et al., 2007). γ
 i j
Rn,γ (i, j) = t(pi ) = tni ≤ tn (4)
3.2 CiteRank

0, otherwise


CiteRank (Walker et al., 2007) was designed specif-
where γ < 1 is the retention probability given to
ically for ranking papers in a citation network. Cit-
attach more weight to a recent patent and decrease
eRank performs a random walk on an aggregated
the weight as the patent keeps ageing. The con-
citation graph but initiates the walk from a recent
tagion matrix can then be written as Equation 5
paper chosen with the probability that depends on
(using Equation 3 and 4):
its age. Authors estimated parameters of the ran-
dom walk by fitting papers’ CiteRank score to the N −1
number of citations accrued by papers over some
X
EM N,α,γ = αi R i N (5)
time period. Let us suppose M is a transfer ma- i=1
trix with elements Mij = 1/Lj if paper j cites i
and 0 otherwise. The probability that a researcher and hence the score of a patent pj at the end of
follows the citation links to encounter a paper is aPtime period [ti , tN ] is given by EM N (j) =
defined as in Equation 2: i EM N (i, j). For our experiments we consider
the best possible settings by empirically setting
α = 0.1 and γ = 0.3. Elsewhere in the paper we
~ = I0 .~
C ρ +(1−α)2 M 2 .~
ρ +(1−α)M.~ ρ +... (2) refer to the EM score as T for uniformity and ease
of comprehension.
where I0 is an identity matrix, ρi =exp−agei /τ is
the probability of initially selecting paper i, agei 4 Experimental Setup
is the age of the paper and τ is the characteristic
4.1 Datasets
decay time. In this paper, we consider α = 0.5 and
τ = 2.6 years, as specified by Walker et al. (2007). For this study, we used two datasets: (1) European
Patent (EP) collection from the MAtrixware RE-
3.3 Time-Attentive Rank search Collection (MAREC)2 and (2) US patents
dataset collected by Kogan et al. (2017) that spans
Our proposed model Time-Attentive Ranking is
the period between 01-01-1926 and 11-02-2010.
based on the notion that ‘An inventor or patentee
To the best of our knowledge, there exists no study
can find patents by following citations links back
of a similar kind on the European Patents, which is
in time from a particular patent’. The number of
why we chose to work with the former collection
paths that can be attenuated between patent pi and
from MAREC. However, this presents a unique
pj can be expressed as a contagion matrix M given
challenge of finding a respective gold-standard
by Equation 3:
list of “milestone” patents, which is not available.
Hence for this collection, we resort to a qualitative
N −1
X evaluation as described in later sections. To com-
MN,α = αi Ai N (3) pare our proposed approach’s performance against
i=1
the state-of-the-art and perform a quantitative eval-
where An is the adjacency matrix of patents cit- uation, we repeated our experiments on the USPTO
ing each other for a particular year tn and α is the dataset as well. Additionally, we also performed
probability of following a citation link. The more validation of our model’s performance by a deep
paths there are from patent pi to pj , the higher learning technique as suggested in Chung and Sohn
the likelihood that an inventor will find pj by fol- (2020) to identify a patent’s grade in determining
lowing citation chains from pi , which is similar to in value.
α-Centrality (Bonacich, 1987) and Katz centrality 4.2 Preprocessing
(Katz, 1953) metrics. Since the existing contagion
matrix does not account for time and hence weights MAREC collection We only considered granted
each edge equally, the authors propose a retained patents from the ‘EP’ collection. For uniformity,
2
adjacency matrix which is given by Equation 4: http://www.ifs.tuwien.ac.at/imp/marec.shtml
we removed patents that had some metadata miss- and Time-Attentive Rank score T . As mentioned
ing, such as classification codes or patent citations. earlier, each patent is endorsed with several classi-
We also did not consider the non-patent citations fication codes that classify them into sectors, cat-
since they are out of the scope of our study. We egories, sub-categories etc. The highest level of
pre-processed the data to only keep the citations classification is according to sectors (A-H). Each
between patents that were issued within 1976-2008, sector consists of several categories (A61K, A61P,
removing thereby the citations to patents issued be- ...), each category consists of several sub-categories
fore 05-1976. Hence, we were left with a network and so on. A single patent can belong to several
consisting of only EP-EP patent citations formed sectors and categories.
out of 251,664 patents having 350,164 citations.
Table 1: Top-5 Patents by citation count
USPTO collection Unlike the well-known
NBER patent data, the dataset provided by Kogan PatentID No. of Citations
et al. (2017) has a vastly improved coverage. We EP0037691 125
EP0272189 121
pre-processed the data to only keep the citations EP1049021 121
between patents that were issued within the time EP0364618 121
period of 01-01-1926 and 11-02-2010, removing EP0527247 121
thereby the citations to patents issued before
01-01-1926. The resulting citation network 5.1.1 Complete Network
analyzed in this paper is composed of 6,237,625
From Table 2, we can observe significant changes
patents and 45,962,301 citations between them.
in the ranking of the patents. The tables reveal that
5 Results and Analysis there are more recent patents granted after the year
2000 in the top-10 list ranked by Time-Attentive
5.1 MAREC Patents Ranking than that produced by either PageRank
In this section, we present a qualitative of the re- or CiteRank. To be precise, the Time-Attentive
sults followed by a comprehensive analysis of the ranking method includes five patents granted after
same. We ran the experiments on networks that the year 2000 in the top-10 list, while for PageRank
were spliced in time (yearly) over a ten year period and CiteRank, it is four out of ten for both. Of
(1998-2008). The same set of experiments were course, the difference is even more pronounced as
carried out on networks comprising of all patents, we go deeper in the lists, say, top-15, top-20 and so
patents belonging to a certain sector and patents on, which we could not present here due to space
belonging to a certain category. On studying the in- constraints.
degree and out-degree of patents, we observed that For comparison, the list of top five patents on
the degrees are very skewed, i.e., only a handful of the basis of citation count is presented in Table
patents gets a large number of citations while most 1. One can observe that none of the three metrics
of the patents in the network have less than ten cita- ranks the patents from Table 1 in their top five
tions. Hence, this network follows a similar pattern list. In fact, within the top fifteen results, only
to that of a scholarly article citation network (al- patent EP0272189 and EP0364618 feature in the
beit with more skewness). Hence, the network-flow lists compiled according to PageRank and CiteR-
algorithms that can be employed with paper cita- ank scores, while patent EP0527247 is listed by
tion networks can also be adapted here. Moreover, PageRank only. The rest do not find a place in
there is a strong correlation between the in-degree the top-10 of any network-based score list. This
and out-degree of the patents in both collections, corroborates our initial hypothesis that is simply
which implies that highly-cited patents tend to be acquiring a high citation count does not indicate
cited by other highly-cited patents and to cite other the importance of a patent.
highly-cited patents (Ren et al., 2018).
5.1.2 Network for a particular sector
Qualitative Comparison of Top Patents : For Next, we perform the same set of experiments over
an intuitive understanding of how the different individual sectors of patents. Similar to the trend
network-flow metric scores affect the rank, it is shown by the complete network, for sector B which
important to observe the top-ranked patents accord- has the highest number of patents, we observe from
ing to the PageRank score P , CiteRank score C Table 3 that Time-Attentive Ranking features the
Table 2: Top-10 patents for 2008 ranked by scores
Rank PatID Title Date #Citations

1 EP0251752 Aluminum-stabilized ceria catalyst compositions and method of making 29-06-1987 111
same.
2 EP1304455 Particulate filter for purifying exhaust gases of internal combustion en- 17-10-2002 103
gines
3 EP0728435 Cyclone dust extractor 20-02-1996 87
PageRank score P

4 EP1031939 COMPOSITE IC CARD 16-11-1998 69
5 EP1261147 A method and system for simultaneous bi-directional wireless commu- 21-05-2001 78
nication between a user station and first and second base stations
6 EP0776864 Process for the aerobic biological purification of water 10-07-1996 34
7 EP1466940 Carbon fiber composite material and process for producing the same 13-04-2004 57
8 EP1400858 PHOTORESIST STRIPPER COMPOSITION 21-06-2002 28
9 EP1059092 Use of complexes among cationic liposomes and polydeoxyribonu- 08-06-1999 19
cleotides as medicaments
10 EP0534904 Imidazolylmethyl-pyridines. 21-09-1992 23

gines
3 EP0728435 Cyclone dust extractor 20-02-1996 87
4 EP1031939 COMPOSITE IC CARD 16-11-1998 69
5 EP1835243 Evaporator with electronic circuit printed on a first side plate 26-02-2007 21
6 EP1261147 A method and system for simultaneous bi-directional wireless commu- 21-05-2001 78
nication between a user station and first and second base stations
7 EP1466940 Carbon fiber composite material and process for producing the same 13-04-2004 57
8 EP0364618 Multiple signal transmission device. 18-10-1988 121
9 EP0776864 Process for the aerobic biological purification of water 10-07-1996 57
10 EP1400858 PHOTORESIST STRIPPER COMPOSITION 21-06-2002 28

Table 3: Sector B patents of 2008 ranked Table 4: Category A61K patents of 2008 ranked

Rank Patent ID Date Rank Patent ID Date
1 EP0728435 20-02-1996 1 EP0776864 10-07-1996
PageRank P

PageRank P

2 EP0008860 20-07-1979 2 EP0728435 20-02-1996
3 EP0095603 07-05-1983 3 EP0071564 19-07-1982
4 EP1142619 26-09-2000 4 EP0002210 17-11-1978
5 EP0466535 18-06-1991 5 EP0447285 27-02-1991

1 EP0728435 20-02-1996 1 EP0776864 10-07-1996
CiteRank C

CiteRank C

2 EP1304455 17-10-2002 2 EP0728435 20-02-1996
3 EP1142619 26-09-2000 3 EP1835243 26-02-2007
4 EP0534904 21-09-1992 4 EP1568666 22-02-2005
5 EP1731327 10-06-2005 5 EP0770375 13-09-1996

1 EP0728435 20-02-1996 1 EP0776864 10-07-1996
TimeAttentiveRank T

TimeAttentiveRank T

2 EP1329412 10-10-2000 2 EP0527247 08-08-1991
3 EP1489033 05-06-2004 3 EP0364618 18-10-1988
4 EP1306147 23-10-2002 4 EP0272189 17-12-1987
5 EP1674419 21-12-2005 5 EP0728435 20-02-1996

more recent patents in their top five as compared 5.1.3 Network for a particular category
to their counterparts.
While it is interesting to study the complete net-
work and find the most influential patents as identi-
fied by Time-Attentive Rank, it does not deliver us
Table 7: RBO among the full ranked
Table 5: RBO@20 for 2008 Table 6: RBO@20 for A61K
lists
P C T P C T
P C T
P – P –
P –
C 0.4981 – C 0.3430 –
C 0.8548 –
T 0.3921 0.5741 – T 0.2568 0.3963 –
T 0.6307 0.7270 –

a lot of information. On the other hand if we limit ment can be defined as the proportion of S and T
the patent citation network by categories, it could that are overlapped at depth d. Rank-biased Over-
provide us some insights as to which technologies lap falls in the range [0, 1], where 0 means disjoint,
have been gaining momentum in the last few years and 1 means identical. While RBO is the agree-
of the patent data. The total number of categories in ment score between two indefinite lists, we are
the patent database exceeds hundred. Not surpris- more concerned with the top-k elements in the lists
ingly, the distribution of patents against categories and hence RBO@k provides us a better measure to
is also skewed. For brevity, we present only the compare the top-ranked elements. It is imperative
results for the most popular category A61K. to note that RBO > RBO@k. For our case, we
From Table 4, we observe a certain peculiar- empirically consider k = 20 and p = 0.9.
ity. None of the top five patents ranked by the Tables 5, 6 and 7 present the RBO confusion
Time-Attentive Ranking mechanism is a post-2000 matrix. We can clearly observe a pattern here.
patent. This is interesting because it implies that The overlap between CiteRank and TimeAttentive
while Time-Attentive rank gives more weightage ranked lists are certainly more than the overlap
to recent citations, it does not bias towards recent (agreement) between PageRank and TimeAtten-
patents, thus maintaining a balance between older tive Rank, which confirms our intuition that recent
and newer patents. So, the top-ranked patent in all patents receive more preference in the weighted
three cases is the same indicating that EP0776864 citation measures rather than unweighted citations
is indeed the most important patent in category of PageRank.
A61K.
5.2 USPTO Patents
Metric for comparison of ranked lists: Since our
For this collection, we adopt a different approach
hypothesis hinges on the ranking of patents over
for carrying out our experiments. The experiments
a network metric based score, it is imperative that
on the MAREC patents were solely based on net-
the lists generated by PageRank and CiteRank and
work flow metrics, which we could not assess
TimeAttentiveRank will be different in their order-
quantitatively due to the lack of a standard base-
ing of elements (ranks). As the lists are quite long,
line. Instead, for the US patents collections, we
their scores are not directly comparable, and for a
compare our approach against the state-of-the-art
given depth d the two lists may not even have the
Re-scaled PageRank method proposed by Mari-
same set of elements, we will have to resort to in-
ani et al. (2019) to identify milestone patents. As
definite ranking (Webber et al., 2010). To this end,
a second objective, we wanted to determine the
we employ rank-biased overlap (RBO) to measure
value added by textual content in determining a
the similarity and agreement between the two lists.
patent’s worth. This objective stems from similar
The RBO values for the year 2008 compared over
studies on patents where it was shown that exploit-
the complete list of ranked results is presented in
ing the multimodal nature of patents yields better
Table 7. The Rank-Biased Overlap is defined as in
prediction performance (Chakraborty et al., 2020).
Equation 6.
For this purpose, we adapt the deep learning ap-
∞
X proach proposed by Chung and Sohn (2020). Due
RBO(S, T, p) = (1 − p) pd−1 .Ad (6)
to the incompatibility of NLP based approach pro-
d=1
posed by Chung and Sohn (2020) and network flow
where S and T are two indefinite ranked lists. p metrics-based approaches such as the one by Mari-
stands for user’s persistence, which determines ani et al. (2019) and ourselves in this paper, we only
how steep is the decline in weights: the smaller p, adopt the deep learning approach (DEP-net) to de-
the more top-weighted is the metric. Ad , agreee- termine a patent’s grade, which is another measure
of patent’s importance. As per Chung and Sohn to uncover these significant patents. The complete
(2020), a patent’s quality is assigned one of three list of these patents can be found in Appendix C
grades (A, B, or C) based on the average number of (Strumsky and Lobo, 2015). Presence in the
of forward citations per year. The deep learning list of significant patents by Strumsky and Lobo
approach is briefly summarised below: is a binary variable: a patent is either in the list
or not. We can therefore study the ability of the
• A patent grade (A, B, or C) is assigned based metrics to rank these outstanding patents as high as
on a threshold determined by the average for- possible, in agreement with the objectives of this
ward citations accrued per year by the patent. paper. While there are 175 significant patents in
• Textual content (abstract and claim) from the the Strumsky-Lobo list, we restrict our analysis to
patent data is extracted along with several those patents that were issued within our dataset’s
other indices such as number of claims, num- temporal span and remove those that are absent in
ber of inventors, number of backward cita- our dataset. This leaves us with M = 112 significant
tions, number of IPCs, etc. patents.

• Abstract and claims are transformed (vector- 5.2.2 Comparison against baselines
ized) into word embeddings as matrices. In this section we inspect the top-ranked patents.
For simplicity, we focus on the top-10 patents as
• A deep neural network composed of Bi-LSTM ranked by PageRank P and Re-scaled PageRank
layer is added to the CNN structure using mul- R and our Time-AttentiveRank T scores (Table
tiple filters that fuses the four components (ab- 8). From Table 8, we can observe that the top-10
stract, claims, indices, network-metric score) patents by Re-scaled PageRank span a wider tem-
as input to train and evaluate a patent’s quality. poral range (1942–2010) than the top-10 by PageR-
Finally, we also evaluate the patent quality for ank (1942–1996), which is a direct consequence of
test data. the age-bias removal. The same temporal span is
It is to be noted that we add an extra component to retained by the Time-Attentive Rank as well. How-
the original model proposed by Chung and Sohn ever, it is noteworthy that our proposed method can
(2020), i.e., the network-metric score. Both the re- pick more (3) patents from Strumsky-Lobo’s list
scaled PageRank score (R) and the Time-Attentive of significant patents. Among the ten top-ranked
Rank score are fed separately as inputs to the deep patents, two are from 2010 (the last year in the
neural model. To simplify things, we retained the dataset) and received only one citation. This hap-
parametric setting of the neural model as proposed pens because only a few among the most recent
by Chung and Sohn (2020). Finally, the features patents received citations, which results in tempo-
from the abstracts, claims, indices and network- ral windows with a large fraction of patents with
flow metrics are fused and used as inputs to the zero citations. Thus, within such a temporal win-
fully connected layer. The loss function was cross- dow, a patent can achieve large T score thanks to
entropy, and the activation function was softmax. one single citation. A possible solution for this
We label this model as DEP-netPlus (as we add issue is to only include the patents whose temporal
value to the DEP-net model). windows contain a certain minimal number of in-
coming citations. Another observation is that both
5.2.1 Expert-selected historically significant the Re-scaled PageRank and Time-Attentive rank
patents do not necessarily rank patent with grade A in a
Strumsky and Lobo (2015) listed 175 patents care- higher position, so the ranking is not solely depen-
fully selected “on the basis of consultation with dent on the citation count but also on the network
engineers, scientists, historians, textbooks, maga- structure.
zine articles, and internet searches”. The patents in
the list “all started technological pathways which 5.2.3 Performance comparison against
affected society, individuals and the economy in DEP-net
a historically significant manner” (Strumsky and To illustrate the importance of including network-
Lobo, 2015). These significant patents thus pro- flow based metric as a component, we performed
vide a good “ground-truth” set of patents that can the patent grade classification as described in
be used to discern the ability of different metrics (Chung and Sohn, 2020). We used the same dataset
Table 8: Top-10 patents ranked by Network-metric scores. Asterisks mark the Strumsky-Lobo significant patents.
Rank PatID Title Date #Citations Grade

1 4683195 Process for amplifying, detecting, and/or-cloning nucleic acid se- 28-07-1987 1956 A
quences
2 4683202 Process for amplifying nucleic acid sequences 28-07-1987 2169 A
3 4237224 (*) Process for producing biologically functional molecular chimeras 02-12-1980 285 B
4 4395486 Method for the direct analysis of sickle cell anemia 26-07-1983 71 B
PageRank score P

5 4723129 Bubble jet recording method and apparatus in which a heating element 02-02-1988 1962 A
generates bubbles in a liquid flow path to project droplets
6 3813316 Microorganisms having multiple compatible degradative energy- 28-05-1974 16 C
generating plasmids and preparation thereof
7 5536637 Method of screening for cDNA encoding novel secreted mammalian 16-06-1996 422 A
proteins in yeast
8 4558413 Software version management system 10-12-1985 1956 A
9 4358535 Specific DNA probes in diagnostic microbiology 09-11-1982 436 A
10 2297691 SElectrophotography 06-10-1942 588 B

1 7764447 Optical element holding device, lens barrel, exposing device, and device 27-7-2010 1 C
producing method
2 4237224 (*) Process for producing biologically functional molecular chimeras 02-12-1980 285 B
Re-scaled PageRank score R

3 2297691 Electrophotography 06-10-1942 588 B
4 7749477 Carbon nanotube arrays 06-07-2010 1 C
5 7784029 Network service for modularly constructing a software defined radio 24-08-2010 1 C
6 5536637 Method of screening for cDNA encoding novel secreted mammalian 16-07-1996 422 A
proteins in yeast
7 4683195 Process for amplifying, detecting, and/or-cloning nucleic acid se- 28-07-1987 1956 A
quences
8 5523520 Mutant dwarfism gene of petunia 04-06-1996 1139 A
9 4395486 Method for the direct analysis of sickle cell anaemia 26-07-1983 71 B
10 4683202 Process for amplifying nucleic acid sequences 28-07-1987 2169 A

1 4683202 Process for amplifying nucleic acid sequences 28-07-1987 2169 A
2 4237224 (*) Process for producing biologically functional molecular chimeras 02-12-1980 285 B
3 2297691 Electrophotography 06-10-1942 588 B
TimeAttentiveRank score T

4 D268584 (*) Personal computer 12-04-1983 3 C
5 7749477 Carbon nanotube arrays 06-07-2010 1 C
6 7784029 Network service for modularly constructing a software defined radio 24-08-2010 1 C
7 5536637 Method of screening for cDNA encoding novel secreted mammalian 16-07-1996 422 A
proteins in yeast
8 5225539 (*) Using recombinant DNA to produce an altered antibody 06-07-1993 549 A
9 4683195 Process for amplifying, detecting, and/or-cloning nucleic acid se- 28-07-1987 1956 A
quences
10 4395486 Method for the direct analysis of sickle cell anemia 26-07-1983 71 B

Table 9: Performance matrix for DEP-netPlus. Best results are marked in bold.

DEP-net DEP-netPlus
Measure A grade (%) B grade (%) C grade (%) A grade (%) B grade (%) C grade (%)
Precision 78.00 51.48 74.85 79.03 52.14 75.01
Recall 75.53 46.65 73.22 74.67 45.98 73.30
F-measure 76.74 48.95 74.03 76.84 49.06 74.15

of 296,933 USPTO patents pertaining to “semicon- superiority of our model in capturing not only the
ductor” technology collected within the temporal “importance” of a patent but also in evaluating the
span of 2000 to 2015. We carried out the same pre- patent’s grade.
processing steps along with down-sampling of the
data or certain classes to maintain uniformity. The 6 Conclusion and Future Work
results of experiments performed with an additional
In this paper, we proposed a method to proactively
component, i.e., our proposed TimeAttentiveRank
identify milestone patents that have been granted
score to the deep learning model which we refer to
in recent years. We compared the performance
as DEP-netPlus are presented in Table 9.
of three network-flow algorithms for this purpose
From the table, we can clearly observe that the on two different datasets. On the second dataset,
classification model is enhanced by the inclusion we used a deep-learning-based approach to fuse
of a network flow metric that account for the net- patent content along with network flow metrics, to
work effect due to citations. This also confirms the compare against state-of-the-art and discovered
that our proposed approach results in better Phillip Bonacich. 1987. Power and centrality: A fam-
performance both in identifying “milestone” ily of measures. American Journal of Sociology,
92(5):1170–1182.
patents as well as improving the patent grade
prediction. From the experimental results, we Sergey Brin and Lawrence Page. 1998. The anatomy
summarily concluded that raw citation count is of a large-scale hypertextual web search engine.
not enough to capture the importance of a patent Computer Networks and ISDN Systems, 30(1):107 –
117. Proceedings of the Seventh International World
since it does not take into account the age of Wide Web Conference.
citations. When accounted for the same using a
balanced metric like Time-Attentive ranking, we Péter Bruck, István Réthy, Judit Szente, Jan Tobochnik,
are guaranteed to identify potential patents that and Péter Érdi. 2016. Recognition of emerging
technology trends: class-selective study of citations
are likely to spur technological growth in the near in the u.s. patent citation network. Scientometrics,
future. We also identified top patents per category 107(3):1465–1475.
and sector, which can help in the identification
Richard S. Campbell. 1983. Patent trends as a techno-
of niche areas for innovation. Although patent logical forecasting tool. World Patent Information,
retrieval is a recall-oriented task, these criteria 5(3):137 – 143.
may also help in re-ranking the results against a
keyword search for patents. Mark P. Carpenter, Francis Narin, and Patricia Woolf.
1981. Citation rates to technologically important
patents. World Patent Information, 3(4):160 – 163.
As part of our future work, we would like to
study the importance of geographical location on Manajit Chakraborty, Seyed Ali Bahrainian, and Fabio
Crestani. 2020. Forecasting patent growth by com-
influential patents, such as the country they origi- bining time-series signals using covariance patterns.
nated from, the citations received from other coun- In Proceedings of the Joint Conference of the Infor-
tries and so on. We also plan to experiment with mation Retrieval Communities in Europe (CIRCLE
the various granularity of time such as a month, 2020), Samatan, Gers, France, July 6-9, 2020, vol-
ume 2621 of CEUR Workshop Proceedings.
year, 5-year period, and so on.
P. Chen, H. Xie, S. Maslov, and S. Redner. 2007.
Acknowledgements Finding scientific gems with google’s pagerank al-
gorithm. Journal of Informetrics, 1(1):8 – 15.
We thank the anonymous reviewers for their valu-
able comments. This work was partially supported Park Chung and So Young Sohn. 2020. Early de-
by The Global Structure for Knowledge Networks tection of valuable patents using a deep learning
model: Case of semiconductor industry. Technolog-
project grant (No. 167326) under the SNSF Na- ical Forecasting and Social Change, 158:120146.
tional Research Programme 75 “Big Data” (NRP
75). Giulio Cimini, Andrea Gabrielli, and Francesco Sylos
Labini. 2014. The scientific competitiveness of na-
tions. PloS one, 9(12):e113470.
References Rafael A. Corre Doira and Preeta M. Banerjee. 2015.
Zoltan J Acs, Luc Anselin, and Attila Varga. 2002. Measuring patent’s influence on technological evo-
Patents and innovation counts as measures of re- lution: A study of knowledge spanning and subse-
gional production of new knowledge. Research Pol- quent inventive activity. Research Policy, 44(2):508
icy, 31(7):1069 – 1085. – 521.

Nisa Bakkalbasi, Kathleen Bauer, Janis Glover, and Leo Egghe. 2006. Theory and practise of the g-index.
Lei Wang. 2006. Three options for citation tracking: Scientometrics, 69:131–152.
Google scholar, scopus and web of science. Biomed- Roberto Fontana, Alessandro Nuvolari, Hiroshi
ical Digital Libraries, 3(1):7. Shimizu, and Andrea Vezzulli. 2013. Reassessing
Mark A Bedau, Andrew Buchanan, Devin W Chalmers, patent propensity: Evidence from a dataset of r&d
C Cooper Francis, Norman H Packard, and Noah M awards, 1977-2004. Research Policy, 42(10):1780 –
Pepper. 2011. Evidence in the patent record for the 1792.
evolution of technology using citation and pagerank Adam B Jaffe, Manuel Trajtenberg, and Michael S Fog-
statistics. In ECAL, pages 77–84. Citeseer. arty. 2000. The meaning of patent citations: Report
Johan Bollen, Marko A. Rodriquez, and Herbert Van de on the nber/case-western reserve survey of patentees.
Sompel. 2006. Journal status. Scientometrics, Technical report, National bureau of economic re-
69(3):669–687. search.
M.M.S. Karki. 1997. Patent citation analysis: A policy evaluation of scientific activity. Computer Horizons
analysis tool. World Patent Information, 19(4):269 – Cherry Hill, NJ.
272.
Filippo Radicchi, Santo Fortunato, and Claudio Castel-
Leo Katz. 1953. A new status index derived from so- lano. 2008. Universality of citation distributions:
ciometric analysis. Psychometrika, 18(1):39–43. Toward an objective measure of scientific impact.
Proceedings of the National Academy of Sciences,
Leonid Kogan, Dimitris Papanikolaou, Amit Seru, and
105(45):17268–17272.
Noah Stoffman. 2017. Technological Innovation,
Resource Allocation, and Growth*. The Quarterly Zhuo-Ming Ren, Manuel Sebastian Mariani, Yi-Cheng
Journal of Economics, 132(2):665–712. Zhang, and Matúš Medo. 2018. Randomizing grow-
Wenhui Liao and Sriharsha Veeramachaneni. 2010. ing networks with a time-respecting null model.
Unsupervised learning for reranking-based patent Physical Review E, 97(5):052311.
retrieval. In Proceedings of the 3rd International Deborah Strumsky and José Lobo. 2015. Identifying
Workshop on Patent Information Retrieval, PaIR ’10, the sources of technological novelty in the process
pages 23–26, New York, NY, USA. ACM. of invention. Research Policy, 44(8):1445–1461.
Manuel Sebastian Mariani, Matús̆ Medo, and François
Lafond. 2019. Early identification of important Dylan Walker, Huafeng Xie, Koon-Kiu Yan, and Sergei
patents: Design and validation of citation network Maslov. 2007. Ranking scientific publications using
metrics. Technological Forecasting and Social a model of network traffic. Journal of Statistical Me-
Change, 146:644–654. chanics: Theory and Experiment, 2007(06):P06010.

Manuel Sebastian Mariani, Matúš Medo, and Yi-Cheng William Webber, Alistair Moffat, and Justin Zobel.
Zhang. 2016. Identification of milestone papers 2010. A similarity measure for indefinite rankings.
through time-balanced network centrality. J. Infor- ACM Trans. Inf. Syst., 28(4):20:1–20:38.
metrics, 10(4):1207–1223.
Xiaobing Xue and W. Bruce Croft. 2009. Automatic
Jean-Francois Molinari and Alain Molinari. 2008. A query generation for patent search. In Proceedings
new methodology for ranking scientific institutions. of the 18th ACM Conference on Information and
Scientometrics, 75(1):163–174. Knowledge Management, CIKM ’09, pages 2037–
2040, New York, NY, USA. ACM.
Francis Narin et al. 1976. Evaluative bibliometrics:
The use of publication and citation analysis in the