<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Overview of Graph Search and Beyond</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Omar Alonso</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Microsoft Mountain View</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>CA USA</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Marti A. Hearst</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>UC Berkeley Berkeley</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>CA USA</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Jaap Kamps</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>University of Amsterdam Amsterdam</institution>
          <country country="NL">The Netherlands</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>Modern Web data is highly structured in terms of entities and relations from large knowledge resources, geo-temporal references and social network structure, resulting in a massive multidimensional graph. This graph essentially unies both the searcher and the information resources that played a fundamentally di erent role in traditional IR, and \Graph Search" o ers major new ways to access relevant information. Graph search a ects both query formulation (complex queries about entities and relations building on the searcher's context) as well as result exploration and discovery (slicing and dicing the information using the graph structure) in a completely personalized way. This new graph based approach introduces great opportunities, but also great challenges, in terms of data quality and data integration, user interface design, and privacy. We view the notion of \graph search" as searching information from your personal point of view (you are the query) over a highly structured and curated information space. This goes beyond the traditional two-term queries and ten blue links results that users are familiar with, requiring a highly interactive session covering both query formulation and result exploration. The workshop attracted a range of researchers working on this and related topics, and made concrete progress working together on one of the greatest challenges in the years to come.</p>
      </abstract>
      <kwd-group>
        <kwd>Graph search</kwd>
        <kwd>Semantic search</kwd>
        <kwd>Personalization</kwd>
        <kwd>Exploration</kwd>
        <kwd>Query suggest</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>Categories and Subject Descriptors</title>
      <p>H.3.3 [Information Storage and Retrieval]: Information
Search and Retrieval|Query formulation, Search process,
Selection process
Copyright c 2015 for the individual papers by the papers’ authors. Copying
permitted for private and academic purposes. This volume is published and copyrighted by
its editors.</p>
      <p>SIGIR Workshop on Graph Search and Beyond ’15 Santiago, Chile
Published on CEUR-WS: http://ceur-ws.org/Vol-1393/.</p>
    </sec>
    <sec id="sec-2">
      <title>1. INTRODUCTION</title>
      <p>Information on the Web is increasingly structured in terms
of entities and relations from large knowledge resources,
geotemporal references and social network structure, resulting
in a massive multidimensional graph. This graph essentially
uni es both the searcher and the information resources that
played a fundamentally di erent role in traditional IR, and
o ers major new ways to access relevant information. In
services that rely on personalized information like social
networks, the graph plays an even more important role, in other
words: you are the query.</p>
      <p>Graph search a ects both query formulation as well as
result exploration and discovery. On the one hand, it
allows for incrementally expressing complex information needs
that triangulate information about multiple entities or entity
types, relations between those entities, with various lters
on geo-temporal constraints or the sources of information
used (or ignored), and taking into account the rich pro le
and context information of the searcher (and his/her peers,
and peers of peers, etc). On the other hand, it allows for
more powerful ways to explore the results from various
aspects and viewpoints, by slicing and dicing the information
using the graph structure, and using the same structure for
explaining why results are retrieved or recommended, and
by whom.</p>
      <p>This new graph based information seeking approach
introduces great opportunities, but also great challenges, both
technical ranging from data quality and data integration to
user interface design, as well as ethical challenges in terms of
privacy; transparency, bias and control; and avoiding the
socalled lter bubbles. Graph search is already available today
in many avors with di erent levels of interactivity. Social
network-based services like Facebook and LinkedIn provide
exibility to search their personal network form many
diverse angles. Web search engines like Google and Bing rely
more on using graphs to show related content as a
mechanism to include other possible contexts for a given query.
Clearly, it is not limited to web, and can be applied to
other highly structured data. Just to give an example, the
hansards or parliamentary proceedings are fully public data
with a clear graph structure linking every speech to the
respective speaker, their role in parliament and their political
party. Graph search allows to explore politics from the
viewpoint of individual members of parliament or government.</p>
      <p>At a high level, graph search seems limited to familiar
entity types (e.g., Facebook entities) and templates. How
far can this scale? Will this work on truly open domains?
There is a huge potential to use the graph to go beyond
recommendations for new friends and contacts or semantically
related content. Unlocking the potential of richer knowledge
sources for new search strategies requires us to think outside
the box, by combining di erent insights from IR, semantic
search, data integration, query expansion and user interfaces
to name a few.</p>
      <p>The rest of this report is structured in the following way.
Section 2 discusses the open research questions raised by
searching highly structured data from a personal point of
view. Next, in Section 3) we discuss the four keynotes who
helped frame the problems and reach a shared
understanding of the issues involved amongst all workshop attendees.
Rose Marie Philip talked about personalized post search at
Facebook, Swee Lim about graph search at Linkedin, Doug
Oard about good uses for crummy knowledge graphs, and
Alex Wade about Microsoft academic graph. Section 4
discusses the six contributed papers, which were presented in a
boaster and poster session. Finally, Section 5 provides
preliminary discussion of the results and progress made during
the workshop.
2.</p>
    </sec>
    <sec id="sec-3">
      <title>OPEN RESEARCH QUESTIONS</title>
      <p>We view the notion of \graph search" as searching
information from your personal point of view (you are the
query) over a highly structured and curated information
space. This goes beyond the traditional two-term queries
and ten blue links results that users are familiar with,
requiring a highly interactive session covering both query
formulation and result exploration.</p>
      <p>
        This raises many open questions:
IR Theory What happens if search gets personal? Does
this break the classic dichotomy between users and
documents, as users are nodes in the social network
data themselves? What is the consequence of ultimate
personalization, as the local graph di ers for all users?
As the local graph structure is key, does this obviate
the need for large central indexes? Do these types of
requests t in the classic paradigm (e.g., Broder's
taxonomy)? How does this shift the balance between the
control of the searcher and the ranker over the result
set?
Data Integration Building a knowledge graph requires
massive data integration at many levels: are there
tradeo s in simplicity and level of detail (such as the classic
knowledge representation trade-o )? What levels of
granularity and comprehensiveness are needed for
effective deployment? What quality is needed: is any
noise acceptable? How to deal with near duplicate
detection, con ation, or entity disambiguation?
Use Cases and Applications Rather than a universal
solution, graph search is particularly useful for speci c
types of information needs and queries. What are
the data and tasks that make graph search works?
What kind of scenarios that would bene t from a graph
model? In what context can switching perspectives by
showing results from the vista of other persons useful?
Query formulation How to move from singular queries to
highly interactive sessions with multiple variant queries?
What new tools are needed to help a searcher construct
the appropriate graph search query using re nements
or lters to better articulate their needs, or explore
further aspects? How can we augment query
autocompletion to actively prompt user to interactively construct
longer queries exploring di erent aspects?
Result Exploration There is a radical shift towards the
control of the searcher|small changes in the query can
lead to radically di erent result sets|how can we
support active exploration of slices of the data to explore
further aspects? Unlike traditional facetted search
options, the result space is highly dynamic, how can we
provide adaptive exploration options tailored to the
context and searcher, at every stage of the process?
Evaluation How do we know the system is any good? How
to evaluate the overall process, given its personalized
and interactive nature? Can we rely on the direct
evaluation of query suggestions and query
recommendations? Are there suitable behavioral criteria for in
the wild testing, such as longer queries, multiple
lters, longer dwell-time, more active engagement, more
structured-query templates? Can we use are
standard experimental evaluation methods from HCI and
UI/UX design?
Privacy Access to personal data is fraught ethical and
privacy concerns, is there is similarly structured public
data for scienti c research? As an extreme form of
personalization, how to avoid the uncanny cave, lter
bubbles and echo chambers? How ethical is it to
privilege a particular query re nement suggestion over the
many other possible candidates?
Further discussion on the challenges of graph based search
can be found in [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ].
3.
      </p>
    </sec>
    <sec id="sec-4">
      <title>KEYNOTES</title>
      <p>Four invited speakers helped frame the problems and reach
a shared understanding of the issues involved amongst all
workshop attendees.
3.1</p>
    </sec>
    <sec id="sec-5">
      <title>Personalized Post Search at Facebook</title>
      <p>
        The opening keynote was given by Rose Marie Philip
(Facebook) on \personalized post search at Facebook" [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ].
      </p>
      <p>There are over a billion people and over a trillion posts on
Facebook. Among these posts, there are uniquely
personalized answers to many search queries. The goal of Facebook
post search is to help people nd the most personally
relevant posts for each individual query, tailored to the content
of people's networks. In this talk, I will present some of our
work to build a search product that uses personalized graph
signals in ranking. I will also give an overview of query
modi cation, posts retrieval and ranking of results.
3.2</p>
    </sec>
    <sec id="sec-6">
      <title>Graph Search at Linkedin</title>
      <p>
        The second keynote in the morning was given by Swee
Lim (Linkedin) on \graph search at Linkedin" [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ].
      </p>
      <p>Linkedin is the largest professional social network. Linkedin's
graph and search systems help our users discover other users,
jobs, companies, schools, and relevant professional
information. I will present the evolution of these systems, how they
support current use cases, their strengths and weaknesses,
our next generation systems, and how we intend to leverage
these systems to perform graph searches.
3.3</p>
    </sec>
    <sec id="sec-7">
      <title>Good Uses for Crummy Knowledge Graphs</title>
      <p>
        The rst keynote in the afternoon was given by Doug Oard
(University of Maryland) on \good uses for crummy
knowledge graphs" [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ].
      </p>
      <p>
        In 1993, Ken Church and Ed Hovy suggested that
before we ask how well some new technology meets the need
we envision for it, we should pause and rst re ect on the
question of whether { now that we know something about
what can be built { we are envisioning the right uses for
what we have. They titled their paper \Good Applications
for Crummy Machine Translation" [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]. At about that same
time, information retrieval researchers obliged them by
(generally without having read their paper) starting to work
on cross-language information retrieval; arguably the best
application for crummy machine translation ever invented.
Now we have some crummy knowledge graphs { and this
time we have read the Church and Hovy paper { so perhaps
the time is right for us to ask whether we have yet envisioned
good uses for crummy knowledge graphs. In this talk, I will
seek to seed that discussion.
3.4
      </p>
    </sec>
    <sec id="sec-8">
      <title>Overview of Microsoft Academic Graph</title>
      <p>
        The second afternoon keynote was given by Alex Wade
(Microsoft Research) with an \overview of Microsoft
academic graph" [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ]
      </p>
      <p>The Microsoft Academic Graph is a heterogeneous graph
containing scienti c publication records, citation
relationships between those publications, as well as authors,
institutions, journals and conference \venues" and elds of study.</p>
    </sec>
    <sec id="sec-9">
      <title>ACCEPTED PAPERS</title>
      <p>We requested the submission of short, 4{5 page papers to
be presented as boaster and poster. We accepted a total
of 6 papers out of 8 submissions after peer review (a 75%
acceptance rate).</p>
      <p>
        Jadeja and Shah [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ] investigate data driven ways to
visualize and navigate graph or tree structured data. Navigating
or traversing highly curated graph data is an understudied
problem, and hierarchical or tree visualizations can help
create order and overview. When visualizing the data from the
viewpoint of a particular node makes any graph data (such
as social network data) look like a tree with the starting
node (a person with all context) as point of origin.
      </p>
      <p>
        Sabetghadam et al. [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ] investigates ways of \reranking"
results based on a graph traversal approach for multimodal
IR, that is hoped to be robust over di erent distributions
of modalities. The use case of multimodal IR in a curated
data space with rich context presents a challenge, as di erent
features and scores on di erent modalities will be very
differently distributed in very di erent probability spaces. An
application of Metropolis-Hastings as sampling/estimation
method is suggested as (partial) solution.
      </p>
      <p>
        Sakamoto et al. [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ] investigate captioning or summarizing
results in highly curated graph data. Succint descriptions
are essential for e ective graph exploration, and requires to
take the context and structure into account. The paper
discusses a particular graph of words, sentences and documents,
and also touches upon semantic annotations, which would
move the document and text space to an entity space, with
all documents and text linked to a particular category or
entity.
      </p>
      <p>
        Santisteban and Carcamo [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ] investigates a variant of the
classic Tanimoto or Jaccard similarity measure able to deal
with assymmetry in directed graphs and subsumption
hierarchies. Similarity measures are central in IR, and related
distance measures central in graph data. The discussion is
motivated by a use case of \paradigmatic" structures.
      </p>
      <p>
        Tong et al. [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ] investigate category and word relation
graphs for retrieving trouble shooting information/documents,
addressing the classical IR problem of human assigned
controlled terms versus document free text in the context of a
curated data space and rich context (at least in principle).
The paper o ers an interesting graph approach is outlined,
mapping terms to categories, for both requests and
documents. Making this graph level explicitly available to users
o ers interesting new possibilities, and opens up ways to
map the noisy term occurrence space to the curated,
concept and entity based space of the category codes. Hence
this paves the way to a semantic, entity based view.
      </p>
      <p>
        Yu et al. [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ] investigates the strength of connections in an
entity graph, speci cally a scholarly network with a rich
entity graph available as public data. This is an interesing use
case with a curated data space and rich context, plus an
interesting dynamic structure over time. The paper proposes
to take the strength (or weakness) of connections into
account | here as simulated blind feedback | turning a
network into a weighted network of the simple graph into a
valued graph.
5.
      </p>
    </sec>
    <sec id="sec-10">
      <title>CONCLUSIONS</title>
      <p>
        The workshop brought together researchers from a range
of areas in information access, who worked together on
searching information from your personal point of view over a
highly structured and curated information space. One of
the main lines of discussion was the considerable industrial
activity around social graphs. The most famous example
is Facebook Graph Search, a feature that allows users to
perform more sophisticated searches on their social network
[
        <xref ref-type="bibr" rid="ref11">11</xref>
        ]. Bing has been integrating Facebook into their web
search results for the last couple of years. Similarly, Google
has been annotating search results with Google+ pro les.
And all the rest of the search industry is moving in the
same direction.
      </p>
      <p>There are also crucial links with work on searching
structured data, and work on the appropriate query languages,
in particular as part of semantic search. These branches of
research in particular focus on complex querying of
structured text or data, whereas the graph search addresses also,
and perhaps primarily, the process of constructing series of
complex queries interactively. This is directly related to
exploratory search and sense making. The graph structure
provides natural facets for exploring the data, from a
local point of view, allowing for a more dynamic structure
than traditional faceted search using rigid, global,
hierarchical structure. This challenges our understanding of search
user interfaces design and evaluation, with search results
moving from the found links, to the HIT page as snippets,
and now to query suggestion as previews of possible query
extensions.</p>
      <p>Graph Search has fundamental consequences for
information access and o ers tremendous opportunities for building
new systems and tools that allow users to explore
information from many di erent angles, shifting control back to the
user. This is a radical departure from current systems where
the machine learning dominate the interaction: the entire
information space is determined by the user, and the user is
in the driver's seat when expressing her needs and exploring
the space of options interactive.</p>
      <p>Acknowledgments This research is funded in part by the
Netherlands Organization for Scienti c Research (ExPoSe
project, NWO CI # 314.99.108; DiLiPaD project, NWO
Digging into Data # 600.006.014).</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>O.</given-names>
            <surname>Alonso</surname>
          </string-name>
          and
          <string-name>
            <given-names>J.</given-names>
            <surname>Kamps</surname>
          </string-name>
          .
          <article-title>Beyond graph search: Exploring and exploiting rich connected data sets</article-title>
          .
          <source>In ICWE'15: Engineering the Web in the Big Data Era</source>
          , volume
          <volume>9114</volume>
          <source>of LNCS</source>
          , pages
          <volume>3</volume>
          {
          <fpage>12</fpage>
          . Springer,
          <year>2015</year>
          . URL http://dx.doi.org/10.1007/978-3-
          <fpage>319</fpage>
          -19890-
          <issue>3</issue>
          _
          <fpage>1</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>O.</given-names>
            <surname>Alonso</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. A.</given-names>
            <surname>Hearst</surname>
          </string-name>
          , and J. Kamps, editors.
          <source>GSB'15: Proceedings of the SIGIR'15 Workshop on Graph Search and Beyond</source>
          ,
          <year>2015</year>
          .
          <article-title>CEUR-WS</article-title>
          . URL http://ceur-ws.
          <source>org/</source>
          Vol-
          <volume>1393</volume>
          /.
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>K. W.</given-names>
            <surname>Church</surname>
          </string-name>
          and
          <string-name>
            <given-names>E. H.</given-names>
            <surname>Hovy</surname>
          </string-name>
          .
          <article-title>Good applications for crummy machine translation</article-title>
          .
          <source>Machine Translation</source>
          ,
          <volume>8</volume>
          :
          <fpage>239</fpage>
          {
          <fpage>258</fpage>
          ,
          <year>1993</year>
          . URL http://dx.doi.org/10.1007/ BF00981759.
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>M.</given-names>
            <surname>Jadeja</surname>
          </string-name>
          and
          <string-name>
            <given-names>K.</given-names>
            <surname>Shah</surname>
          </string-name>
          .
          <article-title>Tree-map: A visualization tool for large data</article-title>
          .
          <source>In Alonso et al. [2]</source>
          , pages
          <fpage>9</fpage>
          <lpage>{</lpage>
          13. URL http://ceur-ws.
          <source>org/</source>
          Vol-
          <volume>1393</volume>
          /.
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>S.</given-names>
            <surname>Lim</surname>
          </string-name>
          .
          <article-title>Graph search at linkedin</article-title>
          . In Alonso et al. [
          <article-title>2], page 5</article-title>
          . URL http://ceur-ws.
          <source>org/</source>
          Vol-
          <volume>1393</volume>
          /.
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>D. W.</given-names>
            <surname>Oard</surname>
          </string-name>
          .
          <article-title>Good uses for crummy knowledge graphs</article-title>
          .
          <source>In Alonso et al. [2], page 6</source>
          . URL http://ceur-ws.
          <source>org/</source>
          Vol-
          <volume>1393</volume>
          /.
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>R. M.</given-names>
            <surname>Philip</surname>
          </string-name>
          .
          <article-title>Personalized post search at facebook</article-title>
          . In Alonso et al. [
          <article-title>2], page 7</article-title>
          . URL http://ceur-ws.
          <source>org/</source>
          Vol-
          <volume>1393</volume>
          /.
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>S.</given-names>
            <surname>Sabetghadam</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Lupu</surname>
          </string-name>
          ,
          <article-title>and</article-title>
          <string-name>
            <given-names>A.</given-names>
            <surname>Rauber</surname>
          </string-name>
          .
          <article-title>Leveraging metropolis-hastings algorithm on graph-based model for multimodal ir</article-title>
          .
          <source>In Alonso et al. [2]</source>
          , pages
          <fpage>14</fpage>
          {
          <fpage>18</fpage>
          . URL http://ceur-ws.
          <source>org/</source>
          Vol-
          <volume>1393</volume>
          /.
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>K.</given-names>
            <surname>Sakamoto</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Shibuki</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Mori</surname>
          </string-name>
          , and
          <string-name>
            <given-names>N.</given-names>
            <surname>Kando</surname>
          </string-name>
          .
          <article-title>Fusion of heterogeneous information in graph-based ranking for query-biased summarization</article-title>
          .
          <source>In Alonso et al. [2]</source>
          , pages
          <fpage>19</fpage>
          {
          <fpage>22</fpage>
          . URL http://ceur-ws.
          <source>org/</source>
          Vol-
          <volume>1393</volume>
          /.
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>J.</given-names>
            <surname>Santisteban</surname>
          </string-name>
          and
          <string-name>
            <given-names>J. T.</given-names>
            <surname>Carcamo</surname>
          </string-name>
          .
          <article-title>Unilateral jaccard similarity coe cient</article-title>
          .
          <source>In Alonso et al. [2]</source>
          , pages
          <fpage>23</fpage>
          {
          <fpage>27</fpage>
          . URL http://ceur-ws.
          <source>org/</source>
          Vol-
          <volume>1393</volume>
          /.
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>N. V.</given-names>
            <surname>Spirin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>He</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Develin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K. G.</given-names>
            <surname>Karahalios</surname>
          </string-name>
          , and
          <string-name>
            <given-names>M.</given-names>
            <surname>Boucher</surname>
          </string-name>
          .
          <article-title>People search within an online social network: Large scale analysis of facebook graph search query logs</article-title>
          .
          <source>In CIKM'14</source>
          , pages
          <fpage>1009</fpage>
          {
          <fpage>1018</fpage>
          . ACM,
          <year>2014</year>
          . URL http://doi.acm.
          <source>org/10</source>
          .1145/2661829. 2661967.
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>B.</given-names>
            <surname>Tong</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Yanase</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Ozaki</surname>
          </string-name>
          , and
          <string-name>
            <given-names>M.</given-names>
            <surname>Iwayama</surname>
          </string-name>
          .
          <article-title>Information retrieval boosted by category for troubleshooting search system</article-title>
          .
          <source>In Alonso et al. [2]</source>
          , pages
          <fpage>28</fpage>
          {
          <fpage>32</fpage>
          . URL http://ceur-ws.
          <source>org/</source>
          Vol-
          <volume>1393</volume>
          /.
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>A. D.</given-names>
            <surname>Wade</surname>
          </string-name>
          .
          <article-title>Overview of microsoft academic graph</article-title>
          . In Alonso et al. [
          <article-title>2], page 8</article-title>
          . URL http://ceur-ws.
          <source>org/</source>
          Vol-
          <volume>1393</volume>
          /.
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>Y.</given-names>
            <surname>Yu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Z.</given-names>
            <surname>Jiang</surname>
          </string-name>
          , and
          <string-name>
            <given-names>X.</given-names>
            <surname>Liu</surname>
          </string-name>
          .
          <article-title>Random walk and feedback on scholarly network</article-title>
          .
          <source>In Alonso et al. [2]</source>
          , pages
          <fpage>33</fpage>
          {
          <fpage>37</fpage>
          . URL http://ceur-ws.
          <source>org/</source>
          Vol-
          <volume>1393</volume>
          /.
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>