<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Automated Construction and Analysis of Political Networks via Open Government and Media Sources</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Diego Garcia-Olano</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Marta Arias</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Josep Llu s Larriba Pey</string-name>
        </contrib>
      </contrib-group>
      <abstract>
        <p>We present a tool, www.whoyouelect.com, to generate real world political networks from user provided lists of politicians and news sites. Additional output includes visualizations, interactive tools and maps that allow a user to better understand the politicians and their surrounding environments as portrayed by the media. As a case study, we construct a comprehensive list of current Texas politicians, select news sites that convey a spectrum of political viewpoints covering Texas politics, and examine the results. We propose a "Combined" co-occurrence distance metric to better re ect the relationship between two entities. A topic modeling technique is also proposed as a novel, automated way of labeling communities that exist within a politician's "extended" network.</p>
      </abstract>
      <kwd-group>
        <kwd>political science</kwd>
        <kwd>network science</kwd>
        <kwd>text mining</kwd>
        <kwd>open data</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>Introduction</title>
      <p>We live in an age of over-information, where we must constantly lter
information, process it and make educated decisions be it at home, at work or when we
vote. However in regards to voting, we are inundated by the media with news
stories on a national level, which should in theory lead to a better informed
populace, but more often than not, we only consume media outlets which rea rm
our political beliefs and thus entrench us in them, creating straw men of di ering
views and increasing polarization of voters. This situation is di cult, but at least
manageable when a presidential election occurs, as we need only sort through
the information related to the histories, views, and promises of a small eld of
candidates to make our decision. The situation is however much worse on the
local or state level where the opposite occurs. There are vastly more decisions
to be made, and we do not receive enough information regarding all candidates
and election races in addition to the aforementioned issue of media bias, both
self imposed through the choices we make as media consumers in addition to the
biases of the media sources themselves. This overwhelming ood of information
results in voters making uninformed or partially incomplete decisions at best
or at worst sees them abstaining completely from the process. For instance, the
last election for Governor of Texas in 2014 saw the Republican candidate Greg
Abbott beat the Democrat one Wendy Davis by a 20% margin, 2.8 million votes
versus 1.8 million votes[ElE]. However one must keep in mind that Texas has
approximately 16.7 million eligible voters of whom 4.8 million voted thus giving
the contest a dismal 27.5% of eligible voter participation. That number bumps
up only to 32.5% if one uses registered voters, but that is still low. Another way
of looking at it is to observe that the most powerful executive political position
of Texas, a position that directly a ects the daily lives and welfare of 27 million
Texans was selected by 16% of its possible eligible voters.
1.1</p>
      <sec id="sec-1-1">
        <title>Problem description and background</title>
        <p>There exists a vast literature on the use of network analysis to study important
social and political phenomena [FJ1,FJ2,KHJ,MT1,LA1]. This work was in fact
inspired in part by a presentation of one such paper describing a novel use of
natural language processing and network analysis techniques to describe the
network of drug cartels in Mexico[EJ1]. In it, the authors perform text mining,
partially by hand and partially automated, on a single book about the subject
"Los Se N~ora's del Narco" by Anabel Hernandez and then derive a visual network
from the actors and links discovered. This co-occurrence based network provides
a simple and concise high-level view of the actors and entities involved in the
text. This general idea of combining text mining and network science to both
summarize content while allowing for the discovery of interesting relationships
can be applied on the enormous, but under utilized, amount of information
available in online news. Such processing of largely unstructured text combined
with simple, exible and powerful tools to explore and understand it would be
of great use for voters, journalists and researchers alike.
1.2</p>
      </sec>
      <sec id="sec-1-2">
        <title>Overview of System</title>
        <p>In an e ort to provide more insight into primarily local and state wide contests,
but also including federal elections pertaining to a speci c state, we decided
to build a system "WhoYouElect.com" that could take one or many candidate
names as input, along with a list of online new sources, and then retrieve all the
articles pertaining to the candidates from them. The system then using
natural language processing, information retrieval and network analysis techniques
automatically generates the network of all the politicians, organizations,
businesses and locations associated with each candidate inputted. The system makes
it easy to add new sources to pull content from and additionally provides two
types of visualizations: an individual close up "star" view, commonly referred
to as an ego network, that allows a user to view the entities (politicians,
businesses, etc.) most associated with a candidate along with the articles and context
in which they co-occurred, and an "extended" world view that is a global view
that shows links between a politician and associated entities, and also the links
between those entities themselves; thus allowing for the detection of communities
and other traditional network analysis measures.</p>
      </sec>
      <sec id="sec-1-3">
        <title>Description of Case Study: Texas Politics</title>
        <p>In 2015. the Texas State Congress, composed of the House of Representatives and
Senate, has 181 members, 31 senators and 150 representatives. On the federal
level within the US Congress, Texas has 38 total members, 2 senators and 36
representatives. 27 other State Level Elected O cials are also studied including
the Governor, Lieutenant Governor, etc. In total we have 246 Texas politicians
composed of 74 Democrats and 172 Republicans in our study.
1.4</p>
      </sec>
      <sec id="sec-1-4">
        <title>Organization of Document</title>
        <p>The organization of this document is as follows. Chapter 2 will cover related
works and additional background. Chapter 3 will describe the system structure
and methods by which we automatically gather, store and process articles for
politicians. Chapter 4 focuses on presenting the network analysis tools developed
and available at WhoYouElect.com. Finally Chapter 5 contains conclusions.
2</p>
      </sec>
    </sec>
    <sec id="sec-2">
      <title>Related Work</title>
      <p>The types of graphs we will be constructing are undirected heterogeneous
networks with weighted edges. Heterogeneous means the graph will contain di erent
node types. In our case, nodes can be people, organizations, politicians, locations,
bills, or miscellaneous. For an introduction and overview of the state of the art
of heterogeneous networks and mining techniques see the following [MR1,SY1].
The graphs could be considered to be Social Networks involving political and
nonpolitical actors or "noisy" Political Networks due to the inclusion of nodes
and relationships not involving politicians.</p>
      <p>For the moment we are only considering one "co-reference" relationship type,
meaning one possible edge between each pair of nodes whose weight will be based
on various distance metrics and as such do not nd ourselves in the multiplex
context which is more adept for studying complex networks. For an extensive
examination of the eld, uses, and visualization tools see [KM1,DM1]. Adapting
the system to include more than one edge type by considering other features
in addition to co-reference or perhaps leveraging linked datasets could be
interesting. Two recent works in this area seem promising; one exploring boosting
speci cally in the case of missing or incomplete linked data involves mining a
knowledge base using additional textual context, i.e. "evidences", for named
entity disambiguation [LY1] and the other constructs a probabilistic model that
captures the "popularity" of entities and the distribution of multi-type objects
appearing in the textual context of an entity using meta-path constrained
random walks over networks to learn weights associated with entity linking [SW1].
The later task of characterizing relationships between nodes is usually handled
by deriving a topic model from the corpus of text available, all of the news
articles gathered for a given politician in our case, and assigning the most probable
"topic" to an edge based on learned Bayesian probability models [CJ1,WC1]. An
overview of Latent Dirichlet Allocation (LDA) or alternatively Latent Semantic
Indexing (LSI) to produce topic models is found in [ND1]. Because the focus of
this initial phase was to construct a working proof of concept and the inclusion
of edge labeling via topic modeling would be a nice, but unnecessary step, it has
been left for future work.</p>
      <p>There exist a good deal of prior work that has focused on deriving
information networks from unstructured news and other web texts[PB1,MT1,TF1,ND1].
Two introductions and overviews on the process of deriving networks from text
may be found [DJ1,HJ1]. Of the prior works cited some rely on hand crafted
networks "extracted through a time and e ort consuming manual process based
on interviews and questionnaires collected"[MT1], while others rely on
organizations such as the European Media Monitor [PB1] providing them with access to
an article lookup system that while impressive in the breadth of sources
available, constrains them to only sources from that list. Additionally and speci cally
in reference to the European Media Monitor and another considered media
aggregation service MITs Mediacloud, in the cases where we noticed an overlap in
available sources between their listings and the ones we are considering for our
case study (the Houston Chronicle for instance), the search results from the
original news source internal search engine always returned more results for speci c
entity queries than either the EMM or Mediacloud service which points to an
additional quality assurance weakness. Other works avoid the actual aspect of
retrieving content from news sites by using a point in time snapshot of curated
news corpora released through the Linguistic Data Consortium or the New York
Times[TF1,ND1]. Similarly, [MT1] leverages paid-for search engine results, only
grabbing the rst twenty results for each query and then only utilizing the
snippet of text present in Yahoos search results page as opposed to all the content
within the actual article itself. In the end we were unable to nd any work that
leveraged the publicly available search engines present in most news websites.
By utilizing this mechanism and allowing the exibility to pull content from any
news site which ful lls that requirement, our system allows site administrators
to create a context in which to search and in doing so curate the content and
thus satisfy the needs of di erent users.</p>
      <p>Information retrieval aspects of the texts used in the prior works aside, the
prior works all use NLP to extract entities and then leverage either similarity
metrics based on some combination of entity co- occurrence, textual contexts of
and shared between entities, and the correlation of entities and hyper links found
in documents [PB1,MT1] or topical modeling [TF1,ND1] to infer relationships
of interest. The textual context presented in [MT1] is limited in that it does not
consider entire documents, rather only text snippets from search engine results,
but is robust in its evaluation of metrics quantifying the use of co-occurrences
metrics for labeling relationships as positive or negative. Based on the number
of results, they produce four metrics of similarity: the Jaccard Coe cient, the
Dice Coe cient, Mutual Information (MI) and the Google-based semantic
relatedness [CR1] and evaluated them against a small hand-crafted Policy Network.
Although interesting this approach does not scale as an evaluation approach.
The works of [TH1,PB2] like [PB1] uses the EMM for content, but are novel
in that as opposed to using similarity distance metrics or topic modeling, they
also use natural language processing with an initial seed of hand crafted
syntactic templates to learn syntactic patterns which paraphrase certain prede ned
relations from articles and uses them to label relationships between entities.
3</p>
    </sec>
    <sec id="sec-3">
      <title>Automated Construction of Graphs</title>
      <p>In our case study, we constructed the graphs for 246 active Texan politicians.
This political environment was chosen to illustrate the use of the system, but
could just as easily have been a list of politicians for any city, state, or country.
The system components which deal with the construction of graphs use only
Python, some associated open libraries, MongoDB, and occasional Bash scripts,
and as such is very light from a technical requirements view point. The code for
constructing graphs discussed here is available on github1. The general outline
of the process for constructing graphs follows.
3.1</p>
      <sec id="sec-3-1">
        <title>Adding Politicians from Open Government Sources</title>
        <p>We leveraged the Sunlight Foundations Openstates.org API2to obtain a list of
both active and inactive members of Texas congress returned as JSON and
saved them locally. Federal Representatives are obtained from the GovTrack.us
API. The inactive members API returns a set of 112 prior State representatives
and though they will not be analyzed and no articles speci c to them will be
retrieved, we still will include them as possible domain knowledge to leverage
during disambiguation of entity types during the processing of articles stage.
Throughout this work "entities" is an umbrella term encompassing any actors
of interest including politicians, organizations, bills, etc.</p>
        <p>We obtained a list of non Congressional, state wide elected o cials (Governor,
Lieutenant Governor, etc.) making use of the Secretary of State of Texas
website3. Finally, in order to get federal representatives and senators for the state of
Texas, we leveraged the GovTrack.us API4. At this point we have four les
referring to the active and inactive state representatives, state level elected o cials,
and federal level elected o cials. We load them all and standardize formatting
of elds and save them into our "entities" mongo database.</p>
        <sec id="sec-3-1-1">
          <title>1 http://github.com/diegoolano/whoyouelect/tree/master/generate network</title>
          <p>2 http://openstates.org/api/v1/legislators/?state=tx&amp;active=true</p>
        </sec>
        <sec id="sec-3-1-2">
          <title>3 http://www.sos.state.tx.us/elections/voter/elected.shtml</title>
          <p>4 https://www.govtrack.us/api/v2/role?current=true&amp;state=TX</p>
        </sec>
      </sec>
      <sec id="sec-3-2">
        <title>Adding News Sources</title>
        <p>Now that the entities have been added to our database, we add the news sources
for use in our case study. A subset of Texas newspapers with the highest
circulation, the Dallas Morning News, Houston Chronicle, and the Austin American
Statesman, representing the spectrum of conservative, centrist and progressive
areas within Texas were selected along with two sites, the Texas Observer and
Texas Tribune which focus on Texas politics and issues. In addition, the New
York Times was selected to provide an outside context.
3.3</p>
      </sec>
      <sec id="sec-3-3">
        <title>Data Acquisition by Template Modi cation</title>
        <p>For each desired source we need to gather articles pertaining to our politicians
list. Weve created two template web scrapers, one based on the Python package
BeautifulSoup which is fast, but doesn't handle pages rendered with javascript
well and another based on the Web.Selenium Python package which is a headless
browser that works in all cases, but is slower. Each template version is a folder
containing two les, one le which calls the internal search URL for a given news
source, and then saves a JSON le of the article URLs, titles, and dates retrieved,
and another le invoked after the rst that then grabs the actual article content
for each URL obtained from step one and saves it into a separate JSON le,
one per article which contains the title, URL, article text, the news-source itself,
time, and an identi er. This procedure of template editing, one folder per source,
is straightforward for a web developer with experience to setup and test however
simplifying this procedure requires considerable e ort via inferring structure of
pages probabilistically and could lead to potential loss of information.
3.4</p>
      </sec>
      <sec id="sec-3-4">
        <title>Running &amp; Storing the Web Search Results for Active Entities</title>
        <p>This script then takes the article JSON results and runs some light post
processing on each of them to detect the language of the article text and insert
the candidate name as a key to index for quick future lookups before saving
the result in MongoDB. Currently the system is tailored for articles in English,
but also works for Spanish and any languages supported by MITIE5, an open
source Named Entity Recognition engine that given a document identi es
possible named entities and tags them as "Organization", "Location", "Person" or
"Miscellaneous".</p>
        <p>In addition to these tag types, during processing we look for and include two
additional tag types, "Politician" and "Bill". We alter "Person" tags to be the more
speci c "Politician" tag type if we nd that entity to be preceded or followed
by a politician position title such as "Senator" Bob or Bob, the Senator from
District 8, etc. More likely however, Person entities will be labeled as Politicians
if the entity name is found in our entities database of active and inactive
politicians that are labeled as Politicians when initially entered into the database.</p>
        <sec id="sec-3-4-1">
          <title>5 https://github.com/mit-nlp/MITIE</title>
          <p>Although it could be read as such, we do not mean to imply that Politicians
are not People. The Bill entity type refers to legislation and we use simple naive
heuristics looking for the phrases SB, HR or HB during the processing stage as
these refer to Senate Bill, House Resolution and House Bill respectively.
3.5</p>
        </sec>
      </sec>
      <sec id="sec-3-5">
        <title>Processing Article Results per Active Entity</title>
        <p>Following the completion of the above, the script 6 which is the heart of the
processing step of the retrieved articles for a given entity is called. The script
queries MongoDB to gather all of the articles found for the person, pre- lters
out "sports" and uninteresting results, and then processes the remaining articles
in the following manner:
1. Find the date retrieved for an article and assert its validity and if
necessary, change it to follow the format YYYY-MM-DD. If an invalid date is
seen, assign a default date of 2000-01-01.
2. Split the article into sentences, and verify that the article text is
non-empty otherwise skip it. This step may seem trivial, but in actuality is
of critical importance because we are dealing with uncertain input.
3. Run named entity recognition over the sentences to return the named
entities and tags per sentence and verify that the candidate we are searching
for appears. This step also identi es the cases where a pay wall exists for
a given article and the returned article contains only the rst few lines of
the actual article that does not include the name of the candidate himself in
which case we skip the article entirely.
4. Disambiguate the entities found. As the prior step found entities and
tags by sentence, we must now consolidate the entities list and do a sort of
heuristic co-reference resolution to infer that two entities found in actuality
refer to the same entity. For instance, if we nd a "Barack Obama" tagged
"politician" entity in one sentence and in the next sentence we nd a "Mr.
Obama" entity, we remove the second entity and use the rst in its place.
This step is of particular importance and uses some heuristics to discover
interesting data common to political texts in English such as political parties,
positions and location relationships. For instance, if the entity "D-Houston"
is found in a sentence, the system knows to look forward or backward within
the sentence for a politician since in American political texts in that
construct implies "Democrat from Houston". Its important to note that this
stage uses heuristics that are largely language speci c and as such, would
need to be adapted or just discarded for use with other languages. By the
end of this step, we have a dictionary of all the unique "disambiguated"</p>
        <sec id="sec-3-5-1">
          <title>6 https://github.com/diegoolano/whoyouelect/tree/generate network/</title>
          <p>get articles for person then nd and save relations.py
entities discovered in the article.
5. Filter entities further.7 This optional step utilizes a user created look
up table of "uninteresting" entities to be excluded from the overall graph.
For instance, in our case the entities for "Texas Legislature" and "Congress"
and other entities which provide little distinguishing capacity are excluded.
6. Check if an article is a candidate listing or something similar in
which case discard it as noise. A simple ratio of unique entities divided by
the number of sentences in the article was used, and if that ratio was 10 or
greater, the article was skipped. This metric was developed during testing of
the system when it was noticed that the occasional article would take very
long compared to the normal use case, and on inspection, it would be an
article that contained an unusually high number of disambiguated entities
with respect to the number of sentences in the article. Even in the case where
the listing was a list of candidates for o ces or winners of statewide races, it
would naively create a great deal of spurious relationships where they didnt
exist.
7. Discovery of relationships8. At this point, the system goes through
each found instance of the politician being processed and for each, gathers
all the entities that occur on the same sentence as it, within 3 sentences of
it, or farther than three sentences away and stores the results in a matrix.
8. Veri cation and construction of relationships. We take the
matrix from the prior step and construct the relationships for each instance
of the main politician found with other entities. All the relationships that
have been created with respect to the main politician will be used later
to construct the "star" individual view graph. The system constructs the
relationships between all the other non-main politician entities for the
"extended" network if the initial script was called with the "include larger" ag.
9. Save intermediate results. We now have a results object for the article
that we need to save to our global results set for the politician. This process
goes through the relationships formed in this article and for each sees if it
exists in the global result set already and if so adds its information as a new
"instance" for that existing edge. If it doesnt exist in the global result set,
a new edge gets added. Additionally, there exist actually two functions to
handle this procedure. It was discovered during development that while most
articles take well under a second to process and save, there were some which
7 The optional lter entities, veri cation/construction of relationships and saving to
intermediate object is done mostly in https://github.com/diegoolano/
whoyouelect/tree/generate network/verify and save relations.py.
8 Discovery of relations functionality is predominantly in https://github.com/
diegoolano/whoyouelect/tree/generate network/entity funcs.py
took a large amount of time; "large" meaning anything over 5 seconds. It
was observed that these articles were ones where the number of candidate
instances found multiplied by the number of entities and the number of
sentences was found to be usually greater than a certain threshold (8000 is the
threshold we ended up using). Thus we developed an additional function
speci cally optimized to these larger instances by pre-computing the hashes
for lookups.
10. Save article metrics. Once an article has been processed and saved,
or alternatively skipped, metrics for the article are saved internally.</p>
          <p>Once all articles have been processed, we save the resulting article metrics
to our global metrics le and then save our global result set and some other
variables into a python "pickle" le, essentially an internal tar le for python
that we compress to save space. This later step was done largely as a time saving
mechanism during development of the system but also allows for a developer to
access the processed data immediately without having to reprocess articles.
3.6
4
4.1</p>
        </sec>
      </sec>
      <sec id="sec-3-6">
        <title>Generate Individual Star and Extended Graphs</title>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>Overview of Who You Elect Visualization Tools</title>
      <sec id="sec-4-1">
        <title>Individual "Star" Ego Network View</title>
        <p>When "Inner Network" is selected from the "Table of Contents" view, the
politicians derived ego network is visually displayed with the entities (ie, people,
politicians, organizations, locations, etc.) which have most co-occurred with him
being placed closer to his central position. By "most co-occurred" we refer to
overall counts independent of whether the co-occurrences were in the same
sentence, near sentences or farther. The following gure shows the graph produced
for Democratic State Representative of District 51, Eddie Rodriguez, who we
will be using as a running example. In the right hand side area we see that
362 articles were obtained from the six sources with the bulk coming from the
Houston Chronicle (136), Austin American Statesman (133) and Texas Tribune
(60). Additionally we see that 6226 entities were discovered along with the exact
break down of counts for each entity types shown. We are presented with the
option to lter by date range, lter by which news sources to include, show more,
less or all ("Show Full") nodes on the screen.</p>
        <p>Clicking on the center node, brings up the information associated with Eddy
Rodriguez including name, party, position, district number, wiki description (if
found), a map of the politician's district and a list of the entities he is most
associated with. Clicking on any non-central node, shows a list of the articles in
which that node and Eddy Rodriguez appear (including date, article title, and
URL ) which maybe lter by media source, and highlights the sentences where
they co-occur together in that article. Along the bottom and left hand side of
the screen, the user can lter the results to only show Politicians or
Organizations, etc and also change the distance metric used for calculating edge weights.
The "Article URLs" link at the bottom right displays a pop up window of a
sortable table of all 362 articles including their URL, date, number of sentences,
number of unique entities, and number of relations created from it. The
functionality for "Show Stats" is of particular interest and when clicked provides
both a time line histogram of how many articles were published by month for
the central politician along with a comprehensive view of their most associated
entities under di erent distance metrics. The "Top Associated" tabs along the
top of the window each show what are the resulting most associated entities by
entity type if di erent distance metrics are used. For instance, the prior gure
has the Top Associated "same sentences" tab currently selected, highlighted in
red, and as such we see a ranked list of the entities which have the most "same
sentence" co-occurrences with Eddie Rodriguez separated by entity type
(Politicians, Organizations, Person, Location, Bills, Misc). We observe that "Dawna
Dukes" is the Politician with the most same sentence occurrences with Eddie
Rodriguez. The 2nd and 3rd tabs refer to distance metrics which additionally
take into account co-occurrences within 3 sentences ("near" in the 2nd case)
and farther (co-occurrence within the same article, but farther than 3 sentences
away in the 3rd case) The fourth Top Associated metric "same article only (not
near)" refers to entities that occur the most at a distance from the main
politician being studied. This listing can be used as a sort of specialized, local term
frequency inverse document frequency (TF-IDF) measure because it allows the
user to observe which entities occur the most at a distance from the politician of
study, and from that, it can be inferred that the strength of the relationship is
lessened. The nal Top Associated "Combined" metric is a proposed
combination of the rst three top metrics with an additional ratio term which penalizes
relations with high "far" distance co-occurrence counts with respect to their
same sentence and near ones. The proposed "Combined" metric is de ned as:
weight = (same sent + .5 * near sent + .1 same article) * boosting
where boosting = combined co-occurrences / same article co-occurrences
The coe cients associated with penalizing "near sentences" and "same article"
co-occurrences (.5 and .1 respectively above) could and should be improved
by having a person with domain knowledge, a political scientist specializing in
Texas Politics in this instance, view the ranked results for each entity type of
various, di erent politician graphs and then reorder the results of each if necessary
thereby assessing their accuracy. It would then be a relatively straightforward
process to use these newly labeled rankings to update the coe cients to produce
results that more re ect the opinions of domain experts.
4.2</p>
      </sec>
      <sec id="sec-4-2">
        <title>Extended View with Communities</title>
        <p>The 2nd generated network is a global view of the entities and is an undirected
graph with edges weighted according to the "Combined" metric described
already. The prior individual "star" view could have at most N-1 edges displaying,
where N is the number of entity nodes, whereas this "extended" graph on the
other hand could possibly have N*(N-1)/2 edges if it is fully connected, i.e. if
all nodes have connections with all other nodes. For this reason, in order to
derive meaningful insight into the network it is necessary to be able to search for
"communities" amongst the nodes and to lter out edges based on the weight,
i.e. "importance", of an edge between two entities. The idea of detecting
communities in a network is similar conceptually to that of clustering in multivariate
analysis and machine learning, and refers to a densely connected group of nodes
that are well separated from the rest of the network. More formally and
commonly, the de nition of a community entails that the number of intra-community
edges amongst the nodes of a single community be greater than the number of
inter-community edges. There are a vast number of detection algorithms that
can be used, but for our case we decided to go with the Louvain method[BV1]
since it works on weighted graphs, provides a hierarchy of clusters, and is fast to
run even on large graphs. We leverage an open-source JavaScript
implementation of the method and D3.js to produce visualizations such as in the following
gure that shows the network formed by articles obtained and processed for
Eddie Rodriguez. A possibly better approach would be to implement a temporal
based community detection approach with soft membership rules such as [LYR1]
to explicitly account for the evolution of communities over time. The system
defaults to searching for 25 communities and uses a link threshold of 15 to only
display relationships above that amount, though both quantities are set by URL
variables which may be set by the user. The Extended view spatially positions
nodes within the same community together and assigns them the same color.
Selecting a node, displays a list of most related nodes and their types and weights
in the right hand column while also visually only displaying the nodes and links
which are connected to the selected node. Along the top of the visualization the
user may choose to size nodes by various common network metrics such as
"Degree", "Page Rank", "Inverse Transitivity", "Strength", "Number of Articles"
or "None" to keep them the same size. Along the left hand side, the
communities are listed and may be expanded to see the nodes within a community or
for more detail, the user may select "Community Analysis" which pulls up a
detailed report of the articles and nodes, and network measures that de ne each
community. The system also includes other mechanisms with the goal of helping
nd and highlight potentially interesting relationships to aid the end users data
exploration. These appear under the "Stats Area" header in the left hand side,
and include "Article Appearances", "Clustering Coe cient", "Degree
Centrality", "Node Strength", "Page Rank", and "Inverse Articles Strength". Clicking
on any of these headers displays an ordered list of the nodes with the highest
number of "Article Appearances", etc.
4.3</p>
      </sec>
      <sec id="sec-4-3">
        <title>Automated Summarization of Communities</title>
        <p>If we select the Community Analysis link for Eddy Rodriguez , we can see the
details pertaining to a speci c community detected within the context of his
retrieved and processed articles. The information there provides details into who
the central gures are within a community and additionally lists the articles
that are most prevalent within it. It would be useful however to have a way
of automatically providing a description of the community at a higher level in
order to give a more easily digestible global perspective of it. In that way we can
then label all communities and allow the end user an additional perspective into
the summaries as a whole. One way to do that is by treating the articles of a
given community collectively as a single corpus. We can then analyze the corpus
using an initial TF-IDF procedure to lter terms and reduce noise, followed by
performing Latent Dirichlet Allocation to derive topics. Since we know how many
entities from a community occurred in each article found in the community, we
can weigh these articles by their relative importance. Additionally, we consider
those articles with only one entity from the community as noise and exclude
them from the corpus. As a proof of concept in the following table we show the
results of applying this technique to a community of Eddy Rodriguez's using
1grams with the initial amount of topics being set to 5. The initial topics number
is set low because if the number of communities is su ciently high, we would
hope that each community would encapsulate at most a few topics though this
again varies per community and is based on the number of entities, articles,
expansion/conductance values and the overall modularity of the communities
We observe that due to the relatively low number of topics there is some overlap
of concepts as highlighted in the second "single-words" topic in the above table,
where the blue words refer to "agriculture" terms (farmers market, farm to
table caucus) and the red refer to "redistricting" terms associated with articles
discussing a lawsuit involving the re-drawing of district maps that would change
U.S. Rep. Lloyd Doggett of Austins district to include Latino neighborhoods of
San Antonio. Additionally for this proof of concept, we are not taking advantage
of the stochastic nature of the results returned by LDA, which are the likelihoods
of a document belonging to any particular topic. The results above use just the
most likely topic for assignment of documents, and as such lose the additional
information provided in the posterior values of the model. This information will
change the results quite a bit if a sizeable number of the articles have more than
one topic with high probability, and is left for future work.
5</p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>Conclusions</title>
      <p>In this work we presented a tool that generates real world political networks from
user provided lists of politicians and news sites. The downloaded and processed
article data for each politician were enriched with data obtained from various
open sources in order to facilitate veri ed politician meta-data and provide some
structure along with the unstructured article texts. In addition to the newly
created networks, various visualizations, and tools that allow for the exploration of
a politician and their environment were automatically generated. As of now the
maps are derived from open-source geographic shape les, so making maps for
new studies and areas consists in largely just nding or constructing the
appropriate shape le and then simply using them instead of the Texas based shape
les. We showed that the proposed Combined co-occurrence distance metric
better determines the strength of the relationship between two actors in a graph
as compared with the other more traditional metrics used in the literature. The
proof-of-concept use of topic modeling for labeling speci c communities within
a politicians extended network is interesting and warrants further exploration
and development. We have left as future work the performing of an extensive
statistical study of the obtained graphs and media results, but have provided
tools that allow a user to access these results individually or collectively.
6</p>
    </sec>
    <sec id="sec-6">
      <title>Acknowledgements</title>
      <p>We thank the anonymous reviewers for their helpful comments. Research at UPC
was partially funded by SGR2014-890 (MACDA) project of the Generalitat de
Catalunya and MINECO project APCOM (TIN2014-57226-P).</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [ElE] Elect Project: http://www.electproject.org/2014g [FJ1]
          <string-name>
            <surname>James H. Fowler</surname>
          </string-name>
          <article-title>: Connecting the congress: A study of cosponsorship networks</article-title>
          .
          <source>Political Analysis</source>
          ,
          <volume>14</volume>
          (
          <issue>4</issue>
          ):
          <fpage>456487</fpage>
          ,
          <year>2006</year>
          [FJ2]
          <string-name>
            <surname>James H. Fowler</surname>
          </string-name>
          , Timothy R. Johnson,
          <string-name>
            <surname>James F. Spriggs</surname>
            <given-names>II</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sangick</surname>
            <given-names>Jeon</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Paul J.</surname>
          </string-name>
          <article-title>Wahlbeck: Network Analysis and the Law: Measuring the Legal Importance of Precedents at the U.S. Supreme Court</article-title>
          .
          <source>Political Analysis</source>
          ,
          <volume>15</volume>
          :
          <fpage>324346</fpage>
          ,
          <year>2007</year>
          [KHJ]
          <string-name>
            <surname>Justin H Kirkland:</surname>
          </string-name>
          <article-title>The relational determinants of legislative outcomes: Strong and weak ties between legislators</article-title>
          <source>The Journal of Politics</source>
          ,
          <volume>73</volume>
          (
          <issue>03</issue>
          ):
          <fpage>887898</fpage>
          ,
          <year>2011</year>
          [MT1]
          <string-name>
            <given-names>Matt</given-names>
            <surname>Thomas</surname>
          </string-name>
          , et. al:
          <article-title>Get out the vote: Determining support or opposition from Congressional oor- debate transcripts</article-title>
          .
          <source>Proceedings of EMAIL</source>
          , pp.
          <fpage>327335</fpage>
          .
          <year>2006</year>
          [LA1]
          <string-name>
            <given-names>Alejandro</given-names>
            <surname>Llorente</surname>
          </string-name>
          , Manuel Garcia-Herranz, Manuel Cebrian,
          <source>Esteban Moro: Social media ngerprints of unemployment arXiv:1411.3140v2 [physics.soc-ph] 2014 [</source>
          <article-title>EJ1] Jesus Espinal-Enriquez, Mario Siqueiros-Garca, et</article-title>
          . al.:
          <article-title>A literature-based approach to a narco-network</article-title>
          .
          <source>Social Informatics</source>
          , pages
          <fpage>97101</fpage>
          . Springer,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [MR1]
          <string-name>
            <given-names>Rokia</given-names>
            <surname>Missaoui</surname>
          </string-name>
          .: Mining Heterogeneous Information Nets, EGC Toulouse,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [SY1]
          <article-title>Yizhou Sun, et a Mining Heterogeneous Information Networks</article-title>
          .
          <source>Synthesis Lectures on Data Mining &amp; Knowledge Discovery. Morgan &amp; Claypool Publishers</source>
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [KM1]
          <string-name>
            <given-names>Mikko</given-names>
            <surname>Kivela</surname>
          </string-name>
          , et. al.:
          <article-title>Multilayer networks</article-title>
          .
          <source>Journal of Complex Networks</source>
          ,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          <string-name>
            <surname>[DM1] Manlio De Domenico</surname>
          </string-name>
          , Mason A.
          <string-name>
            <surname>Porter</surname>
          </string-name>
          , Alex Arenas:
          <article-title>MuxViz: a tool for multilayer analysis and visualization of networks</article-title>
          .
          <source>In Journal of Complex Networks</source>
          .
          <year>2014</year>
          [LY1]
          <string-name>
            <given-names>Yang</given-names>
            <surname>Li</surname>
          </string-name>
          , et al.:
          <article-title>Mining Evidences for Named Entity Disambiguation</article-title>
          ,
          <string-name>
            <surname>KDD</surname>
          </string-name>
          <year>2013</year>
          [SW1]
          <string-name>
            <given-names>Wei</given-names>
            <surname>Shen</surname>
          </string-name>
          , Jiawei Han, et al:
          <article-title>A Probabilistic Model for Linking Named Entities in Web Text with Heterogeneous Information Networks</article-title>
          .
          <source>SIGMOD</source>
          <year>2014</year>
          [
          <article-title>CJ1] Jonathan Chang, Jordan Boyd Gaber, David M Blei: Connections between the Lines: Augmenting Social Networks with Text</article-title>
          .
          <source>KDD</source>
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          <string-name>
            <surname>[WC1] Chi</surname>
            <given-names>Wang</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Marina Danilevsky</surname>
          </string-name>
          , et al.:
          <article-title>A Phrase Mining Framework for Recursive Construction of a Topical Hierarchy</article-title>
          .
          <source>KDD</source>
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [ND1]
          <string-name>
            <given-names>David</given-names>
            <surname>Newman</surname>
          </string-name>
          , et al.:
          <article-title>Analyzing entities and topics in news articles using statistical topic models</article-title>
          .
          <source>Intelligence and Security Informatics</source>
          . Springer,
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [PB1]
          <string-name>
            <given-names>Bruno</given-names>
            <surname>Pouliquen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Ralf</given-names>
            <surname>Steinberger</surname>
          </string-name>
          , et al. :
          <article-title>Building and Displaying Name Relations using Automatic Unsupervised Analysis of Newspaper Articles</article-title>
          .
          <source>In Journees internationales dAnalyse statistique des Donnees Textuelles. 8es</source>
          ,
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [MT1]
          <string-name>
            <given-names>Theodosis</given-names>
            <surname>Moschopoulos</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Elias</given-names>
            <surname>Iosif</surname>
          </string-name>
          , et al.:
          <article-title>Toward the automatic extraction of policy networks using web links and documents. Knowledge and Data Engineering</article-title>
          , IEEE Transactions on,
          <volume>25</volume>
          (
          <issue>10</issue>
          ):
          <fpage>24042417</fpage>
          ,
          <year>2013</year>
          [TF1]
          <string-name>
            <given-names>Fangbo</given-names>
            <surname>Tao</surname>
          </string-name>
          ,
          <string-name>
            <given-names>George</given-names>
            <surname>Brova</surname>
          </string-name>
          , et al.:
          <article-title>NewsNetExplorer: Automatic Construction and Exploration of News Information Networks</article-title>
          .
          <source>SIGMOD</source>
          <year>2014</year>
          , June 2227,
          <year>2014</year>
          [DJ1]
          <article-title>Jana Diesner: Extraction and Analysis of Semantic Network Data from Text Data</article-title>
          .
          <source>Semantic Network Analysis Workshop</source>
          , St. Petersburg State, May
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [HJ1] Jiawei Han,
          <string-name>
            <given-names>Chi</given-names>
            <surname>Wang</surname>
          </string-name>
          , et al:
          <article-title>Bringing Structure to Text</article-title>
          .
          <source>KDD</source>
          <year>2014</year>
          [
          <article-title>CR1] Rudi L Cilibrasi and Paul MB Vitanyi.: The google similarity distance. Knowledge and Data Engineering</article-title>
          , IEEE Transactions on,
          <volume>19</volume>
          (
          <issue>3</issue>
          ):
          <fpage>370383</fpage>
          ,
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [TH1]
          <article-title>Hristo Tanev: Unsupervised learning of social networks from a multiple-source news corpus</article-title>
          .
          <source>Multisource, Multilingual Info Extraction &amp; Summarization</source>
          ,
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [PB2]
          <string-name>
            <given-names>Bruno</given-names>
            <surname>Pouliquen</surname>
          </string-name>
          , et. al.:
          <article-title>Extracting and learning social networks out of multilingual news. Social Networks and Application tools</article-title>
          . Skalica,Slovakia,
          <year>Sept 2008</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [BV1]
          <string-name>
            <surname>Vincent D. Blondel</surname>
          </string-name>
          ,
          <string-name>
            <surname>Jean-Loup Guillaume</surname>
          </string-name>
          , et al:
          <article-title>Fast unfolding of communities in large networks</article-title>
          .
          <source>In J. Stat. Mech. and arXiv:0803.0476</source>
          ,
          <year>2008</year>
          [LYR1]
          <string-name>
            <surname>Yu-Ru Lin</surname>
          </string-name>
          , et al:
          <article-title>FacetNet: A Framework for Analyzing Communities and Their Evolutions in Dynamic Networks WWW 2008</article-title>
          , April 2125,
          <year>2008</year>
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>