The paper considers the task of analyzing complex interconnected objects using graph construction. There is no unified tool for constructing graphs. Some solutions can build graphs limited by the number of nodes, while others do not visually display data. The Gephi application was used to construct graphs for the research. Gephi has great functionality for building and analyzing graphs. The subject of research is a politician with a certain set of characteristics. In the paper an algorithm that enables to automate data collection on politicians was developed. One of the main methods of data collecting on the Internet is web scraping. Web scraping software may access the World Wide Web directly using the HTTP, or through a web browser. While web scraping can be done manually by a software user, the term typically refers to automated processes implemented using a software agent. The data was necessary for constructing graphs and their analysis. The use of graphs enables to see various types of relationships, including mediate. This methodology enables to change the attitude towards the analysis of multidimensional objects.
In the modern world there is a large amount of information that can be represented in
the form of objects and the relationship between them. For example, objects can be
ыindividuals, the relationship between which is described by a certain characteristic. In
this case, graphs are often used for analysis [
Political forces often play an important role for the development of scientific and social activities within individual states and the world community. Decisions made by political forces may depend on the individual characteristics of the person (gender, age, fraction) and the relationship between each other.
Information about political figures is also posted on the Internet. It enables to obtain data in sufficient quantities. It is unreasonable to collect data on politicians manually, since there is a lot of information. Therefore, algorithms are being developed that enable to collect data from information sources of the Internet.
To build graphs for political leaders, it is necessary to determine the nodes of the graphs and describe the relationship between them.
Graph plotting is an unnatural process for the computer. With existing software it is
rarely possible to construct large graphs or have the ability to interact. However, with
the correct construction, the graphs enables a qualitative data analysis and identify both
explicit and hidden relationships [
One of the main methods of data collecting on the Internet is web scraping. The general principle of web scraping can be represented as follows: the program code sends a request to the target source and receives a response in the form of HTML code, after which it searches for the required information using the XPath query language.
Requests are most often generated using the HTTP protocol; they use the GET request method to receive data.
The response is an object that is predisposed to receive any data. In data processing, an HTML document is extracted from the response, after which a search is performed, and the result is converted to the required format. This process is called parsing.
Web scraping can be done in two ways: • development of software; • use of third-party software (including API).
If it is necessary to use several information sources at the same time for data
extraction, the second method is usually not used. Automate data collection during the
development of software can be through the use of agent technologies (software agents)
[
Information about politicians is available on various Internet sources. As an
example, information from such sources as VoteSmart and Govtrack is considered [
Graph Visualization of the Characteristics of Complex Objects… 3
• GET request to an information source; • extracting data from the response body (HTML document) using Xpath; • saving the received data with the possibility of visualization.
JSON is selected as the data presentation format. The choice is due to the complex
nested structure of the raw data, which is difficult to process when stored in CSV
format. Meanwhile, the XML format is redundant [
For the analysis of interrelated objects graphical representation of the data are used. A graph consists of vertices (nodes) and edges that form a connection between vertices. In order to determine the nodes and edges, we select the characteristics of the person obtained during data collection:
The figure (see Fig. 2) shows an example of the data collected by a politician in json. Due to the fact that a person is characterized by many characteristics, when constructing a graph for politicians, two approaches arise: • the node of the graph is a person, an edge - some of the properties; • several types of nodes of a person are selected, edges - the correspondence of nodes in the raw data.
In the first approach, for the trivial case, nodes [
For the second approach, the nodes are unique values of characteristics; edges are the relationship between these characteristics in the raw data. To construct such a graph, it is required to consider in what form the information is presented.
The raw data are separate JSON files, which contain information about representatives of some of the largest US states: Texas, California and Florida. To construct a graph it is necessary to explicitly distinguish nodes and edges. We divide the nodes into categories, while all nodes can be connected by edges only through the node identity.
In the typical case a graph is defined by an adjacency matrix, but other methods are often used. In particular, if there are many “zeros” in the adjacency matrix, which indicate that there is no connection between nodes, the graph is specified with a list. The list consists of entries of the following form (1):
< , , , >, where − starting node, − end node, . − edge weight (1)
Graph Visualization of the Characteristics of Complex Objects… 5 In this case the graph will be undirected. The edge only indicates the connection between the nodes, not the nature of the connection. We distinguish the following categories of nodes: • Person; • University; • Religion; • Membership in a civil organization; • Party.
Because of the initial data representation one of the nodes of each edge will be a ‘person’ node.
The graph may be weighted and unweighted. Weight can be introduced if there is a one-to-many relationship between objects. In particular, such a relationship is formed in such node as ‘person – university’ and “person - membership in a civil organization”. In this case, the weight of the edge will be specified and show the relative share of the university / organization for a particular person. Mathematically this is given by a formula (2): ∑ =1 , = 1 ∀ , ℎ (2) − "person" vertex, − institutions connected with vertex j The Python programming language is used to obtain the graph structure. The developed algorithm converts the raw JSON data into nodes and edges, which are stored in separate CSV files. 2.3
There is no unified tool for constructing graphs. Some solutions can build graphs
limited by the number of nodes, while others do not visually display data [
Gephi has great functionality for building and analyzing graphs. To build a graph we want to colorize the nodes depending on the type of a characteristic. For drawing graphs, several algorithms are proposed, we choose Force Atlas. The algorithm is based on minimizing energy (the nodes are iteratively attracted or repelled from each other in the visualization space, depending on their relative position and the presence of connections), which allows graphs to be constructed with a high degree of interaction. First we colorize the nodes according to the type of a considered characteristic.
We construct a graph for the raw data by adjusting the display of output. The resulting graph will have the following form (see Fig. 3). In the figure, the size of a vertex is directly related l to the number of edges adjacent to it, and the thickness of an edge is directly related to its weight. The following colors are used in the figure: • purple – organization; • brown – party; • yellow – religion; • blue – university; • red – a politician from the state of Florida; • green – a politician from the state of Texas; • blue – a politician from the state of California;
Due to the layout method, the graphs were divided into two clusters: Democrats and Republicans, but due to the large number of nodes, the data cannot be analyzed in detail.
Filters can be used to refine the information. For example, the following filtering result will show which university representatives of the states most often graduate from. (see Fig. 4).
Graph Visualization of the Characteristics of Complex Objects… 7 You can also view information about the relationship of an individual object or group of objects. The figure shows the links between the «Henry Cuellar» and «Darren Soto» nodes (see Fig. 5). These persons are close to each other and it can be assumed that their views are similar. As we can see from the graph, despite the fact that they are from different states, they have a connection with the same organizations. Both are Catholic by religion and belong to the Democratic Party. 3
The task of visualizing linked multidimensional objects is relevant in the field of data analysis. The described technology offers a working approach to constructing graphs for an object such as a politician.
The division of one object of an entity of different nature allows you to build relationships between objects, to draw certain conclusions about them. In particular, the most graduating educational institution is the University of California, graduates of this university become democrats. The construction of several graphs allows us to analyze in more detail and find internal relationships.
It seems that for the sake of completeness, it is necessary to supplement the data for all US congressmen, which truth will lead to a significant increase in relations and the need to consider the graph to solve practical problems, whether it be an analysis of educational institutions or lobbying organizations, or religions depending on the states, etc.
The use of such graph models seems extremely promising from the point of view of analyzing a large amount of information not only on such a complex object as a person, but also on organization technology, etc.