Knowledge graphs have been extremely useful in powering diverse applications like natural language understanding. CodeBreaker attempts to construct machine interpretable knowledge graphs about program code to similarly power diverse applications such as code search, code understanding, and code automation. We have built such a 1.98 billion edges knowledge graph by a detailed analysis of function usage in 1.3 million Python programs in GitHub, documentation about the functions in 2300+ modules, forum discussions with more than 47 million posts, class hierarchy information, etc. In this work, we will demonstrate one application of this knowledge graph, which is a code recommendation engine for programmers within an IDE. All user interactions within the application get translated into SPARQL queries, which have quite different characteristics than queries against traditional knowledge graphs such as DBpedia or Wikidata. Aspects of code such as data ow are inherently transitive, hence the SPARQL is complex and requires property paths. One of our goals is to provide these queries as a basis for graph querying benchmarks, while allowing users the ability to interact with a real application built on top of a large graph database.
Several knowledge graphs have been constructed in recent years such as
DBpedia [
In CodeBreaker, we use a set of generic techniques to construct the rst large-scale knowledge graph for code. Speci cally, we developed knowledge graph with 1.98 billion edges from deep analysis of 1.3 million Python programs on Copyright c 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
GitHub. This covered 278K functions, 257K classes and 5.8M methods from 2300+ Python modules, 47M posts from StackOver ow and StackExchange.
One main challenge for building such a knowledge graph is about
representation. We represent the code in a more global way than in prior work based
on program text or ASTs, i.e., in terms of data ow and control ow. This
allows us to analyze connections throughout programs. Speci cally, we capture
the following: (a) which objects get passed as arguments to which methods, (b)
which objects get used to invoke methods (data ow) and (c) which methods
get called before which other ones (control ow). However, global connections
make scalable querying over large graphs harder; querying data- and
controlow is inherently transitive. This can be achieved using SPARQL 1.1 [
CodeBreaker is composed of 1.3 million individual graphs, one for each
program. Therefore, we use the Resource Description Framework [
We demonstrate CodeBreaker3 using a recommendation engine we built into an IDE. Our key focus here is on the sorts of storage and query support needed for use cases of code knowledge graphs. The conference audience will interact with CodeBreaker through a graphical interface, where they can nd 1) the next most likely call given the current method call, 2) popular data science pipelines used by others that are similar to their own, 3) relevant forum discussions based on their own code. For each of these scenarios, we provide SPARQL queries, which can provide the basis for graph querying benchmarks, along with the knowledge graph. 2
We provide a complete interface to show the potential of CodeBreaker in coding assistance. In particular, we integrated CodeBreaker with Jupyter Lab using its Language Server Protocol support, https://github.com/krassowski/ jupyterlab-lsp. 2.1
Next Coding Step We show a developer being provided commonly used next steps, based on the context of their current code. Context means data ow predecessors of the node 3 A video of the demo is available at https://github.com/wala/graph4code/blob/ master/docs/figures/demo_v2.mp4 while the underlying knowledge graph is available at https://wala.github.io/graph4code/ of interest; we take a simple example of the single predecessor call that constructed the classi er. Figure 1 shows a real Kaggle notebook, where users can select any expression in the code and get the most common next steps along with their frequencies. In similar contexts, data scientists typically do one of the following: 1) build a text report showing the main classi cation metrics (frequency: 16), 2) report the confusion matrix which is an important step to understand the classi cation errors (frequency: 10) and 3) save the prediction array as text (frequency: 8). This can help users by alerting them to best practices from other data scientists. In this example, the suggested step of adding code to compute a confusion matrix is actually useful. The existing Kaggle notebook does not contain this call, but the call is very helpful to understand the properties of a classi er. The exact SPARQL query to support this functionality is available https://github.com/wala/graph4code/blob/master/usage_ queries/find_next_step.sparql. The second scenario helps understand data science pipelines similar to existing ones, and use this to understand what types of models other data scientists use given similar code context. As shown in Figure 2, to de ne the pipeline, the developer needs to choose two steps in the pipeline; such as from the point a dataset is read (e.g. read csv) until a t call is performed (e.g., model.fit). In the example program, data ows from read csv to RandomForestClassifier.fit. The interface allows users to query what other classi ers tend to be invoked on the same data as RandomForestClassifier. On the right side of Figure 2, we can see that in Kaggle notebooks, people tend to use RandomForestClassifier, along with Gradient Boost Classifier and K neighbours Classifier to t the same data. The thickness of the arrows denote how frequently these classi ers have been used together. This recommendation gives data scientists options of di erent classi cation models to try. The SPARQL query for gathering the relevant data science pipelines can be found in https://github.com/wala/graph4code/ blob/master/usage_queries/find_similar_flows.sparql. This query taxes many aspects SPARQL, as we need multiple transitive property paths to capture the t calls, multiple regex lters, aggregation and a minus.
…
Select line, set as start of flow
Mark this line as end, find similar flows A developer can nd posts in StackOver ow and StackExchange based on code written so far. From the code, CodeBreaker nds the forum posts with similar ows. Similarities in ow are useful because it implies that the post is discussing the same coding context. However, it is hard sometimes to detect code similarity at a token level since the same object can be called di erently. Since CodeBreaker's graph decomposes the Kaggle code into its semantics, we can take each of the nodes in the data ow for the Kaggle program and issue a SPARQL query which then gathers up relevant StackOver ow posts.
Figure 3 shows how one can show relevant forum posts for the path in the code up to sklearn.svm.SVC.fit. The gure shows one of the posts. Note that the code written in the Kaggle notebook has data owing from a read csv to a train test split to a SVC.fit. This exact ow exists in the retrieved StackOver ow post which also a read csv to a train test split to a fit on an SVC. Similarities in ow are useful because it implies that the post is discussing the same coding context. It is hard to detect code similarity at a token level since the SVC object is called model in the Kaggle notebook and clf SVM radial basis in this forum post. Since CodeBreaker's graph decomposes the Kaggle code into its semantics, we can take each of the nodes in the data ow for the Kaggle program and issue a SPARQL query which then gathers up relevant StackOver ow posts. The corresponding SPARQL query can be found at https://github.com/wala/graph4code/blob/master/usage_ queries/find_stack_overflow_posts.sparql. 3
In this paper, we demonstrated the use of CodeBreaker, the rst large-scale knowledge graph for code, in code assistance within an IDE. In particular, we show how one can get code suggestions, nding similar data science pipelines Right click on line, select
show relevant posts and context-based search in web forums. The knowledge graph is extensible and we made it publicly available to the larger community for use.