Experience of designing different versions of web-based development environment (IDE) for Optimizing parallelizing system and compiler onto reconfigurable architecture is described. Designed system is based on existing tools and frameworks such as Jupyter Notebook and Eclipse Che. Set of requirements for Optimizing parallelizing system components is developed to make it possible to integrate them into web-based development environment accessible through the Internet. Designing portable environment for compiler development, compiler technology demonstration and teaching parallel program development is also described. Newly developed program transformations are shown to be used during program optimizations for FPGA inside the designed web environment. Means of program transformation visualization are described for use with Jupyter Notebook. The work shown demonstrates possibility to organize remote access to library of instruments and tools for program optimizations currently under development that would be convenient for application developers.
During development of a complex software system, like an optimizing compiler as an example, multitude of problems arise that concern overall organization of development process, deployment and usage of developed system. Complexities arise as well with regards to training and inclusion of new developers, organizing outside access to experimental results, like performing demonstrations of particular functions.
Problems that arise are in many ways similar to those that are faced by programmers not familiar with software development for field-programmable gate arrays, as it requires using complex development environments.
At present moment systems that allow to directly transform high level language programs into low-level hardware description for FPGA are either not sufficiently developed or not sufficiently widespread. To alleviate this problem it is sensible to organize remote access to such systems as they are being developed through Internet in form browser accessible integrated development environment, that is complemented with particular extensions that allow developers to iteratively modify their program and resulting low level hardware description, interactively estimate their characteristics. This approach could help solve some of urgent problems that more and more application developers are facing in their first attempts to develop software targeting FPGA.
Lets note main problems and goals considered in this work: 1. Rapid deployment of developer environment that includes all necessary tools for new member of developer team or new client for the system. 2. Remote delivery of intuitive and simple dialogue-based interface that allows developer to create hardware description for FPGA from source program. 3. Development of tools to aid demonstration of certain new features of compiler that is being developed. 4. Using proposed web environment and parts of compiler system for teaching software development and parallel programming. 5. Using parts, select functions of system being developed in scientific research which requires ease of access to experimental findings, results and convenience of reproduction.
Earlier some web interfaces were developed for demonstration of Optimizing
parallelizing system capabilities [
This way of using web access solves only problem #2, however only partially. User has no ability to directly manage what system does and tweak any parameters, while developer has to create specially crafted scenarios for using the system.
This work states the problem of developing more universal and flexible development environment (software service) that would be accessible remotely through Internet and would have useful properties: 1. Service would be based on currently available tools from cloud computing industry, open source software-as-a-service packages and cloud IDEs. 2. Service would solve all of the stated problems to some extent. 3. Service would not require any extra effort for functionality maintenance from developers as they modify and develop underlying compiler itself while it is being updated.
Compiler onto reconfigurable computer architecture is being developed based on
OPS [
Main results of this work include, firstly, new architectural requirements for Optimizing parallelizing system and its components that are gradually adopted in it to allow creation of program optimization tools based on it and creation of education tools for parallel programming and showcasing program optimizations for FPGA programming. Secondly, another outcome of this work is a prototype of web-based development environment for program transformation onto FPGA that is based on OPS and available open-source cloud IDEs.
Further content of this paper is as follows. More detailed requirements for compiler components to facilitate creation of web IDE based on said compiler are described in section “Problem statement”. Section “Jupyterlab employment for OPS” describes approach to create interactive development environment for OPS based on interactive browser-based scientific notebooks and C++ language. Section “Results” describes most important modifications that allow to create minimally functional development environment for experimenting with program transformations onto FPGA based on compiler components. 2
In cloud computing industry there exists multitude of widely employed web-oriented development environments with open source code that allow to make extensions for any particular field of study or subject area, for example to support new programming languages, management of supercomputer clusters, etc.
Eclipse Che [
However, using this particular software as a basis for specialized cloud development environment requires modifications done to Eclipse Che itself as well as to the compiler.
Generally, modifications necessary would include: 1. Separating compiler pipeline into stages with possibility to export and visualize intermediate results between stages 2. Adding new representations to IDE interface and corresponding methods for workspace API. 3. Creation of set of containers to run compiler and its parts inside predefined environment. 4. Refactor compiler source code to loosen tight relations between modules to make it easier to use them inside cloud-based IDE. 3
Jupyterlab [
Because it is actually possible in Jupyterlab to use interactive notebooks with the same programming language that OPS itself is written in, usage scenarios for various typical use cases and user types could be written easily.
Now to allow different users access to different scenarios it is enough to create set of interactive notebooks that would offer following: 1. Examples of how to use key functions of the system, for example working with internal representation, performing program transformation, which is especially useful for educational purposes. 2. Allow users to perform experiments with examining performance effects and other changes made by different program transformations for sample program using external services for testing, performance analysis, etc. 3. Simple and convenient interface for system’s developers to exchange results of their work, data, test results and examples. JupyterLab allows to use several cell data formats that allow visualization of data of any type. To visualize results of program transformations it is needed to process following types of data: 1. Program source code in plain text format or with complex notes for operators, variable occurrences, etc., which HTML is the most convenient format to use. 2. Program code with associated graph representation that have variable occurrences, expressions or operators as nodes, which SVG graphics can represent. 3. Tabular data such as test results, performance measurements, source code metrics, all of which could be represented with Markdown formatted text. 4. Graph representations of source code that are particularly complex, such as lattice graph for multidimensional loop nest, that could be represented with a bitmap.
It is most convenient to use HTML to represent source code of initial program and its version after transformation as shown on Fig. 3, which illustrates 2 results of similar source code fragment detection for the purpose of adding new instructions to softcore CPU. Apart from marking operators by color it is possible to use interactive control elements. Using text data formats such as HTML or SVG makes developing many specific visualizations and passing results around easier.
During development and testing of optimizing program transformations in Optimizing parallelizing system it is useful to have flexible development environment to test transformations alone and in sequence and estimate their applicability for mapping programs onto FPGA. This environment needs to facilitate display of results, inclusion of samples into documentation. Using JupyterLab with specifically developed extensions that are assembled into integral system satisfied these requirements. Fig. 4 illustrates how program transformations are ran for simple programs, particularly for transformation that forms C function into macro-definition of operation set extension in TCE co-design environment.
Examples like these allow to expand documentation for OPS compiler, make source code more self-documenting, add to unit tests and integration tests and create easy to use scenarios for different modules, functions and transformation pipelines. This allows to easily perform experiments with parts of the system, develop and test new sequences of program transformations. 6
To solve previously stated problems we have introduced following improvements and
additions to OPS:
1. Set of container images to facilitate reliable and repeatable builds of OPS itself and
of tools based on it, as well as to simplify execution of OPS tools and compiler for
sample programs.
2. Extra services to test performance of transformed C language programs similar to
earlier developed black box testing system [
Fig. 5 shows a set of main OPS modules that implement independent plugins. This allows to offer access to only limited set of interfaces to the end user, as well as speed up and simplify system build and its deployment. Further work is being done in two main directions: 1. Support for using compiler source code inside cloud-based IDE to make compiler development more convenient. 2. Adding functionality to specialized web IDE that allows to use the compiler and its modules.
Now to start working with OPS source code and OPS compiler it is enough to simply open a given URL inside web browser after which a fresh workspace will be created for the user with full configured development environment. This makes it easy to introduce users to the system and its source code and lowers amount of time needed for setup, helps in education environment. 7
Specialized tools for easy development, employment and educational use of complex software systems could be created on top of modern cloud-based integrated development environments.
To develop functionally sound web-based development environment based on retargetable compiler it is necessary to solve multiple problems with regards to source code modularity, managing system’s external dependencies, making it more portable and developing flexible programming interfaces for accessing its functions. Solving these problems allows to make system development easier as well as allow more convenient use of it, including using it as educational tool for teaching software programming and demonstrating its key capabilities, visualization of experimental data for web IDE users in interactive form.
The reported study was funded by RFBR according to the research project No 1837-00179.