Dart is a programming language introduced by Google in 2011. Today, it is mainly used in Flutter, a crossplatform SDK to build native mobile applications, desktop applications, and web-based applications. Since its introduction, Flutter has gained in popularity and so has Dart. Thus it has become crucial for the industry and the academic community to dispose of the proper tools to analyze, maintain and evolve projects in Dart. However, our research community has yet to propose tools for the static analysis of Dart. In this paper, we share our experience regarding the usage of Pharo 10 to support the analysis of Dart2.10 and we present our early work on a set of open-source tools. Our tools cover the parsing of Dart with SmaCC, the visualization of its AST with Roassal, and its meta-model analysis with Moose and Famix.
applications. However, despite its increasing popularity, the Dart community faces an absence
of academic and industrial tools to analyze this language. Furthermore, among the diferent
ways to improve software quality, model-driven engineering (MDE) and re-engineering has
outstanding impacts, as seen in [
To tackle the Dart programming language software quality tooling, we built a process for analyzing Dart source code and Flutter application. This process comes with the usage of tools to perform the first level of analysis: code parsing and model analysis, alongside with Flutter application visualizations. In this paper, we share our experience on how to leverage diferent tools to perform such first analysis of a new language such as Dart.
Our paper is structured as follows: section 2 presents an overview of the steps we follow to implement our analysis of Dart; section 3 details the parsing step of Dart and our use of SmaCC; section 4 presents two visualizations we introduce for Dart, one for the Dart AST with Roassal and another for the file dependencies in a Dart project with Roassal and Mermaid; section 5 introduces the early implementations of an importer for the Famix meta-model with Moose.
In our work, we want to exploit and reuse the tools’ suit of Pharo, which has shown to be a
reliable static analyzer of legacy languages like Java, Delphi, and C [
Parsing Dart code. The first step of our analysis is to be able to parse and perform symbol
resolution of Dart source code. The goal of this step is to output an Abstract Syntax Tree (AST)
for Dart. To achieve our goal, we need to formally define a grammar using Extended
BackusNaur Form (EBNF) for Dart and to implement its corresponding parser. The implementation
of this parser required us to implement all the Dart entities inside their corresponding Pharo
classes. So that when parsing a Dart declaration of a class, our parser creates a DartClass entity
(and so on for method, attribute, etc.). Inside the existing Pharo tool set, we found two options:
PetitParser23 and SmaCC [
Model analysis. Exploiting an AST with a visualization is practical when analyzing one or
two source code files. However, it becomes more tedious when having to visualize more than a
hundred files in one project. This is where Moose becomes handy [
In the following section, we detail each step of our analysis process of Dart in order. All the tools that we present are work in progress (with many improvements required) and their source code are available in their respective GitHub depot.
The first step to perform advanced analysis of a programming language is to build a parser. Based on the parser output, one will then be able to create visualisations or perform reengineering.
However, unlike EBNF legacy programming languages (or long-live ones such as Java and C), we observe dificulties to find tooling to parse Dart code. Perhaps it is due to Dart being a relatively new language from 2011 and that it is still in evolution. Or, because Google has not introduced an oficial grammar for Dart, despite providing an oficial specification from Google 5, the 5th edition since 20146. Tools to extract an AST are only available as Flutter packages, making them useful only in the context of Flutter development 7.
4Roassal3 : https://github.com/ObjectProfile/Roassal3 5https://dart.dev/guides/language/specifications/DartLangSpec-v2.10.pdf 6https://www.ecma-international.org/publications-and-standards/standards/ecma-408/ 7https://pub.dev/documentation/analyzer/latest/dart_ast_ast/dart_ast_ast-library.html
SmaCC (Smalltalk Compiler-Compiler) is a generator of scanners and parsers for language that comes with an EBNF grammar. SmaCC inputs an EBNF grammar and outputs a parser that match the grammar. SmaCC compilation process follows a two-step process: first, the scanning which converts a stream of characters into a stream of tokens; then, a parsing step which converts the stream of tokens into objects that are defined by the SmaCC user.
h e l l o U s e r :
" h e l l o " <name> ’ userName ’ { { H e l l o U s e r } } ; <name> :
[ A− Za− z ] \w∗ ;
Let’s go over the basic of SmaCC with a grammar example in Listing 1. Using the EBNF grammar, we declare the tokens and the production rules used by SmaCC to output our objects. A token is declared with a unique token name associated with a specific regular expression. In our example, <name> is a token that matches any name starting with a capital letter. Production rules are non-terminal symbols associated with a set of possible production. Unlike tokens, each production rule is a none terminal symbol (the production name, like helloUser) and consists of a list of tokens or keywords, ended optionally by a Smalltalk statement. Keywords are declared using double quotes (like "hello"). A SmallTalk statement is enclosed inside curly brackets. Double curly brackets are used to declare an AST node object associated with a specific production, here HelloUser will be generated as a Pharo class. A token can be caught as an instance variable when associated with a symbol using a single quote inside the production rule. In our example, ’userName’ is an instance variable of the class HelloUser and that catches the value of the token <name>
To kick start our work on our parser, we’ve started from a community open-source grammar of Dart available in ANTLR4 GitHub project8. Although ANTLR grammar is in EBNF, SmaCC requires adding specific statements inside the EBNF file (like in Listing 1). Thus, manual editing is required to adapt an ANTLR4 grammar to a SmaCC grammar. Hopefully, SmaCC comes with a tool that eases this transformation9. In our work, we add a statement to declare a SmaCC node beside each production rule that is interesting to consider in our AST. The grammar we adapt and our SmaccDart parser are available on GitHub10.
Here are some notes to conclude this section. First, the grammar we use from ANTLR reports that the oficial Dart specification is not up-to-date with the actual compiler code 11. However, they are unclear about what exactly difers between their grammar and the oficial Dart specification.
Second, the authors of Dart specification are unclear about what is the proper starting symbol of Dart grammar. They hint that they may be at least two equivalent starting rules: 〈partDirective〉 or 〈libraryDefinition 〉 (see the comment of section 19, page 204 of the specification).
9ANTLR to SmaCC convertor : https://github.com/j-brant/SmaCC/tree/master/rewrites/antlr 10SmaCCDart : https://github.com/Evref-BL/SmaccDart 11see comment line 14 : https://github.com/antlr/grammars-v4/blob/c0ad64169f54727b7262b4eb1f7edd5cae14a5ce/ dart2/Dart2Lexer.g4#L14 Thankfully, SmaCC let us define multiple starting symbols, thus resolving this ambiguity with a SmaCC directive like so: %start libraryDefinition partDirective .
Finally, in Dart, class constructor invocations are heavily used inside return expressions to compose a Flutter widget with multiple instances of other widgets. However, only a capital letter in the identifier distinguishes a constructor invocation from a function expression. This only diference is further exaggerated by the fact that the new keyword is optional in Dart2. Thus, our parser is forced to explore both the entire branch of function expression production rule and the constructor invocation, before choosing (from the longest branches of the two) which one is the proper AST node. This exploration has a significant performance impact on our parser. To solve it, we introduce a Dart refactoring step before the parsing that adds the new keyword to constructor invocations. This resolves the ambiguity between the functionExpression and constructorInvocation production rules, significantly increasing the parsing speed. For instance, using the implementation of the class _MyHomePageState12, the parsing is over 5 times faster with the refactoring (1476ms without refactoring against 271ms with refactoring). Therefore, refactoring the code to remove ambiguities has a significant performance impact on the parser speed, but comes with the limitation of changing the initial code source.
SmaccDart give us the means to analyze Dart code and get a Dart AST. However, SmaCC itself does not provide a visualization tool to help developers visualize an AST (in the latest version of Pharo). To solve this, we use Roassal3 to implement an AST visualization. Roassal3 (or simply Roassal) is an agile visualization engine for Pharo. It provides a versatile framework for developing interactive and dynamic data visualizations in a range of forms, such as 2D images, animations, and interactive widgets. Its interactivity is one of its main benefits.
The conception of a Roassal visualizer can be summarized as: a) declaring shapes (i.e., Roassal UI entities) in a canvas; b) associating the shapes to a model; and c) defining a global layout. Shapes are declared programmatically by visiting the produced AST of the previous step. During this visit, we declare a new RSBox shape for each node visited, and a link between these boxes for each node to children relation in the AST. Each shape can be set to a model entity using the Shape»model: Object method. It allows us to display a text value in each box that corresponds to the AST node type. Furthermore, it allows us to interact with the model, as we explain after. Finally, Roassal comes with a set of pre-loaded layouts that relies on links between shapes. For rendering the tree structure of the AST, we use the RSTreeLayout. The result is shown in Figure 2, from an AST based on a basic hello world Dart class.
The interaction with a Roassal visualization allows us to display more information about the model when interacting with the windows of Roassal. In Figure 2, dragging the mouse cursor over a shape displays an overlay panel containing an overview of the source code declared by 12Code extrated from Google’s tutorial of Flutter https://github.com/flutter/codelabs/blob/ b53da2544a1df2b0456d54cd83f37bc7d68805f0/namer/step_08/lib/main.dart#L53 this node. By clicking a node, we open an Inspector window to the SmaCCDart object of the Roassal model, allowing further inspection of the AST. Our AST visualization is included with the Baseline dependencies of SmaCCDart.
We implement a second visualization that takes advantage of SmaCCDart to give a more abstract understanding of the Flutter code. We develop a Flutter dependencies resolver that looks at the imports/exports dependencies declared in each Dart file of a Flutter application. Our tool explores the content of each Dart file declared in a Flutter project with SmaCCDart. Then it maps the file import dependencies inside a graph, with each outbound edged representing a import declaration to a specific file.
Dependencies are visualized as a graph, where nodes are files and directed edges represent their specific imports. We color files that are declared in the Flutter project in orange, to distinguish them from the external dependencies (in gray). As an example, we analyze a GitHub open-source Dart project called Animsearch13. We produce the import dependencies graph of this Flutter application using a mermaid flowgraph in Figure 3. The result can be exported to a 13AnimSearch : https://github.com/ArizArmeidi/AnimSearch mermaid14 drawing that can be included in markdown text. Thus, developers can share this representation with non-Pharo experts.
The two proposed visualizations are convenient for small projects, but shows their limits at scale. For instance, the AST visualization is eficient for one file at a time, but becomes tedious to visualize the entire AST of one project because of the large amount of nodes. Similarly, a project with a large number of files and dependencies is dificult to represent inside our Mermaid graph. Therefore, to extend the analysis of Dart, we rely on model engineering and build a meta-model for Dart in section 5.
Based on the AST produced by SmaCC, one can build great visualization using Roassal. However, when it comes to analyzing industrial level software systems, these visualizations find their limits due to the large number of entities. Also, the algorithms to create them become increasingly dificult to write and less and less generalizable. It is in this context that one needs metamodelization framework. Out of the existing one, Moose allows one to create easily new 14Mermaid : https://mermaid.js.org/ meta-models to support new programming languages with out-of-the-box compatible tools. In this section, we describe our early work with Moose for Dart project analysis.
To construct the meta-model, we use the Famix framework [
Declaring new entities is semi-automated with Famix since these classes can be generated. As an example, let’s say that we want to add Dart classes in our Dart meta-model. First, we manually declare a Pharo class called FamixDartGenerator, a subclass of FamixMetamodelGenerator. The generator is responsible for defining the entity’s classes, with the entity’s traits and hierarchy. Thus, we add to FamixDartGenerator two methods: defineClasses and defineHierarchy , shown in Listing 2. Note that a Dart class has attributes and methods. In our meta-model, this relationship is translated using the traits TWithAttributes and TWithMethods (last two lines of defineHierarchy in Listing 3. Finally, we execute the method FamixDartGenerator generate and obtain a new package Famix-Dart-Entities, with our FamixDartClass class generated by Famix.
Listing 3: defineHierarchy method’s code defineHierarchy s u p e r defineHierarchy . class − |> #TClass . class − |> #TType . class − |> #THasVisibility . class − |> #TCanImplement . class − |> #TWithMethods . class − |> #TWithAttributes .
Once a Famix entity is declared in our meta-model, it is ready to be created. To create a model, a good approach is to develop an importer, a program capable of visiting an object and returning instances of its corresponding model entities. In our case, the visiting object is the Dart AST, and its corresponding entities, the one we declare in our Famix-Dart meta-model.
Our Famix Importer is a work-in-progress project called Chartreuse-D15. As of today, it is only able to import classes, attributes, and methods. The methods to visit a Dart AST are provided 15Chartreuse-D : https://github.com/Evref-BL/Chartreuse-D by SmaCC, using a trait it generates when we build our parser for Dart. Thus, we gain access to methods that are invoked when a specific AST node is visited, e.g. visitClassDeclaration. Inside, we set our importer to instantiate a new Famix Dart Entity corresponding to this currently visited class. The methods and attributes of this current class will be instantiated upon visiting their nodes.
Once the AST is fully visited, we complete our model with the resolution of the symbols. Symbol resolution associate the model entities between them. Thus, we look for dependencies such as method invocations, variable and attribute accesses, type references, and type inheritances. In our importer, this step takes place directly on the model instance itself.
In our current work on Chartreuse-D, we plan to implement the symbol resolution step after enough Dart entities are included in our meta-model. Our Roadmap plans a release for November 2023 of this importer, as an open-source project available on GitHub. However, this implementation is a specific Famix Importer, which comes with challenges. First, the behavior of the importer is tied to the visitor of the AST, thus our SmaCCDart parser. This means that maintenance of our parser will require maintenance of our importer. Second, Dart is a multiple paradigm programming language, which allows one to do Functional and Object-oriented programming. Therefore, functions can be declared and used inside objects’ methods without restriction. This can prove challenging during the symbol resolution, as we will need to distinguish between inner method invocation and external function invocation. Finally, we need to study if the current Famix traits are suficient for our Dart meta-model.
In this paper, we explore the potential of diverse tools built in Pharo to handle a static analysis of the Dart language. Our attempts cover the multiple steps of the static analysis process. First, we introduce SmaCCDart, a Pharo parser for Dart2 built with SmaCC. Second, we explore the potential of visualization conception made with Roassal3 and propose an interactive AST visualization. Plus, we propose a Flutter dependencies resolver that gives a quick view of the relation between the internal and external files in one Flutter project. Finally, we give an overview of how to conduct MDE with Moose and Famix for a Dart code. In this work in progress, we present the idea of a meta-model implementation for Dart based on the Famix model. We also present our plan to implement an importer that will instantiate our Famix Dart meta-model from visiting the AST produced by SmaCCDart.
Although this paper very much describes works in progress, it encourages the idea that new languages, such as Dart, can still be analyzed by existing tools already in Pharo. We plan on releasing regular updates of our presented tools and on advancing the development of the importer, set to be released by the end of 2023. Future works will be dedicated to the evaluation of our tools and process, using open source projects on online repository to analyze Flutter project written in Dart16. We believe that this work can open the way to new industrial tools 16https://github.com/tortuvshin/open-source-flutter-apps promoting better-analyzing software systems, thus increasing the maintainability and evolution of new software built in Dart.