Lessons and Pitfalls in Building Firefox with Tup Guillaume Maudoux and Kim Mens UCLouvain (Belgium) Email: {guillaume.maudoux,kim.mens}@uclouvain.be Abstract—Build system implementations are surprisingly nu- merous for the single common purpose of assembling software. With this variety, picking the right one is a complex task. And even more difficult is the migration to a new build system, with uncertain benefits at the end. Software maintainers and release engineers need better comparisons of build systems and precise catagorisation on which to base an informed decision. As a first step toward that goal, we experimented building Firefox with Tup in replacement of Make. We report here our experience at migrating and comparing the build systems. We also describe interesting features of Tup and we discus Mozilla’s Firefox usage as a benchmark for build systems. I. INTRODUCTION We started investigating Mozilla’s build system a year ago, in the hope of finding a large code base for testing different build systems in a realistic setup. We settled on Mozilla’s Firefox because their build infrastructure is designed in such a way that different build systems can be plugged in. This is no coincidence, but an ongoing work at Mozilla to update their build system. The rationale being that picking the right replacement for Make would require testing different alternatives, and each alternative would require a complete Fig. 1. Firefox’s build system is split in two phases and can use different port. With a generic build definition, they can test different backends to perform builds. Depending on the intended backend, different build definitions are generated. build systems and let experimentations drive the selection of the next build system. Other considerations, like gaining control over the build definition, and using a widely known their complex codebase. Finally, we accumulated experience language to do so were taken into account when discussing in migrating from one build system to an other, and learned this change [1, 2]. the hard way that the process is theoretically simple but very We started implementing the Tup backend last in August technical in practice. 2017, when Mozilla’s support for it was minimal. It later appeared that Tup was also the next target build system for II. C ONTEXT Mozilla itself, which led to two independent implementations For readers who may not be already familiar with these, we (ours and theirs). Mozilla’s effort to use Tup has focused on briefly depict Mozilla’s build system, Firefox itself and the producing reusable, clean code by fixing one issue at a time. two build systems considered this article: Make and Tup. Our focus was on getting the build to work, regardless of a) Mozilla’s build system: Mozilla’s modular build setup the code quality. Over the time, both implementations have works in two phases, as depicted in Figure 1. moz.build files converged and as of August 2018, Mozilla’s implementation describe the build with a Mozilla-specific python DSL. This should be preferred as it is now complete and is the only one format is parsed and Makefiles are generated in such a way that will be further maintained and updated. that the remaining of the build can be handled by Make. As This paper presents the insights we gained on three main stated before, this design accepts new emitters, or back-ends areas. With this work, we investigated and gained an in-depth to be used in place of the Makefile generator. This is how the knowledge of Tup’s capabilities and interals. We also learned a Tup backend was implemented. Make and Tup are examples lot about Firefox’s build system design and their tactics to tame of backends used for compilation, but the same mechanism — is also used to extract information from the build system to Copyright © by the paper’s authors. Use permitted under Creative Commons be used by other tools. For example, a backend generates a License Attribution 4.0 International (CC BY4.0) compilation database for advanced autocompletion in some In: Anne Etien (eds.): Proceedings of the 12th Seminar on Advanced Tech- niques Tools for Software Evolution, Bolzano, Italy, July 8-10 2019, published IDEs [3, 4]. This flexibility makes it possible to test different at http://ceur-ws.org/ build systems and inspect Mozilla’s Firefox compilation. b) Firefox codebase: Firefox is a libre, open-source again from scratch. A correct build system is one that always browser with a large code base. The core application contained rebuilds what needs to be rebuilt. When an incremental build about 4M lines of code in 2013, and was growing steadily [5]. is complete, the binaries reflect the sources exactly, and the This does not count tests and other configuration files. At that developer need not worry about the build system’s state and time, the full repository contained about 15M lines of code, shortcomings. She therefore can “stay focused on her project”. and has now reached above 36M lines of code as of 2018 [6], To summarize, Tup has three claims, each of which we will which makes it bigger than the Linux kernel and Libre Office try to verify in this paper. according to Open Hub. It is written in a variety of languages. 1) Performance of execution, C++ forms the core application, supplemented by JavaScript 2) Minimal incremental updates, on top of the core engine. The repository contains also HTML, 3) Always correct build outputs. C, python, java and has recently seen the introduction of Rust, Mozilla’s designed systems programming language. A large These features arise from the design and careful implematation code base, with a variety of languages forming an open-source of the algorithms detailed in [12]. They are hard to demonstrate and well known application. All these features makes it a good by experiment as they should hold by design. We have candidate for meaningful build systems comparison. nonetheless tested Tup’s efficiency at building Firefox and c) Make: Speaking of build systems, Make is the refer- gathered the results presented in the next section. ence application in that domain. It was designed at Bell Labs Minimal updates and correctness can be investigated by a in 1976 and was derived and reimplemented many times since deep understanding of how Tup works and have real impacts then [7, 8]. Make’s configuration language used in Makefiles on Tup usage, especially on large software systems like Fire- is well known by a large proportion programmers. Over the fox. It must first be noted that minimal updates and correctness years, Make has seen many disussions about its shortcomings, somehow conflict. Stricter notions of correctness will require and even more discussions about good practices and usage. more rebuilds. While still being minimal, the build system will Most notably, the paper “Recursive Make Considered Harm- rebuild much more than with other build systems, leading to ful [9]” discussed the good usage of Make on large software the impression that it is uselessly recompiling more parts. This projects, advocating for efficiency at the expense of modular- is exactly what happens with Tup. ity. Strinking the right balance between conflicting goals is still a challenge for current system implementations [10]. Make IV. B UILDING F IREFOX WITH T UP has always been the build system of Firefox and is tightly Tup focuses on correctness and speed, at the expense of integrated in its code base. expressivity or usability whenever they conflict with the former d) Tup: From Tup’s home page, we see that “Tup is aspects. In some sense, Tup is minimalist and tries to do one a file-based build system for Linux, OSX, and Windows. It thing well: building. The configuration language of Tup is not inputs a list of file changes and a directed acyclic graph very expressive. It allows to specify commands, inputs and (DAG), then processes the DAG to execute the appropriate outputs with basic support for variables and conditions. To be commands required to update dependent files. Updates are fair, Lua integration is provided for more advanced features performed with very little overhead since Tup implements but was not tested in this article. powerful build algorithms to avoid doing unnecessary work. We will report here first on our experience with Tup, sepa- This means you can stay focused on your project rather than rated in two key aspects: correctness and speed. Expressivity on your build system” [11]. Besides optimization and careful will be discussed later on, in Section VI. implementation, what makes Tup truly special is its ability to trace command executions to obtain real dependencies A. Correctness and outputs. Validating the declared dependencies with the observed file accesses adds a safeguard against imprecise Correctness is a key feature of build systems. It can be declaration and potential speedups that we will discuss later. defined roughly as the property of producing valid products despite optimizations like incremental compilation [10, 13]. In III. R ESEARCH OBJECTIVES Tup, correctness is achieved by maintaining the state of the From the above description of Tup, it is clear that Tup is build in a separate database and by tracing all the file accesses focussed on performance of updates. Detailed explanations on performed by build commands. “avoid doing unnecessary work” and “stay focussed on your Maintaining the state of the build in an external database, project” are to be found later on Tup’s home page or in [12]. allows Tup to detect changes to the build description. When- In a nutshell, avoiding redundant work is the aim of a minimal ever a command is added, removed or modified, Tup will incremental build system. Minimalism ensures that up-to-date take proper actions to update the build when invoked [12]. By outputs are maintained without running the tasks that produce design, this is difficult or impossible to achieve with stateless them. Tup claims that it is minimal in that sense. “Stay build systems like Make. In particular, they cannot detect focused” hides the even more complex idea of correctness. removed commands as there is no trace of them on the next Some build system can get corrupted, or out of sync and it is invocation. This is not specific to Tup. Most (if not all) recent not uncommon for developers to clean their build tree and start build system implementations now rely on persistent storage. 1 tup error: Unspecified output files - A command is writing to files that you didn't specify in ,→ the Tupfile. You should add them so tup knows what to expect. 2 tup error: Expected to write to file 'libxul.so' from cmd 15846 but didn't 3 tup error: Missing input dependency - a file was read from, and was not specified as an input ,→ link for the command. This is an issue because the file was created from another command, ,→ and without the input link the commands may execute out of order. You should add this file ,→ as an input, since it is possible this could randomly break in the future. Fig. 2. Typical errors produced by Tup when inputs and outputs are not properly specified. These errors are reported respectively when a command 1) writes to a file that was not specified as an output, 2) does not write to a file specified as an output and 3) tries to read a file that is not declared as an input. rough super-set of these. By adding all the objects to the same Unlike Make, Tup detects changes to the build com- Tup group linker invocations can depend solely on it, at the mands, triggering the required command invocations. expense of introducing stages in the build, because linking will have to wait for all the objects to be compiled before By tracing file accesses of each and every command in the running. The real dependencies are then obtained at execution build plan (a.k.a. build steps), Tup ensures that dependencies by tracing the command. This ensures accurate detection of and outputs are correctly specified. In particular, Tup will updated inputs despite the over-approximation in the Tupfiles, refuse to build a command that uses undeclared dependencies. and therefore avoids pointless rebuilds of all the libraries when Some typical error messages can be seen on Figure 2. While a single object changes. this allows to detect hidden dependencies and fix them, we often stumbled upon this constraint during our migration Tup’s “groups” feature allows over-approximating the because it interrupts the build, and the Tupfiles need to be dependencies of a build step. generated again. Tup does not allow to bypass these issues. Tracing is used as enforcement of the declared dependencies and for speed optimizations as discussed later. Tup does The technical issue of specifying all the dependencies not dynamically detect and change the build plan from the also arises in a other places such as with unified builds. collected information. The unified builds optimization works by aggregating several For example, we hit this constraint with fake static libraries C/C++ source file together. The compiler only needs to parse (.a.desc files) built by Mozilla in place of the usual static the presumably large set of included headers once for the libraries (.a files) whenever the static library will only be unified set, reducing compilation time on large rebuilds and linked with other libraries, and not exported or otherwise on header changes [14]. This technique speeds up full rebuilds directly used. In that case, the .a.desc file contains only a list significantly at the expense of slower incremental builds for of the objects that should be present in the archive. That list small changes. In this case, the C/C++ compiler receives a is then parsed and the real objects fed to the linker instead of dummy unified input file that includes the original C/C++ the .a archive itself. This avoids the creation of real .a files source files. From the point of view of Tup, the command that can be quite time and disk-space consuming. depends on the dummy unified source file, but also on all the This optimization conflicts with Tup’s need to know all the source files included therein. dependencies of a command explicitly. Whenever a command This situation highlighted for us a specificity of Tup: source links against the .a.desc, it implicitly depends on all the objects files are not required to be specified as inputs. In fact, the listed therein and this lists needs to be explicitly expanded for set of input files is always an implicit input to all the rules. Tup when generating Tupfiles. The rationale being that Tup is able to detect the sources that are accessed by a particular command and source files have no impact on command ordering. The execution order is Tup strengthens the build description by detecting only constrained by generated files, as consumers cannot run undeclared and hidden dependencies. They all must before producers. To relate this to groups, Tup behaves just as be declared in the Tupfiles or Tup will error out. if all the non-generated files belonged to a “sources” group, automatically added as a dependency of all the commands. In Mozilla’s implementation this is now avoided by skipping We experienced this style of never specyfying source files in fake libraries. They are not generated, and linkers are fed rules, and this results in unusual, if not confusing, rules that directly the full list of objects. This is an optimization over have no declared inputs at all. our version. Since the full list of objects needs to be expanded during configuration anyway, there is no reason not to feed it With Tup, there is no need to specify source files as directly to the linker. inputs. All the commands implicitly depend on them. In our first implementation, we avoided this issue by using Tup groups; a feature that allows to some extent to alleviate the need to list all the required dependencies by providing a Tup is also quite pedantic on the outputs that a command is allowed to produce. The command must write to all its declared outputs, and nowhere else. The issue occurred with Tup’s strong correctness guarantees come at the cost Rust code. Due to a lack of integration at the moment, Rust of strict constraints on invoked commands and on the libraries must be compiled in a single step, and therefore build description. There is no free lunch. discard all outputs outside of the expected output library. This prevents Rust from doing caching of builds, or requires to B. Tup performance build Rust libraries outside of the build and outside of Tup dependency tracking. Rust also suffixes libraries with a hash Tup uses a specific data model to achieve high speeds with of their content. To work around these issues, we had to wrap carefully crafted algorithms [12]. We first take a closer look at Rust invocations with a cleanup script that removes extra files the three different aspects of Tup targeting performance, and and renames the generated library. then describe our benchmark on Firefox. First, Tup comes with a monitor that can listen to filesystem events, collecting source changes on the fly. This optimization Tup requires commands to produce the exact list of is nowadays used by all the build systems targetting large code declared outputs, refusing commands that generate bases, like Bazel, Gradle or Pants. The monitor removes the more, less or custom files on different invocations. need to walk the source tree searching for changes on each build system invocation. This optimization is most noticeable Even though these limitations sometimes prevent building for builds in an up-to-date workspace (a.k.a. null builds). In with Tup, it is still possible to trick it in several ways. First, one our tests, it reduced Tup execution time from one second to can invoke some build steps before running Tup. The produced less than a millisecond. This result probably also applies to files are seen as plain source files by Tup. It however removes the build systems cited here-above. Tup’s ability to detect changes and rebuild these files. In the absence of other change detection mechanisms, these files need When everything is up-to-date and Tup’s monitor is to be rebuilt on each invocation. The other option is to build enabled, Tup runs in less than a millisecond. outside of the source tree, because Tup does not track files there (unless configured to). The required outputs can then be moved to the source tree once the command finishes. With this Also, Tup maintains the build graph in such a way that technique, complex commands can maintain a cache across changes and rebuilds can be propagated upwards, without builds. This is otherwise not possible with Tup constraints. The loading the whole dependency DAG [12]. Theoretically, this potential pitfall is that build correctness relies completely in makes Tup very efficient at small rebuilds because the com- the sub-command, as Tup has no knowledge of what happens plexity of the algorithm depends mostly on the size of the outside of it’s monitoring. Reaching that point means that Tup update, and less on the size of the whole repository as is the gets in the way more than it should. Commands such as Rust case with Make. incovations require structural modifications to work properly Finally, Tup’s build graph is enriched with exact dependen- with Tup. Mozilla’s implementation was defered until they cies detected by tracing the commands durig their execution. updated the Rust toolchain to better support Tup [15]. This This ensures that build steps do not depend on files they do was made possible because Rust already intended to support not need, and that there is no spurious rebuilds when such a that usage [16], but may not be feasible with other projects or file changes. That being said, tracing has also a performance affordable to any Tup user. penalty, and can reduce Tup’s speed on filesystem-intensive tasks like linking Firefox’s largest binary [17]. We ran a small-scale experiment to compare build speed Some tools are structurally conflicting with Tup. They with Make and Tup. We selected 47 consecutive pushes to have to be modified or Tup needs to be bypassed. Firefox1 and compiled these changes incrementally in merge order. The pushes were taken from mozilla-inbound, the The complications induced by the strict enforcement of integration branch at Mozilla. A push is a set of commits policies by Tup ultimately provide strong guarantees on the that are added together, and tested as one atomic change validity of the build results. While it is possible to inadver- by Mozilla’s continuous integration system. The number of tently omit a dependency in Make, it is nearly imposible to commits is limited to 47 because it is the largest range do so in Tup. This makes building with Tup more robust to of consecutive commits that appeared to build correctly in configuration errors or undetected interferences. In the long the benchmarking enviroment. Building more commits would term, the balance leans in favor of strict build systems like have required to use different build environments, or looking Tup because migrating to Tup requires solving tricky issues at more commit ranges. With 47 commits, we were able only once, while an incorrect build system can produce subtle to measure the duration of 46 different incremental builds. inconsistencies between the source code and the build products Figure 3 shows the incremental build times of Make and Tup during any build invocation. 1 The full list of 47 pushes is accessible at https://hg.mozilla.org/integration/ mozilla-inbound/pushloghtml?startID=102811&endID=102858 Fig. 3. Duration of three incremental builds for 47 successive changesets with both Make and Tup not visible to the mozbuild frontend. 1h We get a more detailed view of Make and Tup’s relative performance on Figure 4. Average build times with Make (on the left) are compared with average build times with 10min Tup (on the right) for each of the 47 commits. The graph Build duration is in logarithmic vertical scale, so it is difficult to get precise values, but we see that the builds are indeed faster by a similar 1min proportion for most large builds, with Tup being slightly faster. Looking at smaller rebuilds, we see that they get faster with Tup, and sometimes much faster. This high speedup arises 10s when Tup detects that there is nothing to do. In that case, Tup takes one second to search for modifications (and possibly even faster with its monitor running, as explained above), and 1s then stops after deciding that no modifications means nothing to compile. Make on the other hand needs to recurse into all Make Tup the subdirectories and build the whole dependency graph to Build backend detect the absence of anything to do. It further appears that Make executes some build steps on each invocation, making it Fig. 4. Relative speed of Make and Tup at building incremental changesets. impossible to measure the execution time of the Make process The build duration of each changeset with each tool has been averaged over itself without the tasks being run. Unconditionally running the three runs. Durations were rounded up to the next second. tasks means that some were forced to run, most probably to avoid issues with incomplete dependency handling in Make. for each of these 47 commits. Three runs were made for each From that point of view, Make’s lack of correctness guarantees build system to give an idea of the variance. The first measure translates into an execution overhead. represents a full rebuild from clean sources, and shows how much time is needed for a full build of Mozilla’s Firefox. With Tup is at least as good as Make on average, and much the default Make backend, a bit more than 1h 15m are needed faster for small rebuilds. to do so on our Intel Core i5-4310U which is a fairly recent 2 cores/4 threads CPU. All the builds were performed on the same machine, with the same SSD drive. On the graph, we observe a lot of short builds, thanks to V. G RASPING M OZILLA’ S BUILD DESIGN the incremental optimizations performed by Tup and Make. Also, the data series have the same shape, which shows that Another feature of Tup is that it allows easy extraction of both backends compile about the same things. That being the build graph. Tup contains in its database the whole build said, Make builds are generally slower than Tup ones on big graph that was just built. It also provides a tool to draw the rebuilds. Thanks to the three measures, we can affirm that graph. it is no random artifact of the build environment. The most The complete graph is not easy to understand, as it contains probable cause is that the Make build performs more steps thousands of nodes, one for each command and for each than the Tup build. In some places, the Tup build avoids calls file. We took some steps to simplify the graph based on to wrappers and other fixtures used by Make. But it could also command labels and relative depth in the build tree to obtain come from commands that are only described in Makefiles and the simplified version presented in figure 5. python dependentlibs.list LINK LINK 7 commands 21 commands 7 files 21 files LINK AR LINK AR LINK 3 commands 2 commands 4 commands libxpcomglue.a libnspr4.so 3 files 2 files 4 files CXX CC AS RUSTC 1670 commands 1144 commands 127 commands libgkrust.a 1670 files 1144 files 127 files python WebIDL AS XPIDL 1 command 1 command 4 commands xptdata.cpp 2 files 1457 files 4 files python python XPIDL 1 command 5 commands 152 commands 2 files 11 files 2633 files IPDL preprocess python 1 command 109 commands 70 commands 966 files 109 files 1261 files sources 247373 files Fig. 5. A simplified view of Firefox dependency graph. The node grouping algorithm outlines different stages of preprocessing, compiling, linking and the final step of producing dependentlibs.list discussed in Section V. Nodes are grouped by type and by relative depth in the build system. Transitive dependencies have been removed to avoid edges clutter, hiding in particular the fact that all the nodes depend directly on the sources. 909M gecko-dev/.tup (Tup internal state, Tup stores the whole build plan (the build graph) in including and mostly due to Tup db) 2.2G gecko-dev (sources checkout only) a database in an explicit form that can be queried 4.9G gecko-dev/.git (git repository) easily. 5.8G gecko-dev/obj-tup (build outputs) 14G total It is interesting to discover that the build can be summarized Fig. 6. Size of the main parts of our workspace after a complete build with in a handful of nodes, and divides itself naturally in stages. Tup in a clean checkout. There is a preprocessing and generation stage where symlinks are created, where files are generated with python and other custom transformations happen. Afterwards, we observe the stage, where shared objects depend on other shared objects. classical two-stage process of compiling (AS, CC, CXX) to The graphs could be improved through better projections and object files, and then linking into programs, static libs and simplifications. We intend to work further on these aspects to shared libs (LINK and AR). The ultimate stage is an oddity at improve on this work and generalize our results. Mozilla, where a dependentlibs.list is generated that lists all Extracting that graph would not have been easy with the libraries that libxul.so depends upon. This file is used to recursive Makefiles. There have been attempts at extracting preload all these libraries at startup, possibly for performance Makefile-based build system structure but they require to reasons. actually run Make in verbose debug mode and parse its This graph was the best result obtained after several at- output [18]. Tup provides a much more reliable way to access tempts. It groups commands by the maximal depth of their that graph. inputs. Files are merged with the command that produces Concerning that graph, it is worth noting that it requires them. This kind of transformation does not preserve the a large amount of disk space. On our machine, after a fresh acyclicity of the graph, as a loop appears in the linking build of Firefox, we observed the memory usage described in EXPORTS += [ with Files('**'): 'MP3Decoder.h', BUG_COMPONENT = ('Toolkit', 'App Update') 'MP3Demuxer.h', 'MP3FrameParser.h', DIRS += ['src'] ] if CONFIG['MOZ_ENABLE_SIGNMAR']: UNIFIED_SOURCES += [ DIRS += ['sign', 'verify'] 'MP3Decoder.cpp', TEST_DIRS += ['tests'] 'MP3Demuxer.cpp', elif CONFIG['MOZ_VERIFY_MAR_SIGNATURE']: 'MP3FrameParser.cpp', DIRS += ['verify'] ] # If we are building ./sign and ./verify, FINAL_LIBRARY = 'xul' # then ./tool must come after it DIRS += ['tool'] Fig. 7. A simple moz.build file (dom/media/mp3/moz.build) showing how data can be encoded in this restricted python language. This format is used Fig. 8. A more advanced moz.build file, with conditionals, subdirectories and by Mozilla to describe the build steps that need to be performed. bindings between source files and Firefox modules. Figure 6. We see that the internal state requires an amount of space comparable to the source checkout itself. From that the build options can depend on configuration values quick investigations, we discovered that it is mainly due to passed in the CONFIG dictionary. These values are computed dependencies between build steps. further investigations would at configuration time, and allow tweaking the build depending be required to analyze how this database grows with repository on the platform and various other parameters. sizes and programming languages, or possibly other significant The Python format used in moz.build files was picked factors to be discovered. because it is easy to read and Python is well known among The ability of Tup to inspect the build graph comes as a developers, at Mozilla and in general [2]. This input format is nice side-effect of using Tup as a build backend. It shows parsed by the mozbuild toolchain that generates the right build how using different backends can benefit Mozilla outside of files according to the selected backend (Makefiles for make, building per se. If Tup ends up not being used by Mozilla, etc.) but it can also perform some complex operations on its it remains useful for its easily inspect able build graph and own. For example, the backend generates boilerplate unified also for checking the dependencies declared in the moz.build C++ files, and lists of inputs in separate files to circumvent frontend files. command-line length limits. The Tup backend performs more Besides backends for building, this Mozilla-specific ar- operations to work around Tup limitations. For example, it chitecture opens the door to other tools instrumenting the parses some input files to pre-compute the output file names build system. For example, Mozilla already uses that build that cannot be simply deduced. We call this step preprocessing, infrastructure to extract a label for each file, in order to and distinguish it from the parsing and generation done by automatically relate code changes to Firefox components. It is mozbuild. Preprocessing is mostly needed to work around also used to generate compile-commands.json used by linters limitations and restrictions in the build backend. and IDE compilation databases [3]. Other tools such as linters and software maintenance tools could be plugged in there, With its custom build description format in Python, Mozilla making it low hanging fruit for a practical use case of new is independent from a build system in particular. Because research tools. Python is a widely known and general purpose programming language, it is possible to adapt it to multiple purposes and VI. F IREFOX AS A BENCHMARK FOR BUILD SYSTEMS in particular to generate instructions for different backends as As was already hinted before, Firefox is our candidate we did with Tup. As more backends are added, the difficulty benchmark for comparing build systems. In this section, we to generalize the mozbuild pipeline should lower. Also, in the explain in further details the structure of Mozilla’s build sys- process of implementing a backend for a build system, features tem and the features that catched our attention while looking and limitations of said build system appear more clearly and for a benchmarking project for build systems. can be collated. With several build systems executing on the Mozilla’s build system is based on simplified python files same code base, it becomes possible to produce the meaningful named moz.build. Examples of these files can be seen in comparisons that are not yet available. Our future work will Figure 7 and Figure 8. They list the source files that must therefore target more build systems and will try to elicit be compiled, the name of the libraries or programs that must distinctive features for each of them. be produced and various options like compiler flags, headers to be exported and such. The ad-hoc nature of this data Mozilla’s custom build framework can be easily format allows Mozilla to encode other kind of information adapted to different build systems, and could be used like in Figure 8 where information allows to relate files in the to compare them. source tree with Firefox components. That example also shows VII. R ELATED W ORK X. ACKNOWLEDGMENTS The authors would like to thank Mike Shal for sharing Andrey Mokhov et Al. distinguish different kinds of build knowledge about both Tup’s and Mozilla’s build systems as systems [13]. In Mozilla at the moment, there is a recursive well as for his careful review of a previous version of this memo/dumb configuration and preprocessing phase. This one article. Many thanks too to Marie Champenois for producing is followed by a minimalist build system in the common and a better looking Figure 1. strict sense. While not perfect, combining different kinds of build systems provides the ability to work now, and to be R EFERENCES perfectible later. This is the approach used by Mozilla in their [1] Gregory Szorc. Moving Away from Makefile’s. E-mail. build system, and makes for a simplified migration, as it allows Aug. 22, 2012. URL: https://groups.google.com/forum/ maintaining two build systems up-to-date side-by-side. #!topic/mozilla.dev.platform/SACOnl-avMs/discussion. Gligoric et Al. worked on migrating from one build system [2] Gregory Szorc. Moz.Build Files and the Firefox Build to another [19]. They extracted build traces from the old one System. Feb. 2013. URL: https://gregoryszorc.com/blog/ and populated the new build system by factoring out common 2013 / 02 / 28 / moz . build - files - and - the - firefox - build - patterns. Reusing this approach would not take advantage of system/. the existing moz.build architecture used at Mozilla. There is [3] The Clang Team. JSON Compilation Database For- no need to extract traces when the graph is available. mat Specification. URL: https : / / clang . llvm . org / docs / JSONCompilationDatabase . html (visited on 09/21/2018). VIII. R EPRODUCIBILITY [4] Joshua Cranmer. Mach Command to Output Clang This work was performed with the intent to make it fully Compilation Database. URL: https://bugzilla.mozilla. reproducible. The development environment was managed org/show bug.cgi?id=904572 (visited on 09/21/2018). with the Nix package manager in order to allow a completely [5] Ali Almossawi. How Maintainable Is the Firefox Code- reproducible test environment [20]. All the work is available at base? May 15, 2013. URL: http : / / almossawi . com / https://github.com/layus/gecko-tup. The results presented here firefox/prose (visited on 10/05/2018). are all reproducible from the tools provided there. [6] The Mozilla Firefox Open Source Project on Open Hub: Languages Page. URL: https : / / www. openhub. net / p / This also means that our tests were performed within Nix2 , firefox/analyses/latest/languages summary (visited on which imposes some strong constraints on the usual Linux 10/05/2018). builds. Some patches were introduced to circumvent these [7] Stuart I. Feldman. “Make — a Program for Maintaining constraints. This does not invalidate the final results because Computer Programs”. In: Software: Practice and Expe- Firefox was compiled just fine. The issues were mostly with rience 9.4 (Apr. 1, 1979), pp. 255–265. ISSN: 1097- passing custom library paths to a build system that assumes 024X. DOI: 10.1002/spe.4380090402. that all the system libraries comply to Filesystem Hierarchy [8] Richard M. Stallman and Roland McGrath. GNU Make Standard (FHS) [21], which is not the case with Nix. - A Program for Directing Recompilation. 1991. [9] Peter Miller. “Recursive Make Considered Harmful”. IX. C ONCLUSION In: AUUGN. Vol. 19. AUUGN Journal of AUUG Inc 1. AUUG, Inc., Feb. 1998, pp. 14–25. We have conducted an experiment consisting in building [10] Guillaume Maudoux and Kim Mens. “Correct, Efficient, Firefox with Tup. It has shown that Tup is a valid replacement and Tailored: The Future of Build Systems”. In: IEEE for Make on this large real-life software project. While Tup Software 35.2 (2018), pp. 32–37. does not generally bring a significant speedup at building, [11] Mike Shal. Tup Build System. URL: http://gittup.org/ its strength resides in correctness and enforcement of build tup/index.html (visited on 05/15/2018). specifications. We argued that it may be a good investment [12] Mike Shal. Build System Rules and Algorithms. 2009. in the long term, despite the significant refactoring that its URL : http : / / gittup . org / tup / build system rules and introduction requires. Tup is however more efficient than Make algorithms.pdf. at compiling small changesets. It could spare compilation time [13] Andrey Mokhov, Neil Mitchell, and Simon Peyton if such rebuilds are frequent, as could be expected during Jones. “Build Systems à La Carte”. In: ICFP (2018). development on a developer’s machine. Beyond its usage as a URL : https : / / www . microsoft . com / en - us / research / build system, Tup also provides a straightforward data model uploads/prod/2018/03/build-systems.pdf. that allows in-depth inspection of the build. In the future, we [14] Ehsan Akhgari. Unified Builds. Nov. 2013. URL: https: intend to reuse the knowledge and tooling presented here with / / lists . mozilla . org / pipermail / dev - platform / 2013 - Tup to compare more build systems, and look for relevant November/001998.html. metrics to classify them. [15] Add –Build-Plan for ’cargo Build’ by Mshal · Pull Request #5301 · Rust-Lang/Cargo. URL: https://github. 2 https://nixos.org/nix/ com/rust-lang/cargo/pull/5301 (visited on 10/18/2018). [16] Support Producing a ”Build Plan” without Executing Anything · Issue #3815 · Rust-Lang/Cargo. URL: https: //github.com/rust- lang/cargo/issues/3815 (visited on 10/18/2018). [17] Mike Shal. Linking Libxul with Tup and FUSE. Apr. 2013. URL: http://gittup.org/blog/2013/04/3- linking- libxul-with-tup-and-fuse/. [18] Bram Adams et al. “Makao”. In: Software Maintenance, 2007. ICSM 2007. IEEE International Conference On. IEEE, 2007, pp. 517–518. [19] Milos Gligoric et al. “Automated Migration of Build Scripts Using Dynamic Analysis and Search-Based Refactoring”. In: ACM SIGPLAN Notices 49 (Dec. 2014), pp. 599–616. [20] Rok Garbas. Reproducible Development Environments. Sept. 2015. URL: https://garbas.si/2015/reproducible- development-environments.html. [21] LSB Workgroup, The Linux Foundation. Filesystem Hierarchy Standard, Version 3.0. Mar. 19, 2015. URL: http://refspecs.linuxfoundation.org/FHS 3.0/fhs/index. html.