Making CodeCity evolve David Moreno-Lumbreras Jesus M. Gonzalez-Barahona Universidad Rey Juan Carlos and Bitergia Universidad Rey Juan Carlos Madrid, Spain Madrid, Spain dmoreno@bitergia.com jgb@gsyc.es Valerio Cosentino Bitergia Madrid, Spain valcos@bitergia.com appropriate level of granularity. It depicts object- oriented software systems as habitable [2] cities that Abstract one can intuitively explore. Codecity settles on the city metaphor because it offers a clear notion of locality, CodeCity is a software analysis tool with the thus supporting orientation, and features a structural goal of visualizing software systems as interac- complexity that cannot be oversimplified. Codecity tive, navigable 3D cities. They rely on the city represents classes as buildings located in quarters rep- metaphor, that uses the layout of the city in resenting the packages where the classes are defined. order to visualize different metrics about soft- The metrics that Codecity uses are the number of ware systems. There were other implementa- methods (NOM) mapped on the building’s height and tions of CodeCity among the years but there the number of attributes (NOA) on their base size, and are no modern options that contemplate the for packages, the nesting level mapped on the quarter’s representation of time evolution of the city. color saturation: We propose a solution to this time evolution, developing a web version of CodeCity that works with any device that has a web browser. Then, we analyze different projects and show the time evolution of their cities. Finally, we conclude with some needed features and draw- backs that are going to be considered in future works. Index terms— CodeCity, data visualization, vir- tual reality, web, 3D 1 Introduction Codecity [1] is an approach of a 3D visualization which Figure 1: Example of the Jmol Java package city ana- creates cities that look real, due to the combination lyzed with Codecity of layouts, topologies, metric mappings applied at an One of the known limitations of this approach is Copyright © by the paper’s authors. Use permitted under Cre- that CodeCity is developed using an old framework ative Commons License Attribution 4.0 International (CC BY 4.0). (SmartTalk) and it does not have support for sev- In: D. Di Nucci, C. De Roover (eds.): Proceedings of the 18th eral devices. We rely on web development and we Belgium-Netherlands Software Evolution Workshop, Brussels, are going to solve this limitation developing a version Belgium, 28-11-2019, published at http://ceur-ws.org of CodeCity using web technologies and open source 1 software in order to make the tool more universal and evolution of the treemap. every device that has a web browser could visualize these 3D cities. In the next sections, we detailed the 3 Our approach: BabiaXR and Web development and the technologies used. CodeCity with time evolution On the other hand, in terms of interaction, we ar- gue that this visualization type can get more value and 3.1 BabiaXR Visualizations better comprehension adding it a Virtual Reality en- vironment, allowing the user to tour and navigate into There are different technologies based on WebGL that the city. are focused on making 3D scenes in the browser. Specifically, we rely on A-Frame1 , a web framework Another of its limitations is that Codecity doesn’t for building 3D and VR scenes for the web, for the vi- follow a fixed layout for its blocks/quarters, it means sualization render engine, developing, thus a new ver- that every time a code city is generated, the layout and sion of CodeCity that works in any device that has a the position of a package could change. This would web browser and the needed standards included (the be an important drawback in terms of time evolution most important, WebVR), A-Frame is developed on analysis because the city would change every single top Three.js2 . time that it’s generated. We are going to solve that, Before the development of CodeCity, we got used making the city evolve among the time, increasing and to making some usual 3D visualization in this web decreasing the area and height of the buildings as the 3D environment. Starting with BabiaXR3 , a Git- metrics represented changes among the time, fixing the Lab/GitHub organization where the developing of the position of the buildings, fixing the same position of different visualizations are allocated. BabiaXR has each building while the time snapshot changes. the aim of aggregate different components, using A- Moreover, Codecity is strongly bonded to ana- Frame, that can create different types of chart in a lyze the program structure. We argue that the city modern browser, the web is a universal environment metaphor can be more than just analyze code, like and any device that has a modern browser that sup- analyze the contributions, users, and other kinds of ports WebVR can visualize the charts that the Babi- metrics related to the community. This is one of the aXR components produce, making it more universal future works (section 4) that is going to be analyzed. and easy to use. The first step of BabiaXR was to make the most common visualizations, the pie chart, 2 Related works the 3D and 2D bar chart, and the bubbles chart. The development of these visualizations is separated in We are going to separate the related works in two cat- some A-Frame components, geosimplebarchart for 2D egories, one related to the implementations of the city bar chart, geo3dbarchart for 3D bar chart, geopiechart metaphor and the other related to the time evolution for pie chart and geobubbleschart for bubbles chart. implementations about treemap algorithms. Regard- Moreover, there are components related to data man- ing the city metaphor implementations, Juraj Vin- agement, these components can query, filter data and cur et all. [3] propose a Virtual Reality city for an- add behavior to the visualization components. There alyzing software, made it with non-web technologies, are two components that query data, querier json for Steinbrückner and Lewerentz [4] propose stable city querying JSON files and querier github that query the layouts for evolving software systems, using a different GitHub API. The component filterdata can filter the layout than a treemap. Getaviz [5] is another tool that data retrieved from a querier component and the com- uses the city metaphor in order to generate structural, ponent vismapper that maps the data filtered by a behavioral, and evolutionary views of software systems filterdata component to geometry attributes of the dif- for empirical evaluation. In terms of interaction with ferent charts, it ”prepares” the data and save it in the the city, one good example is CityVR [6] that uses the entity that it is defined. same metrics as the original Codecity but adds interac- tions with a VR headset using the controllers and the In the repository, there is a user guide where all the sight direction. Regarding the evolution of a treemap information and the steps in order to create a 3D and layout, Willy Scheibel et all. propose EvoCells [7], a VR dashboard are defined. treemap layout algorithm for evolving tree data, using One example of these charts is the one shown in 2: rectangular areas. Another field of treemap evolution One of the advantages of A-Frame is that extends research is the Voronoi treemaps, these treemaps have HTML, so using just two sentences of HTML with a not rectangular areas, making the treemap more flexi- JSON data can generate a 3D and VR visualization in ble in order to evolve the areas, Avneesh Sud et all. [8] 1 https://aframe.io/ proposes a dynamic Voronoi treemap algorithm that 2 https://threejs.org/ explores this dynamic changes that can be used as an 3 https://github.com/babiaxr 2 Figure 2: Usual visualizations of BabiaXR the web environment. For example, the next HTML Figure 4: BabiaXR CodeCity examples code and JSON data will generate the figure 3: we defined a unique identifier for each building in or- 1 2 ... der to change its area and height in the time evolu- 3 < / a-entity> ... limitations that the original CodeCity has. In terms 6 < / a-scene> of interaction, A-Frame includes by default the possi- bility of entering the scene in Virtual Reality mode, 1 2 [ { " key " : " David " ," key2 " : " 2019 " ," size " :9} , therefore, the functionality of clicking a quarter with 3 { " key " : " Pete " ," key2 " : " 2011 " ," size " :8} , the mouse or a controller if it using a VR device has 4 ...] been implemented in order to show more information about the building/quarter clicked/hovered, showing the name of it as a title on top of it. 3.3 Creating a City We separate the creation of a city corresponding of a software system in two parts, the first one is related of how we retrieve the data that we are going to visual- ize as a city, we analyze a repository using Graal [10], Graal leverages on the Git backend of Perceval [11] and enhances it to set up ad-hoc source code analy- Figure 3: 3D bar chart generated with BabiaXR com- sis. Thus, it fetches the commits from a Git repos- ponents itory and provides a mechanism to plug third-party tools/libraries focused on source code analysis. More- over, Graal can retrieve similar metrics as the origi- 3.2 BabiaXR Codecity nal CodeCity, we use Graal to obtain analyze projects The CodeCity version of BabiaXR is developed with using code complexity analysis (CoCom)4 , retrieving the goal of exploring the known limitations of the orig- metrics like the number of lines of code or the num- inal CodeCity, the approach is also inside the compo- ber of functions that the file code has. Graal stores nents pack, specifically, the components related to the the data in an ElasticSearch database, once the data CodeCity visualization are codecity-block for blocks is there, we run a Python code included in the Babi- and codecity-quarter for quarters. The first step in aXR components repository in order to get the JSON order to create this visualization is the layout selec- with the data that will be injected in the CodeCity tion, we rely on the treemap pivot algorithm [9] for component of BabiaXR, summarizing: the blocks and quarters positions, it shows a consis- tent layout and a desired aspect-ratio of the building 1. Run Graal specifying the ElasticSearch database area (making them the best ”real building” possible and the repositories that are going to be analyzed. appearance). 2. Run the code generate structure codecityjs.py Moreover, there is an option for changing the ap- with the time evolution flag and the snapshot ar- pearance of the buildings for real building models load- gument filled and it will return a set of JSON data ing gTLF models, making more realistic cities and im- with all the information needed of the city. proving the notion of locality. As A-Frame extends HTML and each building is defined as an HTML tag, 4 https://github.com/chaoss/grimoirelab-graal#backends 3 Once we have run the previous code, in the set of the project and the quarters are defined as the tree of JSON files returned, there is one that has the folders that the file belongs to. As metrics, we rep- starting information about the city evolution, called resent the lines of code as the height of the buildings main data.json, it has a structure that defines the data and the area represents the number of functions that files needed of the time evolution, the sampling days the file has. Moreover, the figures represent a selected that have been used and the time field where the evo- time snapshot, starting with the time snapshot of the lution has been made: 5th of November, 2019. Then going back 6 months 1 { (the 5th of May, 2019) and going back 18 months (the 2 " date_field " : " field " , 3 " sampling_days " : " 180 " , 5th of May, 2019). The demo available in the repro- 4 " init_data " : " data_X_tree.json " , duction package6 gives more information because there 5 " time_evolution " : true , 6 " data_files " :[ are more time samples and it is possible to see the 7 { 8 " date " :1573001804.136289 , evolution of the city in a completed and visual way. 9 " file " : " data_X.json " 10 }, 11 ... 12 ] 13 } This file is the starting point and has to be defined as the main data when the component is written in HTML. The data files follow the next structure, start- ing for the main point, that has all the needed build- ings and quarters tree in order to create the city: 1 [{ 2 " block " : " name " , 3 " blocks " :[ 4 { 5 " block " : " name_child " , 6 " items " :[ 7 { Figure 5: Angular CodeCity of the 5th of November, 8 " id " : " file_path " , 9 " area " : 1 , 2019 10 " height " :2 11 }, 12 ... 13 ] 14 }, 15 ... 16 }] Then, the next files just have a list with the new values of the buildings: 1 [ 2 { 3 " id " : " file_path " , 4 " name " : " file_name " , 5 " height " : 1 , 6 " value " : 2 7 }, 8 ...] Then, just creating a simple HTML file, loading the Figure 6: Angular CodeCity of the 5th of May, 2019 dependencies needed (from A-Frame, BabiaXR com- ponents and the file time evol.js), and defining the pa- rameters of the codecity-quarter component, we can see the city of the project analyzed evolving among the time: 1 2 < / a-entity> 3 ... 4 < / a-scene> 4 First Applications The following figures (5, 6, 7) represent the analysis of the project Angular5 . Each building represents a file Figure 7: Angular CodeCity of the 5th of May, 2018 5 https://github.com/angular/angular We can observe how the city evolves and how the 4 buildings decrease their area and some of them dis- On the other hand, all the metrics seen were met- appear, as this is an evolution from present to past, rics related to modules or packages. Thus, the next some of the files maybe not in the past version, so the step is to look for the right way to represent different space that they filled is changed to empty terrain. For kind of data further than software systems structure, example, focusing on the center of the city, the central using different kinds of metrics. Specifically, we are quarter is related to some packages that Angular in- going to deep in other software development metrics. cludes in their code, if we observe the evolution, there For instance, the distribution of the commits/reposi- are some buildings that disappear, those buildings are tories, the relationships between the people, the time related to some packages that have been included in evolution of the community, etc. Therefore, we have Angular in the latest version, their space is not filled to face the challenge of move this different kind of data in older versions. to a city, thinking about what will represent the build- There is a reproduction package6 available in order ing, the districts, etc. Making the representation easy to reproduce all the steps in a controlled environment. to understand and easy to answer questions that the common visualizations don’t do. Discussion Finally, another research branch is to approach other kinds of metaphor apart from the city, for ex- The proposed solutions to the known limitations of ample, islands and planets. the original CodeCity has been developed success- fully. Using the HTML identifiers of each building Acknowledgment fixes the non-fixed location of the buildings, making the same location of each building as the city evolves, This work was supported by the Spanish Government just changing the area and the height of each building (IND2018/TIC-9669). in each time snapshot. In terms of the interaction, the possibility of adding Virtual Reality to the city and the References development using web technologies solve the limita- [1] R. Wettel and M. Lanza. Visualizing software sys- tion of the interaction of the original CodeCity, mak- tems as cities. In Proceedings of VISSOFT 2007 ing the BabiaXR version more universal and able to (4th IEEE International Workshop on Visualiz- use in any device that has a modern web browser that ing Software For Understanding and Analysis) , supports WebVR. On the other hand, we are analyz- pages 92–99, 2007 ing similar software metrics of the original CodeCity, it needs to add effort to this research and changes the [2] R. Wettel and M. Lanza. Program comprehension type of metrics analyzed to new ones that make sense through software habitability. In Proceedings of to represent using the city metaphor. 15th International Conference on Program Com- prehension (ICPC 2007). IEEE Computer Soci- Conclusion and Future Work ety, 2007. This approach is a proof of concept of a tool that has [3] Vincur, J.; Navrat, P. ;Polasek, I. VR City: Soft- to evolve. We were focused on replicate CodeCity in a ware Analysis in Virtual Reality Environment. modern and open source environment in order to make 2017 IEEE International Conference on Soft- it more universal and easy to use, in this case, the web ware Quality, Reliability and Security Companion environment, and we were focused on the time evolu- (QRS-C), 2017. tion feature. The first versions of this tool are just the beginning and it needs more effort on the interaction [4] F. Steinbrückner, C. Lewerentz. Representing de- part, we propose to add more functionality to the user velopment history in software cities, 2010 5th in- interface, for example, adding two types of cities, one ternational symposium on Software visualization smaller as mock-up in order to see the entire city in a (SOFTVIS), New York, USA, 2010, pp. 193-202. small space and take advantage to the location of the modern VR glasses. And the other one is to make a [5] Baum, D.; Schilbach, J.; Kovacs, P.; Eisenecker, city as the same ”size” as an actual city and tour the U.; Müller, R. Getaviz: Generating Structural, user into it, for example, driving a car or even flying. Behavioral, and Evolutionary Views of Software Other future work could be the feature of adding more Systems for Empirical Evaluation. 2017 IEEE information about the software in the city, with more Working Conference on Software Visualization user interaction than click/hover. (VISSOFT), 2017. 6 https://gitlab.com/thesis-dlumbrer/repr-pckg-benevol- [6] Merino L Ghafari M Anslow C Nierstrasz. 2019 CityVR: Gameful Software Visualization. In- 5 ternational Conference on Software Maintenance VIS), 2001 and Evolution (ICSME), 2017 [10] Valerio Cosentino, Santiago Dueñas, Ahmed Ze- [7] Willy Scheibel; Christopher Weyand; Jürgen rouali, Gregorio Robles, Jesus M Gonzalez- Döllner. EvoCells – A Treemap Layout Algorithm Barahona. Graal: The Quest for Source Code for Evolving Tree Data. 9th International Con- Knowledge 2018 IEEE 18th International Work- ference on Information Visualization Theory and ing Conference on Source Code Analysis and Ma- Applications (IVAPP), Madeira, Portugal, 2018 nipulation (SCAM), 2018 [8] Avneesh Sud; Danyel Fisher; Huai-Ping Lee. [11] Santiago Dueñas, Valerio Cosentino, Gregorio Fast Dynamic Voronoi Treemaps. Seventh In- Robles, and Jesus M. Gonzalez-Barahona. Perce- ternational Symposium on Voronoi Diagrams in val: Software project data at your will. Proceed- Science and Engineering, (ISVD 2010), Quebec, ings of the 40th International Conference on Soft- Canada, June 28-30, 2010 ware Engineering: Companion Proceeedings, 2018 [9] Ben Shneiderman M; Ordered Treemaps Lay- outs. Information Visualization, 2001. (INFO- 6