This paper presents analysis tools for evaluating crowdsourced geometry-based design proposals for urban planning. With the Quick Urban Analysis Kit, an online platform, citizens are able to manipulate objects and create a preferred layout over a case study area. Given that our case study is on a meso-cale, our analysis is focused on the layout and plot configuration. The proposed analysis tools range from simple counting of object types and a buffer analysis to clustering and spatial autocorrelation tools. Besides these form-based criteria, perception-based criteria are also proposed to link the participating subject's assessment of the designs with the layout. Techniques deployed include supervised machine learning methods, statistical spatial tests, and simple calculations of the area size and frequency of objects.
In recent decades, the contribution of citizen participation has led to the improvement of democratic governance and other adjacent fields. These contributions are thought to establish a sense of citizenship, increase positive attitudes and strengthen responsive and accountable states [Pateman, 1970; Mansbridge, 1997; Gaventa & Barrett, 2010]. Conversely, scholars are also skeptical about the perceived autonomy of citizen participation, citing external influences such as elite capture, lack of civic capacities, or other factors [Bonfiglioli, 2003; Golooba-Mutebi, 2004; Banerjee et al., 2010]. We propose new forms for citizen participation in the urban planning process. Two main factors are important in our vision. Firstly, the hurdles for participation must be as low as possible. Any discussion that requires citizens be physically present are exclusionary, and do not leverage on contemporary communications technologies to include as many citizens as possible for more representative feedback. Secondly, tools must be provided to facilitate the creation of creative solutions to problems, and in particular design proposals. Surveys and voting systems are inadequate for this purpose.
American designer and social scientist Elisabeth Sanders wrote about how people can contribute as co-designers [Sanders, 2002]. In order for this contribution to happen, however, a designer must faciliate access to the experience of the user. For Sanders, people express these experiences by talking, thinking, doing, using, knowing, feeling and dreaming. While the first four activities are explicit and observable, the latter are more tacit and latent. To access these levels of experience, she proposed applying “make tools”. “Make tools” enable people to express themselves in many ways. For instance, cognitive toolkits that help people create maps and 3D models can show how they perceive and understand a place, as such tools force people to think and express themselves in novel ways. We elaborate on Sanders’ concept and use make tools in the participatory urban design process for the layperson. This combination of citien science and design is what we name ‘Citizen Design Science’ [Mueller et al., 2017].
Crowdsourced participatory urban design may be regarded as a specific case in collective intelligence. Previous work on collective intelligence is divided into two parts, participatory design and crowdsourcing, with the former practiced in design communities and the latter researched in urban computing [Peffers et al., 2007]. Yu and Nickerson [Yu & Nickerson, 2013] integrate the two domains, namely human intelligence with machine processes, and postulate a crowdsourced idea generation process that facilitates the combination of ideas. Our work also combines the two domains but we focus instead on design evaluation, since our platform already enables effective idea generation from the crowd.
This paper describes some options for evaluating crowdcreative design proposals for redeveloping urban areas. We take a township in South Africa as a case study area. Using a 3D geometry viewer and editor we explain how a Citizen Design Science project on meso-scale can be conducted and analysed.
Quick Urban Analysis Kit (qua-kit) was developed by Artem Chirkin at the Chair of Information Architecture at the ETH Zurich [Chirkin & Koenig, 2016]. The software interface is an online viewer retrievable via http://quakit.ethz.ch/viewer. This viewer can show 3D objects which are either movable or static. The main function of this platform is the manipulation of object positions in two dimensions, including rotation. It is not possible to place blocks on top of one another. The user can make modifications with the left mouse button, right-click to change the point of view and use the scroll wheel to zoom in and out (Figure 1). A mouse is more intuitive than a touchscreen because the latter would require an additional key for further object modification.
This simple web application enables non-expert designers to modify given geometry layouts according to their individual preferences. The focus is on configuring geometries, not on the infrastructure or creating new items. The final layout can be saved and submitted with optional comments on the user’s design motives or further explanations. Participants can also vote and comment on other particpant-proposals and reflect on their own ideas and preferences.
The ease of use is a key factor for citizen science studies. Qua-kit offers the opportunity for design without any instructions by designers. Design tasks can be formulated such that participants can solve it in a few minutes. It is possible to see it as a tool that gamifies design problemsolving.
One of the tool’s drawbacks is that objects cannot be directly edited. This reduces participant creativity, but also ensures that they only focus solely on the configuration of objects.
Figure 2 shows a neighbourhood in Khayelitsha, an informal settlement 22km southeast of Cape Town. The project Empower Shack developed prototypes of new shacks which can be extended to two storey accommodations, thereby using the space more efficiently. The residents of the neighbourhood were involved in the rearrangement of shacks, although they did not use the qua-kit.
The site was prepared for the MOOC lectures Smart Cities (https://www.edx.org/course/smart-cities-ethx-ethx-fc-03x0) and approximately 500 students submitted their proposals via qua-kit. This paper does not focus on the results of the students’ work as research is ongoing. Instead, it presents design criteria and techniques to make the mass of designs useful for designers and decision-makers. The data from the participants are not pictures, but geo-data. This allows for a wide variety of evaluation options for the data which improves precision, in comparison to pictures which must be pre-analysed with image recognition methods. 3
Our data analysis distinguishes between form-based and perception-based criteria. Form-based criteria quantify the layout of buildings and the appearance of objects. Perception-based criteria formulate conclusions on the participants’ perception of the area that can be made by analysing the geometry.
To keep the different analysis tools clear and commensurable, we present them in form of profiles. We explain the method and purpose of each analysis and elaborate on the pros and cons.
Pro: The comparisons between different building types (high-rise, mid-rise, low-rise) can be useful for decision makers. Several other standard design criteria can be deduced: given the area, plot size and number of floors, quantities like the Gross Plot Ratio, Gross Floor Ratio, the number of units, and the density of objects are easily computed. The advantage of the frequency analysis is that it is simple and thus easy to understand for designers and decision makers.
Method: The objects in the circuit (buffer) of a particular object type are counted.
Purpose: This analysis considers the position of objects and assesses the mutual appearance of objects, and shows interrelations between objects.
Pro: This analysis is the perfect for association rule mining. The results can reveal insights into the citizen’s subconscious decision-making processes e.g. what to build along a river, or which building typologies should be built next to each other.
Contra: A carefully considered interpretation is essential to prevent far-fetched outcomes as not every result from the assocaition rule mining algorithm is inherently meaningful.
Method: Streets, and public and private spaces are automatically added as an additional layer based on the distance between buildings. A street is detected if the distance between buildings are within the boundaries of a minimum distance x and a maximum distance y. For our case study, these boundaries could be set to ! = 1$ and % = 4$. Distances greater than y are interpreted as open spaces. If an open space is accessible via a street, it is public; otherwise, it is private (Figure 3).
Purpose: Researchers can draw indirect conclusions about the street network and the placement of public and private spaces.
Pro: This additional layer helps researchers understand the organisation of the area.
Contra: Distance-based assignments of streets and space demand some general interpretation in advance. If the scale of the site is clear, the participants’ interpretation of space will match the automatic detection. If, for instance, the area of our use case would not be introduced as an informal settlement in South Africa, one could interpret the site as a suburban area in North America instead, for example. Also, the algorithm could misidentify some public squares.
Method: According to Ching [
Pro: The two form criteria are well studied in architecture and urban planning and already implemented in algorithms [Chen, et al., 2007].
Contra: While the axis may be detected using the distance between buildings, strict symmetries are supposed to be rarely identifiable in human-made designs, especially in the qua-kit tool which allows for flexible rotation and placement of buildings with no “magnetic features”.
Method: The 2D plots from different participants (optional: all) are superimposed. The merging of data can be done visually and geometry-based, too. Is is optional to deploy a hot spot analysis like the Gets Ord GI* [Mitchell, 2005; Getis & Ord, 1992; Ord & Getis, 1995] in advance. Purpose: The heat map shows preferred areas for particular objects and also directly reveals the spatial distribution of object types.
Pro: This methods allows the visual merging of proposals from different participants. All other proposed analyses are applied for separate designs.
Contra: The plots of the buildings are decontexualised. Method: The 2D centroids of buildings are calculated and used for spatial clustering, e.g. with the DBSCAN algorithm [Ester et al., 1996].
Purpose: On meso-scale, the space is subdived into smaller neighbourhoods like blocks. The preferred number and block sizes can be accessed through clustering.
Pro: The clustering approach allows segmentation to happen objectively.
Contra: A cluster is not neccesarily meant to be a block. The examples in Figure 5 show that parameters in the cluster algorithm need to be carefully adjusted for the case study area, but even this does not guarantee a satisfactory result.
Method: A spatial autocorrelation, e.g. Global Moran's I [Moran, 1950] is applied for different objects types. Purpose: The test reveals if object typologies appear dispersed or clustered in the area. If they are clustered, the buffer analysis can give some indication of the interrelation between objects.
Pro: The autocorrelation test is an objective measurement for a very subjective aspect for the perception of space. Contra: The dispersion of buildings can have many reasons, making simple interpretations hard.
Method: Human intelligence is required to label design proposals regarding creativity. Crowdsourcing internet platforms like Amazon Turk or Crowdfower are used to present workers screenshots of the participants’ designs. One option is to show two designs and ask which is perceived to be more creative. By using the Microsoft TrueSkill algorithm [Herbrich et al., 2007], the designs can be ranked. Kanzjon et al. have evaluated the design creativity of mobile devices using factors like novelty, value and surprise [Grace et al., 2015].
Purpose: Creativity is hitherto a very subjective impression and there are no rules or algorithms for making a decision. Creativity is an important factor for the uniqueness and individuality of a design and needs to be assessed by humans.
Pro: The analysis can be extended for supervised machine learning. The 2D layout of the plot is labeled with the creativity index according to the workers’ result. Contra: A large number of workers need to be employed to ensure unbiased labeling and to compensate for unreliable works.
Method: Similar to the creativity analysis, we want objective criteria that can be adhered to when human feedback is given for design proposals. The concern of the meta information analysis is not how other humans perceive the participant’s design but what the participant thinks of their own proposal. By proposing categories for the main purpose or idea of the design (e.g. safety, dominance of greenery, accessibility,...), the designs can be labeled, and used for the application of supervised machine learning to the geometry and purpose.
Purpose: The idea is to identify characteristics in the geometry in order to infer the main purpose of the design. Pro: This analysis allows very subjective characteristics of designs to be quantified.
Contra: The success of this method is not predictable. Participants may not be clear about what the main idea behind their designs is.
The qua-kit tool is designed for online participation with citizens. The methods in this paper provide options to precisely evaluate a large number of designs which are based on geo-information. Questions may be asked about whether these presented online design tools offer enough options for the participants to express their ideas, or if designers can gain any valueable inputs from these tools. The form-based criteria are simple to understand, but how can the results be interpreted? Especially the opportunity of letting non-experts design provides the chance to access knowledge that cannot be expressed semantically (e.g. by surveys). The evaluation should therefore concentrate on identifying the design’s semantic meaning how to make conclusions from the layout to the design idea beyond the proposal. This question is about to be answered by findings about how space is perceived [Bhatt et al. 2012, 2013, 2014]. Assuming that the design process is guided by the participant’s experience of space (i.e. the perception of space), it makes sense to analyse the design proposals by criteria that are found out to have a significant influence on the space organisation [Ching, 1979]. Axes and symmetries are already mentioned and it is for discussion, how and to which extent the other perception-based criteria should be applied. 5
This paper presents a brief summary of evaluation methods for crowdsourced geometric data on the meso-scale. The techniques are to be used in participatory urban planning studies. The introduced design tool qua-kit allows nonexperts to view 3D geometries and make simple edits. We use a case study from a neighbourhood near Cape Town to show options to evaluate the data on meso-scale. The most comprehensible instruments for analysis are the frequency and buffer analysis. Streets and public or private space can be determined by appropriate distance rules while cluster algorithms make small neighbourhoods automatically identifyable. The overlays of plots from different participants can be regarded as a heat map and visualise favoured spots for particular objects. Besides these formbased criteria, we propose to apply supervised machinelearning to connect the layout of buildings to subjective design criteria. The application of these evaluation tools will testify their usefulness or uselessness. Especially for accessing perception-based criteria, it is crucial to have easy understandable and distinct labels for the proposals. A further option is to apply unsupervised machine learning to the geometries to identify different categories of proposals or similarities in the design.
The research was conducted at the Future Cities Laboratory at the Singapore-ETH Centre, which was established collaboratively between ETH Zurich and Singapore's National Research Foundation (FI 370074016) under its Campus for Research Excellence and Technological Enterprise programme. Moreover, we wish to thank Artem Chirkin who has developed the Quick Urban Analysis Kit, for giving us access to the toolkit and geometric data from the edX MOOC Smart Cities to conduct our research.