Planning Support Systems are (PSS) are software tools developed for assisting planning professionals. Despite the potential attributed to these tools in supporting planning processes, their adoption and use in planning practice is rather low. One major reason for this is their low usability. This paper presents the motivation, objectives, approach and current status of a PhD research that aims to support developers and researchers with strategies for improving PSS usability.
Planning Support Systems (PSS) refer to a set of tools that incorporate different
functionalities such as spatial analysis, modelling and visualisation, and that are
designed to support planning professionals in dealing with the complex nature of
planning tasks. Much potential is attributed to these tools [
Currently, other computer-based tools, not primarily dedicated to assist planning
professionals, are more commonly used in planning practice than PSS. These are, for
example, Geographic Information Systems (GIS), Computer Aided Drafting (CAD)
software, and sketching and visualisation tools [
Research argues that PSS use could assist land use planners in their decision
making process by providing a more systematic and evidence-based approach to
planning problems [
To date, research has identified the factors that hamper the adoption and use of
PSS. Factors can involve instrumental, human, organisational and institutional
characteristics. One of the most important factors has been identified in the low
usability of PSS [
In particular, PSS have been stated to be “far too generic, complex, inflexible,
incompatible with most planning tasks, oriented towards technology rather than
problems, and too focused on strict rationality” [
It is worth remarking that conducting usability evaluation requires specific skills that PSS designers usually do not possess. While there are many books and websites that describe evaluation methods, novice evaluators actually need more guidance on how to plan and perform the overall evaluation. As long as the use of PSS connotes frustration and bad experiences, it is likely the adoption and use of PSS to stay relative low.
The present thesis is dedicated to advancing the research on PSS usability and UX in the endeavour to contribute towards improving PSS adoption and use. 2
This thesis examines factors that influence PSS usability and consequently can have obstructive or supportive effects on the adoption and use of this software. It Proc. of CHItaly 2015 Doctoral Consortium, Rome (Italy), September 28th 2015 (published at http://ceur-ws.org). Copyright © 2015 for the individual papers by the papers' authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors.
furthermore provides strategies and recommendations that might help developers, researchers and land use planners improve PSS usability, adoption and use.
Specifically, this research aims to identify usability constraints and opportunities for the adoption and use of PSS by land use planners. To achieve this aim, the following research questions (RQ) are addressed: (RQ 1) What affects usability of PSS for planners?
Influence factors of PSS usability are not only restricted on limitations of PSS
technology. According to the definition in ISO 9241-210 [
For some usability influence factors merely the awareness of the planning community can be increased. For others, strategies for improving the situation can be identified. This thesis aims to define such strategies, especially in the ambit of software design, and provide tools for improving PSS development and evaluation. 3
The theoretical framework of this research draws on the contributions of two scholars:
[
The usability problem concept is used for describing and reporting usability problems. In this research, it was chosen because it allows dissecting usability problems and analysing their components. In fact, it suggests breaking down usability problems into five components: cause, breakdown, outcome, design change and context (see Figure 1a). Breakdown refers to an inappropriate action, such as a user performing a wrong action or misunderstanding a feedback that triggers a usability problem. The severity of breakdowns can differ. Cause is the origin of a breakdown and of a usability problem. An example of a cause is a design fault. Outcome is the consequence of a breakdown. Two different types of outcomes are distinguished: user behaviour and performance. Examples of the first type are the user is dissatisfied or tries another action. Task failure and/or loss of time are examples of the performance type. Design change refers to modifying an aspect of the system. An example is introducing a new feature. Context relates to each usability problem and its components occurring in a specific context. For instance, users of different Proc. of CHItaly 2015 Doctoral Consortium, Rome (Italy), September 28th 2015 (published at http://ceur-ws.org). Copyright © 2015 for the individual papers by the papers' authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors.
organisations might have different requirements for the design of a new feature.
The innovation-decision process describes the stages that an individual undergoes from being exposed to an innovation for the first time to deciding whether to continue to use or rejecting the innovation (see Figure 1b). Five stages are distinguished in this process: I) knowledge, i.e. the individual is made aware of the existence of the innovation and gains an understanding of its functionality, II) persuasion, i.e. the individual forms a positive or negative attitude towards the innovation, III) decision, i.e. the individual makes a decision whether he/she considers adopting or rejecting the innovation, IV) implementation, i.e. the individual starts using the innovation with some uncertainty and V) confirmation, i.e. the individual decides whether to continue to use or reject the innovation. The number of individuals who adopt an innovation in a specified period of time is given by the rate of adoption. Various factors influence the rate of adoption, such as communication channels or the nature of the social system. However, it has been recognised that from 49 to 87 percent of variance in rate of adoption can be explained by five attributes of innovations. These are relative advantage, compatibility, complexity, trialability and observability. Compatibility has been defined as the “degree to which an innovation is perceived as consistent with the existing values, past experiences, and needs of potential adopters” [15:224]. Evidence suggests that the higher the compatibility of the innovation, the higher the rate of adoption.
This process was chosen because it allows examining the potential adoption process of an innovation from the perspective of potential users without necessarily taking into account, for example, organisational factors as suggested by other theories (e.g. Technology Organisation Environment Framework).
This research examines the influence of usability problems on the innovationdecision process, specifically for PSS. By doing this, it assumes that usability problems (breakdowns) occur when potential adopters use PSS, i.e. during the knowledge and implementation stage, and it rather ignores that potential adopters might have positive experiences. Furthermore, this research does not take into account that other factors than usability, such as of organisational and social nature, might affect the innovation-decision process, specifically for PSS.
While breakdowns occur in the knowledge and implementation stage, this research suggests that the other components of usability problems, i.e. the causes, outcomes and possible design changes, influence potential adopters in the persuasion, decision and implementation stage. For instance, in the persuasion stage, potential adopters form their attitude towards the innovation and perceive the degree of the innovation’s compatibility based on the experience (outcome) they had during the knowledge stage. Furthermore, this research expects that planning professionals’ decisions whether to adopt and use a specific PSS can be both, negatively and positively influenced by, for example, a negative outcome (e.g. dissatisfaction of potential adopter) and a design change (e.g., improved information presentation), respectively.
While the innovation-decision process claims compatibility to directly affect the rate of adoption, this research assumes that the lack of compatibility of PSS with planning professionals’ needs, to be a major cause for the occurrence of usability problems. Indeed, this research supposes the rate of adoption of PSS to be low because of the occurrence of usability problems. Proc. of CHItaly 2015 Doctoral Consortium, Rome (Italy), September 28th 2015 (published at http://ceur-ws.org). Copyright © 2015 for the individual papers by the papers' authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors.
A literature review has been performed to understand the state-of-the-art in PSS use and usability and what are potential reasons for low usability of PSS from the perspective of experts in the field. From the literature review it mainly emerged that user-centred design and usability evaluation of PSS are rather rare. Thus, poor fit of PSS to user requirements was reported.
To support PSS developers intended to perform usability evaluations, a framework, called PSS_EvalF (short for PSS Evaluation Framework), has been developed in collaboration with usability experts that provide the necessary guidance for non-expert evaluators on how to plan and carry out evaluations.
Interviews with academic experts in the field of planning have been conducted in Australia and Switzerland to understand what planning students are taught at different educational institutions in relation to learning how to use PSS.
Interviews have been carried out with planning professionals to understand what software is used in planning organisations, what this is influenced by (including educational background of planning professionals) and what the context of use is. The interviewees worked for different planning organisations (i.e. private and government) in Australia and Switzerland to examine potential differences.
An online resource (http://docs.aurin.org.au/projects/planning-support-systems/) that comprises PSS developed and/or applied in Australia and internationally, as well as some information on their functionality and use requirements has been created. Proc. of CHItaly 2015 Doctoral Consortium, Rome (Italy), September 28th 2015 (published at http://ceur-ws.org). Copyright © 2015 for the individual papers by the papers' authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors.
A user test following PSS_EvalF has been carried out with a small group of land use planners working in Australia to evaluate three PSS that perform Land Suitability Analysis (LSA) tasks. The three PSS have been selected based on the PSS in the online resource. The user test allowed identifying usability problems and usable characteristics of the three PSS as well as planners’ mental models and requirements.
Based on this outcome, recommendations for designing more usable PSS were defined. Furthermore, lessons learned from conducting the user test allowed refining PSS_EvalF.
Next steps of the research are: i) validating PSS_EvalF through its adoption by potential users, i.e. developers, for
PSS evaluation. ii) about drawing conclusions from the knowledge gained through the activities undertaken. Through converging the outcomes of the interviews and user test, it is aimed to formulate a set of guidelines for improving PSS usability and/or heuristics for facilitating the design and evaluation of PSS. 5
As stated above, the results of this thesis will be a series of strategies that if implemented, potentially improve the usability of PSS for land use planners. Two strategies are presented as follow.
PSS_EvalF represents a framework providing guidance on performing usability and UX evaluation that has been developed specialised for the evaluation of PSS.
Inspired by a more general evaluation framework provided in [
composed of six activities which are briefly documented in the following. A more
complete version can be found in [
To clarify what is the scope of the evaluation and what should be achieved once the evaluation is carried out, evaluation goals are defined as a first activity. Typical goals of PSS evaluation might be: • Assess the ease of use of a PSS for its adoption by planning professionals • Compare the functionality and usability of multiple PSS • Identify design suggestions for the next version of a PSS
As a second activity, more specific questions that underpin the goals and should be answered through the evaluation, are formulated. A question can be “is the PSS usable?” and furthermore “is the user interface easy to navigate? Is the terminology confusing because it is inconsistent? Is the feedback provided to users sufficient? Is the response time too slow?”.
PSS_EvalF proposes inspection and/or user-based methods and illustrates their advantages and disadvantages in order to facilitate the choice. Among user-based methods, PSS_EvalF suggests using: • thinking-aloud technique, • questionnaire and interview for assessing user satisfaction and other hedonic qualities of UX, • video and screen recording for complex tests and for reviewing participant’s interaction and behaviour if something is unclear, and • measures of user performance.
Many practical issues have to be considered when conducting an evaluation. For
example, before beginning with an inspection, it should be ensured that all evaluators
i) use the same inspection criteria, ii) have the same understanding of the criteria and
iii) apply them in the same way. Novice evaluators might consider using Nielsen’s
heuristics [
In user-based methods, the following issues have to be addressed: • Choice of the participants • Choice of the experimental design (between- or within-subjects design) • Choice of the task(s) • Choice of the facilities and equipment.
This activity informs evaluators about the ethical issues involved in user-based methods. PSS_EvalF provides an example of a consent form and a plain language Proc. of CHItaly 2015 Doctoral Consortium, Rome (Italy), September 28th 2015 (published at http://ceur-ws.org). Copyright © 2015 for the individual papers by the papers' authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors.
Before actually running the evaluation, decisions have to be made about how data
are analysed and presented. PSS_EvalF refers to books [
The following recommendations emerged from the user test and are intended to
support developers in designing more usable PSS:
• Layout, colours and fonts for the presentation of data, legend and items have to be
carefully chosen to make it easier to understand for the user.
• The terminology used in PSS should be close to the one used by planners and in the
regions and countries where the PSS are applied.
• PSS should allow users to seek visual information in a way that is effective, e.g. by
following Shneiderman's [
This thesis advances the research on PSS usability and UX in that, to our knowledge, it represents a first work that puts the emphasis on usability by relying on current body of knowledge in usability and therefore providing a solid basis for further work.
As a novel contribution, this research developed a framework for performing usability and UX evaluation of PSS, called PSS_EvalF. By providing people with little expertise on conducting usability evaluations with six activities specialised for the Proc. of CHItaly 2015 Doctoral Consortium, Rome (Italy), September 28th 2015 (published at http://ceur-ws.org). Copyright © 2015 for the individual papers by the papers' authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors.
evaluation of PSS usability, it aims to improve evaluations and usability of PSS.
A user test allowed identifying usable characteristics and design recommendations for both, PSS that address LSA and PSS in general. This contributes to increasing developers’ awareness and understanding of what are planning professionals’ needs and expectations. Only by knowing these, PSS that consider user-oriented aspects, are more usable and capable of supporting good user experiences, can be developed.
An insight into the state-of-the-art of PSS adoption has been provided. Differences in the working context, software use and requirements of planning professionals working for different planning organisations (i.e. private and government) in Australia and Switzerland have been identified.
An insight into the state-of-the-art of planning students’ education of PSS use has been provided. This insight has emphasised the controversy between researchers stressing the low use of PSS in planning practice and the limited provision of GIS and PSS courses for planning students and limited encouragement to attend such courses at educational institutions.
Another contribution has been the development of an online resource that lists a large set of PSS developed around the world. Where available, information in relation to technical specifications, analysis functionality, use requirements and user assistance of the PSS has been provided. This information supports planning professionals in choosing appropriate PSS, improving PSS application as well as raising the planning communities’ awareness of PSS availability and potential. 7
I would like to thank my supervisors, C. Pettit, M. F. Costabile and R. Lanzilotti, for the support and guidance provided in this research. Proc. of CHItaly 2015 Doctoral Consortium, Rome (Italy), September 28th 2015 (published at http://ceur-ws.org). Copyright © 2015 for the individual papers by the papers' authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors.