Opening the Box Meta-level Interfaces Needs and Solutions Alan Dix Talis, 43 Temple Row, Birmingham, B2 5LS, UK and Birmingham University, Edgbaston, Birmingham, UK alan@hcibook.com http://www.hcibook.com/alan/papers/SUI2011-meta/ ABSTRACT 2.2 Appropriation This paper begins by considering reasons why some form In particular "plugability and configuration" is one of the of meta-level interface may be required for modifying or design principles for appropriation [9]. Indeed several of exploring existing user interfaces, from obvious functional the design principles discussed in [9] are related to meta- reasons of customisation and personalisation to more level user interactions; while appropriation is possible political and social goals such as education and using the interface as given, the user has greater flexibility empowerment. The paper considers examples of systems if she can peek under the hood (design principle "provide developed by the author and others, and uses these to visibility") and tinker inside ("plugability and present a number of techniques and principles for effective configuration") and share the results with others meta-interactions. Some of these concern more surface ("encourage sharing"). manipulation, and others deeper levels of code and meta- 2.3 End-user Empowerment descriptions of the application and UI. It concludes that One advantage of appropriation is the sense of ownership meta-interaction may be a key element for future liberal and empowerment it engenders. A sense of control is society. important for well being, and the act of tinkering gives this, Keywords whether to improve the user interface for its original customisation, personalisation, end-user programming, purposes, or make it do something completely novel. end-user empowerment, appropriation While this is important for all users it is particularly 1. INTRODUCTION relevant for those in developing countries, or the The topic of this workshop brings together a number of disadvantaged in developed countries, who can be doubly areas on which I have worked or that have been of personal disadvantaged in a world where access to information is concern. This paper will discuss some of these areas of central to economic and political power [1]. concern and then look at general principles and techniques Existing technology can be appropriated by traditionally that can be used to address them. disadvantaged groups; for example, Jensen reports how 2. WHY META? mobile phones allowed fishing boats in Kerala, southwest While it hardly needs stating for this workshop, to many it India, to obtain higher prices for their catches [12] and we may seem that meta-level interactions are simply the have all seen the impact of social media in recent popular preserve of the hobbyist or techie. However, they are both uprisings across North Africa and the Middle East. ubiquitous and of broad benefit. However, if those closer to need are in a position to create, 2.1 Customisation and Personalisation modify or adapt existing software and hardware the results Of course meta-level user interfaces are common. Every are likely to be more appropriate than tools designed time a user drags a palette to the side of the screen, selects a primarily for an urban, middle-class, western environment. ringtone or modifies the style definition in a document, she This may be the end user, but Marsden et al. argue the case is engaging in an adaptation of the user interface. for 'human access points', local experts, in their case local However, we also know that beyond a few examples like health workers, who are given the tools to create and adapt this few users actually customise despite having problems mobile-phone administered questionnaires [16]. Prompted or gripes that could be dealing with through simple by various workshop discussions [17, 20], we have selection of options (for example, turning off some of the explored the potential for a range of mobile phone-based 'smart' features in Word). Improving even these basic adaptations including compete coding via the mobile-phone features can have a major impact on user experience. screen [10]. 2.4 Education interfaces even if they complain about the things that are Often modifications to user interfaces require a high degree wrong! The key problem is not lack of understanding but of expertise; so education is needed in order to use them. lack of immediate benefit. We are creatures who heavily However, if well designed, meta-level interactions hold the discount the future; effort now for future gain is hard. If potential to be a means for education in themselves; as customisation can be made closer to the point of use it generations of children who have fiddled with old car becomes more likely. One example are dialogues that ask engines can testify. Education, of course, also contributes for a decision, but have a tick box to say "always do this". to empowerment. This is effectively asking you set a preference, but at a The Query-by-Browsing (QbB) intelligent database point in time when you are in the middle of doing the interface is an example of this. QbB generates SQL: requisite action. The benefit is clear and the cost (in terms queries based on user record preferences, but then reflects of clicks and mental effort) low. Furthermore this is all set this back to the user both by highlighting the records within the context of a concrete example of use (see also selected by the query and by exposing the query itself [7]. next point) The user can comprehend the system via the concrete 3.2 Progressive Disclosure –Where It happens record selections, but in the process learn the SQL that The preferences and customisation of many applications produce it (although not the machine learning algorithms are buried in a "preferences" menu item far away from the which generate the queries). actual interaction. Somewhere in a preferences panel you 2.5 Privacy and Auditability set parameters whilst guessing vaguely what they might be The control of privacy settings in social applications such about. However, others connect customisation closer to the as Facebook, has become a big issue. Höök also argues thing it affects. Back in 1995, Marsden [15] advocated the that this is an issue likely to be important in future advantages of a systematic policy suggesting a 'screw' ubiquitous computing applications [11]. Indeed the very metaphor where every component has a small screw icon in openness in low-level architecture required for rich the bottom right hand corner. Clicking the screw 'undoes it' context-sensitive features in itself creates privacy issues revealing the circuitry within, and potential the ability to [8]. Many approaches to privacy, in ubiquitous computing unscrew other sub-components (see Figure 1). and elsewhere, focus on restricting information flow. However I have long argued that it is the eventual use of the information that is most critical [6]; that is systems that expose what happens to information both currently (visibility) and in the past (auditability) are far more likely to support the user's ability to manage information disclosure. 2.6. Comprehensible Behaviour and Trust Closely related is the issue of trust, not just for financial and or personal security, but also at a mundane level of whether we decide to use particular application features. Figure 1. Screw Metaphor from [15] This is especially important when systems make choices (a) screw in – UI (b) screw out – metaUI automatically for us. The kind of openness needed to allow a user to adapt a system is very similar to that needed to Today in the Apple Dashboard just such a mechanism is allow a user to believe in what it is doing already. found on widgets. Instead of a screw a little 'i' for information icon, clicking it 'turns around' the widget The record listings in Query-by-Browsing [7] are an showing settings behind. Strangely the iPhone reverted to a example of this as they may be comprehensible to the user, special place for settings rather than associating them even if the SQL is not, giving the user confidence that the closely with their application. query will continue to be appropriate for unseen records. Another example is MICA, which makes suggestions for GUI customisation based on user activity, but also "includes a description of why MICA is making recommendations and how it generated them" [5], precisely to support Hook's "predictability and transparency" principle [11] and so engender trust. 3. TECHNIQUES AND PRINCIPLES So if meta-level investigation and modification is a good thing, how can it be achieved? 3.1 Cost and Benefit – When it happens Figure 2. Mac OS Dashboard widget (a) front – UI (b) back– metaUI Sometimes people don’t customise because they don’t know how. However many experts do not customise their 3.3 Tools of Revelation have a huge gulf between them, but where each pair of A similar approach is to use some form of external 'tool' for successive levels overlap with an easy transition. This meta-level modifications. This happens in the real world; sounds like a hard problem, but there are examples that Figure 3 shows a stud detector, which detects the wooden achieve this to varying extents. HyperCard had a smooth studs in a wall so that you can screw into them. The transition from use to customisation and then to wooden structure is hidden behind plasterboard and programming. In consequence, many who would never wallpaper, but the stud detector reveals it – the "provide consider themselves programmers created complex visibility" appropriation principle [9] in the physical world. HyperCard applications. Xerox Buttons were another example, where a non-technical user might just use the button, then peek at its code and change a file name, and perhaps, over time, start to understand some of the code that drove the familiar user-interface actions [14]. Could the Excel formula to VB step be more like this? 3.5 Ease of collaboration Another of the appropriation principles is "encourage sharing" [9]. In Nardi and Miller's classic study of spreadsheet use [18], they describe the collaboration Figure 3. Wall Stud Detector between Buzz and Betty Note that "provide visibility" does not mean the same as "When Buzz helps Betty with a complex part of the Nielsen's "visibility of system status" evaluation heuristic spreadsheet such as graphing or a complex formula, his [19], as this usually refers to the essential information about work is expressed in terms of Betty’s original work. He the system for normal use. Instead, if systems reveal a little adds small, more advanced pieces of code to Betty’s basic more (such as a mobile phone showing signal strength not spreadsheet: Betty is the main developer and he plays an just whether or not a call can be made), then the user can adjunct role as consultant." use this in unexpected ways (such as waving the phone The fact that spreadsheets have relatively smooth about to seek out better signal). transitions (at least between levels of formula use) make Beaudouin-Lafon's 'instrumental interaction' [2] and in this collaboration possible. Note especially that Betty is particular Toolglasses [3], follows the same principle as the able to do a lot herself, and probably extends this over time stud detector advocating the use of 'instruments' as a means (education). Furthermore Betty is able to determine her for modifying and interacting with objects. own level and understand when to seek help. 3.4 Smooth Transitions Spreadsheets, by their nature allow them to be passed When creating means for user to modify their environment around. It is far rarer to see other kinds of configurations there is often a temptation to try to do everything – the shared. In UNIX systems, a lot of configuration is in text spectre of Turing equivalence rises and before long a files, such as .login or .profile, and expert users will move simple end-user customisation tool becomes a full-blown these around. However, it is near impossible to simply take and complex programming language. The effort to produce one person's Word settings and apply them to another users something that could, in principle, do everything often ends machine. Xerox Buttons [14] were a simple idea, a button up with something that, in practice, is good for nothing. that executed some Lisp code, but were surprisingly However, the alternative is often to have very different powerful, in part because you could mail them round, means for simple and more complex modifications, so that creating a community. Maker cultures emerge when users hit barriers; for example, moving from Excel people can share ideas and, even better, artefacts. formulae to Visual Basic. 3.6 From Configuration to Code Mathematicians face a similar problem when modelling Spreadsheets, Xerox Buttons, Query-by-Browsing and 'differential manifiolds' curved spaces such as the surface HyperCard are all examples where the user can move in of the Earth or the curved space-time of general relativity. steps from doing things to raw coding. When looking at They effectively paper the curved space with flat Euclidean near-end-use development, one of the design lessons was surfaces (which are easier for a mathematician to handle), "reduce the gap between design and execution" [10]. but if you try to use a single flat surface there is at least one "In general, bridging the gaps between environment and point where things go very wrong, like the place where the language, design and use, test and bug report [...] foil is all folded up at the end of an Easter egg. Instead features found in many end-user or near-use software mathematicians use a collection of small patches, which such as spreadsheets (eliding data, code and execution), overlap in a 'smooth' manner. Yahoo! Pipes (design close to execution), and One can envisage customisation working like this, with programming by example (use is design)" different levels of customisation (perhaps ending up at At Talis we are working on tools to bridge this gap for open-source code), where the two ends (use and coding) linked open data [4] as exposed, for example, in data.gov.uk. This is building on Callimacus, where RDFa 8. Dix, A. Beyond intention – pushing boundaries with embedded in a web page turns it into a UI generation incidental interaction. Proc. of Building Bridges: template, opening up application building to ordinary web Interdisciplinary Context-Sensitive Computing, developers [13]. (Glasgow University, 9 Sept 2002) 3.7. Meta-Representations for Meta UIs http://hcibook.com/alan/papers/beyond-intention-2002/ As well as being the subject of user interaction, semantic 9. Dix, A. 2007. Designing for appropriation. In Proc. of data of some form seems to be a key element of future user HCI2007 Volume 2. (2007). BCS, 27–30. interactions. Whether mashing data for the web or http://www.bcs.org/content/conWebDoc/13347 connecting digital devices in the living room, effective 10. Dix, A., Kozhissery, R., Ravichandran, R. and meta data about devices, applications and their interactive Dayanand, D. Content Development Through the potential seems an essential start point for more flexible Keyhole. in Proc. of EISE2009, Expressive Interaction machine initiated activity, for machine activity to be for Sustainability and Empowerment, (2009) 67–78. explicable, and for users to be able to interrogate and http://hcibook.com/alan/papers/EISE2009-Keyhole/ modify it. Model-based user interfaces are clearly one way to achieve this, but there could be other solutions, similar to 11. Höök, K. Steps to take before intelligent user interfaces the way applications expose meta-information for Apple become real. Interacting with Computers, 12 (2000) Scripting on Mac OS or via COM on Windows. 409--426,. 4. CONCLUSIONS 12. Jensen, R. The Digital Provide: Information We have discussed various principles and methods for (Technology), Market Performance, and Welfare in the meta-level interactions., and also some of the reasons why South Indian Fisheries Sector, Quarterly Journal of this is 'a good thing'. As we enter an era of open data and Economics, 122(3), (2007) 879–924. mashups the ability to digitally tinker seems not just a 13. Leigh, J. and Wood, D. RDFa as a Query Language. hobby, but a key enabler of a broad-based civil society. Semantic Technology Conference. (June 2010) REFERENCES 14. MacLean, A., Carter, K., Lövstrand, L., and Moran, T.. 1. Beardon, H., Munyampeta, F., Rout S. and Maiso User-tailorable systems: pressing the issues with Williams, G. ICT for Development, Empowerment or buttons. In Proc. CHI '90. (1990) ACM Press, 1990, Exploitation: Learning from the Reflect ICTs project. 175–182 ActionAid. (2005) http://www.actionaid.org.uk/1413/ict_for_development 15. Marsden, G. Overcoming Design and Execute Modes in _empowerment_or_exploitation.html User Interface Design Environments. in Proc. of HCI 95 people and Computers (1995), 133-137 2. Beaudouin-Lafon, M. Instrumental interaction: an interaction model for designing post-WIMP user 16. Marsden, G., Maunder, A. and Parker, M. People are interfaces. in Proc. of the CHI '00. ACM Press, (2000) people, but technology is not technology. Phil. Trans. 446–453. R. Soc. A (2008) 366, 3795–3804. 3. Beaudouin-Lafon, M and Mackay, W. Reification, 17. Mobile Design Dialog. (Cambridge. 3–4 April 2008) polymorphism and reuse: three principles for designing webpage: http://www.cs.swan.ac.uk/mobdesign/ Mobile visual interfaces. in Proc. of AVI '0). (2000) ACM Design Dialog discussion: Press ,102-109. http://mobiledesigndialog.nexo.com/ 4. Bizer, C., Heath, T. and Berners-Lee, T. Linked Data – 18. Nardi, B.and Miller, J. An ethnographic study of The Story So Far. International. Journal on Semantic distributed problem solving in spreadsheet Web and Information Systems, 2009. development. In Proceedings of the 1990 ACM conference on Computer-supported cooperative work 5. Bunt, A., Conati, C., and McGrenere, J. Supporting (CSCW '90). (1990) ACM, Press, 197–208. interface customization using a mixed-initiative approach. In Proc. of IUI '07. (2007) ACM Press. 92- 19. Nielsen, J. Heuristic evaluation. In Nielsen, J., and 101. Mack, R.L. (Eds.), Usability Inspection Methods, (1994) John Wiley & Sons, New York, USA. 6. [Di90] Dix, A. (1990). Information processing, context and privacy. in Proc. of INTERACT'90, (1990) North- 20. Winter School on Interactive Technologies. (HP Labs in Holland. 15–20. http://hcibook.com/papers/int90/ Bangalore, 2nd & 3rd February 2009). UK-India Network on Interactive Technologies. 7. [DP94] Dix, A. and Patrick, A. (1994). Query By http://www.ukinit.org/02122008/winter-school- Browsing. Proc. of IDS'94: The 2nd International interactivetechnologies Workshop on User Interfaces to Databases, (Lancaster, UK, 1994), Springer Verlag. 236–248. http://hcibook.com/alan/papers/QbB-IDS94/