Pervasive information visualization and interaction are fundamental tools to support learning in smart cities (SCL), for example to promote sustainable behaviours and social interaction. Building on a review of existing work, we identify the main limitations of traditional approaches based on large displays and smart-phones apps, first from a technological point of view then connecting to implications for design, user interaction and experience. In the paper we propose a set of authoring primitives at different semantic levels, ranging from more generic internet of things (IoT) primitives to more domain specific approaches connected to learning in SCL applications.
We focus more in detail on new design opportunities, developing possible scenarios of interest that involve SCL. The use of a toolkit for rapid prototyping is proposed as a valuable support instrument for application design and development.
smart objects; internet of things; authoring; smart city learning.
H.5.2 [Information interfaces and presentation (e.g., HCI)]: User Interfaces; H.5.3 [Information interfaces and presentation (e.g., HCI)]: Group and Organization Interfaces
Studies demonstrate that social connections in cities
stimulates creativity and improves work quality [
Smart cities present, by definition, a strong technological
component. In Technology Enhanced Learning (TEL), the
role of technology is to direct, foster thinking and facilitate
the acquisition of higher order skills [
Traditional technology is a limiting factor: mobile devices and large screens support a very strict and confined set of interaction strategies. It’s often not possible to tailor the user experience to properly fit the specific scenario because technology is too limiting. Our goal is to design aiming at the best possible strategy for the users, building the technology around this process and avoiding the constraints typically introduced by more general-purpose hardware/software combinations.
We claim there is a space of opportunity for SCL
applications in adopting novel ubiquitous computing approaches
like tangible user interfaces (TUIs) and augmented objects
(AOs). These technologies have already been found
effective in supporting learning [
Tangible user interfaces denote systems that rely on
“tangible manipulation, physical representation of data and
embeddedness in the real space”, allowing for an embodied
interaction with digital information. Embodied interaction,
as defined by Dourish [
Embodied interaction takes the interaction “off the screen” to the real world by distributing inputs and outputs in space rather than in time, desequentialising interaction and reducing the gap between where the information is created and where it is accessed. In this picture TUIs seamlessly integrate both representation and control of computation into physical artifacts: “By treating the body of the device as part of the user interface -an embodied user interfacewe can go beyond the manipulation of a GUI and allow the user to really directly manipulate an integrated physicalvirtual device”. When these artifacts also resemble and retain the functionalities of traditional objects, they can be called smart or augmented objects.
TUIs and smart objects (SOs) allow interaction designers to be free to experiment with new types of metaphors, taking advantage of the users’ physical skills and providing interfaces which exploit people’s knowledge with the everyday non-digital world.
End user development can also play an important role in
this scenario [
The concept of smart-city has also been used in many different context and is associated with distinctive and innovative aspects that are often quite different. Big diversities are observed on the reasons why different cities are defined as smart.
This situation is the consequence of the lack of a clear and recognized definition of smart city.
Komninos [
Smart cities are also a powerful ecosystem for learning. Smart city learning aim to support the improvement of all key factors contributing to the regional competitiveness: mobility, environment, people, quality of life and governance. The approach is aimed at optimizing resource consumption and saving time improving flows of people, goods and data1.
Education in this context is pursued as a bottom-up process, where person and places are central. Smartness from a learning perspective exists both in the ambient data collected and among the communities that exists within a city. The separation between student and teacher will fade out. Their role will be content or situation dependent: everybody will be a learner and the relation between persons will get a bigger role.
Technologies like mobile devices, tags, web based
applications, geographical information and e-learning systems
1http://www.mifav.uniroma2.it/inevent/events/sclo/
have already been used to develop smart city learning
applications on the field [
Smart city learning applications consist in the
implementation of urban informatics techniques and approaches to
promote innovative engagement strategies [
No doubt that among the consequences of such attention
there is an acceleration in supporting the integration and
embedding of ICT within physical environments to realize
what has been defined the everyware [
Augmented Objects and TUIs for SCL applications We identified a list of primitives at different semantic levels useful to describe, design and author applications for SCL supported by SOs and TUIs:
Touch as the ability to detect interactions like a simple touch, swipes, multiple taps et simila; Multi-axial rotation detects with sufficient precision object rotation and tilt; Shake and displacement intended as the ability to detect when the object is shaken vigorously or is being physically moved.
Led light can be used as a low fidelity output, more complex communication strategies can be implemented, like blinking, color fading and led matrix; Haptic defined as vibration pattern that differ in intensity and duration; Sound intended as simple beeping or composition of multitone sounds.
Untethered operation augmented object should work independently and autonomously, without being hooked to any external device that provides connectivity or power support; Easily embeddable technology should be easy to integrate into objects, without altering the original function and nature of the object; Energy autonomous the objects should be as much autonomous as possible, effective energy usage, battery efficiency and energy harvesting can help at this regard. “Generic/Domain Specific” overlapping level
Sensor data collection intended as the opportunity to employ data gathered in real time from the surrounding ambient. Domain specific data can include for example air pollution, geolocation, temperature; Data visualization as the ability to support visualization of simple information through low fidelity output. Adapting to the domain implies that only a specific subset of data visualization strategies are suitable. It is also important to trigger the learning process and help the users reflect on their experience; Data processing involves elaboration of data coming from sensors and/or from third party services.
Object sharing intended more as a dimension, SOs allow to freely move from the private and personal sphere to sharable artifacts and community objects, continuously exploring the space in between; Background/foreground interaction should be possible when the object is in the center of attention, but also when in background, providing context information possibly through glances and nudging; Distributed interaction with several augmented objects orchestrated as a single user interface; Multimodal object interaction based on more than one strategy, e.g. touch and speech recognition on the same object; Sensor based it’s an enabler for physical manipulation, since smart objects do not provide a dedicated interaction interface (like a keyboard or buttons), the interaction happens with the object itself.
Reflection occurs when reflecting on previous experience and behaviours; Behaviour change the explicit goal is to modify or improve a specific behaviour; Data enabled knowledge is extracted directly from collected data; Social occurs when it’s the result of a community process; Game based process gamification and situated games where smart objects extend and improve the gameplay and the learning experience.
Although the need for a set of authoring primitives is driven by the possibility to follow a more lightweight design approach and to better adapt to the context, defining an appropriate semantic level can be intricate since it is closely related to the skills of the target group (end-users, developers, designers, etc). A series of challenges are also connected to the authoring process: • Several semantic levels require several competencies, which implies that collaboration among experts is fundamental to address complexity along multiple dimensions; • Proposed primitives can be combined in multiple ways, picking the best alternatives for each application can still be challenging; • Data are a valuable source of knowledge to improve the design process, analytics are important to keep track of the process and spot opportunities for improvements; • The learning process can follow several approaches.
Based on the specific domain, can be challenging to find the most effective; • Promoting creativity is essential when dealing with different target groups, it is important to balance and define the primitives in a way that are useful to guide the design without introducing heavy constraints that can hinder creativity and original solutions.
Example of SCL applications resulting from this approach can be situated augmented games where players interact with smart pones that are also location aware. Pones can be displaced in the environment, shared between players, can guide the gameplay, support different interaction modalities and provide simple triggers for reflection, supporting the learning experience.
Another example can involve the use of augmented objects to facilitate cooperation between communities in the city. The process of urban planning in Norway require the municipalities to gather feedback from communities of citizens. Difficulties has been encountered in providing value in the process, since traditional methods do not fit well when dealing with children for example. Using smart objects that can be physically manipulated and provide an engaging experience can motivate and attract participants to collaborate and at the same time stimulate their creativity.
Tiles2 is a rapid prototyping toolkit for AOs, it is composed
by: (i) a set of abstracted physical interaction primitives and
2http://tilestoolkit.io
composition rules, (ii) hardware modules with sensors and
actuators to generate and consume primitives, (iii) a
software framework to allow manipulation and use of primitives
within a specific application logic. TILES toolbox aim at
supporting the iterative process of building prototypes of
interactive objects, using abstract primitives developed as a
bridge to gracefully support the transition between design
and implementation steps [
TILES is a promising project that fits very well with the generic authoring primitives defined above. It is designed taking into account many of the challenges usually found in prototyping toolkits. TILES’ flexible software framework and event-driven messaging system allow to develop abstractions at different semantic levels for the coding process. This helps to match more closely the skills of end users, developer, designers and possibly other categories.
In this article we proposed a new way to empower technology in SCL scenarios. Starting from the literature, the limitations of current technological patterns were highlighted, then AOs and TUIs were introduced as a viable alternative to more traditional approaches. TILES is then proposed as a valuable toolkit for rapid prototyping with SOs. Our claim is that the proposed primitives can be combined and used to author and design SCL applications that empower SOs, IoT and TUIs. Strengths and challenges of the authoring process were also presented.
Using this approach we believe it will be possible to successfully address several of the challenges that characterize research and interaction in SCL scenarios. More research and feedback are needed to better ground the claims proposed.