=Paper=
{{Paper
|id=Vol-2801/paper5
|storemode=property
|title=The BODYSOUND case. A tangible prototype for co-designing "intangible" healthcare solutions
|pdfUrl=https://ceur-ws.org/Vol-2801/paper5.pdf
|volume=Vol-2801
|authors=Laura Cipriani,Andrea Ascani,Carla Sedini,Massimo Bianchini,Stefano Maffei
|dblpUrl=https://dblp.org/rec/conf/etis/CiprianiASBM20
}}
==The BODYSOUND case. A tangible prototype for co-designing "intangible" healthcare solutions==
https://ceur-ws.org/Vol-2801/paper5.pdf
The BODYSOUND case
A tangible prototype for co-designing "intangible" healthcare solutions
Laura Cipriania, Andrea Ascania, Carla Sedinia, Massimo Bianchinia and Stefano Maffeia
a
Politecnico di Milano, Design Department, via Durando 38A, Milano, Italy
Abstract
This paper explores the role of tangible and intangible technologies in the development of
healthcare solutions that actively involve patients and caregivers in the ideation and
development phases through co-design and co-creation processes. In the first part of the
document, we frame the characteristics of patient innovation - the phenomenon of user-driven
healthcare (Olivera et al. 2015) - in relation to the category of solutions developed, processes
and emerging technologies. The second part is focused on a case study called BODYSOUND
(a pilot of a H2020 European research) and analyses the co-design process adopted to develop
a product-service system for rehabilitation, based on a series of testing of tangible and non-
tangible technologies in an attempt to identify a range of opportunity and scenarios. The final
part systematizes the results of the analysis and tries to identify a series of challenges that bring
this kind of solutions to the market and to users.
Keywords
Patient innovation, Co-design processes, Tangible interfaces, Touchless technologies,
Rehabilitation, Healthcare
1. Introduction: Framing the patient innovation perspective
Patient Innovation (PI) can be delineate as a phenomenon belongs to the wider field of Grassroots
Innovation. In particular, PI has be defined as “a network of activists and organizations generating
novel bottom-up solutions for sustainable development and sustainable consumption; solutions that
respond to the local situation and the interests and values of the communities involved” (Seyfang &
Smith, 2007: 585).
Another research (Dreier et al., 2016) identified three common characteristics of Patient Innovation.
The first one is independence: in most cases, the idea comes from personal condition and to face a
personal issue. The second one is repetition: in many cases the identified solution already exists, but
the patient did not know. The third one is sharing: generally, when a solution is found, patients tend to
share their positive experience with other people in their same condition lacking (without?) a connection
with doctors and healthcare professionals.
Upstream of this phenomenon, there is an emerging lack of confidence in the healthcare system: as
recently reported by Eurispes, 47.4% of Italian citizens are more inclined to opt for self-diagnosis and
self-care (Eurispes, 2017). Involving people to envision new solutions in the care process could
represent the right choice instead of leaving them to the do-it-yourself care and medicine. Society
changes bring inevitable new healthcare needs: as highlighted by OECD Health Statistics 2018 (OECD,
2018), the health system should be more people-centered, taking also the advantage of the evolution of
digital technologies to prevent and facing possible life-threatening diseases.
Proceedings of ETIS 2020, November 16–20, 2020, Siena, Italy
EMAIL: laura1.cipriani@polimi.com (A. 1); andrea.ascani@polimi.com (A. 2); carla.sedini@polimi.com (A. 3)
massimo.bianchini@polimi.com (A. 4); stefano.maffei@polimi.com (A. 5);
© 2020 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Wor
Pr
ks
hop
oceedi
ngs
ht
I
tp:
//
ceur
-
SSN1613-
ws
.or
0073
g
CEUR Workshop Proceedings (CEUR-WS.org)
2. New opportunities about assistive technology in co-design
The eHealth 2012-2020 Action Plan1 of the European Commission provides a roadmap to empower
patients and healthcare workers. The Action Plan link up devices and technologies, fund research
towards the personalized medicine of the future, support research, development and innovation in
eHealth and wellbeing to address the lack of available user-friendly tools and services, promoting at the
same time policy dialogue and international cooperation on eHealth on a global level.
Accessibility to digital (intangible) solutions development tools can help patients to operate in a
more specific dimension of personal fulfilment. These types of solutions are easier to deploy and, partly,
also for non-experts (small investments in terms of time, cost and expertise). The regulatory and
certification phase of the process, known to be one of the most critical, especially in the healthcare
sector, is another factor that facilitates intangible (services) solutions. For these reasons, there are many
solutions having these characteristics that are designed or developed within patient innovation
processes. Among other advantages, e-health allows to develop solutions that are advantageous from
the point of view of care: personalized medical care, portability, continuity of treatment, reaching more
patients, involvement of more patients and quantified measurement of efficiency.
Focusing on Grassroots Innovation and co-creation processes, Fab Labs and makerspaces2 are
recently emerging as enabling places to involve users, including patients, in co-design processes.
Community-based labs are places where creative professionals, makers and amateurs, practitioners and
researchers, can meet together and access to knowledge, technologies and competences, where they can
easily experiment and prototype tailor-made solutions moreover sharing their results with a broader
audience.
3. Role of tangible and intangible interfaces in co-design solution for
rehabilitation: the state of the art in Italy
In Italy, beyond the bottom-up healthcare solutions developed by patients, caregivers or medical
staff, the number of tele-health service-products, e-health app based and digital platform has increased:
among the 150 solutions mapped in MakeToCare2 report (Maffei et al., 2019), 31 are digital services
(apps, platforms) already released or forthcoming, and 71 are product-service systems. These last
solutions have physical/tangible evidence (products) but are also accompanied by a service component,
more or less structured (IoT connected objects, such as a sensorized band supported by a mobile
application).
The research reports MakeToCare1 and MakeToCare2 (Maffei et al., 2017; Maffei et al., 2019)
explored some Italian examples of solutions developed with the involvement of patients, interesting
for: typology of the solution developed, developers and actors involved, interface types (tangible,
touchless, haptic, etc.).
The following cases have been selected as examples to demonstrate that tangible and touchless
technologies can support the development of heterogeneous solutions for rehabilitation or tele-
rehabilitation, taking advantage of gamification, data collection and different combinations of interfaces
for obtain solution of personalized care.
1
European Commission (2012). eHealth Action Plan 2012-2020: Innovative healthcare for the 21st century (ec.europa.eu/digital-single-
market/news/ehealth-action-plan-2012-2020-innovative-healthcare-21st-century)
2
Besides, in many cases, Universities, Research Centres, Hospitals, etc. take part in these networks, or even promoting them (e.g.
POLITECNICO at Politecnico di Milano, Lab4Living at Sheffield Hallam University, UCL Centre for Co-production in Health Research at
University College London, Helix Centre located at the St Mary's Hospital in London but managed by Imperial College London and The
Royal College of Art), giving Fab Labs and Makerspace a more “reliable” reputation when trying to involve different stakeholders like
Academy, Government, Civil Society, and Enterprises.
Table 1
Cases studies
Project Typology of the solution Developers and actors Interface types
developed involved
Rehability Serious games that allow patients Developed by Imaginary touch
www.rehability.me to perform a personalized srl, Co-designed with haptic
2014 rehabilitation therapy with patients suffering from
constant medical supervision stroke, multiple sclerosis
through a series of games played or Parkinson's disease
on tablet.
Mirrorable Interactive tele-rehabilitation Developed by the touchless
en.fightthestroke.org/mirror designed for children with motor founders of the *tangible
able-online disabilities resulting from lesions association (objects)
2014
of the central nervous system, FightTheStroke in
based on the ability to stimulate collaboration with the
motor learning by activating the CNR Neuroscience of the
mechanism of Mirror Neurons Università di Parma,
through the observation, chaired by Prof. Giacomo
imitation, physical interaction Rizzolatti and families of
with objects or virtual interaction children in a post-ictal
with other children with similar state
needs.
Superpower Me Augmented facemask for the Developed by Santa wearable
scfablab.unisi.it/?portfolio=s orthopaedic correction of Chiara Fab Lab and *touch (app)
uperpowerme maxillofacial disorders in Department of Medical
2018
children. The facemask embeds Biotechnologies,
temperature and pressure Policlinico Le Scotte,
sensors to monitor wear time University of Siena
and effectiveness of the therapy.
By wearing it, the child virtually
becomes a superhero who gains
power by fighting against
monsters displayed on a
smartphone application.
CARE Lab Computer Assisted REhabilitation Developed inside mixed reality
https://www.dongnocchi.it/ LABoratory is a high-tech semi- Fondazione Don Carlo touchless
@servizi/care-lab immersive sensorized room for Gnocchi (Onlus), thanks
2016
rehabilitation, driven by VITAMIN to the collaboration
(Virtual realITy plAtform for between U.O.
Motor and cognItive Neuropsychiatry and
rehabilitatioN) which provides to Rehabilitation of the
the user with specific contents Evolutionary Age and
that allow to set up a path of Innovation Development
motor and/or cognitive Department.
rehabilitation targeted and
adaptable to the needs of each
individual patient.
BODYSOUND Interactive system aimed at Developed by Polifactory touchless
siscodeproject.eu/polifactory stimulating the motor inside SISCODE H2020 wearables
2019 reactivation of children through project in collaboration (haptic)
music. It is based on choreutics with FightTheStroke, Co- *touch (app)
and uses touchless technologies designed with children
for converting movement into with a post-ictal state
sound (transform a and their families,
"choreography" into a "melody”), therapists, and
and to collect movement data, policymakers
with particular attention to the
needs of children with cerebral
palsy. It uses sound and a haptic
feedback system through
wearable devices to generate a
multi-channel feedback system
useful to guide kids in the correct
execution of movements.
These selected solutions show some new horizons of physical rehabilitation, starting from the
potential that tangible and touchless technologies can bring in terms of data collection within these care
processes. Another emerging aspect is related to new care environments that are becoming more and
more adaptive: from home to new hybrid spaces distributed in the city. The third aspect is related to the
enhancement of physical aids through the integration of digital solutions, IoT objects that integrate
sensors, actuators, software, apps to enhance the effectiveness of treatment.
In all these case, patients or patient associations were involved in the development process and all
the solutions offered a degree of customization. Another point is connected with the scale of the
solution, which is able to affects the design of interfaces and their versatility: when referring to
environmental-based solutions it is much more common to opt for touchless technologies, which are
more suitable for an overall data collection (of several users at the same time), while solutions with a
wearable interface are more suitable for a selective data collection (biometric data) of single individuals.
In the next section, we will analyse the entire development process of BODYSOUND project
describing the development process through its phases of design, prototyping and testing of different
types of interfaces (tangible and intangible) in relation to the needs of the patients/stakeholders.
3.1. Role of tangible and intangible interfaces in co-design and development
of a mobility reactivation solution: BODYSOUND
Polifactory is developing a pilot project within the European project SISCODE, Co-design for
society innovation and Science (siscodeproject.eu)3 to investigate the various physical-motor needs of
children diagnosed with infantile cerebral palsy based on the principles of proprioception, with a
specific focus on the translation of movement into sound stimuli.
The result is BODYSOUND, a product-service system based upon a co-design process carried out
with children, caregivers, therapists and with the support of FightTheStroke association
(fightthestroke.org)4 which has been developed for almost a year.
3 SISCODE is a project funded by the European Community (Horizon 2020) aimed at stimulating the use of co-creation methodologies in
RRI (Responsible Research Innovation) and Science and Innovation Policies. Coordinated by Politecnico di Milano, SISCODE is composed
of a multidisciplinary consortium of 17 partners from 13 different European countries.
4
FightTheStroke is established as a social promotion association in 2014. Following its transformation into a Foundation, from 4/10/2019, it
was entered in the register of legal entities of the Prefecture of Milan (Italy). Few of his goals are: responding to the need for knowledge of
families impacted by the management of a survivor of Stroke and Cerebral Palsy Childhood; educate to the awareness that children, even the
3.1.1. Co-design process and prototypes
Materialization/tangibilization is an effective way to share information about design, its purposes
and use both within academic and design teams but also with potential users. It’s also useful to
investigate and develop new design concepts, acquiring knowledge about relevant phenomena in
design, with particular attention to prototypes as experimental components, means of inquiry and
research archetypes (Wensveen and Matthews, 2014). Since experimental design research concerns also
human beings, prototypes can and will be used as boundary objects (Star, 1989; Star and Bowker, 1999)
to stimulate communication and conversation and to manage different viewpoints. For example,
boundary objects can enhance the collaboration between communities of practice (Wenger, 1998)
through co-creation, co-design and even co-prototyping processes. Indeed, the research and
experimentation that led to the development of the solution used tangible technologies and interfaces at
different stages of co-design with patients and caregivers.
The process of involvement of children suffering from cerebral palsy and their families carried out
for the development of BODYSOUND was initially based on designing and prototyping a series of
tangible experiences (based on sound manipulation). Indeed, through the use of quick&dirty
prototyping technology and experiments using prototypes as ‘technology probes’ (Hutchinson et al.
2003), the experience and comprehension of sound can be facilitated also via other senses, like touch
or sight.
The first workshops with children tested the use of technologies that could transform the intangibility
of sound into something physical, indeed tangible (e.g. Makey Makey, littlebits, etc.) and other rapid
prototyping tools, aimed at building tangible and cognitively accessible interfaces for children.
3.1.2. Tangible interfaces to experiment with kids
A first workshop (co-design and experimentation lab) was organized through several
activities/sessions, starting with playing a do-it-yourself theremin, then switching to a flat piano
interface built with Makey Makey and conductive ink. After those, the children have used a Kinect based
system to generate sound with the body and later they have created sound starting from SoundMoovz, a
motion-activated wearable
Thanks to these sessions, the researchers observed the degree of interest and involvement of the
young users in musical activities in order to carry out a first user experience analysis.
A second workshop (meet&code workshop) hosted in Facebook's Milanese headquarter, was
focused on making the children aware of the intangibility of sound through the tangibility of movement.
About 20 children participated and the group was equally composed both by children affected by
cerebral palsy and children who were not. All participants played three main roles: deejays,
choreographers and dancers. The first category was the one in charge of reproducing sound playing
with a magnified interface based on synth modules of littleBits, while the choreographers gave
instruction to the dancers about which movements were to be accomplished.
All these tests were important steps that helped us to choose the fundamental characteristics of the
final solution such as portability, resulting from caregivers’ need to have a motor
stimulation/reactivation tool out of the care and medical contexts, adaptable to different needs but also
to different pathologies (customization), using the right technologies in terms of usability but also
possible scalability of the system.
unborn ones, can be affected by brain damage; inspiring new generations and encouraging research and adoption of 'disruptive' therapies for
people with a neurodevelopmental problem.
Table 2
Project timeline and interface development
DESIGN PHASES ACTIVITY INTERFACES
Preliminary phases challenge definition no interface
co- design workshop physical interface (tools)
Co-design phase experimentation lab physical interfaces
touchless interfaces
workshop meet and code with kids physical interfaces
Prototyping first development touchless interface
test phases with kids beta test: touchless interface
Test phases test phases with therapists beta test: touchless interface
service co-creation physical interface (tools)
Service Co-design beta test: touchless interface
Phases
Developing phase second development wearable interface (haptic)
3.1.3. The result
The result of this process and its test is a virtual system where gamification elements help the motor
stimulation and – possibly - reactivation of the limbs by encouraging the children/users to use and move
the plegic part through the execution of a series of choreographies. Guided by the visual interface of
the game, the child can perceive the movement performed and the position in the playing space through
its own reflection in the monitor in the form of an avatar. Besides, the system can detect gestures through
a simplified system of motion capture and return in real-time one or more sound feedbacks, producing
a melody when performing the correct movement.
The system uses a touchless technology (Microsoft Azure Kinect) for body tracking, although space
coordinates and the angles between nodes of human body, and an audio-video system in combination
with a software developed by our team. Every function and interaction of the software (calibration,
activity selection, degree of difficulty and speed, user profiles, collection, analysis and data history) is
managed through a dashboard that is given to a therapist who assists during the use of BODYSOUND.
The child will see all the back-end data related to the various sessions filtered through a visual and/or
sound feedback that highlights the good performance of the session, and that motivates him/her to
continue in the following sessions. From the dashboard, instead, it is possible to see the frequency of
the activities for each single profile, the correctness of the movements and observe the trend in the
medium/long term.
After carrying out a series of user tests, and taking into account the needs of the categories of users
analysed, it was decided to introduce a greater user involvement by developing a multi-channel
feedback system, to guide the child to the correct execution of the movement: in addition to the visual
feedback of the avatar and the auditory feedback of sound, we are integrating a set of haptic feedback
through a wearable device as an augmentative and more performative experience in terms of motor
reactivation.
Figure 1: First test with therapists
As initially hypothesized, and then confirmed through the first tests, we observed that the immersion
in the virtual and multisensory environment has transformed the repetition of tedious training exercises
into stimulating and involving activities. Children perceived the experience as a playful-recreational,
non-rehabilitative activity and responded to the stimuli even with plegics limbs spontaneously without
the need to be stimulated in doing so.
Figure 2: Test with kids
The system, that was created for this precise category of users may be also extended to all children
(and not only) without particular impairments. Inclusivity is an additional characteristic that makes
training sessions less tedious and more similar to a recreational and playing moment.
4. Conclusions
Design-driven Patient Innovation is based on the observation and the recognition of a fact: patients
(especially chronic patients) facing everyday challenges connected with their status, become experts of
their disease and of the all problems related to it.
New areas of research and new technologies can orienteer and contribute to the emergence of new
types of solutions, such as software for remote rehabilitation, wearable devices or IoT aids (where
products are integrated by apps and software) that will make possible the development of rehabilitative
or gamified platforms.
Within co-design processes physical or virtual interfaces can become an important tool to enable
users: for research phases, and for prototyping phases, not only for the final development of solutions.
The nature of the chosen interfaces (tangible, touchless, wearables) depends very much on the subject
that develops the solution: hybrid places like Fab Lab and makerspace have an advantage over the
development of tangible interfaces, due to the nature of the place and the available technologies.
In the development of product-service systems with higher level of complexity, different forms of
interfaces, often addressed to different categories of users, coexist; this is particularly evident in the
healthcare sector, also because of the need for heterogeneous data collection.
For this type of solutions, the challenge is scale-up: how from prototyping or pilot is possible a
change of scale, especially when we refer to the development of product-service systems, working on
the accessibility of the regulatory and certification phase of the process, notoriously one of the most
critical in the development process of healthcare solutions.
Another challenge concern places. The use of hybrid places (open and distributed in the city,
research centres, hospitals, schools, laboratories and businesses) is particularly relevant, since these
spaces foster collaboration between user-patients, designers and healthcare specialists, sharing the
access to a repertoire of technologies and experience and giving the possibility to develop demonstrators
that allow users and other relevant stakeholders to know and touch the results of new innovation models.
5. References
[1] Dreier, J. P. N., 2016. Exploring the diffusion of patient innovations: a multiple-case study (Doctoral
dissertation).
[2] Eurispes, 2017, Rapporto Italia 2017.
[3] European Commission, 2012. eHealth Action Plan 2012-2020: Innovative healthcare for the 21st century
(ec.europa.eu/digital-single-market/news/ehealth-action-plan-2012-2020-innovative-healthcare-21st-
century)
[4] Hutchinson, H., Mackay, W., Westerlund, B., Bederson, B. B., Druin, A., Plaisant, C., ... & Roussel, N., 2003,
April. Technology probes: inspiring design for and with families. In Proceedings of the SIGCHI conference
on Human factors in computing systems (pp. 17-24).
[5] Maffei, S., Bianchini, M., Parini, B., & Delli Zotti, E. 2017. MakeToCare. Un ecosistema di attori e soluzioni
user-centered per l'innovazione nel campo dell'healthcare. Libraccio Editore.
[6] Maffei, S., Bianchini, M., Parini, B., & Cipriani, L.,2 019. MakeToCare2. La patient innovation in Italia tra
progetto e mercato. Libraccio Editore.
[7] Matthews, Ben & Wensveen, Stephan. (2015). Prototypes and prototyping in design research.
[8] Oliveira, P., Zejnilovic, L., Canhão, H. and von Hippel, E. (2015). Patient innovation under rare diseases and
chronic needs. Orphanet Journal of Rare Diseases
[9] Seyfang, G., & Smith, A. (2007). Grassroots innovations for sustainable development: Towards a new research
and policy agenda. Environmental politics, 16(4), 584-603.
[10] Star, S.L. (1989). The Structure of Ill-Structured Solutions: Heterogeneous Problem Solving, Boundary
Objects and Distributed Artificial Intelligence. In Huhns and Gasser (eds.): Distributed Artificial Intelligence.
Morgan Kaufmann.
[11] Star, S.L. and G. Bowker (1999). Sorting Things Out. MIT Press.
[12] Wenger, E. (1998). Communities of Practice: Learning, Meaning, and Identity. Cambridge: Cambridge
University Press