This paper explores the role of tangible and intangible technologies in the development of healthcare solutions that actively involve patients and caregivers in the ideation and development phases through co-design and co-creation processes. In the first part of the document, we frame the characteristics of patient innovation - the phenomenon of user-driven healthcare (Olivera et al. 2015) - in relation to the category of solutions developed, processes and emerging technologies. The second part is focused on a case study called BODYSOUND (a pilot of a H2020 European research) and analyses the co-design process adopted to develop a product-service system for rehabilitation, based on a series of testing of tangible and nontangible technologies in an attempt to identify a range of opportunity and scenarios. The final part systematizes the results of the analysis and tries to identify a series of challenges that bring this kind of solutions to the market and to users.
The eHealth 2012-2020 Action Plan1 of the European Commission provides a roadmap to empower patients and healthcare workers. The Action Plan link up devices and technologies, fund research towards the personalized medicine of the future, support research, development and innovation in eHealth and wellbeing to address the lack of available user-friendly tools and services, promoting at the same time policy dialogue and international cooperation on eHealth on a global level.
Accessibility to digital (intangible) solutions development tools can help patients to operate in a more specific dimension of personal fulfilment. These types of solutions are easier to deploy and, partly, also for non-experts (small investments in terms of time, cost and expertise). The regulatory and certification phase of the process, known to be one of the most critical, especially in the healthcare sector, is another factor that facilitates intangible (services) solutions. For these reasons, there are many solutions having these characteristics that are designed or developed within patient innovation processes. Among other advantages, e-health allows to develop solutions that are advantageous from the point of view of care: personalized medical care, portability, continuity of treatment, reaching more patients, involvement of more patients and quantified measurement of efficiency.
Focusing on Grassroots Innovation and co-creation processes, Fab Labs and makerspaces2 are recently emerging as enabling places to involve users, including patients, in co-design processes. Community-based labs are places where creative professionals, makers and amateurs, practitioners and researchers, can meet together and access to knowledge, technologies and competences, where they can easily experiment and prototype tailor-made solutions moreover sharing their results with a broader audience. 3. Role of tangible and intangible interfaces in co-design solution for rehabilitation: the state of the art in Italy
In Italy, beyond the bottom-up healthcare solutions developed by patients, caregivers or medical staff, the number of tele-health service-products, e-health app based and digital platform has increased: among the 150 solutions mapped in MakeToCare2 report (Maffei et al., 2019), 31 are digital services (apps, platforms) already released or forthcoming, and 71 are product-service systems. These last solutions have physical/tangible evidence (products) but are also accompanied by a service component, more or less structured (IoT connected objects, such as a sensorized band supported by a mobile application).
The research reports MakeToCare1 and MakeToCare2
The following cases have been selected as examples to demonstrate that tangible and touchless
technologies can support the development of heterogeneous solutions for rehabilitation or
telerehabilitation, taking advantage of gamification, data collection and different combinations of interfaces
for obtain solution of personalized care.
1 European Commission (2012). eHealth Action Plan 2012-2020: Innovative healthcare for the 21st century
to perform a personalized rehabilitation therapy with constant medical supervision through a series of games played on tablet.
Interactive tele-rehabilitation designed for children with motor disabilities resulting from lesions of the central nervous system, based on the ability to stimulate motor learning by activating the mechanism of Mirror Neurons through the observation, imitation, physical interaction with objects or virtual interaction with other children with similar needs.
Augmented facemask for the orthopaedic correction of maxillofacial disorders in children. The facemask embeds temperature and pressure sensors to monitor wear time and effectiveness of the therapy. By wearing it, the child virtually becomes a superhero who gains power by fighting against monsters displayed on a smartphone application.
LABoratory is a high-tech semiimmersive sensorized room for rehabilitation, driven by VITAMIN (Virtual realITy plAtform for
Motor and cognItive rehabilitatioN) which provides to the user with specific contents that allow to set up a path of motor and/or cognitive rehabilitation targeted and adaptable to the needs of each individual patient.
srl, Co-designed with patients suffering from stroke, multiple sclerosis or Parkinson's disease
founders of the
association
FightTheStroke in collaboration with the CNR Neuroscience of the
Università di Parma, chaired by Prof. Giacomo Rizzolatti and families of children in a post-ictal state
Chiara Fab Lab and Department of Medical
Biotechnologies, Policlinico Le Scotte, University of Siena
Fondazione Don Carlo Gnocchi (Onlus), thanks to the collaboration
between U.O.
Neuropsychiatry and Rehabilitation of the
Evolutionary Age and Innovation Development
Department.
touch haptic touchless *tangible (objects)
wearable *touch (app) mixed reality touchless
Interactive system aimed at
stimulating the motor reactivation of children through music. It is based on choreutics and uses touchless technologies for converting movement into
sound (transform a "choreography" into a "melody”), and to collect movement data, with particular attention to the needs of children with cerebral palsy. It uses sound and a haptic
feedback system through wearable devices to generate a multi-channel feedback system useful to guide kids in the correct execution of movements.
Developed by Polifactory inside SISCODE H2020 project in collaboration with FightTheStroke, Codesigned with children with a post-ictal state and their families, therapists, and policymakers touchless wearables
(haptic) *touch (app)
These selected solutions show some new horizons of physical rehabilitation, starting from the potential that tangible and touchless technologies can bring in terms of data collection within these care processes. Another emerging aspect is related to new care environments that are becoming more and more adaptive: from home to new hybrid spaces distributed in the city. The third aspect is related to the enhancement of physical aids through the integration of digital solutions, IoT objects that integrate sensors, actuators, software, apps to enhance the effectiveness of treatment.
In all these case, patients or patient associations were involved in the development process and all the solutions offered a degree of customization. Another point is connected with the scale of the solution, which is able to affects the design of interfaces and their versatility: when referring to environmental-based solutions it is much more common to opt for touchless technologies, which are more suitable for an overall data collection (of several users at the same time), while solutions with a wearable interface are more suitable for a selective data collection (biometric data) of single individuals.
In the next section, we will analyse the entire development process of BODYSOUND project describing the development process through its phases of design, prototyping and testing of different types of interfaces (tangible and intangible) in relation to the needs of the patients/stakeholders. 3.1. Role of tangible and intangible interfaces in co-design and development of a mobility reactivation solution: BODYSOUND
Polifactory is developing a pilot project within the European project SISCODE, Co-design for society innovation and Science (siscodeproject.eu)3 to investigate the various physical-motor needs of children diagnosed with infantile cerebral palsy based on the principles of proprioception, with a specific focus on the translation of movement into sound stimuli.
The result is BODYSOUND, a product-service system based upon a co-design process carried out with children, caregivers, therapists and with the support of FightTheStroke association (fightthestroke.org)4 which has been developed for almost a year. 3 SISCODE is a project funded by the European Community (Horizon 2020) aimed at stimulating the use of co-creation methodologies in RRI (Responsible Research Innovation) and Science and Innovation Policies. Coordinated by Politecnico di Milano, SISCODE is composed of a multidisciplinary consortium of 17 partners from 13 different European countries. 4 FightTheStroke is established as a social promotion association in 2014. Following its transformation into a Foundation, from 4/10/2019, it was entered in the register of legal entities of the Prefecture of Milan (Italy). Few of his goals are: responding to the need for knowledge of families impacted by the management of a survivor of Stroke and Cerebral Palsy Childhood; educate to the awareness that children, even the
Materialization/tangibilization is an effective way to share information about design, its purposes
and use both within academic and design teams but also with potential users. It’s also useful to
investigate and develop new design concepts, acquiring knowledge about relevant phenomena in
design, with particular attention to prototypes as experimental components, means of inquiry and
research archetypes (Wensveen and Matthews, 2014). Since experimental design research concerns also
human beings, prototypes can and will be used as boundary objects
The process of involvement of children suffering from cerebral palsy and their families carried out
for the development of BODYSOUND was initially based on designing and prototyping a series of
tangible experiences (based on sound manipulation). Indeed, through the use of quick&dirty
prototyping technology and experiments using prototypes as ‘technology probes’
The first workshops with children tested the use of technologies that could transform the intangibility of sound into something physical, indeed tangible (e.g. Makey Makey, littlebits, etc.) and other rapid prototyping tools, aimed at building tangible and cognitively accessible interfaces for children.
A first workshop (co-design and experimentation lab) was organized through several activities/sessions, starting with playing a do-it-yourself theremin, then switching to a flat piano interface built with Makey Makey and conductive ink. After those, the children have used a Kinect based system to generate sound with the body and later they have created sound starting from SoundMoovz, a motion-activated wearable
Thanks to these sessions, the researchers observed the degree of interest and involvement of the young users in musical activities in order to carry out a first user experience analysis.
A second workshop (meet&code workshop) hosted in Facebook's Milanese headquarter, was focused on making the children aware of the intangibility of sound through the tangibility of movement. About 20 children participated and the group was equally composed both by children affected by cerebral palsy and children who were not. All participants played three main roles: deejays, choreographers and dancers. The first category was the one in charge of reproducing sound playing with a magnified interface based on synth modules of littleBits, while the choreographers gave instruction to the dancers about which movements were to be accomplished.
All these tests were important steps that helped us to choose the fundamental characteristics of the final solution such as portability, resulting from caregivers’ need to have a motor stimulation/reactivation tool out of the care and medical contexts, adaptable to different needs but also to different pathologies (customization), using the right technologies in terms of usability but also possible scalability of the system. unborn ones, can be affected by brain damage; inspiring new generations and encouraging research and adoption of 'disruptive' therapies for people with a neurodevelopmental problem.
ACTIVITY challenge definition co- design workshop experimentation lab first development test phases with kids test phases with therapists service co-creation workshop meet and code with kids physical interfaces INTERFACES no interface physical interface (tools) physical interfaces touchless interfaces touchless interface beta test: touchless interface beta test: touchless interface physical interface (tools) beta test: touchless interface
The result of this process and its test is a virtual system where gamification elements help the motor stimulation and – possibly - reactivation of the limbs by encouraging the children/users to use and move the plegic part through the execution of a series of choreographies. Guided by the visual interface of the game, the child can perceive the movement performed and the position in the playing space through its own reflection in the monitor in the form of an avatar. Besides, the system can detect gestures through a simplified system of motion capture and return in real-time one or more sound feedbacks, producing a melody when performing the correct movement.
The system uses a touchless technology (Microsoft Azure Kinect) for body tracking, although space coordinates and the angles between nodes of human body, and an audio-video system in combination with a software developed by our team. Every function and interaction of the software (calibration, activity selection, degree of difficulty and speed, user profiles, collection, analysis and data history) is managed through a dashboard that is given to a therapist who assists during the use of BODYSOUND. The child will see all the back-end data related to the various sessions filtered through a visual and/or sound feedback that highlights the good performance of the session, and that motivates him/her to continue in the following sessions. From the dashboard, instead, it is possible to see the frequency of the activities for each single profile, the correctness of the movements and observe the trend in the medium/long term.
After carrying out a series of user tests, and taking into account the needs of the categories of users analysed, it was decided to introduce a greater user involvement by developing a multi-channel feedback system, to guide the child to the correct execution of the movement: in addition to the visual feedback of the avatar and the auditory feedback of sound, we are integrating a set of haptic feedback through a wearable device as an augmentative and more performative experience in terms of motor reactivation.
As initially hypothesized, and then confirmed through the first tests, we observed that the immersion in the virtual and multisensory environment has transformed the repetition of tedious training exercises into stimulating and involving activities. Children perceived the experience as a playful-recreational, non-rehabilitative activity and responded to the stimuli even with plegics limbs spontaneously without the need to be stimulated in doing so.
The system, that was created for this precise category of users may be also extended to all children (and not only) without particular impairments. Inclusivity is an additional characteristic that makes training sessions less tedious and more similar to a recreational and playing moment.
Design-driven Patient Innovation is based on the observation and the recognition of a fact: patients (especially chronic patients) facing everyday challenges connected with their status, become experts of their disease and of the all problems related to it.
New areas of research and new technologies can orienteer and contribute to the emergence of new types of solutions, such as software for remote rehabilitation, wearable devices or IoT aids (where products are integrated by apps and software) that will make possible the development of rehabilitative or gamified platforms.
Within co-design processes physical or virtual interfaces can become an important tool to enable users: for research phases, and for prototyping phases, not only for the final development of solutions. The nature of the chosen interfaces (tangible, touchless, wearables) depends very much on the subject that develops the solution: hybrid places like Fab Lab and makerspace have an advantage over the development of tangible interfaces, due to the nature of the place and the available technologies.
In the development of product-service systems with higher level of complexity, different forms of interfaces, often addressed to different categories of users, coexist; this is particularly evident in the healthcare sector, also because of the need for heterogeneous data collection.
For this type of solutions, the challenge is scale-up: how from prototyping or pilot is possible a change of scale, especially when we refer to the development of product-service systems, working on the accessibility of the regulatory and certification phase of the process, notoriously one of the most critical in the development process of healthcare solutions.
Another challenge concern places. The use of hybrid places (open and distributed in the city, research centres, hospitals, schools, laboratories and businesses) is particularly relevant, since these spaces foster collaboration between user-patients, designers and healthcare specialists, sharing the access to a repertoire of technologies and experience and giving the possibility to develop demonstrators that allow users and other relevant stakeholders to know and touch the results of new innovation models. 5. References