The BODYSOUND case A tangible prototype for co-designing "intangible" healthcare solutions Laura Cipriania, Andrea Ascania, Carla Sedinia, Massimo Bianchinia and Stefano Maffeia a Politecnico di Milano, Design Department, via Durando 38A, Milano, Italy Abstract This paper explores the role of tangible and intangible technologies in the development of healthcare solutions that actively involve patients and caregivers in the ideation and development phases through co-design and co-creation processes. In the first part of the document, we frame the characteristics of patient innovation - the phenomenon of user-driven healthcare (Olivera et al. 2015) - in relation to the category of solutions developed, processes and emerging technologies. The second part is focused on a case study called BODYSOUND (a pilot of a H2020 European research) and analyses the co-design process adopted to develop a product-service system for rehabilitation, based on a series of testing of tangible and non- tangible technologies in an attempt to identify a range of opportunity and scenarios. The final part systematizes the results of the analysis and tries to identify a series of challenges that bring this kind of solutions to the market and to users. Keywords Patient innovation, Co-design processes, Tangible interfaces, Touchless technologies, Rehabilitation, Healthcare 1. Introduction: Framing the patient innovation perspective Patient Innovation (PI) can be delineate as a phenomenon belongs to the wider field of Grassroots Innovation. In particular, PI has be defined as “a network of activists and organizations generating novel bottom-up solutions for sustainable development and sustainable consumption; solutions that respond to the local situation and the interests and values of the communities involved” (Seyfang & Smith, 2007: 585). Another research (Dreier et al., 2016) identified three common characteristics of Patient Innovation. The first one is independence: in most cases, the idea comes from personal condition and to face a personal issue. The second one is repetition: in many cases the identified solution already exists, but the patient did not know. The third one is sharing: generally, when a solution is found, patients tend to share their positive experience with other people in their same condition lacking (without?) a connection with doctors and healthcare professionals. Upstream of this phenomenon, there is an emerging lack of confidence in the healthcare system: as recently reported by Eurispes, 47.4% of Italian citizens are more inclined to opt for self-diagnosis and self-care (Eurispes, 2017). Involving people to envision new solutions in the care process could represent the right choice instead of leaving them to the do-it-yourself care and medicine. Society changes bring inevitable new healthcare needs: as highlighted by OECD Health Statistics 2018 (OECD, 2018), the health system should be more people-centered, taking also the advantage of the evolution of digital technologies to prevent and facing possible life-threatening diseases. Proceedings of ETIS 2020, November 16–20, 2020, Siena, Italy EMAIL: laura1.cipriani@polimi.com (A. 1); andrea.ascani@polimi.com (A. 2); carla.sedini@polimi.com (A. 3) massimo.bianchini@polimi.com (A. 4); stefano.maffei@polimi.com (A. 5); © 2020 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Wor Pr ks hop oceedi ngs ht I tp: // ceur - SSN1613- ws .or 0073 g CEUR Workshop Proceedings (CEUR-WS.org) 2. New opportunities about assistive technology in co-design The eHealth 2012-2020 Action Plan1 of the European Commission provides a roadmap to empower patients and healthcare workers. The Action Plan link up devices and technologies, fund research towards the personalized medicine of the future, support research, development and innovation in eHealth and wellbeing to address the lack of available user-friendly tools and services, promoting at the same time policy dialogue and international cooperation on eHealth on a global level. Accessibility to digital (intangible) solutions development tools can help patients to operate in a more specific dimension of personal fulfilment. These types of solutions are easier to deploy and, partly, also for non-experts (small investments in terms of time, cost and expertise). The regulatory and certification phase of the process, known to be one of the most critical, especially in the healthcare sector, is another factor that facilitates intangible (services) solutions. For these reasons, there are many solutions having these characteristics that are designed or developed within patient innovation processes. Among other advantages, e-health allows to develop solutions that are advantageous from the point of view of care: personalized medical care, portability, continuity of treatment, reaching more patients, involvement of more patients and quantified measurement of efficiency. Focusing on Grassroots Innovation and co-creation processes, Fab Labs and makerspaces2 are recently emerging as enabling places to involve users, including patients, in co-design processes. Community-based labs are places where creative professionals, makers and amateurs, practitioners and researchers, can meet together and access to knowledge, technologies and competences, where they can easily experiment and prototype tailor-made solutions moreover sharing their results with a broader audience. 3. Role of tangible and intangible interfaces in co-design solution for rehabilitation: the state of the art in Italy In Italy, beyond the bottom-up healthcare solutions developed by patients, caregivers or medical staff, the number of tele-health service-products, e-health app based and digital platform has increased: among the 150 solutions mapped in MakeToCare2 report (Maffei et al., 2019), 31 are digital services (apps, platforms) already released or forthcoming, and 71 are product-service systems. These last solutions have physical/tangible evidence (products) but are also accompanied by a service component, more or less structured (IoT connected objects, such as a sensorized band supported by a mobile application). The research reports MakeToCare1 and MakeToCare2 (Maffei et al., 2017; Maffei et al., 2019) explored some Italian examples of solutions developed with the involvement of patients, interesting for: typology of the solution developed, developers and actors involved, interface types (tangible, touchless, haptic, etc.). The following cases have been selected as examples to demonstrate that tangible and touchless technologies can support the development of heterogeneous solutions for rehabilitation or tele- rehabilitation, taking advantage of gamification, data collection and different combinations of interfaces for obtain solution of personalized care. 1 European Commission (2012). eHealth Action Plan 2012-2020: Innovative healthcare for the 21st century (ec.europa.eu/digital-single- market/news/ehealth-action-plan-2012-2020-innovative-healthcare-21st-century) 2 Besides, in many cases, Universities, Research Centres, Hospitals, etc. take part in these networks, or even promoting them (e.g. POLITECNICO at Politecnico di Milano, Lab4Living at Sheffield Hallam University, UCL Centre for Co-production in Health Research at University College London, Helix Centre located at the St Mary's Hospital in London but managed by Imperial College London and The Royal College of Art), giving Fab Labs and Makerspace a more “reliable” reputation when trying to involve different stakeholders like Academy, Government, Civil Society, and Enterprises. Table 1 Cases studies Project Typology of the solution Developers and actors Interface types developed involved Rehability Serious games that allow patients Developed by Imaginary touch www.rehability.me to perform a personalized srl, Co-designed with haptic 2014 rehabilitation therapy with patients suffering from constant medical supervision stroke, multiple sclerosis through a series of games played or Parkinson's disease on tablet. Mirrorable Interactive tele-rehabilitation Developed by the touchless en.fightthestroke.org/mirror designed for children with motor founders of the *tangible able-online disabilities resulting from lesions association (objects) 2014 of the central nervous system, FightTheStroke in based on the ability to stimulate collaboration with the motor learning by activating the CNR Neuroscience of the mechanism of Mirror Neurons Università di Parma, through the observation, chaired by Prof. Giacomo imitation, physical interaction Rizzolatti and families of with objects or virtual interaction children in a post-ictal with other children with similar state needs. Superpower Me Augmented facemask for the Developed by Santa wearable scfablab.unisi.it/?portfolio=s orthopaedic correction of Chiara Fab Lab and *touch (app) uperpowerme maxillofacial disorders in Department of Medical 2018 children. The facemask embeds Biotechnologies, temperature and pressure Policlinico Le Scotte, sensors to monitor wear time University of Siena and effectiveness of the therapy. By wearing it, the child virtually becomes a superhero who gains power by fighting against monsters displayed on a smartphone application. CARE Lab Computer Assisted REhabilitation Developed inside mixed reality https://www.dongnocchi.it/ LABoratory is a high-tech semi- Fondazione Don Carlo touchless @servizi/care-lab immersive sensorized room for Gnocchi (Onlus), thanks 2016 rehabilitation, driven by VITAMIN to the collaboration (Virtual realITy plAtform for between U.O. Motor and cognItive Neuropsychiatry and rehabilitatioN) which provides to Rehabilitation of the the user with specific contents Evolutionary Age and that allow to set up a path of Innovation Development motor and/or cognitive Department. rehabilitation targeted and adaptable to the needs of each individual patient. BODYSOUND Interactive system aimed at Developed by Polifactory touchless siscodeproject.eu/polifactory stimulating the motor inside SISCODE H2020 wearables 2019 reactivation of children through project in collaboration (haptic) music. It is based on choreutics with FightTheStroke, Co- *touch (app) and uses touchless technologies designed with children for converting movement into with a post-ictal state sound (transform a and their families, "choreography" into a "melody”), therapists, and and to collect movement data, policymakers with particular attention to the needs of children with cerebral palsy. It uses sound and a haptic feedback system through wearable devices to generate a multi-channel feedback system useful to guide kids in the correct execution of movements. These selected solutions show some new horizons of physical rehabilitation, starting from the potential that tangible and touchless technologies can bring in terms of data collection within these care processes. Another emerging aspect is related to new care environments that are becoming more and more adaptive: from home to new hybrid spaces distributed in the city. The third aspect is related to the enhancement of physical aids through the integration of digital solutions, IoT objects that integrate sensors, actuators, software, apps to enhance the effectiveness of treatment. In all these case, patients or patient associations were involved in the development process and all the solutions offered a degree of customization. Another point is connected with the scale of the solution, which is able to affects the design of interfaces and their versatility: when referring to environmental-based solutions it is much more common to opt for touchless technologies, which are more suitable for an overall data collection (of several users at the same time), while solutions with a wearable interface are more suitable for a selective data collection (biometric data) of single individuals. In the next section, we will analyse the entire development process of BODYSOUND project describing the development process through its phases of design, prototyping and testing of different types of interfaces (tangible and intangible) in relation to the needs of the patients/stakeholders. 3.1. Role of tangible and intangible interfaces in co-design and development of a mobility reactivation solution: BODYSOUND Polifactory is developing a pilot project within the European project SISCODE, Co-design for society innovation and Science (siscodeproject.eu)3 to investigate the various physical-motor needs of children diagnosed with infantile cerebral palsy based on the principles of proprioception, with a specific focus on the translation of movement into sound stimuli. The result is BODYSOUND, a product-service system based upon a co-design process carried out with children, caregivers, therapists and with the support of FightTheStroke association (fightthestroke.org)4 which has been developed for almost a year. 3 SISCODE is a project funded by the European Community (Horizon 2020) aimed at stimulating the use of co-creation methodologies in RRI (Responsible Research Innovation) and Science and Innovation Policies. Coordinated by Politecnico di Milano, SISCODE is composed of a multidisciplinary consortium of 17 partners from 13 different European countries. 4 FightTheStroke is established as a social promotion association in 2014. Following its transformation into a Foundation, from 4/10/2019, it was entered in the register of legal entities of the Prefecture of Milan (Italy). Few of his goals are: responding to the need for knowledge of families impacted by the management of a survivor of Stroke and Cerebral Palsy Childhood; educate to the awareness that children, even the 3.1.1. Co-design process and prototypes Materialization/tangibilization is an effective way to share information about design, its purposes and use both within academic and design teams but also with potential users. It’s also useful to investigate and develop new design concepts, acquiring knowledge about relevant phenomena in design, with particular attention to prototypes as experimental components, means of inquiry and research archetypes (Wensveen and Matthews, 2014). Since experimental design research concerns also human beings, prototypes can and will be used as boundary objects (Star, 1989; Star and Bowker, 1999) to stimulate communication and conversation and to manage different viewpoints. For example, boundary objects can enhance the collaboration between communities of practice (Wenger, 1998) through co-creation, co-design and even co-prototyping processes. Indeed, the research and experimentation that led to the development of the solution used tangible technologies and interfaces at different stages of co-design with patients and caregivers. The process of involvement of children suffering from cerebral palsy and their families carried out for the development of BODYSOUND was initially based on designing and prototyping a series of tangible experiences (based on sound manipulation). Indeed, through the use of quick&dirty prototyping technology and experiments using prototypes as ‘technology probes’ (Hutchinson et al. 2003), the experience and comprehension of sound can be facilitated also via other senses, like touch or sight. The first workshops with children tested the use of technologies that could transform the intangibility of sound into something physical, indeed tangible (e.g. Makey Makey, littlebits, etc.) and other rapid prototyping tools, aimed at building tangible and cognitively accessible interfaces for children. 3.1.2. Tangible interfaces to experiment with kids A first workshop (co-design and experimentation lab) was organized through several activities/sessions, starting with playing a do-it-yourself theremin, then switching to a flat piano interface built with Makey Makey and conductive ink. After those, the children have used a Kinect based system to generate sound with the body and later they have created sound starting from SoundMoovz, a motion-activated wearable Thanks to these sessions, the researchers observed the degree of interest and involvement of the young users in musical activities in order to carry out a first user experience analysis. A second workshop (meet&code workshop) hosted in Facebook's Milanese headquarter, was focused on making the children aware of the intangibility of sound through the tangibility of movement. About 20 children participated and the group was equally composed both by children affected by cerebral palsy and children who were not. All participants played three main roles: deejays, choreographers and dancers. The first category was the one in charge of reproducing sound playing with a magnified interface based on synth modules of littleBits, while the choreographers gave instruction to the dancers about which movements were to be accomplished. All these tests were important steps that helped us to choose the fundamental characteristics of the final solution such as portability, resulting from caregivers’ need to have a motor stimulation/reactivation tool out of the care and medical contexts, adaptable to different needs but also to different pathologies (customization), using the right technologies in terms of usability but also possible scalability of the system. unborn ones, can be affected by brain damage; inspiring new generations and encouraging research and adoption of 'disruptive' therapies for people with a neurodevelopmental problem. Table 2 Project timeline and interface development DESIGN PHASES ACTIVITY INTERFACES Preliminary phases challenge definition no interface co- design workshop physical interface (tools) Co-design phase experimentation lab physical interfaces touchless interfaces workshop meet and code with kids physical interfaces Prototyping first development touchless interface test phases with kids beta test: touchless interface Test phases test phases with therapists beta test: touchless interface service co-creation physical interface (tools) Service Co-design beta test: touchless interface Phases Developing phase second development wearable interface (haptic) 3.1.3. The result The result of this process and its test is a virtual system where gamification elements help the motor stimulation and – possibly - reactivation of the limbs by encouraging the children/users to use and move the plegic part through the execution of a series of choreographies. Guided by the visual interface of the game, the child can perceive the movement performed and the position in the playing space through its own reflection in the monitor in the form of an avatar. Besides, the system can detect gestures through a simplified system of motion capture and return in real-time one or more sound feedbacks, producing a melody when performing the correct movement. The system uses a touchless technology (Microsoft Azure Kinect) for body tracking, although space coordinates and the angles between nodes of human body, and an audio-video system in combination with a software developed by our team. Every function and interaction of the software (calibration, activity selection, degree of difficulty and speed, user profiles, collection, analysis and data history) is managed through a dashboard that is given to a therapist who assists during the use of BODYSOUND. The child will see all the back-end data related to the various sessions filtered through a visual and/or sound feedback that highlights the good performance of the session, and that motivates him/her to continue in the following sessions. From the dashboard, instead, it is possible to see the frequency of the activities for each single profile, the correctness of the movements and observe the trend in the medium/long term. After carrying out a series of user tests, and taking into account the needs of the categories of users analysed, it was decided to introduce a greater user involvement by developing a multi-channel feedback system, to guide the child to the correct execution of the movement: in addition to the visual feedback of the avatar and the auditory feedback of sound, we are integrating a set of haptic feedback through a wearable device as an augmentative and more performative experience in terms of motor reactivation. Figure 1: First test with therapists As initially hypothesized, and then confirmed through the first tests, we observed that the immersion in the virtual and multisensory environment has transformed the repetition of tedious training exercises into stimulating and involving activities. Children perceived the experience as a playful-recreational, non-rehabilitative activity and responded to the stimuli even with plegics limbs spontaneously without the need to be stimulated in doing so. Figure 2: Test with kids The system, that was created for this precise category of users may be also extended to all children (and not only) without particular impairments. Inclusivity is an additional characteristic that makes training sessions less tedious and more similar to a recreational and playing moment. 4. Conclusions Design-driven Patient Innovation is based on the observation and the recognition of a fact: patients (especially chronic patients) facing everyday challenges connected with their status, become experts of their disease and of the all problems related to it. New areas of research and new technologies can orienteer and contribute to the emergence of new types of solutions, such as software for remote rehabilitation, wearable devices or IoT aids (where products are integrated by apps and software) that will make possible the development of rehabilitative or gamified platforms. Within co-design processes physical or virtual interfaces can become an important tool to enable users: for research phases, and for prototyping phases, not only for the final development of solutions. The nature of the chosen interfaces (tangible, touchless, wearables) depends very much on the subject that develops the solution: hybrid places like Fab Lab and makerspace have an advantage over the development of tangible interfaces, due to the nature of the place and the available technologies. In the development of product-service systems with higher level of complexity, different forms of interfaces, often addressed to different categories of users, coexist; this is particularly evident in the healthcare sector, also because of the need for heterogeneous data collection. For this type of solutions, the challenge is scale-up: how from prototyping or pilot is possible a change of scale, especially when we refer to the development of product-service systems, working on the accessibility of the regulatory and certification phase of the process, notoriously one of the most critical in the development process of healthcare solutions. Another challenge concern places. The use of hybrid places (open and distributed in the city, research centres, hospitals, schools, laboratories and businesses) is particularly relevant, since these spaces foster collaboration between user-patients, designers and healthcare specialists, sharing the access to a repertoire of technologies and experience and giving the possibility to develop demonstrators that allow users and other relevant stakeholders to know and touch the results of new innovation models. 5. References [1] Dreier, J. P. N., 2016. Exploring the diffusion of patient innovations: a multiple-case study (Doctoral dissertation). [2] Eurispes, 2017, Rapporto Italia 2017. [3] European Commission, 2012. eHealth Action Plan 2012-2020: Innovative healthcare for the 21st century (ec.europa.eu/digital-single-market/news/ehealth-action-plan-2012-2020-innovative-healthcare-21st- century) [4] Hutchinson, H., Mackay, W., Westerlund, B., Bederson, B. B., Druin, A., Plaisant, C., ... & Roussel, N., 2003, April. Technology probes: inspiring design for and with families. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 17-24). [5] Maffei, S., Bianchini, M., Parini, B., & Delli Zotti, E. 2017. MakeToCare. Un ecosistema di attori e soluzioni user-centered per l'innovazione nel campo dell'healthcare. Libraccio Editore. [6] Maffei, S., Bianchini, M., Parini, B., & Cipriani, L.,2 019. MakeToCare2. La patient innovation in Italia tra progetto e mercato. Libraccio Editore. [7] Matthews, Ben & Wensveen, Stephan. (2015). Prototypes and prototyping in design research. [8] Oliveira, P., Zejnilovic, L., Canhão, H. and von Hippel, E. (2015). Patient innovation under rare diseases and chronic needs. Orphanet Journal of Rare Diseases [9] Seyfang, G., & Smith, A. (2007). Grassroots innovations for sustainable development: Towards a new research and policy agenda. Environmental politics, 16(4), 584-603. [10] Star, S.L. (1989). The Structure of Ill-Structured Solutions: Heterogeneous Problem Solving, Boundary Objects and Distributed Artificial Intelligence. In Huhns and Gasser (eds.): Distributed Artificial Intelligence. Morgan Kaufmann. [11] Star, S.L. and G. Bowker (1999). Sorting Things Out. MIT Press. [12] Wenger, E. (1998). Communities of Practice: Learning, Meaning, and Identity. Cambridge: Cambridge University Press