In this workshop paper we present an overview of our research into understanding how to explain complex machine learning (ML) health platforms to members of the general public who might benefit from them, specifically those who have Type 2 Diabetes (T2D). The availability of home health sensor technology is increasing; however, it is unclear how to explain these platforms to potential users so that they can make an 'informed decision' on the adoption of that platform within their home. Through a user-centered-design approach, we have completed a case study with three studies that have (1) Given an overview of a complex ML platform, that of SPHERE; (2) Identified how the participants would like us to explain this content and (3) Created and validated an explanation document that presents, at a high-level the SPHERE platform. We present our findings on the priority of understanding how and why the platform can help them over the technical detail of the platform itself.
In many parts of our daily lives, Artificial
Intelligence (AI) and Machine Learning (ML)
have become ubiquitous in assisting our
decision making, e.g., suggesting films to
watch on Netflix [
In the domain of healthcare, Holzinger et al
[
In the medical field, there is a legal and
ethical requirement for the patient and clinician
to go through a process of ‘informed consent’
[
To understand how we should explain
complex ML/AI platforms to members of the
general public, we conducted a case study that
focused on the SPHERE platform and members
of the general public with Type 2 Diabetes
(T2D), where most of the care takes place
outside clinical settings [
The seven devices
Water sensor
Appliance sensor b d g Environmental
Sensor SPHERE home gateway a c f
Electricity (mains)
e
Wearable The ten sensors
Silhouette sensor Vibrations Electricity Silhouette Movement Light levels Air pressure Motion Humidity Temperature
Appliance usage
Using a user-centered-design methodology to define the explanation of the SPHERE platform, we first completed semi-structured interviews with eight members of the SPHERE team who had built and maintained the system. After this, we ran a second study which presented alternative designs about the platform’s hardware (figure 2a-c), the ground truthing of the data (figure 2d-f) and the ML process unsupervised learning (figure 2g-i) to nine people with Type 2 diabetes and members of their households who might also have to live with this domestic health technology. From the findings of these two studies, we created an explanation document (figure 4) that presents and explains the SPHERE platform to the general public who had T2D. Finally, we ran a validation study that reviewed how the explanation document was used in an onboarding/set-up session with technicians and how the SPHERE system and the document was interpreted and understood. 2.1.
Our first challenge was to understand what SPHERE was capable of, its processes, hardware and ML/AI requirements. With this aim in mind, we conducted semi-structured interviews with eight out of eleven of the team members. The team members had been working on the project from two to six years and had mixed roles within SPHERE (2 x Deployment technicians, 3 x ML experts, 1 x Hardware engineer, 1 x Researcher and 1 x Community liaison).
By interviewing these team members with a diverse range of roles within SPHERE, we were able to gain an overview of all aspects of the complex platform. We conducted the interviews individually within a universitybased meeting room, audio-recorded and then transcribed verbatim. Using affinity diagramming and a bottom-up approach we created a total of 681 post-it notes (Machine Learning x 245, Research x 63, Community Engagement x 68, Hardware x 100 and Deployment Technician x 205). Once the five job roles (deployment technicians, machine learning, research, hardware and community liaison) had been initially coded into themes, the post-it notes were organized by the first author into 35 further themes that were then broken down into three overarching themes. These overarching themes were (1) Hardware and Network; (2) Installation, Training and Data gathering; (3) Machine learning and Data visualization. We then transferred these themes into a Microsoft Word document. At that stage, the first author merged any duplicated content. We then asked the eight core team members who took part in the interviews to review the document to confirm the draft document was technically correct.
These three overarching themes helped us define the platform, for example, capturing seven sensor devices (Figure 1a-g) and ten individual sensors (Figure 1) with technical and positioning limitations. We also captured the installation process where the deployment technicians will visit a participant’s home four times (survey, installation, maintenance and removal) and that the data collected is saved on a hard disk within the participants home and with their permission and processed through supervised and unsupervised machine learning.
Once we had gained an understanding of the complex platform, our next challenge was to define how to present the information to our participants. For this study we focused on one area of each of the overarching themes: For Hardware & network we selected the most technically complex sensor, the ‘environmental sensor’ (figure 2a-c), for Installation, training & data collection, we selected ‘ground truth’ (figure 2d-f) as this process informs the ML algorithms. For Machine learning & data visualization, we selected 'Unsupervised learning' (figure 2g-i) as this is the more speculative form of ML. Through a design workshop with six participants (three university researchers and three members of a community engagement charity), we focused on the ‘environmental sensor’ (figure 2a-c) and created three alternative designs that presented the platforms information at different technical levels, detail, approaches to language and visual elements. We then, used these design decisions to create three alternative designs for the further two areas of the platform, ‘ground truth’ (figure 2d-f) and ‘unsupervised learning’ (figure 2g-i).
We presented these nine designed documents (figure 2) to nine participants who either had T2D or lived with someone who did. The nine participants (five female, four male) were aged between 25 to 74, with a varying education level ranging from that of entry-level to PhD. Six participants had T2D, and three participants lived with someone who did. All participants owned a smartphone, four participants had an IoT device such as Amazon Alexa or Google Home. Two participants (AD2 and AD6) had weather stations at home and due to this had prior knowledge of sensors and their capabilities. The Environmental Sensors were presented first with the alternative designs alternated (using the Latin square method), then the Ground Truth and finally Unsupervised Learning.
For all three areas (environmental sensor, ground truth and unsupervised learning), the participants considered the alternative design with the most technical information and detail to be far too complex, scary or off putting. The participants additionally preferred the language as used in the simpler design alternatives as it used common language an non-technical words.
For the environmental sensor (figure 2a-c), the participants requested that the image of the sensor be the version from figure 2c, with the sensor measurements as in figure 2a in both centimeters and inches. They requested an understanding of where the position of the sensors within the home, however, they did not like the list in figure 2a or the storyboard in figure 2b as they provided unnecessary information (the deployment technician would fit the sensor). They preferred the more structural visual approach to the rules of the sensor placement as in figure 2b and requested more of a description of what each sensor did.
With the ‘ground truth’ (figure 2d-f) the participants considered the simpler version (figure 2f) to be just enough information and were positive with the storyboard flow. The other two alternatives (figure 2d and 2e) were both thought of as too much information and not relevant to the participants as the deployment technician would complete the process.
Finally, for ‘unsupervised learning’ the participants were confused by the charts and graphs considering figure 2i as the better description with a few changes. These changes included the change of an icon so that it fits the descriptive text better and combining the whole of figure 2i with the righthand side of figure h, here showing the participant how the ‘unsupervised machine learning’ works and showing the results in an understandable chart.
Using this feedback, we then updated the
page designs (figure 3) to match the participants
preferences. For the environmental sensor
(figure 3a), we created an illustration to present
the sensor placement location and added
information about the sensor’s limitations as
suggested by Cai et al [
Our next challenge was to validate this explanation document (figure 4) to understand if we had created a translation of the SPHERE platform that potential participants would feel they could use to make an ‘informed decision’. Overall, the participants liked the document, all understanding at a high-level the data collected and how that data would be used to identify their daily activity. The participants did ask for a number of updates (e.g. page order, image updates and a reduction of pages within the document) and even though they understood the platform (at a high-level) they wanted to understand why SPHERE was useful to them as individuals with T2D.
Our next steps are to investigate how we can incorporate the findings from the validation study so that we reduce the number of pages and not just explain the technical aspect of the SPHERE platform but also understand how to explain why this platform would be beneficial to the participants without influencing their decision in consenting to have the platform within their home. Additionally, we wish to investigate the best medium to presenting this content (Paper or video) and understand how this explanation document can work within the first steps of creating a process for the selfinstallation of the SPHERE platform.
We would like to thank Sue Mackinnon, Jess Linington, Zoe Banks Gross and Fiona Dowling from Knowle West Media Centre for their support on this project. We would additionally like to thank the SPHERE team members who engaged in this project and for taking their time to explain their work to us.
This work was completed through the SPHERE Next Steps Project funded by the UK Engineering and Physical Sciences Research Council (EPSRC), Grant EP/R005273/1. explanation