Demonstration of a Sensor-based App for Self-Monitoring of Medicine Intake Selima Curci Alessandro Mura Abstract University of Cagliari University of Cagliari Accurate adherence to prescribed medications is essen- selimacurci@gmail.com alessandro.mura193@gmail.com tial for the effectiveness of therapies, but several studies Daniele Riboni show that when patients are responsible for treatment ad- University of Cagliari ministration, poor adherence is prevalent. Existing apps riboni@unica.it to support self-administration of drugs may interfere with the normal routine of patients by providing unnecessary re- minders. More sophisticated solutions, including the use of smart packaging and ingestible sensors, are currently restricted to patients involved in a few clinical studies. In this paper, we demonstrate a novel app to support self- administration of drugs without interfering with the patient’s routines. The system relies on cheap wireless sensors at- tached to medicine boxes to detect medicine intake. The app uses machine learning to detect intake events, and active learning to improve recognition based on the user’s feedback. In the demonstration, we show a working pro- totype of the system, which includes a Web dashboard for physicians to monitor the rate of intakes. Author Keywords Medicine intake monitoring; e-health; activity recognition. Copyright is held by the author/owner(s). CHItaly ’17, September 18-20, 2017, Cagliari, Italy. ACM Classification Keywords H.5.m [Information interfaces and presentation (e.g., HCI)]: Miscellaneous System overview The system is depicted in Figure 1, and includes a smart- phone running the smart reminder app, tiny Bluetooth sen- sors, named tags, attached to medicine boxes (Figure 2), and communication with the cloud to acquire data about tags and to process the data at the server side. The app ex- ploits user feedback to fine-tune medicine intake recognition to the patient’s habits. A Web-based dashboard is available to clinicians for inspecting the history of medicine intakes of their patients. Figure 2: A tag stuck to a medicine box. The app is part of the DomuSafe research project1 , and can be used both by patients participating to the project’s exper- imental evaluation, and by other users. The app data (ther- to the experiment is identified by a unique code; the associ- apies, motion data, medicine intakes, mood and pain val- ation among codes and identities is known by the clinician ues) are periodically communicated to a server in the cloud. only. The server provides a Web-based dashboard through which clinicians can evaluate the adherence to prescriptions of The app has been developed for the Android platform, us- patients participating to the evaluation, and inspect the ing the Weka [2] libraries for implementing the ML algo- trends of pain and mood. Communication with the cloud is rithms. In the current implementation, we adopt Estimote2 done through an encrypted channel. The data are stored on stickers as our tags. Stickers, as the one shown in Figure 2, the server in an anonymous form. Each patient participating have an adhesive side that makes it easy to stuck them to medicine boxes. Their communication range is sufficient to cover most apartments. Stickers are disposable and have a life time of approximately one year. At the time of writing, their cost is ten US dollars each. The Web dashboard is implemented in PHP and HTML5. A detailed description of our system, including machine learning methods to detect intake actions based on sensor data, can be found in [1]. App interface Figure 1: System overview. The app’s name is DomuPharm; the app can be down- loaded for free from Google Play. As in normal pill reminder 1 2 http://sites.unica.it/domusafe/ https://estimote.com/ (a) Adding a new therapy (b) Associating a tag to a therapy (a) List of therapies (b) Current medicines to take Figure 3: New therapy and tag association Figure 4: Therapies and current schedule apps, the patient manually fills his/her therapies and the within a one hour interval from the prescribed time). It is red prescribed times of intake (Fig. 3(a)). However, as shown if the intake is missed. It is grey if the medicine is yet to be in Figure 3(b), through the smart reminder app, the patient taken. also associates each therapy to a colored tag, which is ac- tually a tiny Bluetooth low energy (BLE) beacon with an When moved, the tag broadcasts packets containing its integrated accelerometer, and sticks it to the medicine box. identification number and tri-axial acceleration. Those pack- ets are acquired by the app and analysed by a machine The user can inspect the list of therapies (Figure 4(a)). learning algorithm, which is in charge of classifying the Therapies can be modified, removed, and paused. While movements of the medicine box in either “intake action” a therapy is paused, the app does not monitor its intake. Of or “other action”. course, a paused therapy can subsequently be re-activated. The “Today” activity shows the therapies to be taken in the When the app detects a medicine intake action at the pre- current time of the day (Figure 4(b)). The relative item is scribed time, the patient receives an unobtrusive screen no- green if the medicine has been taken at the right time (i.e., tification (the lowest notification shown in Figure 5(b)) that immediately draw his/her attention, and the upper notifica- tion shown in Figure 5(b) is issued. Therefore, when the patient takes the drug at the right time, the notification is much less invasive than when he/she misses the intake. Anyway, the patient can modify the behavior of notifications by accessing the settings entry. The patient can either con- firm that he/she forgot to take the prescribed drug, or may report a misprediction of the app, indicating the actual time of intake. The app also has a “performance” function dis- playing the rates of correct intakes and average mood and pain values in the current day, week and month. Conclusion and future work In this demonstration, we show a novel system to support self-administration of medicines through context-aware re- minders that do not interfere with the normal routine of the patient. The system is based on a smartphone app and (a) Intake data (b) Context-aware reminders cheap wireless sensors to be attached to medicine boxes. Future work includes experimenting our system with real Figure 5: Intake data and reminders patients, and improving the interface and recognition accu- racy based on the experimental findings. produces a vibration of the smartphone but no sound. That Acknowledgments notification reminds the patient to confirm or refute the de- This work was partially supported by the “DomuSafe” project, tection, and to fill the scales of pain and mood (Figure 5(a)). funded by Sardinia regional government (CRP 69, L.R. 7 agosto 2007, n.7). The mood section contains three icons that represent the emotional states: happy, neutral, and sad. The pain section References includes five icons to rate the pain experienced by the pa- [1] Selima Curci, Alessandro Mura, and Daniele Riboni. tient, from no pain to unbearable pain. The icons of mood 2017. Toward Naturalistic Self-Monitoring of Medicine and pain change color when selected. Hence, the user has Intake. In Proc. of the 12th Biannual Conference of the an immediate feedback about his/her action. The values Italian SIGCHI Chapter (CHItaly), to appear. of mood and pain scales can be inspected by clinicians on [2] Mark A. Hall, Eibe Frank, Geoffrey Holmes, Bernhard their Web dashboard for further analyses. On the contrary, Pfahringer, Peter Reutemann, and Ian H. Witten. 2009. when the app detects that the intake was skipped, the pa- The WEKA data mining software: an update. SIGKDD tient is alerted with both a vibration and a ring, in order to Explorations 11, 1 (2009), 10–18.