Poor diet and physical inactivity are important factors that contribute to the obesity outbreak. erefore, healthy eating habits are crucial for physical well-being. In this paper, we present the concept design and an early stage evaluation of PHARA, a personal health augmented reality assistant that recommends healthy and similar products to people in their everyday lives. We evaluated a content-based recommender system in a desktop environment (n = 15) to measure the perceived quality, as well as behavioral intentions of users. In addition, we evaluated the user interface and measured participants' perceptions of usefulness and ease of use. Whereas perceived usefulness and perceived ease of use are good, more work is required towards improving the accuracy and diversity of recommendations.
•Information systems →Decision support systems; Recommender systems; •Human-centered computing →Mixed / augmented reality;
Nowadays, recommender systems are increasingly used to support
decision-making in health-related contexts, suggesting users how
to improve their eating, exercising or sleeping behavior. An
example is the use of recommendations as a basis to algorithmically
derive balanced meal plans that meet nutritional guidelines for the
user [
Recommender Systems. van Pinxteren, Youri et al.[
Visualization. e work of C. Siawsolit et al. [
Augmented Reality. e most comparable system to PHARA is
the work of Ahn, Junho et al.[
e design of PHARA is inspired by related work, JITKM goals, personal health tracking challenges and the opportunities presented by recent advances in AR head-mounted display (HMD) technologies, such as Microso Hololens2. In this section, we describe the system architecture and the design of PHARA’s user interface.
Architecture. e architecture of PHARA is shown in Figure 1. We envision an immersive system where users can wear the HMD freely and execute tasks using input such as voice commands or hand gestures. With a reactive design3 in mind, data is streamed on demand from the servers to ensure system responsiveness. Computing of recommendations and predictions take place in the server, releasing the HMD from any other computation other than tracking and recognizing printed labels and barcodes to identify products. A Web Application also is served in a separate client, through which users can introduce and synchronize their devices with personal data. 2hps://www.microso.com/en-us/hololens 3hp://www.reactivemanifesto.org/
User Interface. e interface of PHARA is inspired by the work
of Heun, Valentin et al [
In this section, we present the design of our user studies. Results are discussed in the next section. We recruited 15 participants (3F, 12M; mean age: 27.6, SD: 5.88) via word-of-mouth. We asked them to participate in two studies to collect their thoughts towards the recommender system, and an early paper prototype to understand how PHARA can be used in real world seings. A short oine evaluation was conducted aerwards to test recommendation accuracy and diversity.
Study 1: Recommender System. We built our system using a crowdsourced database from Open Food Facts, using a contentbased approach and a similarity index to estimate similarities between food products. We generated recommendations based on similarities of products to provide recommendations about Similar Products, Healthy Products and Based on your Prole. Participants were presented with a Web browser application where they were required to create a prole with their personal data (allergies, height, weight, age and activity level). Aerwards, they were asked to select 10 favorite products to train the system. Aer creating a prole, a dashboard was shown (see Figure 3) were they could see a list of recommended products. e task was as follows: Select any products that you would like to have for dinner.
e system showed a visualization of ”MyPlate”, with suggested
indications for a balanced meal, based on Healthy Eating Plate
created by experts at Harvard School of Public Health and Harvard
Medical School4. Participants had to add products until they were
happy with their selection. When clicking on a product from the
list, the system showed the product information and three lists,
Similar Products, Healthy Products and Based on your Prole, each
including six recommendations. Aer nishing their selection for a
healthy plate, we asked users to ll out the Rese[
Study 2: Paper Prototype. We presented packages of dierent
food products from the local supermarket to the participants on
a table. We reused the design of the components from Study 1’s
application and printed them on paper, see Figure 4. Participants
were informed about the AR system and the concept in general. We
also created cards with recommendations from the system, based on
similar products and healthy alternatives. Participants were asked
to think-aloud their ideas during the experiment, and to imagine
the following scenario you are in the grocery store and you want
to get a selection of food products you would likely buy. e task
started with participants picking up any product from the table
and then, as PHARA would (automatically) do, the corresponding
detail component card (Figure 2c) was manually aached to the
front-side of the product package. When participants asked for
recommendations of healthier or similar products the
corresponding recommendations card (Figure 2a) was aached to the package
and we asked them to pick one of the recommended products from
the table. When they did so, and while holding both products,
we aached the corresponding product detail card (Figure 2c) to
the newly picked product. Aerwards, we asked participants to
compare the products using both the printed information in the
box and PHARA’s detail cards and chose the one they would likely
buy. We repeated this procedure until participants declared they
were satised with their selection. At the end of the task, we asked
participants to ll out a technology acceptance questionnaire [
Study 3: Oline Evaluation. An oine experiment was conducted by collecting activity data of the users that used the Web application recommender system. Using this data, we calculated the proportion of the recommendations that were actually suitable for the user (precision) and the variation of items in the recommendations (diversity) of the system. 4hps://www.hsph.harvard.edu/nutritionsource/healthy-eating-plate/
Recommender system. We summarize the results of Study 1 in Figure 5. Participants indicated they perceived the system as useful ¹Median = 4º on a 5-point Likert scale (one: strongly disagree, ve: strongly agree). ey also indicated that they were familiar with the products in the database ¹Median = 4º. e interface that was shown to them appeared to be adequate ¹Median = 4º. Other factors such as condence, novelty, satisfaction and behavioral intentions tended to be rated lower ¹Median = 3º. e main comments of the participants at the end of the session were related to the quality of the data which is reected in their behavioral intentions ¹Median = 3º, indicating that some products were not easy to nd due to missing information about the product. ere were also comments related to the diversity of recommendations ¹Median = 3:5º, indicating that similar products appeared too oen in the recommendations.
Paper prototype. Results are presented in Figure 6. Participant feedback tended to be positive in general (Median >= 6) on a 7-point Likert scale (one: unlikely, seven: likely), indicating that they found the system to be intuitive, easy to use and learn. However, some participants were concerned about the exibility of use ¹Median = 5º, given the HMD hardware that they would wear with ceived ease of use of PHARA. the real HDM device. Some participants had comments during the think-aloud session “e information cards look intuitive and easy to read, however more details on the object would be useful”. When asked to compare the information printed on the box with the augmented component, a participant mentioned that “Actually, this component complements the information in the box”. When asked to hold both products and compare them, a participant mentioned: “In this way, it is easy to compare both products, the information is consistent. Sometimes reading the package can be misleading and/or dicult to read”.
Oline Evaluation. Our initial test showed that recommendations had a low precision and recall, (see Figure 7). However, the numbers for Healthy Alternatives also illustrate the interest of participants in healthy products. ese metrics illustrate participant’s feedback in study 1, where they indicated that recommendations of products seemed to be useful, but factors such as accuracy and diversity were an issue. is was noted particularly when participants faced incomplete descriptions of food products, or missing values. Participants tended to stick with familiar or similar products that they felt condent with. Crowdsourced databases present a lot of opportunities by providing a large set of items supported by a devoted community of users. However, uncertainty in data quality is a challenging factor in terms of processing and presentation to the end user to be addressed. 6
We have introduced PHARA, an AR system to support decisionmaking at grocery stores and described the results of three preliminary user studies. e obtained results are encouraging and suggest that PHARA’s concept is likely a viable way to promote the adoption of healthy food buying behaviors. Future work will rene the prototype and explore its eects in larger and more diverse audiences and seings. We will explore dierent strategies to improve diversity and precision of food recommendations in future studies and dierent visualizations will be evaluated on how to eectively communicate personal health data to users.
e research has been partially nanced by the KU Leuven Research Council (grant agreement no. C24/16/017).