The purpose of this study was to evaluate the usability of a gamified application (WellWe) among public health nurses (PHN) and families with toddlers and preschool-aged children. A post-test design was used. After providing access to WellWe and using it during one health visit in a child health clinic, data were collected with structured system usability scale (SUS) from PHNs (n=5) and families (n=15). WellWe had satisfactory usability with the total mean SUS-score of 65,2. The mean SUS-score was 71,5 among PHNs' and 58,8 among families' evaluations. Most favorable evaluations concerned learning the use of WellWe and managing to use it without technical support. The least favorable evaluations concerned users' opinion on their feeling of confidence when using the application and their willingness for frequent use. WellWe is considered moderately feasible with respect to usability, but there is need for further improvements. Especially the functionality needs refinement, different functions need integration and streamlining. Family provides an important context for child socialization, in which health-related behaviors are shaped (Campbell & Hesketh, 2006, Dalton & Kitzmann, 2008, OECD 2013). Children's wellbeing, balanced growth and development depend on the wellbeing of their families (Moilanen, Räsänen, Tamminen, Almqvist, Piha & Kumpulainen, 2010). Thus, interventions directed to the whole family should be favored. Interventions that emphasize family-centered approach facilitate detecting each family's individual needs and supporting their own resources in an empowering manner (Mikkelssen & Frederikksen 2011). Natural setting for the implementation of such interventions in Finland is the child health clinics, which reach a wide range of families with children under school age, and, thus, play a significant role in promoting the wellbeing of children and families (STM, 2009, Government Decree, 338/2011). The use of health technology and the development of family-centered child health services have been highlighted in political guidelines (Government Programme, 2015). Technological solutions and their ease of use as well as independence of time and place attracts people of today and are ever more present in everyday lives of families. New technology is evolving rapidly, providing the opportunity to develop and implement innovative and child-friendly interventions (Baranowski, Buday, Thompson & Baranowski, 2008, Quelly, Norris & DiPietro, 2015). Favorable findings from a recently published review on digital interventions to promote children's healthy nutrition and physical activity contributed to the development and implementation of future digital interventions (Quelly et al., 2015). Gamification, defined as “the use of game design elements in non-game contexts” (Deterding, Dixon, Khaled & Nacke, 2011), can increase the enjoyment and engagement (Hamari, Koivisto & Sarsa, 2014), as well as enhance learning and at some extent also health behaviors among adults (Looyestyn Kernot, Boshoff, Ryan, Edney & Maher, 2017). Among children, gamified approach showed to increase the attractiveness of health interventions and to be potential in promoting their health (Parisod et al., 2014). Our exploration on related work showed, that research on digital and gamified health promotive interventions has evolved during recent years.
According to a recent review, digital health promotive interventions are feasible and acceptable
methods among children and families
In addition to these existing gamified interventions, several others are currently under development,
such as a family-based gamified healthy eating app for parents with over 5-year-old children in UK
We developed a gamified WellWe-application to promote family wellbeing and to facilitate familycentered health counseling in child health clinics. We targeted families with toddlers and preschoolaged children.
The development of WellWe followed a typical iterative game development process
First, the idea and concept of WellWe were created in a multidisciplinary group. The idea and concept were based on previous evidence and theory, and supplemented with knowledge and experience from experts in various fields of health. After this, an alpha version of WellWe was developed with experts from the fields of information technology and graphics. Second, the alpha version of WellWe was tested among healthcare professionals (n=26), targeting system quality (ease of use), content quality (relevance and visualization). The data were collected through semi-structured group interviews after the participants tested the application. Based on the testing, the application was improved and further developed into beta version. Third, the beta version of WellWe was tested among healthcare personnel from child health clinics (a medical doctor and PHNs) (n=5), targeting system quality (usability and functionality), content quality (usefulness, understandability and visualization). The data were collected through open group discussion after a demonstration of WellWe. During the third step, also the comments about the visualization and understandability were acquired from the research group’s and acquaintance’s children (n=3). Based on these steps, the prototype (the release candidate version) of WellWe was finalized. The data reported in this article were collected during the testing session of the release candidate version of WellWe. (Fig. 1)
WellWe was created as a responsive single-page application (SPA) to support as wide a variety of target devices as possible. This also made the usage of the application simple for the users by removing the need for any installation procedures; only an up-to-date browser and an Internet connection were required. The client application was built with modern HTML5 technologies and frameworks (e.g. AngularJS). The communication with the Node.js –based back end application was done as a RESTful Web Service (Representational state transfer).
Gamified approach was applied to WellWe and implemented as a child-friendly theme (an amusement park theme) and reward-like system (a balloon with a rhyme) in monitoring. WellWe includes four parts: physical activity, nutrition, family resources and daily rhythm. In addition to the child-friendly theme as a gamified element, we included reward-like system to monitoring. In WellWe, monitoring means that families move cards (representing for example different nutritional items) relevant to their behavior into the Ferris wheel and after each card move, a balloon with a rhyme (fact about healthy behavior) appears to the screen. (Fig. 2). Families monitor their wellbeing by going through the four parts before entering to the health visit in child health clinic. A session lasts around 15 to 30 minutes depending on the individuals and whether the children are participating in the use of the application. PHNs receive information on the families’ wellbeing through WellWe. The information about family is also available in a statistical format (Fig. 3) and PHNs use this information in health counseling with families. The aim of WellWe intervention is to promote family wellbeing and facilitate family-centered health counseling in child health clinics. 4. Aim The purpose of this study was to evaluate the usability (SUS) of WellWe among PHNs and the families with 1.5- and 4-year-old children. The data presented in this paper is part of a larger feasibility study.
A post-test design (Fig. 4) and purposive sampling were used. After the study was approved by the Ethics Committee of the University of Turku, Finland (6/2015/26) and relevant permissions were retrieved from the Social and Health Board, five child health clinics from Southwest Finland were recruited to the study. Participation was voluntary and after signing informed consents, PHNs (n=5) were instructed to the study procedures. Families were recruited through the PHNs by sending all the eligible families (n=109) an information letter. Eligibility criteria for families were as follows: Ability to communicate in Finnish language; participation in the regular 1.5 or 4- year health visit in one of the five child health clinic during the data collection time (October to November 2015). Families were provided instructions for the access to WellWe in the information letter, and all the families who registered into WellWe (n=25) were sent an online link to the consent sheet and SU scale (Webropol). Because of low participation rate through online survey, paper-based questionnaires were sent to the registered families by post. Eventually 15 families participated into the study. PHNs were asked to fulfill the paper-based SU scale after few health counseling sessions.
Usability of WellWe was measured with System Usability Scale (SUS), which has proved to be a valid and reliable tool to evaluate the usability of products, systems and services. It has been used in many studies and can be applied to a wide range of technologies. It consists of ten statements (items) to which respondent answers according to own perception using 5-point Likert scale ranging from 1= “strongly disagree” to 5= “strongly agree”. SUS scores in total have a range of 0 to 100 (100 being the best possible score). (Brooke, 1996.)
Descriptive statistics were used to estimate the mean values for each item in SU scale using IBM SPSS
version 23. Usability scores were calculated from mean values using specific SUS-calculation formula
in Excel
PHNs’ and families’ combined evaluation of the usability of WellWe was satisfactory with the mean SUS-score of 65,2. The usability was found good among PHNs evaluations (mean SUS-score 71,5) and satisfactory among families’ evaluations (mean SUS-score 58,8). Specifically, both PHNs and families felt that most people could learn to use the application quickly, it could be used without technical support and its use does not require learning a lot of things beforehand. In contrast, both PHNs and families did not feel very confident using the application. Conflicting evaluations were found in some items, while PHNs giving favorable evaluations and families being of the opposite opinion. (Table 1)
USABILITY STATEMENTS/ITEMS*) 1. I think that I would like to use this application frequently 2. I found the application unnecessarily complex 3. I thought the application was easy to use 4. I think that I would need the support of a technical person to be able to use this application 5. I found the various functions in this application were well integrated 6. I thought there was too much inconsistency in this application 7. I would imagine that most people would learn to use this application very quickly
10. I needed to learn a lot of things before I could get going with this application *) Scoring: 5=strongly agree and 1=strongly disagree.
(N=5)
MEAN
(SD)
The purpose of the present study was to explore the feasibility from the perspective of usability of
WellWe among PHNs and the families with 1.5- and 4-year-old children. According to the results the
usability of the WellWe was satisfactory. The usability was found good among PHNs’ evaluations and
satisfactory among families’ evaluations. Some evaluations were found even conflicting and opposite,
with PHNs giving more favorable evaluations. This may have been due to the WellWe- training,
which PHNs received before the use of WellWe. In addition, some of the PHNs were also involved in
the development process, and thus, may have better understanding as regards to the use and purpose of
WellWe. This may have caused the results to be biased towards too positive. Ideally, SUS should be
used after the respondent has tested the system being evaluated, but before any conversations take
place
Favorable evaluations were given related to the learning to use WellWe without any technical support by both, the PHNs and families. This finding is partly in contradiction with the most unfavorable evaluations given by both. Participants did not feel confident using WellWe, nor were eager to use it frequently. The lack of confidence may refer to imprecise user instructions and those need to be reviewed and clarified. Other possible reasons may be inexperience in using digital devices and uncertainty of the reasons for using WellWe. Also, difficulty to implement new methods fluently into ones working routines from the PHNs point of view, may decrease the confidence, but also the eagerness of using WellWe frequently. Lack of time introducing new technologies and hectic everyday life with small children may decrease the willingness to use WellWe frequently.
Since families thought the functions in WellWe were not well integrated and its use was seen somewhat awkward, the system needs refinement and different functions need better integration and streamlining. This may also be the question of gamified elements implemented in WellWe. Since we aimed at attracting small children with a child-friendly theme, it may have caused some adults feeling of awkwardness, especially in situations where children were not participating to the use of WellWe. This may have influenced to the fluent use of WellWe, for example by slowing down the monitoring. Which, in turn, may have caused a feeling of less successful integration of different functions and an unpleasant experience for parents. If gamified elements have hindered the functionality of WellWe, this needs to be addressed in the future development of it.
Gamified elements need reflection also from the perspective of the children, since optimally WellWe is used together with the children. In this study half of the participated children were 1,5-year-old toddlers and their participation in these kinds of activities is limited. On the other hand, rest of the children were 4-year-old preschoolers and their participation is often dependent on the attractiveness of the activity. The question is, was WellWe enough fun and playful to foster their participation. Thus, gamified elements need reviewing. We also need to think the optimal and relevant target group for WellWe. It may be better to target preschoolers, instead of toddlers. On the other hand, tailoring the user interface according to the users, may be of interest. We could implement WellWe with three different user interface modes, for parents alone, for parents with toddlers and parents with preschoolers. This general localization idea increases the possibility to adapt WellWe also into different contexts and cultures in the future. All in all, unfavorable evaluations from the families may be due to the fact that the families, were not involved to the development process until the feasibility study.
This study had some limitations. The low participation rate and small sample size decrease the generalizability of the results, but this is usually the case in usability studies. Low participation rate may have skewed the results. Families participating to these kinds of studies are usually from higher sosio-economical groups. They may have thought that they do not need these kinds of interventions, and thus, their perceptions may have been less favorable. Families, who might have benefitted from such interventions, usually do not attend to the studies. Another limitation was that some of the PHNs were also involved in the development process of WellWe. This may have caused the results to be too favorable from PHNs point of view. Finally, we did not collect any demographic data from the participants other than the ages of the children, which prevented us from performing comparative analyses to make more in-depth interpretations of the results. But, since we aimed to test the usability of the released candidate version of WellWe, these results may be considered sufficient enough to give us directions for further development.
As a conclusion, our findings suggest that there is need to further develop WellWe to enhance its usability. The key improvement needs, emerging from the usability evaluations, concern refining the functionality, unifying the content and functions and streamlining the system, families use while monitoring their wellbeing. Also, instructions and gamified elements need reviewing. These improvements are important in the light of facilitating the implementation of WellWe into health services. These findings may also be used when developing future digital solutions into health services; especially the involvement of end-users should be taken into consideration already from the beginning of the development process.
The authors thank the public health nurses, parents and children who participated to the study. We also thank graphic designer Sara Suvanto for her contributions in the visual design of WellWe. The conducting of the study was financially supported by the Juho Vainio Foundation, Finland.
Quelly, S., Norris, A., & DiPietro, J. (2015). Impact of mobile apps to combat obesity in children and adolescents: A systematic literature review. Journal for Specialists in Pediatric Nursing 21, 5-17.
STM. (2009). Neuvolatoiminta, koulu- ja opiskeluterveydenhuolto sekä ehkäisevä suun terveydenhuolto. Asetuksen (380/2009) perustelut ja soveltamisohjeet. Sosiaali- ja terveysministeriön julkaisuja 2009:20.