In the contemporary era, the pivotal factor influencing the rise and persistence of environmental challenges is human behavior [1]. In fact, human action is making great changes within the ecosystem and the climate through numerous unsustainable behaviors [2].
Every year, about 25% of the plastic we produce, which equates to a staggering 80.5 million tons, ends up in our environment, making plastic a pervasive environmental pollutant [3,4]. Moreover, the use of noneco-friendly elements, such as most detergents, causes enormous problems, including soil structure deterioration, with detrimental consequences for plant well-being [5], and eutrophication, which results from the discharge of detergents into water bodies, leading to depletion of dissolved oxygen levels in the water, which can be harmful to aquatic organisms such as fish, seaweed, corals and other aquatic life.
Human action -e.g. urbanization, light pollution, chemical pollution, plastic pollution, deforestation, noise pollution, and air pollution [6,7,8]-is therefore producing a decrease in terms of biodiversity, contributing to the decline of some species, especially pollinators, such as wild bees [9]. Bees play a pivotal role in delivering a spectrum of ecosystem services that not only enhance human well-being but also sustain the fundamental life support systems of our planet [10]. These ecosystem services are inherently aligned with the pursuit of global sustainable development goals [11]. This underscores the necessity for a fundamental transformation in people's conduct [1]. It is imperative to replace practices such as consumerism, wasteful resource utilization, environmental degradation, contamination, selfish and unjust behaviors with sustainable habits, using the world's resources in ways that will allow human beings to continue to exist on Earth with an adequate quality of life [1]. With the continuous upgrading of educational technology, gameful systems, known for being highly motivating, have often been implemented to promote behavior change approaches or to support positive behaviors [12,13] in different domains, such as transportation and mobility [14,15], health, well-being and physical exercise [16,17], ecosustainability awareness and pro-environment behaviors [18,13], culture and tourism [19], and so forth. Overall, gameful systems present a reliable approach to enhancing and encouraging users to engage in the desired behavior, potentially fostering environmental protection. In fact, some investigations concentrating on the connection between video games and ecological knowledge have demonstrated noteworthy quantitative outcomes. In particular, there is a notable increase in players' motivation to grasp ecological concepts through video games [20].
We therefore developed a serious game called Untitled Bee Game 2 , in which players have the goal of discouraging non-player characters (NPCs) from engaging in specific non-eco-sustainable behaviors while learning notions related to pollution through dialogue with flowers. The long-term goal of the game is to foster positive behavioral change in terms of eco-sustainability, by providing knowledge in an engaging and enjoyable manner. To assess the game's usefulness and qualities, we formulated the following research questions:
RQ1. To what extent was the game perceived by the players as playful and meaningful?
RQ2. How would learning and perceived learning be affected by this game?
RQ3. What design directions should be taken in developing and improving such a game? In this paper, we present the design, development, and evaluation related to the use of Untitled Bee Game.
In Section 2 we present a brief overview of video games and serious games related to raising awareness of environmental issues; then, we present the design and development of Untitled Bee Game. In Section 3, we describe the design and development procedure to reach the final version of the game. In Sections 4 and 5, we present the questionnaires and the methods for both usability and experimental analyses, and then the results. In Section 6, we discuss the findings after the analysis. Finally, in Section 7, we conclude the paper and present the future work needed to improve the game, as well as some recommendations for future research.
Before starting the design phase, we analyzed commercial video games, and serious games used to raise players' awareness of eco-sustainability and biodiversity issues, or to teach about eco-sustainable behaviors in the real world.
In recent years, an increasing number of video games have been used to increase users' environmental awareness. These include Terra Nil3 , Donut County4 [21], Factorio5 , Arma 36 [22], and Animal Crossing: New Horizons7 [20]. Despite being commercial video games, many of these games present elements of environmental or biodiversity awareness simply because of the setting, storytelling, mechanics, or dynamics of the game.
In some commercial video games, the ecological component is pivotal and the educational purpose is part of the design itself. In this case, however, the goal of teaching something to players may also result in a less playful system [23,24] and a weaker flow experience [25]. In recent years, numerous video games have been developed with this purpose in mind -such as Eco8 [24], Plasticity9 [26,27], and Bee Simulator10 . These games, which thus have some characteristics of serious games while remaining commercial, present a more complicated design procedure. Indeed, designing video games with a well-defined purpose seems to be rather challenging. The priority given in the development to the eco-sustainable awareness component could lead these games to lack the playful impact and the flow experience [25] that users often seek in commercial video games [23,24].
The use of serious games has proven to be a useful element in promoting environmentally sustainable behaviors [28], and climate change awareness [29]. Madani et al. [30] compiled a survey of serious games (both digital and physical) used in the context of tackling environmental challenges and awareness raising on ecosustainability. The authors report that the use of serious games can increase motivation and engagement, with beneficial effects on desired outcomes (i.e., environmental awareness); however, the effectiveness of games is not universal for educational purposes, it depends on the design and components of each game and, for this reason, it is difficult to assess the accuracy of the findings. An interesting example of a serious game is Energy Chickens [31]. In the game, virtual chickens are graphically manipulated to reflect the power consumption of the device running the game. The game was found to contribute to being more energy conscious and the participants' energy consumption decreased by 13%. Several serious games exist that are focused on city management [32], such as EnerCities [33], where players can manage different energy strategies for their city by choosing between fossil and renewable sources. In the field of VR games, [34] propose an immersive experience (Climate Connected: Outbreak) to engage players in the topic of climate change, which was found to foster learning and positive attitudes related to climate change.
In the literature taken into consideration, to the best of our knowledge, there do not seem to be relevant examples in terms of complex worlds and gameplay inspired by mainstream 3D games that also employ large amounts of text. Some examples exist that try to achieve this result in a 3D environment with heavy use of text, in the field of cyberbullying prevention [35], where offensive text (to be erased or changed by players) is included in the form of messages from the characters or graffiti. It was important that the teaching came from virtual characters to merge it more naturally with the game environment. We looked at existing commercial games to find mechanics that could fit well with teaching eco-sustainability facts to foster behavior change. In the next sections, we discuss the inspiration for the game and the design choices that were made during the development of Untitled Bee Game.
Serious games and gamification present a more complicated design procedure than video games, since they are designed with a clear serious purpose. For this reason, not all gameful systems and serious games are effective [36,37]. For a purpose-consistent design and effectiveness of these tools, numerous design frameworks and procedures have been developed and adopted [38,39,40]. Design approaches can follow different perspectives: user-centered, technologycentered, game-centered, and context-centered [38,40,41]. Some of these methodologies present collections of game elements that help the designer keep track of game choices [42], while others are more complex methodologies that provide more of a guideline for the design [29,43], and also for the next steps [41].
Many frameworks are based on developments of preexisting models: for instance, the works of [44] and [45] trace their roots to the MDA model [46]. This model strives to amalgamate game design, development, critique, and technical game research by presenting a structural approach to understanding games. The pioneering 6D model by [47] has inspired numerous authors to create additional design frameworks (see [48,49,50,51]) or adapt it in different contexts (see [52,53]). However, although possible design options are numerous, few design frameworks or procedures have been used or studied consistently. Specifically, few frameworks have been used to design tools promoting behavior change or climate change awareness. Among the few frameworks for climate change awareness, in [34], the authors specifically provide guidelines for serious games about climate change, including, among others, adapting game design to the characteristics of the users, involving users actively, and presenting content in new ways to aid learning.
Untitled Bee Game (henceforth UBG) borrows its title and mechanics from Untitled Goose Game11 , a 3D puzzle game for computers and consoles. Untitled Goose Game is part of a rather recent video game genre that could be defined as "animal mayhem games" [54]. These games let players control animals that wreak havoc and punish human beings, generally relying on humor and disruptive game mechanics. Other such games are, for example, Goat Simulator 12 and DEEEER Simulator 13 . This strand of games seems appropriate to focus on the theme of sustainability and environmental preservation: they shift the point of view from humans to animals and let players correct or condemn human behavior as an external observer. This choice was also dictated by the scarcity of perspective change examples in the literature concerning serious games for sustainability, in the face of a relative abundance of such an approach in entertainment games. Moreover, the proposed perspective shift can be put in relation with recent literature that deals with the abandonment of anthropocentrism, in the context of a posthuman turn that views ecology and climate change from a non-human/other-than-human standpoint [55,56,57]. We also suspect that carrying out hostile in-game actions as a human instead of an animal could potentially have negative educational consequences. The game might be thought to suggest that it is advisable to be hostile to peers who do not follow virtuous behavior.
The core mechanic of Untitled Goose Game consists of playing tricks on, and stealing objects from, human non-playing characters (NPCs) to disturb their daily activities. Albeit relying on such a simple mechanic, the game managed to sell one million copies in three months [58]. In addition, its simple, toon-shaded low-poly 3D graphics make for the ideal style for rapid prototyping and development.
Although there is a clear similarity with Bee Simulator (see Section 2.1), the mechanics did not play a particular role in the design of UBG, although it may be used as a model in the future to add new features.
As a "game with a purpose", we carefully designed the game from the contextual information and purpose. We relied on the GamiDoc design framework [41] to write a game design document 14 that would allow the team to communicate and at the same time, keep track of the development in relation to the purpose. The game had no specific target and was inspired by a simple, cartoon graphic style (used in Untitled Goose Game itself) that could be found pleasurable across different genders and ages. The target of the experiments, however, was adults.
The idea behind UBG is to explore the potential of immersing players in a 3D virtual world inspired by recent commercial games, with NPCs reacting to the avatar's actions, to foster novel gameful interactions for learning and positive behavior change.
UBG lets players control a bee in a small town (see Figure 1). We selected a bee because of the importance of bees in the environment [10]. Indeed, facts about bees are part of the learning content of the game, which players are quizzed on. This choice also brought about advantages under the technical aspect of the implementation, such as having to deal with simpler animations. Imitating commercial games may imply adopting control schemes that are a bit difficult to master. Therefore, at the beginning of the game, players are asked to select the control style they prefer: overhead camera or free camera. In both schemes, players control the bee in the third person, but in the latter, they can also rotate the camera (and the bee) freely in all directions. The overall objective is to dissuade the NPCs from pursuing certain behaviors. If correct actions are taken by the player, the garden (a small area where flowers can appear) and the river are influenced: the garden produces flowers and more fish come to the river.
The scenery of Untitled Bee Game. The screenshot shows the small town that is explorable in the game.
Two NPCs were implemented. The first one wastes plastic bottles, while the second one uses polluting substances to wash her car. An icon above the head of the NPCs shows the player what behavior is being pursued. If it is a damaging behavior, players must sting the NPC or drop rubbish on their head. Stinging can be performed at any time, even if the behavior shown by the NPC is already correct. Therefore, we differentiate between two stinging actions the player can perform: correct and incorrect stinging. The game plays different sounds accordingly. If correct stinging is performed, new flowers appear in the garden and new fish appear in the river. Once a flower has appeared, players can interact with it and read facts about the three topics (see Figure 2), one paragraph at a time, shown in a classic video game message box. Once a knowledge paragraph has been unlocked, it becomes possible to read it again in the pause menu.
As a side mechanic, players may also pick up and recycle the rubbish they find. This grants them one additional fish. In addition, rubbish can be thrown at NPCs instead of stinging them to obtain the same effect. If players do not do anything for a certain amount of time, fish start to disappear. It is worth noting that there is no game over and the players' actions are only driven by the possibility to make
UBG was developed using Unity3D 15 , an industrystandard environment in the field of video game development. Some of the 2D resources and images were found on Adobe Stock 16 , while others were created adhoc. The 3D models were either found in the Unity Asset Store 17 and Quaternius.com 18 or created using Blender 19 , a popular 3D modeling tool. The music was either created ad hoc or found online for free 20 .
All the text presented in the game and the quiz questions were created following an extensive literature search on the topic. Following an initial analysis, the final data were selected on the basis of their association with player and NPC behaviors within the game.
Prior to deploying the game and administering it to the participants, we created a REST API based on a Mongo 21 database that communicated with the game during the sessions. For this reason, players were asked to remain connected while playing. We collected the following information in JSON format: i) a game ID; ii) the number of times each NPC was stung; iii) the number of correct actions; iv) the number of incorrect actions; v) the quiz scores; and vi) the paragraph reading times.
Players were mainly recruited in Italian university facilities (University of Trento, University of Verona, and University of Turin) and there were no exclusion criteria except being underage. Recruitment requests were circulated by two lecturers among bachelor students primarily. They could choose between a browser-based, a Mac, and a Windows version. They were both Italian and English speaking and it was possible for them to change the game language at the beginning of the game. The procedure consisted of playing for approximately 20 minutes and responding to an online questionnaire. The game assigned a random alphanumeric ID to each player to be entered in the final questionnaire, which granted complete anonymity. The procedure was employed in two phases: the usability test and the main experiment. The main differences were (i) the version of the prototype, and (ii) the content of the final questionnaire, administered after the post-session quiz, as described below.
Game session. Players could play a desktop or a web version of the game. After being shown their ID, they were taken to the initial quiz (pre-test) where they could express a choice among 4 to several questions related to eco-sustainability to establish a baseline of knowledge about sustainable behavior. This quiz established a baseline for each participant. After the initial quiz, players had to complete a brief tutorial about the game commands and mechanics. Then they could start exploring the environment and stinging the humans showing undesirable behavior. In so doing they earned a fish and a flower, and then had to pollinate a flower to reveal a new fact. After revealing a certain number of facts, players were taken to the second quiz (post-test) with an additional 4 questions on the same topics as the initial one.
The content and order of the questions in the two quizzes were the same for all participants, both in the usability test and the experimental phase. Accordingly, the eco-sustainability facts were shown to all players in the same order. The game informed players about whether their answer was correct or incorrect, and in the latter case it also displayed the correct one. However, the pre-test and post-test quizzes presented radically different questions to prevent users from answering correctly based on the previous answers.
Usability test design. To collect usability data, we used the System Usability Scale (SUS), which had to be filled out at the end of the game. The choice of using this tool stems from its reliability and its widespread use for usability studies [59]. Moreover, we added the item "Do you have any suggestions, feedback, comments that can help us in improving the game?" to collect more detailed information about possible improvements in the game. We recruited N=9 participants (female = 4; male = 5) and had them play the first version of the prototype, where they had to unlock 16 flowers and answer 16 quiz questions in total (8 pre-session and 8 post-session). Each participant tried the game through the web or desktop version and answered the online questionnaire on Google Form 22 . Data in the literature [60] suggest that for usability studies a sample size of 10 Β± 2 is recommended to detect at least 80% of usability problems. During the usability data collection, we still collected data on playfulness, meaningfulness, and learning perception to check whether usability could in any way negatively affect these aspects. Hence, we selected some items from reliable questionnaires: Q1-"This game gives me an overall playful experience" from the playfulness dimension of the GAMEFULQUEST [61], Q2-"I think this was an important activity" and Q3-"I believe this activity has been beneficial to me" from the value/usefulness dimension of IMI [62], and Q4-"The game contributed to teach me some of the most relevant environmental issues" from the perceived learning dimension of MEEGA360 [63]. We then interviewed the participants in a think-aloud session.
Experimental design. We recruited N=28 participants (age: π = 26.62; ππ· = 8.80; 13 identified as female, 15 22 https://www.google.it/intl/it/forms/about/ identified as male), among which N=18 completed the game, and N=24 completed the final questionnaire, of which N=14 completed both the game and the final questionnaire. In this phase, players were administered a refined version of the prototype, where many things were changed according to the previous round of feedback. In particular, the number of paragraphs to read and the number of quiz questions were both lowered to 12 in total. After playing the game, participants had to fill out a questionnaire where they were asked to provide: (i) demographic information and gaming habits; (ii) an informal assessment of the degree of sustainability of their behavior; (iii) engagement and value provided by the game by using the items from the GAMEFULQUEST, the IMI, and the MEEGA360 described above, and two more items -Q5-"Thanks to the game, I am likely to inquire about how to adopt other sustainable behaviors in the future" and Q6-"Thanks to the things learned in the game, I am likely to adopt more sustainable behaviors in the future"-created ad-hoc to assess how much the game may have influenced users to become informed about environmental issues and adopt environmentally sustainable behaviors in the future. All the materials were available both in English and Italian.
We report here both the results from the usability test (expressed mainly in terms of SUS scores and feedback received) and the main experiment.
The game's SUS overall score (62.2) was below the set threshold for a good level of usability (68), indicating marginal usability; hence, additional improvements were needed [64]. For an exhaustive overview of SUS results, see Table 1.
Questionnaire results. The results of the additional items have identified that during this first application, not all users found the game completely entertaining (Q1 π = 3.33, ππ· = 1.11, πππ = 4). However, they recognized its educational power and its importance (Q2 π = 3.44, ππ· = 1.01, πππ = 4; Q3 π = 3.66, ππ· = 1.11, πππ = 4; Q4 π = 3.88, ππ· = 0.78, πdπ = 4).
Feedback comments. At least one participant (U1) found that the game showed too much text. Therefore, we lowered the number of flowers to 12. This meant less text to read (12 facts instead of 16) and less playtime. Three participants (U3, U4, U8) mentioned that the objective was not completely clear or that the icons were difficult to interpret. A participant wrote that they would have liked to know how important it was to read the flower facts (U1). This would have allowed them to answer more correctly to the final quiz. As a solution, we added a summary of the objectives in the pause menu. Another participant (U9) mentioned the difficulty of understanding interactions with fish within the game. We therefore decided to specify this more clearly. The controls were a bit difficult for at least two participants (U3, U7). This led us to offer a choice to the players at the beginning, asking which control scheme they preferred. However, one participant (U1) liked how they worked. Finally, one participant (U7) stated that the background music was a bit annoying. This stresses the importance of giving players an option menu to personalize the experience. Think-aloud session. After the usability testing, we interviewed the participants informally in a think-aloud session to gather additional feedback. There were several points that were raised in this session that had not emerged as part of the written comments. We summarize the main topics that emerged during the think-aloud session in Table 2 as coded comments. The table includes both suggestions that were later implemented into the game and those that were not, followed by an explanation.
The suggestion "Add flowers to encourage exploration" was not followed because the game only allows to unlock flowers in a dedicated area to optimize the traveling time and the experiment duration, but will be considered in the future; "Add a deeper narrative" was not followed because, as mentioned, the game was optimized to be completed in a reasonable amount of time, so we have not yet included a story behind the characters or the avatar, but it is a direction for future work.
Added Add a progress bar for the quizzes Yes Make tutorial more visual Yes Include recycling mechanics Make it easier to unlock more fish Add flowers to encourage exploration Add a deeper narrative Yes Yes No No Table 2: Suggestions from the think-aloud session, with an indication as to whether they were followed.
What follows is a summary of the results obtained with the main experiment, conducted after the usability test. We proceed to describe and analyze the data obtained 23 https://posit.co/ from the questionnaire, the change in quiz performance in pre-test and post-test, and the feedback received at the end.
All the analyses were conducted using RStudio 23 (version 2023.03.0). Among the participants, N = 24 answered the questionnaire. The results (Figure 3) show an overall positive evaluation of the tool. Answers were given on a 5-point Likert-type scale ranging from strongly disagree to strongly agree (See Table 3). Most of the players found the game playful (63%); however, a few subjects thought the game was not playful, indicating the need to bring changes to the game in the future (12%). Interestingly, almost all the players found the game meaningful (96% and 83% to Q2 and Q3 respectively), probably because of the importance of the themes it dealt with, and they felt they learned something from the experience (Q4 positive results = 88%). Lastly, 58% of the players reported the will to inquire (Q5) and then adopt other sustainable behaviors in the future thanks to the game (Q6). We then ran a linear regression to analyze whether the expressed playful level could be explained by users' prior experience with video games. The results show no correlation between the two variables (π΄πππ’π π‘πππ Β² = β0.07, πΉ1,12 = 0.14, π > 0.05), suggesting that the game is equally perceived despite users' experience with video games. To assess whether the meaningfulness and playfulness of the game, and the perceived learning were conveyed by the amount of information read, we ran some linear regressions between the reading time and the playfulness value (Q1), the interaction of the items related to meaningfulness (Q2 and Q3), and the perceived learning (Q4). The results show that no correlation occurred between the reading time and the expressed game meaningfulness (π΄πππ’π π‘πππ Β² = 0.58, πΉ1,12 = 2.54, π > 0.05), and playfulness (π΄πππ’π π‘πππ Β² = β1.82, πΉ1,12 = 0.301, π > 0.05). The analysis showed a significant relationship between reading time and perceived learning (π΄πππ’π π‘πππ Β² = 1, πΉ1,12 = 5.79, π < 0.001).
Among the participants, N = 18 completed an entire game session, and did the post-test quizzes. To answer RQ2, we investigated the difference in scores between the pretest and post-test quizzes. The post-test quiz obtained generally higher scores (π = 4.55; ππ· = 1.46; πππ = 5) than pre-test one (π = 3; ππ· = 1.23; πππ = 2.5). At first glance, it was possible to see that only 5 players did not get an improvement, of which 4 maintained the same score, and only one had a lower score. We then analyzed whether these differences in users' answers before and after the use of the game were significant. A Shapiro-Wilk test was conducted, indicating that the two distributions of both variables differed from the normal distribution (Pre: π = 0.79, π = 0.001; Post: π = 0.83, π = 0.005). Considering this result, we opted for a non-parametric test. We then performed a one-tail Wilcoxon signed-rank test [65] (π = 11, π < 0.005) and found a statistically meaningful disparity in the results at the in-game ecosustainability questionnaire before and after playing the game. Effect size calculation (π = 0.49), which for the Wilcoxon signed-rank test is based on the formula π = π βπ β [66], suggested a moderate magnitude of the effect [67]. The post-game results are not explained by the subjects' prior experience with video games (π΄πππ’π π‘πππ Β² = 0.02, πΉ1,12 = 1.32, π > 0.05) or the flower reading time (π΄πππ’π π‘πππ Β² = 0.76, πΉ1,16 = 4.49, π > 0.05).
A boxplot showing the pre-test and post-test scores on the eco-sustainability quizzes, taken at the beginning and at the end of the game session. There were 6 questions in each quiz.
We were able to extract a substantial number of comments from the feedback collected with the final questionnaire, which helped us answer RQ3. We conducted a thematic analysis by coding all feedback manually under macro-themes. We summarize the comments here, grouped by theme. In Table 4, we provide a detailed breakdown of the main comments grouped by participants.
Participant IDs There should be additional areas E1, E2, E4 T1. Repetitiveness: E1, E2, and E4 mentioned that they would have liked to have different areas to explore and to find flowers in. This mirrored what emerged during the usability testing. E7 also mentioned that the positioning of the garden and the humans may be exploited to take shortcuts as the mechanics do not encourage exploration. T2. Text style and length: many users expressed that they would have liked to read less or simplify the text. E6, E11, E14, and E16 found that the text was too verbose. E6, E10, and E18 stated that they would have liked less formal language. T3. Voice over: this theme is largely connected to Theme 2. In particular, E3, E13, and E16 suggested employing text-to-speech techniques or voiceovers to decrease the amount of text to be read, especially for people with dyslexia. T4. Unclear goals: E8 stated that they found the stinging context button to be somewhat difficult to activate or interpret. E9, E14, and E19 asked for more succinct indications as to what to do with the flower and the fish. E19 suggested to include more visual cues. T5. Font readability: flowers taught players their facts with a pixelated font, which was chosen to be reminiscent of old-school video games. In this context, however, it produced mixed results. E18, E20, and E24 had difficulties reading the font and this made them want to skip the messages (see Figure 2). T6. The fun in playing as a bee: overall, the game was considered a nice experience by E6 and E19. E6 and E11 stated that the game was fun; E16 stated that they liked the concept of being a bee; E9 and E18 explicitly stated that being mean to humans made them happy or satisfied. T7. Mastery of controls: although E6 explicitly stated that controls were easy to master, E10, E16, E17, and E19 expressed their difficulty in mastering them. Finally, only 4 players did not report any comments.
We assessed the enjoyability and usefulness of UBG in two phases: a usability testing phase and the main experiment. All in all, the experience reported by users was positive, although there were some problems in the usability phase. Comments were particularly useful in the usability testing phase since usability appeared to be suboptimal according to the SUS. Regarding the main experiment, we observed that there was indeed an increase in quiz performance, thanks to the knowledge presented in the game, which is an encouraging finding that contributes to answer RQ2. Interestingly, quiz performance is not explained by self-reported experience with video games according to our analysis. This suggests that potential problems are not likely to have influenced learning. The level of reported playfulness does not seem to be influenced by experience with video games. In this respect, in addressing RQ1, we can say that in the end the prototype was quite enjoyable for every participant and the difficulties with the controls did not invalidate the experience.
Several lessons can be learned by looking at the feedback collected to answer RQ3:
1. Consider implementing a voiceover: The proposal of removing some text and/or adding a voiceover was shared by 29% (7) of participants. In general, reading was fatiguing for players, and adding recorded speech or using text-to-speech techniques seems like the sweet spot between teaching notions and not overwhelming players, although we suspect that learning could be influenced by the two different methods. Speaking of text more in general, our findings mirror those of [34], where the authors suggest avoiding too much text and using instead more visual and impactful feedback.
Four players thought that either the controls were difficult to master or that they could be improved, while one player found them easy to master. This underlines the importance of considering not only all the possible peripheral configurations but also all the possible levels of experience with video games. Choosing the right game genre is key: some games, like first-person shooters, for example, require very specific skills such as aiming with the mouse.
3. Too much freedom may lead to unclear goals: We obtained 24 complete questionnaires in the face of 18 completed game sessions. This fact is likely to be related in some way to T4 (Unclear goals), but it could also be influenced by the fact that the game, albeit giving instructions, leaves players rather free to explore. We therefore recommend future endeavors in serious games to consider the aspect of directing players to the goals in the most straightforward and clear way possible. Finally, 5 players explicitly stated that either the game was a good experience or that controlling a bee and stinging the humans was fun. This contributes to addressing RQ1 and suggests that being mean to humans can be perceived as satisfactory.
There are several limitations to the present study. The serious game employed is still in an early version and lacks a number of features that could make the experience more enjoyable (e.g., a more complex narrative and high-level goals that go beyond the stinging task alone). Since one of the aims of the study was to assess the educational potential of the game, the instructions and the mechanics guided players rather strictly towards reading and responding to quizzes, while it could be turned into a more relaxed activity in future versions. The second limitation is the limited number of participants involved in the study. A greater sample could be recruited or a follow-up knowledge retention test could be introduced to further support the learning trend observed. It could be argued that telling players about the final quiz may have encouraged them to read more carefully, thus influencing the final results. Future directions may include a between-subjects experiment, where the treatment group does not know about the final quiz.
We developed a 3D serious game, mainly inspired by Untitled Goose Game, about eco-sustainability and evaluated it with a usability test and an experiment. Although the usability test turned out suboptimal, the feedback collected allowed us to deliver an overall enjoyable experience in the following phase. Our quiz results coming from the main experiment indicate that it is indeed possible that players learn by means of such a game. Moreover, most users felt encouraged to learn more about eco-sustainability in the future. Some participants even stated that controlling a bee and stinging the humans was fun. However, the sample size involved in the study was limited. Future work should investigate this tendency in larger samples and with a more advanced prototype that includes, among others, a narrative and more exploration. Future work may also take advantage of more comprehensive questionnaires.
Recommendations: The insights gained from UBG allow us to state some recommendations for future games on sustainability awareness. First, as noted in the literature [30], the design procedure matters: the effectiveness of a game is not universal. Design procedures should follow specific frameworks or procedures, and design choices should fit the game's purpose and context (see [41] and [68]). Second, as suggested in the literature [69], fun can convey the effectiveness of serious games and gameful systems. Thus, it is necessary to analyze the context in which the game will be used and the target users to make the game fun. Third, interaction is essential in the learning process. As our data suggests, learning through interaction in play is more fun than text-based learning, especially avoiding repetition of patterns. It is therefore necessary to identify a method to make the way information is shown to players entertaining. Data in the literature suggest limiting repetition and giving space to novelty [42]. Serious games are developed with specific purposes in mind; hence, it is essential to include certain paths to follow, to prevent users from getting lost, clearly defining the objectives of the game, and at the same time, leaving some freedom for users. Lastly, as suggested by the data in the literature [70] and player comments, consider using narrative and storytelling to accompany the player along the way.
We acknowledge the support of the PNRR project FAIR -Future AI Research (PE00000013), under the NRRP MUR program funded by the NextGenerationEU.