Graph literacy involves interpreting graphical information correctly, requiring a broad range of knowledge to generate inferences about different types of graphs (e.g., [1,2]). Graph reading is ability to fluently extract and use information from graphs [3] Individuals who are proficient in reading and interpreting graphs tend to process more complex information and accurate conclusions while viewing line or bar graphs than individuals with lower graph literacy [4]. However, after learners become proficient in graph literacy, there are additional challenges since graphs can be misleading and require critical graph reading skills.
A misleading graph is based on valid data, but the visual appearance of the graph is not aligned with its numerical values, distorting the message of the graph. Several manipulation techniques can be used to create misleading graphs. For example, scales of the axes can be inverted, or the baseline of y-axis can be set larger than zero, creating conflicts between spatial features (e.g., height of the bars) and conventional features of the graph (e.g., axes labels and scales) [5,6]. Consequently, readers may misinterpret graphs if they only rely on visual features of a graph. Misleading graphs immerged even in media and governmental communications during the covid pandemic [7,8]. Moreover, producing misleading graphs might not always be intentional rather than individuals' gap in knowledge to create well-formed graphs [9]. Hence, the responsibility of identifying and interpreting misleading graphs is passed on to individuals, and the level of critical graph reading skills becomes a pivotal determinant.
Prior studies suggest that learners who lack critical reading skills often struggle to identify misinformation, but pre-emptive (prebunking) interventions can increase learners' ability to identify misinformation [10]. Sterling pre-emptive interventions offer a promising approach to deal with misinformation, which is based on inoculation theory [11]. Inoculation in a misinformation context refers to building resistance against false information by preemptively exposing learners to weakened forms of misinformation, which originates from concepts of vaccination, i.e., controlling the exposure of a virus and slowly building up resistance [12,13]. Inoculation theory is based on two main mechanisms [11,13]. First, the aim of forewarning is to motivate resistance (a desire to defend oneself from manipulation attacks). Second, the aim of a pre-emptive refutation (preexposure to a weakened example of the manipulation attack) is to provide people with specific knowledge that they can use to refute future manipulation attacks. Thus, the pre-emptive interventions apply vaccination principles to knowledge, where learners are 'inoculated' with a weakened form of persuasion (misinformation) to build immunity against similar attempts faced in the future [10,15]. Subsequently, learners might demonstrate increased competence to identify misinformation.
Game-based learning environments (GBLEs) offers a medium to integrate inoculation in a more 'active' way compared to more traditional and passive learning materials such as text-based misinformation campaigns [16,17]. Thus, GBLEs may offer opportunities to increase learners' critical graph reading skills [17].
GBLEs offer advantages over traditional educational approaches by rendering more interesting and engaging instructional tasks, enhancing knowledge acquisition, skill development, and learning outcomes [18,19,20]. While utilizing inoculation theory in GBLEs has shown promising results for improving critical reading skills [21], there is a lack of research in the graph literacy domain. Research findings are inconclusive regarding the role of GBLEs in promoting positive emotions (e.g., curiosity) that stimulate learners' desire for knowledge that benefit their learning outcomes [22]. Thus, designing GBLEs that support, and nurture learners' epistemic curiosity could serve as a powerful motivator for developing critical graph reading skills.
A recent systematic literature review [21] indicated that the use of GBLEs in critical reading education had emerged after 2021. The surfacing research might spring from the growing importance of critical reading skills in the today's information maze [23]. Moreover, the rising threat of misinformation might further lead to increase of published papers regarding GBLEs' usage in developing and supporting critical reading skills. Kiili and colleagues [21] found that most GBLEs designed to improve critical reading skills were based on inoculation theory and took a preemptive intervention approach. Simple choice-based simulation games were one of the most popular types of GBLEs and provided a storyline where the learner was either a misinformation producer or a factchecker.
Bad News, a simulation-based GBLE [16], is one example of a game designed to support critical reading skills. It requires learners to produce and spread fake news on social media to gain popularity and credibility as a news publisher. The game applies the process of active inoculation to make learners more skeptical towards the selected misinformation strategies. Bad News introduces earnable six badges to a to teach learners about common misinformation strategies: (1) impersonating another person, (2) creating provocative emotional content, (3) amplifying existing group polarization, (4) generating their own conspiracies, (5) discrediting opponents, (6) practicing trolling. The results demonstrated that Bad News significantly reduced the perceived reliability of tweets that embodied common misinformation strategies and made learners more attuned towards them.
Another GBLE, Harmony Square, let's learners to witness how misinformation brings chaos to Harmony Square [24]. Narrative takes more political aspect and tries to demonstrate the effects of misinformation on residential area. Gameplay includes producing misinformation, and gather as much "likes", and following as possible. The learners' complete levels themed by different misinformation techniques (trolling emotions, amplification, and escalation). The game also uses active inoculation to build resistance against misinformation for learners by letting them produce the misinformation. This might enhance memory retention and extend the duration of the protective effect against misinformation [16]. The game reduced the perceived reliability of misinformation, increased confidence in learners' ability to spot misinformation, and made learners less likely to share misinformation in social media [24].
In sum, the review [21] showed that GBLEs seem to demonstrate positive results for increasing critical reading skills, even though the field is still in maturing stage. Even though, critical reading games are present in the game-based learning literature, they are still developed to focus on specific areas (e.g., news, social media posts) rather than focusing on the misinformation in wider areas like voting or society problems. Moreover, the review revealed that critical graph reading was not addressed in any of the reviewed papers.
For learner to effectively read and interpret graphs, cognitive load plays a major part [25]. By minimizing cognitive load and keeping visual complexity on reasonable level, allows learners to retrieve and process the information effectively.
The general cognitive ability emerges as the primary predictor of graph reading performance [3]. The general cognitive ability, defined as the capacity to tackle novel problems, thus becomes crucial in unfamiliar graph reading tasks. In addition, visual processing and analogical reasoning are have been recognized as influential in graph comprehension [3].
Leading models of graph comprehension have demonstrated three distinct processes that learners utilize to draw inferences from graphical representations (e.g., line or bar graphs; [6]). The initial process is encoding the visual patterns to recognize the primary elements in the graph (e.g., lines with different slopes). The process also includes making visual judgments of the elements (e.g., determining locations along a scale, assessing the slope, or measuring the length).
The second process involves translating identified visual features into conceptual relations [6]. For instance, differences in the size of spatial elements (e.g. varying bar heights) are utilized to demonstrate the change and differences in quantity of the variables. Spatial elements refer to components found within the pattern, such as different height bars, or ascending or descending trends.
The last process involves recognizing and deducing information from basic (conventional) elements in graphs (e.g. labels of the axes, legends, numerical values on the scales) and integrating this information with the information extracted during the previous two processes [6]. For example, in bar and line graphs, it is required to recognize the variables displayed on the x-and y-axes and the values these variables acquire.
Correctly interpretating a graph relies on the spatial and conventional features aligning with learners' spatial-to-conceptual mappings [6]. Spatial and conventional feature conflicts may occur when a graph's visual and contextual elements do not match. For example, the heights of bars may be incoherent because of the scaling of y-axis values. In the case of conflicts, learners, particularly learners with lower graph literacy, might be led to misinterpret the graphs visual representation. However, number of empire studies focusing on critical graph reading is very limited, especially among adolescents, and needs additional studies.
Epistemic curiosity is an epistemic emotion. Epistemic emotions are defined as affective states that motivate critical reflection and inquiry [26]. They are emotions that relate to knowledge and the generation of knowledge. Epistemic emotions arise from the cognitive qualities related with thinking, understanding, and learning. Epistemic curiosity, defined as an innate thirst for knowledge, may inspire learners to generate innovative ideas, bridge gaps in their understanding, and persevere when confronted with complex challenges [27]. Curiosity emerges from an information gap or inconsistency between what the learner knows and what they want to know [28]. Curiosity steers a learner to seek, obtain and utilize new information. Nakamura and colleagues [29] found that positive appraisals, cognitive puzzles, novelty, and task or topic satisfaction may trigger epistemic curiosity. Moreover, higher epistemic curiosity tends to be simulated more likely by complex situations, such as identifying misleading information on graphs, possibly motivating learners' engagement with the learning material.
This study is a part of an on-going project in which we are developing a GBLE for teaching critical graph reading. In this paper we report the evaluation results of the first prototype of the MediaWatch game. This study has two objectives. First, to examine the effectiveness of MediaWatch, a GBLE grounded in inoculation theory, in improving critical graph reading skills. Second, to assess whether learners' selfreported epistemic curiosity increased after they learned critical graph reading with MediaWatch. To achieve these objectives, we conducted an intervention study by randomly assigning learners to one of two conditions: a game condition and a control condition. Our research questions and hypotheses are as follows: RQ1: Are there differences in the degree of change in graph reading task scores from pre-to post-test between the game and control conditions?
Hypothesis 1a: Learners' misleading graph interpretation task score will increase significantly more from pre-to post-test in the game condition than in the control condition.
Hypothesis 1b: Learners' graph comparison task score will increase significantly more from pre-to post-test in the game condition than in the control condition.
RQ2: Are there differences in epistemic curiosity from pre-to post-test after learners finished learning with MediaWatch?
Hypothesis 2: Learners who play MediaWatch will demonstrate a significant increase in epistemic curiosity after game-based learning.
One hundred and one 15-20-year-old (n = 101; Mage = 16.80, SDage = .71; 48% females) high-school students completed this study and were recruited from a public school in Finland. The participants were randomly assigned to one of two conditions at the beginning of the study: 1) the game condition, where learners played a game called MediaWatch, and 2) a control condition, where they engaged with their usual classroom lecture that did not include any graph reading content. The control condition without any treatment was used to control the possible learning effects of the employed graph reading test. One participant from the game condition was excluded from analyses due to not playing MediaWatch.
MediaWatch is a web-based GBLE that aims to support critical graph reading skills. Each player works as a fact-checker on a fictional island called Sahramoa (see Figure 1: left). The island is inhabited by four different villages, which each play a role in contributing to different environmental crises (see Figure 2: right). MediaWatch is a fact-checking institute on the island that assigns tasks to players. The institute was established to ensure that misleading information is not published in the local news media. MediaWatch receives regular reports from each village and checks the content before releasing them as public news.
The player's job is to fact-check the reports by interpreting multiple types of graphs (e.g., line and bar graphs) and selecting a title that best aligns with the graph (see Figure 2: left). The tasks that a player completes include both manipulated and well-crafted graphs. Three manipulation techniques are included: reversed x-axis, y-axis not starting from zero, and yaxis range being too wide. In the case of manipulated graphs, players are presented with four title options: one that is correct, one aligned with the manipulation, and two that are incorrect altogether. The title options for well-crafted graphs include one correct and three incorrect titles. Once the player selects a title, they will receive feedback from a mentor character called Guido about the correctness of their title selection. Guido also explains how the graph was manipulated and reveals the village's motive for using a manipulated graph in their environmental report (see Figure 2: right). The feedback also highlights the manipulation to ensure players notice it, and an example of a well-crafted graph is presented next to the manipulated graph (see Figure 2: right).
After completing a task, the player earns experience points from a correct answer (selected title). Earned experience points determine the player's rank in the game. There are four ranks in total: intern, assistant, fact-checker, and chief fact-checker, which were designed to help players reflect on their performance. The game also includes a credibility meter. Correct answers increase credibility and incorrect decrease it. If credibility falls to zero, the player must start the game from the beginning.
MediaWatch was designed around inoculation theory through narrative and game design. Specifically, two mechanisms of inoculation theory were applied. First, the narrative is used to warn the player about manipulated graphs and villages' attempts to deceive the player. The aim of such forewarning is to motivate players to defend themselves from manipulation attacks. Second, the game actively and pre-emptively exposes the players to misleading graphs in a safe fantasy environment, underlining the used graph manipulation techniques, and how they were misled (feedback). While playing MediaWatch, the players will reinforce their resistance against manipulated graphs, and the game aims to equip players with specific knowledge about graph manipulation techniques that they can use to refute future manipulation attacks.
Graph reading assessment. To measure the effectiveness of playing MediaWatch on critical graph reading skills, a multiple-choice assessment was administered to both conditions before the intervention (pre-test) and after the intervention (post-test). Participants had 40 seconds to respond to each graph interpretation and graph comparison task. All graphs displayed quantifiable data related to phenomena commonly encountered in geography classes (e.g. population growth, annual rainfall). To minimize the impact of prior knowledge in geography on the results, specific labels and titles were obscured. For example, specific references to countries and areas in titles were substituted with generic terms like "one area" or "one country"; similarly, in data labels, names of countries and areas were replaced with sequential alphabet letters starting from A. The assessment included two types of tasks: graph interpretation tasks and graph comparison tasks. Graph interpretation task type was adopted from [30,6]. Specifically, the assessment included sixteen graph interpretation tasks, of which four were well-crafted graph tasks, and 12 were misleading graph tasks (Figure 3: left). The manipulation methods used in misleading graph tasks were reversed x-axis (four items), y-axis not starting from zero (four items), and y-axis with too wide range (four items). The mean score from misleading graphs is referred to as the misleading graph interpretation score. Graph comparison task type was adopted from [31]. The assessment included six graph comparison tasks (Figure 3: right) that can be considered as near transfer tasks. A graph comparison task includes two graphs from which one is misleading. Half of the graph comparison tasks contained y-axis not starting from zero manipulation, and the other half reversed x-axis manipulation. The mean score from graph comparison tasks is referred as graph comparison score.
Epistemic curiosity scale was adopted from [32] and translated to Finnish. It was measured using a 5point Likert scale (1=strongly disagree, 5=strongly agree) and had 6 items with following example: "I am really curious to know more about this topic". The curiosity items were averaged to measure the degree of epistemic curiosity before and after game-based learning.
Math fluency was assessed as prior research has shown that basic numerical abilities are key predictors of performance in reading graphs [3]. Math fluency was measured with six multiple-choice items. The items measured math competences needed in interpreting the graphs of the graph reading assessment. An example question: "How many times more white squares are there than black circles in the picture?"
Graph familiarity was measured with six 5-point Likert scale items (1=strongly disagree, 5=strongly agree). Participants were asked to reflect how familiar they are with bar and line graphs (e.g., "I am familiar with bar and line graphs").
The study was conducted during a regular school day in a classroom. Participants used their own computers to access all research materials and the MediaWatch game.
First, a researcher provided instructions and details about the study, as well as reminded participants about their rights. Next, all participants received a randomly generated code, which they used to log in to the web-based questionnaire. Prequestionnaire included consent, demographics (e.g., age, gender, high-school grade level), as well as math fluency test, and self-report items to gauge learners' familiarity with graphs and their degree of curiosity (only in the game condition). After the prequestionnaire, participants completed the graph reading assessment. Next, participants of the game condition accessed the MediaWatch game with their codes and played the game through during 30 minutes playing session. The game containing a total of nine graph interpretation tasks. The control condition continued their usual class session, which was unrelated to graph reading or graphs, for 30 minutes. Subsequently, both conditions completed the graph reading assessment as a post-test, and the epistemic curiosity was measured again in the game condition.
First, graph assessment pre-test and post-test scores were calculated ratios of correct items over total items for the misleading graph interpretation task variable and graph comparison task variable. We utilized normalized change scores in our analysis which calculate the maximum possible change from pre to post-test on misleading graph interpretation tasks and graph comparison tasks [33].
Statistical analyses were performed using RStudio [version R 4.1.3] [34], utilizing the 'dplyr' package [35]. Since the misleading graph interpretation and graph comparison task data were not normally distributed and contained outliers, a Wilcoxon ranked-sum test was chosen to examine differences in pre and post-test scores between the game and control conditions (RQ1). A Shapiro-Wilk test was performed to determine whether curiosity pre/post variables were normally distributed. The results revealed that the data were non-normally distributed, W = .948, p = .025. Thus, a Wilcoxon signed-rank test was performed to address the non-normal distribution and to examine the differences in curiosity between pre-and postmeasurements (RQ2).
On average, the learners of the game condition completed a singular MediaWatch graph interpretation task in 4.09 seconds (SD = 1.58).
Learners completed 68% of the game's tasks correctly (overall), while manipulated tasks had 66.3%, and well-formed tasks had a 72% accuracy rate. In addition, 20% of the responses to the manipulated graph tasks aligned with the manipulation. Lastly, 71% of the incorrect responses to the manipulated graph tasks were aligned with the manipulation.
Table 1 shows the descriptive statistics for study variables. To assess the internal consistency of the used measures, Cronbach's Alphas were calculated. Graph familiarity (α = .78), misleading graph interpretation (αpre = .81; αpost = .80), graph comparison (αpre = .68; αpost = .71), and curiosity (αpre = .95; αpost = .95) had at least acceptable internal consistency. Well-crafted graph interpretation (αpre = .33) and math fluency (α = .22) had poor internal consistency, which is understandable due to the ceiling effect.
Wilcoxon ranked-sum tests were conducted to examine if learners in the game and control conditions had any pre-existing differences. The results showed that learners in the two conditions did not differ on math fluency (W = 1148, p = .315), graph familiarity (W=1171, p=.603), and interpretation skills of wellgrafted graphs (W = 1148, p = .135). The age difference was significant (W = 1001.5, p = .049, r = .20), but the effect size was small. A χ2 test revealed that the game condition (boys n = 27; girls n = 23) and control condition (boys n = 24; girls n = 25) did not differ significantly in the proportion of boys and girls, χ2 (1) = 0.25, p = 0.617. Based on these results, we concluded that random assignment produced conditions that were satisfactorily equivalent among these basic characteristics.
The results from Wilcoxon ranked-sum test indicated there were significant differences in misleading graph interpretation change scores between the game (Med = .50) and control condition (Med = .14), W = 906.5, p = .012, with a small to medium effect size of r = .25 (Figure 4).
Another Wilcoxon ranked-sum test revealed that there were no significant differences in graph comparison change scores between the game (Med = 0) and control conditions (Med = 0), W = 1358.5, p = .558, r = .06 (Figure 4).
Wilcoxon signed-rank test was performed to examine whether there were differences in curiosity scores from pre-to post-test for learners assigned to the game condition. The results showed there were no significant differences between pre-test curiosity (Med = and post-test curiosity (Med = 3) in the game condition, W = 368, p = .23, r = .45.
The present study examined the effectiveness of a GBLE called MediaWatch on learners' developing critical graph reading skills. We also examined whether learning with MediaWatch increased learners' epistemic curiosity towards graph literacy after game-based learning. Our results indicated that the game condition demonstrated significant improvement (pre to post-test) in interpreting misleading graphs after playing MediaWatch compared to the control condition, supporting our hypothesis (1a). This finding is consistent with previous research indicating that inoculation based critical reading games can improve learning outcomes [21].
Interestingly, there was no differences in conditions when it came to graph comparison task. This finding led to reject our hypothesis (1b) assuming that learners' graph comparison tasks score will change significantly more from pre-to post-test in the game condition than in the control condition. It is possible that this task did not measure interpretation of misleading graphs properly. As the task includes both a well-crafted and a misleading graph side by side, the manipulation is easier to spot, and the questions are also simpler. Graph comparison task is not as well-established in literature as the graph interpretation task, which has been examined also with eye tracking measures. Future research could investigate the processing of graph comparison tasks with eye tracking and think-aloud methods to evaluate its suitability for graph reading assessments.
Regarding the second research question, the results showed that playing MediaWatch did not significantly change intensity of learners' epistemic curiosity. Thus, we rejected our hypothesis (2). We can only speculate on the possible explanations for this finding. Curiosity was only measured before and after the game but not during gameplay. Thus, critical information is missed on whether learners experienced curiosity while they interacted with the GBLE. It is possible that some learners were curious to learn more about misleading graphs and manipulation techniques while playing the game, but the things that they learned in the game already satisfied their curiosity. On the other hand, it is also possible that the graph literacy topic did not interest learners to trigger curiosity. Invoking curiosity was not considered in the design of the game and that may also explain why there were no differences in curiosity scores.
One limitation of this study is that the intervention was short and included only nine graph interpretation tasks from which six were misleading. Accordingly, a longer intervention (multiple playing sessions) would be needed to better evaluate the usefulness of the current MediaWatch implementation [36], [37]. As we did not conduct a delayed post-test, we do not know how permanent the achieved learning effects are. Moreover, our graph reading assessment did not include a clear transfer task and thus, the results cannot be generalized to other types of manipulated graphs.
Despite the limitations, the results demonstrated the promise of GBLE in supporting learners' ability to interpret misleading graphs.
This study contributed to the field of critical reading games by demonstrating that a graph reading game that utilizes features of inoculation theory can help to build resistance against graph manipulation techniques. Our findings indicates that even a short pre-emptive intervention in the classroom context, can enhance learners' ability to interpret misleading graphs. Thus, MediaWatch proved some promise to be used in schools.
Future researchers should utilize eye-tracking devices while learners read and interpret varying graph types with MediaWatch to provide a deeper insight into specific graph reading processes to inform the design of game elements that can support learners' critical graph reading skills. Additionally, epistemic curiosity should be measured while learners read and interpret graphs using other methods, including emote-aloud protocols [38], where the learner verbally expresses their experience of curiosity during the gameplay. We might get more coherent comprehension what made learner curious and what might have triggered it. Furthermore, since curiosity appears to be experienced while performing tasks, measuring it solely before and after game session, and not during, might be a potential avenue for direction to take in the future endeavors.
Measuring graph reading processes and epistemic curiosity in real-time during gameplay could serve to inform how to adapt the game mechanics to best serve the development of critical graph reading skills and support different learning needs.
This work was supported by Strategic Research Council (SRC) established within the Academy of Finland under Grants [335625, 358250].