CCS CONCEPTS
of the most important tasks for children is learning to develop their knowledge, for which availability of resources at their own reading level is a must. Traditionally, the teacher was the one pointing students to resources they could gather from the library. Nowadays, children spend considerable time online and rely on materials from various websites and apps to help with their homework. In their quest for educational materials, school children directly (and indirectly) rely on search and recommender systems to identify resources that might be useful for them.
Recommender systems (RS) are designed to ease decision-making and reduce information overload.
These characteristics make them particularly useful for children, who are known to struggle with
judging relevance and quality of content presented to them in online environments [
We are interested in studying multiple aspects of the paradigm using search and recommendation
technology in the classroom (see Figure 1). In this article, we explore (i) the added value of a RS to
assist children in their autonomous quest for information and (ii) the degree to which children take
advantage of RS. To control scope, and in order to fit in our framework, we focus on recommendations
generated for the purpose of facilitating completion of inquiry assignments in the classroom seting. We
depend upon a simple popularity-like strategy to generate recommendations. In this case, suggested
resources are chosen based on their frequency of occurrence in the dataset that resulted from the
study presented in [
Children have dificulties with selecting relevant and reliable sources, while ignoring irrelevant
ones. They could benefit from some guidance, and it is typically the teacher who undertakes this role.
Unfortunately, in a digital environment this is not very practical. RS could play a pivot role. Thus, the
question we tackle is “to what extend do children use recommendations and how to improve this usage”.
Lessons learned (i) ofer insights on children’ perceptions of RS when used to solve classroom-related
tasks and (ii) provide foundations for RS that take into account content and context characteristics to
enhance the recommendation process. More importantly, findings prompt us to acknowledge factors
that can directly impact RS and their efectiveness: from expert involvement (to provide instruction
on what RS are and their potential benefits within the classroom seting) and interface design (to
make recommendations visible and accessible), to the level of detail needed on the source of the
recommendations to inspire trust in and define authority of the generated suggestions.
To beter understand if children would choose to bypass browsing resources in favor of directly turning
to resources curated and recommended for them, we recruited 33 child participants in the 4th and
5th grades (ages 9 to 11; 19 males and 14 females; demographic data that contextualizes participants’
experience with online environments is shown in Figures 2 and 3). Following the study framework
presented in [
To locate information to answer the aforementioned questions, children used a simple interface
that resembles a popular search engine: Google. Instead of the common “I’m feeling lucky” option,
they were presented with an option that, if selected, would lead them to a suggested resource
that they could use to complete the question sheet. For recommendation purposes, we simulated
a popularity-based strategy. For each of the 16 prompts (4 topics, 4 questions), we treated the
resource that was most often selected as relevant by participants in the search study presented in [
To understand how children perceived recommendations, we conducted two diferent experiments using two disjoint groups of children. The 15 children in S1 could either search for resources, or instead access recommendations when selecting the “Suggestions by other children” buton. For the remaining 18 children in S2, the buton instead read “Suggestions for you” (see Figure 4 for an illustration of the interface children used for study purposes).
As reported in Table 2, based on the query logs generated as a result of the aforementioned experiments, children in S1 used recommendations to address 15% of their assigned questions, a percentage that decreased to 10% in S2. At the same time, the number of search sessions is comparable for S1 and S2, but children in S2 used more queries per search session to locate the needed information (average 5.8 queries per search sessions for S1 vs. 8.9 for S2). From these results, we believe recommendations were a contributing factor towards minimizing the time and efort required from children to locate relevant resources that contained the data necessary to complete the tasks presented to them.
We noticed that 32% of the search sessions in S1 included the use of the recommendation alternative at least once; a proportion that decreased to 16% for S2. We atribute this to distrust, as the source of the recommendations was not explicitly mentioned to the children in S2, unlike participants in S1 who were made aware that suggestions were based on children’s use of resources.
Another insight that emerged from query log analysis refers to frequency of query repetition: children repeated their queries more often when using the search option than when using the recommendation option (85 total queries, as opposed to 12 queries). This, compounded with the fact that more often a query was repeated when search was used first followed by a recommendation than vice versa (10 vs. 6), lead us to believe that when the recommendation option was used, participants successfully completed their inquiry, i.e., no further action was needed.
In addition to query logs, we examined the results of a short survey presented to the child participants. Survey questions were meant to capture children’s perception on RS and their willingness to use them as a proxy of search engines to complete class assignments. As shown in Table 3, during S1 the use of the recommendation option was uniformly distributed. That was not the case for S2, when the majority of the children favored searching over depending on recommendations. Furthermore, 85% of the children in S1 stated that they would use the recommendation option in the future, a percentage that decreased to 50% among S2 participants. Yet, more children in S2 than S1 would be open to trying the recommendation option, if presented with that choice in the future. This could suggest that children, in retrospect, see the value of recommendations and are willing to use them, just not in this iteration as they lacked information to identify the source of the recommendations. The “Suggested for you” label was too vague for them to trust: not knowing how recommendations were generated could have influenced the likelihood of relying on them to complete an assignment. The main reasons why children used the recommendation option were because they felt “It showed what I needed” and because it was “Useful and complete: it showed me the right sources of information”, still 2 of the children stated not trusting the recommendations (see Figure 5).
Given the struggles children face with successfully completing information discovery tasks in the classroom environment, we argued that RS could help ease this process for them. With that in mind, we conducted an initial study to understand children’s perceptions and usages on RS. From the direct observation of the students engaged with the tasks, it emerged that they enjoyed recommendations and expected them to facilitate the completion of their tasks at school. Nonetheless, they also believed that it would be necessary to receive in advance instructions and/or insights on how to use RS. Based on the inferences made from query log analysis, responses to surveys, and completion of the presented assignments (i.e., responses to prompted questions), we can conclude that when used, recommendations have a positive impact on the achievement of inquiry-related tasks in the classroom seting. It became apparent, however, that children did not really trust and were less willing to turn to suggested resources, if they did not know the source of the recommendation and its authority (i.e., triggered by “Suggestions by other children” vs. the more vague “Suggestions for you”).
The study findings prompt us to explore other suggestion-generation strategies that are not only popular among peers, but are also suitable to the target audience and the classroom context. We are also interested in interface design, so that the recommendation option is visible, intuitive to use, and transparent (i.e., declares explicitly the source of the recommendations). For this, participatory design sessions involving children and teachers will be the next step. Lastly, it became apparent during our preliminary studies that new algorithmic and interface developments will not be enough, which is why we will team up with teachers to develop the type of materials that can support classroom instruction, so that children can be more cognizant of the recommendation process and thus take full advantage of presented suggestions, even beyond completion of schools assignments.
We appreciate the participation in our study of the students in the Leonardo da Vinci school in Lugano and Istituto Comprensivo Antonio Stoppani in Milan. We want to especially thank Meis Huibers for her art. Work partially supported by National Science Foundation, award 1565937.