Finding, Understanding and Learning: Making Information Discovery Tasks Useful for Children and Teachers

Abstract

We present our ongoing e orts on the development of a search environment tailored to 6-15 year-olds that can foster learning though retrieval of materials that not only satisfy the information needs of users but also match their reading abilities. YouUnderstood.me is an enhanced environment based on a popular search engine speci cally designed to help students deal with search for learning tasks, and allow teachers to track their progress. An initial assessment conducted on YouUnderstood.me and well-known (children-oriented) search engines based on queries generated by K-9 students, showcases the need for this type of environment.

1. INTRODUCTION

The use of Web technologies is increasingly becoming a relevant and valuable asset for children's education [ 12 ], both because it enhances the class environment and it introduces children, from early stages of their lives, into today's information society [ 18 ]. Unfortunately, as described by Danby [ 3 ], incorporating technology with more traditional activities into early childhood education is not a trivial task. Children use search engines on a daily basis to locate materials that can help them with di erent academic tasks, from nding information for a class to discovering the meaning of a word [ 12 ]. While the use of search engines for the enhancement of learning tasks is very common, they are not designed with children in mind, and thus a number of issues arise when used by this audience [ 8 ]. An important barrier is showcased by the fact that search engines are not always successful in understanding children's information needs, expressed in long Search as Learning (SAL), July 21, 2016, Pisa, Italy The copyright for this paper remains with its authors. Copying permitted for private and academic purposes. natural language or ambiguous queries [ 1 ]. Other prominent issue is evidenced by the results of the survey conducted by Bilal et al. [ 1 ], which identi es that out of 300 retrieved results to satisfy information needs of 7 graders, only 1 matched their reading level. This is concerning since it is hard for children to comprehend texts with readability levels that do not match their own. Furthermore, given that children as web users, \di er widely in their reading pro ciency and ability to understand vocabulary, depending on factors such as age, educational background, and topic interest or expertise" [ 2 ], it is imperative to tailor the complexity of results to the speci c needs of each child, and not just to generalize based on a label such as age or grade. As reported by Lennon and Burdick [ 14 ], reading for learning takes place when the reader comprehends 75% of a text. This represents an appropriate balance that allows the reader to positively understand the text, while also nding challenges in the reading process that will motivate him to improve his skills [ 14 ]. Therefore, unless the retrieved resources match the reading skills of users, reading for learning, and learning as nal goal, as a part of the online information seeking process cannot take place.

In response to the issues that a ect to the information seeking process, we discuss our ongoing e orts to develop a web search environment designed to help K-91 students in nding adequate online materials. We focus on audience comprised of 6 to 15 year-olds, since these ages refer to children from their initial search experiences through their \graduation" to adult search tools. YouUnderstood.me (YUM) aims to enhance search engines so that they can be used as a tool to facilitate learning, rather than just retrieving information. The main goal of YUM is to improve the information seeking process and increase children's comprehension of retrieved materials by combining diverse functionalities to overcome search engines de ciencies encountered by children. YUM makes the information retrieval process e ective and e cient by (i) taking advantage of readability formulas, a popular search engine, a search intent module, and a query recommendation tool as well as (ii) providing each student with a personal account which keeps track of current readability level and feedback given to previously retrieved resources, enabling YUM to update the predicted reading level2 of students over time. Teachers can also bene t of YUM as they have access to the constantly evolving reading levels of students, allowing them to better adapt classes' materials and pace. 1K-9 refers to grades prior to high school sophomores in the education systems in countries such as USA or Canada. 2We consider a reading level of a student to be the maximum readability level of texts he can understand.

The novelty of YUM lays on creating an environment based on existing search engines that not only serves students in retrieving resources relevant to their information discovery tasks, but also ensures that those resources have appropriate reading levels to each speci c user. Furthermore, YUM builds a bridge to establish a direct relationship between teachers and students, where teachers can follow the progress in readability levels among the students and further foster the learning process. Finally we contribute with an initial study of (children-oriented) search engines conducted over a sample of children written queries, which will be made public to the research community.

RELATED WORK

A number of studies have targeted the issue of search personalization [ 2, 6, 11, 21 ]. The authors in [ 6, 21 ] argue for the need to personalize search results to satisfy diverse users' needs and preferences. However, while their personalization strategy focuses on parameters such as authority of web pages or atypical search sessions, respectively, we focus on parameters that can aid the learning acquisition process, i.e., readability levels of retrieved results. Personalization based on readability has also been explored [ 2, 11 ]. While Collins et al. [ 2 ] demonstrate, based on the results of an extensive query-log analysis, that readability is a valuable signal for relevance of retrieved resources, Jatowt et al. [ 11 ] highlight the need for suitable readability levels on resources retrieved as a result of queries on complex topic formulated by non-experts. We agree on the importance of readability in personalizing web searches, which is why YUM is designed to present its users resources they can read and understand. Our e orts to create a search environment that addresses issues K-9 students encounter while conducting information seeking tasks are further encouraged by the conclusions reported by Huumerdeman and Kamps [ 10 ], who argue in favor of the need to connect literacy and search engines.

Related to search environments speci cally designed for children, the authors in [ 9 ] introduce a search user interface that takes the user's age as a parameter for adaptation. Similarly, YUM focuses on adapting the search environment to the needs of children, but from a reading comprehension standpoint, to facilitate the search-as-learning task. The authors in [ 8 ], on the other side, present an adaptive search user interface that aims at enhancing the search process for 7-to-12 year old children. The focus of their research is on developing a new search environment. YUM instead, focuses in incorporating modular capabilities that can be applied to improve the functionality of popular search engines preferred by children [ 1 ], in terms of the needs and expectations of children. The closest environment to the one we propose is the one described in [ 20 ]. However the application proposed by Usta et al. only o ers grade level ltering, which is a constraint, since students' reading abilities may di er even in a same class and improve over the time [ 19 ]. In addition, their environment is not based on known search engines, which children tend to favor [ 1 ].

While a number of search engines have been developed to aid children, they are not optimal to conduct information discovery tasks for learning purposes as discussed in [ 8 ] and Section 4. To the best of our knowledge, YUM is the only education-oriented environment that considers readability levels as well as queries that potentially lead to the retrieval of child-targeted resources to aid K-9 students in completing successful information seeking tasks. 3.

YUM is an online environment built around a search engine, which aims to make the search process valuable for children. Opposed to similar environments [ 20 ], YUM is not meant to be treated as a new, child-oriented search engine, since studies [ 1 ] show that children prefer popular search engines. Instead, YUM acts as an intermediate layer between the child and an existing search engine (Google Safe Search), to facilitate the interaction between the two of them. For doing so, YUM puts into practice strategies oriented to address issues children face when using popular search engines, as well as strategies that can enhance the search experience to foster learning. A description of the mentioned strategies is provided below.

Search Intent. Children tend to write natural language queries, instead of short, keyword-based ones that search engines usually expect [ 17 ] making children unable to successfully complete information seeking tasks. In addition, children also tend to misspell words, but not necessarily in the same fashion as average users. For example, children commonly repeat letters in a word to emphasize it, such as in "faaaaast", which can cause search engines to misunderstand the intended meaning of the word. YUM leverages our previous research work QuIK [ 4 ], a search intent module designed for children which addresses common patterns in each query Q written by a child including, but not limited to, diminutives, emphasis, children trendy terms or children speci c misspellings, and transforms Q into a new keyword query capturing the information expressed by the child in a way that can be easier for search engines to comprehend. Query Suggestion. Even if a search intent module can identify the most likely intent for each query, users have di erent interests and needs, which is why when dealing with ambiguous queries, it is only each speci c user who knows the purpose of his search. With this is mind, YUM takes advantage of our previously-developed ReQuIK [ 5 ], a query recommender tailored to children, and provides alternatives for the initial query that the user can select to better inform the search process. ReQuIK is based on a multi-criteria strategy that examines traits commonly associated with children and suggests queries that (i) are associated with children topics, (ii) lead to the retrieval of resources with levels of readability matching those of the K-9 audience, and (iii) are diverse.

Filtering by Readability. Even when the search engine has understood the intent of a child query and retrieves results that match the information needs expressed by users, the suitability of retrieved resources is still not assured. K-9 students nd di cult to understand documents containing complex or technical vocabulary. For example, in the case where a child is looking for information about chemistry, retrieving a scienti c publication would not be adequate, while retrieving information from an elementary chemistry book would. If the retrieved documents are too complex, children may not succeed in completing their information discovery tasks. In order to avoid this situation, YUM incorporates a ltering strategy based on readability levels. This strategy ensures that the retrieved documents match, to a degree, the reading ability of each individual user. YUM allows users to go through a one-time process where they can select their grade level, which is originally used as a target to eliminate resources that are not within half a grade level above or below the grade of the corresponding student. For estimating the readability of retrieved resources, YUM uses the Flesh-Kincaid readability formula [ 7 ]. While we expect to develop our own readability formula in the future, we initially selected this formula given that it is considered a standard by educators and institutions for measuring readability. Tracking. K-9 students have diverse reading abilities, which can di er even in same grade class, and progressively improve over time [ 19 ]. Consequently, a one-size- ts-all strategy is not applicable for conducting successful information-seeking tasks that lead to the retrieval of resources individual users can understand. YUM employs an adaptive strategy based on explicit feedback that users can provide by specifying whether the resources retrieved were \Too Easy", \OK" or \Too complex" for them. Children might not be experts in determining the readability of a document, however, YUM takes advantage of their perception over the multiple documents they have read, to obtain estimates about their reading skills. We treat the problem of predicting the current readability level of users as a constraint satisfaction problem, where each feedback provided by a student generates a constraint that needs to be satis ed by the readability of the student. For example, a student s giving a feedback of \Too complex" to a document of readability level 5 would generate the constraint rs < 5, stating that the readability rs of s should be lower than 5. As showed in Equation 1, the predicted readability for s is the one that maximizes the amount of constraints satis ed.

ci2C rs = arg max X (f (ci) if r satisf ies ci r 0 otherwise (1) where r 2 R = f0; 0:5; : : : ; 8:5; 9g represents every possible readability value for the student and C is the set of constraints created based on the feedback provided on retrieved resources by s. According to reports in [ 2 ] users' reading pro ciency needs to be estimated based on both current and past searching process. Thus, Equation 1 considers the time stamps of the created constraints, favoring those created more recently and discarding the ones created outside current academic year. For doing so, f (ci) is a function that starts at value 9 for a new constraint ci and decreases by 1 for each month since the corresponding feedback was provided until 0. We selected 9 as the number of months to consider as this represents the average length of an academic year. Initially, YUM de nes two base constraints that represent one grade of deviation from the current readability of the student: rs < ps + 0:5 and rs > ps 0:5 where ps represents the prior readability of student s based either on the grade level selected the rst time YUM is used or the rs value for the previous academic year. These constraints give YUM a starting level, that will be adjusted as the student uses the environment. 3.1

YUM for teachers

Teachers can also bene t from using YUM within the class environment. Work setting standards have changed from a vertical structure, where only the top individuals of the pyramid had to think critically and the lower parts just followed directions, to an horizontal structure, where each individual is expected to collaborate with others and solve important problems using identi cation, searching, synthesizing, and communication skills [ 15 ]. Given this change, education plans oriented to meet the new requirements of the current industry, such as the Common Core State Standards Initiative (CCSS), have been developed. CCSS requests educators to make an emphasis on higher level thinking during reading and focus on the acquisition of skills such as research and comprehension using digital tools, including search engines [ 15 ]. Furthermore, studies showcase the bene t of in-class exercises such as exploratory talks, where students are asked to solve a problem in groups discussing information found on resources obtained using a search engine [ 13 ]. Unfortunately, teachers might not be able to propose such a task to their students and lead discussions, if students have problems using search engines, whether they are struggling to nd the right queries or not being able to understand the retrieved documents. YUM can help teachers overcome those issues so that they can focus on the discussion, rather than the manner in which students should formulate queries or the type of results they access. Furthermore, YUM can serve as a monitoring tool that allows teachers to check students' progress, based on the resources they have retrieved and their provided feedback in terms of complexity. We believe that YUM can not only facilitate learning when children use it for their information discovery assignments, but it can also help teachers within the classroom environment by addressing the challenge of seamlessly integrating technology to perform everyday classroom activities [ 3, 13 ].

4. INITIAL STUDY

YUM is more than a search engine for children. Instead, it is an enhanced web environment that incorporates features oriented towards facilitating and fostering learning as a result of conducting successful information seeking tasks online. In this initial assessment we expand on the analysis framework presented in [ 8 ] to demonstrate the need of environments such as YUM. For doing so, we examine a number of popular search engines oriented to children3 as well as Google, given that children tend to prefer it over others [ 1 ].

Due to the lack of benchmarks available for evaluating search-related tools focused on young users, we collected our own sample of queries written by children. This sample includes 300 unique queries written by 50 children between the ages of 6 and 15. For creating it, we asked various K-9 teachers in the Idaho (USA) area to propose their students an information discovery task for which the students had to create queries. The domain of the task was open, however, most of the children looked for information about lms and animals, generating queries such as \When is nding Dory coming out?" and \How many cheetahs are in the world?". We submitted these queries to each of the aforementioned search engines and examined their respective retrieved resources as well as the challenges children need to overcome when using these engines. We discuss below details pertaining to each of the aspects considered for our assessment and present an overview of our initial ndings in Table 1. Di culty to retrieve adequate resources. Children are known to struggle when composing queries, often creating queries that are not what search engines expect [ 17 ]. Based on our assessment using children queries, we observed that for 21% of the queries, (child-oriented) search engines considered in this analysis did not retrieve any result or the results 3Kiddle.co, KidRex.org, SafeSearchKids.com and Gogoolingans.com Di culty to retrieve adequate resources Average readability(Flesh) bCyhotsheenuser 12.4 Non adequate contents Mobile friendly that were retrieved did not correspond with what the child would expect, opposed to the 12% for which YUM was in same situation. As an example, the query \lollipop" retrieved resources about the Android Operating System rather than resources about candies or songs, which is what a child would expect.

Readability. The readability level of resources retrieved in response to a child query is also a relevant aspect to explore to quantify the success of a search from a reading for learning perspective. We computed the average readability level of the top-N results retrieved in response to children queries. Given that \children are known to systematically go through retrieved resources and rarely judge retrieved information sources" [ 17 ] we computed the readability scores reported in Table 1 based on the top-3 documents retrieved in response to each query. For measuring the readability level of the retrieved resources, we selected the Flesch formula [ 7 ], as it is considered an standard nationwide. Recall that YUM lters our retrieved resources that do have a complexity level within +/- 0.5 deviation from the reading level of each user, assuring that retrieved resources can be comprehended by its users. Therefore, we only computed the average readability levels of resources retrieved in response to queries posted on (child-oriented) search engines considered in this analysis. As shown in Table 1, the readability levels of retrieved resources are on average above 10, and even one of the search engines (SafeSearchKids) retrieved resources that average 15.6 in terms of readability levels.

General experience. The quality of a search engine is not only determined by its retrieved results, the general search environment is also important [ 8 ]. We observed that the presence of ads was recurrent among the search engines considered in this study. These ads were usually indistinguishable from relevant retrieved resources, which can be confusing, and more importantly, sometimes not ltered for children, advertising products unsuitable for children. For example, we found ads that referred to drug rehabilitation programs or anti-aging products among results retrieved by SafeSearchKids in response to queries such as \frozen characters". We also noticed that platform adaptability was an issue for some of the search engines, since they showed poor support for small screens, such as the ones from phones or tablets, making it hard for a child to use the same system in all platforms. This supposes a signi cant drawback, given that 71% of children frequently access the internet through a tablet [ 16 ]. Finally, most of the search engines showed no or poor support for helping children improve their queries. Google and Gogooligans suggested query reformulations while typing. However, these suggestions were not tailored to children or did not go beyond dictionary based auto completion. For example, when \Sven" (the name of a character from the Disney movie Frozen) was typed, \Seven" (a movie not rated for children) was given as a suggestion in most of the search engines, which doesn't capture the intended meaning of the query considering that it was written by a child. YUM meets the three criteria described, by excluding ads, being adaptable to smaller screens and supporting children to improve their queries by providing suggestions or using the most likely search intent. 5.

CONCLUSIONS

In this paper we presented YUM, an online environment that addresses issues children face when using popular search engines to conduct information seeking tasks. YUM can facilitate the learning that can occur while reading resources that are retrieved as a result of a child-initiated search. As part of our ongoing research e orts, we leverage the use of popular search engines, search intent and query suggestion modules we have developed, a readability-based ltering strategy and a novel tracking strategy, to enhance the search-for-learning tasks conducted online and informing teachers of the progress of their students, in terms of reading and comprehension. We conducted an initial assessment using queries written by K-9 children and demonstrated the need for environments such as YUM. We plan to extend YUM by implementing a number of enhancements. We are aware that the Flesch formula currently used in YUM may not be precise enough. Therefore, we will build our own readability assessment tool, which will go beyond counting terms and syllables, and instead will consider web-page speci c metadata as well as in-depth language information, such as syntax and semantics. An exploration of di erent ltering strategies will also be conducted based on web page authority and the level of maturity of the content retrieved, so that retrieved resources are more suitable to children. We also plan to explore and incorporate new ways of collaborative searching between students and teachers, which could further enhance the learning while searching tasks. We are also aware that children may not provide explicit feedback for all the resources they read. Therefore, we also plan to explore ways of obtaining feedback in a implicit ways, such as analyzing the time spent reading the resources. Finally, a more in-depth study will be conducted to better understand, quantify, and showcase the correlation between learning and information discovery tasks conducted using enhanced web search environments. Since the developmental stages and information needs of K-9 children are broad, we will conduct these studies based on more speci c age ranges, such as 6-8 and 9-12.

References

[1] D. Bilal and M. Boehm . Towards new methodologies for assessing relevance of information retrieval from web search engines on children's queries . Qualitative and Quantitative Methods in Libraries , 1 : 93 { 100 , 2013 .

[2] K. Collins-Thompson , P. N. Bennett , R. W. White , S. de la Chica, and D. Sontag . Personalizing web search results by reading level . In CIKM , pages 403 { 412 , 2011 .

[3] S. Danby . Going online: young children and teachers accessing knowledge through web interactions . Educating Young Children: Learning and Teaching in the Early Childhood Years , 19 ( 3 ): 30 , 2013 .

[4] N. Dragovic , I. Madrazo , and M. S. Pera . \ Is sven seven?": A search intent module for children . In ACM SIGIR , 2016 .

[5] N. Dragovic , I. Madrazo , and M. S. Pera . A multi-criteria strategy to recommend queries for children . In Under review , 2016 .

[6] C. Eickho , K. Collins-Thompson , P. N. Bennett , and S. Dumais . Personalizing atypical web search sessions . In ACM WSDM , pages 285 { 294 , 2013 .

[7] R. Flesch . A new readability yardstick . Journal of Applied Psychology , 32 ( 3 ): 221 , 1948 .

[8] T. Gossen , J. Hempel , and A. Nu rnberger. Find it if you can: usability case study of search engines for young users . Personal and Ubiquitous Computing , 17 ( 8 ): 1593 { 1603 , 2013 .

[9] T. Gossen , M. Kotzyba , and A. Nu rnberger. Knowledge journey exhibit: Towards age-adaptive search user interfaces . In Advances in Information Retrieval , pages 781 { 784 . Springer, 2015 .

[10] H. C. Huurdeman and J. Kamps . Supporting the process: Adapting search systems to search stages . In Information Literacy: Moving Toward Sustainability , pages 394 { 404 . Springer, 2015 .

[11] A. Jatowt , K. Akamatsu , N. Pattanasri , and K. Tanaka . Towards more readable web: measuring readability of web pages based on link structure . ACM SIGWEB Newsletter , (Winter): 4 , 2012 .

[12] S. Knight . Finding knowledge{what is it to 'know' when we search? 2014 http://goo .gl/LQEhXc.

[13] S. Knight and N. Mercer . The role of exploratory talk in classroom search engine tasks . Technology, Pedagogy and Education , 24 ( 3 ): 303 { 319 , 2015 .

[14] C. Lennon and H. Burdick . The lexile framework as an approach for reading measurement and success . Electronic publication on https://goo.gl/WiPlsj, 2004 .

[15] D. J. Leu , E. Forzani , C. Burlingame , J. Kulikowich , N. Sedransk , J. Coiro , and C. Kennedy . Assessing and preparing students for the 21st century with common core state standards . http://goo.gl/wdbTB6, pages 219 { 236 , 2013 .

[16] Ofcom . Children and parents: Media use and attitudes report . 2014 http://goo.gl/g6x9ph.

[17] S. Y. Rieh , K. Collins-Thompson , P. Hansen , and H.-J. Lee . Towards searching as a learning process: A review of current perspectives and future directions . Journal of Information Science , 42 ( 1 ): 19 { 34 , 2016 .

[18] A. Sadaf , T. J. Newby , and P. A. Ertmer . Exploring pre-service teachers' beliefs about using web 2.0 technologies in k-12 classroom . Computers & Education , 59 ( 3 ): 937 { 945 , 2012 .

[19] T. Shin , M. L. Davison , J. D. Long , C. - K. Chan, and D. Heistad . Exploring gains in reading and mathematics achievement among regular and exceptional students using growth curve modeling . Learning and Individual Di erences , 23 : 92 { 100 , 2013 .

[20] A. Usta , I. S. Altingovde , I. B. Vidinli , R. Ozcan , and O . Ulusoy . How k-12 students search for learning?: analysis of an educational search engine log . In ACM SIGIR , pages 1151 { 1154 , 2014 .

[21] H. Wang , X. He , M.-W. Chang , Y. Song , R. W. White , and W. Chu . Personalized ranking model adaptation for web search . In ACM SIGIR , pages 323 { 332 , 2013 .