=Paper=
{{Paper
|id=Vol-2699/paper16
|storemode=property
|title=Information Retrieval from Hypermedia, with a Navigable Overview, at the End of Primary School
|pdfUrl=https://ceur-ws.org/Vol-2699/paper16.pdf
|volume=Vol-2699
|authors=Sandrine Moschetti-Rome,Franck Amadieu,Cécile van de Leemput
|dblpUrl=https://dblp.org/rec/conf/cikm/Moschetti-RomeA20
}}
==Information Retrieval from Hypermedia, with a Navigable Overview, at the End of Primary School==
Information retrieval from hypermedia, with a navigable overview, at the end of primary school Sandrine Moschetti- Romea,, Franck Amadieua and Cécile van de Leemputb a Laboratory CLLE (CNRS, UT2J), University of Toulouse, 5 Allée Antonio Machado, 31058 Toulouse, France b Faculty of Psychological Sciences and Education, Department of Work and Consumer Psychology (PsyTC), Solbosch Campus - CP 122, Avenue F.D. Roosevelt, 50 1050 Brussels, Belgium Abstract At the end of primary school, pupils begin to learn how to manage information web search. This activity is complex for young learners. Thus, they carry out their first learning and use in a "protected hypertext environment". Despite this material adaptation, cognitive processes remain complex and most of pupils need navigation aid. This study aimed to examine how a navigable overview of the structure of the hypermedia could support navigation and promote better performance in answering questions from non-linear hypermedia, considering children’ personal differences. We hypothesized that both information retrieval skills in a non-linear printed document and motivational aspects (self- efficacy about these skills) would predict navigational behaviour and performance. Moreover, we expected that a navigable overview would help when personal resources are low. A sample of pupils schooled in priority education was examined. Our hypotheses were partially confirmed. Pedagogical applications were discussed. Keywords 1 Information retrieval, hypermedia, skills and self-efficacy, navigable overview, children 1. Introduction comprehension: for a review, see Amadieu and Salmeron [4]. In addition, children learn to manage four different tasks: 1. retrieve and At the end of primary school (4th and 5th extract factual information; 2. make simple grades), in Science, History and Geography, inference; 3. interpret and integrate information children are made aware of web research with their own knowledge (locate and relate guided by specific questions2. They work in a two or three information situated in different “protected” digital learning environment [1]: nodes); 4. review and evaluate content. In the different physical structures of hypermedia, in present study, we focused on non-linear which content and navigation functionalities hypermedia where children navigated are limited by the designer. There are two types exclusively by embedded links to perform the of navigation functionalities: keyword querying first and third tasks. In this context, we aimed and/or the selection of relevant links sometimes to better understand the effects of a navigational embedded in a navigation aid like menu or a overview of the hypermedia structure and map [2], [3]. Most of studies showed that a personal factors on navigation and navigational map supports a better Proceedings of the CIKM 2020 Workshops, October 19-20, Galway, Ireland EMAIL: sandrine.rome1@univ-tlse2.fr (A.1); franck.amadieu@univ-tlse2.fr (A.2); cecile.van.de.leemput@ulb.ac.be (A.3) ORCID: https://orcid.org/0000-0002-8144-8450 (A. 1); https://orcid.org/0000-0003-3390-7146 (A.2); https://orcid.org/0000-0002-6369-2804 (A. 3) ©️ 2020 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) 2 French curriculum, cycle 3, page 6 performance. We synthesized below the state of this navigation aid according to personal recent scientific knowledge on (1) cognitive characteristics need more investigations. demand for the two tasks studied, in terms of navigation and treatment of information nodes, 1.2. Influences of individual and on (2) the main personal predictive variables searching information with differences and overview, with navigational overview for students at the end of young students primary school Prior knowledge of the subject matter has a 1.1. Cognitive demand for principal effect on performance. But with young learners, the level of this resource is information retrieval from often very low. Also, reading linear printed hypermedia document skills are powerful predictors of navigation and performance, and browsing Compared to a printed linear informative behaviour mediates the relationship between document, a non-linear hypermedia allows a these skills and performance [9], [15], [16]. more flexible reading path. However, the However, over the past two decades, linear hypermedia structure makes information informational materials are uncommon in retrieval task more complex, and navigation children's literature. From the beginning of behaviour are closely related to hypertext elementary school, students learn to read reading success [5], [6]. Information searchers printed non- linear document with help of meta- have to access relevant information while they textual clues. Although these skills are not yet don’t know the extent of the hypermedia and fully mastered at this age, we hypothesized that only one page is visible at a time. An effective they could support a task of information strategy consists in discovering the entire retrieval from hypermedia. Indeed, the document before reading the questions [7]. structural characteristics of a printed non-linear After, they analyze question and make inferences document have similarities with a hypermedia with their prior knowledge and reading material [17]. Otherwise, studies conducted with high (To answer this question, what do I already know school students have shown that motivational and what information could I find in the aspects are predictive of web search documents, where and how to access it?). To achievement [18]. For complex academic tasks, locate relevant nodes, they make inferences from motivational factors (self- efficacy about hyperlinks about the content of the corresponding reading) have also been identified as predictors nodes. When hypermedia suffer from/has a lack of reading task success [19]–[21]. However, to of cohesion, they use their knowledge about the our knowledge, the predictive power of this subject matter to fill in the "blanks" and build a personal characteristic has not been tested with suitable text representation. Then, without children, in the specific context of information forgetting the goal of their research, they "flies retrieval from hypermedia. Moreover, recent over" or ignore irrelevant parts and directly access studies on children suggest that an overview relevant information to answer questions [7]–[9]. could help to answer questions from text. Complexity is increased when the task requires Indeed, Salmerón and Garcia [15] found that, locating and integrating information from under certain conditions, an overview had a different nodes. “Very good level pupils” have main effect on performance in information motivational, attentional, cognitive and retrieval, for answering inferential questions. metacognitive resources to match up to these However, they didn’t found any interaction in cognitive demands. However, for most of them, the relationship between skills and at the end of primary school, explicit learnings performances. Overview effect would be of information retrieval strategies are in the limited when learners' reading comprehension process of being acquired [10]–[12]. Thus, skills were too low: in this case, children navigation may be "hazardous" and poor couldn’t take full advantage of this navigation performance are usual [10], [13], [14]. It seems aid [22]. In contrast, Fesel and colleagues [23] that an overview may support information found that overview might support children retrieval tasks in a hypermedia for young with low prior knowledge. Furthermore, the pupils. But as we detailed above, the effects of interactions between personal variables and navigation on the one hand and performances 2. Method on the other hand are still unclear. 2.1. Design and participants 1.3. Our study: aims, framework and hypotheses Before the study started, we informed the headmasters of the experimental schools, the parents and the children. We also obtained the The present study examined both the authorizations required by French law. One predictive power effects of new personal hundred and nineteen learners from two priority variables (i.e. information retrieval skills in a education schools participated at the printed non-linear document and self-efficacy experience. (Mage = 10,06 years, SD = 0,63 ; 61 about these skills) and navigable overview on females et 39 males). Before any data navigation and performance. In addition, we collection, they were divided, into two groups: examined the moderator effect of overview in the test group (with overview) and the control relationships between personal variables and group (without overview). These learners rarely navigation behaviour and performance: used tablets in class and only to do exercises, take photography and produce simple H1: Main effect of the overview. Overall, an documents. In addition, they regularly learned overview displays the structure of the how information retrieval works from linear hypermedia and allows direct access to relevant and non-linear printed documents, but never pages (avoids the use of backward buttons and from hypermedia or from web. In addition, intermediate pages opening) and helps to relate students' prior knowledge was very low. information situated on different nodes. So, we Nineteen participants were excluded from the expected that, compared to the opposite data analysis due to incomplete data (these navigation condition, overview should support participants did not attend all sessions in the more accurate navigation (H1a) and study). performance should be better (H1b). H2: Effect of personal characteristics. A 2.2. Material printed non-linear document shares common structural characteristics with a hypermedia. 2.2.1. Device and hypermedia Thus, information retrieval skills in a printed non-linear document (here, a double-page) Two versions of a hypermedia document should have a positive influence on navigation have been designed with the software Sparkle (H2a) and performance (H2b). Moreover, we 2.6.2 Visual Web Design [24]: with and expected to find similar results with previous without overview. studies regarding the relationship between motivational aspects and navigation on the one hand and performance on the other hand. In other words, self-efficacy about information retrieval should support navigation (H2c) and performance (H2d). H3: Interaction effect. An overview could support students with low level skills. Furthermore, considering motivation is related to instructional condition, we hypothesized that overview could also help students with low Figure 1: Example of node of the personal resources. Thus, overview would experimental hypermedia. On the left, without condition the effect of skills and self- efficacy overview. At the center, the same node with about information retrieval, on navigation and overview. And, on the right, the overview performance. In other words, without overview, opened. information retrieval skills and self- efficacy about these skills would have a greater effect on The two versions included texts (nearly 600 navigation (H3a) and performance (H3b). words), graphical elements and data tables distributed on 10 nodes. Hyperlinked words and Self-Efficacy about information retrieval backward buttons were embedded in each node. in non-linear printed document. Data about The versions differed in the presence or not of self-efficacy was collected from self-ratings an overview. Overview was implemented at the based on Bandura’s Children’s Self-Efficacy top of each node. It showed exactly the Scale [26]. They indicated the strength of their hypermedia structure. Like the city map, a red efficacy beliefs on a 100-point scale, ranging in point indicated where the reader was in his 10-unit intervals from 0 (“I feel I can’t do it at navigation path (Fig.1). To navigate, in both all”) to complete assurance, 100 (“I feel I can versions of the hypermedia, participants could do it at all”). Questions were related to reading click on hyperlinks in the text and backward and information retrieval from expository buttons. In addition, the test- group could click printed non- linear document: for example, “To on the Overview to display it full screen and which extent do you feel able to read an make it interactive. The hypermedia were read expositive document that includes texts, on iPAD tablets (9.7" screen size) belonging to pictures, graphics and data tables?” (translated schools. from French). Chronbach’s alpha = 0.78. A mean score of self-efficacy was computed from 2.2.2. The task the items ratings. The task consisted of retrieving relevant 2.3.2. Dependent variables information from hypermedia in order to answer three questions: one explicit question Performance at the task. The scale was: 0 (i.e. factual; the answer required finding one point for wrong answer or no answer; 0.5 point piece of information located in one node) and for a partial correct answer, and 1 point for a two implicit questions (i.e. inferential; the full correct answer. A mean score of answer required finding and relating two or performance was computed from the three three pieces of information located in different items. nodes). All questions were given at the Navigation accuracy. The quality of beginning of the session. Students wrote their navigation was assessed from accuracy ratio answers with paper and pencil. [27]: the total number of target nodes of the predefined path visited / the total number of open nodes (excluding clicks in order to open 2.3. Measures the overview page, for the test group). We 2.3.1. Individual differences computed the ratio for the global path. It was equal to 1 if all relevant nodes were visited and Prior knowledge. Prior knowledge about equal to 0 if none of them were visited. A mean production and consumption of electricity was score of navigation accuracy was computed assessed by 9 open ended questions during a from the three accuracy ratios. collective session. For each question, the scoring consisted of 0 point for a wrong answer 2.4. Procedure or no answer, 0.5 for a partial correct answer, and 1 point for a correct answer. The study was included in a Science and Information retrieval in non-linear printed Technology learning sequence about document skills. We used a test adapted from production and consumption of electricity PIRL assessment tool [25]. The document which lasted nine sessions (2 sessions per week, included texts, pictures, graphics and data on average). The experimenter conducted all tables. In accordance with our problematic, we sessions. Prior knowledge, skills and self- focused on retrieving and extracting factual efficacy were measured in the first session. The information and, making simple inference. 2nd and 4th sessions consisted in a Students answered four explicit or implicit familiarisation with hypermedia and questions. Each item was scored as following: experimental conditions (hypermedia about 0 for no answer or wrong answer; 0.5 for a production of electricity). The 6th session was partial answer; 1 for a correct answer. A mean the experimental session (hypermedia about score was computed from the four items. consumption of electricity). All questions were given on questions- answers sheet (paper and pencil format) before starting the information 3.2. Performance retrieval. Participants wrote their answers respectively under each question. They could The model for the performance scores process questions in any order and they could revealed a significant model, Rcoor2 = .31, come and go between the hypermedia and the F(6,79) = 7.44, p < .00. In accordance with our questions as much as they needed. They could hypotheses, overview supported performances, ask help at the experimenter for reading β = .91, t(85) = 4.96 p < .001. Performance were questions and writing answers. Finally, there also improved by skills, β = .36, t = 2.85, p = was no time constraint. During 5th, 7th, 8th and .006 and by self-efficacy, β = .38, t(85) = 2.84, 9th sessions, over the experimentation, the p = .006. Moreover, prior knowledge had a experimenter proceeded to structure new significant effect but at the limit of the knowledge and finally assessed the level of conventional threshold, β = .19, t(85) = 2.05, p acquisition about this knowledge. = .044. About interactions, our hypotheses were partially confirmed. There was no interaction 3. Results between skills and guidance, β = -.32, t(85) = - 1.76, p = .083. But the interaction between self- First, we checked with t-tests that the groups efficacy and guidance was significant, β = - .40, did not differ on the cognitive factors assessed. t(85) = -2.21, p = .003. Navigable overview As expected, there were no differences between moderated self- efficacy effect. Overview groups in their scores for prior knowledge, t(85) sustained participants with lower level of self- = -.88; p = .38, skills, t(85) = 0.75, p = .45; and efficacy (Fig.2). self-efficacy, t(85) = 1.68, p = .09. In order to test the effects of guidance condition (i.e. with vs. without overview) and personal differences (skills; self-efficacy), we performed multiple linear regression analyses with interaction terms, on each dependent variable. Skills and self-efficacy were entered as z-standardized variables. The guidance condition was entered as a contrast-coded dummy variable. In our model, we entered prior knowledge as a control variable but we expected a non-significant effect because level was very low in both groups, M = .14, SD = .17 Figure 2: Predicted value of and for test-group, M = .17, SD = .20. performance; effects of self-efficacy according to guidance condition 3.1. Navigation accuracy A quick descriptive analysis of the navigation historic showed that all participants 4. Discussion of the test-group had systematically use overview for their navigation. The model for The study investigated the effects of a the navigation accuracy scores appeared navigable overview and individual significant, Rcorr² = .21, F(5,80) = 5.58, p < .001. characteristics (skills about information As expected, overview supported better retrieval from printed non-linear document; navigation accuracy β = .93, t(85) = 4.78, p < self- efficacy about these skills) on navigation .001. But in contrast with our hypotheses, and performance: main effects and interactions. neither skills influenced navigation β = .25, Our assumptions were partially confirmed but t(85) = 1.93, p = .056, nor self-efficacy, β = .27 tended to support or complement previous , t(85) = 1.93, p = .057. And, no interaction was studies. observed: neither between overview and skills, β = -.11, t(85) = -.54, p = .59 nor between overview and self- efficacy, β = -.26, t(85) = - 1.34, p = .18. 4.1. Predictive variables of variables, we computed the average navigation performance and navigation accuracy. Yet, information retrieval with young previous studies [7], [8], [16] suggested that the learners effect of predictive variables would differ depending on the type of question. Regarding Predictive variables of navigation: our motivational aspects, we measured self- study confirmed the main effect of an overview efficacy before participants were informed of on the quality of navigation [4], [22]. In the task conditions. However, it was possible addition, while the test-group had the choice to that self-efficacy may have evolved after the navigate via the links embedded in the node or sessions of familiarization. Thus, for future via the overview, all participants systematically research, it would be appropriate to collect a used overview. Thus, the acceptability of this measure of self-efficacy after the exploratory navigation aid would be very strong with phase and after explaining to each group the children. It would help to make lexical or task and de navigation condition. However, semantic correspondences between the despite these limitations, this study allowed us question keywords and relevant nodes. It to identify directions for instructional allowed direct access to the target pages recommendations. First, teacher should (avoiding intermediate pages). Over this design explicitly teach the similarities and the factor, we tested new personal variables: skills differences between different reading materials. about information retrieval from a printed non- It would help children to transfer and adapt their linear document and self- efficacy about these personal resources acquired in printed non skills. We found no main effect related to linear document to hypermedia. Until students personal variables. Maybe participants would have acquired sufficient personal resources, not have transferred their cognitive and designers and teachers should ensure that motivational resources on navigation strategies reading materials are equipped with a in hypermedia; they would not have identified navigation aid like an overview. But in order to the implicit structural similarities between a children benefit fully from overview, they need printed non-linear document and a hypermedia. to be provided with explicit learning about it. Predictive variables of performance: our study confirmed that cognitive resources have a 5. Conclusion major effect on performance. Like linear reading skills [9], [15], [16], non-linear reading The study supported that both personal and skills would help to process the selected pages design factors predict digital reading with and link them. Our study also highlighted the young students. For an information retrieval importance of taking into account motivational task, a navigational overview could be an aspects when the task is complex and, above all, effective aid. Moreover, we highlighted the the influence of instructional choices on importance of both cognitive and motivational motivation [21]. Participants who felt they had variables about reading a non-linear document. low self-efficacy would have perceived that overview could give them autonomy. 6. Acknowledgements 4.2. Limits of the study and This research was funded by the National further works Agency for Research (LETACOP project). We thank Pierre-Vincent Paubel, engineer, for his About navigation accuracy: we could not be help in collecting and treatment of logs files. sure that a low level reflected a lack of efficiency: it could be explained by a navigation 7. References behaviour motivated by the pleasure of handling the material over the task. Indeed, as mentioned earlier about this research, [1] M. Macedo-Rouet et J.-F. Rouet, participants have no previous experience with « Documents multiples : processus et o reading hypermedia; it was possible that the difficultés de lecture », Canopé, n n° 48, p. novelty effect had negatively influenced p.13, juill. 2011. navigation accuracy. About dependant [2] N. Gauducheau, F. Cuisinier, et C. grade school students », Appl. Cogn. Psychol., Garitte, « Utilisation d’hypermédia et vol. 22, no 3, p. 389‑406, avr. 2008, doi: recherche d’information chez les enfants », in 10.1002/acp.1415. Ergonomie des documents électroniques, [11] R. Cerdán, R. Gilabert, et E. Vidal- Presses Universitaires de France., Paris: Presses Abarca, « Selecting information to answer universitaires de France, 2008, p. 103‑128. questions: Strategic individual differences [3] J.-F. Rouet, C. Ros, A. Goumi, M. when searching texts », Learn. Individ. Differ., Macedo-Rouet, et J. Dinet, « The influence of vol. 21, no 2, p. 201‑205, 2011, doi: surface and deep cues on primary and 10.1016/j.lindif.2010.11.007. secondary school students’ assessment of [12] M. G. Cataldo et J. Oakhill, « Why Are relevance in Web menus », Learn. Instr., vol. Poor Comprehenders Inefficient Searchers? An 21, no 2, p. 205‑219, avr. 2011, doi: Investigation Into the Effects of Text 10.1016/j.learninstruc.2010.02.007. Representation and Spatial Memory on the [4] F. Amadieu et L. Salmerón, « Concept Ability to Locate Information in Text », J. Maps for Comprehension and Navigation of Educ. Psychol., vol. 92, no 4, p. 791‑799, 2000. Hypertexts », in Digital Knowledge Maps in [13] J.-F. Rouet et A. Potocki, « From Education, D. Ifenthaler et R. Hanewald, Éd. reading comprehension to document literacy: New York, NY: Springer New York, 2014, p. learning to search for, evaluate and integrate 41‑59. information across texts / De la lectura a la [5] J. Naumann et L. Salmerón, « Does alfabetización documental: aprender a buscar, Navigation Always Predict Performance? evaluar e integrar información de diversos Effects of Navigation on Digital Reading are textos », Infancia Aprendiz., vol. 41, no 3, p. Moderated by Comprehension Skills », Int. 415‑446, juill. 2018, doi: Rev. Res. Open Distrib. Learn., vol. 17, no 1, 10.1080/02103702.2018.1480313. févr. 2016, doi: 10.19173/irrodl.v17i1.2113. [14] J.-F. Rouet, The skills of document use: [6] L. Salmerón, H. I. Strømsø, Y. from text comprehension to Web-based Kammerer, M. Stadtler, et P. van den Broek, learning. Mahwah, NJ: L. Erlbaum Associates, « Chapter 4. Comprehension processes in 2006. digital reading », in Studies in Written [15] L. Salmerón et V. García, « Reading Language and Literacy, vol. 17, M. Barzillai, J. skills and children’s navigation strategies in Thomson, S. Schroeder, et P. van den Broek, hypertext », Comput. Hum. Behav., vol. 27, no Éd. Amsterdam: John Benjamins Publishing 3, p. 1143‑1151, mai 2011, doi: Company, 2018, p. 91‑120. 10.1016/j.chb.2010.12.008. [7] L. Salmerón, J. Naumann, V. García, et [16] J. Naumann et F. Goldhammer, I. Fajardo, « Scanning and deep processing of « Time-on-task effects in digital reading are information in hypertext: an eye tracking and non-linear and moderated by persons’ skills and cued retrospective think-aloud study: Scanning tasks’ demands », Learn. Individ. Differ., vol. and deep processing in hypertext », J. Comput. 53, p. 1‑16, janv. 2017, doi: Assist. Learn., vol. 33, no 3, p. 222‑233, juin 10.1016/j.lindif.2016.10.002. 2017, doi: 10.1111/jcal.12152. [17] É. Bautier, J. Crinon, C. Delarue- [8] J.-F. Rouet, « What was I looking for? Breton, et B. Marin, « Les textes composite: des The influence of task specificity and prior exigences de travail peu enseignées? », knowledge on students’ search strategies in Repères, no 45, p. 63‑79, juin 2012, doi: hypertext », Interact. Comput., vol. 15, no 3, p. 10.4000/reperes.136. 409‑428, juin 2003, doi: 10.1016/S0953- [18] J. Naumann, « A model of online 5438(02)00064-4. reading engagement: Linking engagement, [9] C. Hahnel, F. Goldhammer, J. navigation, and performance in digital Naumann, et U. Kröhne, « Effects of linear reading », Comput. Hum. Behav., vol. 53, p. reading, basic computer skills, evaluating 263‑277, déc. 2015, doi: online information, and navigation on reading 10.1016/j.chb.2015.06.051. digital text », Comput. Hum. Behav., vol. 55, p. [19] Y. S. Lee et M. Jonson-Reid, « The 486‑500, févr. 2016, doi: Role of Self-Efficacy in Reading Achievement 10.1016/j.chb.2015.09.042. of Young Children in Urban Schools », Child [10] J.-F. Rouet et B. Coutelet, « The Adolesc. Soc. Work J., vol. 33, no 1, p. 79‑89, acquisition of document search strategies in févr. 2016, doi: 10.1007/s10560-015-0404-6. [20] H.-F. Shang, « Online Metacognitive Strategies, Hypermedia Annotations, and Motivation on Hypertext Comprehension », Educ. Technol. Soc., no 19 (3), p. 321–334, 2015. [21] J. T. Guthrie, A. Wigfield, et W. You, « Instructional Contexts for Engagement and Achievement in Reading », in Handbook of Research on Student Engagement, S. L. Christenson, A. L. Reschly, et C. Wylie, Éd. Boston, MA: Springer US, 2012, p. 601‑634. [22] L. Salmerón et V. García, « Children’s Reading of Printed Text and Hypertext with Navigation Overviews: The Role of Comprehension, Sustained Attention, and Visuo-Spatial Abilities », J. Educ. Comput. Res., vol. 47, no 1, p. 33‑50, juill. 2012, doi: 10.2190/EC.47.1.b. [23] S. Fesel, E. Segers, et L. Verhoeven, « Individual variation in children’s reading comprehension across digital text types », J. Res. Read., vol. 41, no 1, p. 106‑121, févr. 2018, doi: 10.1111/1467-9817.12098. [24] D. Trambusti et D. Wilcox, Sparkle 2.6.2 Visual Web Design. 2011. [25] « PIRLS 2016 : évaluation internationale des élèves de CM1 en compréhension de l’écrit. Évolution des performances sur quinze ans. », p. 4. [26] A. Bandura, « Guide for constructing self-efficacy scales », in Self-efficacy beliefs of adolescents, Greenwich, CT: Information Age Publishing., vol. 5, F. Pajares et T. Urdan, Éd. 2006, p. 307‑337. [27] F. Amadieu, C. Bastien, et T. André, « Les méthodes on-line 1: analyse des parcours », in Ergonomie des documents électroniques, A. Chevalier et A. Tricot, Éd. PUF, 2008, p. 251‑270.