Categorical Information Improves the Effectiveness of Refutation Texts Irini Skopeliti (eskopel@upatras.gr) Department of Educational Science and Early Childhood Education, University of Patras Rio, Patras, Greece Stella Vosniadou (svosniad@phs.uoa.gr) Department of Philosophy and History of Science, University of Athens Athens, Greece, and School of Education, The Flinders University of South Australia, Australia Abstract Wu, 1997; Kendeou & van den Broek, 2005, 2007). However, refutation text studies have not adequately addressed the We present the results of an empirical study, which investigated the question of how much information as well as what kind of effects of the amount of information and the type of information information should be refuted. refuted in refutation texts. The study compared the effects of three In the present study we investigated the effects of refuting refutation texts on elementary school children’s understanding of the more information as well as of adding categorical information. scientific concept of the Earth. One text refuted only the belief that More specifically, the texts used refuted prior knowledge the Earth is flat, the second refuted in addition the belief that people cannot live at the ‘bottom’ of the spherical Earth because they would representing the belief that the Earth is flat, and the belief that fall down, and the third added the categorical information that the people cannot live at the bottom of the Earth because they will Earth is an astronomical object. The term ‘categorical information’ is fall down. In addition, we investigated the effect of adding in used to refer to information about the category to which a concept the text information about the scientific category to which the belongs. We hypothesized that the text refuting more information as Earth belongs. Conceptual change research has shown that well as the one that included the categorical information that the learning science often requires re-assignment of the category Earth is an astronomical object would be more effective in improving to which a concept belongs (Chi, 2013; Vosniadou & students’ scientific knowledge about the Earth. The refutation of Skopeliti, 2005). categorical information is important because it is generic and carries a great deal of implicit information that can guide new learning. The findings showed that the text that included more information as well Categorical Information and Conceptual Change as the categorical information facilitated scientific understanding Categorization is one of the most powerful learning most. The results support the hypothesis that both the amount of mechanisms (Chi, 2013; Medin & Rips, 2005; Vosniadou, information and the kind of information being refuted need to be 2008). When an object is assigned to a given category, all the taken into consideration when deciding what to refute in a refutation text. characteristics of the category are transferred to this object. For example, if the ‘Earth’ is assigned to the category of physical objects, then the characteristics of physical objects, Keywords: refutation texts; categorical information; learning; conceptual change known to the learner, are assigned to it. Prior research has shown that young children do indeed assign the Earth to the category of physical objects and transfer to it characteristics of Introduction physical objects, such as stability, support and the belief in an Refutation texts ‘up/down’ gravity (Vosniadou & Skopeliti, 2005). Categorization is a dynamic rather than a static process.. Science texts are difficult to understand, especially for young, During development many concepts may change category, i.e. elementary school children (Goldman & Bisanz, 2002; plants are considered to belong to the category of non-living Graesser, Leon, & Otero, 2002; Vosniadou & Skopeliti, in things at first but later change category and are considered to preparation). One possible reason for these difficulties is that belong to the category of living things (Carey, 1985). This science texts do not take into consideration children’s process of re-categorization is an important part of the incompatible prior knowledge and their possible conceptual change process that takes place, particularly in misconceptions (Mikkila-Erdmann, 2002; O’Reilly & learning science. The re-categorization of plants from the McNamara, 2007). In situations where an expository text category of “non-living” to the category of “living” things is presents counter-intuitive scientific information which may accompanied by important changes in the characteristics not be supported by readers background knowledge, the use of applied to pants. refutation texts - the texts that acknowledge students’ Many of the misconceptions children form are the result of alternative conceptions and then explicitly refute them- assimilating scientific information into their background produces better understanding results than non-refutation texts knowledge without changing their initial categorizations (Broughton & Sinatra, 2010; Diakidoy, Kendeou, & (Vosniadou & Skopeliti, 2013). When conceptual change is Ioannides, 2003; Dole, 2000; Guzzetti, Williams, Skeels, & 1 78 achieved, a new scientific concept has been formed which has children’s categorizations and their Earth shape models. In either been re-assigned to a different existing category, or an conclusion, the results supported the hypothesis that there is a altogether new category has been formed (Chi, 2013). In most change in the categorization of the Earth from a physical to an cases (i.e., concepts like heat, energy, force, matter, etc.) the astronomical object, and that the re-categorization of the Earth new, scientific category does not exist and needs to be as an astronomical object may precede children’s full constructed with the help of specific instruction.   understanding of the Earth as a spherical planet that rotates around its axis and revolves around the sun. Previous studies have shown that when children’s initial The concept of the Earth beliefs about the Earth are addressed, their understanding of Previous research has shown that children have considerable the spherical shape of the Earth is improved. A study difficulty understanding that the Earth is a sphere that rotates comparing a control group that received traditional instruction around its axis and revolves around the Sun (Vosniadou & with an experimental group that received additional Brewer, 1992). It has been argued that this is the case because information about the spherical shape of the Earth and gravity children categorize the Earth as a physical object and apply to based on simulations, showed much improved understanding it the characteristics of physical objects, which are found on of the scientific concept of the Earth for the experimental the Earth, such as solidity, stability and up/down gravity. group (Vosniadou, Ioannides, Dimitrakopoulou, & These in turn constrain children's understanding of the Papademetriou, 2001). In another study, Hayes, Goodhew, scientific information about the earth and become the cause Heit, and Gillan (2003) based on the findings of Vosniadou for children to form various kinds of misrepresentations of the and Brewer (1992) used non-refutational texts and videos to Earth. explain the beliefs of flatness and up/down gravity that A substantial body of cross-cultural research supports the constrain the scientific understanding about the earth. The conclusion that during the preschool years children construct results again showed significant differences between the an initial concept of the earth based on interpretations of experimental and the control group. In the present study, we everyday experience in the context of lay culture. According investigated the effects of the refutation of the same two to this initial concept, the Earth is a flat, stable, stationary, and beliefs as well as of the categorical information that the Earth supported physical object. Objects located on the earth obey is an astronomical object.. the laws of up/down gravity, and space is organised in terms Three refutation texts were compared. The first dealt with of the dimensions of up and down. The sky and solar objects the belief that the Earth is flat (Text 1), the second also refuted are located above the top of this flat earth, which is thought to the belief that people cannot live at the bottom of the Earth occupy a geocentric universe (Nussbaum, 1985; Vosniadou & because they will fall down (Text 2), and the third added to the Brewer, 1992, 1994). above the information that the Earth is an astronomical object The scientific concept of the Earth, to which children are rather than a physical object (Text 3), i.e., a spherical planet in exposed at least as soon as they enter elementary school, a heliocentric solar system. More specifically, Text 1 violates practically all of the characteristics that apply to explicitly stated “the Earth is not flat but spherical, like a very children’s initial Earth concept. According to the scientific big ball” and explained that “the Earth looks flat to us who concept, the Earth is a planet, i.e., an unsupported, spherical, live on a very small piece of it, like Greece”. Text 2 added to astronomical object, which rotates around its axis and revolves this the information that “gravity is the force that holds us on around the sun, in a heliocentric solar system. People live all the Earth so that we do not fall down. It pulls everything that around the spherical Earth and gravity operates towards the is found on the Earth’s surface towards the center of the centre of the earth. Understanding the scientific concept of the Earth. Because of gravity people can live on all the parts of Earth requires that children re-categorize the earth to a new the spherical Earth without falling”. Finally, Text 3 also stated ontological category, that of an astronomical object. “the Earth is one of the planets of our solar system. Today we Vosniadou & Skopeliti (2005) examined in detail this know that our solar system is heliocentric. All planets revolve hypothesis and showed that indeed there is a shift in children’s around the sun. The sun is located in the middle of the solar categorizations of the Earth from a physical object to an system. The Earth is also a planet with all the characteristics astronomical object, and that this shift is related to children’s of planets. All planets are spherical, round like very large understanding of the scientific model of the Earth. In this balls”. study 62 1st and 5th grade children were shown 10 cards with We hypothesized that Texts 2 and 3 that refuted more the words ‘SUN’, ‘MOON’, ‘STAR’, ‘EARTH’, ‘PLANET’, information would be better in improving children’s ‘HOUSE’, ‘CAT’, ‘ROCK’, ‘TREE’, and ‘CAR’ and were understanding compared to Text 1. However, it was also asked three categorization questions. The results showed that hypothesized that Text 3, which contained the categorical the great majority of the children were able to distinguish information that the Earth is an astronomical object, would be physical from astronomical objects and that there was a more effective than the other two.This is the case because developmental shift in their categorizations of the Earth. Many categorical information is generic and allows inferences about of the younger children thought that the Earth belongs with the the characteristics of astronomical objects while it prevents physical objects, while practically all of the 5th graders inferences of the sort that the Earth is stable and in need of categorized the Earth with the astronomical objects. support, and of ‘up/down’ gravity. Furthermore, significant correlations were obtained between 2 79 Method and of 2 if they revealed exposure to scientific information, which was however not fully understood. No responses were Participants scored as 0. Thus, for each student the total score could range Eighty-one 3rd grade students (mean age 8 years and 4 months) from 0 to 42. from a middle-class school in Athens, Greece, participated in A group type (text1*text2*text3) ANOVA was used to test the study. 3rd graders were selected because previous studies whether there were any significant pretest differences between have shown that although they are exposed to scientific the two experimental groups assigned to the different text information about the Earth they do not yet fully understand conditions.. The results showed no statistically significant the Earth concept. Furthermore, this is the youngest age at differences between the three experimental groups in the which we expected the children to be able to read and pretest [F(1,78)=.710, n.s.; η2=.004]. understand a simple science text. Children’s total scores in all the common (explicit and inferential) questions in the pretest and posttest were subjected Materials to a mixed two-way ANOVA with text type as a between subjects variable and pre-post test performance as a within The materials used were: (a) three different refutation texts subjects variable. The results showed that all refutation texts which referred to the Earth, and (b) a questionnaire consisting improved significantly children’s responses in the posttest of 14 open-ended questions. As mentioned earlier, Text 1 [F(1,78)=49.742, p<.001] (see Table 1) and the size effect for refuted the belief that the Earth is flat and explained that it is this analysis [η2=.387] exceeded the convention for a large spherical (310 words). Text 2 included all the information effect (η2=.025). found in Text 1 and also refuted the belief that people can fall off the Earth and explained that there is force called gravity Table 1: Mean Score and Mean Score Difference in Children’s that holds people on to the surface of the spherical Earth (410 Responses in All the Common Questions in the Pretest and the words). Text 3 included all the information found in Texts 1 Posttest as a Function of Text Type and 2 and also added the information that the Earth is an astronomical object (510 words). All texts were of comparable readability level (Flesch index: 57.84 for text 1, 61,32 for text Text Mean Std. Error Mean Difference 2 and 73,24 for text 3). 1 Pretest 18.950 .586 .500 The questionnaire was used both as a pre-test and as a Posttest 19.450 .529 post-test. The questions were divided in two groups: (1) 2 Pretest 19.457 .443 .743 explicit questions, some of which arose directly from all the Posttest 20.200 .400 texts (common explicit), and some only from Text 3; and (2) 3 Pretest 19.038 .514 2.347 inferential questions, the answers to which were not explicitly Posttest 21.385 .464 stated in the texts but could be inferred from it. Again, some inferential questions could be inferred from all texts (common The analysis also showed a statistically significant inferential), while others could be inferred only from Text 3. interaction between text type and students difference in performance from pretest to posttest [F(2,78)=7.438, p<.001]. Procedure The effect size for this analysis [η2=.160] was found to exceed the convention for a large effect (η2=.025). The students who Children were randomly assigned to one of the experimental read Text 3 showed greater improvement in their responses in conditions. More specifically, each classroom was randomly the posttest compared to the pretest, as opposed to the students given one of the three texts to read. The pre-test was who read Text 1 and Text 2 (Figure 1). administered first. When all children finished answering the pretest, the answer sheets were collected and a written copy of one of the refutation texts was distributed. They had 15 minutes to read it on their own. At the end of the reading period, the experimenter read the text aloud in order to make sure that the children could decode all the words. Then the written texts were withdrawn and the posttest was administered.. Results Scoring. Two judges agreed on a scoring key, which they used to score independently children’s responses in all the questions. The judges’ agreement was calculated at 95% [Kendall’s tau τ=0.939; Ν=81; p<.001]. All disagreements were discussed until a common score had been achieved on all the items. Children’s responses to each of the 14 questions received a score of 3 if they were scientific, of 1 if they were Figure 1: Mean Scores of Children’s Responses in all “initial” (if they were consistent with the flat Earth model), Common Questions as a Function of Text Type 3 80 Post hoc comparisons using the LSD test indicated that the Table 4: Mean score and Mean Score Difference in Children’s mean score difference from pretest to posttest for Text 3 was Responses in the Common Inferential Questions in the Pretest statistically significant only when Text 3 was compared to and the Posttest as a Function of Text Type Text 1. The mean score difference between Text 2 and Text 3 was in the expected direction but not statistically different (see Text Mean Std. Error Mean Difference Table 2). Similarly, the mean score difference between Text 1 1 Pretest 8.650 .366 .050 and Text 2 was not statistically different. Taken together, these Posttest 8.700 .345 results suggest that Text 3, improved children’s responses in 2 Pretest 9.314 .277 .229 the posttest more than the other two texts, but the difference Posttest 9.543 .261 was statistically significant only between Text 1 and Text 3.. 3 Pretest 9.885 .321 .846 Posttest 10.731 .303 Table 2: Mean Score Difference in Children’s Responses in all the Common Questions Using the LSD post-hoc Analysis Significant interactions were obtained only in the case of the inferential questions [F(2,78)=6.410; p<.005]. The η2 effect Texts Mean Difference Std. Error Significance size value for this analysis [η2=.141] was high. Figure 2 is a 1 2 -0.629 .662 n.s. pictorial representation of these results. 3 -1.012* .703 p<.05 2 1 0.629 .662 n.s. 3 -0.383 .612 n.s. 3 1 1.012* .703 p<.05 2 0.383 .662 n.s. A similar mixed ANOVA was conducted separately for the common explicit and the common inferential questions only. In the case of the common explicit questions the analysis showed statistically significant improvement in children’s responses from pretest to posttest in all three texts [F(1,78)=20.058; p<.001; η2=0.205]. No statistical significant interactions were obtained [F(2,78)=.637; n.s.; η2=0.006], despite the fact that Text 3 had a greater effect in children’s Figure 2: Mean Scores of Children’s Responses in the responses in the posttest compared to the other two texts ( see Common Inferential Questions as a Function of Refutation Table 3). Text Type Table 3: Mean score and Mean Score Difference in Children’s It appears that Texts 1 and 2 slightly improved children’s Responses in the Common Explicit Questions in the Pretest responses in the posttest, while Text 3 had a greater impact on and the Posttest as a Function of Text Type children’s posttetst explanations compared to the other two texts. Additional post-hoc analyses using the LSD test showed Text Mean Std. Error Mean Difference that this difference was in favor of Text 3 compared to the 1 Pretest 10.300 .383 .450 other two texts (Table 5). Posttest 10.750 .324 2 Pretest 10.143 .290 .514 Table 5: Mean difference of children’s responses in the Posttest 10.657 .245 inferential questions in the posttest using the LSD post-hoc 3 Pretest 9.500 .336 1.154 analysis Posttest 10.654 .284 Texts Mean Difference Std. Error Significance 1 2 -0.754 .431 n.s. In the case of the common inferential questions a 3 -1.633* .457 p<.001 statistically significant improvement in children’s responses 2 1 -0.754 .431 n.s. from pretest to posttest in all three texts was obtained 3 -0.879* .398 p<.05 [F(1,78)=20.058; p<.001; η2=0.205] (see Table 4). 3 1 1.633* .457 p<.001 2 .879 .398 p<.05 4 81 It seems that, once again Text 3, which included the that   caution   needs   to   be   exercised   as   to   how   categorical   categorical information, improved children’s responses in the information  can  be  best  included  in  science  texts,  particularly   inferential questions more than the other two texts. in   situations   where   there   are   reasons   to   believe   that   the   scientific   category   does   not   already   exist   in   the   conceptual   repertoire  of  the  student.       Discussion   Science texts are difficult to understand, especially for young elementary school children (Goldman & Bisanz, 2002; References Graesser, Leon, & Otero, 2002; Vosniadou & Skopeliti, in Broughton, S. H., & Sinatra, G. M. (2010). Text in the science preparation). Refutation texts that take into consideration classroom: Promoting engagement to facilitate conceptual children’s incompatible prior knowledge and possible change. In M. G. McKeown & L. Kucan (Eds.), Essays in misconceptions and explicitly refute them are a good means to honor of Isabel Beck (pp. 232-256). New York, NY: promote text understanding in these cases (Dole, 2000; Guilford Press. Diakidoy, Kendeou, & Ioannides, 2003; Guzzeti, et al., 1997; Carey, S. (1985). Conceptual Change in Childhood. Mikkila-Erdmann, 2002; O’Reilly & McNamara, 2007). The Cambridge, MA: Bradford Books, MIT Press. results of the present study are consistent with the results of Chi, M.T.H. (2013). Two kinds and four sub-types of previous research in view of the fact all the refutation texts misconceived knowledge, ways to change it and the improved children’s responses in the posttest. learning outcomes. To appear in International Handbook In addition, the results of the present study indicate that the of Research on Conceptual Change (2nd edition). (pp.49- refutation text that included the categorical information was 70). New York: Routledge. more effective than the other texts in improving children's Chi, M.T.H., & Roscoe, R.D. (2002). The processes and understanding of the scientific information about the Earth. challenges of conceptual change. In M. Limon and L. The children who read Text 3, which included the information Mason (Eds.), Reconsidering Conceptual Change: Issues about the categorization of the Earth, improved their responses in Theory and Practice. Kluwer Academic Publishers, The in the inferential questions in the posttest more compared to Netherlands, pp 3-27. their peers who read the other two texts. This finding is Diakidoy, I.-A.N., Kendeou P., & Ioannides C., (2003). consistent with our hypothesis that categorical information is Reading about energy: The effects of text structure in more important than other types of information, because it is science learning and conceptual change. Contemporary generic and carries with it a great deal of implicit information Educational Psychology, 28(3), pp. 335-356. that can be used by the learner to guide new learning Dole, J.A. (2000). Readers, Texts, and Coceptual Change (Vosniadou & Skopeliti, 2005). However, since Text 3 also Learning. Reading & Writing Quarterly, 16, pp. 99-118. included more information than the other two Texts, more Goldman, S.R. & Bisanz, G.L. (2002). Toward functional experiments are need in order to further test the power of analysis of scientific genres: Implications for presenting categorical information alone. understanding and learning processes. In J. Otero, J. A. Overall, the present results support the conclusion that Len & A.C. Graesser (Eds.) The Psychology of Science when using refutation text, it is important to consider what Text Comprehension. Mahwah, NJ: Erlbaum. kind of information is being refuted as well as how much Graesser, A., Leon, J., & Otero, J. (2002). Introduction to the information is refuted. It appears that texts are more effective psychology of science text comprehension. In J. Otero, J. when more information is refuted as well as when the A. Len & A.C. Graesser (Eds.) The Psychology of Science refutation includes categorical information (Skopeliti & Text Comprehension. Mahwah, NJ: Erlbaum. Vosniadou, 2006, 2009). This finding has important practical Guzzetti, B.J., Williams, W.O., Skeels, S.A., & Wu, S.M., implications and needs to be considered by curriculum (1997). Influence of text structure on learning designers, authors of science text, and of course teachers. counterintuitive physics concepts. Journal of Research in Learning science very often requires changes in the Science Teaching, 34, 701-719. ontological category to which an object belongs.   These   Hayes, B.K., Goodhew, A., Heit, E., & Gillan, J. (2003). The changes   are   considered   to   be   the   most   difficult   conceptual   role of diverse instruction in conceptual change. Journal of changes   to   achieve   (Chi, 2013; Chi & Roscoe, 2002; Thagard, Experimental Child Psychology, 86, pp. 253-276. 2013; Vosniadou   &   Skopeliti,   2013).   In   the   case   of   the   Earth   Kendeou, P., & van den Broek, P. (2005). The effects of concept  it  was  not  too  difficult  to  add  the  information  that  the   readers’ misconceptions on comprehension of scientific Earth   is   an   astronomical   object   in   the   present   texts,   because   text. Journal of Educational Psychology, 97, pp. 235-245. the   children   already   had   formed   a   category   of   astronomical   Kendeou, P., & van den Broek, P. (2005). The effects of objects.   However,   it   might   be   more   difficult   to   refer   to   the   readers’ misconceptions on comprehension of scientific scientific   category   when   this   category   does   not   exist   in   the   text. Journal of Educational Psychology, 97, pp. 235-245. conceptual   repertoire   of   the   student.   For   example,   reference   Kendeou, P., & van den Broek, P. (2007). Interactions to   the   category   ‘interactions’   for   concepts   like   force,   or   heat,   between prior knowledge and text structure during which  are  initially  categorized as  substances  or  properties  of   comprehension of scientific texts. Memory & Cognition, objects  might  not  carry  much  meaning  or  explanatory  power   35, pp. 1567-1577. for  a  student.  Taking  this  into  consideration,  it  has  to  be  clear   5 82 Medin, D. L. & Rips, L. J. (2005). Concepts and Categories: Society, (2325-2330), Mahwah, NJ: Lawrence Erlbaum Memory, meaning, and metaphysics. In K. J. Holyoak & Associates. R. G. Morrison (Eds.), Cambridge handbook of thinking Vosniadou, S., & Skopeliti, I. (2013). Conceptual change and reasoning (pp. 37-72). Cambridge: Cambridge from the framework theory side of the fence. Science University Press. & Education, DOI 10.1007/s11191-013-9640-3. Mikkilä-Erdmann, M. (2002). Science learning through text: The effect of text design and text comprehension skills on conceptual change. In M. Limón, & L. Mason (Eds.) (2002). Reconsidering conceptual change. Issues in theory and practice (pp. 337-356). Dordrecht, The Netherlands: Kluwer Academic Publishers. Nussbaum, J. (1985). The earth as a cosmic body. In R. Driver, E. Guesne, & A. Tiberghien (Eds.), Children’s ideas in science (170-192). Milton Keynes: Open University Press. O'Reilly, T., & McNamara, D. S. (2007). The impact of science knowledge, reading skill, and reading strategy knowledge on more traditional “high stakes” measures of high school students’ science achievement. American Educational Research Journal, 44, 161-196. Skopeliti, I., & Vosniadou, S. (2006). The Influence of Refutational Texts on Children’s Ideas about the Earth. In R. Sun & N. Miyake (Eds.) Proceedings of the 28th Annual Conference of the Cognitive Science Society (pp. 2608), Vancouver, Canada. Skopeliti, I. & Vosniadou, S. (2009, April) The Influence of Categorical Information in Refutational Texts. Paper presented in the G. M. Sinatra (Chair) Symposium “Mechanisms for Facilitating Conceptual Change Through Text-Based Interventions” at the AERA Annual Meeting 2009, San Diego, California. Thagard, P. (2013). Conceptual change in the history of science: Life, mind, and disease. In S. Vosniadou (Ed.), International handbook of research on conceptual change (pp. 374-387). New York: Routledge. Vosniadou, S. (2013). Conceptual change in learning and instruction: The framework theory approach. In S. Vosniadou (Ed.), The international handbook of conceptual change (2nd ed., pp. 11–30). New York: Routledge. Vosniadou, S. & Brewer, W.F., (1992). Mental models of the earth: A study of conceptual change in childhood. Cognitive Psychology, 24, 535-585. Vosniadou, S. & Brewer, W., (1994). Mental models of the day/night cycle. Cognitive Science, 18, pp. 123-183. Vosniadou, S., Ioannides, C., Dimitrakopoulou, A., & Papademetriou, E. (2001). Designing learning environments to promote conceptual change in science. Learning and Instruction, 11, pp. 381-419. Vosniadou, S. & Skopeliti, I. (in preparation). Readers can form misconceptions from reading science text similar to those revealed in science education and developmental studies. Vosniadou, S. & Skopeliti, I., (2005). Developmental Shifts in Children’s Categorization of the Earth. Στο B. G. Bara, L. Barsalou, & M. Bucciarelli (Eds.) Proceedings of the XXVII Annual Conference of the Cognitive Science 6 83