Evaluation Needs of Webble Technologies in an E-Learning Laboratory Case Study

Abstract

Webble Technology is an advanced current form of Meme Media on the Web. The authors use Webbles for the implementation of Web-based interactive laboratories. There arises a particular question for the perception of added values which result from peculiarities of Meme Media technologies. This should not be confused with the question for the laboratories' usability. The focus of the present investigations is on the perception and, perhaps, appreciation of implementing ideas of memetics by different groups of users.

Introduction

When new technologies enter an application domain, technology providers are always assuming the invention’s beauty and success. However, the proof of the pudding is the eating of the pudding.

The authors of the present paper are enganged in a comprehensive endeavor of introducing Meme Media technologies [ 1 ] into Web-based applications aiming at learning support at school. The technology of choice is “Webble”.

The acronym Webble [ http://www.meme.hokudai.ac.jp/WebbleWorldPortal/ ] abbreviates Web Pebbles, where “Pebble” is short for “Pad Enhanced Building Block Lifelike Entity”. In this descriptional phrase, the term “Pad” again is short for “IntelligentPad” according to [ 1 ].

One might ruoghly understand IntelligentPad as a middleware having a number of quite desirable features. Those who are more ambitious understand the IntelligentPad approach as a way to implement Memelike building blocks intended to enable knowledge representation and evolution. The concept “Meme”, being intended to resemble the words memory and gene, was coined by Richard Dawkins [ 2 ]. Yuzuru Tanaka took up the challenge to carry over Richard Dawkins’ Fig. 1. Screenshot of the Webble-Based Solar Biker ideas to software technology [ 1 ]. Susan Blackmore is providing a general perspective at the reach of those ideas and approaches [ 3 ].

The authors are using Webble technologies for the implementation of a series of interactive laboratories of which the so-called Solar Biker is a prototypical example. The key question is to what extent the quality of the new technology is accepted.

The Solar Biker Laboratory

The authors’ e-learning Solar Biker project has been inspired by some real toy kit originally developed for educational purposes by Peter Thron et al. in Ilmenau, Germany, but nowadays available as a commercial product [ 4 ]. The authors’ project has no immediate commercial goals and, thus, does not interfere with the source of inspiration.

Figure 1 is intended to give an impression of the current state of implementation.

The Solar Biker Laboratory is an interactive playground on the Internet. The authors are aiming at a series of similar Web laboratories providing useful content for the playful acquisition of knowledge at school. Questions concerning access to and administration of those laboratories are beyond the limits of the present paper. The focus of the present investigations is on the evaluation of the impact of the novel technology.

For this purpose, this chapter is providing a sketch of what is in the Webble technology seen from the perspective of the Solar Biker educational application.

In particular applications such as the present one, the Webble technology is providing a repository of building blocks as illustrated in figure 2. The building blocks are pads.

Every pad has its model view controler architecture.

What a pad does and how it looks is defined internally in so-called slots. By setting a slot value, you can modify both the appearance and the functionality of a pad. This includes the position of a pad.

From the user’s point of view, pads may be used very much like Lego building blocks. You take them and plug one on top of the other. In the example shown in figure 1, the sun is plugged on top of the environment.

Plugging pads together establishes connections between particular slots of these pads. The data flow between slots allows for a coordinated functionality of the individual components of a composite pad.

For illustration, the solar biker on display in figure 1 will be driven by some energy provided by the solar cell in the left lower corner. This means that the solar cell is providing some slot value to the moving pads of the solar biker. When a cloud is plugged onto the environment pad, this cloud pad is sending some slot value to the environment pad. This value is used to modify, in fact to diminish, the value of the energy slot delivered by the solar cell pad. How much the cloud pad is dimming the solar cell energy value depends on the position Fig. 2. Repository of the cloud pad on the environment pad. In such a way, direct manipulation of the pads on the playground dtermines the behavior of the composite pad under construction.

In an expert mode, all details are accessible to the human manipulating pads. One may freely choose pads, modify them and plug them together with other pads. The result is always a composite pad the structure of which may be seen as a tree.

For educational purposes, the freedom of access has to be controled according to the knowledge and skills of the learners. Didactic intentions such as a certain degree of exploration in learning may be a further source of access regulation.

Technology may be modified to support playful learning and construction. One of the key features is automatic plug-in. When one pad is moved over another pad which it fits, the right slots are connected automatically. The whole construction process is just a playful drag and drop.

The Meme Media Peculiarities

The Meme concept has been introduced and the term has been coined with high ideological ambitions [ 2, 3 ]. There is–roughly speaking–the idea of non-biological evolution as observable in areas such as fashion, architecture and, perhaps, religion.

Yuzuru Tanaka has been excited by the idea to foster evolution of ideas by means of information and communication technologies [ 1 ]. In response to the challenge, he took the initiative to develop and implement digital meme concepts. Tanaka coined the term Meme Media and developed what was then called Meme Media technologies in its prominent variants of IntelligentPad and IntelligentBox.

Webble technologies [ 5 ] are a modern Internet-enabled version of IntelligentPad. The way from conventional IntelligentPad to current Webbles may be studied on the basis of a few representative IntelligentPad publications such as [ 6 ], [ 7 ], [ 8 ], [ 9 ], [ 10 ], [ 11 ], and [ 12 ].

IntelligentPad, in general, and Webble technology, in particular, is more than just another middleware for the implementation of Web services. A short illustration from the Solar Biker application shall clarify the pecularities under consideration.

Solar Biker Lab Evaluation

As said above, the focus of the present investigations is on the evaluation of the impact of the novel technology.

There will be usability tests and the like as usual. But these questions are not within the focus of the present paper. Instead, the authors want to investigate to what extent and with what impact users are perceiving the peculiarities of the Webble technology.

There is a need to distinguish two basic categories of users. First, there are those setting up e-learning contents for the intended learner audience. Second, there are the learners by themselves. 4.1

The Media and Technology Perception by Authors

Those who set up content of interactive laboratories such as the Solar Biker may be teachers, tutors, content providers or software specialists. Due to the mediatization of our contemporary society [ 13 ], their affinity to information and communication technologies is continuously changing. Nevertheless, we still have to assume a low familiarity with digital media technology among teachers.

The authors do not expect to have access to a larger community of authors. Thus, they are planning for systematic qualitative investigations [ 14 ] of way in which authors cope with the novel potentials of the technology. 4.2

The Media and Technology Perception by Learners

In contrast, learners are–on the average–persons who are, at least to some extent, familiar with information and communication technologies. They have to be kept from dealing with too many technological detail in addition to the main learning task and from being bothered with technicalities of handling the software. Learners playing with the digital Solar Biker toy kit shall learn about solar energy and photo-voltaic technology. The should be encouraged to play with the system and to explore, for illustration, the effects of putting more clouds to the sky.

Both for quantitative and qualitative approaches [ 15 ], there is a large variety of settings.

Shall we compare two groups of learners where the one group has the physical toy kit [ 4 ] available whereas the other one uses the virtual laboratory over the Internet? What are the details we are looking at and asking for? 4.3

Survey of the Evaluation Approach

To avoid confusions and inappropriate expectations, the authors are not going to present evaluation results within this paper. Instead, the authors are planning (i) to present the Solar Biker laboratory under consideration and then, based on the DERIS audience’s impression of and knowledge about the system, (ii) to discuss their intended evaluation approach. The authors’ hope is for guidance and advice.

In summary, we need to investigate the acceptance of the Solar Biker with regard to the two main target groups: teachers and learners. Some acceptance factors will probably affect both of the groups–e.g. if students learn something about solar technology when using the Solar Biker. However, in some aspects the expectations towards the Solar Biker may differ–maybe it is less important for the teachers, that their students have fun while using the Solar Biker. Therefore two main research questions are outlined, which take the different acceptance factors of the two groups into account. These questions will lead us through the whole evaluation process. The main questions are completed through some more focussed questions.

Research question for the group of learners

– How do young people between 12 and 16 years with different education levels experience the Solar Biker as playful learning tool? – Which learning methods and scenarios encourage the usage of the solar biker, e.g. how should the introduction be like? What exactly can be learned with the Solar Biker? What are the entertaining factors of the Solar Biker? Learning about solar technology includes some factual knowledge, e.g. to know which components are necessary to build up a Solar Biker, skills, e.g. like being able to put the components together, as well as motivational aspects, e.g. if students would like to know more about solar technology. Two learning scenarios will be tested: guided learning [ 16 ] and discovery learning [ 17 ]. A guided learning scenario is defined by instructions of a teacher, e.g. ‘Please build up a Solar Biker, and use the environment pad first.’ The discovery learning scenario is also characterised by an instruction, but a more open one, e.g. ‘You have 20 minutes time to use the Solar Biker as you like’. Entertaining factors will be operationalized as feelings of control, emotional pleasantness (positive feelings like fun) and sovereignty [ 18 ]. We will focus on the students’ age between 12-16 years. In Germany it can be expected that students from the seventh grade onwards will have basic knowledge about chemical and physical foundations of solar technology. Solar technology is not a regular topic in school until the tenth grade.

Research question for the group of teachers

– How do school teachers experience the Solar Biker concerning the learning process of their students and the potential of the system to encourage teaching? – In which learning scenarios can the Solar Biker be used? Which target groups can be reached? What can be learned with the Solar Biker?

In this target group we will focus on the same learning methods, namely guided learning and discovery learning as two possible use cases. We would like to know from the teachers which age the students need to have to deal with the Solar Biker appropriately and how they assess the learning effect. 4.4

Research Design

We would like to continue enhancing the system of the Solar Biker. To be able to influence the design of the Solar Biker constantly by the results of the evaluation, we choose a formative evaluation as our basic research design [ 19, 15 ]. The first step will be to ensure that the usability of the system is given and to explore for which age the Solar Biker is most appropriate. As a second step we will conduct a qualitative group discussion with teachers from schools with different education levels, ranging from high school level (’Gymnasium’) to secondary modern school (’Hauptschule’), for which school system the Solar Biker is an appropriate learning tool and in which class levels it can be used best. The results of the group discussion will be integrated in the third step of the evaluation process, a quasi-experimental setting. Hereby it will be compared how the Solar Biker is used in two contrasting education levels. The two different learning methods will be compared within that setting as well. The main focus will be on the perception of the Solar Biker as a new possibility to deal with digital content. Additionally the help function will be evaluated as well as possibilities of collaborative learning (e.g. to send a running biker per mail to a friend). As a last step the results of the experimental study will be discussed again with the teachers in a group discussion to find out how teachers assess the learning progress of their students.

Usability Test A good usability means that the interaction between the user and the computer is experienced as efficient, effective and satisfying. The concept focuses how targets and tasks can be fulfilled and solved through the use of the system. Hence a usability test will detect the main problems within the user-systeminteraction as a basis of our evaluation [ 20 ]. a) The first usability test will be conducted with a sample of students ranging from 12-16 years. There should be a good balance between students with a very high computer literacy and students with a very low computer literacy. Because the use of the Solar Biker is maybe influenced by knowledge about solar technology and skills in subjects like physics and chemistry, we will compare students with a higher and a lower education level. b) The other usability test with the teachers from natural and technical sciences will also take the different level of computer literacy into account. In that stage of the evaluation it is necessary to consider the teachers’ subjects, because the usability is maybe, as said before, influenced by the knowledge about solar technology, nonetheless the Solar Biker is an interdisciplinary object.

The participants will get the general task to complete the Solar Biker, but there will be also some special tasks to fulfil, to outline the weak spots, e.g. what happens, if somebody build up a completely wrong constellation and how the help system can assist in these situations. The empirical method used here is an observation, which will use both structured and open criteria to describe the behaviour of the participants. The test will end with a very short interview including some open questions to detect weak spots which the participants would like to remark. Here, too, we will ask in detail how the help function is perceived–a good result would be, if the participants receive the help function as a very individual helping hand and personal assistance.

The concept of usability is not able to explain the usage and experience of the system in general, e.g. if it is fun to use the Solar Biker and if the uniqueness of the technology is experienced as an added value. These aspects will be clarified through the next steps of the evaluation. But before the results of the usability test will end in a revise of the Solar Biker.

Group Discussion I The participants in the group discussion [ 21 ] will range from the lower to the higher education levels. The teachers will get an introduction into the Solar Biker system. Afterwards they will be asked for which age and class level the Solar Biker is useful and if they think that they would use it in their schools and why. Furthermore the innovative potential of the Solar Biker in particular and interactive laboratories modelled with Webble technology in general will be discussed.

Quasi-Experimental Setting The quasi-experimental setting [ 19 ] will be conducted in two schools with a different education level. Depending on the results of the group discussion two different class levels will also be integrated. The effort of doing research in three different school levels would probably not be appropriate, but this decision does very much depend on the results of the group discussion with the teachers. For the moment we assume surveying and comparing two different school systems. The sample will take two classes out of these two different school levels, so that we will have a sample of four classes with approximately 80 students. These four classes will be surveyed, two out of every school level, from which one will use the guided learning scenario, the other one the discovery learning approach.

To use the Solar Biker within a guided learning scenario will be operationalized by using the Solar Biker in a computer lap during a school lesson. The other class will use the Solar Biker as part of homework, without a specific task but a minimum of time to spend. The teachers will be provided with a standardised introduction to the topic ‘solar technology’ (e.g. a movie) and a standardised instruction. Hereby the influence of the teacher on the learning result will be reduced.

Before the quasi-experimental study there is a high need to conduct qualitative interviews as pre-test to clarify some key dimensions of the usage, e.g. what specific factors influence the acceptance of the Solar Biker (e.g. ‘What do you like about the Solar Biker?’) and how students evaluate the unique character of the system. Therefore some scenarios will be given, like ‘Imagine you could send the biker as mail to your friends; would you do so? Why?’ or ‘Imagine you could put the biker on different countries on a map, e.g. Australia and Sweden; would that be interesting?’

Students will get a standardised questionnaire before they use the solar biker, e.g. to ask for the different computer literacy levels and their general usage of interactive media. The knowledge about solar technology and the interest in chemistry, physics etc will be asked as well, finalising the first questionnaire with some socioeconomical variables, e.g. sex. The second questionnaire will be given to the students after they used the system, contending closed and open questions. Within that questionnaire items concerning the use of the solar biker as a playful learning tool will be in the focus of interest, which means to integrated dimensions of knowledge (e.g. ‘Which components are needed for a solar biker?’), emotional (e.g. ‘I don’t like to use the internet for learning purposes’) and cognitive (e.g. ‘The internet offers good possibilities to learn’) attitudes, emotional (e.g. ‘To handle the solar biker is pleasant to me’) and cognitive (e.g. ‘The solar biker explains solar technology in an understandable way’) opinions and intentions (e.g. ‘I would like to use applications like the Solar Biker in school more often’).

The main focus within these dimensions will be on the perception of the Solar Biker as a unique possibility to deal with digital content, therefore it will be asked, as how unique, new and novel it is valued (by means of an semantic differential) and which features are liked the most, e.g. to send objects per mail, to get individual help, to explore the possibilities how to stick components together etc. Group Discussion II As a finishing method again a group discussion with the teachers is planned, so that the teachers can assess the learning process and the usage of the Solar Biker in school. At least it will be discussed, which benefits are perceived (e.g. to collaborate with their students, to monitor their learning progress, that students get individual assistance). Therefore a scenario will be outlined to extend the discussion towards the possibility having a complete interactive laboratory, so that the direction of further software development will integrate the perspective as teachers and learners at one time.

Outlook An additional evaluation could ask for the different perception and usage of a ‘real-life’ Solar Biker in comparison to the digital one. The comparative study would have to look carefully for possibilities to combine the use of ‘real-life’ tool kits and digital ones in school. We decided to conduct the basic evaluation first, because we need to know about the best learning scenarios for the solar biker before comparing it to a well established learning method. 5

Acknowledgements

This first version of this manuscript has been submitted when the first and the third author have been visiting the Meme Media Lab. at the Hokkaido University Sapporo, Japan, the place where the technology under consideration has its origins.

The authors are particularly greatful to Yuzuru Tanaka for encouraging their publication activities and to Micke Kuwahara for providing Webble World [ 5 ].

The present research and development has been partially supported by the Thuringian Ministry for Culture within the project iCycle under contract PE-004-2-1. This article was processed using the LATEX macro package with LLNCS style

References

1. Tanaka , Y. : Meme Media and Meme Market Architectures: Knowledge Media for Editing, Distributing, and Managing Intellectual Resources . IEEE Press & WileyInterscience ( 2003 )

2. Dawkins , R. : The Selfish Gene . Oxford University Press ( 1976 )

3. Blackmore , S.: The Meme Machine . Oxford University Press ( 1999 )

4. Woll Solar Systeme . http://shop.solaranlage.com/shop/solarbiker-p- 267 .html [latest access on Nov. 9 , 2009 ] ( 2009 )

5. Kuwahara , M.: http://www.meme.hokudai.ac.jp/webbleworldportal/ [last access on Sept. 20 , 2009 ] ( 2009 )

6. Tanaka , Y. , Imataki , T.: IntelligentPad: A hypermedia system allowing functional compositions of active media objects through direct manipulations . In: Proceedings of the IFIP 11th World Computer Congress . ( 1989 ) 541 - 546

7. Tanaka , Y. : Intelligentpad as meme media and its application to multimedia databases . Information & Software Technology 38 ( 1996 ) 201 - 211

8. Tanaka , Y. : Meme media and a world-wide meme pool . In: MULTIMEDIA '96: Proceedings of the fourth ACM international conference on Multimedia , New York, NY, USA, ACM ( 1996 ) 175 - 186

9. Tanaka , Y. , Fujima , J. , Sugibuchi , T. : Meme media and meme pools for re-editing and redistributing intellectual assets . In: Revised Papers from the nternational Workshops OHS-7, SC-3 , and AH-3 on Hypermedia: Openness, Structural Awareness , and Adaptivity, London, UK, Springer-Verlag ( 2002 ) 28 - 46

10. Ito , K. , Tanaka , Y. : A visual environment for dynamic web application composition . In: HYPERTEXT '03: Proceedings of the fourteenth ACM conference on Hypertext and hypermedia , New York, NY, USA, ACM ( 2003 ) 184 - 193

11. Tanaka , Y. , Ito , K. , Fujima , J.: Meme media for clipping and combining web resources . World Wide Web 9 ( 2006 ) 117 - 142

12. Tanaka , Y. , Fujima , J. , Kuwahara , M. : Meme media and knowledge federation . In: KI '08: Proceedings of the 31st annual German conference on Advances in Artificial Intelligence , Berlin, Heidelberg, Springer-Verlag ( 2008 ) 2 - 21

13. Krotz , F. : Mediatisierung: Fallstudien zum Wandel von Kommunikation . VS ( 2009 )

14. Kuckartz , U. : Qualitative Datenanalyse: computergestu¨tzt. Methodische Hintergru¨nde und Beispiele aus der Forschungspraxis . VS Verlag ( 2007 )

15. Tisdal , K.M. , B. , D. J., Gallagher , M. : Researching with Children and Young People: Research Design, Methods and Analysis . SAGE Publications Ltd . ( 2009 )

16. Kirschner , P.A.S.J.u.C.R.E. : Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching . Educational Psychologist 41 ( 2006 ) 75 - 86

17. Loyens , S.M.M. , Gijbels , D. : Understanding the effects of constructivist learning environments: Introducing a multi-directional approach . Instructional Science 36 ( 2008 ) 351 - 357

18. Fru¨h, W., Schulze , A.K. , Wu¨nsch, C.: Unterhaltung durch das Fernsehen: Eine molare Theorie . UVK Verl .-Ges., Konstanz ( 2002 )

19. Bortz , J. , Do¨ring, N.: Forschungsmethoden und Evaluation: Fu¨r Human- und Sozialwissenschaftler; mit 87 Tabellen. 4 ., u¨berarb. aufl., nachdr. edn. Springer-MedizinVerl., Heidelberg ( 2009 )

20. Rubin , J.: Handbook of usability testing: How to plan, design, and conduct effective tests . Wiley, New York ( 1994 )

21. Lamnek , S. : Gruppendiskussion: Theorie und Praxis. 2 ., u¨berarb. u. erw. aufl. edn. UTB, Stuttgart ( 2005 )