=Paper=
{{Paper
|id=None
|storemode=property
|title=Educational Resources Packaging Standards SCORM and IMS Common Cartridge - The Users Point of View
|pdfUrl=https://ceur-ws.org/Vol-681/paper11.pdf
|volume=Vol-681
}}
==Educational Resources Packaging Standards SCORM and IMS Common Cartridge - The Users Point of View==
https://ceur-ws.org/Vol-681/paper11.pdf
Educational Resources Packaging Standards SCORM
and IMS Common Cartridge – The Users Point of View
1
Kati Clements, 2Àgueda Gras-Velázquez, 1Jan M. Pawlowski1
1
Global Information Systems, University of Jyväskylä, 40014 Jyväskylä Finland,
2
European Schoolnet, Rue de Lalaing 24, 1040 Brussels, Belgium
kati.clements@jyu.fi, agueda.gras@eun.org, jan.pawlowski@jyu.fi
Abstract. This paper reports on an exploratory study analyzing the Educational
Resources’ packaging standards SCORM and IMS Common Cartridge’s (CC)
regarding interoperability from the point of view of key users: teachers. The
two specifications SCORM and CC to package Educational Resources have
been developed to help the users to re-use Learning Objects from Learning
Object Repositories (LORs) in Learning Management Systems (LMS) of
schools. In our study, we found that teachers find packaging solutions highly
useful, especially the interoperability between LORs and LMSs. Teachers also
appreciated that they can modify packaged content after it has been uploaded to
the LMS. The teachers also strongly appreciated the additional functionalities
of CC packages while teaching courses online or giving home work/extra
assignments to their students.
Keywords: Educational Resources, content packaging standards, Learning
Object Repositories, SCORM, IMS Common Cartridge, teachers
1 Introduction
This paper shows the views of teachers on interoperability between Learning Object
Repositories (LOR) and Learning Management Systems (LMS) supported by two
content packaging standards: SCORM and IMS Common Cartridge (CC). We set up a
testing experiment and survey to find out 1) whether teachers would get any
additional value from using these standards and 2) whether tools used to support these
standards are reasonable for teachers to use in their everyday teaching.
According to The IEEE Learning Technology Standards Committee, a Learning
Object is defined as "any entity, digital or non-digital, that may be used for learning,
education or training" [1]. Learning Objects in this paper are defined as digital objects
which can be used, re-used or referenced during a technology-supported learning
process [2]. Educational Resources can be defined as Learning Objects that can be
employed in technology-supported learning [3]. Learning Object Repositories are
collections of Learning Objects that are accessible to users via a network without
prior knowledge of the structure of the collections [4]. Distribution of Educational
Resources is most commonly done via LORs. This study investigates the views of
teachers re-using and sharing Learning Objects from LORs with the help of
interoperability standards SCORM and IMS Common Cartridge. In their everyday
79
lesson preparation, teachers use Educational Resources in LMS such as Moodle [5].
LMS help teachers to manage digital contents, to aggregate blended learning and give
homework to their students.
Recent research involving packaging standards SCORM and IMS Common
Cartridge has been focusing on the technical specifications [6],[7],[8], without
investigating the teachers’ point of view. The assumed benefits of SCORM and CC
have not been verified by the end user group, leading to the questions of this study:
Are the teachers willing to use SCORM and CC packages in their busy schedule of
designing Educational Resources for their classes. Is the use of content packaging yet
another technical problem which the teachers will not be able to handle with a
reasonable amount of training?
This study aims to examine what the real users, the teachers, think of using these
Learning Object packaging standards and ask if they can see the interoperability
between LOR and LMS facilitating the work they are doing in their everyday lesson
plan creation process.
When trying to evaluate the impact of a learning technology standard to the users, it
is important to realize that specifications cannot be evaluated by users as users do not
work with them directly. Rather, interoperability specifications are implemented in
software tools offering a set of functionalities to the end user. End users can then
make use of the tool in practice and that use can be evaluated. The evaluation results
need to be analyzed in detail to assess whether problems are caused by the
interoperability specifications or by the functionality provided by the tool, or the user
interface through which the functionality is made available to the end user. [9]
However, it can be evaluated whether the users have understood and used the key
concepts of a specification (such as packaging or metadata categories). To avoid this
problem, we look at the key concepts and functions enabled by the standards. As
SCORM and CC are both standards which are widely implemented in tools, we can
map the standards’ concepts and the resulting functionalities in the tools, which are
then assessed in practical experiments. Therefore we tested the standards by using the
tools keeping in mind that the interfaces of these players could affect the minds of the
users.
2 Open Educational Resources Packaging Standards
In the following, we discuss the content packaging specifications SCORM (Sharable
Content Object Reference Model) and IMS Common Cartridge (CC).
2.1. SCORM
The Sharable Content Object Reference Model (SCORM) was created to help the re-
usability, interoperability, portability, access, maintenance and adaptation of Learning
Objects. [10] SCORM is a collection of standards and specifications that enable
learning platforms to find, import and deliver learning content in a standardized way.
SCORM specifies how Learning Objects must be created in order to ensure
interoperability across different platforms and tools. [7]
80
SCORM was originally intended for use in self-study, computer-based training
scenarios rather than in interactive scenarios between teachers and students.
SCORM’s crucial functionality ‘sequencing’ supports the learner by allowing him/her
only to navigate into parts of the package based on the previous learning assignments
fulfilled. However, it can be said that SCORM was created to help teachers when
transferring collections of Learning Objects, typically from an LOR to an LMS.
To date, most popular Learning Management Systems support SCORM objects [11]
among these the system of Moodle. SCORM packages can be uploaded as single,
unmodifiable entries to Moodle.
2.2. IMS Common Cartridge
IMS Common Cartridge (CC) was supposed to enhance SCORM, offering more
flexibility and support for assessments, web 2.0 standards, content authorization,
collaborative forums and outcomes reporting [7]. The aim of IMS Common Cartridge
was not to compete with SCORM but to support to blended learning rather than self
learning. According to IMS [12] the IMS Common Cartridge specification’s
objectives for facilitating teaching include increasing flexibility, sharing and re-use.
CC enables teachers to assemble lesson plans taking only parts of an CC package or
integrating the whole package into their LMSs. This study aimed to find out how
teachers saw the difference of interoperability when using IMS Common Cartridge
packages integrated into a Moodle LMS.
IMS Common Cartridge is not as widely supported by different LMSs as SCORM
perhaps because there are not yet as many tools to support it. However, LMS
Platforms have gained growing interest towards IMS Common Cartridge support. It is
still gathering momentum, which should increase by the announcement [13] from the
popular open source platform Moodle to start supporting IMS Common Cartridge
packages in the spring of 2010 on. Users can upload CC packages as a whole into the
Moodle system much in the same way as uploading SCORM packages.
3 Test Setting
Adopting Standards and Specifications for Educational Content (ASPECT) is a Best
Practice Network for educational content that aims at improving the adoption of
learning technology standards and specifications [14]. Standards and interoperability
experts produce recommendations that are implemented by tools and content
providers before being tested by teachers during school pilots. Tests were carried out
in order to demonstrate in which way the implementation of Standards and
Specifications leads to greater interoperability and cross-border re-use of content [15].
The Learning Resource Exchange (LRE) is a pan-European federation of Learning
Object Repositories [16]. The service is offered to stakeholders providing digital
content, such as ministries of education, commercial publishers, broadcasters, cultural
institutions, and other non-profit organizations offering online content to schools [17].
The LRE was used as the testing LOR in the ASPECT project and it provided the
possibility to get the same package available in all the different formats (web page,
81
SCORM package, CC package, SCORM in Icodeon player and CC in Icodeon
platform) in its metadata (see section 3.2 for further information).
Our main research aim is the validation of artifacts: the standards SCORM and CC
as well as corresponding tools. As these standards are widely implemented in tools
and rely upon them in practical experiments, the use of these tools can be considered
a valid evaluation of the standards [9]. This paper mainly looks at the design
evaluation [18] in an experimental setting [19] and essentially the usability aspects
[20]. Both SCORM and IMS Common Cartridge have different versions which
support different features. This study focuses on the key feature of compatibility and
portability of resources as it can be seen as a critical feature for teachers when re-
using learning objects from repositories.
3.1. Validation Group
A teacher workshop was organized for a group of 44 mathematics and science
teachers equally split, from Portugal, Lithuania, Romania and Belgium. The four
countries were chosen randomly among the participating countries to the ASPECT
project. Teachers were selected to have the following characteristics: teachers of
mathematics, science and/or technology, some experience with ICT and Learning
Resources and currently teaching students aged 6 to 21. Before selecting teachers on
the basis of these profiles, the strategy for finding volunteers varied from specifically
contacting teachers who, in one way or another, had collaborated with the selection
team before (e.g., Portugal) to publishing an open call for volunteers (e.g., Lithuania).
Initial observations showed that the ICT skills of this group of teachers ranged from
little knowledge (a few even unsure of what Google was) to advanced (users of
LaTeX ). Up to 80% of the teachers had advanced ICT skills. The group of advanced
teachers was strongly represented. The validation results must be seen in this light:
Teachers participating in European projects tend to be more motivated towards ICT
and improving their teaching methods. These teachers have clearly higher levels of
motivation and are very eager to share their knowledge and expertise. As a
consequence, it must be taken into account that if these teachers find the project tasks
and concepts too difficult or uninteresting, one can be reasonably certain that this will
apply even more so to teachers with average levels of ICT competence.
3.2. Tests Description
The workshop combined straight-forward assignments and direct feedback gathering
from the teachers in the form of interviews and two surveys. The test session was
organized in May 2010, concentrated on the integration of Resources into Learning
Management Systems and content packaging. In preparation for the tests, teachers had
already learned how to create basic courses on Moodle platform. They also had
learned how to browse the Learning Object Repository, LRE.
In the tests, teachers were initially asked to create a normal lesson plan using the
Moodle learning platform in a “traditional” way by combining different Resources.
Then they were asked to repeat the same task using a Resource on the same topic that
82
had been ‘packaged’ by ASPECT content developers using both the SCORM and
IMS Common Cartridge packages. The test task was to create a simple lesson plan
made up of some text, an image, a quiz and a forum, on the topic of thermodynamics
for Moodle, using four different approaches:
1) Using non-packaged content
2) Using the entire SCORM package (created from the non-packaged content)
3) Using an entire IMS Common Cartridge package (created from the non-
packaged content, with a forum added)
4) Picking up parts from the IMS Common Cartridge package
The tests were designed to serve both as a basic training on the use of different types
of packaged content and their features (necessary as the teachers had no previous
experience with this kind of content) and at the same time obtain their reactions as
rewards to usefulness in their everyday teaching, interest and facility to use. Each
teacher had an empty Moodle course and editing rights. Each teacher designed the
same lesson plan four times as described before. Teachers searched for the resource in
question in the LRE, where it was provided in the 3 formats, both to view and
download. For the non-packaged lesson plan, teachers had to use Moodle options to
create the quiz and the forum themselves. Both the SCORM and Common Cartridge
packages had the quiz included in the package, and the latter also contained a forum.
Additionally, teachers were presented with a dozen additional Resources packaged
as SCORM and Common Cartridge to browse through and see their benefits,
independently of the topic of the Resources. In all cases, to view the packaged
content, Icodeon’s Common Cartridge Platform and SCORM player were used as the
tools to show the packaged content.
4 Results Analysis
As the sample size of the survey (n=44) was small, the results of the statistical
analysis can only give us some indication on the teachers’ attitudes. The survey
results were backed up by a qualitative analysis of interviews of the teachers.
Generally, the teachers reacted to SCORM Resources in much the same way that
they treated unpackaged content; for example, they did not see much difference
between having a SCORM Resource and a PowerPoint presentation. While they saw
that a SCORM package could include more than one resource, they did not use it any
differently than PowerPoint; both types of content were integrated into an LMS as a
single, unmodifiable entity. In comparison, the teachers were very enthusiastic about
CC content packaging. After importing a CC package into Moodle, the teachers could
remove parts that they did not need, edit the content and change the order of different
resources. Many teachers requested instructions on how to adapt Moodle to use CC
packages and even some teachers expressed an interest in using CC to package their
own content in order to share it with other teachers.
Most teachers had little interest in simply viewing and playing SCORM or CC
packages. On the other hand, apart from the option to upload the complete packages
into an LMS and have the different parts of the package converted into Moodle
format, the teachers liked the possibility of being able to embed only parts of a
83
Common Cartridge package in the LMS, or even blogs or websites, which is not
possible to do with items from within SCORM packages. While this second option
did not take advantage of the LMS's features, teachers liked to have the possibility to
only integrate the parts of the cartridge into their courses that they liked or thought
were relevant to their lesson. In the survey,
• 25% said that taking an entire course in SCORM format and using it in Moodle
(or their school's own System) would be extremely helpful
• 39% said that taking an entire course in CC format and using it in Moodle (or
their school's own System) would be extremely helpful
• 43% said that taking a piece of the Learning Resource from one of the CC
packages and using it with their other teaching materials would be extremely
helpful
Most of the teachers who did not see the approaches as extremely helpful, saw that
the packages could be useful for them in limited cases like when giving homework or
teaching an online course. The survey results indicate that teachers see the
interoperability between LORs and LMSs created by the specifications SCORM and
CC as useful for their everyday teaching – especially when the packaging allows them
to alter the content after it was uploaded or selecting only bits of the package before
uploading to LMS. IMS Common Cartridge specifies this interoperability.
Part of the objective of this research was not only to find out whether or not the
teachers could see content packaging useful, but to find out if they could actually
manage working with these standards with the ICT skills that they possess. Our initial
hypothesis was that the tools supporting these standards have not yet developed
enough to be easy enough for the teachers to use. However, according to the survey,
normal web pages were unsurprisingly the easiest to use. 75% of the teachers found
using entire SCORM packages really easy or reasonable, whereas they admitted
having some problems when using CC packages, whether it was the package as a
whole, or taking parts of it. However, interestingly none of the teachers evaluated that
any of these methods as impossible to use in their every day teaching (see Fig. 1).
45 %
40 %
35 %
30 %
25 %
20 %
15 %
10 %
5%
0%
Really Easy Reasonable Some problems Quite Impossible to use
Complicated in everyday
teaching
Using normal web page
Using a SCORM package
Using an IMS Common Cartridge package
Taking parts of an IMS Common Cartridge package
Fig.1. “How easy/difficult was it to create a lesson plan...”.
84
This study indicates that even though SCORM packages might not be the preferred
solution for teachers, the longer period of development seems to have helped it to be
easier to use than IMS Common Cartridge packages, which was not supported by
Moodle before the spring of 2010. There seems to be a need to develop the tools for
these standards to be more usable for teachers in the future. However this result would
also indicate that the teachers did not like the easiest option best, which would suggest
that they were able to look beyond the interfaces of the players into the ideas of
interoperability and the standards.
Part of testing SCORM and IMS Common Cartridge with users, we looked at the
opinions regarding the technical interfaces which show the contents of the packages.
Teachers were asked to think of three typical use cases from their everyday teaching
life:
A) Showing Educational Resources to students in their class rooms
B) Teaching an online course
C) Giving online homework/extra credit work to the students
60 %
50 %
40 %
30 %
20 %
10 %
0%
Web page SCORM Common SCORM IMS Common I think they
Player view Cartridge package in Cartridge are basically
player view Moodle package in the same and
Moodle I don't care
which one I
use
...you are just showing the materials to the students in your class?
...you are going to teach an entire online course?
...you are giving them online homework/extra credit work?
Fig. 2. Interface preferences of teachers in three use scenarios.
Overall, Fig. 2 shows that when it comes to just showing Resources in the classroom,
that is easiest to do from a normal webpage, or even that it does not matter so much
what the interface is. However, when you are creating an online course or giving
homework to students, normal web page is no longer enough. Half of the teachers
thought that using IMS Common Cartridge packages in Moodle would be the ideal
way to teach online or give assignments in the form of homework or extra credit to
the students. In the interviews with teachers it became obvious that Learning
Management Systems like Moodle are widely used in schools and therefore standards
85
that create interoperability between content and these LMSs, have additional value for
teachers.
If we want students to study independently, you can give them a SCORM package
that they unzip and then use. But if you want, for example, to integrate a package in
Moodle, because in Portugal we use Moodle a lot, probably Common Cartridge is
good, because we can prepare everything and import it into Moodle." (ICT Teacher
from Portugal) “
After the workshop, teachers were asked, what would they prefer to use after
learning about the functionalities of SCORM and IMS Common Cartridge packages
and interoperability between LORs and LMSs. Most of the teachers ended up
preferring IMS Common Cartridge packages (87%), leaving only 7% to prefer normal
web pages and 9% to prefer SCORM packages. This result supports the finding that
teachers were genuinely excited about the prospects of IMS Common Cartridge after
one day of training and lesson plan making. The teachers also seemed to support the
solution that was the most adaptable to their own needs rather than the solution that
was easiest to use.
5 Conclusions
Open educational materials and other web-based resources lead to new opportunities
for sharing and re-using content. [21] European teachers are seldom aware of the
content packaging standards SCORM and IMS Common Cartridge but understand the
added value of re-using Educational Resources in their everyday teaching.
In this paper, we have presented the teachers’ view on the interoperability between
LORs and LMSs while using SCORM and CC standards. Teachers showed special
interest towards CC packages and in particular their use in Moodle system. In detail,
the teachers enjoyed the possibility of editing a package, taking some elements and
mixing them with their own teaching Resources very much in the same way as they
do in with the non-digital Resources in their classrooms. CC supports this kind of
interoperability, which is the key finding of this research as it aims to solve one of the
biggest problems in the field [22]. In this study, the teachers said that they were more
willing to use a solution that would be suitable for them than the solution which
seemed the easiest to use. Also no teacher believes that packaging standards are
impossible to use in everyday teaching, even though they can see some problems and
complications in the process. After a standard is finalized, it takes a long time before
tools are developed that actually deliver the functionality to end users in a way that is
useful and usable [23]. Hence, for the success of the standard among users, it is
crucial to develop CC tools and improve their usability.
We also need to recognize that many teachers still struggle to obtain the basic IT
skills which are more essential for their day-to-day work even if these teachers were
optimistic about content packaging – teachers with lower ICT skills might have a
different point of view. In our opinion, training in content packaging standards may
be something that remains of interest to a fairly small number of European teachers.
However, it is not imperative for the teachers to know that these standards exist. What
is vital for them is that the process of lesson preparation using Educational Resources
86
will be smooth and quick. Further development of these standards and the tools
around them is the way of assuring re-use of Educational Resources.
Acknowledgements
This work has been done in the project “ASPECT: Adopting Standards and
Specifications for Educational Content” (http://aspect.eun.org/), funded by the
European Union, reference number 417008. The authors wish to thank José Moura
Carvalho, Pascal Craeye, Delia Oprea, Svetlana Kubilinskiene, Ingo Dahn, Joris
Klerkx, Lars Ingesman, Anicet Yalaho and Alenka Kavcic for their help in organizing
the workshops and setting up the tests. The authors also wish to acknowledge the
present work could not have been carried out without the help and enthusiasm of the
44 teachers from Belgium, Lithuania, Portugal and Romania.
References
1. IEEE Learning Technology Standards Commity.: IEEE Standard for Learning
Object Metadata. Available at http://ltsc.ieee.org/wg12/ site visited 4.6.2010.
2. WILEY, D.: Connecting Learning Objects to Instructional Design Theory: a
Definition, a Metaphor, and a Taxonomy. Utah State University: Digital Learning
Environments Research Group, The Edumetrics Institute (2000)
3. McGreal, R.: Learning Objects: A Practical Definition. Int’l J. Instructional
Technology and Distance Learning, vol. 1, no. 9, pp. 9 (2004)
4. Mohan, P.: Learning Object Repositories. Proceedings of the Informing Science
and IT Education Joint Conference, Flagstaff, Arizona, USA (2005)
5. Dougiamas, M., Taylor, P.: Moodle: Using Learning Communities to Create an
Open Source Course Management System. In D. Lassner & C. McNaught (Eds.),
Proceedings of World Conference on Educational Multimedia, Hypermedia and
Telecommunications, pp. 171-178. Chesapeake, VA: AACE. (2003)
6. Qu, W., Nejdl, W.: Towards Interoperability and Reusability of Learning
Resources: a SCORM-conformant Courseware for Computer Science Education.
IEEE International Conference on Advanced learning (2002)
7. Gonzalez-Barbone, V., Anido-Rifon, L.: From SCORM to Common Cartridge: A
step forward. Computers & Education Volume 54, Issue 1, January 2010, Pages
88-102 (2009)
8. Rey-López, M., Díaz-Redondoa, R., Fernández-Vilasa, A., Pazos-Ariasa, J.,
García-Duquea, J., Gil-Sollaa, A., Ramos-Cabrera, M: An extension to the ADL
SCORM standard to support adaptivity: The t-learning case-study. Computer
Standards & Interfaces, Volume 31, Issue 2, February 2009, pp 309-318 (2008)
9. Duval, E.: Learning Technology Standardization: Making Sense of it All. ComSIS
Vol 1, No. 1, (2004)
10. ADL Technical Team.: ADL Guidelines for Creating Reusable Content with
SCORM 2004 (2008) Available at: http://www.adlnet.org. site visited 4.6.2010.
87
11. Gonzalez-Barbone, V., Anido-Rifon, L.: Creating the first SCORM object.
Computers & Education, pp. 1634–1647 (2008)
12. IMS Global Learning Consortium: IMS Common Cartridge Specification.
Available at: http://www.imsglobal.org/commoncartridge.html Site visited
7.6.2010.
13. Moodle.: Development: IMS common cartridge. Available at:
http://docs.moodle.org/en/Development:IMS_common_cartridge. Site visited
8.6.2010.
14. Massart, D., Chaudron, S., Ayre, J., Klerkx, J., Heckmann, P., Ravet, S.,
Ingesman, L., Gras-Velázquez, À., Pawlowski J.: D-1.2.1 ASPECT First progress
report. ASPECT deliverable. Confidential (2009)
15. Gras-Velázquez, À. Clements, K., Yalaho, A., Ayre J., Ingesman, L., Van Assche,
F., Blamire R., Vuorikari, R., De Four, H.: D-6.2 Protocol of Experimentation
with National Adaptations. ASPECT deliverable. Confidential (2009)
16. Massart, D.: The EUN Learning Resource Exchange (LRE). The 15th
International Conference on Computers in Education (ICCE2007) Supplementary
Proceedings, vol. 1, pp. 170-174 (2007)
17. Ternier, S., Massart. D., Campi, A., Guinea, S., Ceri, S., Duval, E.:
Interoperability for Searching Learning Object Repositories - The ProLearn Query
Language, D-LIB Magazine, vol. 14 (2008) Available at
http://dlib.ejournal.ascc.net/dlib/january08/ceri/01ceri.html Site visited 4th June
2010.
18. Hevner, A.R., March, S.T., Park, J., Ram, S.: Design Science in Information
Systems Research. MIS Quarterly, vol. 28, 75-105 (2004)
19. Zelkowitz M. V., Wallace, D.: Experimental models for validating computer
technology, IEEE Computer, vol. 31, pp. 23-31. (1998)
20. Nielsen, J.: Usability Engineering. Academic Press, Boston (1993)
21. Vuorikari, R.: Insight Special Report: Why Europe Needs Free and Open Source
Software and Content in Schools, 2004.
22. Johnstone S.: Open Educational Resources Serve the World: Sharing Educational
Resources over the Internet. Educause Quarterly, Vol. 28, No. 3, 2005.
23. Duval, E., & Verbert, K. On the Role of Technical Standards for Learning
Technologies. IEEE Transactions on Learning Technologies, vol. 1, no. 4 (2008)
88