=Paper=
{{Paper
|id=Vol-2121/paper9
|storemode=property
|title=Proposed Standard for Metadata Tagging with Pedagogical Identifiers
|pdfUrl=https://ceur-ws.org/Vol-2121/paper9.pdf
|volume=Vol-2121
|authors=Jeanine A. DeFalco
}}
==Proposed Standard for Metadata Tagging with Pedagogical Identifiers==
https://ceur-ws.org/Vol-2121/paper9.pdf
73
Proposed Standard for Metadata Tagging with
Pedagogical Identifiers
Jeanine A. DeFalco 1, 2
1 US Army Research Laboratory, 2 Oak Ridge Associated Universities
jeanine.a.defalco.ctr@mail.mil
Abstract. The constantly evolving domain of adaptive training warrants an ex-
amination of standards for adaptive instructional systems (AIS), specifically in
the area of metadata tagging as it relates to the methods to curate, apply, and
evaluate content in the authoring process. Metadata tags provide information not
only to authors of AIS’s, but are key in driving learning object searches. Metadata
tagging represents key micro-communications between systems and search en-
gines, and its absence limits the ability to meaningfully consume and exchange
information with learning management systems. In the effort to standardize tools
and methods for authoring content for generalizable AIS’s, this paper will advo-
cate for establishing a standard on metadata tagging methodology for learning
objects that would include pedagogical identifier codes. The establishment of
this standard would not only ease efforts in the development and implementation
of learning designs for AIS’s, but would promote efforts to expand the aims of
AIS’s to push beyond domain specific content mastery tasks to support broader
cross-disciplinary discriminate intelligence in learners.
Keywords: Standards, Adaptive Instructions Systems, Metadata tagging, Peda-
gogical identifiers.
1 Introduction
The constantly evolving domain of adaptive training warrants an examination of stand-
ards for adaptive tutoring systems, specifically in the area of metadata tagging.
Metadata tags provide information not only to authors of tutoring system, but are key
in driving learning object searches. Metadata tagging represents key micro- communi-
cations between systems and search engines, and without it, limits the ability to mean-
ingfully consume and exchange information with learning management systems. As
such, in the effort to standardize tools and methods for authoring content for adaptive
instructional tutoring systems (AIS’s), this paper will limit its scope to reviewing the
gap that exists in the report regarding the metadata management requirements and ex-
isting solutions for European Union (EU) Institutions and Member States [1], critique
the current IEEE Learning Object Metadata (LOM) standard regarding the interactivity
element of the Educational category in the LOM data model [2], and propose a revision
of this element for AIS’s to standardize pedagogical identifier codes (PIC) based on
Back to Table of Contents
74
Bloom’s Revised Taxonomy in order to support efforts to create learning environments
that move beyond mere content mastery to supporting learners’ discriminate intelli-
gence [3].
2 Gap in Report of ISA Data Standards Governance
In 2014, the Interoperability Solutions for European Public Administrations (ISA) Pro-
gramme reported on their commissioned study aimed at recommending coordination
of data standards, governance, and management within and between different organi-
zations at both the national and EU level to promote greater interoperability between
information systems. The results of this study identify the set of high-level processes
for managing the lifecycle of metadata, propose a governance structure for data stand-
ards including roles and responsibilities, as well as identify tools and best practices for
managing and consuming data standards [1].
To begin with, the authors of this report identify the need for standardizing primary
data models and reference data, and distinguish the relevant obstacles for addressing
both. Data models -- as a collection of entities, their properties and relationship to each
other -- represent conceptual or real-world elements. When changes are intended for
data models, what is needed is a strongly managed implementation plan aligned with a
software upgrade cycle [1]. Reference data, on the other hand, is data used to organize
and categorize data consisting of codes and descriptions or definitions. Since changes
in reference data is more loosely linked to the functionality of applications, it does not
have a disruptive effect on functionality [1]. Examples of reference data includes type
codes (codes and values to categorize an object by type), and descriptive taxonomies
and vocabularies that consist of stable lists of codes and values of real-world objects.
For the purposes of this paper, the advocacy for standards regarding metadata tagging
is less a call for change in standards pertaining to existing data models, but rather is a
call to design and create current reference data codes targeted for AIS’s as it relates to
educational activity learning objects, which would include redesigning and creating
descriptive type codes based on domain established pedagogical taxonomies.
Within this context, then, the gap in the ISA report as it relates to metadata tagging
standards can be found in how the ISA addressed and proposed standards for the dif-
ferent phases of the lifecycle of metadata [1]. The different phases of the lifecycle of
metadata were identified as: documentation; maintenance and updating; sharing and
reusing; and designing and creating [1]. Accordingly, there was a recommendation to
establish standards for metadata management, documentation, and representation, but
the area of designing and creating was identified as outside the scope of the report [1].
Importantly, this omission is particularly relevant to adaptive instructional designers,
because the designing and creating of metadata standards is arguably an area of neglect
that if remedied could have a positive effect on how content is identified and then re-
used in designing an adaptive instructional environment. What follows is an analysis
of the current flawed IEEE Learning Object metadata model standard [2], as well as a
critique as to how the educational category of this schema can be redesigned by way
of creating pedagogical identifiers.
Back to Table of Contents
75
3 IEEE LOM Conceptual Data Schema
The IEEE Learning Object Metadata (LOM) is a metadata model consisting of con-
ceptual data schema designed in a way to allow for extensions to the established
schema, such as new vocabularies or taxonomies [2]. These vocabularies and taxono-
mies act as qualifiers for the nine established sections or categories: general, lifecycle,
metametadata, technical, educational, rights, relation, annotation, and classification
[2]. The IEEE LOM standard specifies the semantics and syntax via attribute elements
so learning objects can be used, re-used, or referenced in a learning platform.
Within the educational category, there are eleven elements with accompanying de-
scriptors: interactivity type; learning resource type; interactivity level; semantic den-
sity; intended end user role; context; typical age range; difficulty; typical learning time;
description; and language [2]. It is this first element, the interactivity type that is flawed
in its current iteration, and most specifically warrants a reexamination and a design
revision for standardizing metadata tagging for AIS’s.
4 Proposal for Pedagogical Identifier in Educational
Category of IEEE LOM
4.1 Tagging learning objects to support learning designs
The argument to design, create, and standardize pedagogical tagging rests in the issue
that content and learning objects in and of themselves do not necessarily lead to greater
learning. Rather, content is merely the vessel through which learning occurs. Tagging
content absent of a contextualization of how the content can support learning limits its
potential applicability across a range of domains, and does not help AIS designers and
authors to find content that best supports a learning objective.
What is needed in addition to tagging content and learning objects is the learning
objective: identifying how the content supports critical thinking and discriminate in-
telligence. Discriminate intelligence here means the ability to distinguish between not
just right and wrong answers, but between good-better-best answers, and generating
unique solutions to ill-defined problem sets. Therefore, prescribing a standard for
metadata tagging that identifies what skills learning objects support would not only
contribute to broad educational aims across a range of domains, but it would allow for
learning objects to be reused and referenced across domains that might normally be
filtered out because of limiting tagging identifiers.
The importance of creating high quality consistent metadata for learning resources
has been an area of concern in the e-learning area since the inception of the LOM as a
standard, as is the recognized need for better use of pedagogies to enhance instruc-
tion/learning [4]. Nitchot and Gilbert [5] advocate for authors to provide metadata in-
formation that would assist in searching for pedagogically relevant Web pages that
would requires tagging with a corresponding capability and a context.
Back to Table of Contents
76
Some suggested approaches to tagging has included using a collaborative approach
for tagging to overcome problems associated with establishing a formal ontology cre-
ation [6]. Others have proposed revising educational metadata profiles to characterize
digital educational resources that modifies the existing elements of IEEE LOM
schema, but this work is still limited to identifying the resource itself and does not
incorporate knowledge domain or learning outcome ontologies or taxonomies [7].
As has been identified by other researchers [8], if learning systems seek semantic
interoperability and optimization of enhanced instruction/learning, they should incor-
porate and adapt an existing ontology that includes a pedagogical framework that can
standardize tagging of learning objects. In that context, then, this paper advocates not
only for a revision of the LOM metadata model, but to adopt the existing ontology of
Bloom’s Revised Taxonomy and adapt it to achieve improved interoperability for AISs
developers and authors.
4.2 IEEE LOM Interactivity Type in Educational Category
Currently, the IEEE LOM description for the Interactivity Type element within the Ed-
ucational Category is described is as follows: “active: active learning is supported by
content that directly induces productive action by the learner; expositive: expositive
learning occurs when the learner’s job mainly consists of absorbing the content ex-
posed to them; mixed: a blend of active and expositive interactivity types” [2]. Not
only is this a reductionist model of learning, it is does not allow for tagging elements
that covers learning objectives beyond the cognitive domain, namely, the affective and
sensorimotor domains. While a recent analysis of metadata formats for data sharing [9]
noted that the LOM standard complemented by the MODS (Metadata Object Descrip-
tion Schema) allows for entering an identifier element to extend its definition, and in
this way, it is suitable for describing links to learning objects, it still does not establish
a standardized model or process applicable for AIS developers or authors. Accord-
ingly, not only should this particular reference data model be revised to change the
limited definitions of learning types, but for the purposes of establishing standards for
AIS’s, this metadata model should redefine the interactivity element and create tagging
identifiers according to established pedagogical taxonomies.
4.3 Pedagogical identifiers for adaptive instructional systems
The advocacy for using pedagogical identifiers is not a new idea. Hadji, Choi, and
Jemni’s [10] presents a conceptual model of a courseware generation system that uses
a pedagogical scenario model to support pedagogical flexibility in the adaptive
courseware generation system. Notably, they compose pedagogical scenarios using a
pedagogical identifier code that is given to reference each pedagogical scenario stored
in a repository, including pedagogical objectives that identify the objective to achieve
through the pedagogical scenario being modeled. Further, they identify the following
pedagogical models which while not comprehensive are established examples: presen-
tation, problem solving, discussion, brainstorming, games, simulation, role playing,
case study, project design method, question and answer method [10]. These models
Back to Table of Contents
77
also include an activity-sequencing that can be used to apply a particular pedagogic
scenario.
Essentially, the Hadji, Choi and Jemni model [10] is a sound one, except the authors
rely on the prior 1956 iteration of Bloom’s Taxonomy [11] (see Figure 1) [12] in which
the ordering of cognitive skills was organized according to nouns:
- Knowledge (Novice)
- Comprehension (Novice)
- Application (Average)
- Analysis (Average)
- Synthesis (Expert)
- Evaluation (Expert)
Fig. 1. Bloom’s original 1956 Taxonomy [12].
In 2001, however, Bloom’s Taxonomy [11] was revised [3] and it is here where we part
ways with the Hadji, Choi, and Jemni’s model of pedagogical identifiers.
4.4 Bloom’s Revised Taxonomy of 2001 in standardizing metadata
tagging
Bloom’s Revised Taxonomy of 2001 [3] (see Figure 2) was the result of a group of
instructional researchers, cognitive psychologists, and curriculum theorists’ efforts to
emphasize higher-order thinking development of learners, highlighting skill develop-
ment. In this revision, the taxonomy uses verbs instead of nouns, and swaps the highest
of the thinking skills “evaluation” with “creation,” – a meaningful and substantive
change.
Back to Table of Contents
78
Fig. 2. Comparison of Bloom’s Original 1965 vs. Revised 2001 Taxonomy [13].
Today in face-to-face classrooms, Bloom’s Revised Taxonomy continues to be used
as a tool to balance assessments with assignments for the purposes of insuring all orders
of thinking are addressed in students’ learning [13]. As a well-established taxonomy
within the field of Education and Educational Psychology, not only has Bloom’s Re-
vised Taxonomy become the backbone of many teaching philosophies, but it has been
used in curriculum designs to identify content that best works as a vessel through which
to teach skills.
The importance of Bloom’s Revised Taxonomy discussion lays in the fact that to
propose an effective design for the standardizing metadata tagging for AIS’s, the in-
clusion of pedagogical identifiers should rest both on widely accepted industry prac-
tices and be useful for the aims and purposes of designing and reusing AIS learning
objects. Adopting a standard that would encourage AIS designers and authors to use
metadata tagging that contextualizes content within Bloom’s taxonomical framework
could aid in more effective selection of content to support learning aims, as well as
broaden the scope of possible learning objects that can be incorporated in a learning
course that might otherwise be overlooked if tagged ineffectively and limited by mere
content identifiers.
In sum, this proposed standard is in line with the ISA efforts to maintain a standard
for structural metadata management for different phases of the lifecycle of metadata,
in particular by addressing the ISA’s gap of addressing designing and creating of
metadata tagging for learning objects [1]. More importantly, however, adopting a
metadata tagging standard that includes pedagogical identifiers could help influence
AIS’s designers and authors to think beyond merely populating courses with domain
Back to Table of Contents
79
specific content, but rather think more purposefully about designing environments sup-
port both content mastery and discriminate intelligence.
5 Conclusion
In all, the value of establishing a metadata tagging standard for AIS’s that includes
pedagogical identifiers based on Bloom’s Revised Taxonomy is anchored in the fact
that content, in and of itself, does not promote greater learning nor support the devel-
opment of discriminate intelligence. Rather, it is the way in which a learning environ-
ment is designed and implemented, the pedagogy used, and the outcomes one seeks to
achieve through content that yields robust learning outcomes and improved discrimi-
nate intelligence.
Ultimately, the significance of establishing a standard for AIS’s with Bloom’s Re-
vised Taxonomy pedagogical identifiers allows for a more manageable and reusable
method of employing a metadata tagging tool for authoring adaptive instructional
courses, particularly across a range of domains whose learning objectives and desired
outcomes are not limited by a narrowly constructed understanding of what learning
activities look like and do. Future work in this area should include how the proposed
meta-data standard would be applied, the types of pedagogical information that would
be included in this standard, how this standard would be mapped in relevant systems, a
more in-depth analysis and comparison of Bloom’s Revised Taxonomy versus other
established learning taxonomies, as well as determining whether other elements of the
IEEE LOM metadata standard warrant similar design revisions for AIS standards.
Acknowledgements
This research was sponsored by the US Army Research Laboratory and was accom-
plished under Cooperative Agreement Number W911NF-17-2-0152. The views and
conclusions contained in this document are those of the authors and should not be in-
terpreted as representing the official policies, either expressed or implied, of the Army
Research Laboratory or the U.S. Government. The U.S. Government is authorized to
reproduce and distribute reprints for Government purposes notwithstanding any copy-
right notation herein.
References
1. Dekkers, M., Loutas, N., Goedertier, S., Karalopoulos, A., Peristeras, V., Wigard, S.
Metadata management requirements and existing solutions in EU Institutions and Member
States. Innovation and the Public Sector, European Commission (2015).
2. IMS Global Learning Consortium. IMS Metadata Best Practice Guide for IEEE 1484.12. 1-
2002 Standard for Learning Object Metadata. http://www. imsglobal.
org/metadata/mdv1p3/imsmd_bestv1p3.html last accessed 2018/03/27.
Back to Table of Contents
80
3. Anderson, L., Krathwohl, D. R., et al (Eds.). A Taxonomy for Learning, Teaching, and As-
sessing: A Revision of Bloom’s Taxonomy of Educational Objectives. Allyn & Bacon, Bos-
ton, MA (2001).
4. Sungkur, R. K., Raheeman, I., & Musbally, Y. USING SEMANTIC WEB AND
ONTOLOGIES TO CREATE INTELLIGENT AND PERSONALIZED SYSTEMS FOR
LEARNING PURPOSES. IJAEDU-International E-Journal of Advances in Education 2(5),
332-340 (2016).
5. Nitchot, A., & Gilbert, L. Does the Web Contain Pedagogically Informed Materials? The
COSREW Outcomes. Electronic Journal of e-Learning 13(5), 390-410 (2015).
6. Bateman, S., Brooks, C., & McCalla, G. Collaborative tagging approaches for ontological
metadata in adaptive e-learning systems. In Proceedings of the fourth international work-
shop on applications of semantic web technologies for e-learning (SW-EL 2006) (pp. 3-12)
(2006).
7. Solomou, G., Pierrakeas, C., & Kameas, A. Characterization of educational resources in e-
learning systems using an educational metadata profile. Journal of Educational Technology
& Society 18(4), 246 (2015).
8. Faqihi, B., Daoudi, N., & Ajhoun, R. TOWARD A NEW TREATMENT APPROACH OF
LEARNING CONTENT IN CLOUD ERA. Journal of Theoretical and Applied Information
Technology 83(2), 215 (2016).
9. Stočes, M., Šimek, P., & Pavlík, J. Metadata Formats for Data Sharing in Science Support
Systems. AGRIS On-line Papers in Economics and Informatics 9(3), 61-69 (2017).
10. Hadji H., Choi, H., Jemni M.: Pedagogically-Driven Courseware Content Generation for
Intelligent Tutoring Systems. International Journal of Fuzzy Logic and Intelligent Systems
12(1): 77-85 (2012).
11. Bloom, B., Krathwohl, D. Taxonomy of Educational Objectives: The Classification of Ed-
ucational Goals. Longmans, Green, New York, NY (1956).
12. Vanderbilt University Center for Teaching. https://cft.vanderbilt.edu/guides-sub-
pages/blooms-taxonomy/ last accessed 2018/03/27.
13. Wilson, L. https://thesecondprinciple.com/teaching-essentials/beyond-bloom-cognitive-
taxonomy-revised/ last accessed 2018/03/27.
Back to Table of Contents