The constantly evolving domain of adaptive training warrants an examination of standards for adaptive tutoring systems, specifically in the area of metadata tagging. Metadata tags provide information not only to authors of tutoring system, but are key in driving learning object searches. Metadata tagging represents key micro-communications between systems and search engines, and without it, limits the ability to meaningfully consume and exchange information with learning management systems. As such, in the effort to standardize tools and methods for authoring content for adaptive instructional tutoring systems (AIS's), this paper will limit its scope to reviewing the gap that exists in the report regarding the metadata management requirements and existing solutions for European Union (EU) Institutions and Member States [1], critique the current IEEE Learning Object Metadata (LOM) standard regarding the interactivity element of the Educational category in the LOM data model [2], and propose a revision of this element for AIS's to standardize pedagogical identifier codes (PIC) based on Bloom's Revised Taxonomy in order to support efforts to create learning environments that move beyond mere content mastery to supporting learners' discriminate intelligence [3].
In 2014, the Interoperability Solutions for European Public Administrations (ISA) Programme reported on their commissioned study aimed at recommending coordination of data standards, governance, and management within and between different organizations at both the national and EU level to promote greater interoperability between information systems. The results of this study identify the set of high-level processes for managing the lifecycle of metadata, propose a governance structure for data standards including roles and responsibilities, as well as identify tools and best practices for managing and consuming data standards [1].
To begin with, the authors of this report identify the need for standardizing primary data models and reference data, and distinguish the relevant obstacles for addressing both. Data models --as a collection of entities, their properties and relationship to each other --represent conceptual or real-world elements. When changes are intended for data models, what is needed is a strongly managed implementation plan aligned with a software upgrade cycle [1]. Reference data, on the other hand, is data used to organize and categorize data consisting of codes and descriptions or definitions. Since changes in reference data is more loosely linked to the functionality of applications, it does not have a disruptive effect on functionality [1]. Examples of reference data includes type codes (codes and values to categorize an object by type), and descriptive taxonomies and vocabularies that consist of stable lists of codes and values of real-world objects. For the purposes of this paper, the advocacy for standards regarding metadata tagging is less a call for change in standards pertaining to existing data models, but rather is a call to design and create current reference data codes targeted for AIS's as it relates to educational activity learning objects, which would include redesigning and creating descriptive type codes based on domain established pedagogical taxonomies.
Within this context, then, the gap in the ISA report as it relates to metadata tagging standards can be found in how the ISA addressed and proposed standards for the different phases of the lifecycle of metadata [1]. The different phases of the lifecycle of metadata were identified as: documentation; maintenance and updating; sharing and reusing; and designing and creating [1]. Accordingly, there was a recommendation to establish standards for metadata management, documentation, and representation, but the area of designing and creating was identified as outside the scope of the report [1]. Importantly, this omission is particularly relevant to adaptive instructional designers, because the designing and creating of metadata standards is arguably an area of neglect that if remedied could have a positive effect on how content is identified and then reused in designing an adaptive instructional environment. What follows is an analysis of the current flawed IEEE Learning Object metadata model standard [2], as well as a critique as to how the educational category of this schema can be redesigned by way of creating pedagogical identifiers.
The IEEE Learning Object Metadata (LOM) is a metadata model consisting of conceptual data schema designed in a way to allow for extensions to the established schema, such as new vocabularies or taxonomies [2]. These vocabularies and taxonomies act as qualifiers for the nine established sections or categories: general, lifecycle, metametadata, technical, educational, rights, relation, annotation, and classification [2]. The IEEE LOM standard specifies the semantics and syntax via attribute elements so learning objects can be used, re-used, or referenced in a learning platform. Within the educational category, there are eleven elements with accompanying descriptors: interactivity type; learning resource type; interactivity level; semantic density; intended end user role; context; typical age range; difficulty; typical learning time; description; and language [2]. It is this first element, the interactivity type that is flawed in its current iteration, and most specifically warrants a reexamination and a design revision for standardizing metadata tagging for AIS's.
Proposal for Pedagogical Identifier in Educational Category of IEEE LOM
The argument to design, create, and standardize pedagogical tagging rests in the issue that content and learning objects in and of themselves do not necessarily lead to greater learning. Rather, content is merely the vessel through which learning occurs. Tagging content absent of a contextualization of how the content can support learning limits its potential applicability across a range of domains, and does not help AIS designers and authors to find content that best supports a learning objective.
What is needed in addition to tagging content and learning objects is the learning objective: identifying how the content supports critical thinking and discriminate intelligence. Discriminate intelligence here means the ability to distinguish between not just right and wrong answers, but between good-better-best answers, and generating unique solutions to ill-defined problem sets. Therefore, prescribing a standard for metadata tagging that identifies what skills learning objects support would not only contribute to broad educational aims across a range of domains, but it would allow for learning objects to be reused and referenced across domains that might normally be filtered out because of limiting tagging identifiers.
The importance of creating high quality consistent metadata for learning resources has been an area of concern in the e-learning area since the inception of the LOM as a standard, as is the recognized need for better use of pedagogies to enhance instruction/learning [4]. Nitchot and Gilbert [5] advocate for authors to provide metadata information that would assist in searching for pedagogically relevant Web pages that would requires tagging with a corresponding capability and a context. Some suggested approaches to tagging has included using a collaborative approach for tagging to overcome problems associated with establishing a formal ontology creation [6]. Others have proposed revising educational metadata profiles to characterize digital educational resources that modifies the existing elements of IEEE LOM schema, but this work is still limited to identifying the resource itself and does not incorporate knowledge domain or learning outcome ontologies or taxonomies [7].
As has been identified by other researchers [8], if learning systems seek semantic interoperability and optimization of enhanced instruction/learning, they should incorporate and adapt an existing ontology that includes a pedagogical framework that can standardize tagging of learning objects. In that context, then, this paper advocates not only for a revision of the LOM metadata model, but to adopt the existing ontology of Bloom's Revised Taxonomy and adapt it to achieve improved interoperability for AISs developers and authors.
Currently, the IEEE LOM description for the Interactivity Type element within the Educational Category is described is as follows: "active: active learning is supported by content that directly induces productive action by the learner; expositive: expositive learning occurs when the learner's job mainly consists of absorbing the content exposed to them; mixed: a blend of active and expositive interactivity types" [2]. Not only is this a reductionist model of learning, it is does not allow for tagging elements that covers learning objectives beyond the cognitive domain, namely, the affective and sensorimotor domains. While a recent analysis of metadata formats for data sharing [9] noted that the LOM standard complemented by the MODS (Metadata Object Description Schema) allows for entering an identifier element to extend its definition, and in this way, it is suitable for describing links to learning objects, it still does not establish a standardized model or process applicable for AIS developers or authors. Accordingly, not only should this particular reference data model be revised to change the limited definitions of learning types, but for the purposes of establishing standards for AIS's, this metadata model should redefine the interactivity element and create tagging identifiers according to established pedagogical taxonomies.
The advocacy for using pedagogical identifiers is not a new idea. Hadji, Choi, and Jemni's [10] presents a conceptual model of a courseware generation system that uses a pedagogical scenario model to support pedagogical flexibility in the adaptive courseware generation system. Notably, they compose pedagogical scenarios using a pedagogical identifier code that is given to reference each pedagogical scenario stored in a repository, including pedagogical objectives that identify the objective to achieve through the pedagogical scenario being modeled. Further, they identify the following pedagogical models which while not comprehensive are established examples: presentation, problem solving, discussion, brainstorming, games, simulation, role playing, case study, project design method, question and answer method [10]. These models also include an activity-sequencing that can be used to apply a particular pedagogic scenario.
Essentially, the Hadji, Choi and Jemni model [10] is a sound one, except the authors rely on the prior 1956 iteration of Bloom's Taxonomy [11] (see Figure 1) [12] in which the ordering of cognitive skills was organized according to nouns:
-Knowledge (Novice) -Comprehension (Novice) -Application (Average) -Analysis (Average) -Synthesis (Expert) -Evaluation (Expert) In 2001, however, Bloom's Taxonomy [11] was revised [3] and it is here where we part ways with the Hadji, Choi, and Jemni's model of pedagogical identifiers.
Bloom's Revised Taxonomy of 2001 [3] (see Figure 2) was the result of a group of instructional researchers, cognitive psychologists, and curriculum theorists' efforts to emphasize higher-order thinking development of learners, highlighting skill development. In this revision, the taxonomy uses verbs instead of nouns, and swaps the highest of the thinking skills "evaluation" with "creation," -a meaningful and substantive change. Today in face-to-face classrooms, Bloom's Revised Taxonomy continues to be used as a tool to balance assessments with assignments for the purposes of insuring all orders of thinking are addressed in students' learning [13]. As a well-established taxonomy within the field of Education and Educational Psychology, not only has Bloom's Revised Taxonomy become the backbone of many teaching philosophies, but it has been used in curriculum designs to identify content that best works as a vessel through which to teach skills.
The importance of Bloom's Revised Taxonomy discussion lays in the fact that to propose an effective design for the standardizing metadata tagging for AIS's, the inclusion of pedagogical identifiers should rest both on widely accepted industry practices and be useful for the aims and purposes of designing and reusing AIS learning objects. Adopting a standard that would encourage AIS designers and authors to use metadata tagging that contextualizes content within Bloom's taxonomical framework could aid in more effective selection of content to support learning aims, as well as broaden the scope of possible learning objects that can be incorporated in a learning course that might otherwise be overlooked if tagged ineffectively and limited by mere content identifiers.
In sum, this proposed standard is in line with the ISA efforts to maintain a standard for structural metadata management for different phases of the lifecycle of metadata, in particular by addressing the ISA's gap of addressing designing and creating of metadata tagging for learning objects [1]. More importantly, however, adopting a metadata tagging standard that includes pedagogical identifiers could help influence AIS's designers and authors to think beyond merely populating courses with domain specific content, but rather think more purposefully about designing environments support both content mastery and discriminate intelligence.
In all, the value of establishing a metadata tagging standard for AIS's that includes pedagogical identifiers based on Bloom's Revised Taxonomy is anchored in fact that content, in and of itself, does not promote greater learning nor support the development of discriminate intelligence. Rather, it is the way in which a learning environment is designed and implemented, the pedagogy used, and the outcomes one seeks to achieve through content that yields robust learning outcomes and improved discriminate intelligence.
Ultimately, the significance of establishing a standard for AIS's with Bloom's Revised Taxonomy pedagogical identifiers allows for a more manageable and reusable method of employing a metadata tagging tool for authoring adaptive instructional courses, particularly across a range of domains whose learning objectives and desired outcomes are not limited by a narrowly constructed understanding of what learning activities look like and do. Future work in this area should include how the proposed meta-data standard would be applied, the types of pedagogical information that would be included in this standard, how this standard would be mapped in relevant systems, a more in-depth analysis and comparison of Bloom's Revised Taxonomy versus other established learning taxonomies, as well as determining whether other elements of the IEEE LOM metadata standard warrant similar design revisions for AIS standards.
This research was sponsored by the US Army Research Laboratory and was accomplished under Cooperative Agreement Number W911NF-17-2-0152. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Army Research Laboratory or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein.