In this paper, we address the problem of competency comparison, providing some heuristics to help match the competencies of users with those involved in task-based scenario components (actors, tasks, resources). Competencies are defined according to a structured competency model based on a domain ontology. We provide a context for recommendation through a learning scenario model. The approach has been implemented by extending an ontologydriven system called TELOS. It has been tested with a learning unit where these comparison relations are used to provide recommendations to users involved in a technology enhanced learning scenario.
actors using various kinds of input resources, producing outcomes and
interacting with other actors
We have built an ontology-based competency model, also presented
in section 2. It is used for the semantic referencing of actors, tasks and
resources in a scenario, and as a basis for recommendation. Unlike
other approaches for an ontology-based recommendation, such as
OWLOLM
In section 3, we describe a method for referencing resources in a learning scenario with such ontology-based competencies. We also address the central problem of competency comparison, providing some heuristics to help match a user’s competencies with those possessed by other actors or involved in task or resources in a scenario.
In section 4, we present an application where these comparison
relations are used to define recommendation agents, to help personalize a
learning scenario. Applications like the one presented here are
implemented as an extension of the TELOS ontology-driven system
Figure 1 presents a simple scenario model, a screen-shot from our
GMOT scenario editor
In the first task, the student reads the general assignment for the scenario and the list of target competencies he is supposed to acquire. In the second one, he builds a table of planet properties that is validated by the professor, using the information in a PowerPoint document (called “Planet Properties”). In the third one, using this table assessed by the professor (“Validated table”), he compares five properties of planets to find out relations between properties, writing a text on his findings (“Validated relations”). In the last task he is asked to order the planets according to their distance to the Sun and to write his ideas on planets that can sustain life.
On the right side of the figure, three recommendation agents have been added to corresponding tasks, in order to provide advice and update the student’s competency model with newly acquired competencies. Their action will be explained in section 4.
As we have pointed out (Paquette and Marino, 2004), educational modeling languages and standards such as IMS-LD (2003) need to be improved with a structured knowledge and competency representation, in order to add semantic references to scenario components. Two main methods are generally used: semantic references from a domain ontology or natural language statements called prerequisites and learning objectives (as in IMS-LD). Both are not sufficient for our purpose.
In most common practice, unstructured natural language statements from a competency referential are used. Such statements have many problems. First, they are not related to domain ontologies that could describe formally their knowledge part. Second, natural language statements are not appropriate for computation. Computationally, they make it difficult to reference and compare competencies assigned to actors, tasks and resources of a learning scenario. The IEEE-RCD (2007) specification allows optional definition elements as “a structured description that provides a more complete definition of the RCD than the free-form description expressed in the title and description”.
Our competency model corresponds to that goal. It has been published in many conferences, journals and books, and also extensively used in instructional design projects. Devedsic (2006, p.260) describes our model as “a competency structure, corresponding to the domain ontology and represented by entry and target competencies related to the nodes in the instructional structure” (the scenario model).
Unlike other approaches based on ontologies, such as OWL-OLM
For that purpose, in our competency model
This model has been implemented (in a TELOS extension) for referencing actors, tasks and resources in the following way. The domain ontology follows the W3C OWL-DL standard. The taxonomy of skills is simplified to a 10-level scale (0-PayAttention, 1-Memorize, 2Explicitate, 3-Transpose, 4-Apply, 5-Analyze, 6-Repair, 7-Synthetize, 8-Evaluate, 9-Self-Control). The performance part is a combination of performance criteria values with four performance levels (0.2-aware, 0.4-familiar, 0,6-productive, 0.8-expert), added to the skill level.
For example, using a domain ontology of solar system planets (shown on figure 3) and a competency referential (or model) based on this ontology, competencies can be associated to a resource from the scenario on figure 1. The competencies describing such a resource, (“Planet Properties”), could be compared with those of a user (Gilbert Paquette) to verify if he has all of them, or some, or none, in his competency model, and offer a recommendation accordingly.
All components of a scenario are thus referenced using comparable competencies, based on the same domain ontology. Resources and tasks in a scenario are referenced by two sets of competencies, one for prerequisite competencies, and the other, for target competencies (i.e. learning objectives).
A user competency model is composed of three main parts
Consider two competencies C1=(K1, S1, P1) and C2=(K2, S2, P2). It will be rarely the case that the three parts will coincide, but we can evaluate the semantic proximity or nearness between C1 and C2, based on the respective positions of their knowledge parts in the ontology and the values associated with the skills and the performance levels.
From a semantic point of view, a recommendation agent evaluates for example if a user’s actual competency is very near, near, or far from the prerequisite or target competencies of a resource or a task or from the actual competencies of another user. The agent can also evaluate if a competency is stronger or weaker than another one according to the levels of its skill and performance parts. Or it can determine if the competency is more specific or more general according to the positions in the ontology of the corresponding knowledge components.
Thus, to take advantage of the competency representation, we need to establish a formal framework for the evaluation of the proximity, strength or generality of competencies. In the next section we define the semantic proximity between knowledge parts of a competency. In section 3.3 we extend the framework to competencies by considering skills and performance .
In this section, we focus only on the knowledge part of two competencies to be compared. Maidel et al. (2008), proposes an approach in which a taxonomy is exploited. Five different cases of matches between a concept A in the resource profile and a concept in the user profile are considered. Various matching scores are given when a concept A in the item profile, a) is the same, b) is a parent, c) is a child, d) is a grandparent or e) is a grandchild of a concept in the user profile. Then, a similarity function is used to combine these scores in order, for example, to recommend news to a user according to his preference.
Maidel et al. state that if the use of taxonomy is not considered, the recommendation quality significantly drops. Our thesis is that, for education, taxonomy is not enough either, for only subsumption relations are exploited. We thus propose to define the semantic proximity between knowledge elements, based on their situation in the domain ontology. Semantic references are components from an OWL-DL ontology that describe the knowledge in a resource. A few examples of these knowledge references are shown on figure 3 that presents part of an ontology for solar system planets1. They can take six different forms solarSystemPlanet is a class reference (C), Neptune is an instance reference, solarSystemPlanet/hasAtmosphere/atmosphere is an object property reference with its domain and range classes (D-oP-R), Earth/hasSatellite/Moon is an object property instance reference (I-oP-I’), solarSystemPlanet/hasOrbitalPeriod is a data property reference with its domain class (D-dP), Earth/hasNumberOfSatellites is a data property instance reference (I-dP).
We have investigated systematically these 6 forms of OWL-DL references to decide on the nearness of two references K1 and K2. For example, a concept (form C) is near its sub classes, super classes, and instances. It is also near an object or data property (forms D-oP-R and D-oP) that has a domain or range identical or equivalent to this concept. A property reference, with its domain and range (form D-oP-R) is near a sub-property or super-property with the same domain and range. It is also near to a subclass or superclass of its domain and range.
Other criteria assert when a reference K1 is more general or more specific than another one K2. For example, K1 is more general than K2 if K1 is a superclass of K2, has K2 as an instance, appears as domain or range of a data or object property reference K2, or contains an instance in the domain or range of a data or object property reference K2. 3.3
Let us now extend the comparison between ontology components to add the skill (S) and performance (P) dimensions of the competency model. Figure 4 presents a few comparison cases between two competencies C1=(K1, S1, P1) and C2=(K2, S2, P2) in the case where K1 is near K2. Other cases are not considered, i.e. comparison fails.
To illustrate the heuristics, the (S, P) couples are represented on a 2dimensional scale (figure 4). Skills are ordered from 0 to 9 and grouped into four classes as follows: !!0,1", !2,3,4", !5,6,7", !8,9"". Performance indicators are grouped into four decimal levels.
For example, a competency C1 with an analyze skill at an expert level is labeled 5.8 (S1 + P1). A competency C2 at a level 7.2 or 6.4 will 1 Unlike other graphic presentation of ontologies, properties are shown as objects (hexagons) between their domain and range classes (rectangles). It this way, the relations between properties are shown on the same graph. Individual are linked to classes by an “I” link. be considered near and stronger than C1 because the synthesize skill or the repair skill are in the same class than the analyze skill, but one or two levels higher in the skill’s hierarchy. On the other hand, a competency C2 at a level 5.2 will be considered very near and weaker than C1 because it has the same skill’s level but with a lower performance level. Other possible competencies in the “far zone” will be considered too far to be comparable. Also, depending on the relationship between K1 and K2, C2 will be defined as equivalent, more general or more specific than C1. These relations between competencies can also be combined to define more complex relationships. For instance, it is possible for a competency reference to be near and stronger and more general than another one.
Recommendation agents are added to a scenario, linked to some of the tasks called insertion points, as in the example of figure 1. The designer defines these agents by a set of rules. In each rule, one and only one of the actors linked to the task at the insertion point is chosen as the receiver of the recommendation. If a triggering event occurs at run time such as “task completed”, “resource opened”, etc., each applicable rule condition is evaluated and its actions are triggered or not, depending on the evaluation of the condition.
A competency-based condition takes the form of a triple: • Quantification takes two values: HasOne or HasAll, which are abbreviations for “the user has one (or all) of its competencies in some relation with an object competency list”. • Relation is one of the comparison relations between semantic references presented above: Identical, Near/Generic, Near/Specific, VeryNear/Generic, VeryNear/Specific, Stronger, Weaker; or any combination of these relations. • ObjectCompetencyList is the list of prerequisite or target competencies of a task or a resource at/around the insertion point.
Lets take the example of a condition like:
HasAll /NearMoreSpecific / Target competencies for Essay When it is evaluated, competencies in the user’s model are retrieved, together with the list of target competencies for the resource “Essay” The evaluation of the relation “NearMoreSpecific” provides a true or false value according to the method exposed in section 3.3. 4.2
The action part of an agent’s rule can perform one or more tasks: give advice to the actor, notify another actor, recommend various learning resources, update the user’s model, propose to jump to another task or to another learning scenario.
All these possibilities have been implemented. On figure 1, we have presented a scenario with three recommendation agents. For example, Recommender agent #1 on figure 1 will verify if the student has succeeded the second task in the scenario (“Build a table…”). It has 3 rules, shown on the screen-shot of figure 5.
The rule “Update User Model” transfers the list of target competencies for the task to the student’s user model if he has succeeded to build a validated table of planet properties. If he has failed, a second rule will send a notification to the professor to interact with the student. Finally, a third rule provides an advice to the student and recommend consulting a resource shown on figure 5. We have produced an implementation for competency-based assistance that has been tested with a few scenarios and recommendation situations. It provides a proof of concept for the general method. It also provides a workbench to investigate further and extend the methods presented here with variants and a larger range of applications.
First of all, extensive experimental validation will help refine the relation for semantic nearness between OWL-DL references. Adding weights to the various cases would improve the quality of the evaluations. For example, one could assert that a subclass or superclass is closer to a class than its instances or one of its defining properties, especially if there are many defining attributes for this class.
Our model of multi-actor learning scenarios embeds the idea of collaboration between learners, and between learners and various kinds of facilitators. Recommendation for groups in collaborative scenarios has not been thoroughly explored yet.
Finally, to improve the practical use of approach, some of the task will have to be partly automated and the ergonomics of the system improved. Still, the approach presented here sets the ground for an open and flexible method for semantically aware recommendation systems.