Competency based learning and e-portfolios are integral parts of modern teaching repertoires. Media and computer technology play an important role in supporting such scenarios. This paper presents an infrastructure consistent of a generic competency database that can be connected to various learning management systems or e-portfolio systems to implement competencebased assessment. The concepts shown in this paper are an important step towards supplementing traditional ways to mark students. Students' achievements are modelled as the acquisition of competencies and represented as such.
Schools have relied on grading their students using ordinal scales or rational scales.
Depending on the educational system they might use percentage grading (Ontario) or
plain numbers from one to six (Germany). There are some concerns about the
educational value of marks in principle, because the coefficient of predicting adult success
is very low [
Grading like other descriptive actions depends on the technology available at the
time. Historically, distances have been measured in feet and sticks because lasers,
GPS or triangulation were not invented yet. Similarly, educational progress has been
described differently throughout history. Grading students in a modern sense using
ordinal scales has only been around since the 20th century [
If we agree with the critics of numerical grading, supplementary assessment concepts are needed. This paper tries to answer the question as to whether grading by using numerical scales could be attributed to lacking technologies. It also deals with the question of how alternatives could be achieved. Assuming it is possible to store students’ competencies electronically on a detailed level, a set of competencies validated by the organization and presented in e-portfolios may be a way of supplementing or even replacing grades. This paper tackles the research question by looking at ways to store competencies consistently for a larger number of people (or organizations) and in a generic fashion. This is a prerequisite for the bigger task of creating a competency based assessment and thus sets the scope of this paper.
The paper is structured as follows. First we look into the current state of research with a focus on (technical) competency models that help to store competencies in a generic fashion. Next we present the competency model developed. Then we present the implementation of the competency database and explain the intended usage including a process model. 2
The Oxford Dictionary of Education defines competencies as “The ability to
perform to a specified standard” [
For instance, the industrial approach sees effective management of competencies
as a way to foster human resources development [
Another approach focuses on the idea that learning objects such as assignments or
documents contain useful metadata [
The COMBA model [
That is why we decided to conceptualize an improved competency model based on existing standards that is generic enough to be useful for any domain and which can be persisted in an competency database in order to allow for reasoning and interinstitutional usage 3
One possibility to model competencies is to integrate them into the learner model.
This way competencies are described as aspects of the situation, the intended learning
outcomes and the dimensions of the indicators [15, p. 248]. Linking the evidences to
the activity model like the LMS Learning Object Metadata standard (LOM) [
The model is generic in terms of looking at aspects and relations of competencies (structural perspective) instead of taking the perspective of the curriculum, faculty, subject or school. Facing diverging pedagogical cultures in different disciplines this is the only way to generalize the concept. The only restriction is based on the earlier definition that a competency must be visible in terms of assessment and that can be formulated in a sentence. It must relate to a performance in the real world. Teacher education classes at our institution have created a de facto standard for formulating competencies this way to simplify the analytical process of extracting metainformation out of the formulated competency:
– Computer science students : implement : service oriented architectures (by programming : webservices) (by designing : interfaces) – Computer science students : create : java programs (by using : dependency injection)
This approach does not impose any requirements regarding the actual domain. It is used in subjects as different as political science and computer science. It also means that competencies are not computed as a result of activities. They are inserted independently. They derive from the curricula or the teacher’s internal course structure. By clearly separating the activity model and the competency model this way it becomes possible to create a big pool of competencies that can be used in various applications. 4
Primarily, we present the features developed. After that we explain the software prototype based on an activity model. The development of a new grading scale presented here is only one application of the database created. As a software framework it stands for itself and can be used in all kinds of human resources oriented applications. Nevertheless, we focus on this use case because it shows the full innovative potential.
The following list highlights some of the features we have developed: – The competency database – Excel import of competencies – Web services (SOAP, SPARQL and REST) – Plugins for learning management systems
Our goal was to implement a prototype that can be used as a read-only competency database. The excel import allows us to role it out at the institutional level. Web services are required to connect the prototype to other software systems like the campus management system or the library. Independency was also reached with the user interface by abstracting from the context with a mixture of webservices and Javascript injection. Most importantly, the persistence layer was implemented using an rdf/owl format stored in a high performance triple store.
Linking competencies to actual performances has been implemented in a very general way. Evidence links may point to websites of the e-portfolio or tasks achieved in a learning management system. This leaves it to the teacher to decide what counts as an evidence. This could even be a link to a website showing exam results which means coming full circle concerning the grading concept. All the systems providing evidences for a competence must implement a webservice that requires a link and a readable name. Showcase implementations include Moodle (LMS system) activities and Liferay (portlet container) group activities. The figure 1 on page 5 shows the implemented system from a component perspective with the example of a university as a learning organization (instead of school or company).
The first step is to agree on a set of competencies and their structure for a given organization. Most of the time a curricula is given which can be used as a starting point. However, in order to arrive at a useful level of detail the teachers in the organization have to enter their course structure in terms of competency graphs into the database. Both specifying the learning goals and ordering them in in terms of learning trails formally are time consuming. Here it is up to the organization to enforce compliance. One could argue that good classes should always be planned properly but reality may be different. One of the advantages of the approach presented in this paper is that once this has been done for an organization this can be used by similar organizations. Existing competency schemes can be used and pooled here. For instance, the European Union has created a competency matrix for language education. Schools have internal curricula in form of pdf documents and universities, too. Using a database capable of full text search is an advantage here even if the process is halted at this point. If competencies are linked to the teacher’s knowledge and perspective the consistency is improved at the institutional level because the teachers’ implicit assumptions are made transparent to their colleagues. Instead of saying that course A requires course B, course A now requires a set of competencies which are linked to other courses giving the student the chance to cherry-pick the courses they like. This reduces the number of students complaining that they are learning certain things twice or that courses do not match their needs. Another advantage is the possibility to tag educational contents to make them more accessible in different scenarios. Having a standard set of activities or lessons ready in order to teach a certain competency may improve the level of teaching and the redundancy of teachers’ preparation. This kind of high level semantics cannot be reached with other approaches like machine learning or rule based systems.
One of the logical consequences of improving transparency is the visibility of conflicts in teaching methodology and basic assumptions. This may cause problems. If two teachers have radically different views on a subject, they will enter conflicting facts into the competency database which need to be moderated. Since this is an organizational problem, it can only be solved by stimulating dialogue between the colleagues in question. Depending on the flexibility of the organization this poses some challenges.
The second step is the actual usage of the competency database for assessment. There are two roles that can be differentiated: the teacher who evaluates the portfolio and the student producing the digital artifacts. We look at the teacher’s perspective first. The process is visualized in figure 2 on page 8.
Select competencies: The first step is for the teacher to select a set of competencies relevant for the course. Only the selected competencies are visible in the course context. Assuming the number of competencies in the repository grows it becomes necessary to filter them intelligently. Moreover, the teacher has the option to mark certain competencies mandatory for the students to pass the course. In the example of an university this done most of the time within the campus management system. Using the competency database, it simplifies the process of creating module descriptions as older competency templates can be reused. Also big changes compared to earlier courses can be visualized as students and pupils might base their decisions to take classes on experiences of older students.
Define order of acquisition: In a subsequent step the order is defined in which the competencies are supposed to be acquired and linked. This way the student is presented with a smaller set of competencies he or she can adapt his or her studies to. This information can be used to structure classes and improve the match of the course level compared to the students’ knowledge and skill set. This is usually done within the campus management system or offline. However, current systems, especially in big organizations, struggle when taking into account cross-disciplinary redundancy in classes.
Link evidences to activities: This is the default state of the system. Students produce electronic artifacts during their studies. These are linked to the competencies with the effect that follow up competencies are presented. This takes place within the e-portfolio system or even learning apps designed to allow the student to reflect his or her progress. We currently work on apps tailored to this purpose, too.
View progress of users: As soon as an activity is linked to a competency it is marked as evidence for this particular competency. The sum of the competencies linked compared to the number of competencies selected earlier shows the overall progress the student has made. Filtering the competencies will result in partial progress view. This way students are not graded in comparison to their peers but in view of the possible degree of competence they can reach in the organization overall.
View evidence links of user: Finally, evidences can be validated, invalidated, commented on or deleted by the teacher. This is the point where traditional evaluation methods are needed to qualify the teacher’s decision concerning the evidences.
From a student’s point of view the system behaves very similarly. Students are not allowed to influence the selection of competencies or their ordering. Students are, however, allowed to link the competencies themselves or for their fellow students to make the process more engaging and to lift some of work from the teacher. The teacher does have the option to counteract problematic behavior by invalidating or deleting evidence links. Furthermore, students are allowed to view the progress of their peers and comment on the decisions the teacher has made concerning the validation.
The process passes through several systems that all deal with the same competencies. This improves the knowledge management within the organization if the assumptions regarding the learning goals are externalized and made accessible at every state of the process. This addresses existing needs within most bigger learning organizations not mentioning the pedagogical value expressed earlier. 6
We argue that when leaving school a set of formulated competencies may be more useful in describing a student’s skills then grades. An interesting argument worthy of a conclusion is that grades are negative (’you only have reached this percentage compared to a perfect student’) whereas competencies much like levels in computer games are positive (’you have reached this level of competency in these subjects’) and more constructive (’you are missing these sub-competencies to be as good in programming as Student X’). However, if the competencies formulated are meant be equally useful in comparing students as the numerical scale of grades there is a need for a standardized assessment and description framework based on competencies. First complete descriptions are available for selected disciplines.
From a pedagogical point of view we analyzed competency based learning, portfolio learning and merged the concepts. Competence based learning offers the chance to grade students with a badge like tag system instead of marks. This way transparency and consistency of the curriculum can be improved. Portfolio learning offers the tools to use an equally generic assessment scheme to go along with the generic competency model. This way activities can be linked as evidences for competencies independent of the subject being taught. The main challenge was the creation of a competency model that conceptually as well as technically could be used in all subjects and organizations (generality). Instead of empirically determining categories we look at competencies (or learning goals) as a plain text formulated concept of what a student should learn. A grammar for formulating standardized competencies was proposed that supplements the enhanced competency model developed. The presented competency model includes inheritance and recursive competencies which makes the competency model more generic then those currently available. Whereas some of these ideas have been present in technological standards for some time we implemented a competency database as a reference in order to make the concept more useful in practice. Here we have achieved what we think is a generic competency database. The second challenge was ensuring consistency and re-usability. Here a lot of technological questions arose concerning the computerlinguistic comparison of plain text competencies, using reasoning to validate the internal logic of learning graphs and developing competency based recommender systems. We have made some progress but it needs a lot of work still.
Next we will facilitate inserting new competency schemes and try to tackle the pending research questions discussed. Furthermore, we are currently developing frontends and plugins for popular learning management systems in order to make the system easier accessible for other organizations. If this is successful, the research in semantic technologies will lead to a supplementary method of grading academic achievement on a nominal scale. This way we strive for a meaningful way of grading students’ progress.