The paper presents how the concept and technology of Open Badges are applied in the GRASS project for acknowledging, grading, awarding and recognizing learners' efforts and achievements in developing their soft skills (like self-regulation, leadership, collaboration, skilled communication, problem solving and innovation). Specifically, the paper describes a 4-phase model, called SAGRADA that was developed to support the process of awarding badges in the context of the GRASS project. The core of the SAGRADA model is a set of metrics, carefully developed for specific practical application cases following well-known pedagogical approaches. The metrics enable measuring and tracking learners' efforts in developing their soft skills. The paper also presents a specific application case based on the SAGRADA model, including the preliminary results of the first observational study, and plans for applying learning analytics methods and techniques in further activities of the GRASS project.
The overall objective of the GRASS (Grading Soft Skills)
project (https://sites.google.com/site/llpgrassproject/) is to
enable educators to continuously observe, support, assess,
and acknowledge the development of learners' soft skills
(e.g., critical thinking, skilled communication, leadership,
and teamwork) by leveraging novel pedagogical
approaches and state-of-the-art ICT tools. The project
partners are especially interested in developing a better
understanding of how Open Badges can be used to support,
motivate, grade and recognize development of one’s soft
skills over time in formal educational settings. To this end,
the project explores the use of Open Badges in the context
of developing learners’ soft skills in several different
application cases, in different (multi)cultural settings, at
different levels of education, as well as across different
education levels. In particular, the project partners include
lower secondary, upper secondary, and university level
educational institutions from four different countries.
Soft skills are closely related to the so-called 21st Century
Skills - a broad set of knowledge, skills, work habits, and
personal traits that are considered highly important for
success in today’s world, especially in modern workplace
settings [
GRASS is an ongoing project, but some preliminary results have already been achieved, and they are presented in this paper. In particular, the paper introduces the SAGRADA model that underlies all developments in the project, and provides a demonstration of how the model is used in practice by describing one of the project’s application cases. Since this application case has started only recently, just the initial results of the first observational study are currently available and reported in the paper. By indicating merits of and challenges in the application of the model, the results, even though preliminary are still quite useful for further developments and experimentation in the GRASS project.
The next section describes the SAGRADA model and ICT framework. Then the paper briefly discusses the metrics used to assess learners' soft skills and shows a practical, working example of how Open Badges are used in GRASS including the preliminary results of the first observational study. Finally, it indicates how specific analytic and Artificial Intelligence-based methods and techniques are planned to be used in conjunction with Open Badges to further support learners in developing their soft skills.
SAGRADA stands for SAmpling, GRAding, Displaying and Acknowledging learners' soft skills. It is a model and ICT framework for observing, measuring, assessing, evaluating, awarding, and recognizing/credentialing learners' soft skills, as well as providing learners with appropriate feedback. It is developed as part of the GRASS project and is used in the project as the foundation for supporting and awarding learners with Open Badges. It has four distinct phases.
The first phase, Fig.1a, assumes that teaching and learning practices in an educational institution include continuous use of various ICT tools and services (e.g., e-portfolios, learning management systems, online office tools, wikis, and survey tools/services), as well as didactic tools, such as simulation tools, assessment tools, rubrics, and so on.
Teachers might use them in class, as well as for preparing their courses, for course administration, for grading their students, and the like. Students/Learners might use them (individually, in small groups, or in the class) as learning aids, as exploration tools, aids in completing their assignments, or simply for making their learning more enjoyable. They may use them individually and/or in groups, i.e. collaboratively (e.g., wiki tools).
Another assumption is that a well-defined, open and expandable set of soft skills metrics is available to teachers, so that they can periodically sample (observe and measure) students’ soft skills. The next section discusses the GRASS project approach to such metrics. The metrics definition and guidelines for their use are assumed to be available online for easier access by multiple teachers and for compliance with the institution’s internal standards.
Observations and measurements must be captured and digitally stored to serve as evidence of the soft skills developed and demonstrated by learners. This digital evidence is needed for the subsequent phases.
In the second phase (Fig.1b) learners’ soft skills are
assessed and graded by awarding Open Badges. Teachers
may award certain badges directly, based on the
observations made in the Sampling phase. They may also
use various other badge-awarding approaches such as
assignment submissions, completing steps/challenges,
nominations and/or collecting points; for a detailed insight
into these approaches, see, e.g. [
Open Badges always include evidence of the achievements they are awarded for. Here the evidence comes from digital traces of the sampling phase and from those of the badgeawarding approach used in this phase. Just like the sampling phase, this phase also assumes the existence of soft skills metrics. The evidence is directly related to the metrics used in the sampling phase, and is indirectly related to them through the badge-awarding criteria used in this badge awarding phase (since the criteria are built on top of the metrics).
Once learners have earned some badges for soft skills, they can display them in their backpacks, on their home pages, job profiles, social networking sites, and other online places of their choice (Fig.1c). Note that badges for soft skills can nicely complement those earned for other achievements (i.e. for “hard” skills demonstrated in specific subject areas), and thus lead to a more comprehensive learner profile. Also, badges for soft skills can represent general skills (e.g., "critical thinker"), or can be contextualized (e.g., "critical design analyst"). This implies that the metrics for soft skills and the related badge-awarding criteria must be carefully designed to facilitate categorization of badges for soft skills as general and contextualized; this is helpful when learners decide about the badge collections they make and share.
Peers, teachers, parents, representatives of other educational institutions, employers, government bodies, and other stakeholders can access a learner’s public badges (Fig.1d). Badges earned for soft skills development can reveal the learner’s personal development (persistence, self-regulation, problem solving, divergent and lateral thinking, creativity, etc.), as well as her/his social participation (e.g., collaboration, communication, negotiation, teamwork, networking, leadership, emotional awareness, differentiation of contributions).
Metrics for development and assessment of soft skills are present in this acknowledging (recognition) phase of the SAGRADA model, as well. For example, if an employer wants to hire a team leader, clicking the evidence and criteria links on an applicant's badge earned for the leadership skill will reveal the underlying metrics (provided, of course, that the badge earning criteria are carefully designed and that all the relevant evidence is made available). The metrics in the criteria and in the evidence might not be displayed in their raw form, but will certainly enrich and detail the applicant's profile. A similar example is that of letters of recommendation that are usually required as part of the application procedure for degree programs at colleges and universities. Referees are often required to briefly assess the applicants' soft skills in letters of recommendation that they write. In such a letter, a reference to the applicant's collection of badges for soft skills is a concise, yet very effective way to reveal the evidence and the metrics used for the assessment of the applicant’s soft skills.
Fig. 1 - SAGRADA model (a) Sampling (observing, measuring) soft skills (b) Grading (assessing, awarding) soft skills
(c) Displaying (sharing) soft skills (d) Acknowledging (recognizing, credentialing) soft skills
Note that the four phases of the SAGRADA model are
typically iterative since soft skills are often developed and
improved iteratively and incrementally, through feedback
and formative assessment, and over extensive periods of
time [
Badge earning and the social recognition that badges tend to bring can act as important incentives for further improvement of one’s soft skills.
At the time of writing this paper, the GRASS project experiments with issuing badges for development of soft skills at 8 partner institutions from 4 European countries. Note that only two partners teach soft skills in designated courses; the other partners teach their regular courses but incite development of soft skills in the context of their courses.
The development of soft skills is measured differently in
each specific application case, i.e., in a specific course in a
partner institution. To address this diversity, the project has
developed a rich, structured set of general soft skill rubrics
to serve as dynamic indicators of the learners' ability to
develop, apply, and improve their soft skills. These sets are
all available online [
To set up an initial measurement model and the associated
set of metrics, the project team relied primarily on the
guidelines and rubrics defined for measurement and
assessment of 21st Century skills [
There are eleven different application cases in GRASS (one or more per partner institution). Some of them are currently using the same Web site to register learners, set badgeearning tasks and challenges, let the learners work on these and make submissions, and award badges; the others run their own installations of badging platforms. The number of registered users per application case ranges from a few dozens to about one hundred.
Interested users can try the GRASS badging platform at http://badgeos.fon.bg.ac.rs/ with the designated test account (username: Han, password: Han). The platform is based on the BadgeOS™ plugin for WordPress. Since the process of active experimentation with the GRASS badging framework has started only in Nov 2014, there is still work to be done in developing a badging system for each application case. Still, badges pertaining to the UB application case are available1. Even though these badges are associated with solving some Java programming tasks, they are awarded for the demonstrated soft skills, not for Java programming expertise. Students can push the earned badges to Mozilla Backpack using the "Backpack" menu (currently not supported for test users).
The badges the students get from GRASS are awarded by the GRASS project as the badge issuer.
The first observational study with students earning GRASS badges in the UB application case was conducted in the second half of Dec 2014. 59 BSc students and 23 MSc students were introduced to the project and asked to try to (optionally) earn some of the soft skills badges for Java programmers. They were also asked to give some short personal comments in their submissions about the badges, the badging process, and about the idea of badges as skills recognition mechanism in general. Both MSc and BSc students were asked to complete their programming assignments associated with badges within two weeks.
The study was designed to walk the students through the phases of the SAGRADA model (1 cycle only). The metrics used for sampling their interaction with the programming tools, as well as interaction with peers and the teacher during the two weeks of their time with the GRASS badging platform were related to the soft skills of enthusiasm, initiative, problem solving and critical thinking: the number and frequency of interaction with the tools, the number of meaningful discussions they have initiated with the teachers related to the badge-earning challenges, the number of meaningful comments they have made about the challenges, etc. In the badge awarding phase of the SAGRADA model, the teachers combined different measurements from the sampling phase before approving the students' submissions for badges.
Table 2 shows some of the results obtained two weeks later. Much more enthusiasm and interest from BSc students are apparent immediately. One reason for the obvious lack of interest from MSc students might be that they are all employed in the IT industry (in Serbia), which might have made them doubtful that badges for soft skills can really "work" there. This is something we intend to explore in our further work on this application case. 1 Badges listed under the "Badges" menu; since these badges require some experience in Java programming, interested users who lack programming experience can submit "OBIE 2015 test" to be awarded badges (as soon as the system administrator receives their submission). 1With that metric value (or higher), the student is a candidate for a badge in the corresponding achievement category. 2Collaborative software development project, group of 3-4 students, estimated effort 8 person-hours. 3Non-empty lines of code include new lines of code, but also modified or deleted lines of code. badges the students earned were demonstrated to groups of students and teachers who had not participated in the study, asking them to take the roles of "other stakeholders" (for feedback purposes only). Both the displaying and the recognition phases have brought useful insights for further developments and experimentation in the GRASS project (see Table 2). In addition, the entire study was extremely helpful to the teachers involved in terms of adjusting the metrics threshold values, since some of the values set initially have proven to be over- or underestimated.
In GRASS, there is a need to analyze students' interaction
with various online learning tools (SAGRADA model,
sampling phase) where they can demonstrate their soft
skills. In addition, the project intends to analyze students’
interaction with Open Badges in terms of taking different
learning paths, as well as their interaction with teachers
and peers in the context of earning Open Badges through
the development of their soft skills (badge awarding phase
in the SAGRADA model). Thus the project partners
intend to use learning analytics methods and techniques to
identify badge pathways in each application case, i.e.,
typical learning paths that students take when striving to
earn badges related to the attainment of a certain soft skill
[
Project partners have considerable experience with different methods, techniques and tools for intelligent data analyses, especially those based on production rules and
students who made submissions for badges
BSc 59 22 good idea for learning Java; good good for motivation; idea; good for future employment; gaming good for skills recognition; strategies university should adopt it; should not sure how badges go be together with formal grades; applied; what if skills recognized by badges become obsolete? The displaying phase of the SAGRADA model seemed to be very intriguing, since many of those who have earned some badges wanted to discuss possible badge collections in their backpacks and further badge sharing opportunities. The recognition phase of the SAGRADA model was attempted only in a minimalistic way: the fuzzy logic (Drools2, jFuzyLogic3, JEFF4), as well as those based on neural networks (Neuroph5). Still, the use of these methods and techniques in the context of GRASS requires more time with running the application cases and more data to be collected from them.
However, scenarios for data analysis and feedback
provision have been carefully considered when
developing the SAGRADA model and the set of soft
skills metrics. For example, metrics and indicators related
to a particular soft skill are intended to be used as a part
of a rule-based system that would be fed by the data
(observations and/or measurements) collected in the first
phase of the SAGRADA model (sampling soft skills).
Such a system would help a teacher to determine the level
of the soft skill demonstrated by his/her students (the
second phase of the SAGRADA model). The system will
be based on a set of interconnected If-Then rules
developed on top of the specification of metrics for soft
skills [
In addition, a system based on a neural network is intended to be used to gain further insights into the development of soft skills and their mutual connections, ultimately enabling detection of, e.g., soft skills that should or should not be taught/developed concurrently. Such a system can be adaptively configured (trained) to reflect the teacher’s own instructional approach and practice.
The ongoing GRASS project deploys Open Badges for encouraging, monitoring, grading and recognizing learners' soft skills in several educational institutions in Europe. The process of awarding badges in the context of the GRASS project is based on the proposed SAGRADA model of sampling, grading, displaying and acknowledging learners' soft skills. The model is aimed at supporting monitoring and inciting continuous development of one's soft skills. It has already been deployed in the project, and the first preliminary results with two different groups of students are discussed in the paper. The project has also developed scenarios for deploying various analytic and AI-based approaches to further support the development and assessment of soft skills.