=Paper=
{{Paper
|id=Vol-1601/CrossLAK16Paper3
|storemode=property
|title=One Competency Data Model to Bind Them All
|pdfUrl=https://ceur-ws.org/Vol-1601/CrossLAK16Paper3.pdf
|volume=Vol-1601
|authors=Jeff Grann
|dblpUrl=https://dblp.org/rec/conf/lak/Grann16
}}
==One Competency Data Model to Bind Them All==
https://ceur-ws.org/Vol-1601/CrossLAK16Paper3.pdf
One Competency Data Model to Bind Them All
Jeff Grann, Capella University, jgrann@capella.edu
Abstract: Although students regularly demonstrate career-relevant competencies in post-
secondary programs, it is difficult to document and communicate these accomplishments
beyond the institution. Portfolios and microcredentials are intended to help solve these
problems, but neither have been widely adopted or regularly expected amongst hiring
managers. This paper speculates that a significant barrier to adoption has been the absence of a
shared competency data model for educators and human resource professionals. A
competency data model used to solve educational technology interoperability problems is
presented. Discussion highlights synergistic projects that, if connected via a shared
competency data model, could help adults advocate for their career goals using evidence from
multiple contexts.
Keywords: competency-based education, data model, transcript, learning analytics
“The diploma, by remaining tied to no standard other than credit accrual and seat time,
provides no useful information about what students have studied or what they can actually do
with what was studied.” (Wiggins, 1989).
Introduction
The field of learning analytics suffers from a dearth of student learning data. Current research relies on indirect
proxies, such as a student’s grade point average, particular course grades, or test performances, instead of
authentic demonstrations of important competencies. For example, 70 percent of the fifty-one empirical studies
included in a recent literature review used GPA as a dependent variable, with the remainder consisting of self-
reports, credits, and exams (Gray, McGuinness, Owende & Carthy, 2014). Such indirect measures may yield
valuable findings relative to institutional productivity, but they are insufficient to provide information about the
processes and correlates of student learning for several reasons. Grades summarize performance relative to
multiple course objectives, intermixing student learning with attendance, timeliness, engagement, punctuality,
and other factors. Grades are also distributed according to a fixed, predetermined academic calendar that is not
sensitive to each student’s specific learning curve. Moreover, historical research has demonstrated grade
inflation patterns over the past several decades that don’t track with increases in students’ study habits or
measured learning outcomes (Arum & Roksa, 2011). Presumably, learning analytics researchers primarily rely
on weak measures of student learning because better measures are not readily available.
The resurgent interest in competency-based education (CBE) has the potential to rectify the
shortcomings of grades for learning analytics research by providing programmatic, direct measures of student
learning over time (Johnstone & Soares, 2014;Ford, 2014). In CBE programs, students advance based on
demonstrated learning rather than on seat time or attendance. Needless to say, assessments are important in CBE
programs. In mature CBE programs, faculty members, subject matter experts, and specialists collaborate to
design assessments that simulate anticipated performance conditions and require students to draw upon multiple
competencies simultaneously (McClarty & Gaertner, 2015). In the United States, the federal Department of
Education has publicly encouraged institutions to develop CBE programming; current estimates project that
about six hundred institutions are either currently designing or already implementing such programming (Fain,
2015).
CBE Interoperability
Several educational technology barriers have limited the growth of CBE. Established CBE programs have long
managed these problems locally by developing custom applications and services, but new entrants can find these
barriers formidable. The Technical Interoperability Pilot (TIP) project surveyed mature CBE programs and
revealed several common interoperability barriers to CBE programming (Leuba, 2015). In response, IMS Global
and the Competency-Based Education Network issued five formal requests for proposals and ten educational
technology companies were selected to develop prototype solutions (see Table 1 for a list of contributing
10
Copyright © 2016 for this paper by its authors. Copying permitted for private and academic purposes.
vendors and use cases). Over a ninety-day period in 2015, these vendors collaborated with one another to
prototype interoperability solutions using a shared competency data model (1). These solutions are envisioned to
help institutions focus on serving the needs of their students and not integrations amongst multiple educational
technologies (2).
Table 1: Vendor participants for CBE use cases and shared data model
Use Case
Vendor Manage Evaluation Program Substantive Record of
Competencies Results Information Interaction Performance
Blackboard X
D2L X
Ellucian X X X
Workday X X X
eLumen X
Flatworld X
Oracle X
Regent X X
Accreditrust X
Learning X
Objects
Properties of a Shared Competency Data Model
A key deliverable of the 2015 CBE interoperability prototypes was a shared competency data model (See Figure
1). This model was used to define a JSON-LD web service supporting the transmission of a student’s extended
transcript (3). Relative to the argument of this paper, there are four key properties of the competency data model
that support its flexibility for CBE interoperability within an institution and potential data integration schemes
that go beyond the institution.
Competency hierarchy
Most CBE curriculum models contain a hierarchical structure of competency statements. Some products support
hierarchical relationships, but they do so in idiosyncratic ways that are specific to each product. The data model
supports competency hierarchies using simple parent-child relationships via “isChildof” and “hasChild” fields.
Another key feature of the data model is the inclusion of a “reference hierarchy” entity that is envisioned to
describe a particular competency in relation to common frameworks. By supporting competency hierarchies in
the data model, vendors help institutions sustain coherent programming across many technology tools for
student-level reporting.
Competency type
Many analysts have noted that CBE programs use inconsistent terminology across institutions. Educational
technology products have addressed this inconsistency by using very generic language in their products, which
rarely supports any program well. The data model supports institution-specific terminology via a “type” field in
the competency table. By supporting this feature of the data model, educational technology vendors empower
institutions to sustain and control their curricular models.
Competency code
Many technology products have proprietary mechanisms for storing competency statements. Institutions
utilizing multiple technology products for CBE programming often need to reference the same competency
statement in different products, such as when assessing students using one tool and storing competency results
in another tool. The prototype team envisioned solving this challenge by using a “competency code” with an
associated effective date for versioning. A competency code is a logical reference to the full competency
statement. It is analogous to a course code, such as PSY101 as a reference to an institution’s introductory
psychology course. Competency referencing via a competency code and effective dates will allow an institution
11
to reference the same competency statement across multiple products without relying on increasingly elaborate
synchronization services for unique ids and versions.
Competency scores
CBE programs need to aggregate student-level data for particular competencies across multiple assessments and
varied time periods. The data model supports such aggregation via a competency score table indexed by a
competency_offering_map table. This approach permits institutions to report student-level competency scores
based on the institution’s assessment strategy for each competency. For example, one competency may be
assessed in multiple courses, whereas another competency may be assessed in only one specific course. The
competency score table also supports competency-specific calculation methods to provide programming
flexibility.
Figure 1. Shared Competency Data Model
12
Speculations on CBE Data Integration
A shared competency data model not only helps institutions deploy their programming; it also advances a
system for connecting higher education institutions with employers via digital credentials. For over a century,
the student transcript has maintained its connection to the Carnegie unit standard for credits (Silva, White &
Toch, 2015), but this focus is now being supplemented at many institutions with an extended transcript (eT) that
includes detailed competency demonstration information (Black, Leuba, Owczarek, Parks & Shendy, 2016). The
eT is intended to help students conceptualize their educational experience in a richer framework than what may
be possible with courses, grades, and credits. Such metacognitive reflection may promote the intentionality and
motivation some students need to complete their degree programs. For example, there is some evidence that
utilization of a competency map dashboard is associated with higher rates of persistence, even after controlling
for several powerful covariates (Grann & Bushway, 2014). The eT is also intended to help students advocate for
their career goals with hiring managers.
Because these extended transcripts are secure digital documents (specifically packaged via JSON-LD),
they could also serve as a basis for a broader data integration scheme. Students regularly participate in a variety
of civic, volunteer, and employment contexts in which career-relevant competencies are acquired and
demonstrated. While not common practice, some employers are also articulating key competencies for positions
in their organizations (Franklin & Lytle, 2015). Currently, these accomplishments are documented individually
on paper via a résumé or a curriculum vitae. While some companies have created text-mining algorithms to
decode and digitize these text documents for analytic and recruitment purposes (such as Burning Glass and
CareerBuilder), a far more efficient solution would be to standardize competency demonstration evidence using
a shared competency data model.
Three synergistic projects make the prospect of a shared competency data model more probable than it
might seem. The Credential Transparency Initiative (CTI) is focused on building a modern credential registry
using a consistent vocabulary (4). CTI intends to include competencies in its credential domain model and to
provide this information using linked data. Similar to IMS Global for educational technology, HR
OpenStandards tries to solve interoperability problems with human resource technical systems (5). Based on a
cursory analysis of HR OpenStandards, a competency data model is used to support hiring and performance
management functions; this seems similar in structure to the shared competency data model. Because both of
these models rely on linked data relationships packaged via JSON-LD, an equivalency between the competency
data models can be established (Allemang & Hendler, 2011) to link the evidence on an extended transcript with
CTI’s credential registry and the competency model used in HR OpenStandards. Moreover, these relationships
linking higher education and employers can operate in both directions and provide a new mechanism to formally
recognize documented competency demonstrations occurring within employment contexts, such as performance
reviews. The Lumina Foundation has also supported the development of a Beta Credentials Framework that
provides a metadata schema for describing competencies in a consistent manner (6). Although only now being
field tested, this framework is conceptually consistent with the competency data model’s concept of “reference
hierarchy” and, if broadly adopted, could help establish much-needed competency equivalencies and exchanges.
If these promising initiatives could be integrated via a common data model, higher education has the potential to
establish a lifelong learning digital credentialing system across multiple contexts that empowers students to
articulate their learning outcomes and advocate for their career goals.
Endnotes
(1) The IMS Global website provides more context and information about this project at the following link:
https://www.imsglobal.org/initiative/enabling-better-digital-credentialing.
(2) The following video provides an overview of the five CBE interoperability barriers and developed prototypes:
https://www.youtube.com/watch?v=Eo5cnT6QVP4
(3) A working prototype of the extended transcript is available at the following link: https://demo-cbl.difference-
engine.com/extended-transcript/.
(4) More information about the Credential Transparency Initiative is available at the following link:
http://www.credentialtransparencyinitiative.org/.
(5) More information about HR OpenStandards is available at the following link: http://www.hropenstandards.org/.
(6) More information about the Beta Credentials Framework is available at the following link:
https://www.luminafoundation.org/resources/connecting-credentials.
13
References
Allemang, D., & Hendler, J. A. (2011). Semantic web for the working ontologist: Effective modeling in RDFS
and OWL, second edition (2nd ed.). Waltham, Mass.: Morgan Kaufmann.
Arum, R., & Roksa, J. (2011). Academically adrift: Limited learning on college campuses. Chicago: University
of Chicago Press.
Black, T., Leuba, M., Owczarek, S., Parks, R., & Shendy, J. E. (2016). Seven things you should know about the
evolution of the transcript. Educause Learning Initiative.
Fain, P. (2015, Sept. 10th, 2015). Keeping up with competency. Inside Higher Ed.
Franklin, C., & Lytle, R. (2015). Employer perspectives on competency-based education. Washington DC:
American Enterprise Institute.
Grann, J., & Bushway, D. (2014). Competency map: Visualizing student learning to promote student success.
Paper presented at the Proceedings of the Fourth International Conference on Learning Analytics and
Knowledge, Indianapolis, Indiana, USA. pp. 168-172. doi:10.1145/2567574.2567622
Gray, G., McGuinness, C., Owende, P., & Carthy, A. (2014). A review of psychometric data analysis and
applications in modeling of academic achievement in tertiary education. Journal of Learning Analytics,
1(1), 75.
Leuba, M. (2015). Competency-based education: Technology challenges and opportunities. Educause Review,
50(5)
McClarty, K. L., & Gaertner, M. N. (2015). Measuring mastery: Best practices for assessment in competency-
based education. Washington D.C.: American Enterprise Institute.
Silva, E., White, T., & Toch, T. (2015). The Carnegie unit: A century-old standard in a changing education
landscape. Stanford, CA: Carnegie Foundation for the Advancement of Teaching.
Wiggins, G. P. (1989). Teaching to the (authentic) test. Educational Leadership, 46(7), 41.
14