Assessing competences for digital creativity Kathy Kikis-Papadakis1 and Foteini Chaimala2 1 Foundation for Research and Technology-Hellas (FORTH), Greece 2 Foundation for Research and Technology-Hellas (FORTH), Greece Abstract. This paper deals with assessment issues pertaining to digital creativi- ty in educational contexts. Following a short introduction about the concept of digital creativity, the first part of the paper discusses tensions and challenges in its assessment– rooted on the complexity of the interplay between creativity and digital technologies. The focus is then turned on teaching competences for digi- tal creativity from an assessment perspective. Presented are areas and compo- nents of the DoCENT framework for digital creative teaching competences, re- lated to the assessment of the teaching and the learning process - namely the ar- eas “creative assessment” (i.e. the use of digital strategies to assess students’ creativity), and “learners’ digital creativity” (i.e. the competences required to enhance students’ digital creativity). A way to operationalize the DoCENT framework for self-assessing purposes through rubrics tool under the five-level conscious competence learning model is then proposed. Concluding remarks re- flect on implications of the work presented for further research and study, out- lining the need to explore the effects of assessing teaching competences for dig- ital creativity on the promotion of students’ creative mind-set. Keywords: Digital creativity, Teacher competences, Formative assessment 1 A Snapshot to Digital Creativity Creativity is considered as a key competence for education, as expressed in EU policy rhetoric, evidenced in national curricula and supported by recent research and aca- demic discourse. EU strategic frameworks have set the enhancement of creativity as a strategic objective at all levels of education and training [1], and EU policy docu- ments outline recommendations for promoting creative ways of teaching and learning [2]. At policy mediation level, creativity is explicitly referred in all national curricula of the EU27 [3]. In addition, recent research shows that teachers consider creativity as a relevant cross-curricular competence; nevertheless, they find it challenging to apply practices, assessments, and technologies to support its development [4]. In recent years, rapid advancements in digital technologies have caused researchers and practitioners to rethink more traditional ways to support the development of stu- dents’ creativity. Henriksen et al. refer to the reciprocal relationship between creativi- ty and technology, mentioning that “technological change is driven by human creativ- ity, and in turn provides new contexts and tools for creative output’, and suggest that Copyright © 2019 for this paper by its authors. Use permitted under Creative Com- mons License Attribution 4.0 International (CC BY 4.0). 2 teaching and learning should emphasize their connection [5, p.28]. Currently there is consensus that the affordances of technologies may have a strong influence on crea- tive processes and achievements. Loveless for example, outlining the potential roles of digital technologies in supporting creativity mentions that “digital technologies can be tools which afford learners the potential to extend or enhance their abilities, allow users to create novel ways of dealing with tasks which might then change the nature of the activity itself, or provide limitations and structure which influence the nature and boundaries of the activity” [6]. Nevertheless, understanding the interplay between “digital” and “creative” still appears as a challenge in academic and research dis- course and teaching practice. The notion of digital creativity expresses the concern to consider issues pertaining to creativity and digital technologies in tandem. Broadly speaking it can be defined as working creatively within a digital medium [7]. In a more comprehensive definition - digital creativity refers to a “purposive imaginative activity, mediated by digital tech- nologies, generating outcomes that are original and valuable in relation to the learn- er” [8]. As applied to education, digital creative teaching consists of applying digital technologies with the aim to support creative pedagogies, i.e. learner-centred ap- proaches, open-ended ethos, synergistic collaboration and knowledge connection. 2 Tensions and Challenges in Assessing Digital Creativity Dealing with assessment issues relating to digital creativity in educational contexts is complex and challenging. Difficulties in assessing digital creativity mainly emerge from the challenge to conceptualize creativity and digital technologies in tandem, and the complex interplay between the two notions. The assessment of creativity per se involves tensions for different reasons, which are mainly rooted both on the dispense notion of creativity and on decisions need to be taken relating to the assessment purpose. Definitional challenges that may be faced by educators trying to assess ‘creativity’ can be illustrated by the questions: Person – is the student a creative person and have his/her learning experiences made thin/her more creative? Process – is the process through which the person learnt or the product was made creative? Product – is the essay or other piece of work creative? [9]. On the other hand, questions relating to the purpose of assessing creativity may include: Do we want measures of creativity for summative assessment purposes, for structures in which quantifiable outcomes are valued (for example national examinations)? Do we want to be able to assess creativity for formative purposes in order to give feedback to individuals on their achievements and ways forward for progression? Do we want to have means to recognise meaningful and original personal expression? [10] According to Henriksen et al. ‘the theme of creativity and assessment revolves around the challenge of navigating tensions in evaluating and assessing creativity’ [5, p.34]. Their study identifies the following dichotomous tensions: Psychometric vs. behavioral (i.e identification of psychometric characteristics of creative individuals, such as cognitive flexibility vs. behavioral measures such as alternative uses test); process vs. product (i.e. focusing on the learning process for creative solutions vs. 3 focusing on the output of the creative activity); individual vs. group (i.e. how to en- gage students in collaborative open-ended projects that support creativity, while also assessing individual performance?); domain general vs. domain specific (i.e. whether creativity is located specifically within domains or whether it is something more gen- eral and extendable). Challenges that arise from decisions to be taken by educators in relation to these tensions include: Exploration and use of a range of alternative as- sessment formats and technics that consider how technology and creativity interact; Definition of criteria for assessing both the creative process and the product of the creative activity; Contextual challenges (e.g. formal-informal settings; disciplinary- transdisciplinary – multi-disciplinary contexts). The assessment of students’ digital capabilities for creativity is no less challenging. Starting points for the discussion of assessing creativity and digital technologies have been offered by Jonassen [11] - who suggests dimensions for assessment along a range of criteria and Sinker [12] - who offers a detailed discussion of the issues asso- ciated with evaluating young people’s creative multimedia production. Nevertheless, work on assessing creativity and digital competences in tandem is still in its infancy [13]. The use of digital technologies for enhancing students’ creativity raises ques- tions about the evaluation and judgement of creative processes and products that are different from more traditional tools. The main challenge according to Loveless (2002) is that it is difficult to find correlations between the use of digital techniques and their attainment in other subjects (for example to identify the indirect effects of the use of ICT on motivation, attitude, and problem solving capability, critical think- ing and information handling abilities) [10]. 3 Teaching Competences for Digital Creativity: the DoCENT project framework For educators to be able to deal with assessment challenges for digital creativity, they need to develop competences so as to undertake assessment tasks effectively (achiev- ing the desired outcome) and efficiently (optimizing resources and efforts). The Do- CENT framework for digital creativity competences – developed in the frame of the DoCENT ERASMUS+ project – proposed a set of competences for digital creative teaching [8]. The methodological processes for the development of the DoCENT framework included: an audit of existing competence frameworks (General compe- tence frameworks, Teacher competence frameworks Teacher educator frameworks, Digital competence frameworks, and creative teaching competences); and negotiation and expert validation of the framework in consultation workshops with teacher educa- tors and teachers. Figure 1 here below presents the areas of DoCENT framework for digital creative teaching competences. Highlighted are areas and competences relating to assessment namely: a) creative assessment – i.e. the use of digital strategies to assess students’ creativity, and b) assessment of learners’ digital creativity – i.e. the competences re- quired to assess students’ digital creative competences. 4 Fig.1. The DoCENT framework of digital creative teaching competences. Highlighted are competences relating to assessment for digital creativity 4 Operationalization of the DoCENT competence framework for assessment purposes This section presents a proposal for operationalizing the DoCENT framework through rubrics for self-assessing competences for digital creativity. Rubrics are useful tools for formative assessment purposes, designed to clarify criteria and standards against which teaching and learning processes can be assessed. Proposed here below are a) criteria for self-assessing the teaching and learning process for digital creativity informed by the DoCENT framework of digital creative teaching competences and b) mastery competence levels, informed by the five-level conscious competence learning model [14]. 4.1 Criteria and sub-criteria for assessing competences for digital creativity Table 1. Criteria and sub-criteria for assessing competences for digital creativity – Focus on the teaching process Criteria Sub-criteria Actively engage Involve students in self-evaluation and peer-evaluation students in assess- Focus on both the learning process and the outcome, so to 5 ment processes encourage students to critically reflect on their learning path, which foster meta- competences, mistakes and progress cognition and critical Use a variety of assessment formats and approaches thinking Use digital technologies to carry out formative and summa- tive assessment (e.g., learning analytics) Use technologies to Apply criteria (e.g., fluency, flexibility, originality, elabora- evaluate students’ tion) for evaluating students’ little-c (personal) creativity creativity Apply tools (e.g., digital rubrics) for evaluating students’ little-c (personal) creativity Table 2. Criteria and sub-criteria for assessing competences for digital creativity – Focus on the learning process Criteria Sub-criteria Divergent and con- Encourage students to identify and solve real-world problems vergent thinking using creative thinking skills, i.e., generate and apply original ideas and solutions by forming remote associations, concep- tual combinations, and approaching problems from different angles (divergent thinking) Encourage students to evaluate and select ideas using deci- sion-making strategies, so to produce the best possible an- swers (convergent thinking) Digital creation and Adopt a “maker culture” which fosters students’ creative expression expression of ideas, experiences and emotions in a range of media, through the creation of digital or tangible objects Allow for knowledge construction processes and expression based on students building, making, storytelling, prototyping, engineering and sharing objects that are relevant to a larger community Information literacy Encourage students to articulate information needs and digital citizen- Encourage students to find information and resources in digi- ship tal environments Facilitate students to organise, process, analyse and interpret information Facilitate students compare and critically evaluate the credi- bility and reliability of information and its sources Encourage students to participate safely, effectively, critically and responsibly in the digital world Creative dispositions Use digital technologies to promote students’ openness to experience, responsible risk taking, tolerance of ambiguity, learning from failure, and viewing challenges as possibilities for learning 6 Computational Stimulate students to solve problems and model systems, as thinking and design well as understand mindsets and behaviors, by drawing on the thinking concepts fundamental to computer science and design think- ing 4.2 Mastery competence levels under the 5-level conscious learning model The five stages of competence, or the "conscious competence" learning model, relates to the psychological states involved in the process of progressing from incompetence to competence in a skill. It involves Stage 1 - Unconscious incompetence (the person is blissfully unaware of their ignorance); Stage II – Conscious incompetence (the person aware of their skills shortage); Stage III – Conscious competence (the person is able to demonstrate their competence with a high level of concentration or focus); Stage IV – Unconscious competence (the person is able to demonstrate their compe- tence with a low level of concentration or focus); Stage V – Shared competence (the person is able to teach others by explaining not only how but also the why’s to achieve a level of competency). In conjunction to the above stages, the following items for the rubrics (relating to the mastery competence level) are proposed, illustrated in Table 3. Table 3. Mastery competence levels – items for the rubric distinction between levels Mastery competence levels Level 1 Level 2 Level 3 Level 4 Level 5 Unconscious Conscious Conscious Unconscious Shared/reflective incompetence incompetence competence competence competence I consider the I need to I perform I perform I can help others Rubric items competence work more to the compe- the compe- to develop the X as non- master the tence X tence X competency X useful competence efficiently efficiently X when I am without mindful thought The above mastery levels can be used both for teachers’ self-assessment of the teaching process (see Table 1 for the according criteria) and for teachers’ assessing students learning process (see Table 2 for the according criteria) for enhancing digital creativity. 5 Concluding Remarks In this paper it has been outlined that assessing digital creativity in educational con- texts is a complex task. Tensions and challenges in this endeavour are rooted on the dispense notion of creativity and on difficulties in conceptualizing creativity and digi- tal technologies in tandem. In the view of these challenges, it has been argued that if 7 teachers are to undertake assessment tasks effectively and efficiently, they need to practice competences relating to digital creativity assessment. Under this concern, we presented the DoCENT framework’s teaching competences for digital creativity relat- ing to assessment issues. We also proposed a way to operationalise the framework for self-assessment purposes through rubrics– by illustrating criteria for assessing the teaching and learning process for digital creativity, and by proposing mastery compe- tence levels informed by the five-level conscious competence theory. The work presented in this paper should be considered as a preliminary step in fa- cilitating the assessment of competences for digital creativity. The applicability and the feasibility for adoption of the proposed tool is yet to be seen through validation studies. Of interest would also be to explore teachers’ competence development path through the five-level conscious competence theory - for example to identify critical incidents that allow the progression from one mastery level to another in relation to competences for digital creativity and challenges in teachers’ mastering the reflective competence level. Finally, investigating implications of the proposed approach for assessing competences for digital creativity on the promotion of students’ creative mind-set would be an area for further research and study. References 1. European Commission: Council conclusions of 12 May 2009 on a strategic framework for European cooperation in education and training (‘ET 2020’). 2009/C 119/02 (2009). 2. European Commission: Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Re- gions: Rethinking Education: Investing in Skills for Better Socio-economic Outcomes, 20.11.2012 COM (2012) 669 final, Strasbourg: European Commission (2012). 3. Wyse, D., & Ferrari, A.: Creativity and education: Comparing the national curricula of the states of the European Union and the United Kingdom. British Educational Research Jour- nal, 41(1), 30–47 (2015). 4. Cachia, R., Ferrari, A., Ala-Mutka, K., & Punie, Y.: Creative learning and innovative teaching: Final report on the study of creativity and innovation in education in the EU member states. Luxembourg: Publications Office of the European Union. doi:10.2791/52913 (2010). 5. Henriksen, D., Mishra, P., & Fisser, P.: Infusing Creativity and Technology in 21st Centu- ry Education: A Systemic View for Change. Educational Technology & Society, 19 (3), 27–37 (2016). 6. Loveless, A.: Creative learning and new technology? A provocation paper. In J. Sefton- Green (Ed.), Creative learning (pp. 61-72). London, UK: Creative Partnerships (2008). 7. Sefton-Green, J. & Brown, L.: Mapping learner progression into digital creativity A state of the art review written for Nominet Trust (2014). 8. Barajas, M., Frossard, F. & Alcaraz-Domínguez, S.: DoCENT – Digital Creativity EN- hanced in Teacher Education Framework of digital creative teaching competences http://www.ub.edu/euelearning/O1_FRAMEWORK_DIGITAL_CREATIVE_TEACHIN G_COMPETENCES.pdf, last accessed 2019/04/07 (2018). 9. Charyton, C., Ivcevic, Z., Plucker, J. A. & Kaufman, J. C. : Creativity Assessment in Higher Education, Handbook of Research on Assessment Technologies, Methods, and Ap- plications in Higher Education, C. S. Schreiner, ed., Hershey: IGI Global, 78-96 (2009). 8 10. Loveless, A.: Literature Review in Creativity, New Technologies and Learning. 2002. hal- 00190439 (2002) https://telearn.archives-ouvertes.fr/hal-00190439, last accessed 2019/04/07 (2018). 11. Jonassen, D.H.: Computers as Mindtools for Schools: Engaging Critical Thinking. 2nd Edition. Upper Saddle River, New Jersey, Columbus, Ohio: Merrill/Prentice Hall (2000). 12. Sinker, R (2000). Making Multimedia: Evaluating young people's creative multimedia production in J.Sefton-Green and R. Sinker (eds) Evaluating Creativity. London: Routledge (2000). 13. Cerrato, A., Siano, G., De Marco, A. & Ricci, C. : The Importance of Spacial Abilities in Creativity and Their Assessment Through Tangible Interfaces. International Conference in Methodologies and Intelligent Systems for Technology Enhanced Learning, 89-95 (2019) 14. Pivotal Education Homepage, https://pivotaleducation.com/hidden-trainer-area/training- online-resources/levels-of-competence/, last accessed 2019/07/04