This investigation aims at developing an Open Educational Resource (OER) to the learning of research methods in doctoral programs in Technology-enhanced Learning (TEL). Under the methodological approach of Educational Design Research, it consists of three phases: context analysis, development and formative evaluation, and semi-summative evaluation. Preliminary results from the first phase revealed that design-based research is the most used research method in TEL and also the one that PhD candidates and PhD holders need more training. Considering this, in the second phase, the OER prototype about design-based research was designed and developed using the H5P plugin on Moodle, where it was possible to insert some gamification strategies. At first, only a section about one research method was developed, so that this prototype can be formatively evaluated before developing the whole course on research methods. At the moment, the formative evaluation is being planned to encompass three cycles of assessment. In the first cycle, the scientific content will be validated by experts; in the second, a survey will be conducted with master's and PhD students; and finally, there will be focus groups with professors, researchers and PhD students. In the last phase, a semi-summative evaluation of the whole course will be performed to make the last adjustments to the OER and finally implement and disseminate the results in doctoral programs in TEL. Open Educational Resources, doctoral education, Educational Design Research Proceedings of the Doctoral Consortium of Sixteenth European
As information technologies have advanced and been more accessible, a vast number of digital resources have become more available for those involved in education. Teachers have been using the Internet to spread their materials and courses, and content in digital format has largely increased. Yet, most of these materials are not open to be freely reused, shared or remixed. With the purpose of overcoming these barriers, the Open Educational Resource (OER) movement was founded to encourage and enable anyone to reuse and share content in an open manner.
OER
can be defined as any teaching, learning or research materials that make use of
2020 Copyright for this paper by its authors. Use permitted under Creative
permission to adapt and reuse them [
material through different
activities, known as the 5R: retain, reuse,
revise, remix, and redistribute. The types of
materials can vary from videos, images and
textbooks, to podcasts, games, and courses [
There are some motivations for educators,
institutions and governments to be involved
with the development and sharing of OER [
However, according to [
Nine European universities and the
European Association for
TechnologyEnhanced Learning (EA-TEL) created the
Doctoral Education for Technology-enhanced
Learning (DE-TEL) project with the aim at
identifying the best practices in doctoral
programs in TEL, developing a proposal for a
new program and developing sustainable Open
Educational Resources (OERs) [
This thesis, in particular, focuses on the development of OERs to doctoral education training and, as a contribution to the DE-TEL project, the OERs are going to cover the research methods for doctoral programs in TEL. Our main research question is which characteristics should an OER developed for doctoral education training have in order to make its use more engaging? and our objectives are: 1. to analyse the use of OERs in doctoral education training; 2. to identify which characteristics can contribute to make OERs an engaging solution to doctoral education training; 3. to develop an OER to the learning of research methods in doctoral programs in TEL; 4. to evaluate the developed OER on research methods in doctoral programs in TEL.
In the following sections the methodological approach and some preliminary results are going to be presented.
This investigation is being developed under
the Educational Design Research
methodological approach. Its planning consists
of three phases [
In the first phase, named context analysis, a systematic literature review (SLR) is going to be carried out. The main goals of this SLR are to analyse the use of OERs in doctoral education training and identify the main challenges and barriers when adopting OERs in higher education, especially in doctoral programs.
Furthermore, a survey will be conducted
among PhD candidates, researchers and
practitioners who study and/or work in the field
of TEL, the context of this study, to identify
which characteristics can contribute to make
OERs an engaging solution to doctoral
education training. The survey will be divided
into four sections. In the first section, there will
be questions related to personal background.
The second section is going to be adapted from
[
Sections three and four will regard the
factors and formats that would make the
participant more likely to select a particular
resource when searching for content. These
sections are going to be adapted from a survey
developed by the Hewlett-funded OER
Research Hub, an open research project based
at The Open University (UK) [
The data from the closed-ended questions will be analysed through descriptive analysis on SPSS, and the data from the open-ended questions will be analysed through content analysis on NVivo, if necessary.
In the second phase, called development and formative evaluation, the conceptualization, design and development of the OER prototype will be carried out considering the previous results from the first phase. Since one of the objectives of the DE-TEL project is to develop OERs on research methods and key topics in TEL, this study will focus on the development of OERs on research methods.
First, a prototype is going to be developed. This prototype will go through cycles of formative evaluation with PhD candidates, professors and researchers in the field of TEL, and as feedback is being received, adjustments and improvements will be carried out.
At the moment, we are studying the instruments and techniques and also searching for some validated questionnaires and scales to formatively evaluate the OER prototype. This formative evaluation will probably consist of three cycles, as presented in Table 1. As the feedback is being received from each cycle, adjustments and improvements will be carried out before starting the next cycle.
In the first cycle, there will be a validation of the scientific content of the prototype. Three experts in research methods are going to be contacted via email to validate the content. The main goal of this cycle is to verify if the content is appropriate to the learning of research methods in doctoral programs in TEL and make the necessary adjustments, before starting the second cycle. To guide the experts’ validation, there will be a survey with questions asking if the content is useful, relevant, accurate, reliable, sufficient and if it meets learners’ needs. There will also be an open-ended question for additional comments.
In the second cycle, the general aspects of the prototype will be evaluated with twenty PhD candidates. They are going to answer a survey consisting of the User Experience Questionnaire (UEQ) and a self-design questionnaire.
According to Díaz-Oreiro et al. [
Díaz-Oreiro et al. [
As a complement to the UEQ, the selfdesign questionnaire will encompass some open-ended questions, such as the most positive and negative aspects of the prototype in their opinion, and suggestions for improvement. These data will be analysed through content analysis on NVivo.
The data collected in this second cycle will be triangulated with the data from the participants’ access to Moodle, where the OER is going to be integrated. On Moodle, it is possible to visualize if the participants accessed the platform, how much time they spent on it, which pages they visualized and interacted with, which tasks they accomplished, etc. The SPSS software is going to be used for statistical data analysis.
In the third cycle, focus groups are going to be conducted. The purpose of these focus groups is to show some preliminary results and shed light on the data collected in the second cycle in order to identify the participants’ opinions about the prototype. There will be two focus groups, one with professors and researchers, and another with PhD candidates. Each group will have from six to eight participants, last from 60 to 90 minutes, and will be conducted online, through the Zoom platform. The questioning route will include questions related to the interactive content created with the H5P plugin on Moodle, the 5R activities related to OERs, negative and positive aspects, and others, to make improvements to the prototype.
After the prototype goes through these cycles of formative evaluation, the final version of the OER will go through a semi-summative evaluation before being disseminated in doctoral programs in TEL. 2.3.
In the third and final phase, after we have a final version of the OER, a semi-summative evaluation is going to be conducted among PhD candidates, professors and researchers who study and/or work in the field of TEL. A survey will be conducted to conclude this investigation and make the last adjustments to the OER if necessary. The data will be analysed using SPSS and NVivo. Finally, the OER is going to be disseminated in the doctoral programs in TEL in Europe.
Figure 2 summarizes the methodological design of this research:
This section will present some preliminary results regarding the development of the first OER prototype on research methods in TEL. This first version is being developed using the H5P tool and plugin on Moodle to test the possibilities of tools that might be useful for the development of OER prototypes.
A survey was conducted by the DE-TEL project to collect information on the current practices and challenges of doctoral education in TEL and find out what topics are useful but have few educational resources. The survey was conducted among students, PhD candidates, researchers, practitioners who study and/or work in the field of TEL, between December 2020 and March 2021, through the online survey tool LimeSurvey whose link was available on the DE-TEL webpage (eatel.eu/de-tel/survey). The DE-TEL partners are using SPSS and Tableau to analyse the collected data.
Preliminary results have revealed that 229 participants from 40 different countries, most of them from Europe, answered the survey. When asked to select the item that best described the general methodological approach of their PhD research, PhD candidates and PhD holders reported that design-based research was the most used research method, followed by quantitative and qualitative methods, as can be seen in Figure 3. Regarding the research methods that they need more training, designbased research was also the most selected research method by PhD candidates and PhD holders, followed by quantitative and qualitative methods as well.
As this research aims at developing OERs to the learning of research methods in TEL and the preliminary results showed that designbased research is the most used research method and also the one which PhD candidates and PhD holders need more training, a prototype of the OER module about designbased research is being designed and developed using H5P tool.
H5P tool (h5p.org), an abbreviation for HTML5 Package, is a completely free and open technology, which enables anyone to create, share and reuse interactive HTML5 content more efficiently, without the need for any technical knowledge.
H5P makes it easy to create rich interactive content by providing several content types for various needs. It is possible to create videos enriched with interactions, presentations with interactive slides, drag and drop tasks with images and text, images with multiple information hotspots, single or multiple choice questions, interactive books integrating several content types, and many others.
Figure 4 shows an example of the content type named drag the words, where it is possible to create text-based drag and drop tasks. In this activity, learners are asked to drag the characteristics of design-based research and drop them into the correct explanation. Then, they can check their answers to see how many responses they got right and a score is generated. They can also choose between retrying the task or visualizing its solution. Figure 4, for instance, presents the solution with the correct responses.
The content type that was adopted in the creation of this OER prototype on design-based research was the interactive book. The H5P interactive book content type allows authors to create courses, books or tests, combining various interactive content types inside of it, such as interactive videos, course presentations, questions and much more, through multiple pages. Figure 4 illustrates a part of the page about the characteristics of design-based research. On the left-hand side of the image, it is possible to visualize the contents of this interactive book. The numbers in the top righthand corner of the image indicate the actual page and the total pages of the book, respectively.
At the end of the interactive book, there is a report displaying the learner’s progress throughout the book (Figure 5). There is the total score, which is the number of points scored by the learner from his/her correct answers to the interactive questions; the book progress, which is the percentage of the visualized pages and the performed interactions with the content (it is not possible to get 100% of book progress only visualizing the pages); and the interactions progress, which is the percentage of the content that the learner interacted with. On the last page of the interactive book, there is also a summary that shows the details of the interactions from each page.
H5P content can be integrated into other platforms, such as Canvas, Blackboard and Moodle. This content about design-based research, specifically, was integrated on the open Tech4Comp platform (moodle.tech4comp.dbis.rwth-aachen.de/), on Moodle, since it is the platform that is being used by the DE-TEL project. It is also possible to create content using the H5P plugin on Moodle. Thus, some editing and complement to this material were performed directly on Tech4Comp Moodle platform.
With this prototype, we intend to insert some OER characteristics according to the answers from the survey which is going to be carried out in the context analysis phase. PhD candidates, researchers and practitioners are going to identify which characteristics can contribute to make OERs an engaging solution to doctoral training and we are going to insert these characteristics into the prototype to see how they work.
With this investigation, we expect to contribute to the practical and theoretical understanding of the process related to the development of technological solutions and innovations to the area of TEL. Design-based Research (or Educational Design Research) is the most used research method in TEL and its process brings both theoretical and practical contributions to the area.
Regarding the practical results, we hope that the developed OER can be a useful tool to the teaching and learning of design-based research, that it helps PhD candidates to deepen their knowledge on this research method, and that its characteristics can contribute to making its use more engaging.
Theoretically, we expect that the developed OER can raise awareness of the importance of the adoption and spread of OERs in education, mainly among doctoral programs, and that this study can support the development of other solutions and innovations to the field of TEL.
I would like to express my gratitude to my supervisors, Professors Luís Pedro and Carlos Santos. This paper and the research behind it would not have been possible without their support, guidance and valuable comments.
This project has received funding from the European Union’s Erasmus Plus programme, grant agreement 2019-1-NO01-KA203060280. 6. References