In the article, the authors study one of the trends in modern education - massive open online courses. MOOCs traditionally feature interactive thematic lectures, implemented by various technical means as well as tasks, tests, forums and comments providing communication between a teacher and a student. From the start of the development and further on, while promoting the courses, the authors were interested in how the listeners assess different elements of MOOCs. Davis's method is used to assess the preference of MOOC training elements. The research was carried out using a questionnaire on Google Forms. As a result of the research, the authors obtained data indicating the attractiveness of different MOOC training elements. The results of this study contribute to the development of knowledge in the field of MOOCs. Identifying, sorting and displaying characteristics according to the TAM theoretical model can facilitate the design and teaching of MOOCs and, therefore, will promote the research and implementation of more advanced MOOC elements. Moreover, the identification of student characteristics will allow a better match of MOOC characteristics with the learning styles of students in the field of information technology for economics and management majors.
Massive open online courses (MOOCs) is a rather new but exceptionally important, under current conditions, trend in the education including online education. Over the last years, the academic community has been studying how to tap the full potential of MOOCs since the number of MOOCs themselves together with the number of listeners undergoing training or retraining continue growing. The authors of this article have developed two massive open online courses published on Stepik, the Russian elearning platform. The objective of the article is to analyze different scientific research on the assessment of utility demonstrated by different elements of these courses for the teaching and educational processes as well as to find out which form of material presentation is most attractive for the respondents and helps them master the course better.
In the research presented in the article, the authors faced two main challenges: first of all, to study the characteristics that contributed to the success of MOOCs depending on student perception, and, secondly, to receive information from the students themselves on their preferences. To meet these objectives, over 400 online surveys were held among students concerning two courses in information technology for economics and management majors.
Student feedback was analyzed using quantitative and qualitative methods to identify characteristics that contributed to the success of MOOC elements in terms of students. First of all, the data of student survey was analyzed using the quantitative method. Characteristics of different MOOC activities were identified and classified using a questionnaire drafted following the recommendations and using the updated Technology Acceptance Model (TAM) by Davis.
The indicated model, despite being proposed by Davis in 1985, is still one of the most common models used to assess the acceptance of systems by users.
The authors drafted a questionnaire based on the Technology Acceptance Model by Davis.
The following training elements were specified as MOOCs elements: Video with teachers on the screen Video with voiceover text Video with captions Practical task Tests Tasks with subsequent analysis It was important for the authors to understand which form of material presentation is most attractive for the respondents and helps them master the course better. 2
The first platforms appeared in the US (Coursera, EDеX, Udacity), then in
Germany (Iversity), Spain (Crypt4you), the UK (Futurelearn), the EU (Open up Ed) [
Russia launched its platforms in 2013-14. They included Lektorium, Universarium, Stepik (Stepic) etc. Some Russian universities entered Coursera with their own academic courses.
In 2015, the group of Russian universities including Lomonosov Moscow State Uni-versity, Peter the Great St. Petersburg Polytechnic University, St. Petersburg State University, National University of Science and Technology MISIS, National Research University Higher School of Economics, Moscow Institute of Physics and Technology, Ural Federal University and ITMO University initiated the creation of the National Open Education Platform (Otkrytoye Obrazovaniye, hereinafter referred to as the NOEP) [2].
The implementation of digital higher education programs is aimed at improving the quality, expanding and updating teaching methods. Today, there is a particular interest in the development of digital education at the state level. The structure of the Digital Education national project, approved in September 2018 by the Presidium of the Council under the President of the Russian Federation for Strategic Development and National Projects pays particular attention to the development of the Digital educational environment [3, 4].
The Digital Educational Environment federal project has been developed to create a modern and safe learning environment by 2024. As part of the digital educational environment implementation, digital competency profiles will be created for all participants of the learning process: students, teachers, and administration. The Modern digital Educational Environment in the Russian Federation project will make online learning an effective, popular and harmoniously integrated part of the existing educational system within several years [5, 6]. At the same time, citizens will have free access to online training courses of any level with the possibility of crediting them by educational institutions and obtaining certificates confirming the successful completion of studies. [7, 8]. This project focuses on the development and improvement of massive open online courses.
Even before the 2020 pandemic, the international MOOCs market showed steady growth [9, 10]. Amid modern conditions, the development and improvement of MOOCs is a particularly important trend in the education and online learning. Since the beginning of 2020, there has been an explosive growth in the number of courses in the MOOC format, in the number of universities and organizations that publish online courses, and in the number of students who register for these courses.
In particular, Dhawal Shah, already mentioned here, noted that on March 15 this year, Class Central saw a large increase in traffic [11]. Since then, over 9 million students (and perhaps even a few celebrities) have visited Class Central to find courses, sending 5.5 million clicks to the MOOC providers themselves.
The ongoing pandemic has fueled interest in online education – MOOC providers, in particular, observed a surge that reminded them of the year of MOOC.
Within 30 days, users registered to more than 10 million courses on Coursera, which is 644% more compared to last year. EdX has become one of the 1000 best websites in the world. The growth in the number of courses and attendance is shown in Table 1. #285,183 (+495,214) #45,692 (+33,412) 0.18 (↑200%) 0.73 (↑52%) Thus, the number of students undergoing training and retraining with the help of MOOCs is growing steadily. The authors of this article were part of the team that developed two massive open online courses hosted on Stepik, the national online learning platform. We, as developers, from the start of the development and further on, while promoting the courses, were interested in how the listeners assess different elements of MOOCs. MOOCs traditionally feature interactive thematic lectures, implemented by various technical means, as well as assignments, tests, communication with teachers and students on the forum. In this article, the authors consider what kind of the material presentation is most attractive for the listeners and helps them master the course better.
Similar studies assessing the utility of various elements of massive open online courses for teaching and learning processes have been carried out abroad. In the earlier papers, Freitas, Morgan, and Gibson, as well as Dabbagh [12, 13], provide information about similar problems and suggest different approaches to solving them, as well as descriptions of various MOOC activities for us to better understand how students communicate with MOOCs and undergo them. In these papers, the authors mention that “…still better indicators are to be developed” to assess the effectiveness of each element.
Researchers from the Kebangsaan University of Malaysia emphasize the importance of such research and note in their article [14] that “Listening to the voices of the students and the types of characteristics that they chose to mention, enables further exploration of their preferences and expectations regarding MOOCs and, accordingly, to future adaptation between students’ preferences and MOOC characteristics…”.
In the research presented in this article, the authors and their colleagues set two main objectives: firstly, to study the characteristics that contributed to the success of MOOC, depending on the perception of students, and, secondly, to learn about student preferences from students themselves.
A number of studies by various authors such as: Arbaugh et al., in 2008; [15], Garrison & Arbaugh, in 2007; [16], Kozan, in 2016; [17], Mills et al., in 2016 [18] were aimed at determining and validating the ratio of different types of training in MOOCS. These studies were conducted at different times, and there were attempts to validate, in particular, the model The Community of Inquiry (CoI) framework theory. However, the authors preferred research based on the results of student surveys, such as in research by a group of scientists from the Kebangsaan University of Malaysia [14], in research by Literat, 2015; [19], Lowenthal, Snelson, Perkins, 2018, [20], GilJaurena, Domínguez, 2018; [21] and also Lin, Cantoni, 2018, [22].
The purpose of the research is to determine the optimal structure of MOOCs. This study was commissioned by the Plekhanov Russian University of Economics (PRUE). And this research will help the Department of Informatics of PRUE to create more effective mass open online courses.
The empirical base of the study features the data of an online survey carried out on two courses in information technology for economics and management majors hosted on the Stepik: “Information systems in the economy. Working with MS Access DBMS” and “Information technology. Working with Excel spreadsheets”. The survey was conducted using Google forms, and statistical processing was performed using MS Excel tools. The sample consists of more than 450 respondents who studied these courses.
Feedback from respondents was analyzed using quantitative and qualitative methods in order to identify characteristics that contributed to the success of the MOOC elements from the listeners’ point of view. First of all, the audience survey data were analyzed using a quantitative method of calculating statistical indicators. The characteristics of the various MOOC activities were identified and classified using a questionnaire drafted following the recommendations and using the updated Technology Acceptance Model (TAM) by Davis [23].
The indicated model, despite being proposed by Davis in 1985, is still one of the most common models used to assess the acceptance of systems by users.
The authors drafted a questionnaire based on the Technology Acceptance Model by Davis.
The following learning elements were specified as part of the MOOCs: Video with teachers on the screen Video with voiceover text Video with captions Practical task Tests Tasks with subsequent analysis
The assessment of the subjective utility of a training element was carried out based on the following statements 1. Using this training element will improve my learning performance. 2. Using this training element will help me complete the assigned learning objectives faster. 3. Using this training element will simplify my learning process. 4. Using this training element will increase the effectiveness of my learning process. 5. Using this training element will broaden my professional horizons. 6. Using this training element will benefit my learning process.
The assessment of the subjective ease of use of this training element was formed on the basis of answers to the following questions: 1. I will be able to easily learn how to use this training element. 2. It will be easy for me to get exactly what I want from this training element. 3. The interface of this training element is intuitive. 4. I will be able to easily adapt this training element to my needs. 5. I will easily become an advanced user of this training element.
This training element can be easily used by anyone.
It was proposed to give answers using the following scale (Table 2).
Many questions are similar which makes the questionnaire more effective, as recommended in [25]. 4
Our research involved seven groups of respondents by age (from 14-17, from 18-21, from 22-28, from 35-45, from 45-55, from 55-65 and over 65). As the analysis of the survey data has shown, the majority of answers were given by the age group of 18-21 (Fig. 1), in total 55% of the number of respondents.
The survey revealed that the majority of the respondents consider MOOCs to be very useful for students and teachers. The respondents say that it helps assimilate the material consciously, gives a chance to make quick corrections, allows students to work independently, and facilitates the work of teachers. A minimal number of the respondents found the use of MOOCs unattractive and not useful both for students and teachers (Fig. 2).
Fig. 2. Results of answers to the question on whether MOOCs have advantages (drafted by the authors) We conducted a study among the population on the experience of using MOOCs.
Half of the respondents experienced (Fig. 3) this educational method only once, while only 8% of the respondents used massive open online courses more than five times, which allows us to conclude that MOOCs are not yet so widely known among the population. Another objective of the survey, set by the authors, is to identify the attractiveness of various educational activities for the respondents that help them master the course better. It was identified that among all the available answer options, there are no clear preferences. However, the respondents found the video format with a teacher on the screen the most useful one, while the video format with captions turned out to be the least convenient one Another question posed to the respondents was “Why do you need a MOOC?” As Fig. 5 shows, there is no definite answer. People want to use MOOCs for completely different purposes. The only thing we can observe is that the majority voted for selfeducation, while the most unpopular answer was “to keep up to date with new developments”.
The main limitation of this study was the small number of courses it covered, although these courses included hundreds of reviews and a high score: 5 out of 5 and 4.9 out of 5.
When studying the subjective utility of MOOCs for people of different ages, it can be concluded based on the survey, that the majority of young people believe that MOOCs will improve their learning performance, while in the 45+ group, the opinion prevails that MOOCs will broaden their professional horizons.
The study of the population’s experience in using MOOCs showed that half of the respondents experienced, as can be seen from Figure 2, this educational method only once, while only 8% of the respondents used massive open online courses more than five times, which allows us to conclude that MOOCs is not yet so widely known among the population.
It was important for the authors to understand which form of material presentation (video with a teacher on the screen, video with voice-over text, practical tasks, tasks with subsequent analysis, tests, video with captions) is the most attractive for the respondents and helps them master the course better. It turned out that the respondents found the video format with the teacher on the screen the most useful one, while the video format with captions turned out to be the least convenient one.
For the development of MOOCs, it is very important to understand what elements need to be improved; what positive or negative characteristics are given by the respondents in terms of information presentation means. Another part of our survey features the analysis of advantages: Use of visual methods Promotes better understanding Colorful Consistency of presentation Contact with the teacher Ease of management, challenges: The learner does not keep up with the presentation Difficulty of perception Cannot enlarge the image, it features small details No feedback, various forms of material presentation: Video with voice-over text; Video with the teacher on the screen; Video with captions; Practical tasks; Tests; Tasks with subsequent analysis.
Many people find this method inconvenient mainly due to the difficulty of perception. The other indicators were also assessed as challenges, however, many found it difficult to answer this question, therefore, these indicators are less objective.
The main advantages of this method are: use of visual methods, promotion of better understanding and colorfulness. Many also believe that this method has benefits overall, but the difficulty of the perception distorts the positive assessment.
Among the definite challenges, the respondents mentioned no possibility to enlarge the image, the presence of small details. The difficulty of perception, no feedback and the likelihood that the learner does not keep up with the training cannot be called unequivocal drawbacks as the opinions of the respondents were divided. However, the majority of respondents agree that such an element of MOOC as video with a teacher on the screen has certain drawbacks.
As for the benefits, the respondents mentioned the use of visual methods, colorfulness, consistency of presentation, promotion of better understanding and contact with the teacher as unambiguous advantages. Ease of management and colorfulness raised doubts among the survey respondents, so they cannot be attributed to ad-vantages.
The majority of respondents mentioned a significant number of drawbacks of this method. The most obvious one is that the learner does not keep pace with the presentation. The others that received the majority of votes are: difficulty of perception, no possibility to enlarge the image and no feedback. We can see that this form of presentation requires improvement.
However, this method also has many unambiguous advantages, such as: use of visual methods, promotion of better understanding, colorfulness, consistency of presentation and ease of management.
When studying different ways of presenting the same element, for example, a video, the authors found that the most preferable for perception is a video with a voiceover text, a video with a teacher on the screen is slightly inferior to it. However, the video with the captions is losing significantly.
In this way of information presentation, the results turned out to be ambiguous and the opinions were divided, however, the main disadvantages were: no possibility to enlarge the image and the fact that the student does not keep up with the presentation.
The respondents mentioned a lot of advantages, such as: promotion of better understanding, use of visual methods, colorfulness, consistent presentation and contact with the teacher. This method was assessed extremely positively.
The results of the respondents in terms of drawbacks were ambiguous, but still 45% think that this method has challenges, the main one being the difficulty of perception.
The key advantages of this method are: use of visual methods and promotion of better understanding. The advantages also included the consistency of presentation and ease of management.
Many respondents indicated the challenges of this method, however, not those listed by us. The only challenge where they were unanimous was the inability to enlarge the images.
The situation with the advantages is clear. All of the proposed advantages (use of visual methods, promotion of better understanding, consistent presentation, colorfulness, contact with the teacher and ease of management) were chosen by the majori-ty.
We also asked why the use MOOCs is needed. The majority said that MOOCs is suit-able for both self-education and instructor-led learning. Many people think that it will also be useful for individual learning of the course. Another part of the respondents thinks that MOOC can be a separate element in the studied discipline.
When asked whether the respondents will use MOOCs in the future, 70% answered yes, 18% said they would not, and 12% have not yet decided. 6
The results of this study contribute to the development of knowledge in the field of MOOCs. Identifying, sorting and displaying characteristics according to the TAM theoretical model can facilitate the design and teaching of MOOCs and, therefore, will promote the research and implementation of more advanced MOOC elements. Moreover, the identification of student characteristics will allow a better match of MOOC characteristics with the learning styles of students in the field of information technology for economics and management majors.
The main limitation of this study was the small number of courses it covered, although these courses included hundreds of reviews and a high score: 5 out of 5 and 4.9 out of 5.
Thus, research scope should be expanded to include additional courses. Moreover, these reviews were collected from different sessions of the same course held over the past year. The research is based on feedback from students who decided to post on the Stepik website and does not include all students who participated in the selected courses. It is also important, that the reviews on the Stepik website were mostly positive and concerned successful courses. Additional information from negative reviews in a similar amount can add information to the list of characteristics that contributed to the success of MOOCs. The authors also note the need to expand the research to other MOOCs platforms.
It should be noted that the characteristics of the forum, which may represent collaboration between students, have been identified as one of the factors promoting the success of MOOCs. Collaborative learning usually involves activities in forums or joint work on tasks or projects. This area can become a new field of research. 7 18. Mills, J., Yates, K., Harrison, H., Woods, C., Chamberlain-Salaun, J., Trueman, S., & Hitchins, M. (2016). Using a community of inquiry framework to teach a nursing and midwifery research subject: An evaluative study. Nurse Education Today, 43, 34-39. 19. Literat I. (2015) Implications of Massive Open Online Courses for Higher Education: Mitigating or Reifying Educational Inequities? Higher Education Research and Development, vol. 6, no 34, pp. 1164–1177. 20. Lowenthal P. R., Snelson C., Perkins R. (2018) Teaching Massive Open Online Courses (MOOCs): Tales from the Front Line. International Review of Research in Open and Distance Learning, vol. 19, no 3, pp. 1–19. 21. Gil-Jaurena I., Domínguez D. (2018) Teachers’ Roles in Light of Massive Open Online Courses (MOOCs): Evolution and Challenges in Higher Distance Education. International Review of Education, vol. 64, no 2, pp. 197–219. 22. Lin J., Cantoni L. (2018) Decision, Implementation, and Confirmation: Experiences of Instructors behind Tourism and Hospitality MOOCs. International Review of Research in Open and Distance Learning, vol. 19, no 1, pp. 1–293. 23. Davis, F. D. (1985). A technology acceptance model for empirically testing new end-user information systems: Theory and results. (Doctoral dissertation). Massachusetts Institute of Technology. Retrieved from http://dspace.mit.edu/handle/1721.1/15192 24. Gerasimova V. G., Romanova Yu. D., Melamud M. R., Zhenova N. A., Sorokina M. Yu. the DAVIS METHOD FOR evaluating the attractiveness OF implementing ELEARNING in PLEKHANOV Russian University of Economics. [in Russian]. Transport business in Russia. 2018. № 4. С. 102-104. 25. Saprykina A. O. Model of acceptance of Davis technology as a means of evaluating the subjective effectiveness of e-portfolio technology [Text] [in Russian]. // Theory and practice of education in the modern world: materials of the VII international journal. scientific Conf. (St. Petersburg, July 2015). — St. Petersburg: Svoje publishing house, 2015. — С. 108-110. — URL https://moluch.ru/conf/ped/archive/152/8483/ (last accessed: 2018/03/20).