This article describes the insights from the analysis of the comparative evaluation of open learning methods in organizational education and will be of most interest to researchers and practitioners with non-university background. The approaches considered focus on the selection of solutions based on the methods, which can be individually adapted in the future and applied, for example, by using artificial intelligence technologies to achieve organizational goals without direct human intervention. In this paper we describe the evaluation methods of the learning forms based on the principles of andragogy and consider the software specifics of investigated methods. The obtained conclusions show that the open source course, online learning communities, training with virtual reality and artificial intelligence can be successfully implemented separately and in combination according to the purposes of organizational development. However, learning with artificial intelligence has also a great potential due to the extensive knowledge base and diverse ways of knowledge mediation.
With the increasing dominance of global megatrends, scientific and economic actors reorient their activities on the needs of the net generation. The rapid digitalization affects all areas of organizations: from finance and reporting management to marketing [6-8, 49]. This tendency also applies to the field of knowledge management. The adaptation to external factors influences organizational change [5, 18, 37, 47].
Since lifelong learning became increasingly important, education is changing in the direction of globalization, digitalization, and specialization [35]. This change means mobility, and learning independent of time and place, increasing internationalization of educational opportunities, strict specification of professional fields. In many countries, lifelong learning is financially sponsored by the state and internal in the companies [50, 53]. Innovations in education have great potential and, like finance and reporting, have a significant impact on organizational management [46, 49, 51].
Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
At the same time, this leads to increasing demand for the most current knowledge. Open education technologies fulfill all requirements: they are freely accessible, cost- and license-free, and offer users both teacher and learner role choice. Companies use open education solutions to provide their employees with transparent access to the selected general knowledge and to strengthen a common knowledge base [9, 33]. Some global corporations build their organizational learning system similar to open source platforms. Small and medium-sized companies are embracing this trend and trying to make their organizational learning digital, virtual, and open [31].
Although the open learning platforms offer most of the video content, the more often other technologies are developed and used for learning purposes there. In addition to well-known video-based training on the online platforms and open libraries, current open education includes the following forms of learning: synchronous e-learning (whiteboard, digital classrooms, open educational games), simulation (open educational virtual games), online learning communities (peer-tutoring, peer-working) [43, 54]. Furthermore, such forms for the open transfer of knowledge are virtual visualization AR-, VR-applications (learning factory), or CPU-based learning and learning with artificial intelligence intended [16]. Historically, all these forms and their particular applications correspond to the specific development of digital education and range from digitization of analog media and conventional learning methods (e. g. e-learning, cloud learning), visual integration of information into the working environment, connecting content with physical systems (e. g. augmented reality, IoT, virtual classroom) to autonomous systems of competence development by combining existing technologies and artificial intelligence (e. g. cognitive systems, collective intelligence). The learning form and their relevance for organizational development are the objects of our investigation.
Our attempt to combine different learning forms is not new, and it differs from the other firstly in the methodology of the study and secondly in the obtained results [16]. 2
Our investigation consisted of four phases. Firstly, a trend analysis of the web search requests was performed to determine which learning forms are currently most relevant in which countries and how this tendency has developed. In the second step, we compared the selected learning forms empirically using deductive usability benchmark analysis. The third phase was based on expert evaluation according to the obtained theoretical and empirical data. Finally, we present the portfolio matrix of proposed solutions for a better identification of learning forms according to organizational learning objectives and learning efficiency. 2.1
Even though autonomation with the increasing use of artificial intelligence reshapes education, traditional learning forms remain relevant. To evaluate these forms qualitative correctly, we selected one option from each group according to such criteria as technology degree, user interaction, variety of learning content, number of participants, etc. These exemplary forms are: open source training (video), online learning communities, virtual reality and learning with and from cognitive systems.
We have limited the range of our investigation to these four options. 2.2
To determine the actuality of the topic, we conducted a definition analysis with Google Trends from the past five years worldwide with the predefined terms: “open source course”, “online learning communities”, “training with virtual reality” and “learning with artificial intelligence” [17]. 2.3
Between all usability analyses, we have chosen one, which aims to compare several alternatives by one user and review of results collected from all users [1]. Here the learning forms are tested in comparison based on standardized tasks. All independent variables are fixed, and the dependent variables are measured. If one learning form is better than another in a benchmark analysis, it does not mean that this also applies to a change in the independent variables, e.g., for another task or users with other previous experiences. The study aimed to evaluate forms of learning based on parameters such as the speed of knowledge transfer, quality of learning, and personal perception. The first parameter determines the time that the user needs to record the presenting content. The second parameter was measured by how effectively the user reproduces the learning material, even after one day. The third parameter reflects the user's feedback on the learning experience. The compared parameters are variable except the predetermined video duration in the first variant, open source course. The speed of knowledge transfer of other learning forms due to user interaction cannot be estimated in advance. The number of correctly answered questions measured the learning quality. A total of four open questions and one with an enumeration of the offered variants were asked. The questionnaire was conducted twice: immediately after the completion of each session and the next day. Similarly, after the completion of each learning form, the data on the personal perception of the learning process were collected. Users prioritized this according to their overall learning experience (on a scale from 1 to 4).
The subject to learn should originate from an already known learning area and contain some new information and be appropriate for a wide variety of respondent groups. This reflects the approach for learning in the organization. As the subject for this study, we have chosen planets of the Solar system, one for each learning form. It must be noted that the quantity of the learning material was the same for each learning form and the same content was taught (distance from the Sun, color of the planet surface, components of the surface, water management on the planet, possibilities to explore the planets). For the first form of learning, we filmed a short video about Venus (the presenter explained the learning material and slides) and showed it to the respondents to simulate the open space course experience. For the online learning communities, we have chosen the Earth as a learning object, because all participants already have some previous knowledge. For the third learning form, training with virtual reality, we have selected a "Weltraum VR" app (as hardware Apachie VR Box), which has already integrated several tasks, among others, to learn some facts about Mars. The unique feature here is that the learner who is virtually on the planet does not perceive the tasks as it is, but notices many things, such as the planet's surface color and substance unconsciously. For learning with artificial intelligence, we have chosen a voice assistant Siri (MacBook), that gave the information about Mercury. The knowledge query was in the form of a questionnaire, in which the user was asked the same questions as regarded content. Although the analysis of Google Trends shows the increasing interest in learning with artificial intelligence, we have not found a better and free solution for our purpose than voice assistance. Unfortunately, this only redirects the user to the internet pages and does not provide any comprehensive information about the searched object. It means that currently, the user has to search and sort out the information himself. Nonetheless, the variations in functionalities and the likelihood of the vision of teaching artificial intelligence continue to increase.
According to the standard rules for such survey art the selected respondent group
consisted of the employed person in the age between 16 and 70 years (the lower limit for
apprentices and school students and the upper limit is for persons with a lot of work
experience) [11, 25]. The precondition for the users was to have little or inadequate
knowledge of the planets of the Solar system. To calculate the number of required
respondents to be tested, we have applied a stochastic sample formula (
2
1+( 2× (1− ))
2
(
The heuristic morphological analysis is suitable for the systematic study of complex issues. Furthermore, this method can be used for the formation of future scenarios. For the evaluation, the processing logic, according to Scriven, was applied [40]. The sequence of the evaluation methodology selected by us corresponds to the evaluation logic [12].
We structured the considered object (open learning form) into its parameters, for which various options are listed. By combining these options, innovative, goal-oriented approaches can be identified.
This method consists of five steps. These are: 1. Definition of the considered object (learning form). 2. Formulation of parameters for designing the requirements of the learning system for adult education. 3. Formulation of performance levels of the parameters. 4. Measurement and comparison (analysis). 5. Prioritization of objects or object components. In this step, the different results are combined into a unified value statement.
This method is presented in the form of a matrix with parameters. The characteristics of each learning form that influence the parameter values can be combined with solutions and presented as a prioritized list of possible solution combinations [36].
Through systematic literature analysis, we have selected and evaluated four parameter categories according to the investigated learning forms [2-4, 13-16, 19-22, 27-29, 3234, 44, 52, 55]. These correspond to the requirements of andragogic, software, and organization.
Andragogical evaluation parameters include: problem orientation (consideration of the specific issue) - in the ICT this parameter is presented, as a sequence of problem-solving tasks; user interaction - integration of the adult into the learning process, involvement and active participation in the learning process; participant orientation - the software is adapted to predefined user groups; the subject-specific experience of the user - the interactions in ICT are designed according to the level of knowledge and are neither too much nor too little demanding; ability for self-paced learning - the learner can determine for himself whether the learning content is relevant for him, whether and how he can vary it; intermediary of the material has a supporting function and allows the learner to understand the correlations of the material and the existing knowledge (i. a. user-generated content).
Special software evaluation parameters are: requirement of special software - some forms of learning cannot be used without specialized hardware, and the others use it partially; experience with the control (control experience) - the level of recognition of the used hard- or software influences the user experience and improves the handling; complexity of the control (control complexity) - this parameter, similar to the experience with the control system influences many other parameters; control type (voice control, haptics, gesture control) - this parameter indirectly influences other parameters, such as complexity of the control, customizability and real-time use; customizability - adjustment or selection of control options; real-time application - use of a learning form at the time the user searches for the problem solution or information; prospects for other applications - option for use for other purposes. General software evaluation parameters are also required for the evaluation of educational applications. They are generally valid and are not included in the evaluation we performed. These include all other facets of software required parameters. Organizational evaluation parameters include ability for permanent integration into organizational learning; suitability for diverse users - the learning form must be applicable for different user groups; speed of the learning process - the learning process is not intended to reduce the work of the employees, but to support it. Learning must be effective and time efficient; number of learners involved – the number of learners simultaneously participating in the learning process.
Our parameter system consists of 17 parameters, which can be assigned to one of the five performance levels in the respective learning form. It means that 340 parameter values are considerable for the learning system, which is established by the organization. Since we evaluate four learning forms according to their parameter levels, only 17 values can be selected as one per parameter from a total of 68. 3
Empirical results of the Google Trends analysis (see Fig. 1) show continuous regression for the interest of the definition “training with virtual reality” and gradual reduction of the search requests for “online learning communities” and “open source course”. In the years 2016 and 2018, the massive increases in search requests for the definition “learning with artificial intelligence” are noticeable. These trends show that the open source issue remains highly relevant, and learning with artificial intelligence is increasingly important. The topics on online learning with communities and with virtual reality are less interesting or known to the users. We may forecast that interest in the subject of artificial intelligence usage in the learning process will continue to grow over the years until the newer technology does not massively penetrate education. Remarkable is the geographical distribution of search requests for defined terms (see Fig. 2): the United States and India show massive popularity of these terms. This distribution does not mean that there is no interest in these definitions in the other countries, but that the search requests there are relatively low. Possible reasons for these trends can be the population of the country and the (desirable) technological development of the country. As the fragments of the results indicate, the topics of online learning communities and training with virtual reality have the greatest actuality in the United States. In comparison, the interest of Indian web-users in the issues open source course and learning with artificial intelligence is particularly high (this can be recognized by the more intensive marking of the country on the map). The data in Fig. 3 were obtained in the heuristic morphological analysis. The speed of knowledge transfer shows the average time in minutes required for the user to gain knowledge. The comparison of the value of this parameter indicates that virtual training is the most rapid to complete. This learning method also shows the best values for the other two parameters and, therefore, the optimal one from the examined learning forms. Other studies have already proven that active learning (by doing) is much more effective than passive learning [24].
On the contrary, learning with artificial intelligence shows the worst values, which can be the consequence of relatively poor learning performance of the solution used. Nevertheless, to the additional question, whether the users could imagine being guided by artificial intelligence in their everyday life and the organization-relevant topics in the future, we received 54 positive answers. The second-best result is measured with an open source course.
While the aggregated data is the relatively consistent, more precise observation of the individual values shows a high degree of diversification. It means that some respondents complete the tasks extremely quickly or are very imprecise in their queries and all forms of learning, which is more related to their personality or experience. Fig. 3. Speed of knowledge transfer, quality of learning and personal perception of tested learning forms (own processing) The advantages of this experimental procedure allow causal conclusions on the learning experience when conducted efficiently and make significant contributions to theory formation. The disadvantage of planning an experiment involves the appropriate determination of the dependent and independent variables as well as a measured option for the test environment.
Through heuristic morphological analysis, we can achieve a quantitative illustration of individual learning form parameters and use them as a basis for the design of education systems based on the combination of selected criteria. The analysis reveals that according to the latest scientific findings, the highest performance levels of the most parameters (12 of 17) indicate learning with artificial intelligence. 9 of 17 highly valued parameters has trained with virtual reality. Although open source course (training) does not seem to provide such high parameter values, it does not require specialized hardware and is relevant for teaching of different contents. These findings can serve to individualize the design of the organizational learning system as well as the selection of particular learning forms.
The following statements can be concluded from the findings of empirical and theoretical analyses: ─ for efficient (fast, high learning performance) real-time knowledge acquisition and transfer for one person is the training with virtual or augmented reality, due to so gained immersive experience, currently the most appropriate approach (e.g., workplace-related knowledge, conflict resolution behavior in a specific situation, etc.); ─ for the transfer of general, informative, non-problem-related knowledge with a high degree of standardization is open source course particularly useful (e.g., preparation for certification audit, general compliance rules); ─ the learning communities offer support for issues that are difficult to resolve (e.g. maintenance-related knowledge for specific parameter settings, preparation of presentations); ─ the learning with artificial intelligence can be used in various ways, for example, as a language assistant to support the quick search for problem-solving methods (reading the content aloud) or to detect the learners' weaknesses and knowledge gaps (the recognition of faulty activity execution through gesture recognition), etc. Besides, learning with artificial intelligence can also be used for selecting appropriate forms of learning within the organization’s learning system. Our study aimed to compare different forms of open education for organizational issues. We have found that each learning form can be ideal for a specific purpose or a particular user. Even though the considered learning methods can be easily implemented in open education platforms, the involvement of those in the organizational development process has a limitation, which mainly affects the learning material. The learning material concerns the learning context because, as soon as it refers to the proprietary corporate know-how, the content is not permitted to be publicly available. It means that only those contents which describe generally valid principles or which are not authentic components of a concept/process/product can be educated as open source with the related learning forms.
From the perspective of the coopetition, an open knowledge pool can be established for a common knowledge base because of the cooperation between companies, from which sector-specific information for open learning can be taken [42, 43].
The orientation of knowledge, research, and industrial organizations towards the digitalization of learning processes has many advantages and changes the educational state of the world [10, 39]. Nevertheless, this can be better integrated with state (financial) support [23, 30, 53].
The findings of our study can be integrated into the design for the subsystem of organizational learning [26].
My thanks to each of the respondents in the benchmark study: without your participation, the evaluation would not have been possible.
We want to express our great appreciation to the three anonymous reviewers of the workshop for their critical reading and insightful feedback.
The survey was supported by the Ministry of Education and Science of Ukraine and
performed the results of the project “Modeling and forecasting of the
socio-economicpolitical road map of reforms in Ukraine for the transition to a sustainable growth
model” (registration number 0118U003569).
16. Gavish, N., Gutiérrez, T., Webel, S., Rodriguez, J., Peveri, M., Bockholt, U., Tecchia, F.:
Evaluating virtual reality and augmented reality training for industrial maintenance and
assembly tasks. Interactive Learning Environments 23(
DOI: 10.21272/bel.2(
DOI: 10.21272/mmi.2017.4-31
50. Vasylieva, T. A., Lieonov, S.V., Petrushenko, Yu. M., Vorontsova, A. S.: Investments in the
system of lifelong education as an effective factor of socio-economic development. Financial
and Credit Activity-Problems of Theory and Practice 2 (23), pp. 426-436 (2017). DOI:
10.18371/fcaptp.v2i23.121202
51. Vasilyeva, T. A., Makarenko, I. A.: Modern innovations in corporate reporting. Marketing
and Management of Innovations Issue: 1, pp. 115-125 (2017). DOI:
10.21272/mmi.2017.110
52. Villegas-Ch, W., Arias-Navarrete, A., Palacios-Pacheco, X.: Proposal of an Architecture for
the Integration of a Chatbot with Artificial Intelligence in a Smart Campus for the
Improvement of Learning. Sustainability (Switzerland) 12(