Decision Support System for Quality Management in Learning Process Nataliia Komleva1[0000-0001-9627-8530], Vira Liubchenko1[0000-0002-4611-7832], Svitlana Zinovatna1[0000-0002-9190-6486], Vitaliy Kobets2 [0000-0002-4386-4103] 1 Odesa National Polytechnic University, 1 Shevchenko av., Odesa, 65044, Ukraine komleva@opu.ua, lvv@opu.ua, zinovatnaya.svetlana@opu.ua 2 Kherson State University, 27 Universitetska st., Kherson, 73000, Ukraine vkobets@kse.org.ua Abstract. The authors describe the design of a decision support system, which allows automating the works on extracting information from the survey results. The generalization of recent publications had confirmed the relevance of the primary purpose of the paper. The domain analysis determined what tasks the decision support system should solve. The results of domain analysis became the base for the requirements specification. Logical and process views represent the system architecture design. A denormalized data structure, which acceler- ates the acquisition of aggregated data in different dimensions, is developed. The system design provides the work with various data sources as well as in- cremental development of the decision support system. Keywords: Decision Support System, Domain, Requirement Analysis, Archi- tectural Design, Data Structure. 1 Introduction The learning process in educational institutions is a systematic and purposeful activi- ty, which allows students to master a complex of knowledge and skills according to their chosen curriculum. Because of the higher education reforms implemented in Ukraine, the issues of control and quality assurance of the learning process are signif- icant; they are regulated by principles of the “Regulations on Accreditation of Study Programs in Higher Education.” The quality of the learning process, in the general sense, means the fit of the real learning outcomes to the requirements stated in the study program. At the same time, quality management should determine the means and techniques for achieving learn- ing outcomes. Until now, numerous procedures managed the learning process pro- gress are introduced. Although, some uncertainty about the choice of practices that increase its effectiveness remains. The variety of such practices is due to the presence of different forms of learning (full-time, part-time, distance, etc.) and the study condi- tions provided by the educational institution. An additional difficulty is the impossi- bility of a complete unification of learning processes, even within the scope of specif- Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). ICST-2020 ic forms and programs of study. Also, quality management of such a complicated process as the educational one involves the processing of large data sets whose pro- cessing results act as control parameters. All the above confirms the need for automa- tion in the tasks of managing the learning process progress. 2 Background Higher education institutions (HEIs) should provide both student satisfaction and their own business process fulfillment. Much HEIs are already collecting data to make decisions about critical changes in the learning process. It is appropriate to use the information technologies for the analysis of collected data. The paper [1] stated that “data refers to data sets that are large and complex that needs to be processed, stored, shared, analyzed, and used to help make smarter deci- sions in both the learning process and the management of the institution.” There are usually four types of databased analytics: Static Reports, Interactive Dashboards, Forecasts, and Recommendations. Educational institutions, like any other organization in the world, depend mainly on their ability to adapt to the environment. Accelerating change in all spheres of social life requires rethinking both goals and ways of organizing and behaving. Mak- ing better decisions in favor of education will undoubtedly affect the quality of educa- tional services, which will give HEIs greater competitiveness. It is well known that “the most competitive countries are those that are making the best use of ICT, which dominate and productively apply knowledge” [2]. Research [3] stated that “emerging higher education leaders will distinguish themselves by forming networks within and across institutions that engage stakeholders in the hard work of extracting actionable information from the data in their information systems, empowering frontline profes- sionals to understand and articulate relationships between the inputs and outputs of educational activities across the institution. The result is informed decision making is driven by mission, quality, cost, and revenue considerations.” Business analytics tools can be a powerful decision-making instrument [4]. “Busi- ness Intelligence systems can be defined as tools to assist and extend decision-making processes and make them more accurate and reliable, based on the knowledge gener- ated by the company’s data than intuitive values and personal experiences” [5]. In the context of increasing competition in the market of educational services, there is a need for special measures to promote activities of HEIs. A particular role in the automation of complex learning processes is played by decision support systems (DSS), which make recommendations based on research and analysis of significant aspects of such processes. Research [6] proposed a multi-criteria DSS for evaluating the transfer of knowledge from HEIs to society. In the process, three phases are com- pleted: strategic options development and analysis, measuring institution attractive- ness by a categorical-based evaluation technique and formulating recommendations. The paper [7] shows the use of DSS to choose university orientation that best fits the student’s skills according to her profile. DSS took part in university orientation programs and helped executive management to make appropriate decisions in direct- ing students to the most appropriate choice. Some research on student success considered the educational performance on par- ticular courses. For example, the study [8] sought to identify and characterize profiles of students based on academic performance in mathematics using random forest and classification and regression tree. At the same time, the authors used features related to individual and family behavior; that is, they associated the student’s learning suc- cess with his background and environment. Usually, experts play an essential role in the development of high-quality DSS. In many cases, they determine the recommended values of the learning process charac- teristics. The work [9] is devoted to the formalization of requirements for experts who take part in group decision support. The authors had developed a communication model and prototype to simulate decision scenarios. 3 Domain analysis and the practical issues The learning process involves numerous synchronous and asynchronous teacher- student interactions, nature, and frequency of which depend on the form and organiza- tion of learning. Traditionally, the effectiveness of the learning process is evaluated by the percentages of “success” and “quality,” which are calculated based on quantita- tive analysis of the grades obtained by students for different assignments. These char- acteristics of the learning process are named observable. As well the learning out- comes are determined by unobservable characteristics, reflecting the students’ ability to understand the learning outcomes, adapt to the learning process, effectively manage their study time and their learning activities, etc. Unobservable characteristics relate to such components of the learning process as the course content, teaching methods, and related teaching materials. The ability to measure and analyze the values of such characteristics allows evaluating the learning process quality more comprehensively and objectively. It is possible to estimate the unobservable characteristics through feedback from students, which is appropriate to organize with the surveys with appropriate question- naires. The purpose of such surveys is to get the students’ opinions in the form, which are suitable for processing and analysis. The survey results give insights about the learning process aspects that should be improved. It is advisable to develop a DSS to simplify the perception of the survey results when deciding on the learning process quality. DSS is designed to provide a complete and objective analysis of the data to assist decision-makers in difficult or uncertain environments [10]. The results provided by DSS differ in their complexity and value to the decision-maker. The simplest option is to provide statistical reports that aggre- gate the results of individual surveys in the form of tables and charts. Such infor- mation only simplifies the perception of the personal survey data; the decision-maker has to make the rest of the conclusions. Dashboards provide a visual representation of data grouped by a specific feature, for example, the answers on the same questions in surveys across different disciplines in the same year, the results of surveys by one course in the current and previous years, etc. Such comparisons make it possible to verify the effectiveness of the changes implemented, or provide the basis for conclu- sions about the need for changes. DSS can provide predictions about the expected efficiency of the learning process under certain conditions if there is sufficient histori- cal data on the results of surveys. Also, the use of information diagnostic methods provides an opportunity to identify the problem characteristics of the learning process and provide recommendations for possible actions in the diagnosed condition. Fig. 1 presents a formalized diagram that demonstrates the place of DSS to support monitoring procedures when implementing a student-centered approach in the learn- ing process. Fig. 1. The decision support system in the learning process For students involved in the learning process, questionnaires with questions relevant to the research purposes are prepared. The survey results are the basis for the DSS, so it is necessary to monitor the quality of the primary data, to control their accuracy, completeness, and clarity [11]. After the preprocessing, which eliminates the noise component of the primary data, the values of the unobservable characteristics are calculated. The diagnostic module determines the deviation of the calculated values of unobservable characteristics from their recommended values that were determined by experts. By type and magnitude of deviations, the recommendation module activates the relevant rules of the knowledge base related to the set of teaching tactics. Teaching tactics are measures taken to im- prove the quality characteristics of the learning process. All information received is provided to the decision-maker to ensure the objectivity and validity of decisions on improving the learning process quality. All related information that allows justifying the decision is saved in the data warehouse. Summarizing the above, we have to develop the data-driven DSS, which means the model of DSS should provide data acquisition, preprocessing, and manipulation. As well, the system should realize such a feature of knowledge-driven DSS as problem- solving for recommendation generating. The high-quality design of every listed DSS components ensures the effectiveness of the system as a whole [12]. 4 DSS requirements description We describe the functional requirements of DSS from the end user’s perspective with use cases. Each use case represents a set of possible sequences of interactions be- tween systems and users in a particular environment and relates to a specific goal. Fig. 2 presents the use case diagram. Fig. 2. The use case diagram of the decision support system The primary users of DSS are Expert and Teacher. The quality management of the course learning process should be performed following the package of quality charac- teristics. The Expert, together with the Teacher, determines the package of unobservable characteristics for the particular course. These characteristics are subsets of the com- plete package of unobservable characteristics of the learning process. As well, the Expert with the Teacher forms the teaching tactics base. The primary data for the DSS are the survey results. Answers on each question of the questionnaire allow getting values for one or several unobservable characteristics. An external Survey System conducts the survey and uploads the results. The structure of the questionnaire is determined by the set of unobservable charac- teristics defined for a particular course. When downloading the results, DSS should control whether the questions of the questionnaire cover all essential characteristics. If so, the questionnaire is complete, and the validity and reliability of the survey results are determined. This procedure is more complicated when using paper questionnaires and is some- what automated when using specialized questionnaires. Statistical processing of survey results is performed to obtain the aggregate charac- teristics of the course. A comparison of the aggregate characteristics with the recom- mended values allows diagnosing the state of the learning process. The methodology for statistical data processing depends on the scales of unobservable characteristics measurement; consideration of this aspect is beyond the scope of this paper. The decision about teaching tactics recommendation is taken based on the results of the learning process diagnosis. Teaching tactics are selected from the knowledge base; they guide according to the nature and intensity of the problem. A set of charac- teristics and their properties are defined for each tactic. The knowledge base deter- mines the fit between particular tactics and unobservable characteristics of the learn- ing process. The fit is subject to adjustment to improve teaching performance. The knowledge base has to be formed before the first use of the DSS; it cannot be empty and has to be verified for a clear interpretation of the rules. Here are some examples of the rules for knowledge base: • IF (Question = “Was the form of methodical material convenient?” AND Answer = “Disagree” AND Count>50%) THEN (“Change the form of methodical material”) • IF (Question = “Did you understand new material in the classroom?” AND Answer = “Not always” AND Count>50%) THEN (“Simplify delivery of new material” OR (“Prepare handouts” AND “Deliver handouts on time” AND “Provide an up- to-date communication channel”)) • IF (Question = “Did you understand new material in the classroom?” AND Answer = “Mostly not” AND Count>90%) THEN (“Change course syllabus”) 5 The architectural design of the DSS DSS is designed to meet the business objectives of the educational institution. The bridge between these abstract goals and the specific working DSS is an architectural project. The main issue of the architectural design is the complexity of the developed sys- tem. Usually, the software architecture is designed using well-known solutions that support the achievement of the business goals. Such a solution, named architectural style, helps to achieve the desired characteristics and behavior of the software system [13]. Based on the specified requirements, it can be argued that the DSS should be de- composed in such a way, which provides the possibility to design modules separately and connects them with small interactions. Such decomposition support further modi- fication and reusability. The design should be based on the sequence of data pro- cessing in the DSS. First, the data should be preprocessed, depending on how the survey was orga- nized. If the survey was conducted using paper questionnaires, then the answers should be digitized, which is scanned and recognized. If the survey was conducted using specialized tools such as Google Form or Survey Monkey, then the answers should be imported into the DSS. The prepared data are then stored in the DSS to allow working with them. The next step is processing and analyzing the data to obtain useful information for making decisions on quality improvement, such as quality diagnostics of the learning process, guidelines for methodological and didactic support changes, etc. Finally, the pro- cessing results are prepared for presentation to end-users. According to the context of the architectural design, it is appropriate to use archi- tectural style Layers, in which each layer has a clearly defined role and responsibili- ties [14]. A 4-layers architecture is sufficient for DSS. Take a closer look at the data sources layers, which provide data delivery. In particular, for scanned paper questionnaires, the layer realizes the text recogni- tion feature. For electronic questionnaires, the layer realizes interfaces to external survey systems. The storage layer is responsible for retrieving data from the data sources and con- verting them, if necessary, to a format that is suitable for future use. For example, in scanned questionnaires, all unanswered questionnaires can be deleted. This layer also provides DSS repository operation. In addition to processing the results of the survey, recommendations on learning process improvement measures should be made. Therefore, DSS requires its knowledge base on the properties of teaching tactics that are implemented in the data storage unit. The analysis layer provides all the procedures for preparing the required analytical results or finding appropriate recommendations. The consumption layer provides a visualization of the analysis results to help the user with information in the decision-making process. Each layer includes several types of components (Fig. 3). Note that DSS should be compatible with the environment, which will determine the specific implementation of the interfaces of the data sources and consumption. There are possible solutions for both direct interactions with the user and the transfer of data from/to other information systems. The initial implementation of DSS does not involve processing large data sets and performing complex computations, so it can be located on a single computing node to provide users with access to conventional network protocols. However, in the future, it makes sense to move to a service architecture through service unbundling [15], which will facilitate the support and further development of DSS. Fig. 3. Structural view of the DSS architecture The functional autonomy of the architectural elements makes the system framework flexible, extensible, and minimizes the “price” of architectural errors in specific im- plementations. 6 The core processes implemented by DSS After designing a static structure, we can describe the processes that ensure the proper functioning of DSS. Let us start with a description of the overall process of DSS op- eration (Fig. 4). There are two principally different ways in which DSS works: the first is activated when a system gets the results of a completed survey, and the second is activated when the system receives a user request. They differ in the frequency of activation and the sequence of actions performed. Fig. 4. Statechart of the DSS overall process The surveys provide results that significantly differ in characteristics such as the pur- pose, which determines the sets of evaluated characteristics and questionnaires, media (paper or electronic), coverage (students of a particular program, faculty or all HEI), frequency (at the end of a semester or once a year). Therefore, when the survey is completed, it is necessary to ensure that the raw data are uploaded to the DSS, and then need in data preprocessing appears. For data from paper questionnaires, prepro- cessing should clear the missed records that may arise due to inaccurate questionnaire completion by respondents. All the survey results should be structured to determine the fit between the particular questions and the analyzed quality characteristics. Final- ly, the structured data should be saved for future processing. The corresponding activ- ity diagram is shown in Fig. 5. Fig. 5. Activity diagram of processing the survey results However, the primary purpose of the DSS is to provide information that supports the decision-makers in the process of making a decision. An activity diagram describing the process of information extraction is shown in Fig. 6. Note that in this process, the results of validity assessment and information diag- nostics of the states are ancillary activities. However, they may also be involved in the future to support particular decisions. Fig. 6. Activity diagram of the information extraction process 7 Data structure The DSS should provide a variety of analytical results. It is desirable to support these capabilities at the level of data structures and to make use of OLAP-technologies. Let us take a closer look at the data model for DSS. After structuring the data, the survey results are presented as a table whose col- umns represent answers the questionnaire qj, and rows represent physical copies of questionnaires Ai filled in by different respondents. At the intersection of column and row, there is the answer vaij, which was given in questionnaire Ai to question qj. Fig. 7 shows a simplified data model for the normalized storage of survey results. A column-oriented table document should be transformed into a row-oriented format while saving the survey results into the database. The compliance between the questions of the questionnaire and the characteristics of the learning process should be defined for providing further analytical processing. The compliance can be considered as metadata in the database (highlighted by the dashed line in Fig. 7). Fig. 7. The normalized data model of the questionnaire database Surveys usually are conducted at the end of the semester to gather the data about the teaching of a particular course at a specified time interval by the specified teach- ing staff. DSS has to process a high volume of data to provide grounded solutions. It usually takes a long time. To accelerate the analysis, we use a denormalized structure [16], which represents a multidimensional cube (Fig. 8). Fig. 8. The denormalized data model of OLAP-cube The developed data model represents an entirely denormalized star schema and makes it possible to analyze the answers given in different terms with movement on the hier- archy levels in each dimension. The values of the measure in each slice serve as the input for further diagnosing the learning process state. 8 Conclusions Decision making in the field of learning quality improvement is a nontrivial issue. We described the design of DSS, which simplifies the decision-making providing useful information and giving the recommendations. The primary value of DSS consists in basing on unobservable characteristics of the learning process, which are not usually taken into account. These characteristics can be received from students’ surveys that represent actual feedback for the implementation of a student-centered approach in the learning process. The DSS includes modules for obtaining values of unobservable characteristics of the learning process and defining their deviations from the recom- mended values, which allows diagnosing the state of the learning process and provid- ing appropriate recommendations for its improvement. The developed DSS is adjusted to the learning process in a specific course, which allows taking into account all its features. The proposed framework allows various software implementations: for working with electronic or paper questionnaires, with the development of its database or using an interface to the database of other infor- mation systems, and so on. We described as well the denormalized data structure, which forms OLAP-cube. Its use can accelerate the acquisition of aggregate data in different sections. The au- tomation of data processing guarantees the complete record of all information re- ceived from participants of the learning process. As well in the future OLAP tech- nique will be involved in business intelligence tasks. Among the tasks are the justifi- cation of teaching tactics guidance, approving of the recommendations feasibility, modeling of the changes in the learning process, the forecasting of the unobservable characteristics for a given prediction horizon, and similar tasks. The results of the system performance can be easily scaled for higher education in- stitutions and their departments if uniform conditions for conducting educational pro- cesses are provided. References 1. Education Analytics. White paper. http://edudownloads.azureedge.net/msdownloads/ Mi- crosoftEducationAnalytics.pdf. (2017) Accessed: 19 Jan 2020 2. Leon-Barranco, A., Saucedo-Lozada, S. N., Avendaño-Jimenez, I. Y., Martínez-Leyva R., Carcaño-Rivera, L. A.: Business Intelligence in Educational Institutions. Research in Computing Science. 96, 43–53 (2015) 3. Soares, L., Steele, P., Wayt, L.: Evolving Higher Education Business Models: Leading with Data to Deliver Results. DC: American Council on Education, Washington (2016) 4. Guster, D., Brown, C. G.: The application of business intelligence to higher education: technical and managerial perspectives. Journal of Information Technology Management. XXIII(2), 42–62 (2012). doi:10.1088/1742-6596/1126/1/012053 5. Mussa, M. S., Souza, S. C., Freire, E. F. S., Cordeiro, R. G., Hora, H. R. M.: Business in- telligence in education: an application of Pentaho software. Revista Produção e Desenvol- vimento. 4(3), 29–41 (2018). doi:10.32358/rpd.2018.v4.274 6. De Almeida, M. V., Ferreira, J. J. M., Ferreira, F. A. F.: Developing a multi-criteria deci- sion support system for evaluating knowledge transfer by higher education institutions. Knowledge Management Research & Practice. 17(4), 358–372 (2019). doi:10.1080/14778238.2018.1534533 7. Hesham, F., Riadh, H.: How can one improve the logistics process of academic orienta- tion? Neural network programming to support the decision-making system in a university career. International Journal of Advanced and Applied Sciences. 7(1), 6–19 (2020). doi:10.21833/ijaas.2020.01.002 8. Sanzana, M. B., Garrido, S. S., Poblete, C. M.: Profiles of Chilean students according to academic performance in mathematics: An exploratory study using classification trees and random forests. Studies in Educational Evaluation. 44, 50–59 (2015). doi:10.1016/j.stueduc.2015.01.002 9. Carneiro, J., Saraiva, P., Martinho, D.: Representing decision-makers using styles of be- havior: An approach designed for group decision support systems. Cognitive Systems Re- search. 47, 109–132 (2018). doi:10.1016/j.cogsys.2017.09.002 10. Burstein, F., Holsapple, C.: Handbook on Decision Support Systems 1. Basic Themes. Springer-Verlag, Berlin Heidelberg (2008) 11. Krisilov, V. A., Komleva, N. O.: Analysis and Evaluation of Competence of Information Sources in Problems of Intellectual Data Processing. Problemele Energeticii Regionale. 1- 1(40), 91–104 (2019) 12. Fayoumi, A. G.: Evaluating the Effectiveness of Decision Support System: Findings and Comparison. International Journal of Advanced Computer Science and Applications. 9 (10), 195-200 (2018). doi: 10.14569/IJACSA.2018.091023 13. Bass, L., Clements, P., Kazman, R:. Software Architecture in Practice. Addison-Wesley Professional (2012) 14. Richards, M.: Software Architecture Patterns. O’Reilly Media (2015) 15. Liubchenko, V.: Queueing Modelling in the Course in Software Architecture Design. In: Shakhovska N, Stepashko V (eds) Advances in Intelligent Systems and Computing. III 871, 550–560 (2019). doi:10.1007/978-3-030-01069-0_39 16. Kungrtsev, A., Zinovatnaya, S.: Simulation model for research of denormalization effi- ciency of a relational database in the information system. Radio Electronics, Computer Science, Control. 1(19), 60–69 (2008)