The authors describe the design of a decision support system, which allows automating the works on extracting information from the survey results. The generalization of recent publications had confirmed the relevance of the primary purpose of the paper. The domain analysis determined what tasks the decision support system should solve. The results of domain analysis became the base for the requirements specification. Logical and process views represent the system architecture design. A denormalized data structure, which accelerates the acquisition of aggregated data in different dimensions, is developed. The system design provides the work with various data sources as well as incremental development of the decision support system.
The learning process in educational institutions is a systematic and purposeful activity, which allows students to master a complex of knowledge and skills according to their chosen curriculum. Because of the higher education reforms implemented in Ukraine, the issues of control and quality assurance of the learning process are significant; they are regulated by principles of the “Regulations on Accreditation of Study Programs in Higher Education.”
The quality of the learning process, in the general sense, means the fit of the real learning outcomes to the requirements stated in the study program. At the same time, quality management should determine the means and techniques for achieving learning outcomes. Until now, numerous procedures managed the learning process progress are introduced. Although, some uncertainty about the choice of practices that increase its effectiveness remains. The variety of such practices is due to the presence of different forms of learning (full-time, part-time, distance, etc.) and the study conditions provided by the educational institution. An additional difficulty is the impossibility of a complete unification of learning processes, even within the scope of specific forms and programs of study. Also, quality management of such a complicated process as the educational one involves the processing of large data sets whose processing results act as control parameters. All the above confirms the need for automation in the tasks of managing the learning process progress. 2
Higher education institutions (HEIs) should provide both student satisfaction and their own business process fulfillment. Much HEIs are already collecting data to make decisions about critical changes in the learning process. It is appropriate to use the information technologies for the analysis of collected data.
The paper [
Educational institutions, like any other organization in the world, depend mainly
on their ability to adapt to the environment. Accelerating change in all spheres of
social life requires rethinking both goals and ways of organizing and behaving.
Making better decisions in favor of education will undoubtedly affect the quality of
educational services, which will give HEIs greater competitiveness. It is well known that
“the most competitive countries are those that are making the best use of ICT, which
dominate and productively apply knowledge” [
Business analytics tools can be a powerful decision-making instrument [
In the context of increasing competition in the market of educational services, there is a need for special measures to promote activities of HEIs. A particular role in the automation of complex learning processes is played by decision support systems (DSS), which make recommendations based on research and analysis of significant aspects of such processes. Research [6] proposed a multi-criteria DSS for evaluating the transfer of knowledge from HEIs to society. In the process, three phases are completed: strategic options development and analysis, measuring institution attractiveness by a categorical-based evaluation technique and formulating recommendations.
The paper [7] shows the use of DSS to choose university orientation that best fits the student’s skills according to her profile. DSS took part in university orientation programs and helped executive management to make appropriate decisions in directing students to the most appropriate choice.
Some research on student success considered the educational performance on particular courses. For example, the study [8] sought to identify and characterize profiles of students based on academic performance in mathematics using random forest and classification and regression tree. At the same time, the authors used features related to individual and family behavior; that is, they associated the student’s learning success with his background and environment.
Usually, experts play an essential role in the development of high-quality DSS. In many cases, they determine the recommended values of the learning process characteristics. The work [9] is devoted to the formalization of requirements for experts who take part in group decision support. The authors had developed a communication model and prototype to simulate decision scenarios. 3
The learning process involves numerous synchronous and asynchronous teacherstudent interactions, nature, and frequency of which depend on the form and organization of learning. Traditionally, the effectiveness of the learning process is evaluated by the percentages of “success” and “quality,” which are calculated based on quantitative analysis of the grades obtained by students for different assignments. These characteristics of the learning process are named observable. As well the learning outcomes are determined by unobservable characteristics, reflecting the students’ ability to understand the learning outcomes, adapt to the learning process, effectively manage their study time and their learning activities, etc. Unobservable characteristics relate to such components of the learning process as the course content, teaching methods, and related teaching materials. The ability to measure and analyze the values of such characteristics allows evaluating the learning process quality more comprehensively and objectively.
It is possible to estimate the unobservable characteristics through feedback from students, which is appropriate to organize with the surveys with appropriate questionnaires. The purpose of such surveys is to get the students’ opinions in the form, which are suitable for processing and analysis. The survey results give insights about the learning process aspects that should be improved.
It is advisable to develop a DSS to simplify the perception of the survey results when deciding on the learning process quality. DSS is designed to provide a complete and objective analysis of the data to assist decision-makers in difficult or uncertain environments [10]. The results provided by DSS differ in their complexity and value to the decision-maker. The simplest option is to provide statistical reports that aggregate the results of individual surveys in the form of tables and charts. Such information only simplifies the perception of the personal survey data; the decision-maker has to make the rest of the conclusions. Dashboards provide a visual representation of data grouped by a specific feature, for example, the answers on the same questions in surveys across different disciplines in the same year, the results of surveys by one course in the current and previous years, etc. Such comparisons make it possible to verify the effectiveness of the changes implemented, or provide the basis for conclusions about the need for changes. DSS can provide predictions about the expected efficiency of the learning process under certain conditions if there is sufficient historical data on the results of surveys. Also, the use of information diagnostic methods provides an opportunity to identify the problem characteristics of the learning process and provide recommendations for possible actions in the diagnosed condition.
Fig. 1 presents a formalized diagram that demonstrates the place of DSS to support monitoring procedures when implementing a student-centered approach in the learning process. For students involved in the learning process, questionnaires with questions relevant to the research purposes are prepared. The survey results are the basis for the DSS, so it is necessary to monitor the quality of the primary data, to control their accuracy, completeness, and clarity [11].
After the preprocessing, which eliminates the noise component of the primary data, the values of the unobservable characteristics are calculated. The diagnostic module determines the deviation of the calculated values of unobservable characteristics from their recommended values that were determined by experts. By type and magnitude of deviations, the recommendation module activates the relevant rules of the knowledge base related to the set of teaching tactics. Teaching tactics are measures taken to improve the quality characteristics of the learning process. All information received is provided to the decision-maker to ensure the objectivity and validity of decisions on improving the learning process quality. All related information that allows justifying the decision is saved in the data warehouse.
Summarizing the above, we have to develop the data-driven DSS, which means the model of DSS should provide data acquisition, preprocessing, and manipulation. As well, the system should realize such a feature of knowledge-driven DSS as problemsolving for recommendation generating. The high-quality design of every listed DSS components ensures the effectiveness of the system as a whole [12]. 4
We describe the functional requirements of DSS from the end user’s perspective with use cases. Each use case represents a set of possible sequences of interactions between systems and users in a particular environment and relates to a specific goal. Fig. 2 presents the use case diagram. The primary users of DSS are Expert and Teacher. The quality management of the course learning process should be performed following the package of quality characteristics.
The Expert, together with the Teacher, determines the package of unobservable characteristics for the particular course. These characteristics are subsets of the complete package of unobservable characteristics of the learning process. As well, the Expert with the Teacher forms the teaching tactics base. The primary data for the DSS are the survey results. Answers on each question of the questionnaire allow getting values for one or several unobservable characteristics. An external Survey System conducts the survey and uploads the results.
The structure of the questionnaire is determined by the set of unobservable characteristics defined for a particular course. When downloading the results, DSS should control whether the questions of the questionnaire cover all essential characteristics. If so, the questionnaire is complete, and the validity and reliability of the survey results are determined.
This procedure is more complicated when using paper questionnaires and is somewhat automated when using specialized questionnaires.
Statistical processing of survey results is performed to obtain the aggregate characteristics of the course. A comparison of the aggregate characteristics with the recommended values allows diagnosing the state of the learning process. The methodology for statistical data processing depends on the scales of unobservable characteristics measurement; consideration of this aspect is beyond the scope of this paper.
The decision about teaching tactics recommendation is taken based on the results of the learning process diagnosis. Teaching tactics are selected from the knowledge base; they guide according to the nature and intensity of the problem. A set of characteristics and their properties are defined for each tactic. The knowledge base determines the fit between particular tactics and unobservable characteristics of the learning process.
The fit is subject to adjustment to improve teaching performance. The knowledge base has to be formed before the first use of the DSS; it cannot be empty and has to be verified for a clear interpretation of the rules. Here are some examples of the rules for knowledge base: • IF (Question = “Was the form of methodical material convenient?” AND Answer = “Disagree” AND Count>50%) THEN (“Change the form of methodical material”) • IF (Question = “Did you understand new material in the classroom?” AND Answer = “Not always” AND Count>50%) THEN (“Simplify delivery of new material” OR (“Prepare handouts” AND “Deliver handouts on time” AND “Provide an upto-date communication channel”)) • IF (Question = “Did you understand new material in the classroom?” AND Answer = “Mostly not” AND Count>90%) THEN (“Change course syllabus”) 5
DSS is designed to meet the business objectives of the educational institution. The bridge between these abstract goals and the specific working DSS is an architectural project.
The main issue of the architectural design is the complexity of the developed system. Usually, the software architecture is designed using well-known solutions that support the achievement of the business goals. Such a solution, named architectural style, helps to achieve the desired characteristics and behavior of the software system [13].
Based on the specified requirements, it can be argued that the DSS should be decomposed in such a way, which provides the possibility to design modules separately and connects them with small interactions. Such decomposition support further modification and reusability. The design should be based on the sequence of data processing in the DSS.
First, the data should be preprocessed, depending on how the survey was organized. If the survey was conducted using paper questionnaires, then the answers should be digitized, which is scanned and recognized. If the survey was conducted using specialized tools such as Google Form or Survey Monkey, then the answers should be imported into the DSS.
The prepared data are then stored in the DSS to allow working with them. The next step is processing and analyzing the data to obtain useful information for making decisions on quality improvement, such as quality diagnostics of the learning process, guidelines for methodological and didactic support changes, etc. Finally, the processing results are prepared for presentation to end-users.
According to the context of the architectural design, it is appropriate to use architectural style Layers, in which each layer has a clearly defined role and responsibilities [14].
A 4-layers architecture is sufficient for DSS. Take a closer look at the data sources layers, which provide data delivery.
In particular, for scanned paper questionnaires, the layer realizes the text recognition feature. For electronic questionnaires, the layer realizes interfaces to external survey systems.
The storage layer is responsible for retrieving data from the data sources and converting them, if necessary, to a format that is suitable for future use. For example, in scanned questionnaires, all unanswered questionnaires can be deleted. This layer also provides DSS repository operation.
In addition to processing the results of the survey, recommendations on learning process improvement measures should be made. Therefore, DSS requires its knowledge base on the properties of teaching tactics that are implemented in the data storage unit.
The analysis layer provides all the procedures for preparing the required analytical results or finding appropriate recommendations.
The consumption layer provides a visualization of the analysis results to help the user with information in the decision-making process.
Each layer includes several types of components (Fig. 3).
Note that DSS should be compatible with the environment, which will determine the specific implementation of the interfaces of the data sources and consumption. There are possible solutions for both direct interactions with the user and the transfer of data from/to other information systems.
The initial implementation of DSS does not involve processing large data sets and performing complex computations, so it can be located on a single computing node to provide users with access to conventional network protocols.
However, in the future, it makes sense to move to a service architecture through service unbundling [15], which will facilitate the support and further development of DSS. The functional autonomy of the architectural elements makes the system framework flexible, extensible, and minimizes the “price” of architectural errors in specific implementations. 6
After designing a static structure, we can describe the processes that ensure the proper functioning of DSS. Let us start with a description of the overall process of DSS operation (Fig. 4). There are two principally different ways in which DSS works: the first is activated when a system gets the results of a completed survey, and the second is activated when the system receives a user request. They differ in the frequency of activation and the sequence of actions performed. The surveys provide results that significantly differ in characteristics such as the purpose, which determines the sets of evaluated characteristics and questionnaires, media (paper or electronic), coverage (students of a particular program, faculty or all HEI), frequency (at the end of a semester or once a year). Therefore, when the survey is completed, it is necessary to ensure that the raw data are uploaded to the DSS, and then need in data preprocessing appears. For data from paper questionnaires, preprocessing should clear the missed records that may arise due to inaccurate questionnaire completion by respondents. All the survey results should be structured to determine the fit between the particular questions and the analyzed quality characteristics. Finally, the structured data should be saved for future processing. The corresponding activity diagram is shown in Fig. 5. However, the primary purpose of the DSS is to provide information that supports the decision-makers in the process of making a decision. An activity diagram describing the process of information extraction is shown in Fig. 6.
Note that in this process, the results of validity assessment and information diagnostics of the states are ancillary activities. However, they may also be involved in the future to support particular decisions.
Surveys usually are conducted at the end of the semester to gather the data about the teaching of a particular course at a specified time interval by the specified teaching staff. DSS has to process a high volume of data to provide grounded solutions. It usually takes a long time. To accelerate the analysis, we use a denormalized structure [16], which represents a multidimensional cube (Fig. 8). The developed data model represents an entirely denormalized star schema and makes it possible to analyze the answers given in different terms with movement on the hierarchy levels in each dimension.
The values of the measure in each slice serve as the input for further diagnosing the learning process state. 8
Decision making in the field of learning quality improvement is a nontrivial issue. We described the design of DSS, which simplifies the decision-making providing useful information and giving the recommendations. The primary value of DSS consists in basing on unobservable characteristics of the learning process, which are not usually taken into account. These characteristics can be received from students’ surveys that represent actual feedback for the implementation of a student-centered approach in the learning process. The DSS includes modules for obtaining values of unobservable characteristics of the learning process and defining their deviations from the recommended values, which allows diagnosing the state of the learning process and providing appropriate recommendations for its improvement.
The developed DSS is adjusted to the learning process in a specific course, which allows taking into account all its features. The proposed framework allows various software implementations: for working with electronic or paper questionnaires, with the development of its database or using an interface to the database of other information systems, and so on.
We described as well the denormalized data structure, which forms OLAP-cube. Its use can accelerate the acquisition of aggregate data in different sections. The automation of data processing guarantees the complete record of all information received from participants of the learning process. As well in the future OLAP technique will be involved in business intelligence tasks. Among the tasks are the justification of teaching tactics guidance, approving of the recommendations feasibility, modeling of the changes in the learning process, the forecasting of the unobservable characteristics for a given prediction horizon, and similar tasks.
The results of the system performance can be easily scaled for higher education institutions and their departments if uniform conditions for conducting educational processes are provided. 5. Mussa, M. S., Souza, S. C., Freire, E. F. S., Cordeiro, R. G., Hora, H. R. M.: Business intelligence in education: an application of Pentaho software. Revista Produção e Desenvolvimento. 4(3), 29–41 (2018). doi:10.32358/rpd.2018.v4.274 6. De Almeida, M. V., Ferreira, J. J. M., Ferreira, F. A. F.: Developing a multi-criteria decision support system for evaluating knowledge transfer by higher education institutions. Knowledge Management Research & Practice. 17(4), 358–372 (2019). doi:10.1080/14778238.2018.1534533 7. Hesham, F., Riadh, H.: How can one improve the logistics process of academic orientation? Neural network programming to support the decision-making system in a university career. International Journal of Advanced and Applied Sciences. 7(1), 6–19 (2020). doi:10.21833/ijaas.2020.01.002 8. Sanzana, M. B., Garrido, S. S., Poblete, C. M.: Profiles of Chilean students according to academic performance in mathematics: An exploratory study using classification trees and random forests. Studies in Educational Evaluation. 44, 50–59 (2015). doi:10.1016/j.stueduc.2015.01.002 9. Carneiro, J., Saraiva, P., Martinho, D.: Representing decision-makers using styles of behavior: An approach designed for group decision support systems. Cognitive Systems Research. 47, 109–132 (2018). doi:10.1016/j.cogsys.2017.09.002 10. Burstein, F., Holsapple, C.: Handbook on Decision Support Systems 1. Basic Themes.
Springer-Verlag, Berlin Heidelberg (2008) 11. Krisilov, V. A., Komleva, N. O.: Analysis and Evaluation of Competence of Information Sources in Problems of Intellectual Data Processing. Problemele Energeticii Regionale. 11(40), 91–104 (2019) 12. Fayoumi, A. G.: Evaluating the Effectiveness of Decision Support System: Findings and Comparison. International Journal of Advanced Computer Science and Applications. 9 (10), 195-200 (2018). doi: 10.14569/IJACSA.2018.091023 13. Bass, L., Clements, P., Kazman, R:. Software Architecture in Practice. Addison-Wesley
Professional (2012) 14. Richards, M.: Software Architecture Patterns. O’Reilly Media (2015) 15. Liubchenko, V.: Queueing Modelling in the Course in Software Architecture Design. In: Shakhovska N, Stepashko V (eds) Advances in Intelligent Systems and Computing. III 871, 550–560 (2019). doi:10.1007/978-3-030-01069-0_39 16. Kungrtsev, A., Zinovatnaya, S.: Simulation model for research of denormalization efficiency of a relational database in the information system. Radio Electronics, Computer Science, Control. 1(19), 60–69 (2008)