Towards an affective tutoring system with multimedia content adaptation

Abstract

Adaptation allows for the personalization of learning through the use of different strategies, for example, the adaptation of multimedia content. An Affective Tutoring System (ATS) adapts the learning process to the emotions presented by the student during the interactive session. However, the development of intelligent tutor systems has become more complex. One problem is the lack of consensus about standards for their implementation, which makes it difficult to generalize their effectiveness. In this research, we propose a model-based development architecture for an ATS that adapts multimedia content in the presence of different levels of anxiety, with the aim of being able to extend its application to multiple learning domains.

Keywords

• eol>ITS
• ATS
• e-learning design
• personalized learning

1. Introduction

Learning is not only associated with the cognitive context of the student but also with other factors such as their personality and emotions. Within the classroom the teacher influences the emotional state of the students and this in turn influences the effectiveness of the process [ 1 ]. However, each person experiences a different reaction to the same situation; therefore, decision making regarding pedagogical activities, thematic content, and presentation of the learning strategy will not have the same effect for an entire work group. A particularity of specific learning disorders and deficiencies in the area of mathematical didactics is the influence that the affective state has on the process. Mathematical anxiety is an example of a persistent emotion in students who face difficulties in carrying out any type of problem involving the use of mathematical knowledge. Among the characteristics of its treatment in a traditional way is the importance of fostering motivation and decreasing anxiety and frustration through personalized learning.

Personalized learning mediated by technology allows us to recognize and respond to different particular factors of users, such as the affective state at a specific time. An Intelligent Tutoring System (ITS) is a technological tool that can collect information from the learner and adapt the instruction to the learner’s needs [ 2 ]. Specifically, an Affective Tutoring System (ATS) adapts its response to the emotions presented by the learner during the interactive session [ 3 ].

Despite the extensive development in the research and creation of ATS, in many cases the possibility of reusing these tools has been limited [ 4 ] [ 5 ]. One of the factors influencing this limitation is the need to analyze the ATS requirements from the software development perspective, considering that the implementation should be agnostic to the technology and the learning domain, but not to the learning objective.

In ITS development, the best known approaches are the client-server architecture and the component architecture [ 6 ] [ 7 ] [ 8 ][ 9 ]. All are based on the basic model of an ITS, which is composed of 3 main modules and a final user interface, as shown in the Figure 1. Despite the effectiveness of the abstract model, there are many software functionalities that are not represented in it. For this reason, the development of an ATS almost always involves a double design stage. On the one hand, obtaining the competent elements of the basic model of an ITS; and on the other hand, software engineering process.

In [ 4 ], the authors propose a 5-layer architecture based on the basic components of an ITS that aims to address the need for ITS development whose effectiveness is replicable to different learning contexts. However, it still separates the definition of learning objectives and instructional design from the implementation of the software system.

On the other hand, there are methodologies for the development of instructional design that take up the stages of classical software engineering methodologies, such as the ADDIE model [ 11 ]. The disadvantage is that, although it makes a more explicit connection between the software and the learning process, it still separates the process of defining the domain and the system.

This research proposes the development of an ATS that takes into account the learner’s levels of anxiety, using a methodology based on the CAMELEON framework [ 12 ]. The objective is to develop a tool that adapts the multimedia content to the affective needs of the user, that can be reusable in different learning contexts, and that the development process unifies the definition of the objectives, elements of the learning process, and the software requirements. The general description of the proposed ATS is showed in Figure 2.

2. Related Work

Research on ATS has been conducted from three main approaches. On the one hand, a set of studies oriented to the identification of emotions in learning from the use of ITS [ 5 ]. Another approach has been the development of emotion measurement methods, which are classified into verbal methods, based on text and voice evaluation [ 13 ] [ 14 ], and nonverbal methods, such as face analysis and biosignals such as heart rate or neural activity [ 15 ]. A final approach has been given regarding the explanation of emotions in the technology-mediated educational context [ 16 ]. The latter is the most recent and with a smaller number of contributions to date. However, in the development of ATS, one of the largest referents is the multimedia cognitive theory of learning [ 17 ], which relates the stimulus derived from multimedia content to the creation of models in working memory and the integration of previous knowledge in long-term memory.

Considering the distribution of studies with respect to research approaches, a need to test the effect on the learning process in different contexts becomes evident. It should also be mentioned that the number of research from Latin American countries is minimal. The effectiveness of these tools in different contexts needs to be reported, since variations among users lead to different results that should be considered in the improvement of their development at the technological level.

3. Model Driven Architecture based methodology

For the development of the ATS that adapts the multimedia content, a methodology based on the use of the model-driven architecture (MDA) is proposed. The development of virtual environments faces increasing difficulties that have been attacked using the model-driven architecture (MDA), this architecture contemplates four basic components for the development of interactive systems: models, language, software and approach. MDA separates the conceptual design, which concretizes the functionality, as a platform independent model (PIM), from the corresponding implementation as platform specific models (PSM). All of them from a computation-independent model (CIM) (Figure 3).

The approach used is UsiXML [ 18 ] which is based on the CAMELEON Framework. This framework, in turn, consists of four development phases, namely: • TasksandConcepts(TC): describe the user tasks, the concepts related to the data model (concepts) that are required to perform these tasks. • Abstract User Interface (AUI): defines abstract containers and individual interaction components. Tasks are associated to containers for execution or to individual objects for manipulation. An AUI is considered as an abstraction of a concrete user interface with respect to the interaction modality. At this level, the user interface consists mainly of the definition of inputs and outputs of the system but nothing is defined about the interaction modality (graphical, voice, touch). • Concrete user interface (CUI): the concrete interface defines an interaction modality and as such the interface is composed of elements that describe it, concrete interaction objects (CIOs) in order to define the interface design and navigation widgets. The CUI is independent of any computing platform, although it makes explicit the appearance and behavior of an end-user interface, it is still a mock-up, which only works within a particular environment. A CUI can also be considered as a reification of an AUI at the top level and an abstraction of the end user interface with respect to the platform. • Fine User Interface (FUI): is the operational part, i.e., the running user interface on a computing platform.

Transformational user interface development finds its motivations in the concept of heterogeneity of information systems. In this case, heterogeneity refers to the variety of contexts of use for which a user interface has been designed. This heterogeneity emphasizes the need to be able to rely on the abstraction of details relevant to specific contexts. From these abstractions, it is possible to obtain specific representations. The advantage of accessing such representations is to be able to reason about a single (task) model and to obtain many different user interfaces. In the Figure 4 it is showed the layers of the CAMELEON framework.

4. ATS software architecture and development of a communication API.

In order to make the ATS development extensible, a set of services was developed that communicate with each other to provide general functionalities and support future specific functionalities. The general scheme is shown in the Figure 5. The modular design allows the software to be used in experimental environments, under conditions not yet fully defined. There is an ”Interaction” layer, consisting of a client that serves as an interface for users. The ”General Functionality” layer is responsible for exposing the API input to the system, registering new Domain modules, making use of the data processing that these Domains provide, and mediating between these modules to store data in the Persistence layer. The ”Specific Functionality” layer is a specialized service that contains what is needed to define a system task with specific objectives, the data schema of that task and the algorithms needed to process it. Finally, there is a ”Persistence” layer that is responsible for storing the data for storage and post-processing. Its main component is the database.

5. ATS implementation: supporting learning of algebra

An ATS to support the learning process for students taking high school algebra courses, using the proposed methodology, has been developed.The system is composed of four independent modules. A tutor module, which manages the interactive activities and processes; a student module, which analyzes and responds to the user’s data and needs; a domain module, which contains the educational resources pertaining to the subject; and finally, a final user interface that allows the student to interact with the system. The main flow of the designed system is described in Figure 6 The user can make use of different activities designed in the system that allow him to enrich his knowledge in the fundamentals of algebra, taking account his emotional state, mainly the mathematical anxiety that he may present during the sessions. The user interface was developed to make the use of the system simple, accessible and satisfactory in relation to the educational objectives it supports. Some examples of the final user interfaces are showed in Figure 7. The tests developed in this study will serve as a basis for improving the design of the system, as well as the teaching strategies that compose it.

6. Preliminary conclusions and future work

From the perspective of user-centered interface design, it is possible to integrate the general elements that constitute an ITS to support the general model and also specific needs, such as adaptation to the learner’s emotions. Following a proper construction of the interaction flow and from the abstractions of the user tasks, the software implementation and the learning objectives can be worked at the same time, which reduces the development time of this type of educational technology. The presentation of the UI directly influences the effectiveness of the pedagogical strategy when using educational systems, since the student takes the role of the end user for whom we must design and evaluate the UX.

As a case study to validate the tool, a study of its effect on the academic performance of a group of high school algebra students in the context of the city of Puebla, Mexico, will be carried out. For this study, users must have an electronic device with internet access, either by Wifi or mobile data, and that has a webcam (computer, tablet or cell phone). Each user will perform a total of 5 interactive sessions with the tutor system over the course of a week. During the first session, the system will perform a knowledge diagnosis to place the user in the activities of the corresponding level within the subject matter. Subsequently, an evaluation of the level of anxiety at the beginning of the training will be performed. During the learning activities, the system will record information about the user, through short questions and video camera monitoring.

The activities to be performed in the interactive sessions will be described at the beginning of each one, since they will be selected as a change in the affective and cognitive state of the student is detected. At the end of the week, the user will carry out an evaluation of knowledge on the topic addressed, as well as an evaluation of the system, by means of a questionnaire of 10 short questions. The system will update the level of anxiety observed and present the user with a final written evaluation.

From the result of this first software validation, we will be able to extend the context of use of the system to other learning domains, with heterogeneous user groups.

Figure 7: ATS final user interfaces

References

[1] R. Mohanan , C. Stringfellow , D. Gupta , An emotionally intelligent tutoring system , in: 2017 Computing Conference , IEEE, 2017 , pp. 1099 - 1107 .

[2] M. H. F. Zarandi , M. Khademian , B. Minaei-Bidgoli , et al., A fuzzy expert system architecture for intelligent tutoring systems: A cognitive mapping approach , Journal of Intelligent Learning Systems and Applications 4 ( 2012 ) 29 .

[3] M. A. Hasan , N. F. M. Noor , S. S. B. A. Rahman , M. M. Rahman , The transition from intelligent to affective tutoring system: a review and open issues , IEEE Access 8 ( 2020 ) 204612 - 204638 .

[4] N. T. Graf von Malotky , A. Martens , General its software architecture and framework , in: International Conference on Intelligent Tutoring Systems , Springer, 2020 , pp. 17 - 22 .

[5] A. A. Castillo Avila , J. M. González Calleros , J. Guerrero García , Revisión sistemática de la literatura sobre los sistemas tutores afectivos: 2001 - 2020 , Revista Brasileira de Informática na Educação 29 ( 2021 ) 928 - 956 .

[6] S. A. Kazi , Vocatest: An intelligent tutoring system for vocabulary learning using the” mlearning” approach ( 2005 ).

[7] P. Dašić , J. Dašić , B. Crvenković , V. Šerifi , A review of intelligent tutoring systems in e-learning, Annals of the University of Oradea 3 ( 2016 ) 85 - 89 .

[8] I. Padayachee , Intelligent tutoring systems: Architecture and characteristics , in: Proceedings of the 32nd Annual SACLA Conference , Citeseer, 2002 , pp. 1 - 8 .

[9] S. Dutt , N. J. Ahuja , M. Kumar , An intelligent tutoring system architecture based on fuzzy neural network (fnn) for special education of learning disabled learners , Education and Information Technologies 27 ( 2022 ) 2613 - 2633 .

[10] A. L. L. Cruces , F. De Arriaga, Reactive agent design for intelligent tutoring systems , Cybernetics & Systems 31 ( 2000 ) 1 - 47 .

[11] B. A. Jones , Addie model (instructional design) ( 2014 ).

[12] G. Calvary , J. Coutaz , L. Bouillon , M. Florins , Q. Limbourg , L. Marucci , F. Paternò , C. Santoro , N. Souchon , D. Thevenin , et al., The cameleon reference framework, deliverable 1.1, version v1. 1 , cameleon project ( 2002 ) campbell , e., maintaining accessible websites with microsoft word and xml , in: Proceedings of XML Europe , 2003 .

[13] R. Z. Cabada , M. L. B. Estrada , F. G. Hernández , R. O. Bustillos , An affective learning environment for Java , in: 2015 IEEE 15th International Conference on Advanced Learning Technologies, IEEE , 2015 , pp. 350 - 354 .

[14] J. M. Martin , A. Ortigosa , R. M. Carro , SentBuk: Sentiment analysis for e-learning environments , in: 2012 International Symposium on Computers in Education (SIIE) , IEEE, 2012 , pp. 1 - 6 .

[15] I. Arroyo , B. P. Woolf , W. Burelson , K. Muldner , D. Rai , M. Tai , A Multimedia Adaptive Tutoring System for Mathematics that Addresses Cognition, Metacognition and Affect, International Journal of Artificial Intelligence in Education 24 ( 2014 ) 387 - 426 . URL: http: //link.springer.com/10.1007/s40593-014-0023-y. doi: 10 .1007/s40593-014-0023-y.

[16] R. E. Mayer , Searching for the role of emotions in e-learning ( 2019 ) 3 .

[17] R. E. Mayer, The promise of multimedia learning: using the same instructional design methods across different media , Learning and instruction 13 ( 2003 ) 125 - 139 .

[18] Q. Limbourg , J. Vanderdonckt , B. Michotte , L. Bouillon , V. López-Jaquero , Usixml: A language supporting multi-path development of user interfaces , in: IFIP International Conference on Engineering for Human-Computer Interaction , Springer, 2004 , pp. 200 - 220 .