The use of new information and communication technologies contribute to improving the quality of the teaching and learning process. Educational multimedia facilitates the creation of teaching materials that guide student learning. The use of interactive media facilitates the understanding of the contents through various forms of information. The purpose is to ensure greater participation and improve the academic performance of students. The objective of this work was the implementation of an educational multimedia system integrated with the Kinect device. The project was developed by applying the cascade model consisting of sequential development. The test scenarios allowed to evaluate the functionality of the software, and the integration of the kinect device. The main objective was to create an interactive, dynamic and fun environment with full mastery of student participation, and teacher acceptance. In general, the use of any audiovisual-interactive medium favors learning through sensory organs such as sight and hearing.
At present, education has become one of the cornerstones of economic, social,
scientific and technological development. It is recognized as one of the most
precious and powerful social goods to achieve individual and collective well-being
[
Academia has had great transformations as human knowledge and
scientific and technological development advancements. Information and
communication technologies (ICT) have been integrated into all levels of education. Its
implementation offers unlimited possibilities to improve the teaching-learning
process [
The educational use that students give to ICTs is directly related to the
teacher’s approach, their needs, and the educational level. In this context, ICTs
are classified into three categories: ICT that supports the transmission of
messages from the sender to the recipient, ICT that supports active learning through
experimentation with the objects of study and ICT that facilitates interaction
to learn [
Multimedia-interactive systems are found in the ICT category that facilitates interaction to learn. Interactive tools are didactic resources that promote the acquisition of knowledge in a more dynamic and participative way for students.
Kinect technology applied in education promotes kinesthetic learning. This
learning consists of understanding and acquiring knowledge through
experimentation and movements. That is, learning through experimentation and practice,
games, models, even physical representations of what is studied, such as a globe
or a human skeleton [
The educational applications that integrate the Kinect device represent a classroom model. This allows exploring and implementing innovative teaching and learning strategies in a learning environment monitored and supervised by the teacher.
The work integrates multimedia technology and the Kinect device to
generate new learning experiences. The digitalization of academic content is used to
later add multimedia effects and create interactive environments. The objective
is to foster creativity, the ability to observe, classify, interact, discover and
complement new information with prior knowledge [
Multimedia consists of using various types of media to transmit, manage or
present information. These media can be text, graphics, images, audio and video,
among others. In the field of computing, multimedia is the combination of specific
software and hardware to store and present interactive content [
The use of multimedia is very extensive since it is present in almost all
forms of human communication. Different industries benefit from multimedia,
for example: for entertainment through the use of special effects in movies and
animation of cartoon characters; medicine through the use of augmented reality
and virtual reality to simulate human body organs; education with countless
applications according to the material and educational level; and among many
other fields. Multimedia is an interactive medium that offers the possibility of
presenting information in different ways, stimulating different senses through
the creation of motivating, pleasant and interactive environments for the human
being [
In this context, ICT applied in education has led to new teaching tools and
paradigms. Educational multimedia are educational materials or resources that
combine different types of information and media. These are monitored through
the computer to improve the teaching and learning process [
The goal is not only to acquire knowledge but also to understand the way
of learning by getting the most of the benefits of ICT in order to transform the
mental structures regarding the use of technology in the educational community.
Figure 1 shows the structure of the cognitive theory of multimedia learning.
The theory centers on the foundations of multimedia education and bases the
application of an interactive multimedia system as a teaching resource [
Multimedia learning refers to the construction of students’ mental models,
through the use of printed or spoken words and static or dynamic images. The
cognitive theory of multimedia learning argues that individuals possess two
systems for processing information: visual and verbal. The auditory input
corresponds to the verbal system, and text input or animation corresponds to the
visual system [
Based on these assumptions and the information obtained from interviews with teachers, the content presented in the multimedia system was designed using visual and verbal media such as images, videos, audios and texts. The design was evaluated by teachers and students to ensure the playful quality of the tool to generate a more attractive and easily understood environment. The integration of the Kinect device allows students to control and interact with the content of the multimedia system without having to establish physical contact with the computer. This is possible, thanks to the integration of a natural user interface that recognizes gestures, voice commands, objects and images. This level of interaction facilitates the psychomotor development of children. The purpose is to improve the body’s expression of the child as well as developing kinesthetic, spatial and communicative intelligence through the movements of the body. 3
The project was developed by applying the cascade model, whose methodological
approach rigorously orders the stages of the process for software development,
so that the beginning of each stage must wait for the end of the previous stage.
Next, the development phases and their results are described [
Requirements Analysis] The following describes the requirements, scope and limitations of the multimedia system.
Objective. To develop and implement a multimedia desktop system that presents
the academic content of the Mathematics and Language subjects of the first
grade of primary school established by the Ministry of Education of Ecuador.
Hardware component. The multimedia system requires the use of the Kinect
device for Xbox360. The device is the interface of interaction between the system
and the users. The goal is for students to learn with body activity through the
use of their hands. The Kinect device is a tool created for games, it contains
SDK (Kit Software Development) libraries that help create interaction. It works
by capturing the human skeleton and then shows the information in the X, Y,
Z plane of its movements [
System modules. The system will consist of two modules. Each module contains tasks and activities to develop and practice.
Mathematics: add and subtract, decomposing numbers, color geometric figures, select correct quantities in additions.
Language: sentence structure, complete words, recognize syllables. 3.2
Following, it is shown the decomposition of the system organized using diagrams that represent its processes and functions.
System scheme. Next, the scheme of operation of the main processes of the system is presented. Fig 2. Represents the process to capture information about the movement of the person in front of the Kinect device and then process it and present it on the screen. Fig. 3 Shows the functional process of the system started with the Kinect device.
System design. Fig. 4 shows the communication between the user, the Kinect device and the computer. The user generates the movement, the sensor receives the movement through the camera and the computer validates the received data to finally show the results on the screen.
Kinect device operation. Fig. 5 shows the operating design of the Kinect device integrated with libraries (SDK) that make it easy to develop applications with Kinect.
Interface Design . The following images show the design of the system interfaces.
Different test scenarios were designed to verify the functionality of the system for each specified component. Table 1 describes the component interaction scenario.
The functional results of each component were 90% successful. The system manages to capture the movements of the human body through the camera of the Kinect device. The tests were carried out in a real scenario with first-year basic students as shown in Fig. 12. The different activities and practices presented in the system were validated with 90% effectiveness. 4
A survey was conducted to assess the usability level of the system. These questions were applied to a total of 34 students in the first year of primary school. Fig.13. represents the percentage of acceptance and interest in using the multimedia system. 87% of the students strongly agreed to use the multimedia system; they also consider it is easy to use and understand.
Fig. 15 represents the ease of system interaction. 76% of the participants mentioned that the interaction through the movements of their hands and body is easy to execute and also innovative. However, 15% of the participants disagreed about the ease of interaction. This is due to the slow interaction when the depth sensor captures the movement of both hands at the same time. 5
The theoretical references promote the use of technological resources of Educational Multimedia in teaching. It is necessary to complement traditional learning styles with technological and innovative resources for students.
The application of multimedia with Kinect technology in the educational field facilitated the teaching and learning process. The use of various types of information creates a pleasant, fun, attractive and motivating environment for the student. It facilitates the understanding of the contents employing sensory organs such as sight and hearing, including the development of kinesthetic intelligence.
The multimedia system is an ambitious project that seeks to become an innovative technological tool in the area of primary education. The Kinect device’s depth camera recognition allows users to interact through their body movements creating a different learning experience.
The educational community agreed that the multimedia system used as a teaching resource improves the academic performance of students in the classroom. This interactive system that does not use cables provides greater security in participation and allows students to stay focused. The acquisition of knowledge is through the display of information and interpretative elements that help to better understand the object of study; It also fosters a socio-technological culture in the institution. 6
For future work related to Kinect technology for educational purposes, or to replicate this research, it is recommended to extend the topics and areas of study according to the academic programs and levels. Also, the academic performance of students, as well as the development or improvement of their skills, and specific competencies related to multiple intelligences must be observed in depth.
To improve the operation of the Kinect device, it is necessary to place it in a fixed position so that the depth chamber correctly senses the person in front of it; otherwise, the device will sense a wrong position without detecting the person. The approximate distance between the person and the Kinect device should be between 1.20m and 2m distance. The distance between the ground and the Kinect must be between 0.70cm and 1.20m high to work without problems; otherwise the depth sensors will not recognize the person, and the interaction with the device will be interrupted.
Finally, the Kinect device must be away from heavy-duty machinery to prevent damage due to problems of loading or unloading voltage. Prevent lights or sunlight from shining directly onto the sensor; Besides make sure that no foreign object blocks the lens of the sensor. These recommendations guarantee the optimal operation of the device.