=Paper=
{{Paper
|id=Vol-2730/paper32
|storemode=property
|title=Computer simulations for teaching an AI algorithm: the case of psychology students struggling with Support Vector Machines
|pdfUrl=https://ceur-ws.org/Vol-2730/paper32.pdf
|volume=Vol-2730
|authors=Federica Somma,Onofrio Gigliotta
|dblpUrl=https://dblp.org/rec/conf/psychobit/SommaG20
}}
==Computer simulations for teaching an AI algorithm: the case of psychology students struggling with Support Vector Machines==
https://ceur-ws.org/Vol-2730/paper32.pdf
Computer simulations for teaching an AI algorithm: the
case of psychology students struggling with Support
Vector Machines
Federica Somma1[0000-0003-4341-3393] and Onofrio Gigliotta1
1 Natural and Artificial Cognition Laboratory, Department of Humanistic Studies, University of
Naples Federico II, Naples, Italy
federica.somma@unina.it
Abstract. Computer simulations can be a great support in education, especially
in learning contexts deemed too complex such as AI. In this work we examined
how a web-based simulation implementing Support Vector Machines (an AI clas-
sification algorithm) can affect perceived learning and cognitive load in a group
of psychology students. Results of this preliminary work suggest that the specific
implemented simulation did not affect perceived learning but may have differ-
ently affected cognitive load in students with different educational backgrounds
(scientific vs humanistic). Differences related to the high-school educational
background are discussed and future perspectives are drawn.
Keywords: Simulations, AI, Support Vector Machines, education, learning
1 Introduction
Computer simulations under different forms have been used not only for basic re-
search [1] [2] [3] [4] [5] but also for educational or edutainment (education plus enter-
tainment) purposes (see for example [6] [7]). Simulation in education can be considered
a methodological support as the individual can directly manipulate and observe the
phenomena under study, thus fostering inquiry-based learning [8]. Inquiry learning is
defined as a process of exploration and manipulation which leads to asking questions,
discovering solutions and testing the findings in search of new understanding of expe-
rience. The use of simulation for teaching reduces the gap between theory and practice,
between concept and application. Students can become familiar with complex tools or
skills, directly observe the operation, hypothesize and design experiments.
Serious games for example [9][10] and simulated worlds are meant to ease the daunt-
ing endeavor of learning topics requiring a dynamical perspective; it is the case of social
psychology theories [10] in which agent-based modeling is crucial in simulating social
phenomena. Moreover, simulations are very useful especially when acquiring a certain
knowledge can be potentially risky and costly, for example in the case of flight-learn-
ing.
Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License
Attribution 4.0 International (CC BY 4.0).
2
Many of the above cited simulations make use of some form of Artificial Intelligence
(AI) under the hood. However, in this historical moment of expansion of AI algorithms
everywhere (see for example the many App boasting AI in their name across various
App stores), we assume it can be useful to disseminate to the general public, or at least
to professionals who may get some advantage for their own work, AI principles.
In fact, for professionals not inclined to hard disciplines like mathematics, computer
science and the like, AI may appear as a magic or, at the most, a very complicated topic;
however, they could benefit from learning some AI principles [11].
Accordingly, the aim of the present pilot study was to investigate perceived learning
and perceived cognitive load with respect to simulation-enhanced learning activity. Per-
ceived learning is indicative, across its three domains (cognitive, affective and psycho-
motor) of how a learner has acquired a specific knowledge without the need to resort
upon traditional form of grading requiring specific questions for each topic [12]. In
particular, the affective component highlights the personal attitude towards a specific
topic. This latter aspect to us is critical because it may mark a future interest in using
that technology for professional use.
Cognitive load or effort is defined as the load that performing a specific task imposes
on the individual's cognitive system [13]. Particularly, our initial hypothesis was that
performing a simulation while reading a related topic content, may have resulted in a
higher perceived learning and in a lower cognitive load.
2 Materials and Methods
2.1 Participants
65 Psychology students [Table 1] of the University of Naples “Federico” II were in-
volved in the experimental session and randomly allocated either to the group without
simulation (Text condition) or the group with simulation (Software condition). Selec-
tion criteria for participants recruitment included normal or corrected-to-normal vision.
Both groups completed an initial questionnaire on general information: gender, age,
qualification and high school field of study (humanistic or scientific). Informed consent
was obtained from all participants.
Group Software
Group Text (No
(With Total
simulation)
simulation)
Males 16 10
Gender N = 36 N = 29 65
Females 20 19
Age 21.6 (1.4) 20.9 (1.1)
Educational Humanistic 19 19 38
background Scientific 17 10 27
3
Table 1. Sample data
2.2 Measures
Users facing difficulty in understanding multimedia education systems, can overload
their mental resources. In order to increase student success in learning, overload should
be avoided. To this end, it is useful to measure the cognitive load with respect to the
use of simulations in order to determine if these multimedia environments are effective
and helpful for learning.
To obtain a measure of perceived cognitive effort the Cognitive Load Scale (CLS)
[13] [14] was administered to participants at the end of the session: CLS is a 9-point
self-rating scale to assess cognitive load in multimedia learning environments, based
on the cognitive load theory paradigm. CLS measures how much mental effort the in-
dividual invested in studying or in solving a task in a complex learning environment,
from very, very low mental effort to very, very high mental effort.
Scientific evidence suggests that self-reports of perceived learning may be a valid
measure of learning. In order to measure perceived learning, we administered the CAP
Perceived Learning Scale [12], that addresses three overlapping domains: cognitive,
affective, and psychomotor learning. CAP is a 9-item self-rating scale used to evaluate
the effectiveness of learning and educational environments. According to the theoreti-
cal background of the CAP scale, Cognitive learning is the recognition of knowledge
and development of intellectual skills, Affective learning is defined as an increase of
positive attitudes toward the content or subject matter, and Psychomotor learning is
associated with skills relating to manual tasks and physical movement that refers to
operations with the physical system, as the computer.
2.3 Procedure
Experimental sessions, due to COVID-19 restrictions, took place via a web-app.
First, participants completed the initial questionnaire on general information. Then,
we presented to the two groups of students an only text learning condition (Text Group)
and a text plus simulation condition (Software Group) and asked both to spend at least
30 minutes studying the provided materials. The text to learn was a descriptive text on
Support Vector Machines (SVM), an AI classification algorithm for breast cancer di-
agnostics; the simulation software, developed by [15], allowed students to directly ex-
perience the SVM simulator during the text reading.
After each session (only text or text plus simulation), students were administered
with Perceived Learning Scale and Cognitive Load Scale. Again, we hypothesized a
role of simulation in lowering cognitive load and leveraging perceived learning.
3 Results
Statistical analyses were conducted using JASP software (Version 0.12.2) [16]. All 65
participants were included in the analysis.
4
Regarding Cognitive Load, two-way ANOVA (with Group and Gender set as inde-
pendent variables) demonstrated that there was no variability between Text and Soft-
ware groups, F (1.04) = 0.422, p = 0.518, η2 = 0.006, thus both groups perceived almost
the same cognitive effort during the learning task (“Text Group” M = 5.722; “Software
Group” M = 6.069) between “neither low nor high mental effort” or “rather high mental
effort”. Similarly, the main effect of the gender variable was not significant, F (1.27) =
2.422, p = 0.125, η2 = 0.035.
A second two-way ANOVA was then conducted, always setting the Cognitive Load
as dependent variable and the Group (Text or Software) and the type of high school
formation (Scientific or Humanistic) as independent variables. The main effect of the
group variable was not significant, F (1.10) = 0.895, p = 0.348, η2 = 0.013, as well as
the main effect of the study field F (1.07) = 0.675, p = 0.414 , η2 = 0.010. However, a
significant interaction Group * Field of study resulted (F (1.61) = 5,384, p 0.024, η2 =
0.079).
Descriptive plots (Figure 3) revealed that, while the scientific studies group per-
ceived almost the same level of cognitive effort in both the software condition (M =
5.80) and text-only condition (M = 6.18), with a slight increase in the latter condition,
the group of students with a humanistic high school educational background perceived
more cognitive effort in the software condition (M = 6.21) compared to the text-only
condition (M = 5.32).
Fig. 1. Cognitive Load scores of Text (without simulation software) and Software (with simula-
tion software) Group separated by Field of study
We then analyzed any group differences in Perceived Learning Scale general index
(CAP): two-way ANOVA results showed significant difference regarding Gender, F
(1.41) = 9.590 , p = 0.003, η2 = 0.127), but not regarding Group, F (1,11) = 2.717, p =
0.104, η2 = 0.036) or regarding interaction effect Gender * Group, F (1,10) = 2.424, p
= 0.125, η2 = 0.032. Particularly, females in both Text (M = 35.050) and Software (M
5
= 34.895) Group expressed higher perceived learning than males in Text (M = 32.438)
and especially in the Software condition (M = 27.000) (Figure 2).
Fig. 2. Perceived Learning Scale general index of Text (without simulation software) and Soft-
ware (with simulation software) Group separated by gender
To better explore the data with respect to the single subscales of the Perceived Learning
Scale, two-way ANOVA was conducted for Cognitive, Affective and Psychomotor
Subscales. Results showed that the perception of affective learning did differ across
Gender, F (1.11) = 13.724, p = <.001, η2 = 0.172, as the gender main effect was statis-
tically significant: particularly females demonstrated higher perception of affective
learning both in Text and Software group.
4 Discussion
Analysis results do not reveal significant differences between the two groups, the one
who reads the text together with a Support Vector Machine simulator [15] and the one
who reads the text without the simulator, regarding perceived learning and cognitive
load. In particular, the perceived cognitive load [13] was rather high in both condi-
tions.
The results, however, highlighted a difference with respect to the educational back-
ground, as the students with scientific higher-education perceived less cognitive effort
in the simulator condition than in the text-only condition, while the students with hu-
manistic background perceived more cognitive effort in the Software condition rather
than in the Text condition. A more balanced sample with respect to the school back-
ground is necessary in order to draw conclusions on these results, however it could be
argued that a more developed inquiry learning style [8], such as that promoted by high
schools oriented to scientific studies, may help students to be more inclined to use a
6
simulator for learning rather than just reading the text. This hypothesis should be ex-
plored in future works.
An interesting aspect, however, emerged with respect to gender: results highlighted
that females demonstrated higher perception of affective learning in both groups, Text
and Software. In addition, affective learning perception was slightly higher in the group
with simulation than in the group without simulation. In contrast, males perceived more
affective learning in the Text group than in the Software group and both measures were
lower than the female’s ones.
Considering the limits of the present study given by the low sample size, which is
even not gender-balanced, it would be interesting to explore in more details the gender
perception difference to understand whether this difference could be interpreted in the
light of the target topic. Breast cancer topic is particularly pregnant for females, who
may have felt emotionally closer to the subject treated and may have perceived more
attitude to the topic. Therefore, the topic itself may have been the factor that influenced
the perception of greater emotional learning. In addition, the females who were in the
group with the simulator perceived even more affective learning: we cannot draw con-
clusions about it, however, it would be interesting to understand if the addition of a
simulation software together with meaningful topics for female participants could in-
volve them more in affective learning. However, due to the disproportion between male
and female participants, these are only hypotheses and there is a need for further studies
with a larger and gender-balanced sample.
It is also possible to reflect on the fact that higher affective involvement in the learn-
ing process could have had an impact on the perceived cognitive load of females in the
Software group: this hypothesis would be in line with literature data that underline how
emotion may relate to cognitive load during learning [17]. Learners’ emotional states
during learning could affect learning outcomes [18]. The processing of emotion and
cognition involves highly connected cortical networks; thus, information encoding
could be affected by emotion and emotion may directly affect cognitive resources and
mental effort investment [19] [20]. It would be interesting for future studies to evaluate
both cognitive load and perceived learning comparing non neutral and neutral topics of
the text and of simulation, to better understand the role that a specific theme has on
general and affective perceived learning, both for males and females.
It is necessary to take in account some limits of the present study: first of all, the
Support Vector Machine simulator was used in its original form, according to [15]'s
study, and not adapted to the sample, so in the future it would be useful to simplify and
tailor the instrument to the specific target. As already mentioned, a larger and more
gender-balanced sample, as well as a more educational background-balanced one,
would have made it possible to obtain more reliable and significant results, especially
regarding the affective scale’s scores. Another limitation of the study is that the CAP
Perceived Learning Scale [11] has been translated specifically for the present study as
no validation with an Italian sample is provided. In the future it would be necessary to
validate the Italian version in order to repeat the study with a validated scale. Finally,
given the restrictions due to the Covid-19 pandemic, it was not possible to carry out the
experimental sessions live, and the remote procedure did not provide for adequate con-
trol of the setting.
7
5 Conclusion
Studying the effects of simulations on perceived learning and cognitive load could be
useful in helping to tune specific software (as that used in for the present paper) to target
a specific population. Probably, the simulator used in the present study is still too com-
plex and could have increased cognitive effort instead of decreasing it; therefore it
would be useful for the future to simplify the usability of the software, perhaps adding
a virtual tutor able to guide the inquiry-based learning; without an efficient guide, in
fact, the risk to hinder rather foster learning is real [8]. Moreover, the usability and
engagement of technological tools is a key criterion that motivate the learning process
[21] [22].
In any case, the present study has highlighted a difference, albeit subtle, between
genders with respect to perceived affective learning, and new research hypotheses have
been put forward to understand whether this difference could be related to the specific
dataset (breast cancer data) used to train support vector machines and to explain how
the AI algorithm works. A difference with respect to scientific and humanistic educa-
tional backgrounds also emerged, that must be explored in more depth. Finally, future
works should focus on using measures of perceived learning and cognitive load to tune
simulation-based learning tools in order to maximize their educational impact on spe-
cific target audiences.
References
1. Miglino, O., Pagliarini, L., Cardaci, M., Gigliotta, O.: TeamSim: An Educational Micro-
world for the Teaching of Team Dynamics. In: Tsihrintzis, G.A., Virvou, M., Howlett, R.J.,
and Jain, L.C. (eds.) New Directions in Intelligent Interactive Multimedia, pp. 417–425.
Springer, Berlin, Heidelberg (2008).
2. Ponticorvo, M., Gigliotta, O., Miglino, O.: Simulative Models to Understand Numerical Co-
gnition. In: Ferrández Vicente, J.M., Álvarez-Sánchez, J.R., de la Paz López, F., Toledo
Moreo, J., and Adeli, H. (eds.) Natural and Artificial Computation for Biomedicine and
Neuroscience, pp. 75–84. Springer International Publishing, Cham (2017).
3. Pacella, D., Ponticorvo, M., Gigliotta, O. Miglino, O.: Basic emotions and adaptation. A
computational and evolutionary model. PLOS ONE 12, 1–20 (2017).
4. Gigliotta, O., Bartolomeo, P., Miglino, O.: Approaching neuropsychological tasks through
adaptive neurorobots. Connection Science 27, 153–163 (2015).
5. Rotondaro, F., Ponticorvo, M., Gigliotta, O., Pinto, M., Pellegrino, M., Gazzellini, S., Dolce,
P., Miglino, O., Doricchi, F.: The Number Interval Position Effect (NIPE) in the mental
bisection of numerical intervals might reflect the influence of the decimal-number system
on the Gaussian representations of numerosities: A combined developmental and computa-
tional-modeling study. Cortex 114, 164–175 (2019).
6. Miglino, O., Gigliotta, O., Ponticorvo, M., Nolfi, S.: Breedbot: An edutainment robotics
system to link digital and real world. Lecture Notes in Computer Science (including subse-
ries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 4693
LNAI, 74–81 (2007).
8
7. Council, N.R.: Learning Science Through Computer Games and Simulations. Washington,
DC: The National Academies Press (2011).
8. Jong, T. de: Technological Advances in Inquiry Learning. Science. 312, 532–533 (2006).
9. Rubinacci, F., Ponticorvo, M., Gigliotta, O. Miglino, O. Breeding robots to learn how to rule
complex systems. In: Merdan M., Lepuschitz W., Koppensteiner G., Balogh R. (eds.) Ro-
botics in Education. Advances in Intelligent Systems and Computing, pp. 137-142. Springer
International Publishing, Cham (2017).
10. Dell’Aquila, E., Marocco, Davide, Ponticorvo, M., Di Ferdinando, A., Schembri, M.,
Miglino, O.: Educational Games for Soft-Skills Training in Digital Environments. Springer
International Publishing, Cham (2017).
11. Gigliotta, O., Miglino, O., Schembri, M., Di Ferdinando, A.: Building up serious games with
an artificial life approach: Two case studies. In: Evolution, Complexity and Artificial Life,
pp. 149–158 (2014).
12. Rovai, A.P., Wighting, M.J., Baker, J.D., Grooms, L.D.: Development of an instrument to
measure perceived cognitive, affective, and psychomotor learning in traditional and virtual
classroom higher education settings. The Internet and Higher Education 12, 7–13 (2009).
13. Paas, F., Ayres, P., Pachman, M.: Assessment of cognitive load in multimedia learning: the-
ory, methods and applications. In Recent innovations in educational technology that facili-
tate student learning, pp. 11–35. Information Age Publishing Inc., Charlotte, NC (2008).
14. Schmeck, A., Opfermann, M., Van Gog, T., Paas, F., Leutner, D.: Measuring cognitive load
with subjective rating scales during problem solving: differences between immediate and
delayed ratings. Instructional Science 43(1), 93-114 (2015).
15. Walsh, P., Lynch, J., Kelly, B., Palu, C., Gigliotta, O., Fuccio, D.: Demonstrating Machine
Learning for Cancer Diagnostics. In: Collaborative European Research Conference, pp. 185-
194, (2019).
16. JASP Team (2020). JASP (Version 0.12.2) [Computer software].
17. Plass, J. L., Kalyuga, S.: Four ways of considering emotion in cognitive load theory. Edu-
cational Psychology Review 31(2), 1-21, (2019).
18. Heidig, S., Müller, J., Reichelt, M.: Emotional design in multimedia learning: Differentia-
tion on relevant design features and their effects on emotions and learning. Computers in
Human Behavior 44, 81–95 (2015).
19. Lazarus, R. S. Thoughts on the relations between emotion and cognition. American psy-
chologist, 37(9), 1019 (1982).
20. Pessoa, L.: On the relationship between emotion and cognition. Nature Reviews Neurosci-
ence 9(2), 148–158 (2008).
21. Ibili, E., Billinghurst, M.: Assessing the Relationship between Cognitive Load and the Usa-
bility of a Mobile Augmented Reality Tutorial System: A Study of Gender Effects. Interna-
tional Journal of Assessment Tools in Education 6(3), 378–395 (2019).
22. Soffer, T., Yaron, E.: Perceived Learning and Students’ Perceptions toward Using Tablets
for Learning: The Mediating Role of Perceived Engagement among High School Students.
Journal of Educational Computing Research 55, 951–973 (2017).