=Paper=
{{Paper
|id=Vol-3353/paper9
|storemode=property
|title=Engaging Students in Secondary Science Learning Through the Scientific and Engineering Practices and Use of Digital Tools
|pdfUrl=https://ceur-ws.org/Vol-3353/paper9.pdf
|volume=Vol-3353
|authors=Jari Lavonen,Cesar Gonzalo Vera-Vasquez,Janna Inkinen,Kalle Juuti,Katarina Salmela-Aro,Klinge Villalba-Condori
|dblpUrl=https://dblp.org/rec/conf/citie/LavonenVIJSV22
}}
==Engaging Students in Secondary Science Learning Through the Scientific and Engineering Practices and Use of Digital Tools==
https://ceur-ws.org/Vol-3353/paper9.pdf
Engaging Students in Secondary Science Learning through the
Scientific and Engineering Practices and Use of Digital Tools
Jari Lavonen 1, Cesar Gonzalo Vera-Vasquez 2, Janna Inkinen 3, Kalle Juuti 4 , Katarina
Salmela-Aro 5 and Klinge Villalba-Condori 6
1,3,4,5
University of Helsinki, Finland
2
Universidad Continental, Arequipa, Perú
6
Universidad Nacional Pedro Henríquez Ureña, Santo Domingo, Dominican Republic
Abstract
Policy documents have suggested that school science should represent real scientific and
engineering practices and use of digital tools, in order to raise students’ low levels of interest
and engagement in science learning. In this study, a quasi-experimental design was used to
test the effect of Project-Based Learning (PBL) and use of digital tools on the engagement
and learning of upper secondary students. The experimental group (n = 29) was taught using
a PBL Newtonian mechanics teaching module, and the control group (n = 25) was taught
using traditional teacher-delivered lessons and practical work. Students’ engagement was
measured using an experience sample (ESM) instrument, and their achievement of learning
outcomes was measured using a cognitive test. We found that the PBL teaching module, which
emphasized collaboration, the use of scientific and engineering practices and use of digital
tools, engaged students in learning and supported them in achieving learning outcomes better
than traditional teaching.
Keywords
Project based learning, Scientific and engineering practices, digital tools, engagement.
1. Introduction
The Organization for Economic Co-operation and Development [10] designate that science is not
just test tubes and the periodic table. Science is the basis of nearly every tool we use every day, like a
simple can opener to the space explorer or the latest medical advances. Nor is science the domain of
scientists only. Everyone now needs to be able to “think like a scientist”: to be able to weigh evidence
and come to a conclusion; to understand that scientific “truth” may change over time, as new
discoveries are made, and as humans develop a greater understanding of natural forces and of
technology’s capacities and limitations.
Students’ low interest and engagement in science learning has been recognised worldwide [11]
[16]. The percentage of students in Finland who agreed or strongly agreed with the statement, “I enjoy
acquiring new knowledge in science” has decreased from 64% in 2006 to 56% in 2016. In 2016, the
percentage of Finnish students who expected to be working in a science-related occupation at age 30
was the lowest among OECD countries [12]. Therefore, in Finland, there is a need to develop more
engaging science teaching.
CITIE 2022: Congreso Internacional de Tecnología e Innovación Educativa, November 08-10, 2022, Arequipa, Peru
EMAIL: Jari.lavonen@helsinki.fi (J. Lavonen); cverav@continental.edu.pe (C. Vera); janna.inkinen @helsinki.fi (J. Inkinen)
kalle.juuti@ helsinki.fi (K. Juuti) katariina.salmela-aro @ helsinki.fi (K. Salmela) kvillalba@unphu.edu.do(K. Villalba)
ORCID: 0000-0003-2781-7953 (J. Lavonen); 0000-0003-4168-5117 (C. Vera); 0000-0003-2856-4601 (J. Inkinen) 0000-0001-8696-2611
(K. Juuti) 0000-0003-1901-4712 (K. Salmela) 0000-0002-8621-7942 (K. Villalba)
©️ 2022 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)
Due to this, the new Finnish curriculum framework has emphasised the importance of student
engagement in science learning [4]. The curriculum has suggested that students’ engagement could be
supported through contextualising learning and focusing on core scientific knowledge and practices,
including practices with digital tools to expand their knowledge acquisition. Scientific practices
involve students in asking questions and defining problems, planning and carrying out
investigations[21], analysing and interpreting data, and developing explanations and designing
solutions. The curriculum is in line with the European Commission’s Horizon 2020 Work Programme
[3], which suggested that school science should represent real scientific practices and cater effectively
to the needs and interests of young people.
Another issue, the various countries and organisations, like OECD and European Union emphasizes
is the use of digital tools in teaching and learning. According to the description in the Finnish
curriculum of what is involved in learning digital competences, students should first be able to do the
following:
• learn to use digital tools in diverse and creative ways
• collaborate and network with digital tools; and
• work with data, information, and
knowledge. Second, the student should be
guided in the following:
• critical and creative knowledge practices, such as searching information and generating
ideas;
• collaborative knowledge-building and the use of knowledge in different situations; and
• constructing and working with abstract artefacts, such as texts and concept maps, and
concrete artefacts, such as scientific models and modelling, with digital tools in different
in- and out-of-school earning environments.
2. Project-Based Learning
Project-Based Learning (PBL) is an approach that lets use multitude of strategies to learning. The
students drive their own learning through inquiry, as well as work collaboratively to research and create
projects that reflect their knowledge. The students learn technology and communication skills, and
advanced problem solvers.
• Krajcik and Shin [7] emphasised the following characteristics of PBL:
• PBL starts with a driving question, that is, a problem to be solved and focuses on the
learning goals of the curriculum that students are required to master
• Students are active in learning and explore the driving question by participating
collaboratively in scientific and engineering practices, including practices with digital
tools, that are central to expert performance in the discipline. They, for example, investigate
questions, propose explanations, and argue for their ideas.
• Students create a set of tangible products, like graphs and tables with digital tools that
address the driving question. These are shared artefacts, that is, publicly accessible external
representations.
The definition of PBL fit well with the description of aims of science education and role of digital
tools in education in Finnish curriculum. The science curriculum emphasises: learning of disciplinary
core ideas, which are important (across science and engineering domains); explanatory (used for
explaining phenomena); generative (used for investigating and solving problems); relevant (personal,
local, global contexts). The curriculum emphasizes also the use and learning of scientific practices or
the ways scientists and engineers study and design environment:
• asking questions, defining problems;
• planning and carrying out investigations;
• analyzing and interpreting data;
• developing and using models;
• using mathematics and computational thinking;
• developing explanations and designing solutions;
• engaging in argument from evidence; and
• obtaining, evaluating, and communicating information
Moreover, the curriculum emphasizes that students have an active role in knowledge building and
collaboration. The aims in the science curriculum are in line with what is described as important related
tpo the use of digital tools in learning as described above. Consequently, the curriculum is in line with
and could be implemented in classrooms through PBL, which is a pedagogical approach designed
according to learning science research outcomes.
Multiple educational projects have been developed by implementing PBL in science education. [5],
examined the effectiveness of Science, Technology, Engineering, and Mathematics project-based
learning lessons on students’ achievement in algebra, geometry, probability and problem solving. Their
study shows higher scores in geometry, probability, and problem solving than those in non-Science,
Technology, Engineering, and Mathematics project-based learning schools.
Also, Han et al. [5] [17] proposed a quasi-experimental design where quantitative data from three
focus groups were collected to assess student engagement within a STEM PBL classroom compared
to a non-STEM PBL classroom. They performed the exploratory factor analysis to more closely
examine the 8 engagement structures and resulted in the creation of two higher order factors,
(1) academic engagement (AE) and (2) behavioral engagement (BE). These results were used to
verify that there exists an improvement in student academic engagement between the intervention
groups, comparing traditional mathematics lessons verses STEM PBL lessons. The results showed that
the academic rigor and relevance provided through STEM PBL lessons increase students’ academic
engagement.
On the other hand, Graig et al. [1] found that students taught through PBL, as a group, matched
performance of conventionally taught students on all science 11th grade and mathematics 9th, 10th,
and 11th grade TAKS achievement measures and exceeded performance by a scale score increase of
133 for the 10th grade science TAKS measure by (B = 133.082, t = 3.102, p < .05).
Also, a recently research by Putri al. [13] present study is to examine the improvement of creative
thinking skill of primary school students in science through project based learning (PBL). Their
research used quasi experiment with non-equivalent control group design. The study involved 45 fifth
grade students at a public primary school in Karawang, West Java. The students were divided into two
groups i.e. experimental group (n=24) and control group (n=21). The students in experimental group
were given instruction through PBL; meanwhile, the control group was involved in traditional
instruction. Creative thinking test was used as pre- test and post-test to both groups. The data were
analyzed by using independent sample t-test to compare the creative thinking score between the
experiment and the control group. The result showed that the students in experimental group had better
creative thinking skill rather than the students in the control group. It can be concluded that project
based learning can effectively improve creative thinking skill of primary school students in science
class.
We approached situational engagement in the context of flow theory [2][14]. To be engaged, a
student should experience content- and context-specific situational interest in the task, which depends
on knowledge, values, and feelings. Second, students’ skills and mastery of a set of specific tasks
should be related to the activity, and the task should be at an appropriate level (situational resources).
Third, students should experience elevated levels of challenge and a desire to persist in a science
learning situation (situational task demands).
However, little is known about students’ engagement in scientific and engineering practices and
how teachers can employ these practices in their classrooms [9][15]. Therefore, a six-lesson (75 min)
PBL- module of basic Newtonian mechanics was designed in partnership with teachers and researchers
and implemented in a class (n = 29) as part of their first comprehensive physics course in upper
secondary school. Moreover, there was a control group of equal size.
The research question of the study was, “How did PBL teaching modules support students’
engagement in science learning and their achievement of expected learning outcomes?”
3. Method
3.1. Design of a PBL teaching module
Nonaka et al. argue that changes in professionals’ [10] [19] practices build on professional learning
or knowledge creation processes that span individual, and group levels where interacting with peers
and seeking help from more expert professionals are important. Therefore, for teacher networks and
partnerships with researchers could support the design and adoption of new pedagogy related to
engaging science teaching, reflection, and sharing and generating new ideas or pedagogy. This type of
research–practice partnership (RPP) has been suggested for bridging the gap between educational
research and practice and consequently, supporting teachers’ professional learning [6] [21].
We engaged in an RRP with 20 secondary science teachers for co-designing, implementing, and
reflecting on teaching units, emphasizing science and engineering practices, student collaboration, and
construction of artifacts to enhance student engagement. This teacher–researcher partnership is
expected to support teachers’ professional learning on pedagogy related to engaging science teaching.
The designed six hour teaching module focused to Newtonian mechanics and the driving question
was decided as: “Why do some objects take different amounts of time to fall from the same height?”
According to the learning outcomes of the module, students who demonstrate an understanding of
basic Newtonian mechanics can do the following:
• analyse data on motion, recognise when objects move with constant or changing velocity,
and use the basic models for motion with constant velocity and constant acceleration in
problem solving;
• analyse relationships among the net force on a macroscopic object, its mass, and its
acceleration and use Newton’s second and third laws in problem-solving;
• apply scientific and engineering practices to design, evaluate, and refine an experimental
design that could be used for modelling previous topics.
Following description of one lesson is made based on the video captured during the lesson. The
teacher starts his lesson through introducing the topic: “This week we will investigate to different kinds
of movements and reasons behind the movements. We will experiment and discuss. And you should
make a report based on your project [Project Based Learning activity]”. The driving question is “why
objects fall to the ground – sometimes at the same time and sometimes not at the same time”.
“In order to understand the driving question and pose relevant research questions we will start with
a demonstration. We will discuss on your observations and your conclusions and the evidence you
use”. In the demonstration same shape but different mass objects were given to fall and it was
happening what was not according to expectations. During the demonstration the teacher push his
students to wonder and come curious and the driving question comes understandable and it was
supportive for students’ thinking and posing relevant research questions for the next phase.
The students were asked write down questions related to the movement of a falling body and
questions related to reasons of the nature of movement. Students start working with the questions.
Later the questions were analyzed and discussed together.
The teacher summarizes the experiences: “This was the orientation to this week. We will explain
later what we observe. You might be wondering now. Let’s start studying the phenomena. However,
we have to split it down and study it part by part according to your questions. This is not a simple
phenomenon. There are many issues. You should have this phenomenon in your mind while we are
answering the guiding question.”
Next, the teacher shows a video clip considering the racket leaving Earth. The teacher explains that
this phenomenon actually belongs also to the topic they will analyze. The teacher asks students to
compare the phenomenon to a ball, which is thrown towards the sky. Similarities and differences were
discussed. The questions, students posed after the demonstration were analyzed and slightly modified.
The teacher told that they will start with the questions related to movement. Among the questions there
were questions, like “is the velocity constant?”; “How the movement change or is not changing?”;
How the velocity change, if it change?”.
Teacher: “Now we will engage in making experiments according to your questions. Here we have
equipment. You are free to use Microcomputer Based Laboratory (MBL) tools. For example ultrasonic
sensor could be useful.” After the introduction, the students start investigating according to their
research questions or engage in collaborative knowledge construction through employing scientific
knowledge practices. The students start planning their investigation according to their questions. The
investigations are focusing to modeling of movement of different falling objects. Student are
systematic in their activities according to the. While students are planning and working the teacher
walk around the classroom. He is asking questions and helping students to focus to the topic. The
teacher asks students, for example, “How do you feel when the lift goes up. Or actually start to go up.
How do you feel when the lift starts to go down? Please, think about what you feel in your body”. This
type of guiding helps students to orient to recognize the link between the net force and change in
movement. After the measurements, the students use long time in negotiating, modeling and
communicating. The models they created were models (artifacts) for two different type of movement
(movement with constant velocity and movement with constant acceleration and the preliminary
reasons for the change in velocity.
The driving question, “Why objects fall to the ground – sometimes at the same time and sometimes
not at the same time?” was partly solved. The students recognized that heavy objects accelerate the
whole falling and light object only a short distance. However, the reason for the change in movement
did not become fully clear.
3.2. Design of the study and data collection
A quasi-experimental design was used. The experimental group (n = 29) was taught using the
designed PBL module. The control group (n = 25) was taught using traditional teacher- delivered
lessons and practical work. The same teacher taught both groups.
The experience sample method (ESM) was used to evaluate how well students in the experimental
and control groups engaged in science learning through a specific smartphone application [14] [8]. In
the ESM questionnaire, 4-point Likert scale items were used to measure students’ situational interest,
skills, and challenges. We operationalized engagement as a state of involvement in a learning task
identified by higher- than-average individual states of interest, skill, and challenge. Moreover, students
were asked to select the type of science practice undertaken and whether they collaborated with other
students when signaled.
We used the taxonomy of Bloom, as a starting point for the design of test items to measure the
learning outcomes of the designed module. The dimension of cognitive processes was reduced to three
levels: remember, understand, and apply and create. Second, we took into account the national
curriculum framework aims and scientific (knowledge) practices emphasized in the PBL model. [18]
The scientific practices involved students in asking questions, developing and using models, planning
and carrying out investigations, analyzing and interpreting data, using mathematics and computational
thinking, developing explanations, engaging in arguments from evidence, and communicating
information. The curriculum has emphasized the use of conceptual and procedural scientific
knowledge in three situations: explaining phenomena scientifically, designing and evaluating scientific
inquiry, and interpreting data and evidence scientifically. The same computer-based test, with 13 items,
was used as a pre- and post-test for both groups. Figure 1 shows two test items.
012. A relevant research question 04. Forces acting on a ball
1 Take a look at this video related to a moving 1 Watch a slow-motion video of someone
sledge. kicking a football.
http://youtube.com/watch?v=3pk8gOFgmn Http://youtube.com/watch?v=v0zowDrCbE s
w 2 Take a screenshot from the video and paste
2 Pose two questions on the basis of which it is it to a drawing program of your choice. Draw all
possible to examine the links/correlation forces acting on the ball in the picture. Name the
betweenvariables relating to the phenomena in forces.
the video.
Figure 1: Two examples of test items.
4. Results
Table 1 shows the engagement of students from each group in scientific practices and working in a
group. Altogether, during the six 75 min lessons, there were 465 randomly-selected engagement
measurements in the experimental group and 445 measurements in the control group. Students in the
experimental group reported participating in more scientific practices.
than those in the experimental group, except for the practice of “using models”. Moreover, students
in the experimental group found most scientific practices and working in a group more engaging than
control group students. However, the differences between the groups were statistically significant only
for the practices of “developing models” and “making arguments”.
Table. 1a.
Scientific practices and percentage of engaging situations in the experimental (n = 29) and control (n
= 25) groups.
Experimental group Control group
Scientific Practices Number of Percentage Number of Percentage
(SP) SP ofengaging SP ofengaging
SP SP
Asking questions 14 6% 13 4%
Developing models 79 38% 62 18%
Using models 19 15% 36 15%
Planning 37 17% 22 7%
investigations
Conducting 102 17% 80 18%
investigations
Analyzing data 98 30% 80 21%
Solving problems 23 9% 22 12%
Constructing 84 26% 67 19%
explanations
Making arguments 19 23% 13 4%
Evaluating 116 32% 111 23%
information
Other 74 6% 147 25%
Total without other 591 26% 506 18%
Table. 1b.
Group work, and percentage of engaging situations in the experimental (n = 29) and control (n = 25)
groups.
Experimental group Control group
Scientific Number of Percentage Number of SP Percentage
Practices (SP) SP ofengaging ofengaging
SP SP
Working in a 93 13% 10 10%
group
There was no statistically significant difference between experimental and control group
performance in the cognitive pre-test. However, the experiment group performed better in the post-
test. The difference in performance was statistically significant (Table 2).
Table. 2.
Experimental and control group students’ performance in pre- and post-tests.
Experimental group Control group F
Mean S.D. Mean S.D.
Pre-test 10.7 3.6 10.2 4.0 0.27ns
Post-
15.3 2.7 12.0 3.9 13.1***
test
5. Discussion
We inferred that the implementation of the designed PBL made the difference. [9] argued that few
research papers have addressed students’ engagement in scientific practices and how teachers can be
guided to organize engaging science education. According to our pilot quasi-experiment, we inferred
that a designed upper secondary PBL Newtonian mechanics teaching module was better at engaging
students in learning and supporting students to achieve expected learning outcomes than traditional,
teacher-delivered lessons and practical work. The percentage of scientific practices in which students
were engaged was about 10% higher than in our previous measurements in traditional Finnish and US
science classes or in our control group class [14].
We interpret our results so that the BPL approach helps a teacher to integrate the curriculum aims
for science learning and aims related to the use of digital tools in learning. Moreover, the PBL approach
is supportive for learning and engagement. We summarize the characteristics of PBL in line with
Krajcik and Shin [7]:
• PBL starts with a contextualizing driving question, that emphasize the learning of
disciplinary core ideas through the use of scientific and engineering practices.
• Students are active in learning and explore the driving question by participating
collaboratively in scientific and engineering practices, including practices with digital
tools, that are central to expert performance in the discipline. They investigate questions,
work with data, information and knowledge propose explanations, and argue for their ideas.
• Students create a set of tangible products, like graphs tables with digital tools that address
the driving question. These are shared artefacts, that is, publicly accessible external
representations. These external representation are used for collaborative development of
models which are describing the phenomena which was inquired.
6. References
[1] Craig, T., Marshall, J. Effect of project‐based learning on high school students' state‐
mandated, standardized math and science exam performance. Journal of Research in Science
Teaching, 56(10), 1307-1488. (2019) DOI: https://doi.org/10.1002/tea.21582.
[2] Csikszentmihalyi, M.. Flow: The psychology of optimal experience. New York: Harper
Perennial.
(1990)
[3] EU.. Horizon 2020: Work programme 2016–2017: Science with and for society. European
Commissiondecision C(2016)1349 of 9 March 2016) (2016).
[4] Finnish Ministry of Education and Culture. Future upper secondary school: National aims and
allocation of lesson hours. (Opetus- ja kulttuuriministeriön työryhmämuistioita ja selvityksiä
2013). (2013).
[5] Han, S., Rosli, R., Capraro, M., & Capraro, R. The Effect of Science, Technology, Engineering
and Mathematics (STEM) Project Based Learning (PBL) on Students’ Achievement in Four
Mathematics Topics. Journal of Turkish Science Education (13), 3-29. (2016)
[6] Henrick, E.C., Cobb, P., Penuel, W.R., Jackson, K., & Clark, T. Assessing Research-
PracticePartnerships: Five Dimensions of Effectiveness. New York, NY: William T. Grant
Foundation.(2017).
[7] Krajcik, J., & Shin, N. Project-based learning. In K. Sawyer (Ed.), The Cambridge handbook
of the learning sciences (2nd ed., pp. 275–297). New York, NY: Cambridge University Press.
(2015).
[8] Lavonen, J., & Villalba-Condori, K. O.. Diseño colaborativo e implementación de herramientas
digitales en educación como parte de programas a nivel nacional en un sistema educativo
descentralizado. Education in the Knowledge Society (EKS), 20, 12. (2019)
https://doi.org/10.14201/eks2019_20_a23
[9] Minner, D. D., Levy, A. J., & Century, J. Inquiry‐based science instruction—what is it
and does it matter? Results from a research synthesis years 1984 to 2002. Journal of Research
in Science Teaching,47(4), 474–496. (2010). DOI: https://doi.org/10.1002/tea.20347.
[10] Nonaka. I., von Krogh. G.. & Voelpel. S. Organizational Knowledge Creation Theory:
Evolutionary Paths and Future Advances. Organization Studies. 27(8). 1179–1208. (2006).
DOI: https://doi.org/10.1177/0170840606066312.
[11] OECD. PISA 2015 results (Volume I): Excellence and equity in education. Paris: OECD
Publishing. (2016). DOI: http://10.1787/9789264266490-en.
[12] Osborne, J., & Dillon, J. Science education in Europe: Critical reflections. London: The
Nuffield Foundation. (2008).
[13] Putri, S., Sumiati, T. & Larasati, I. Improving creative thinking skill through project- based-
learning in science for primary school. Journal of Physics: Conference Series, 1157(2).
(2019).DOI: https://doi.org/10.1088/1742-6596/1157/2/022052.
[14] Schneider, B., Krajcik, J., Lavonen, J. M. J., Salmela-Aro, J. K., Broda, M., Spicer, J., Bruner,
J., Moeller, J., Inkinen, S. J. M., Juuti, K. P. T. & Viljaranta, J. H. Investigating optimal learning
momentsin US and Finnish science classes. Journal of Research in Science Teaching, 53, 400–
421. (2015). DOI: https://doi.org/10.1002/tea.21306.
[15] Villalba-Condori K.O., Adúriz-Bravo A., Lavonen J., Wong LH., Wang TH. Importance of the
Concept of “Competency” in Science Teacher Education: What Are the Professional
Competencies for Science Teachers?. In: Villalba-Condori K., Aduríz-Bravo A., Lavonen J.,
Wong LH., Wang TH. (eds) Education and Technology in Sciences. CISETC 2019.
Communications in Computer and Information Science, vol 1191. (2020) Springer, Cham
[16] T. J. Cateriano-Chavez, M. L. Rodríguez-Rios, E. L. Patiño-Abrego, R. L. Araujo-Castillo,
K. Villalba-Condori, Competencias digitales, metodología y evaluación en formadores de
docentes, Campus virtuales 10 (2021) 153–162.
[17] L. E. Quispe-Bendezú, R. L. Araujo-Castillo, J. E. García-Tejada, Y. García-Tejada, A. S.
Sprock, K. O. Villalba-Condori, Relationship between academic procrastination and
attributions of achievement motivation, International Journal of Learning, Teaching and
Educational Research 19 (2020) 188–205.
[18] R. Paucar-Curasma, K. Villalba-Condori, D. Arias-Chavez, N.-T. Le, G. Garcia-Tejada, I.
Frango-Silveira, Evaluación del pensamiento computacional utilizando cuatro robots
educativos con estudiantes de primaria en perú, Education in the Knowledge Society (EKS)
23 (2022). URL: https://revistas.usal.es/tres/index.php/eks/article/view/26161.
doi:10.14201/eks.26161.
[19] A. Garcia-Holgado, C. Deco, N. Bredegal-Alpaca, C. Bender, K. O. Villalba-Condori,
Perception of the gender gap in computer engineering studies: a comparative study in peru and
argentina, in: 2020 IEEE Global Engineering Education Conference (EDUCON), 2020, pp.
1252–1258. doi:10.1109/EDUCON45650.2020.9125224.
[20] K. O. Villalba Condori, R. Avello MartÃnez, Resiliencia como factor determinante para la
satisfacciÃcon la vida en estudiantes universitarios, EducaciÃMÃSuperior 33 (2019). URL:
http://scielo.sld.cu/scielo.php?script=sci_arttext&pid=S0864-
21412019000300007&nrm=iso.
[21] L. M. Oliva Córdova, H. R. Amado-Salvatierra, K. O. Villalba Condori, An experience making
use of learning analytics techniques in discussion forums to improve the interaction in learning
ecosystems, in: P. Zaphiris, A. Ioannou (Eds.), Learning and Collaboration Technologies.
Designing Learning Experiences, Springer International Publishing, Cham, 2019, pp. 64–76.