Using learning analytics and dispositional learning analytics in teaching is difficult. Examples of their use are required for higher educational institutions and teachers. In this paper, we present a flipped learning approach in online settings (due to COVID-19) with particular emphasis on learning analytics and dispositional learning analytics. For this, an understanding of flipped approaches (i.e., flipped classroom and flipped learning) as well as the role of technology in the teaching context is required and presented. The role of technology includes (1) a digital learning system, (2) a conferencing system, (3) the collection and use of learning analytics and dispositional learning analytics, and (4) content-specific technology. Additionally, our aim is to present students' course feedback results from quantitative research methods course practices (2020, 2021) for preservice teachers (i.e., students; N = 70). The content is highly challenging for these students, causing fear, frustration, anxiety, and boredom. Generally, the results for pedagogy were positive, but the results of students' learning perceptions were lower. Based on the approach and results, discussion with new insights is provided.
Learning analytics has received attention in recent decades as a way to support students’ learning,
improve teaching, and offer personalized support [
The pedagogical approach in this case was built on a flipped learning approach in the course Quantitative Research Methods 2. With the flipped learning approach, more specific pedagogical practices, and the technology used, such as a digital learning environment and conferencing systems (i.e., Zoom and Teams), the possibility to collect both learning analytics and dispositional learning analytics data of students and content-specific technology (i.e., Statistical Package for Social Sciences [SPSS] and Excel) should be considered as a whole. 2.1.
Flipped learning [
For teachers who have more experience with traditional teaching approaches (e.g., lectures, online
lectures, demonstrations), Väisänen and Hirsto [
According to Akçayır and Akçayır [10] and O’Flaherty et al. [9], both positive and negative results from the previous research have been identified. Positive results include that flipped approaches may improve students’ learning outcomes [11], self-regulation [12], and orientation [13]; enhance positive learning experiences [14]; and enhance the possibility of deeper learning when teacher availability in needed points of learning is visible [15]. Moreover, a recent study by Sointu et al. [16] indicate that students consider their teachers as experts in pedagogy and the use of technology in teaching with the flipped classroom approach. Research indicates positive weak to moderate effects on student satisfaction with the flipped classroom in their meta-analysis [17]. Sointu et al. [18] identified five key factors of student satisfaction with the flipped classroom: guidance for teaching approach, aiming for understanding (i.e., combining theoretical information into practice), a safe learning environment, teachers’ content teaching skills, and students’ technological skills. The negative results of flipped approaches often include the need for students’ self-regulating and time management skills [19], as a lack of these skills can increase the unfamiliarity of the approach and cause students to fall behind in their learning [20]. In particular, students with higher task avoidance orientation and lower selfregulation skills should be considered with a flipped approach [21]. Thus, considering the elements of student satisfaction with a novel pedagogical approach and additional data from learning analytics and dispositional learning analytics, the meaningful use of technology may increase the possibility for more positive learning experiences and learning results. 2.2.
Technology should be considered and planned well in flipped teaching. The technology should not be overly complicated for either teachers or students. According to Koehler and Mishra [22], the use of technology should align well with the pedagogical approaches used and the content areas taught during the course. There are several different technologies that can be used for supporting different pedagogical needs [23]. Especially during the recent decade, the development of tools for supporting collaborative and student-centered pedagogies has been emphasized. The COVID-19 pandemic prompted the development of different conferencing systems. Along with different pedagogical approaches, the nature of the content knowledge taught poses its own demands for the technologies used [22]. In addition to teaching the content, the technologies should provide students with experiences of the technologies used within the content discipline, i.e., technologies that are used by experts in the field.
In this course’s online flipped learning approach, several technologies were needed to meet the needs outlined by the Koehler and Mishra [22]. Technologies had to provide the space for students’ collaborative and self-regulated learning activities. To provide students with information about their learning process, the elements of learning analytics had to be integrated. Finally, data analysis software was needed to provide students with experiences using authentic software for conducting quantitative research. To meet these goals, the following technologies were used for teaching and learning: 1. Digital learning environment (called Valamis) 2. Conferencing systems (Zoom and Teams) 3. Collection and use of learning analytics and dispositional learning analytics data from
Valamis 4. Content-specific technology for quantitative research methods (Statistical Package for Social
Sciences [SPSS] and Excel) (1.) The digital learning environment (i.e., Valamis) was used to deliver the learning materials to studentts. These materials included short online videos (theoretical and practical) about the content, handouts based on the videos, tasks based on the videos, and handouts as well as self-correcting quizzes to monitor learning. The aim of the materials was to support students’ learning and to help them understand and learn the content better. Moreover, students were able to complete the assignments during class and group work after the class. Students were often encouraged to do collaborative work with their fellow students. Even though collaborative work was supported, every student had to finalize each assignment, task, and quiz in the digital learning environment. In addition, when using a learning management system for teaching, it should be capable of collecting data from learning analytics and dispositional learning analytics for student support and research.
(2.) The rationale for using two conferencing systems came from the practice: Zoom was used for general teaching in the classroom setting and joint discussions, while Teams was used for individual or small group support for students. Moreover, Teams was the main venue for students to contact the teacher. Each meeting started with a general discussion in Zoom and responses to the questions that participating preservice teachers had. Afterward, students started to work in the digital learning environment and the teacher was available to support their learning either in Zoom or Teams. However, based on the rules made between the students and the teacher, Zoom was kept mainly silent during class because students wanted to concentrate on the materials in the digital learning environments. Thus, we decided that Teams was the venue where students could contact the teacher and help them out whenever needed. More precisely, the aim was to develop a safe environment for learning for students, and Teams provided a venue for this as they were able to discuss individually with the teacher. We also agreed with the students that the teacher check every thirty minutes that everything is okay in Zoom.
The tandem use of Zoom and Teams allowed the teacher to provide stronger support for students,
but it also required the teacher to use two devices (e.g., two computers, a computer and a tablet). The
use of Teams helped the teacher to provide more personalized or small-group support for the students.
Additionally, if several students had challenges with the content and contacted the teacher, in this
case, the teacher was able to form small groups in Teams, explain and overcome the obstacles, and
suggest to the students that they could continue working collaboratively. The use of Teams became
the natural way of working, and students were skilled in using both systems. With challenging
content in an online setting, the tandem use of conferencing systems enables not only students to
contact the teacher but also the teacher to contact individual students and support their regulation,
orientation, self-efficacy, and emotions for learning [
(3.) According to Sointu et al. [18], guidance is highly important in flipped courses. Along with guidance provided by the teacher, the elements of learning analytics were integrated into the digital learning environment to provide students with information about their learning processes. We assume this information provided by the digital learning environment as one form of guidance [24]. Students were also well guided in the use of analytics data themselves. Learning analytics data were collected and visualized for the students from their progress, quizzes, and tasks in the environment. With this, we aimed to guide and provide feedback to the students about their self-regulation and learning processes. With the learning progress data, the teacher could also gain more in-depth knowledge about students’ learning, such as understanding which content students struggled with and which content they found easier.
In addition, we used dispositional learning analytics (i.e., short questionnaires) to gain further insights about students’ orientation to learning (e.g., did they consider the course important for their future career), self-regulation and time-management skills, and emotions (enjoyment, boredom, and anxiety) toward the content. This provided an additional tool for the teacher to understand students’ mindsets better from outside the cognitive perspective. When students had challenges with time management, task avoidance, or emotions, the teacher was able to use more pedagogical practices to help them based on the knowledge gained from dispositional learning analytics data. For instance, the teacher could provide further guidance for the learning and technology, connect the content to the student's future profession, attempt to create the feeling of a safe learning environment online, and show emotional sensitivity and understanding for students’ anxiety toward the content. Overall, the use of both learning analytics and dispositional learning analytics provided the teacher with tools to understand more students’ learning in the online environment and support them when confronting challenges.
(4.) One of the main aims of this quantitative research methods course is to learn how to conduct
quantitative analysis using the Statistical Package for Social Sciences (SPSS) program. Thus, SPSS and
Excel were content-specific programs. The content of the course was divided into six topics for the
six meetings as follows:
1. General information, guidance for the teaching approach and programs used in the course,
and content related to variables and correlation
2. Reliability, composite scores, recoding, normality testing, and correlation
3. Validity, exploratory factor analysis
4. T-tests and non-parametric matches for them
5. Variance analysis and nonparametric matches
6. Regression analysis
SPSS was used mainly in the course content. Excel was also used at meaningful points (e.g., data
management, name and label transformations, calculations). The content of the course is highly
difficult for preservice teachers (see e.g., [
The flipped learning approach was implemented online due to COVID-19. In practice, all course
materials were available in the digital learning environment as self-study materials for the students.
Since the content is difficult for these students, the majority of the students participated in meetings
that had 1-2-week intervals. One meeting lasted three hours. Zoom and Teams were used in tandem
for online teaching and particularly to support the students. Learning analytics were guided for
students to monitor, and the teacher used learning analytics and dispositional learning analytics data
to support the students and better understand their needs for support. Previous research by Sointu et
al. [
In total, 99 preservice teachers participated in the eight-week quantitative research methods practices courses during late fall 2020 and late fall 2021: 40 participants in 2020 and 59 participants in 2021. Each yearly cohort had two classes. The convenience sample of this paper includes the responses of seventy (N = 70; N2020 = 28, N2020 = 42) preservice teachers. The course was a part of a project called Utilization of Learning Analytics in the Various Educational Levels for Supporting Self-regulated Learning (OAHOT) funded by Business Finland through the European Regional Development Fund (2020-2022). Students were invited to respond to the anonymous feedback questionnaire voluntarily based on their informed consent. Moreover, the OAHOT project was approved by the University of Eastern Finland (UEF) institutional review board decision (11/2020). The data were collected via an electronic system with UEF [26] study course form. Professor Jyri Manninen from UEF led the development of the feedback form. The feedback form adapted for this study includes eight questions, in which students were asked to assess the quality of the quantitative methods course on a scale of 0–5 (0 = Rejected, 1 = Sufficient, 2 = Satisfactory, 3 = Good, 4 = Very good, 5 = Excellent). The questions were: (1) Connection between learning objectives and content of the course, (2) Own commitment in studying in the course, (3) Interaction between students and teacher, (4) Learning atmosphere in the course, (5) Course instruction (i.e., teaching), (6) Functioning of the study methods, (7) My own learning in the course, and (8) General grade to the course. Moreover, the feedback form included open-ended questions not reported here. The data were analyzed with SPSS. We report descriptive statistics: mean (M), standard deviation (SD), and percentage distributions.
The results indicate rather positive feedback from the participants (see Table 1). The highest assessments from all measured areas were from the Learning atmosphere (M = 4.34, SD = .72), Course instruction (i.e., teaching) (M = 4.26, SD = .90), Functioning of the study methods (M = 4.23, SD = .87), Interaction between students and teacher (M = 4.22, SD = .84), and Connection between learning objectives and content of the course (M = 4.20, SD = .68). Generally, the course was considered quite successful as indicated by the general grade for the course (M = 4.32, SD = .74). The lower assessments in the course feedback survey were related to the own commitment or learning assessed by students. More precisely, students considered Own commitment in studying in the course (M = 3.57, SD = .99) and own learning in the course (M = 3.44, SD = .96).
These items were also investigated in frequency level as indicated with percentages in Figure 1. The majority of students considered the course very good to excellent in Learning atmosphere (89.2%), Course instruction (i.e., teaching) (81.4%), Functioning of the study methods (77.2%), Interaction between students and teacher (84.0%), and Connection between learning objectives and content of the course (89.2%). However, own commitment in studying in the course (sufficient = 4.3%, satisfactory = 7.2%, good = 31.0%, very good = 40.6%, excellent = 15.9%) and own learning in the course (sufficient = 4.3%, satisfactory = 8.6%, good = 37.1%, very good = 38.6%, excellent = 11.4%) were distributed more evenly.
The aim of this paper was to describe how flipped learning in online teaching was used with learning analytics and dispositional learning analytics. Additionally, we wanted to find some indicators of how students considered this type of approach. Based on the feedback form results, the highest ratings were given for Learning atmosphere, Course instruction (i.e., teaching), Functioning of the study methods, and Interaction between students and teacher. We see these as successful from the perspective of the course design. It can be assumed that the course elements created the feeling of a safe atmosphere, teacher accessibility, functional pedagogical design, and technologies for students. These results align with previous studies within the field of flipped learning [18]. Moreover, the Connection between learning objectives and content of the course was the fifth highest, indicating that the course followed the course curriculum. One important aspect of teaching is to open the curriculum during the courses, and the sequencing of flipped learning makes this possible. This relates to both the content and program (i.e., SPSS) used in the course. Generally, the course was considered quite successful as indicated by the general grade for the course. Altogether, this example provides a concrete approach to how to build a learning analytics-supported course including pedagogical approach considerations, interaction, and a safe atmosphere.
Interestingly from the students’ perspective, their own commitments in studying in the course and
own learning in the course were above three (i.e., at the level of good), but still, they deserve more
attention in the future. These results provide us with challenges that may be avoided by focusing
more on the pedagogy and use of technology. Typically, the area of quantitative research methods
has been perceived as a difficult or even scary topic (see e.g., [
However, it is also important to understand that many of the aspects of creating a successful
course are very much teacher-dependent and vary between teachers (i.e., teacher effect). Additionally,
this may vary between different disciplines and the content of these disciplines. This is particularly
important to notice when students need additional support and how teachers can approach these
support needs. For example, further guidance for the learning and technology, reasoning the content
to the future profession, attempts to create a feeling of a safe learning environment even online, and
emotional sensitivity and understanding particularly for students’ anxiety towards the content (i.e.,
quantitative research methods) can be approached differently by different teachers. In other words,
the approaches used in this study do not automatically work for every teacher. Teachers must
consider their own pedagogical and learning perceptions, skills, and teacher identity. In the case of
this course, the teacher had experience with and understanding of the flipped classroom [
Even though evidence of the functionality of the flipped learning approach online with learning analytics and dispositional learning analytics were found, limitations exist, and future research should be considered. First, even though the data were a representative sample of the course, it still was a rather small convenience sample. In the future, research should attempt to use larger samples and quasi-experimental designs. As noted above, teacher effect may exist in this study, thus sample sizes needed for multilevel modeling should be sought. Second, the results were based on a feedback questionnaire form. In the future, more robust rating scales should be used in investigating the phenomenon. With individual items presented in this study, there were no possibilities to investigate the reliability or the validity. Thus, future research should take these into account too. Additionally, no analysis was performed on the learning analytics data used in this study. Future studies should include learning analytics analysis. Finally, qualitative data from the students’ feedback forms provide an important venue to understand the phenomenon more thoroughly. Thus, future research should consider combining qualitative data and quantitative data with a mixed-methods approach.
This work was supported by a grant from Business Finland through the European Regional Development Fund (ERDF) project Utilization of Learning Analytics in the Various Educational Levels for Supporting Self-regulated Learning (OAHOT) (Grant no. 5145/31/2019). We would like to thank our funder. Additionally, this study was supported by the School of Educational Sciences and Psychology, Special Education unit, and the School of Applied Educational Science and Teacher Education. Thank you to these departments at the UEF. Additionally, our deepest gratitude to the OAHOT project researcher for the work well done. Last but not least, our sincere thanks to all students for participating in the course and responding to the course feedback form. [9] J. O’Flaherty, C. Phillips, S. Karanicolas, C. Snelling, T. Winning. The use of flipped classrooms in higher education: A scoping review. The Internet and Higher Education, 25 (2015): 85–95. https://doi.org/10.1016/j.iheduc.2015.02.002. [10] G. Akçayır, M. Akçayır, The flipped classroom: A review of its advantages and challenges.
Computers & Education, 126 (2018): 334–345. https://doi.org/10.1016/j.compedu.2018.07.021. [11] N. Tusa, E. Sointu, H. Kastarinen, T. Valtonen, A. Kaasinen, L. Hirsto,… M. Mäntyselkä. Medical certificate education: Controlled study between lectures and flipped classroom. BMC Medical Education, 18.1 (2018). https://doi.org/10.1186/s12909-018-1351-7. [12] C.-L. Lai, G.-J. Hwang, A self-regulated flipped classroom approach to improving students’ learning performance in a mathematics course. Computers & Education, 100 (2016): 126–140. https://doi.org/10.1016/j.compedu.2016.05.006. [13] J. Strayer, How learning in an inverted classroom influences cooperation, innovation and task orientation. Learning Environments Research, 15 (2012): 171–193. https://doi.org/10.1007/s10984-012-9108-4. [14] I. T. Awidi, M. Paynter, The impact of a flipped classroom approach on student learning experience. Computers & Education, 128 (2019): 269–283. https://doi.org/10.1016/j.compedu.2018.09.013. [15] M. B. Gilboy, S. Heinerichs, G. Pazzaglia, Enhancing student engagement using the flipped classroom. Journal of Nutrition Education and Behavior, 47.1 (2015): 109–114. https://doi.org/10.1016/j.jneb.2014.08.008. [16] E. Sointu, T. Valtonen, L. Hirsto, J. Kankaanpää, M. Saarelainen, K. Mäkitalo, A. Smits, J.
Manninen, Teachers as users of ICT from the student perspective in higher education flipped classroom classes. Seminar.net – International Journal of Media, Technology & Life-long Learning 15.1 (2019): 1–15. https://doi.org/10.7577/seminar.3402. [17] P. Strelan, A. Osborn, E. Palmer, Student satisfaction with courses and instructors in a flipped classroom: A meta‐analysis. Journal of Computer Assisted Learning, 36.3 (2020): 295-314. https://doi.org/10.1111/jcal.12421. [18] E. Sointu, M. Hyypiä, M. C. Lambert, L. Hirsto, M. Saarelainen, T. Valtonen, Preliminary evidence of key factors in successful flipping: predicting positive student experiences in Flipped Classrooms. Higher Education. The International Journal of Higher Education Research (2022) https://doi.org/10.1007/s10734-022-00848-2. [19] A. J. Boevé, R. R. Meijer, R. J. Bosker, J. Vugteveen, R. Hoekstra, C. J. Albers, Implementing the flipped classroom: An exploration of study behaviour and student performance. Higher Education, 74.6 (2017): 1015–1032. https://doi.org/10.1007/s10734-016-0104-y. [20] Y. Chen, Y. Wang, N. S. Chen, Is FLIP enough? Or should we use the FLIPPED model instead?
Computers & Education, 79 (2014): 16–27. https://doi.org/10.1016/j.compedu.2014.07.004. [21] L. Hyppönen, L. Hirsto, E. Sointu. Perspectives on University Students’ Self-Regulated Learning, Task-Avoidance, Time Management and Achievement in a Flipped Classroom context, International Journal of Learning, Teaching and Educational Research 18.13 (2019): 87–105. https://doi.org/10.26803/ijlter.18.13.5. [22] M. Koehler, P. Mishra, What is technological pedagogical content knowledge (TPACK)? Contemporary Issues in Technology and Teacher Education, 9.1 (2009): 60-70. https://doi.org/10.1177/002205741319300303. [23] T. Valtonen, S. López-Pernas, M. Saqr, H. Vartiainen, E. Sointu, M. Tedre, The nature and building blocks of educational technology research. Computers in Human Behavior, 128 (2022): 107123. https://doi.org/10.1016/j.chb.2021.107123. [24] S. Väisänen, S. Hallberg, T. Valtonen, I.-A. Tervo, J. Kankaanpää, E. Sointu, E., L. Hirsto, Pupils’ experiences of learning analytics visualizations in supporting self-regulated learning in an elementary school classroom. Seminar.net – International Journal of Media, Technology & Lifelong Learning. To appear. [25] E. Sointu, M. Saqr, T. Valtonen, S. Hallberg, J. Kankaanpää, C. Tuominen, L. Hirsto, Emotional behavior in quantitative research methods course for preservice teachers. Learning analytics approach, in: Proceedings of Society for Information Technology & Teacher Education International Conference. AACE. 2022. URL: https://www.learntechlib.org/primary/p/220997/. [26] UEF [University of Eastern Finland]. University-level surveys. Course feedback, 2022. URL: https://kamu.uef.fi/en/have-a-say-and-give-feedback/.