The pandemic has affected all areas of life with education being no different. Distance learning has become the new normal in all levels of education. Learning analytics as a tool best utilized by the abundance of data should now be more lucrative and needed than ever. Prior to the change, we surveyed the students of two Finnish higher education institutes, Helsinki university and Satakunta university of applied sciences, about their opinions, attitudes and expectations of learning analytics and ethics associated with it. This study is a follow-up surveying only the students of university of applied sciences and aims to find changes that might have been brought forth by the sudden and drastic switch to distance learning only. In addition, this was a chance to gather information on the students' experiences on distance learning, and how they feel about the so called new normal. The results seem to indicate that while students' expectations towards learning analytics haven't changed, their attitudes to security of their data and ethical conduction of analytics have become significantly stricter. We also found out that students have diverse feelings about distance learning in general, indicating changes required to post-pandemic education.
The drastic change brought upon by the global pandemic has affected everyone.
Education has had to undergo a transformation to cope with the change, higher
education not being an exception
Copyright © 2021 for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0)
extreme limitations to travelling, contact, all forms of physical interaction between
societies. In education, this commonly meant change from physical contact lessons to
full-on online education. What higher education institutes had already been offering to
the world in smaller proportions became the new standard across all levels of education.
The focus of this follow-up study was to find out if students' attitudes, expectations
and opinions about learning analytics and ethics associated with it have been changed
by the pandemic. The original study was completed in the beginning of spring 2020
and was therefore done prior to the COVID-19 virus reaching pandemic status.
This paper is structured as follows. First, we present related studies with focus on articles related to learning analytics and ethics. Next, we discuss the topic of ethics in learning analytics in more detail. Then we disclose the results of the original study and present the setup of this follow-up study, including the design changes and conduction of the survey. Finally, the results are presented, discussed and a conclusion is drawn. 2
As stated in the introduction the basis for this study heavily references similar studies
done that measured the effects of the pandemic in education and the previous studies
done by our research group on the use and application of learning analytics in higher
education. The aforementioned
Learning analytics as a subfield of analytics is the measurement, collection, analysis
and reporting of data about learners and their contexts, for purposes of understanding
and optimizing learning in the environments in which it occurs
Fig 1. Students study progression monitoring graph in a study management system.
Depicts the completed, ongoing, and planned studies, f. ex. in Autumn 2020 the student has completed 15 credits, enrolled into 25 credits (including the completed) and is therefore 5 credits short of the school’s semester objective of 30 credits for Autumn 2020.
Data ethics always follow analytics since use of data comes with its own questions and
worries about safety, rights, and privacy. These questions and worries consider data
ownership, consent and handling of the students’ data. What are the rights of the data
owner, do the students really understand what they are enabling, where are the
boundaries of privacy set? While ethics can be seen as a concept of conduct that affects
the safety of learning analytics and rights of the data handler and owner, privacy can
also be separated from the three as a continuous negotiable boundary that is guided by
the ethics surrounding.
The previous study on the students’ perception about learning analytics and ethics
served as comparison to this study. In the previous study both students from a university
and a university of applied sciences were surveyed. The students of the previous study
found learning analytics to be a potential and welcome tool to aid them create their
personalized learning paths but did also convey their worries about data safety and data
transparency. Learning analytics was generally seen as a good addition but the data it
needs to be useful must be collected and handled by ethically justifiable methods. This
was made even more evident by the university students, who were stricter about the
usage of their data, its collection and handling.
The basis for this and the previous study are the ongoing and shortly concluding
learning analytics standardization projects in the Finnish higher education scene funded
by the national ministry of education
Prior to the previous study our research group also did a study on the application of learning analytics in different learning models where both students and teachers’ perspectives were considered. In the study it was found that teachers involved were positively taken by especially lower level analytics tools that would allow them to better facilitate course design on their terms and level of skill. The students of this study outlined similarly to the students of the previous and current study that while learning analytics is a welcomed addition to help them advance their studies there are ethical considerations considering the pros and cons of using their data to produce learning analytics results. It was also found during this study that the students want to know how their data is used to produce these results and that generally they would like to know more about how learning analytics work. (Nevaranta et al. 2019)
During the pandemic several studies and publications with similar scopes and
objectives were completed which were referenced on this article. As the online learning
became the new normal across education levels the allure of learning analytics seemed
to increase
The study was conducted in early 2021 with an electronic survey. The participants were
the students of Satakunta University of Applied Sciences. The survey was based on the
previously given survey
A total of 133 students (N=133) answered the survey. Most students were between 20 to 25 years or 31 to 40 years old. The age distribution of all respondents is displayed in Figure 2.
Fig 2. The age distribution of all respondents in the survey.
The student distribution between different majors is displayed in Figure 3.
Fig 3. The major subjects of all respondents.
As seen in the figure, most of the students were either business or technical majors.
There are some minor differences in distributions compared to the earlier survey,
The survey contained all the questions from the similar survey conducted a year earlier
Questions about students' knowledge of analytics and about their attitudes and perceptions towards learning analytics and the access to their study information.
Questions about students' expectations towards learning analytics, i.e. what they wish they could gain from it and how it could (and should) help them in their studies.
Questions about ethical and security issues associated with learning analytics. Questions about students' attitudes and experiences towards distance education during the Covid-19 pandemic.
The questions were provided with an electronic form using Microsoft Forms. Some additional descriptions of some of the concepts were provided alongside questions to ensure that the students understood all the questions correctly. These included the description of analytics, learning analytics and the project under which the study was done. The new statements about distance education and its effects included statements about how the students' learning has been affected, has the studying become harder or easier and do they find themselves receiving enough study counseling. The statements can be found in Table 4. 5.3
The results were first analyzed with Microsoft Excel. For additional analysis, the data was converted into quantitative form (where applicable) and analyzed with custommade Python scripts. The two central forms of analysis were hence 1.
Quantitative analysis of students' answers to questions 1 to 21 and differences to last year's results Qualitative analysis of open answers and the differences to last year's study results We also constructed correlation matrices between all questions in 2020 and 2021 surveys, respectively. This was done to ensure the validity of answers and to see whether there were any unforeseen connections between individual questions. 6
In this section, the results of the study are presented. When applicable, the corresponding results of the 2020 instance of the survey are also included. It should be noted that in 2020 the survey was conducted in both, University and University of Applied Sciences. To keep the results comparable, only the results of the latter are included in the comparison. 6.1
The significance of difference between distribution of answers was calculated with a two-tailed t-test. The difference was statistically significant (p < 0.05) in questions 2, 3 and very significant (p < 0.01) in question 4. Hence, it seems that in 2021 the students were less confident about what learning analytics means, whether IT systems help organizing their studies and whether learning analytics help them to follow their own progress. This could be due to different forms of data collection last year. Students were asked to take the survey and at the same time they were also informed about the goals of the APOA-project and shortly about learning analytics. This year in the pandemic situation the same forms of data collection was not possible. In the student union’s national survey, answers of the students in SAMK indicated that students in their own opinion are good or at least adequate in study preparedness, which also includes timetabling (SAMMAKKO, 2021). Possible reasons for that can be found in the open questions section. However, the absolute difference in mean values was still rather small.
The answers to questions in category 2, students' expectations towards learning analytics, are displayed in Table 2. 8. I wish that LA would help me follow my progress in studies 9. I wish LA would help me design my studies better 10. I wish LA would help visualize my progress in studies 4,110 4,211
-0,100 4,200 4,195
0,005 4,145 4,263 -0,118 11. I wish teachers would use LA to follow my progress 12. I wish that the institution would use LA to progress my studies 3,972
3,872 3,966 3,835 0,100 0,131
p 0,350 0,966 0,259 0,385 0,259 As seen in the table, the students' expectations towards learning analytics have not changed between the two observed years. The students still wish that analytics would help them to follow, visualize and design their studies better. Moreover, the students still hope that teachers and institutions would utilize learning analytics to follow their progress in their studies.
The answers to questions in category 3, measuring students' attitudes towards ethics and security of learning analytics are displayed in Table 3. 13. I get enough information about my data collected by institution 14. I can affect how my data is collected and handed in institution 15. I know what data is collected by institution 16. I accept the collection of my data by institution 17. I accept the utilization of my data by institution 18. Institution should follow laws and regulations to keep my data safe 19. My data collected by institution can be used for my own benefit 20. My data collected by institution can be used for institutions benefit 21. I know that my data is removed by institution after my graduation 2020
2021 3,131
2,248 3,179
2,571 2,876 2,030 As seen in the table, the students' attitudes towards the ownership and safe handling of their own data have changed in a year. The students still accept the utilization and collection of their data similarly to 2020 (questions 16 and 17), but in all other questions there is a significant change. Notably, the students feel that they don't receive as much information about data collection and usage (questions 13, 14 and 15) and they also have stronger opinions about secure storage and ethical usage of their own data (questions 18 to 21).
In the 2021 survey there was also a fourth category to measure students' perceptions
about distance and online education during the Covid-19 pandemic. The answers to the
fourth category are displayed in Table 4.
Studying remotely has been more difficult than regular learning
My digital and IT skills are sufficient for studying
I have received enough instruction for studying
Visualizations in Moodle help in tracking my study progress
Visualization in Loki helps in tracking my study progress
2021
3,152
4,439
3,553
3,833
3,909
As seen in the table, the students haven't found distance learning as more difficult and
feel that they have sufficient IT skills for learning remotely. Most of the students seem
to find visualizations useful in tracking their study progress as well. As a side note,
Moodle is the common open-source LMS used by many institutes globally
The survey included some open questions as well. The first three were the same as the
questions asked in 2020
The first question was about students' perception on using their data to progress or track their studies. Most of the answers were positive about this, such as (translations from Finnish by authors): "I find it positive that progress is tracked, and actions are taken if anomalies are detected. This way nobody gets left behind." "I think it's a good thing that teachers and tutors can follow my progress. This improves the discussion." "Anything that's helpful can be used." However, several students that were positive about utilization of their data also raised a concern about the privacy and security of this: "A good thing as long as data security in ensured." "As long as my study data is used to promote or guide my studies, this is ok. I don't want my data exposed to any outsiders." "The fact that my data is collected and can be observed by anyone else makes me a little uncomfortable. But for promoting my studies it is ok as long as the data is not shared with anyone else." There were also students who did not think that the data was utilized to full extent: "Students and teachers need a lot of training for this." "Should be utilized better." "Very good idea, but in practice we seem to be very far from actually utilizing data in counseling." Finally, some students were negative about the whole idea: "Fully negative, I don't want my data to be collected. Why should I justify this?" The second open question was about what kind of added value the students hoped learning analytics would bring to their studies. The answers were roughly divided into two categories. First, there were students who saw the potential of analytics, such as "Knowing where to progress and where to focus more." "Easier tracking of one's progress in studies." "Clearer and more visible learning goals etc." The other group consisted of students who did not find any potential benefits. Most of them were not skeptical about the benefits of analytics as such, but rather argued that the teachers are not likely going to utilize analytics in teaching: "Now that I know that this kind of data is collected, I assume that it will be utilized as well. Until now there have been no signs of using any kind of data by teachers or counselors" "Nothing. The teachers will track [the progress] like they have always done." "Nothing, the opposite. I don't think the teachers can utilize the data at all." Majority of answers to the second open question (N=34) were positive, only 7 of the answers were classified as negative.
The third open question was about whether the students see learning analytics as an ethically sustainable part of education in the future. There were 53 answers in total. The students who answered the question positively (N=16) listed results such as "I don't see any problem; I have nothing to hide." "I don't see ethical problems. Education is about completing things and it's important that the teacher gets the necessary information." There were also students who felt negatively towards this (N=6): "Not much. Traditional human contact would be nice, as would having fewer students so that the teacher will at least know everyone's names." "I should be able to refuse without consequences." Many students (N=24) however agreed that learning analytics can maybe conducted ethically if special consideration is paid towards data security and ethics in general: "I don't find analytics itself to be an ethical problem, but the rules and data handling need to be clearly announced." "If analytics is transparent and students know what they are committed to, I don't see a problem." "It's ethically acceptable as long as students give their permission and can see the data collected concerning them without any delays." Finally, we wanted to know what kind of changes the pandemic had to studying. Notably, the students' answers were mostly very strongly positive or negative. Positive comments were mostly related to flexible schedules and not needing to travel, such as However, many students found distance learning as a very negative experience: "It's easier to study while eating breakfast at home in my pajamas." "Easier to schedule things I do since traveling won't take any time" " I have a better computer and more displays at home than in school." "Schedules are hard to meet when you spend all your time at home." "I have no structure in life." "Concentrating and self-motivating at home are nearly impossible." 14 "I have no human contacts, all hobbies are forbidden, 14h per day looking at a computer screen [...] I have lost my mental sanity and have no interest in school." There was a total of 125 answers to the final question, which seems to indicate that the students had strong feelings about distance learning. In 44 of these the students reported positive feelings about distance learning while 61 answers indicated negative. The remaining included students who already had studied remotely before the pandemic. 7
It seems that the pandemic has changed some attitudes towards learning analytics strongly, while some attitudes have remained similar. The questions in category 1 (Table 1) reveal that the students still find analytics in general as a positive thing like the study a year ago. However, now the students feel less confident about analytics helping them to follow their own study progress. It is possible that due to the pandemic, the students now need to rely heavier on the analytics and may since feel less satisfied with the results. Students' expectations towards analytics (category 2, Table 2) seem to be like what they were before distance learning: they hope that learning analytics would help them follow, design, and visualize the progress of their studies.
The biggest changes in students' attitudes were in the third category (Table 3), which was about students' attitudes about ethics and security of learning analytics. It seems that students now feel significantly less informed about the collection, utilization, and removal of their data. The students still accept the collection and utilization of their data for their own benefit but seem to be much stricter about the ethical and security issues related to their data. Again, we find it likely that the students are more concerned about these issues because they have been dealing with them a lot more than before. Students did not have as much experience about distance learning and about ethical issues related to it when we conducted the survey last time before the pandemic started.
While the questions in the fourth category (Table 4) seem to indicate that the students' attitudes towards distance learning are quite neutral, the open answers draw a different picture. In fact, many students seem to have very strong positive or negative attitudes towards distance learning. While many of them appreciate the flexible schedules and working from home, many of them are worried about lack of motivation and the (sometimes devastating effect of) missing human contacts. Since the division seems to be quite deep, the organizations should seriously consider providing distance learning chances for some students even after the pandemic while returning to normal routines with those who want it.
The students' open answers reveal that many students are positive about the potential possibilities of learning analytics, but at the same time they are worried about the safety of their data and ethical conduction of analytics. It also seems that many students don't find the teachers' competence about utilizing analytics as sufficient. Again, some students seem to think that their data can be utilized freely (as they "have nothing to hide"), but in general there seems to be a stronger demand towards safer conduction of analytics. This is a positive thing, because it may indicate that the students are more aware of data collection and its effects on them. 8
We conducted a survey collecting students' perceptions about learning analytics and ethical and security issues related to them. The study was a follow-up to a study conducted a year ago, and our goal was to find out whether the pandemic had any effects on students' perceptions. Based on the results, it seems that the students' expectations towards analytics have not changed, but their attitudes toward data security and ethical conduction of analytics have become much stricter. We also found out that the pandemic has had a diverse effect on students: while some have enjoyed the flexibility and working from home, others felt strongly that their motivation had decreased and loss of contacts had been close to unbearable.
There are naturally limitations to this study, which we need to address in future studies. Unfortunately, due to different scheduling of the courses, we were not able to collect data from university students like we did a year ago when conducting the first iteration of the study. Hence, a comparison like what we did earlier was not possible this time. Hence, in the future we are going to conduct the survey in the university as well. Other future work should include finding the connection between students' perceptions and their actual utilization of learning analytics based on the statistics. We are also very interested to find out what kind of effect the return to contact learning will have on students' attitudes towards ethical conduction of learning analytics. AnalytiikkaÄly project. (2019). Research, development and implementation of learning analytics in higher education. Referenced on 31.05.2021. Retrieved from https://analytiikkaaly.fi/in-english/ APOA project. (2019). Learning analytics in the Universities of Applied Sciences. Referenced on 31.05.2021. Retrieved from http://apoa.tamk.fi/in-english/ SAMMAKKO. (2021). “Aika yksin on jäänyt.” Etäopiskelun tulos koonti 2021. Student unions survey on distance and online learning. (Finnish only). Referenced on 04.06.2021. https://sammakko.fi/wpcontent/uploads/2021/05/Eta%CC%88opiskelukysely2021_Tuloskoonti_06052021_ KORJATTU.pdf