Analytics and its various subfields such as data analytics and learning analytics enjoy varying popularity as measured by public attention. Commonly, applications of learning analytics in the European area have been talked about lately partly because of the still fresh European General Data Protection Regulation (GDPR). Higher education institutions in Finland have all become active in standardizing learning analytics in recent years, with multiple nationwide multi-institutional projects conducted around the topic. In this article, we present a study where we surveyed the students about the ethical issues of data collection and usage in learning analytics. The survey was conducted in two universities in Finland: University of Helsinki and Satakunta University of Applied Sciences. In addition to finding out students' general perceptions of the topic, we wanted to find out if there are differences in the attitudes and opinions between different types of higher education institutes. Based on the results, students seem to be quite positive about the possibilities of learning analytics but are also concerned about the safety and usage of their personal data. The university students seem to be even more cautious, but also less informed about the data handling procedures.
The need for constant improvement is not a strange thing in today’s economy. We look for new and innovative ways to improve what we do in our everyday life and education is no different. Analytics is part of this improvement process where we are slowly progressing towards the efficient use of big data and as of the last decade or so, we have slowly been seeing subfields of analytics merge that factor in this trend. Learning analytics is the equivalent of using analytics to improve education and the learning process. As stated by numerous sources and studies, learning analytics is the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning in the environments in which it occurs. (Siemens, 2013)
Many countries have been involved in the study of learning analytics as of late and
so has the Finnish higher education community as well. The usual scope of such studies
has been the shaping of and application of learning analytics.
The focus of the study is to find out students' attitudes and opinions about learning analytics and the ethical dilemmas often associated with it. We also wanted to find out if there are any defining deviations between students’ attitudes and opinions in these two differing higher education models. It should be noted that the study was concluded during the beginning of spring 2020 and was not affected by the 2020 COVID19pandemic since the survey for the study was done prior to the virus reaching pandemic status. (Finnish Institute for Health and Welfare, 2020)
This paper is structured as follows. First, we present related studies with focus on articles related to learning analytics and ethics. Next, we discuss the topic of ethics in learning analytics in more detail. After this, we present the setup of our study, including participants, the design and the conduction of the survey. Next, the results are presented and discussed. Finally, we present our conclusion and some ideas for future research. 2
As previously stated there has been a substantial amount of studies in the subfield of
learning analytics in the last decade or so some of which will be used as original study
sources for our study. Learning analytics applications studies compared to many
dominant new fields like big data and AI have thus far been limited to a few but fruitful
studies and ever since the beginning of 2010’s several research communities have been
formed around the role of data analytics in education.
What do we actually mean when we talk about learning analytics? Early definitions
More recently Teasley (2019) stated that learning analytics has ever since become
increasingly broad as a subfield of analytics including now but not limited to the
aforementioned methods like data mining and statistics, but also semantics, learning
theories and more precisely focused studies. Since this would suggest that learning
analytics is getting more precision-based as more varying methods are used to do more
narrowed and focused studies, the methods to improve the learning experience for
individual students further would tempt to do more in-depth research about the primary
benefactors of said analytics. (Teasley, 2019)
The application of all and any analytics does come with its own questions about data
ethics. Generally, ethics in technology question the problems that technological
advancements, digital tools and methods produce. This can range from privacy to
autonomy to discrimination by data.
To draw a line from data ethics to LA, we have to consider the owners and producers of data in question. In a learning environment, this would mean institutes and students. These educational institutes can gather data about their students to facilitate the education requirements dictated by law which can be used for LA but also collect data for other varying purposes if they have been given permission to do so which also could benefit LA among other things. (Data protection law 1050/2018, 2 chapter 6 §) To satisfy the framework requirements for education qualifications and be able to offer studies institutes need to possess certain information about its students. This contains but is not limited to personal information (previous education record, social security details etc.) and depending on the extent of digitalization, the amount of student data on each individual student can rightfully cause concern among students.
Now to elaborate on the reasoning for the study, the ethics point of view; it is a
common conception that the dual model presents a divide in the train of thought in the
general population which stems all the way from second degree education level, called
the upper secondary level, where the dual model is first proposed. (Ministry of
Education and Culture Finland, 2020) You typically choose between getting a
profession fast through vocational school or entering high school (also known as
general upper secondary education, fin. lukio) to get more literate before moving
onwards to third educational level (which is the university level in Finland, National
Qualification Framework (NQF) 6 and up).
In this section, the research setup is presented. The data gathering was spread between two universities in Finland: University of Helsinki (which is the largest university in the country) and the Satakunta University of Applied Sciences. The reasoning behind this was to find out if the students in different universities have different perceptions about use of learning analytics and the ethical perspectives associated with it. To further diversify the student population, the study in University of Helsinki was conducted in a programming Massive Open Online Course (MOOC), where many students come from outside university. However, in this article we have filtered out the answers from students who did not have a status as a student in either of the selected universities when the survey was conducted. 4.1
The study was conducted as an online survey. In the university, the survey was attached to a programming MOOC. The MOOC in question is a popular programming MOOC aimed for students with multidisciplinary backgrounds and can be completed by students outside universities as well. In addition, the MOOC works as an entrance exam: the students who pass the MOOC with a high enough grade can apply to be students in the Department of Computer Science. Because of this, the MOOC has been quite popular. More than 5,000 students in total have started the current implementation. The survey was included in one of the weekly exercise rounds in the MOOC. To motivate the students in answering, one point was awarded for it.
In the university of applied sciences, the survey was conducted through basic face to face (F2F) methods. The same survey was used but the students were actively encouraged to participate in it by engaging them. Students of two separate campuses in the university of applied sciences were surveyed in their respective locations (Pori and Rauma), and motivated to participate by awarding traditional Finnish overall badges (a Finnish student custom in which students collect badges to overalls that represent your faculty) upon survey completion.
The survey was constructed with Google Forms and Microsoft Forms. Forms were selected to conduct the survey in the educational institutes as they offered adequate form functionalities and were deemed viable solutions for surveying. The survey questions were composed mostly of multichoice questions and statements based on the 5-step Likert scale and open form questions. 4.2
The participants for the survey were a group of students from both of the higher education institutes. The total number of replies received in the survey was 201 of which 56 were from the university and 145 from the university of applied sciences. 652 students answered the survey in the university programming MOOC, but only 56 of these had an official status as university students. Hence, the other answers were excluded from this study. Student numbers are displayed in Table 1.
Biggest age group was 20 to 25 year olds (58% and 46%, respectively). Second largest age group was 26 to 30 (21% and 25% respectively). Hence, a total of 79% and 71% of the students in the survey were between the ages 20 and 30. 73% of university students and 43% of university of applied sciences students answered to at least one of the open questions at the end of the survey, which can be considered as a good total percentage.
The division of students between different majors is displayed in Figures 1 and 2.
As seen from the figures, the largest groups were Math and Science and Engineering, respectively. The students in the university had more diverse backgrounds than students in the university of applied sciences. It should be noted that the major of Social and Human in university of applied sciences consists mostly of business majors. 5
As the idea was to collect data on students' ethical perceptions about learning analytics, the majority of the questions were constructed around this theme. The form of data collection was an online survey that was made to reflect the themes of the study in learning analytics and its ethics. A total of twenty multichoice questions and statements and three open questions divided into three categories were included in the final survey. The themed categories were: 1. 2. 3.
Attitudes: Students' attitudes towards the application of learning analytics. The first section also measures students' current knowledge about analytics. Expectations: What students expected learning analytics to be, and for whom are they collected.
Ethical issues: How students viewed the possibilities and problems posed by
learning analytics and associated individual data in learning environments.
This survey was then replicated into two exact copies, which were then shared with
students in both institutes early 2020. The questions and/or statements in these
categories were based on the themes found in the student-teacher workshops made in
APOA-project (Avain Parempaan Oppimiseen Ammattikorkeakouluissa) and
published by the Jyväskylä University of Applied Sciences late 2019
In this section, we present the results from the surveys conducted in both universities. First, the means and differences between the two universities for all three themes are displayed. After that, we discuss the answers to open questions.
In Table 2, the average answers for questions in Theme 1, attitudes are displayed.
As seen in the Table, there seems to be no major differences between the two universities. However, the students in the university of applied sciences seem to value learning analytics and IT systems in general higher than the university students. The university students also seem to have a tighter attitude towards institutions utilizing data in analytics. The differences in questions 2, 3 and 6 are statistically significant (two-tailed T-test, p < 0.05).
The mean values and differences for questions under Theme 2, expectations are displayed in Table 3. 7. I wish that LA would help me follow my progress in studies 8. I wish LA would help me design my studies better 9. I wish LA would help visualize my progress in studies 10. I wish teachers would use LA to follow my progress 11. I wish that the institution would use LA to progress my studies
According to results, the students' wishes towards learning analytics were quite similar in both universities. Most students hope that utilization of analytics would help them to design and visualize their studies better. Notably, the students also wish that learning analytics would provide better chances for teachers to follow their progress in studies. There are no statistically significant differences between statements under Theme 2.
The mean values and differences for questions under Theme 3 are displayed in Table 4.
Table 4. The mean values for statements in Theme 3, measuring students' perceptions towards ethical considerations of LA. 'Institution' in statements refers to the university where the data was collected. The difference is calculated similarly to previous tables.
The differences with statistical significance are bolded.
Statement 12. I get enough information about my data collected by institution 13. I can affect how my data is collected and hande in institution 14. I know what data is collected by institution 15. I accept the collection of my data by institution 16. I accept the utilization of my data by institution 4.055 3.911 4.232 3.745 17. Institution should follow laws and regulations to keep my data safe 18. My data collected by institution can be used for my own benefit 19. My data collected by institution can be used for institutions benefit 20. I know that my data is removed by institution after my graduation
Notably, the answers to the third category were somewhat different between the two universities. In fact, the difference is statistically significant in all questions under Theme 3 except for question number 16. The biggest absolute differences were in statements 19 and 20. The first one was about attitudes towards institutions using data for their own benefit (such as marketing), which the students from the university of applied sciences found more acceptable than the students from university. The university students also seemed to lack knowledge about their personal data being erased after their graduation, which may indicate that the university of applied sciences had informed the students better about the topic.
In addition to questions answered in the Likert scale of 1 to 5, the survey contained three open questions. In general, the students in the university were more verbose in their answers, but there were no huge differences in the tone or content. The first question asked the students about their opinions in using their own data to progress their studies. The students were also asked to consider both positive and negative aspects of the topic.
In general, the students found the possibilities of data utilization positive (all comments are translated from Finnish and in some cases re-worded and/or shortened by authors).
"Data collection is a good thing for course design and implementation" "I found the data collection as positive thing, since my data benefits me" "I think the data can help me study more effectively"
In a modern society where data usage is common especially younger generation seems to be more accepting of it. On the other hand, most students expressed their concerns about the safety and ethicality of data storage and handling by third parties: "I found it risky if third parties are used to handle the data" "I'm wondering if my data is used to something I don't know about" "I am very suspicious about any data collection or utilization where the participants are not clearly informed about the procedure" "The students should be able to restrict the utilization of their data" Now that GDPR is in effect the results could be affected in one way or the other about data safety; GDPR made the public more aware about data protection but at the same time it stipulated it. In the second open question, the students were asked about the additional value learning analytics could produce for them. Again, most of the students considered analytics as potentially quite useful:
Personal guidance and adaptive learning materials and courses were repeated in most of the answers. It seems that the students find this as one of the most important thing learning analytics can offer them. Analytics generally is seen as added value to an existing system and in LA the benefits are easily perceived.
As the final open question, the students were asked to consider if they find learning analytics as ethically justifiable part of future studies, where student instruction is based on data collection and analysis. While previously students gave positive feedback about LA, in the utilization of analytics, they found both positive and negative chances: "In principle, it's in everyone's interest to make learning more effective. Practical procedures may prove to be problematic" "If the learning results are improved, the development is ethically provable. The ethical problems may arise from data collection and storage." "In the future lot of the studies are likely implemented over internet. Hence, LA will be needed more." "I find LA ethically sustainable as the results will benefit students. Students should have a chance to limit the data collection and usage."
Some students mentioned that even though learning analytics has potential ethical issues, it is still ethically more acceptable than most of the data collection happening already in different systems, such as social media platforms. 7
As seen on the results, the students in general seemed to find the possibilities learning analytics offers quite positive, but were worried about whether their data is stored and utilized safely and ethically sustainable. The biggest things that students value in analytics are visualizing their progress and the possibility to get assistance when required, even without requesting for it. From the future analytics, the students seem to wait for personalized and adaptable learning environments and materials. This is perfectly understandable, as in online education (especially in the form of massive open online courses), there is often a lack of communication and personal guidance. Hence, the idea of learning environments that fit to everyone's skill levels and learning preference must sound tempting.
Students' worries about data safety and misconduct with their data are also perfectly understandable. In general, most of the students seem to agree that data collected about their learning can be used in analytics as long as the results benefit themselves. However, the students find it problematic if data is used for anything else (even inside the institutions) or delivered outside institutions. From a perspective of learning analytics researchers, we find this attitude quite reasonable: as long as learning analytics is conducted openly and in collaboration with students, they seem to have no problems with it.
Although students in both participating universities seemed to mostly agree on the ethical issues of learning analytics, there were some significant differences. The students in university seemed to be more cautious about the usage of their data. For example, while students in university of applied sciences seem to find the usage of their data to benefit the institution within acceptable boundaries, the students in university find it significantly less acceptable. Similar differences can be found with other statements and in open comments as well: the comments from university students were more verbose, and the same basic message was repeated in them: using data is perfectly fine as long as it is used to benefit students and not shared with anyone outside the institution.
There are some likely explanations for the differences between the two universities.
First, the students in university of applied sciences are likely more familiar with the
concept of learning analytics and with the procedures associated with it (such as data
safety for example), due to the ongoing APOA project
It should also be noted that in the survey, we first introduced the concept of LA to the students in order to give them an idea about it, so they would not rush into the survey with a false understanding. Even though we created the survey with unequivocality in mind, while presenting the survey in university of applied sciences we also encouraged people to use the neutral answer “I don’t know” when they did not feel like they fully understood the question as to prevent falsifying the data. No similar procedure was used in university, which may have skewed the results in some degree against or in our favor. These details and their effects could be ruled out by replicating the research setup in future on different groups in different educational institutes.
Overall, it seems that the students have strong opinions about the subject. For example, it is quite typical that openly answered questions, particularly at the end of a survey do not collect that many replies. In the survey, we had many very interesting and well thought out answers on all three categories. The fact that university students' answers were much longer could indicate more interest on the topic, but likely explanation may also be that the topic was already familiar to students in university of applied sciences. Moreover, it is possible (although cannot be confirmed) that students in the latter case answered the survey mostly with mobile devices while the students in university used laptops, which enables easier text input. From the students point of view the survey did seem to be a tad too long, which we found out when interviewing some students about it after completion. The amount of unreplied open answers supports this.
By now, with the accelerated digital leap forced on us by the 2020 pandemic, the collection of student data should have become more obvious in a modern educational environment. More and more of the lessons are held via digital tools, conferencing systems, learning management systems etc. supplemented by digital registers that keep track of the students' studies and progress. As it is at least somewhat likely that some of the newly widely adapted digital tools will become part of the permanent workflow after the pandemic as well, the need for extensive use of learning analytics has become imminent. For this reason, we should listen to students' perceptions carefully to come up with ethically sustainable analytics. LA is seen as a necessary form of analytics somewhat comparable to healthcare analytics, where the data primarily serves the individual, but also the public and common welfare. 8
In this study, we found out that students find learning analytics as a potentially promising tool for tracking and visualizing their own progress. They also see the potential of analytics to create adaptive and personalized learning paths for students with different skills, mindsets and learning preferences. However, the students are also worried about the safety of their data and the transparency of the data collection and utilization. In short, the students feel that learning analytics can and should be used, but only if the data collection and handling is done in ethically justifiable methods. Curiously, university students were stricter about this than their counterparts in the university of applied sciences were.
In future, we are planning to expand the study to contain more universities and other institutions, and to collect even more data about students' backgrounds to find out if there are factors that may explain the attitudes and expectations towards analytics. It would also be useful to find out how much the universities have formally informed the students about the data safety procedures used in the institutes, f.ex. how GDPR has been implemented and voiced in a institute. Finally, it would be interesting to combine the survey data with performance data from courses to find out if the students' success or failures in courses affect their attitudes towards analytics. Data ethics should be given more consideration in future studies since it ultimately affects in which way the public opinion, attitudes and expectations, sway and as the world of IT it will evolving and changing constantly. AnalytiikkaÄly project. (2019). Research, development and implementation of learning analytics in higher education. Referenced on 21.04.2020. Retrieved from https://analytiikkaaly.fi/in-english/ Data protection law 5.12.2018/1050 with all its changes (Finnish only). Referenced on 29.04.2020. Retrieved from https://www.finlex.fi/fi/laki/ajantasa/2018/20181050#a1050-2018
Finnish National Agency for Education. (2020). Qualification frameworks - Finnish National Agency for Education (OKM). Referenced on 25.04.2020. Retrieved from https://www.oph.fi/en/education-and-qualifications/qualifications-frameworks Ministry of Culture and Education Finland. (2019). Higher education institutes, science agencies, research institutes and other public research organizations. Referenced on 22.04.2020.
Retrieved from https://minedu.fi/en/heis-and-science-agencies