How are Learning Analytics Considering the Societal Values of Fairness, Accountability, Transparency and Human Well-being? -- A Literature Review Eyad Hakami* and Davinia Hernández-Leo Pompeu Fabra University, Barcelona, Spain eyad.hakami01@estudiant.upf.edu davinia.hernandez-leo@upf.edu Abstract. The scientific community is currently engaged in global efforts towards a movement that promotes positive human values in the ways we formulate and apply Artificial Intelligence (AI) solutions. As the use of intelligent algorithms and analytics are becoming more involved in how decisions are made in public and private life, the societal values of Fairness, Accountability and Transparency (FAT) and the multidimensional value of human Well-being are being discussed in the context of addressing potential negative and positive impacts of AI. This research paper reviews these four values and their implications in algorithms and investigates their empirical existence in the interdisciplinary field of Learning Analytics (LA). We present and highlight results of a literature review that was conducted across all the editions of the Learning Analytics & Knowledge (LAK) ACM conference proceedings. The findings provide different insights on how these societal and human values are being considered in LA research, tools, applications and ethical frameworks. Keywords: Learning Analytics, Fairness, Transparency, Accountability, Wellbeing/ Well-being 1 Introduction The interdisciplinary field of Learning Analytics (LA) borrows methods from Artificial Intelligence (AI) and goes together with several related areas of research in Educational Technology to understand and enhance learning. Certainly, Education is one domain where AI is having an increasingly relevant role and impact. According to the latest Innovating Pedagogy report [36], “AI-powered learning systems are increasingly being deployed in schools, colleges and universities, as well as in corporate training around * Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0) 122 the world”. The emergence of the LA field has emphasized this trend and raised discussion about the possible positive and negative futures that can be envisaged considering the AI potential [27]. Although AI systems can bring benefits, they also present inherent risks, such as biases, reduction of human agency due to lack of transparency, decrease of accountability, etc. Therefore, societal initiatives (e.g. policy makers) and the AI scientific community are currently engaged in global efforts towards a movement that promotes positive human values in the ways we formulate and apply AI solutions. As the use of intelligent algorithms and analytics are becoming more involved in how decisions are made in public and private life, societal values of Fairness, Accountability and Transparency (FAT) are being discussed in AI research to address potential negative and positive impacts of AI. In addition, there are demands and efforts for considering AI impacts on all aspects of human wellbeing. The IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems [71] recognizes in a recent report that prioritizing ethical and responsible AI has become a widespread goal for society, and the design of intelligent systems should directly address important issues of transparency, accountability, algorithmic bias, and value systems. This research paper reviews these four values and their implications in algorithms and investigates their presence in the field of Learning Analytics (LA). First, we introduce the main concepts this paper revolves around, which are Learning Analytics, and the four values of FAT and Wellbeing. Then we analyze and highlight results of a literature review that was conducted across all editions of the Learning Analytics & Knowledge (LAK) ACM conference proceedings. The findings provide different insights on how these societal and human values are being considered in various LA tools, applications and ethical frameworks. 2 Research context The research context of this paper is framed around a) data involvements in Education in the form of Learning Analytics that include, but are not limited to, AI methods and techniques, b) the problem of algorithmic bias as an active example of potential harmful impacts of using advanced data-driven algorithms, followed by societal concepts of fairness, accountability, and transparency, from the perspective of their relevance to preventing bias and ensuring positive AI impacts, and c) the notion of wellbeing as a multidimensional value, viewed from both perspectives of its theoretical background and the global efforts of promoting positive wellbeing impacts out of intelligent or autonomous systems (A/IS). 2.1 Data in Education As people and devices are increasingly connected online, society is generating digital data traces at an extraordinary rate [6]. The term “Big Data” is used to reflect that a quantitative shift of this magnitude is in fact a qualitative shift demanding new ways of 123 thinking, and new kinds of human and technical infrastructure [74]. Like many other sectors, Education has been affected by what commonly known as data revolution. Collecting reliable performance data for the purpose of tracking learning progress is being considered an essential feature for improved educational systems. Learning Analytics. Big and small data approaches are present in Education in the form of Learning Analytics (LA). Learning Analytics are the processes of collection, measurement, analysis and reporting of learners’ data for the purpose of understanding and optimizing learning and the environment in which it occurs [42]. By merging data techniques and analytics into learning technologies, data-driven tools and algorithms (e.g. analytics dashboards, recommender systems, intelligent tutoring systems ITS, etc.) are being designed and developed for understanding and enhancing learning. Arguably, the concerns of LA applications are driven by not only finding ways to enhance learning, but also by validating the complex processes used in this direction and evaluating their wider impacts. 2.2 Bias in Data Analytics In the case of data collection and analysis, bias is always a major threat. To be biased means to be prejudiced for or against individuals or groups in ways considered unfair. Bias in data analytics can occur because the data collected are biased, or the humans who collected them are biased. The way people collect data can have significant influence on results that they obtain by analyzing the data [51]. Whereas cognitive socially-driven bias is an example of the human bias that can affect processes of collecting and analyzing data, the matter of data selection and generalizability is a typical example of how a data set can be biased. In addition, when software and AI methods are involved in data analytics, they may reproduce different forms of bias and impact a large scale of stakeholders: “algorithmic decision procedures can reproduce existing patterns of discrimination, inherit the prejudice of prior decision makers, or simply reflect the widespread biases that persist in society” [12]. Algorithmic Bias. Algorithms are widely defined as sequences of problem-solving operations conducted based on sets of rules and instructions to lead to predictable or desirable outcomes. The term algorithm in the context of this paper refers to the advanced computational algorithms that have capabilities from AI and machine learning, allowing them to autonomously make decisions based on statistical models or decision rules [39]. Even by this meaning, the limits of the term algorithm are determined by social engagements rather than by technological or material constraints [21]. Algorithmic bias can occur when algorithms reflect the implicit values of people who are involved in training the algorithm. Ways that people may be affected by algorithmic bias include being consciously and unconsciously subjects for forms of mistreatment (e.g. discriminatory, unfairness), and making different types of decisions depending on biased algorithmic outcomes. 124 2.3 Fairness, Accountability and Transparency (FAT) As the use of algorithms and analytics are increasing and becoming more involved in multiple decision-making processes, social topics such as fairness, transparency, and accountability (FAT) are receiving more attention in research from the perspective of their relevance to preventing bias, and ensuring more ethical algorithmic practices. Regardless issues of data agency in the deployment of algorithms and analytics, new questions started to rise in the direction of shaping the ethical framework of decision- making algorithms. The ethical concerns these questions discuss go beyond the actual work of algorithms, mostly focus on the design and development phases of training an algorithm: How can fair algorithms be designed and developed? [65], how can we develop algorithms that are more transparent and accountable? [39], and how can we produce machine-learning algorithms that autonomously avoid discriminating against users and automatically provide transparency? [14]. Algorithmic Fairness. Oxford dictionary defines fairness as the “impartial and just treatment or behavior without favoritism or discrimination”. As bias, by some means, is the lack of fairness and the excess of discriminatory, fairness can be understood as the lack of bias. Algorithmic fairness typically means that algorithmic decisions should not create discriminatory scenarios, but it is still a complicated topic because the definition of fairness is largely contextual and subjective [77]. With that in mind, some scholars and activists have been presenting multitude of technical definitions and solutions to substantially prevent algorithmic bias and maximize fairness and transparency. Algorithmic Transparency. Transparency is generally considered a means to see the truth and motives behind actions [4]. In data-driven models and algorithms, transparency is understood as openness and communication of both the data being analyzed and the mechanisms underlying the models [40]. Some researchers considered algorithmic transparency as a way to prevent discrimination; assuming that when people understand how system works, they are more likely to use the system properly and trust the designers and developers [39]. Another applicable perspective of transparency in algorithms is about its ability to provide reasons for an autonomous decision (e.g. demonstrating reasons behind selections made by a recommender system). This view proposes that transparency in algorithms follows the sequence of logic: observation produces insights that create the knowledge required to govern and hold systems accountable [3]. Yet, full transparency can be significantly harmful. Therefore, transparency is just one approach toward the ethics and accountability of algorithms [20]. Algorithmic Accountability. Accountability refers to processes by which actors provide reasons to stakeholders for their actions and the actions of their organizations [63]. While people are responsible for reasoning their actions, algorithmic accountability concerns are driven by drawing the responsibility circle of algorithmic 125 decisions. A critical question to define algorithmic accountability is: who is responsible for actions and decisions of an algorithm created by humans and able to make decisions without explicit human intervention? One answer on this suggests that accountability of algorithmic decisions must be derivable from the methods and data used by the algorithm in order to generate the decision [16]. Thus, accountability in algorithms and their application begins with the designers and developers of the system that relies on them [15]. Subsequently, questions that are more specific might be asked in order to hold algorithms accountable: What are the consequences of using an algorithm for individuals and societies? How influential are these consequences and how many people may be affected by? To what extent they are aware of the algorithmic mechanism that decides for them and drives their decisions and opportunities? What are the possibilities for algorithmic bias and discrimination to be occurring and leading to negative impact on the public? How this can be avoided from the early phases of designing and developing an algorithm? How can it be fixed if it happens during the implementation of the algorithm? What are the strategies of optimization and the techniques of intervention? 2.4 Well-being For the purposes of aligning ethical considerations to intelligent systems’ design, the term “well-being” refers to an evaluation of the general quality of life of an individual, and encompasses the full spectrum of personal, social, and environmental factors that enhance human life and on which human life depend [71]. Therefore, human wellbeing should not be perceived as a value of one dimension, and evaluations of wellbeing and the impacts of A/IS on wellbeing domains must be done with a consideration that human wellbeing is inseparably linked to the wellbeing of society, economies, and ecosystems. Measuring Well-being. Wellbeing can be reliably measured [48 and 71]. Measuring wellbeing has become a target for several national and international institutions for the purpose of better understanding whether, where and how peoples’ life is getting better (e.g. European Social Survey [24], OECD Better Life Index [48]). Subjective and objective indicators are being used by such institutions to measure wellbeing of individuals and societies. While subjective indicators are used to collect data about how people perceive the state of their wellbeing, objective indicators are used to gather observable data to measure wellbeing (e.g. incomes, graduation rates, etc.). A question that has been recently asked is: what are the potential impacts, positive and negatives, on the various wellbeing dimensions that include but are not limited to: feelings, community, culture, education, economy, environment, human settlement, health, government, psychological wellbeing, satisfaction with life and work. [34]. Value Systems. Whatever their level of autonomy and their capacity to learn and make decisions, intelligent systems are required to incorporate societal and moral values into their technological developments at all phases of creating the system: analysis, design, 126 construction, implementation and evaluation [17]. When creators of AI systems are not aware that indicators of well-being, including traditional metrics and all other personal and social indicators that improve quality of life, can provide guidance for their work, they also miss innovation that can boost well-being and societal value. A representative illustration of this concept is autonomous vehicles. The discussion is commonly centered in how they may save lives, but less is argued about their potential to reduce greenhouse gas emissions or to increase work-life balance or the quality of time. In education, for example, technology-enhanced learning implies that the presence of information and communication technologies in education has to be in a framework distributed for educational value creation at all levels. If we only use metrics of learning performance when designing and developing educational tools and systems, we may lose other relevant well-being facets such as effects in socio-emotional aspects, self- regulation, workload of teachers and learners, the inclusion dimension, etc. 3 LAK Literature Review In this literature review, we investigated empirical existence of the four values of FAT and Wellbeing in LA research. The search was conducted across all the ten editions of Learning Analytics & Knowledge (LAK) conference proceedings from 2011 to 2020. 3.1 Method This review is limited to LAK conference proceedings, as they, to a certain extent, reflect the work and results related to LA community. The search aimed to answer the following questions: ● To what extent are the concepts of FAT and Well-being existent in LAK papers? ● How do the LAK papers present and face these concepts? A conventional search on the full texts of all LAK companion proceedings (from LAK11 to LAK20) was conducted by using the following keywords: fairness, accountab*, transparen*, and wellbeing/ well-being. The textual search covered every paper published in LAK proceedings according to tables of contents in ACM digital library. Since these conceptual keywords are relatively new to the field of LA, everything related to the topic was read, and judgments were made based on textual analysis aimed to identifying contexts of each keyword. 3.2 Quantitative Results A total of 49 papers include one or more of the keywords used in the search. As shown in Table 1, there is a modest increase in the number of papers that tackle the four concepts across the years (from 2-5 in LAK11-15 to 7 in LAK16-20). The table shows 127 the detail about the evolution across years in the use of each concept by LAK papers. In total, over 75% of the papers (37 out of 49) mention the concept of “transparency”. 22% and 18% of the papers include the terms “accountability” and “fairness”, respectively. And only 7 papers (14%) mention the term “well-being”. Table 1: Number of papers per each keyword across the ten LAK proceedings. Some papers include more than one keyword, so the horizontal total represents papers per year/proceedings 3.3 FAT in LA Ethical Frameworks In their endeavor to map ethical and legal basis informing LA practices, [54] cited the notions of transparency, accountability and fairness among other approaches aiming to Well- Transparency Accountability Fairness Total being LAK11 2 - - - 2 LAK12 5 - - - 5 LAK13 3 - - - 3 LAK14 - 2 - - 2 LAK15 2 1 1 - 2 LAK16 5 1 - 3 7 LAK17 7 2 2 - 7 LAK18 4 3 2 1 7 LAK19 5 1 3 1 7 LAK20 4 1 1 2 7 LAK All 37 11 9 7 49 solve complex data-centered ethical problems. In the range of these ethical approaches, legal frameworks attempt to make such complexities more palatable by reducing them to a series of principles. According to [33], the principles of fairness, accountability and transparency in existing international privacy frameworks can influence the whole design cycle of LA systems. A review of eight existing LA policies for higher education was presented by [72] and discussed how these policies had tried to address notable challenges in the adoption of LA. The results of this review showed that all the eight policies had ensured that processes on student (and staff) data must be transparent. More insights on how data can be handled transparently were extracted from those eight policies and were interpreted by [72] as follows: 1) the methods used to collect data have to be disclosed to the subjects of the data collection; 2) the information about how data will be stored needs to be provided; 3) Users need to be notified about where their data has travelled in any integration process between multiple entities and informed about any changes made to the analytics process. In the direction of establishing an ethical literacy for LA, [70] borrowed multiple frameworks from the field of technical communication to guide discussion on the ethics of LA “artifacts”: data visualization, interactive dashboards, and LA methodology (gather, predict, act, measure, and refine). “When guided by such frameworks, an ethical literacy for LA will answer the question: Who generates these artifacts, how, 128 and for what purpose, and are these artifacts produced and presented ethically?” [70]. Lack of accountability is a potential consequence of inaccurate or incomplete data that may be used in LA models. On that, the ethical literacy proposed by [70] described the need for understanding limitations of data in LA models as a limitation of accountability. FAT in a Personal Code of Ethics. A draft personal code of ethics for LA practitioners was developed by [38] to consider whether such a code might determine the ethical responsibilities for individuals within the field of LA. This code considered the principles of fairness, accountability, and transparency as following: Fairness. An ethical code of fairness for individuals involved in LA practices could be: “I will recognize that fairness and justice entitle all persons access to, and benefit from, the contributions of education and to equal quality in the processes, procedures and services being conducted through the use of data”. Accountability. Although this personal code of ethics included parts that may define personal accountability, the authors concluded that there is currently no way in which individuals can be held accountable to any code. Given the scale and complexity of institutional LA systems, “it may be impossible to trace an individual’s actions without substantial, possibly unrealistically sophisticated, accounting systems being implemented”. Considering the need to distinguish between what is mandatory (professional obligation) and what is aspirational (moral guide) when applying personal ethical codes, [38] offered different contexts to explain to what degree can individuals be held accountable in LA practices. An example on what might be considered a mandatory code is: “I have a responsibility to act for the benefit of learners and to avoid any action that would harm the learner and their educational opportunity”. The following quote could be considered an aspirational personal code for individual accountability in LA: “I will ensure that I understand analytic processes (algorithms, statistics) that I employ. I will strive to promote accuracy, honesty and truthfulness in the science, teaching and practice of learning analytics” [38]. Transparency. The code also encouraged LA practitioners for more transparency: “I will ensure that data practices are transparent to those whose data I work with” [38]. Yet, being transparent regarding LA practices seems not to be an individual call. 3.4 Institutional Transparency Educational institutions may need to set policies that reveal information about what data is collected, how they are used, etc., in ways that are technically and intellectually accessible to all relevant parties [31]. As [22] agreed, providers of analytical services have to demonstrate a transparent treatment for personal data. To make this possible, [56] suggested that addressing the practical implementations of being transparent regarding the collection and use of personal data could force companies and institutions 129 to address practical policies and clarify their thinking. In a later work, the authors provided more insights on how higher education institutions should strive to be transparent. They suggested that institutions should allow students to: (1) know what data is collected, by whom, for what purposes, who will have access to this data downstream and how data might be combined with other datasets (and for what purposes); (2) be aware of the potential benefits that they may access in exchange for their data; (3) access to, and feedback on, the analyses that result from collection of their data, as this can support LA in its goal of not only providing institutions with a clearer understanding of how students learn, but also what students find useful [69]. 3.5 Transparency and Data Transparency was considered a problematic affair since the first efforts in both research and innovation within the LA field. While the issue of privacy was an alarm trigger to the ethics of LA, issues of transparency and openness about tracking learners’ data have been a corner stone in such discussions. The main reason for this early attention to transparency is the nature of analytics as it derives from data. “It is not surprising that many outstanding concerns in LA center on data” [66], and it is often said that lack of transparency about data collection can cause unease among data subjects [22]. Therefor, “it should always be clear to a person that she is being tracked” [23]. 3.6 Implications of Transparency in LA Transparency for Understanding, Sense-making and Reflection. Investigations on the appropriate use of data in online education asked whether the transformation of data sets into measures and indicators is transparent and sensible [46]. Various LA applications (dashboards, recommenders, predictors) have adopted the concept of transparency as a method to support users’ understanding and sense-making. According to [43], advances in visualization tools provide a great opportunity for researchers to develop visualizations that can improve transparency and therefore increase awareness and support reflection. An evaluation by [61] was conducted on a dashboard they had created to “empower students to reflect on their own activity, and that of their peers, in open learning environments”. [60]. Open Learner Models (OLMs) were regarded by [37] as powerful means to enhance transparency, increase understandability and support reflection. In a similar vein, [76] described how the use of analytics can be framed in a pedagogical model, where students viewed the analytics as a guideline for sense- making that can empower them to regulate their learning process. For LA prediction models, it was indicated that transparency related to the reasons why and how certain predictions are made is essential in order for teachers and students to understand how best to act upon the predictions [50]. Also, [26] showed how an LA recommendation could make more sense when the rationale behind it is transparent for the learner. According to a hypothesis by [47], “a more complex (i.e. black-box) model performs better, while a transparent model, despite given less accurate results, may be more 130 valuable thanks to a higher degree of explainability”. Recently, a study was conducted by [1] and aimed to investigate the impact of complementing Educational Recommender Systems (ERSs) with transparent and understandable OLMs that provide justification for their recommendations. The survey results indicated that complementing an ERS with an OLM has an overall positive impact on the students’ engagement and enhances their acceptance of the system [1]. Additional work is needed to generalize such findings by comparing the effect between a transparent recommendation and a traditional black-box recommendation on students’ motivation to follow the recommendation, and eventually, accept the tool [5, 49]. Transparency for Acceptance and Adoption. It has been noticeable by the LA research community that transparency is one effective way toward more acceptance for LA practices among users and stakeholders. An early heed of that was stated by [66] in his effort toward envisioning LA as a research and practice domain: “A proactive stance of transparency and recognition of potential learner and educator unease of analytics may be helpful in preventing backlash”. This vision was supported by [10] who suggested that transparency can effectively benefit LA in overcoming challenges related to social acceptability. In addition, [73] found in a study aimed to understand LA privacy issues through students’ own perception that transparency and communication are key levers for LA adoption. As also argued by [13], transparent modelling approaches such as decision trees allow teachers and learners to scrutinize analytics suggestions and reflect on them, which can lead to more agency of teachers and learners, therefore can lead to easier adoption. Transparency to Build Trust. One of the earliest attempts to put transparency in LA innovation was by integrating a reputation system to a participatory learning platform for the goal of facilitating trust between users, by making actions and feedback transparent and allowing users to track their own learning and that of others [9]. Also, [41] found that transparency regarding what data is used, who data is shared with, and how algorithmic design choices are determined represent essential components for building trustworthy educational predictive models. Another proposition by [64] goes in line with discussions on the trustworthiness of AI, stating that providing educators with a level of control on an LA tool can ensure that the models are transparent and do not act as a black box for human interpretation. Transparency and the Option to Opt-out. In several papers, Prinsloo and Slade presented the option to opt-out of the collection of certain types of data as a potential way to increase transparency [55, 56, 67 and 68]. The review of eight LA policies by [72] also indicated that multiple LA policies had taken such an option in consideration. Examples on these considerations, as summarized in this review included that users should be given the option to opt out of the data collection processes without any consequences, and that LA mechanisms must allow specific data to be withdrawn at any time. However, some other policies in this review stated that such an option is not 131 available, because of the impossibility of delivering courses and supporting students without having their data stored in information systems [72]. Transparency to Support LA Co-design. Incorporating different resources of LA stakeholders and users (e.g. researchers, subject experts, students and teachers) into the design of analytical tools can improve usability and usefulness of these systems [18]. According to this argument, challenges of power-balance in such a ‘co-creation strategy’ for LA can be reduced through a clear distribution of roles and a high level of transparency among the different co-designers. On a practical level, [59] provided a student-centered design that applied deferent methods to engage students in the design, development and evaluation of a student facing LA dashboard. Transparency was underlined as a core contribution of this design, which “emphasis on fully utilizing the user-centered process, not just for initial requirements gathering, so that the design and development process of Student Facing LA systems is fully transparent, from the initial analysis stage all the way to final evaluation” . Transparent LA Tools. Deferent perceptions have been proposed to describe when an LA tool is considered transparent. According to [62], an analytical tool supports transparency if users know what data about them is collected and who can see information about them. A stricter view considered an LA tool transparent when the users understand the whole process behind analytical outcomes [7]. Transparent LA Research. A research method was presented by [29] as an approach to conducting LA research. An important aspect of this method is the transparency on how a research work might contribute to a ‘fully complete LA’. The method stated that researchers should “articulate the extent to which their work is constituent and contributes to an existing or future LA agenda, and/or it is aggregate and incorporates prior LA constituent research, in order to deliver a more complete LA” [29]. 3.7 Institutional Accountability Institutions and policy makers have to ask, “How can we use algorithmic decision- making in higher education to ensure, on the one hand, caring, appropriate, affordable and effective learning experiences, and on the other, ensure that we do so in a transparent, accountable and ethical way?” [58]. A paper by [33] showed how LA process requirements can be derived from an existing privacy framework (i.e. GDPR) by transforming legal requirements into systems requirements. This work provided a list of design requirements for LA including that “the institutions must be able to demonstrate that they have systems in place (policies and procedures) that uphold the protection of personal information and minimize risk of breaches”. [33]. 3.8 Algorithmic Accountability Ways in which analytic devices become effective factors in learning has led to demands for greater algorithmic accountability, to ensure the pedagogic goals of analytic devices 132 are transparent across all stakeholders [35]. As researchers should demand a rigorous level of accountability from LA devices, educators and students should also be encouraged to demand accountability to whatever level of detail they require [30]. LA devices shape or are shaped by learning contexts; and to make them eligible for learners and teachers they require careful analysis on the theory behind any given learning-target [35]. Thus, the implications of LA are not only critical for human inference and decision making, but also for algorithmic accountability [2]. 3.9 Accountable Learning The findings of a study by [32] showed that when the design of interactive features and analytics focus on contextual knowledge, it could foster learning of the conceptual knowledge that courses are typically accountable for. According to [44], “learning analytics has the potential to shape the curriculum, through enabling new kinds of learning practices that favor efficient and accountable ways of being over disciplinary knowledge-building or knower-building”. For example, self-assessment can work as a tool to make students accountable for their learning [53]. 3.10 Fair LA Outcomes Fair Measurement. As LA often aims to measure learning, [45] discussed issues related to the fairness and validity of these measures. In her work toward establishing methodological foundations of measuring learning in LA, she stated that the different demographical and cultural backgrounds of participants can lead to biased responses to indicators used to measure learning. “This means that the measures may be confounded, causing unfairness for one group or another and certainly confusing any interpretations about what is being measured” [45]. Fair Instruction. Inaccurate data models about students can affect not only the learning measurement but the learning itself too. In the context of LA algorithms used to inform intelligent tutoring systems, [19] assumed that a fair outcome is when students from different demographical backgrounds reach the same level of knowledge after receiving instruction; no matter how long it took them to reach this level. On that, they proposed that adaptive educational algorithms, such as knowledge tracing, can contribute to preventing inequities between different groups of students by allowing them to go through the curricula in their own pace. However, such adaptive educational algorithms can still be unfair (e.g. favoring fast learners over slow learners) when they rely on inaccurate models of student learning [19]. Fair Prediction. Considering that predictive modelling has been one of the core research areas in the field of LA, and with such models are deployed in a variety of educational contexts, [28] presented a method for evaluating the fairness in predictive student models through “slicing analysis”, an approach in which model performance is evaluated across different categories of the data. Although they argued that most of the prior work to define and measure predictive fairness are still insufficient for LA 133 research, the researchers indicated that LA have to satisfy the existing legal concepts of fairness and should aspire even higher standers of fairness in the educational systems. While slicing analysis as an exploratory methodology can be used only to measures predictive fairness and not to correct it, they argued that measurement is a necessary condition for correcting any detected unfairness [28]. In this context, a point of view by [75] described LA dashboards as tools that offer a great promise to address bias- related challenges in prediction models, “as by visualizing the data used by predictive models end-users can potentially be made aware of underlying biases”. 3.11 LA to Support Well-being Educational institutions have legal and moral obligations to demonstrate care for the wellbeing and growth of students, leading them to success in their education [22 and 57]. The support of student well-being was mentioned among the purposes that have encouraged students, in a study by [73], to welcome the university collecting and using of their data. In another study by [25] aimed to investigate perceptions of students and instructors of the potential benefits and risks of using LA, instructors also considered improving the overall learning experience and well-being of their students among the most important uses of LA. It is in the interests of education providers to devote LA for supporting students in developing social skills as well as domain knowledge [52]. Examples for such a potential include a paper by [11] aimed at exploring the potential of LA for improving accessibility of e-learning and supporting disabled learners. This work provided a comparative analysis of completion rates of disabled and non-disabled students in online courses and outlined how LA can identify accessibility challenges and disabled students’ needs [11]. 3.12 Value-sensitive LA Design A relevant paper by [8] introduced two cases of applying the Value Sensitive Design (a methodology from the field of Human–Computer Interaction) to support ethical considerations and system integrity in LA design. Both cases demonstrated that Value Sensitive Design could be an applicable approach for balancing a wide range of ethical and human values in the design and development of LA. Through a conceptual investigation of an LA tool developed to visualize online discussions in a learning platform, the researchers found that the following values supported by the LA tool can be in tension with other values: autonomy, utility, ease of information seeking, student success, accountability, engagement, usability, privacy, social wellbeing (in the sense of belonging and social inclusion), cognitive overload, pedagogical decisions, freedom from bias, fairness, self-image, and sense of community [8]. 3.13 Summary of Qualitative Results 134 Table 2: Summary of the qualitative results from LAK literature review Topics LAK Papers (As ordered in section 3) (As numbered in the References) FAT in LA Ethical Frameworks [33], [54], [70], [72] FAT in a Personal Code of Ethics [38] Institutional Transparency [22], [31], [56], [69] Transparency and Data [22], [23], [66] Implications of Transparency in LA: Transparency for Understanding, Sense- [1], [5], [26], [37], [43], [46], [47], making, and Reflection [49], [50], [61], [76] Transparency for Acceptance and Adoption [10], [13], [66], [73] Transparency to Build Trust [9], [41], [64] Transparency and the Option to Opt-out [55], [56], [72] Transparency to Support LA Co-Design [18], [59] Transparent LA Tools [7], [62] Transparent LA Research [29] Institutional Accountability [33] Algorithmic Accountability [2], [30], [35] Accountable Learning [32], [44], [53] Fair LA Outcomes: Fair Measurement [45] Fair Instruction [19] Fair Prediction [28], [75] LA to Support Well-being [11], [22], [25], [52], [73] Value-sensitive LA Design [8] 4 Conclusions The global efforts toward positive impacts of AI-powered systems on humans’ well- being continue to establish societal guidelines for such systems to remain human- centric, serving humanity’s values and ethical principles. Although the LA community is increasingly concerned about ethics, the societal values framing the notion of Responsible AI have been approached only to a limited extent and are scattered across LA research. Most cases focus on transparency. Yet, truly research around positive impacts of LA should be addressed from a holistic perspective that goes beyond 135 transparency and considers accountability and ways by which LA systems contribute to diverse dimensions of human well-being in and beyond the educational scenarios. To do so, there is a need for addressing metrics and techniques to help educational technology stakeholders in safeguarding human values and well-being when they design, develop, implement and evaluate LA tools and solutions. Acknowledgment. This work has been partially funded by the EU Regional Development Fund and the National Research Agency of the Spanish Ministry of Science and Innovation under project grants TIN2017-85179-C3-3-R, RED2018- 102725-T. D. Hernández-Leo acknowledges the support by ICREA under the ICREA Academia program. E. Hakami acknowledges the grant by Jazan University, Saudi Arabia. References 1. Abdi, S., Khosravi, H., Sadiq, S. and Gasevic. D.: Complementing Educational Recommender Systems with Open Learner Models. In Proceedings of the 10th International Conference on Learning Analytics and Knowledge (LAK ’20), pp. 360- 365. ACM, New York, NY, USA (2020). https://doi.org/10.1145/3375462.3375520 2. Alhadad, S., Thompson, K., Knight, S., Lewis, M. and Lodge, J.: Analytics-enabled teaching as design: reconceptualisation and call for research. In: Proceedings of the 8th International Conference on Learning Analytics and Knowledge (LAK '18), pp. 427- 435. ACM, New York, NY, USA (2018). DOI: https://doi- org.sare.upf.edu/10.1145/3170358.3170390 3. Ananny, M. and Crawford, K.: Seeing without knowing: Limitations of the transparency ideal and its application to algorithmic accountability. Journal of New Media Soc 20(3), 973–989. (2018) 4. Balkin, J M.: How mass media simulate political transparency, Cultural Values 3 (4). 393-413. (1999). DOI: 10.1080/14797589909367175 5. Bodily, R. and Verbert, K.: Trends and issues in student-facing learning analytics reporting systems research. In: Proceedings of the Seventh International Learning Analytics & Knowledge Conference (LAK '17), pp. 309-318. ACM, New York, NY, USA (2017). DOI: https://doi.org/10.1145/3027385.3027403 6. Bollier, D.: The Promise and Peril of Big Data. The Aspen Institute, CO, USA (2010). https://assets.aspeninstitute.org/content/uploads/files/content/docs/pubs/The_Promise _and_Peril_of_Big_Data.pdf 7. Buckingham Shum, S., Sándor, A., Goldsmith, R., Wang, X., Bass, R. and McWilliams, M.: Reflecting on reflective writing analytics: assessment challenges and iterative evaluation of a prototype tool. In: Proceedings of the Sixth International Conference on Learning Analytics & Knowledge (LAK '16), pp. 213-222. ACM, New York, NY, USA (2016). DOI: https://doi.org/10.1145/2883851.2883955 8. Chen, B. and Zhu, H.: Towards Value-Sensitive Learning Analytics Design. In: Proceedings of the 9th International Conference on Learning Analytics & Knowledge (LAK19), pp. 343–352. ACM, New York, NY, USA (2019). DOI: https://doi.org/10.1145/3303772.3303798 9. Clow, D. and Makriyannis, E.: iSpot analysed: participatory learning and reputation. In: Proceedings of the 1st International Conference on Learning Analytics and 136 Knowledge (LAK '11), pp. 34-43. ACM, New York, NY, USA (2011). DOI= http://dx.doi.org/10.1145/2090116.2090121 10. Clow, D.: The learning analytics cycle: closing the loop effectively. In: Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (LAK ’12), pp. 134–138. ACM, New York, NY, USA (2012). DOI: https://doi.org/10.1145/2330601.2330636 11. Cooper, M., Ferguson, R. and Wolff, A.: What can analytics contribute to accessibility in e-learning systems and to disabled students' learning?. In: Proceedings of the Sixth International Conference on Learning Analytics & Knowledge (LAK '16), pp. 99-103. ACM, New York, NY, USA (2016). DOI: https://doi.org/10.1145/2883851.2883946 12. Crawford, K. and Schultz, J.: Big Data and Due Process: Toward a Framework to Redress Predictive Privacy Harms. Boston College Law Review 55 (93). 13-64. NYU Law and Economics Research Paper No. 13-36. (2014). Available at SSRN: https://ssrn.com/abstract=2325784 13. Cukurova, M., Zhou, Q., Spikol, D. and Landolfi, L.: Modelling Collaborative Problem-solving Competence with Transparent Learning Analytics: Is Video Data Enough?. In: Proceedings of the 10th International Conference on Learning Analytics and Knowledge (LAK ’20), pp. 270-275. ACM, New York, NY, USA (2020). https://doi.org/10.1145/3375462.3375484 14. Datta, A., Tschantz M & Datta, A.: Automated Experiments on Ad Privacy Settings: A Tale of Opacity, Choice, and Discrimination. (2014). Available at: https://arxiv.org/abs/1408.6491 15. Diakopoulos, N. and Koliska, M.: Algorithmic Transparency in the News Media. Journal of Digital Journalism. (2017). DOI: 10.1080/21670811.2016.1208053 16. Dignum, V.: Responsible autonomy. In: Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI-17). Melbourne, Australia (2017). https://doi.org/10.24963/ijcai.2017/655 17. Dignum, V.:. Ethics in artificial intelligence: introduction to the special issue. Journal of Ethics and Information Technology 20(1). (2018). https://doi.org/10.1007/s10676- 018-9450-z 18. Dollinger, M. and Lodge, J M.: Co-creation strategies for learning analytics. In: Proceedings of the 8th International Conference on Learning Analytics and Knowledge (LAK '18), pp. 97-101. ACM, New York, NY, USA: (2018). DOI: https://doi.org/10.1145/3170358.3170372 19. Doroudi, S. and Brunskill, E.: Fairer but Not Fair Enough On the Equitability of Knowledge Tracing. In: Proceedings of the 9th International Conference on Learning Analytics & Knowledge (LAK19), pp. 335-339. ACM, New York, NY, USA (2019). DOI: https://doiorg.sare.upf.edu/10.1145/3303772.3303838 20. Dörr, K N. and Hollnbuchner, K.: Ethical Challenges of Algorithmic Journalism. Journal of Digital Journalism 5(4), 404-419 (2017). DOI: 10.1080/21670811.2016.1167612 21. Dourish, P.: Algorithms and their others: Algorithmic culture in context. Journal of Big data & society. (2016). https://doi.org/10.1177/2053951716665128 22. Drachsler, H. and Greller, W.: Privacy and analytics: it's a DELICATE issue a checklist for trusted learning analytics. In: Proceedings of the Sixth International Conference on Learning Analytics & Knowledge (LAK '16), pp. 89-98. ACM, New York, NY, USA (2016). DOI: https://doi.org/10.1145/2883851.2883893 23. Duval, E.: Attention please!: learning analytics for visualization and recommendation. In: Proceedings of the 1st International Conference on Learning Analytics and Knowledge (LAK '11), pp. 9-17. ACM, New York, NY, USA (2011). DOI= http://dx.doi.org/10.1145/2090116.2090118 137 24. European Social Survey.: Europeans’ Personal and Social Well-being. (2015). https://www.europeansocialsurvey.org/docs/findings/ESS6_toplines_issue_5_persona l_and_social_wellbeing.pdf 25. Falcão, T., Mello, R., Rodrigues, R., Diniz, J., Tsai, Y. and Gašević, D.: Perceptions and expectations about learning analytics from a brazilian higher education institution. In Proceedings of the Tenth International Conference on Learning Analytics & Knowledge (LAK ’20), pp. 240–249. ACM, New York, NY, USA (2020). DOI: https://doi.org/10.1145/3375462.3375478 26. Ferguson, R. and Buckingham Shum, S.: Social learning analytics: five approaches. In: Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (LAK '12), pp. 23-33. ACM, New York, NY, USA (2012). DOI=10.1145/2330601.2330616 http://doi.acm.org/10.1145/2330601.2330616 27. Ferguson, R., Brasher, A., Clow, D., Griffiths, D. and Drachsler, H.: Learning Analytics: Visions of the Future. In: 6th International Learning Analytics and Knowledge (LAK) Conference. Edinburgh, Scotland. (2016) 28. Gardner, J., Brooks, C. and Baker, R.: Evaluating the Fairness of Predictive Student Models Through Slicing Analysis. In: Proceedings of the 9th International Conference on Learning Analytics & Knowledge (LAK19), pp. 225-234. ACM, New York, NY, USA (2019). DOI: https://doi-org.sare.upf.edu/10.1145/3303772.3303791 29. Gibson, A. and Lang, C.: The pragmatic maxim as learning analytics research method. In: Proceedings of the 8th International Conference on Learning Analytics and Knowledge (LAK '18), pp. 461-465. ACM, New York, NY, USA (2018). DOI: https://doi.org/10.1145/3170358.3170384 30. Gibson, A., Aitken, A., Sándor, A., Buckingham Shum, S., Tsingos-Lucas, C. and Knight, S.: Reflective writing analytics for actionable feedback. In: Proceedings of the Seventh International Learning Analytics & Knowledge Conference (LAK '17), pp. 153-162. ACM, New York, NY, USA (2017). DOI: https://doi.org/10.1145/3027385.3027436 31. Haythornthwaite, C.: An information policy perspective on learning analytics. In Proceedings of the Seventh International Learning Analytics & Knowledge Conference (LAK '17), pp. 253-256. New York, NY, USA (2017). DOI: https://doi.org/10.1145/3027385.3027389 32. Hickey, D., Kelley, T. and Shen, X.: Small to big before massive: scaling up participatory learning analytics. In: Proceedings of the Fourth International Conference on Learning Analytics And Knowledge (LAK '14), pp. 93-97. ACM, New York, NY, USA (2014). DOI= http://dx.doi.org/10.1145/2567574.2567626 33. Hoel, T., Griffiths, D. and Chen, W.: The influence of data protection and privacy frameworks on the design of learning analytics systems. In: Proceedings of the Seventh International Learning Analytics & Knowledge Conference (LAK '17), pp. 243-252. ACM, New York, NY, USA (2017). DOI: https://doi.org/10.1145/3027385.3027414 34. IEEE P7010 Draft Recommended Practice for Assessing the Impact of Autonomous and Intelligent Systems on Human Well-being. In IEEE P7010/D4, pp.1-103. (2020). http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8960682&isnumber=8960 681 35. Knight, S., Anderson, T. and Tall, K.: Dear learner: participatory visualisation of learning data for sensemaking. In: Proceedings of the Seventh International Learning Analytics & Knowledge Conference (LAK '17), pp. 532-533. ACM, New York, NY, USA (2017). DOI: https://doi.org/10.1145/3027385.3029443 36. Kukulska-Hulme, A., Beirne, E., Conole, G., Costello, E., Coughlan, T., Ferguson, R., FitzGerald, E., Gaved, M., Herodotou, C., Holmes, W., Mac Lochlainn, C., Nic Giolla Mhichíl, M., Rienties, B., Sargent, J., Scanlon, E., Sharples, M. and Whitelock, D.: 138 Innovating Pedagogy 2020: Open University Innovation Report 8. Milton Keynes: The Open University. (2020). https://iet.open.ac.uk/file/innovating-pedagogy-2020.pdf 37. Kump, B., Seifert, C., Beham, G., Lindstaedt, S N. and Ley, T.: Seeing what the system thinks you know: visualizing evidence in an open learner model. In: Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (LAK ’12), pp. 153–157. ACM, New York, NY, USA: (2012). DOI: https://doi.org/10.1145/2330601.2330640 38. Lang, C., Macfadyen, L P., Slade, S., Prinsloo, P. and Sclater, N.: The complexities of developing a personal code of ethics for learning analytics practitioners: implications for institutions and the field. In: Proceedings of the 8th International Conference on Learning Analytics and Knowledge (LAK '18), pp. 436-440. New York, NY, USA (2018). DOI: https://doi-org.sare.upf.edu/10.1145/3170358.3170396 39. Lee, M K.: Understanding perception of algorithmic decisions : Fairness, trust, and emotion in response to algorithmic management. Journal of Big data & society, 1–16. (2018). https://doi.org/10.1177/2053951718756684 40. Lepri, B., Oliver, N., Letouzé, E., Pentland, A. and Vinck, P.: Fair, Transparent, and Accountable Algorithmic Decision-making Processes: The Premise, the Proposed Solutions, and the Open Challenges. Journal of Philosophy & Technology 31(4), 611- 627. (2018). 41. Li, W., Brooks, C. and Schaub, F.: The Impact of Student Opt-Out on Educational Predictive Models. In: Proceedings of the 9th International Conference on Learning Analytics & Knowledge (LAK19), pp. 411-420. ACM, New York, NY, USA (2019). DOI: https://doi-org.sare.upf.edu/10.1145/3303772.3303809 42. Long, P. and Siemens, G.: Penetrating the Fog: Analytics in Learning and Education. Educause Review 46(5), 30–32. (2011). 43. McAuley, J., O’Connor, A. and Lewis, D.: Exploring reflection in online communities. In: Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (LAK ’12), pp. 102–110. ACM, New York, NY, USA (2012). DOI: https://doi.org/10.1145/2330601.2330630 44. McPherson, J., Tong, H L., Fatt, S J. and Liu, D.: Student perspectives on data provision and use: starting to unpack disciplinary differences. In: Proceedings of the Sixth International Conference on Learning Analytics & Knowledge (LAK '16), pp. 158-167. ACM, New York, NY, USA (2016). DOI: https://doi.org/10.1145/2883851.2883945 45. Milligan, S K.: Methodological foundations for the measurement of learning in learning analytics. In: Proceedings of the 8th International Conference on Learning Analytics and Knowledge (LAK '18), pp. 466-470. ACM, New York, NY, USA (2018). DOI: https://doi-org.sare.upf.edu/10.1145/3170358.3170391 46. Milligan, S., He, J., Bailey, J., Zhang, R. and I. P Rubinstein, B.: Validity: a framework for cross-disciplinary collaboration in mining indicators of learning from MOOC forums. In: Proceedings of the Sixth International Conference on Learning Analytics & Knowledge (LAK '16), pp. 546-547. ACM, New York, NY, USA (2016). DOI: https://doi.org/10.1145/2883851.2883956 47. Niemeijer, K., Feskens, R., Krempl, G., Koops, J. and Brinkhuis, M: Constructing and Predicting School Advice for Academic Achievement: A Comparison of Item Response Theory and Machine Learning Techniques. In Proceedings of the 10th International Conference on Learning Analytics and Knowledge (LAK ’20), pp. 462-471. ACM, New York, NY, USA (2020). https://doi.org/10.1145/3375462.3375486 48. OECD. How's Life? 2017: Measuring Well-being. OECD Publishing, Paris (2017). https://doi.org/10.1787/how_life-2017-en. 49. Papamitsiou, Z. and Economides, AA.: Can't get more satisfaction?: game-theoretic group-recommendation of educational resources. In: Proceedings of the 8th International Conference on Learning Analytics and Knowledge (LAK '18), pp. 409- 139 416. ACM, New York, NY, USA (2018). DOI: https://doi- org.sare.upf.edu/10.1145/3170358.3170371 50. Pardo, A., Mirriahi, N., Martinez-Maldonado, R., Jovanovic, J.: Dawson, S. and Gašević, D. Generating actionable predictive models of academic performance. In: Proceedings of the Sixth International Conference on Learning Analytics & Knowledge (LAK '16), pp. 474-478. ACM, New York, NY, USA (2016). DOI: https://doi.org/10.1145/2883851.2883870 51. Pelánek, R., Rihák, J. and Papoušek, J.: Impact of data collection on interpretation and evaluation of student models. In: Proceedings of the Sixth International Conference on Learning Analytics & Knowledge (LAK '16), pp. 40-47. ACM, New York, NY, USA (2016). DOI: https://doi.org/10.1145/2883851.2883868 52. Potts, B., Khosravi, H., Reidsema, C., Bakharia, A., Belonogoff, M. and Fleming, M.: Reciprocal peer recommendation for learning purposes. In: Proceedings of the 8th International Conference on Learning Analytics and Knowledge (LAK '18), pp. 226- 235. ACM, New York, NY, USA (2018). DOI: https://doi- org.sare.upf.edu/10.1145/3170358.3170400 53. Prestigiacomo, R., Hadgraft, R., Hunter, J., Locker, L., Knight, S., Hoven, E. and Martinez-Maldonado, R: A Translucent Approach to Understand How Teachers Talk About Classroom Data. In Proceedings of the 10th International Conference on Learning Analytics and Knowledge (LAK ’20), pp. 100-105. ACM, New York, NY, USA (2020). https://doi.org/10.1145/3375462.3375475 54. Prinsloo, P. and Slade, S.: An elephant in the learning analytics room: the obligation to act. In: Proceedings of the Seventh International Learning Analytics & Knowledge Conference (LAK '17). pp. 46-55. New York, NY, USA (2017). DOI: https://doi.org/10.1145/3027385.3027406 55. Prinsloo, P. and Slade, S.: An evaluation of policy frameworks for addressing ethical considerations in learning analytics. In: Proceedings of the Third International Conference on Learning Analytics and Knowledge (LAK '13), pp. 240-244. ACM, New York, NY, USA (2013). DOI= http://dx.doi.org/10.1145/2460296.2460344 56. Prinsloo, P. and Slade, S.: Student privacy self-management: implications for learning analytics. In: Proceedings of the Fifth International Conference on Learning Analytics And Knowledge (LAK '15), pp. 83-92. New York, NY, USA (2015). DOI: https://doi.org/10.1145/2723576.2723585 57. Prinsloo, P. and Slade, S.: Student Vulnerability, Agency and Learning Analytics: An Exploration. Journal of Learning Analytics 3(1), 159–182. (2016). 58. Prinsloo, P.: The increasing impossibilities of justice and care in open, distance learning. Presentation In: EDEN Research Workshop, Oldenburg, Germany (2016). http://www.slideshare.net/prinsp/the-increasingimpossibilities-of-justice-and-care-in- open-distance-learning 59. Quincey, E., Briggs, C., Kyriacou, T. and Waller, R.: Student Centred Design of a Learning Analytics System. In: Proceedings of the 9th International Conference on Learning Analytics & Knowledge (LAK19), pp. 353-362. ACM, New York, NY, USA (2019). DOI: https://doi-org.sare.upf.edu/10.1145/3303772.3303793 60. Santos, J L., Verbert, K. and Erik, D.: Empowering students to reflect on their activity with StepUp!: two case studies with engineering students. In: Proceedings of ARETL'12 2nd workshop on Awareness and Reflection. 931. (2012) 61. Santos, J L., Verbert, K., Govaerts, S. and Duval, E.: Addressing learner issues with StepUp!: an evaluation. In: Proceedings of the Third International Conference on Learning Analytics and Knowledge (LAK '13), pp. 14-22. ACM, New York, NY, USA (2013). DOI= http://dx.doi.org/10.1145/2460296.2460301 62. Scheffel, M,. Drachsler, H. and Specht, M.: Developing an evaluation framework of quality indicators for learning analytics. In: Proceedings of the Fifth International 140 Conference on Learning Analytics And Knowledge (LAK '15), pp. 16-20. ACM, New York, NY, USA (2015). DOI: https://doi.org/10.1145/2723576.2723629 63. Shewbridge, C; Fuster, M. and Rouw, R.: Constructive accountability, transparency and trust between government and highly autonomous schools in Flanders. OECD Publishing, Paris, France (2019). https://doi.org/10.1787/c891abbf-en. 64. Shibani, A., Knight, S. and Buckingham Shum, S.: Contextualizable Learning Analytics Design: A Generic Model and Writing Analytics Evaluations. In: Proceedings of the 9th International Conference on Learning Analytics & Knowledge (LAK19), pp. 210-219. ACM, New York, NY, USA (2019). https://doi- org.sare.upf.edu/10.1145/3303772.3303785 65. Shin, D. and Park, Y J.:. Role of fairness, accountability, and transparency in algorithmic affordance. Journal of Computers in Human Behavior 98(March), 277-284. (2019). https://doi.org/10.1016/j.chb.2019.04.019 66. Siemens, G.: Learning analytics: envisioning a research discipline and a domain of practice. In: Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (LAK ’12), pp. 4–8. New York, NY, USA (2012). DOI: https://doi.org/10.1145/2330601.2330605 67. Slade, S. and Prinsloo, P.: Learning analytics: ethical issues and dilemmas. American Behavioral Scientist. (2013). https://doi.org/10.1177/0002764213479366 68. Slade, S. and Prinsloo, P.: Student perspectives on the use of their data: between intrusion, surveillance and care. In: 8th EDEN Research Workshop, 27-28 October, Oxford, UK (2014). 69. Slade, S., Prinsloo, P., and Khalil, M.: Learning analytics at the intersections of student trust, disclosure and benefit. In: Proceedings of the 9th International Conference on Learning Analytics & Knowledge (LAK19), pp. 235-244. ACM, New York, NY, USA (2019). DOI: https://doi-org.sare.upf.edu/10.1145/3303772.3303796 70. Swenson, J.: Establishing an ethical literacy for learning analytics. In: Proceedings of the Fourth International Conference on Learning Analytics And Knowledge (LAK '14), pp. 246-250. New York, NY, USA (2014). DOI= http://dx.doi.org/10.1145/2567574.2567613 71. The IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems: Ethically Aligned Design: A Vision for Prioritizing Human Well-being with Autonomous and Intelligent Systems, First Edition. IEEE. (2019). https://standards.ieee.org/content/ieee-standards/en/industry- connections/ec/autonomous-systems.html 72. Tsai, Y. and Gasevic, D.: Learning analytics in higher education --- challenges and policies: a review of eight learning analytics policies. In: Proceedings of the Seventh International Learning Analytics & Knowledge Conference (LAK '17), pp. 233-242. New York, NY, USA (2017). DOI: https://doi.org/10.1145/3027385.3027400 73. Tsai, Y., Whitelock-Wainwright, A. and Gašević, D.: The privacy paradox and its implications for learning analytics. In: Proceedings of the Tenth International Conference on Learning Analytics & Knowledge (LAK ’20), pp. 230-239. ACM, New York, NY, USA (2020). DOI: https://doi.org/10.1145/3375462.3375536 74. UNESCO Institute for Information Technologies in Education.: Learning Analytics. (2012). Retrieved from: https://iite.unesco.org/publications/3214711/ 75. Verbert, K., Ochoa, X., De Croon, R., Dourado, R. and De Laet. T.: Learning analytics dashboards: the past, the present and the future. In: Proceedings of the Tenth International Conference on Learning Analytics & Knowledge (LAK ’20), pp. 35–40. ACM, New York, NY, USA (2020). DOI: https://doi.org/10.1145/3375462.3375504 76. Wise, A F., Zhao, Y. and Hausknecht, S N.: Learning analytics for online discussions: a pedagogical model for intervention with embedded and extracted analytics. Proceedings of the Third International Conference on Learning Analytics and 141 Knowledge (LAK ’13), pp. 48–56. ACM, New York, NY, USA (2013). DOI: https://doi.org/10.1145/2460296.2460308 77. Yang, K. and Stoyanovich, J.: Measuring Fairness in Ranked Outputs. In: Proceedings of the 29th International Conference on Scientific and Statistical Database Management (SSDBM '17). Article 22, 6 pages. ACM, New York, NY, USA (2017). DOI: https://doi.org/10.1145/3085504.3085526