In the contemporary digital age, recommender systems (RSs) shape the way in which people interact online and offline: from social media to music streaming, from e-commerce to news websites, suggested contents and products have the spotlight on platforms' interfaces and influence individuals' interests and priorities. RSs have recently been addressed by European regulations such as the Digital Services Act, whose impact on the design and management of online platforms can already be observed. While algorithmic recommendations, as the output of RSs, are aimed at improving user's experience by reducing the information overload, they can give rise to ethical concerns related to privacy, autonomy and fairness, and generate risks such as misinformation, filter bubbles and epistemic fragmentation. RSs have even been featured in legal cases involving the endangerment of minors through social media challenges and the recruitment of terrorists: this evidence underlines their deep impact on society. However, the concept of recommendation lacks a unified understanding due to the variety of domains in which the corresponding term is used. In fact, if the context of use is not specified, what is referred to as a recommendation includes not only the output of RSs, which may influence users without constraining their freedom, but also the outcomes of decision support systems (DSSs) or automated decision-making systems (ADMSs), whose impact on individuals is direct and often does not depend on their choice. This contribution aims to propose a framework for the ontological differentiation between the different concepts of algorithmic recommendation. The differentiation is based on the identification of the subject who has the responsibility and autonomy to decide whether to follow the recommendation. The adoption of this framework has the potential to improve the ethical scrutiny and auditing of AI technologies, which are required by European regulations like the DSA, for what concerns RSs, and the AI Act, regarding DSSs and ADMSs.
1. Introduction
In the contemporary digital age, recommender systems (RSs) shape the way in which people
interact online and offline: from social media to music streaming, from e-commerce to news websites,
suggested contents and products have the spotlight on platforms’ interfaces and influence
individuals’ interests and priorities. Because of the risks related to their nudging potential, RSs
deployed by online platforms are now subject to the transparency requirements of the Digital
Services Act [
However, the concept of recommendation lacks a unified understanding due to the variety of
domains in which the corresponding term is used. In fact, if the context of use is not specified, what
is referred to as a recommendation includes not only the output of RSs, which may influence users
without constraining their freedom, but also the outcomes of decision support systems (DSSs) or
automated decision-making systems (ADMSs)2, whose impact on individuals is direct and often does
not depend on their choice. The conceptual boundary between RSs and DSSs has not been clearly
established, considering that “there is still no accepted definition of DSS” [
However, this argument does not provide a defined boundary that allows to distinguish precisely, from the recipient’s perspective, whether a recommendation is the output of DSSs or RSs, as it does not clarify whether the person that directly faces the implications of the decision can choose whether to follow the suggestion of the system. For example, in the healthcare domain, a recommendation about keeping the appropriate heartbeat will have very different implications if it is produced by a runner’s wearable device or by a Holter monitor worn by a patient under anaesthesia in the operating room: although the content of the recommendation is the same, in the former case it is “consumed” by the person directly concerned by it (i.e. the runner whose heartbeat is being measured), while in the latter case it is “consumed” by a third person who decides whether it will impact the person directly involved (i.e. the surgeon).
This example highlights a guiding question for the ontological differentiation between the different concepts of algorithmic recommendation: who has the responsibility and autonomy to decide whether to follow the recommendation? To attempt an answer, we consider three subjects: the system (S), the agent (A) and the patient (P): S is the technology produces the recommendation; A evaluates the recommendation and decides whether to follow it; P directly bears the consequences of following the recommendation. The relationship between these subjects can determine a taxonomy that allows to distinguish between RSs, DSSs and DMSs: • • •
If A = P and S ≠ A, we have a RS. The subject who receives a recommendation from the AI system and can choose whether to follow it is the same who directly bears the consequences of following it. Therefore, the recommendation can influence but cannot constrain the choices of the subject who is exposed to it (e.g.: a YouTube user sees a list of recommended videos, clicks on one of them and watches it).
If S ≠ A, S ≠ P and A ≠ P, we have a DSS. The subject who receives a recommendation from the AI system decides which impact it will have on another person who is not actively involved in the decision-making process but bears its consequences. This is typical of domains where a specific expertise is required, like medicine or law (e.g.: a judge decides for how many years a culprit should be convicted based on his recidivism score).
If S = A and A ≠ P, we have an ADMS. The recommendation coming from the AI system is
directly enforced on the subject who must bear its consequences without any
human-in-the2 In fact, the AI Act [
loop intervention. The recommendation de facto becomes an automated decision (e.g.: in the
UK in 2020, when final high-school exams could not be taken due to the pandemic, A-level
grades, which determine admission to university, were assigned by the so-called Ofqual
algorithm without any mediation by teachers or schools [
The system-agent-patient (SAP) framework would contribute to establishing whether an algorithmic recommendation comes from a RS, DSS or ADMS from the perspective of its human recipient, thereby bringing conceptual clarity on the distribution of responsibility for the output of these AI systems, each of which has different implications on society. In fact, while RSs influence individuals indirectly through nudging strategies [11], DSSs and, even more so, ADMSs constrain the autonomy and freedom of the subjects who bear the consequences of following the recommendation but are not responsible for choosing whether to follow it. Therefore, the ontological differentiation based on the SAP framework has the potential to improve the ethical scrutiny and auditing of AI technologies, which are required by European regulations such as the DSA, for what concerns RSs, and the AI Act, as regards DSSs and ADMSs. [11] M. Jesse, D. Jannach, Digital nudging with recommender systems: Survey and future directions. Computers in Human Behavior Reports, 3, 100052, 2021. URL: https://doi.org/10.1016/j.chbr.2020.100052