Robot Companions as Case-Scenario for Assessing the “Subjectivity” of Autonomous Agents. Some Philosophical and Legal Remarks Elettra Stradella2 and Pericle Salvini3 and Alberto Pirni1 and Angela Di Carlo1 and Calogero Maria Oddo3 and Paolo Dario3 and Erica Palmerini1 Abstract.1 duties to autonomous robots from a legal and philosophical In this paper the European flagship project proposal Robot standpoint. In talking about rights and duties with respect to robots Companion for Citizens (RCC), grounded on the idea of capable of autonomous decisions, one should face the implications developing robot companions for citizens, is taken as a case that inevitably these terms rise, especially in the field of law. scenario for investigating the feasibility of ascribing rights and The paper is organized as follows: next section briefly explores duties to autonomous robots from a legal and philosophical standpoint. In talking about rights and duties with respect to robots the concept of autonomy, as well as the technologies that will be endowed with autonomous decision capabilities, one should face developed in the framework of the RCC project. In Section 3 the the implications that inevitably these terms rise, especially in the nexus between autonomy and duties is explored from a field of law. The paper points out the technological problems philosophical point of view. Section 4 deals with the rationale at related to the application of the notion of duty to robots and the the basis of the recognition of a subjective status to robot problems deriving from attributing a legal subjectivity to non- companions. It explores the cases of attribution of subjectivity to human entities such as robot. entities other than persons in Europe and attempts to extend such cases so as to include robotic agents as well. Finally in Section 5, the question concerning the need for having an autonomous 1 INTRODUCTION subjectivity with respect to robot companions acting in the legal The legal problem of robotics, or legal gap, as it has been environment is analysed. defined by [1], is the consequence of the new possibilities offered by technological advancements in artificial intelligence and robotics components (perception, computation and actuation), 2 FROM CURRENT ROBOTICS TO namely the possibility to have autonomous machines. In robotics, ROBOT COMPANIONS FOR CITIZENS: the term autonomy in general refers to the ability to perform a task AN OVERVIEW in an unknown environment for a prolonged period of time without The concept of autonomous agent applies to systems being human intervention. An autonomous robot can be defined as ‘a either physically instantiated or not. The former case refers to machine that collects information from the surrounding embodied agents, such as robots, i.e. those agents having both environment and utilises them to plan specific behaviours which brainware and bodyware and thus being directly capable of allow it to carry out actions in the operative environment’ [2]. The physical actions, while the latter refers to agents that have not an current legal systems, from East to West, are not ready to deal with evident physical instantiation, such as the case of non-human robots that exhibit autonomous behaviours in human-inhabited operators in financial transactions (e.g., in stock exchange markets environments. The most remarkable illustration is provided by the or in business-to-business platforms managing industrial supply case of the Google Car. As a matter of fact, although the car is chains). capable of driving autonomously, namely without the need of a Autonomous agents present both significant Scientific and human being, by law there must be a person on board, just for Technological (S&T) challenges and related Ethical, Legal and liability purposes. Things get even more complicated, from the Societal (ELS) implications, with particular reference to liability regulatory point of view, if robots are endowed with learning aspects associated to the deployment of autonomous agents in capabilities. society. In this paper the European flagship project proposal RCC Autonomy is inherently multi-scale depending of the layer of (http://www.robotcompanions.eu), grounded on the idea of the control hierarchy being awarded with a degree of autonomy, or developing robot companions for citizens, is taken as a case involving environmental or human influence in the decisional loop. scenario for investigating the feasibility of ascribing rights and Autonomy may span from low-level control (e.g., in tracking a reference trajectory in the joint space of a robot), to task planning 1 and execution given a specific objective (e.g., in identifying Scuola Superiore Sant’Anna, Pisa, DIRPOLIS Institute 2 University of Pisa, Pisa, Department of Law optimal trajectories while navigating between two locations), to the 3 Scuola Superiore Sant’Anna, Pisa, The BioRobotics Institute definition of specific objectives given a general objective (e.g., the sequence of intermediate stops in product distribution chains), to RCC S&T programme will raise ELS issues, including liability management of energetic resources (e.g., energy saving and battery aspects, which will be carefully managed and investigated in the charge policies), to cloud robotics (e.g., agents sharing decisions RCC workplan, by means of dedicated and interdisciplinary teams and experiences over ICT infrastructures), to interaction and composed by roboticists, experts in ethics, and lawyers. In this communication (e.g., the case of the “Chinese room thought paper, we will start to approach such ELS issues, by focusing on experiment”), to the decision of strategic objectives in abstract the feasibility of ascribing rights and duties to robots. form, etc. All such layers and scenarios, from the low-level to the abstract one, present subtle aspects while attempting to define autonomy, as 3 WHICH AUTONOMY? A PROVISIONAL well as to differentiate an automatic control from a degree of OVERVIEW WITHIN THE SPHERE OF autonomy. As a matter of fact, the concept of autonomy is directly DUTIES connected to automatic control, though autonomy is much more When we try to focus such complex range of claims and issues controversial. Influence of past experience on future behaviours is through the lens of ethics, we must admit the necessity of dealing not sufficient to characterize autonomy versus automatic control: a with a mass of problems, which are far from being captured and simple integrator is influenced by past experience, but nobody solved by both traditional and contemporary ethical theories [11]. would assert the integrator to be an autonomous machine (rather, it The “Robot Companion” framework could indeed constitute a is automatic, as a fundamental block of traditional control and good chance to renew the toolbox of ethics, and surely the concept automation theory). of autonomy is one of the most questioned in such field of ethics, A peculiar characteristics of an autonomous agent is the ability the robot ethics, which takes seriously into account the new to develop and learn automatic behaviours and policies, and a challenges introduced into the ethical domain through the higher degree of autonomy may be associated to a shift from low- developments of robotics. level control towards higher order functions (as it is occurring to Thus, just an overview to the topic of autonomy within the advanced robotic systems: Justin [3], the Jazz player robot contemporary literature confirms that the debate has now achieved musician [4], indoor and outdoor service robots [5], [6], just to a level of maturity [12], [15]. This is perhaps a sign of the fact that mention a few) in applying novel and emerging machine learning current technological developments seriously begin to lay down the approaches (as it is the case of the “Formal theory of creativity, conditions for being able to discuss on such a topic, beyond any fun, and intrinsic motivation” [7]). Previous experience and science fiction presuppositions. Moreover, another “travel into environmental constraints radically influence and may introduce infinity” might occur to the researcher who wanted to reach a bifurcations in shaping the evolution of agents endowed with sufficiently wide competence about the so-called robot ethics or machine learning methods or embodiment of computational machine ethics [16], [17], [2] that constitutes the unavoidable functions [8], [9]. framework for the attempt developed below. What are the associated ELS implications (particularly, with The contemporary debate about robot ethics has developed respect to liability aspects), given the potentially unmanageable some interesting results in such frame, firstly connected to the and unpredictable variety of learning experiences and operational health-care robots [18], [19], but also to the particular context of scenarios for agents being instantiated in unstructured physical child-care robots [20]. Furthermore, autonomy is an undoubtedly environments? relevant task also for robotic warfare [21], [22]. Such questions will concern next generation robots, such as In order to take a step forward in such framework, it could be those that will be developed within the “FET Flagship Candidate useful to take a step back, by examining briefly, from another point Robot Companions For Citizens” (RCC). The RCC S&T of view, the concept of autonomy and the theoretical conditions of programme proposes a radically new approach to develop its attribution to an agent. It is surely trivial to affirm that assessing machines and to truly deploy them in society as RCC Platforms: the status of autonomous agents with respect to robots is a HealthCompanion, ExploreCompanion, WearableCompanion, problematic issue. In this context, we would briefly explore an WorkCompanion, UniversalCompanion. articulation of the nexus between autonomy and duties [23] The RCC highly ambitious programme is summarized by the (another of the key-concepts of an ethical toolbox for robotics) that RCC cross-domain grand scientific challenge: “To unveil the could support a less trivial way of posing that issue. natural principles of simplexity, morphological computation and Starting with a short definition of duty, it is possible to recall a sentience and to translate the resultant scientific knowledge into paradigmatic statement drawn from Th. Reid’s Essays on the design principles and fabrication technologies for Robot Active Powers of Man (1788) [24]. Duty is neither something that Companions that effectively and safely act, interact and adapt to belongs exclusively to an agent («It’s up to you!»; «You must, over their physical and social environment”. and beyond any considerations!»), nor something that is In particular, sentience is the ability to integrate perception, intrinsically related to action («This action should be done!» «It’s cognition and action in one coherent scene and context in which impossible not to do that»). Rather, duty is structurally and action can be interpreted, planned, generated and communicated inseparably connected to both, or to agent and to action at the same [10]. Morphological computation is a novel paradigm asserting the time. In other terms, duty is a relationship between agent and role of materials in taking over some of the processes normally action that triggers “spontaneously” and “mandatory” when a attributed to control [10]. Simplexity comprises a collection of certain situation occur. For example: I see a person falling while solutions that can be observed in living organisms that, despite the she is walking in front of me and immediately I feel / perceive the complexity of the world in which they live, allows them to act and duty (as subject) to help her to get up. project the consequences of their actions into the future. Simplexity By remaining within the framework of duty, this (apparently) can be described as a property of living systems such that they can simple situation opens (at least) three areas of questioning. One is cope with the complexity of their world [10]. The highly ambitious related to the time of reaction, or: What does “immediately” in 4 LEGAL SUBJECTIVITY AND ENTITIES such a context mean? A second point regards the verb used in such OTHER THAN PERSONS: POSSIBLE situation: What do “feel the duty” or “perceiving the duty” mean? INCLUSION OF ROBOT COMPANIONS? Last but not least: Which is the meaning of the word “agent”, in relation to this situation? All these areas are widely discussed, in In debating whether, one day, robots will have rights and duties, philosophy as well as in neurosciences, but also in roboethics (see it is crucial to start wondering whether and how robots could [25], [26], [27]). For the purposes of this paper, the authors could become legal subjects, instead of ever remaining an object of the just sketch synthetically the third one – and only a little portion of law. such problematic area. Understanding the cases in which legal subjectivity is The concept of agent, in relation to the claim of duty – and to recognized to entities other than natural persons serves the purpose such specific duty («help the person who is in trouble») –, needs at to answer the question: is a legal subjectivity for robots needed (or least the clarification of a central aspect. Any duty implies a power, useful)? conceived as “to be able to do something”: if I have the duty to do In this context it is important to underline immediately that the a certain action, I must also have the power to do that action, I concept of “subjects” and “subjectivity” that it is used in this paper must be able to do what I am “obliged” to do. Otherwise, no does not refer to the philosophical notion, widely accepted by practical question can exist, i.e. any question of ethical relevance. modern and contemporary philosophy. The use of these terms is a It has been R.M. Hare [28] to identify this point with deep strictly legal use, functional to the aims of the authors. sharpness. Nevertheless, one has first to consider that the meaning and the In its turn, the “power-to-do” issue should deal with a double nature of the “legal person” and “legal subject” concepts are still question: firstly, with an external condition, that can be called “the controversial. While nobody doubts that the human being possibility side”: I had the duty to help the person who had fallen represents the legal person par excellence, it is not unanimous how in front of me, but there was a ditch along the street (or another they acquire their legal capacity – namely the capability of being physical impediment) between me and her that I have not been able entitled of rights and duties – and whether other entities, which are to exceed it. Consequently, the possibility to fulfil such a duty has not human beings, could be considered legal person in a specific been denied to me. legal system. Secondly, the “power-to-do” issue should deal with an internal With respect to the first aspect, some scholars believe that the condition, which is – on its turn – intrinsically double. So, there is legal capacity is a natural feature of human beings, so that legal what can be called “the first level capacity side”, I should have the systems can just recognize it by law; on the contrary, others think ability to perform exactly the action I am obliged to do: I can do that the legal capacity is a legal status that law awards to certain precisely the action of helping her to get up, for example, as I entities, as argued in Kelsenian theory. It is quite evident that the exactly know how to approach her and to surround his her latter approach eases awarding the status of legal person to entities shoulders while she is stretching out his her arms to get up. But it is that are not persons. Associations, foundations and organisations also possible to distinguish a “second level capacity side”, that are a significant example; indeed, the experience in existing legal implies the ability to do more than one sole action in order to orders shows that considering them as legal person gives rise to answer to the duty-question in that / such situation (“help her to get several issues and that the rationale of such an option has to be up”). The agent can choose among different possibilities, all found in patrimonial responsibility [29], [30]. oriented to the goal of helping: I can grab her arms, or I can bend Nonetheless, further questions arise from the possibility of over, so she can lean on me. Still, I can try to stop the traffic, since assigning the legal personality to entities that are not composed by she fell in the middle of a road and this is surely the first priority in a group of people, but are individual entities other than human order to help her. In other words, I can value by myself – “in beings. Can we speak about them as a legal person in legislation, complete autonomy”– what is the best action to do in this specific since they are not people in real life? Some theories argue that the situation. concepts of “person” and “human being” do not overlap at all. The The entire question, related to both an external condition and to scientific and technological progress in biology and medicine has (at least) an internal one, could be considered as the core of every led to rethink, especially at a philosophical level, these notions and possible discourse about the attribution of autonomy to an agent. the opportunity of including some stages of human life in the The authors have consciously chosen an example with multiple category of “person”; at the same time, they started to assume that facets related to a task implying movement. And they are also other living beings, such as animals and plants, or even intelligent aware that anyone of these trivial examples opens enormous things would be considered as a person [31]. Engelhardt, for problems of implementation, if it was possible to transfer the terms instance, believes that autonomous agents only, thanks to their of such question to robots – and even larger problems would need potential capacity of self-determination, can be considered as a to be questioned if the aim of this paper was to consider duties less person, irrespective of their human or non human nature [32]. related to physical aspects. In any case there are no doubts that robotic technologies, Nevertheless, a crucial point remains here at stake: It is possible whatever the level of autonomous capacity to determinate their to attribute duties to robots – and to open the discourse about this actions would be, cannot be included in the notion of person. The topic – without asking whether robots [can?] support [or not?] the intrinsic qualification of person prevents to assimilate to this set of conditions this section has tried minimally to enlighten? ethical and juridical category any entity without a naturalistic Moreover, if this paper wanted to frame this issue at a greater dimension of life and self-awareness. distance, the authors would realize that it was only a half of a The Italian constitutional framework (and the constitutional sphere, which finds its ideal completion in the legal dimension. framework of several European Union Member States) grounds on the “personality” principle to be interpreted as the general recognition of the fundamental human rights for every human being, independently from their citizenship, economic, social pain (physical and emotional – not strictly psychic because this conditions. The “recognition” of “inviolable” rights means, in the would attribute to animals a “psyche” that could be ascribed to the Italian constitutional context (Article 2), that human rights are the possibility to self-determine in right and wrong). authentic base of society, and the human being is the true scope of In this framework the recognition of subjectivity is directed the legal system and of the public organization of power. In the above all to the protection against behaviours aimed at European Charter of Fundamental Rights, Article 1 introduces the (gratuitously) inflicting pain, and to clear - though partially - the concept of human dignity: this is the leading criterion in the relationship between the animal and its owner from a strict definition of the axiological paradigm that can guide the possible dimension of property rights. attribution of some subjective status to robotic technologies. Recently a theory has been developed in France – the Therefore, sometimes the law itself individuates a distinction Marguénaud’s approach – according to which refusing to recognize between person and subject (or other forms of “subjectivities”), human rights to animals does not mean denying at all the providing hypotheses of differentiation between the two notions. protection of certain animal interests. Another approach, supported This possible differentiation grounds on the distinction between by Joel Feinberg, an American law philosopher, considers animals two “laws”: the law of legal rules (the positive law) and the law of equivalent to elderly, disabled people and minors from a legal point society, “intrinsic to society as principle and rule of coexistence” of view. As a consequence, they would be necessarily represented [33]. The positive law has the mission to recognize the “subject”, in order to fulfil their rights (Council of Europe, 2006). whilst the law of society would recognize the “person”: this means In Europe, the first laws on animal protection were approved at that the positive law can create (legal) subjects that are not persons, the beginning of the XX century. Since 1968 the Council of Europe but never denying the human being, with her form and substance approved five Conventions for the protection of: animals during and, before her, the capability to live [34]. international transport (1968, revised in 2003); animals kept for Three are the main cases of differentiation between person and farming purpose (1976); animals for slaughter (1979); vertebrate subject, and the individuation of subjects that are not included in animals used for experiments (1986); pet animals (1987). the notion of person, that we can find in the European Member Provisions for animal rights have been included in the national States law: i) unrecognized organizations and some kind of Constitutions of Switzerland (1992, 2000) and Germany (2002), corporations without legal personality; ii) conceived baby before while the EU Lisbon Treaty (2007) states that the Union and the birth; iii) animals. Member States shall, since animals are sentient beings, pay full This paper aims at individuating the rationales that base these regard to the requirements of animal welfare. In the United States, various recognitions in order to assess the eventuality to extend despite the Constitution does not mention animals, a US federal some of them to the possibility of recognition of robots’ judge was asked to rule on whether animals take benefit of the subjectivity. constitutional protections against slavery as human beings; thus, In the first hypothesis (i) the subject is a sort of summation of the judge ruled that slavery is uniquely a human activity, as those natural persons that act in order to pursue common scopes, both terms have been historically and contemporaneously applied, and economic-proprietary and not. The rationale seems to be the there is no basis to construe the Thirteenth Amendment as applying recognition to these entities of a legal capacity necessary to carry to non-humans. out the activities legally appertained to the single natural persons b) Jurisprudence and case-law in various European countries that make them. unanimously confirm the existence of a human right to the In the second (ii) a status including (fundamental) rights is protection of biosphere, the equilibrium among species, and set up recognized to a subject that is potentially (natural) person: this a right of future generations to a healthy environment and a subject must be protected under the umbrella of the principle of sustainable management of environmental resources and human dignity [35-37]. ecosystem. Animals are of course part of this reality and, from this The rights to life, to psychic and physical integrity, are not legal point of view, they can be seen as instrumental goods to the goods lavished by the legal system on individuals. They directly protection of human rights, and therefore recognized as subjects (or derive from the belonging to a human society. Because the human subjectivities) to be protected by the legal system. being is person just for her evident existence in society; although c) Animals, and pets in particular, have an “emotional” relation the embryo cannot be considered as a person, it is a “human with humans, contributing to their wellbeing and the development reality” in which we find all the dignity of the future human being. of their personality. The main objective of the Western It is possible to try an assimilation with the third category: (iii) constitutionalism and the aim pursued by legal systems as animals. described by the most important Constitutional Charters in Europe In the core values of constitutionalism certainly we can and in the other Western Countries is undoubtedly the development individuate the base for the protection of rights of non-human of personality, the happiness, or the fulfilment of a strong species [38]. interpretation of the human dignity principle. In this third a) The constitutionalism protects the human being because she framework the recognition of subjectivity would constitutionally is holder of goods – for example physic and emotional integrity – ground on the protection and promotion of a “relational good” [39]. that cannot be limited or abolished without determining an In order to investigate the possibility to give the RCs a injustice: the limitation or the abolition would directly prejudice subjectivity, it is necessary to understand whether some of these the condition of happiness of humans. From this point of view the elements could regard robotic technologies as well. creation of a “protective” status of subjectivity for the animal Certainly b) can be excluded without need of motivations. would derive from the consideration that the animal has got Indeed some reflections could be made about a) and c). “sentience” too. How animal sentience could be described? It is With regards to a) the definition of “sentience” is decisive, in evident that in this case “sentience” may be intended as the the specific meanings applied to animals and to RCs, as briefly capacity to feel sensations of pain and pleasure, and in particular discussed in Section 2. The animal’s sentience is today quite well known by difficulties of awarding robots full capacity; the same we would ethologists: they underline that “a fairly solid body of information encounter also by accepting that robots are simply mandated by about what animals are feeling is collected by indirect means. They their users, because the latter option equally requires to confront have been assembled about states of suffering experienced by farm the issue of capacity for the deputy. On the other hand, it appears animals such as pain, fear, frustration and deprivation” [40]. They rebuttable under two aspects: it is counterintuitive, because of the use various methods in order to define a pain assessment in animals detachment and possibly the physical distance between the primary [41], and the results provide evidence that the animal would be actor and his supposed offshoot; most of all, it does not take into able to experience negative sensations similar than the human ones, account the limited, but not inexistent, autonomous decision- suitable to raise the demand of justice mentioned above [42-44]. making ability the robots companions are doomed to have. Another The different content of “sentience” in the animal in comparison possibility is to consider the companion robots as autonomous to RCs prevents the recognition of a legal subjectivity for animal agents, endowed with the status of subjects, but capable of entering and the (prospective) recognition of a legal subjectivity for robots into transactions under certain constraints. The reduced capacity of to be ascribed to the same rationale. minors or of the mentally impaired, known and disciplined in the With regards to c), it is worthy to point out that the RC could current legal systems, could be taken as a model for regulation. build (is supposed to build) a “personal” relationship with the Under this special regime, robots would be entitled to act validly individuals who “use” it, and that examples of robotic technologies but only with regards to transaction of minor importance and value, with emotional-relational functions already exist (e.g., the case of those that are needed in order to satisfy the basic necessities of the well-known Paro robotic therapeutic seal). Nevertheless, their users (See, for instance, art. 409, comma 2, of the Italian Civil because of the extreme subjectivity of the capacity of an entity (or Code). simply a thing) to represent an emotional object and an instrument Another practical reason suggests to investigate the possibility of happiness for an individual, this element does not seem of awarding some kind of legal capacity to the robots companion, sufficient to ground the recognition of legal subjectivity (that could that is the issue of liability for damages. Ensuring the safety of otherwise concern televisions, cars, computers, etc.). these devices trough careful design and manufacturing does not exclude that accidents might occur either to their users or to third parties. Hence the crucial question of who and under which 5 ROBOT COMPANIONS ACTING IN THE circumstances is responsible for the damages brought about by LEGAL ENVIRONMENT: IS THERE A robots. The stance taken on the status of autonomous agents of RCs NEED FOR AN AUTONOMOUS becomes decisive in order to frame properly the problem of SUBJECTIVITY? liability. More precisely, it is necessary to appreciate whether the existing rules about producer’s liability or liability deriving from Assigning legal capacity to RCs as an acknowledgment of their the ownership or possession of things apply; or if the technology is peculiar status of “sentient beings”, comparable to animals, is an so highly developed and advanced, and provided with a certain issue to let open at the present moment. Nonetheless, the option of degree of decision-making ability, that the rationale underlying recognizing them as persons in a legal sense has to be analysed those sets of rules cannot operate. The concern should be about from a more empirical and functional perspective as well. First of fixing a general divide between traditional machines, that can be all, the prospect of creating companion robots devoted to assist designed and manufactured so that their behaviour will be elderly and disabled people requires to provide them with the predetermined or predictable by the constructor and afterwards ability of rendering basic services that go beyond acts of pure mastered by their user; and sophisticated robots, that do not material care. People with reduced capacity to move around, to correspond to this archetype. If the robots companion belong to this carry weights or even to speak out their wishes in verbal ways have latter category and cannot therefore be entirely controlled, we need to be assisted and helped also in purchasing goods, such as food, to part from a rule that assigns liability precisely on the basis of the drugs, newspapers, bus tickets. This means that the technology power that the subject who responds for the damages can exert would be more helpful and worthy whereas robots were provided over the sphere of the actual agent. Again, the basic structure of with the ability of performing legal transactions. Many operations a most legal regimes regarding injuries caused by minors and companion robot could be asked to carry out effectively imply incompetent persons could be taken as a model rule. The two cases entering into a contract. Assigning legal capacity to a robot, in this share some common features: the limited capacity of the agent, not sense, could solve the problem of having a centre of imputation of sufficient to held her fully responsible for the damages he has the effects deriving from the agreement and avoiding the contract produced; but also an independence of action, more or less to be considered void. Such an option may appear redundant substantial, that the agent exhibits and that accounts, at the same because the transactions done by robots are deemed to be time, for the possibility of the guardian to be exonerated by elementary and of minor value; moreover, they are normally demonstrating not to be at fault (or to have adopted every immediately executed, hence most often contractual remedies reasonable precaution in order to avoid accidents). would not be called to intervene. Nevertheless, one cannot exclude Recognising the autonomy of RCs, be it limited and only in principle that disputes will arise and that the problem of “functional”, may result in the potential attribution of duties or identifying the contractual parties, and their capacity of entering a obligation, deriving from the agreements undertaken or stemming transaction will become controversial. Therefore the need of by the wrongs committed. Nevertheless the legal mechanisms thus referring the contract to someone to be held responsible with evoked, both contractual and non contractual liability, are not self- regards to its effects remains. A plain answer could be to consider sufficient. If robots do not have assets to make up for their robots as a sort of extension of their users’ will and physical body, obligation or to compensate for damages, to hold them liable so that any act they execute is directly referable to them. On the without providing a vicarious responsible will not make sense. The one hand, this solution would circumvent the conceptual supplier would not get paid, the victims could not recover damages, if we stick to the previous examples. The prospect of According to this functional perspective it seems inappropriate assigning legal capacity to the RCs for practical, instead of to use “binding” legal concepts like rights and duties (and ontological, reasons definitely requires to implement other autonomy) are, and, instead, it appears more suitable a case-by- instruments through which these can be achieved. case application of existing legal instruments provided for other machines. 6 CONCLUDING REMARKS ACKNOWLEDGEMENTS Besides an ethical and social problem concerning robotic technologies, there is also a legal one. Simply speaking, the former The research leading to these results has received funding from the problem deals with whether it is right or wrong to carry out European Union Seventh Framework Programme (FP7/2007-2013) research in a specific way or field or to deploy robots in certain under grant agreement no. 289092 RoboLaw project. This work contexts or for certain tasks. Many relevant arguments in favor and was supported in part by the EU under the CA-RoboCom Project against robotics research and applications have been raised by (FP7-ICT-2011-FET-F, P.N. 284951). scholars in the last years [45-52]. On the contrary, the legal problem does not seem to care too much about the issue of robots’ legal subjectivity, whilst it should be a preliminary question to REFERENCES pose. [1] Y.H. Weng, C.-H. Chen, C.-T. Sun. Toward the Human-Robot Co- Of course every attempts to regulate new and emerging Existence Society: On Safety Intelligence for Next Generation technologies should be accompanied by careful ethical and social Robots. International Journal of Social Robots, 1:267-282, ( 2009). analyses as well as risks and safety analysis. Too often science and [2] P. Lin, K. Abney, G.A. Bekey (eds), Robot Ethics. The Ethical and technology have been embraced uniquely on the basis of political, Social Implications of Robotics, Cambridge, The Mit Press 2012. economic and/or scientific interests. The truism that the possibility [3] http://www.youtube.com/watch?v=93WHRSKg3gE to do one thing (e.g. make robots autonomous) is not enough for [4] http://www.autonomousrobotsblog.com/robotic-musicianship/ [5] www.irobot.com justifying its accomplishment is even more true in case of [6] P. Salvini, G. Teti, E. Spadoni, C. Laschi, B. Mazzolai, P. Dario machines which should interact or coexists with human beings. Peccioli: The Testing Site for the Robot DustCart. Focus on social In addressing the issue of rights and duties of autonomous and legal challenges Ieee Robotics And Automation Magazine - vol. software or robot agents, therefore, a preliminary question should 18, Issue 1 : 59:67 (2011) be concerned with the ethical and social implications ensuing from [7] J. Schmidhuber. Formal theory of creativity, fun, and intrinsic their deployment. motivation (1990-2010). IEEE Transactions on Autonomous Mental On the other hand, the issue of rights and duties should be Development, 2(3), 230-247, (2010). considered as a “second level” topic to be addressed: it is necessary [8] R. Pfeifer, F. Iida, G. Gómez. Morphological computation for to assess the (legal) possibility and significance of a recognition of adaptive behavior and cognition. International Congress Series, 1291, 22–29, (2006). subjectivity for autonomous agents. In other words, ‘to define [9] R. Pfeifer, J.C. Bongard. How the Body Shapes the Way We Think A regulations and control mechanisms to ensure sound and consistent New View of Intelligence. The MIT Press 2006; M.P. Wellman, A. behaviours’ maybe is not enough. Greenwald, P. Stone, Autonomous Bidding Agents: Strategies and The RoboLaw project, funded by the European Commission Lessons from the Trading Agent Competition, Cambridge, The Mit (EC) in the Seventh Framework Programme (www.robolaw.eu) Press 2007. aims at providing the EC with new knowledge on the regulation of [10] The Robot Companions for Citizens Consortium, The Robot robotics technologies. The most relevant result of the project will Companions for Citizens Manifesto, www.robotcompanions.eu, July be a White Paper guidelines for the regulation of emerging robotic 2012. technologies. However, the RoboLaw goal is not just to provide [11] R. von Schomberg, From the ethics of technology towards an ethics roboticists with legal regulations for bringing their inventions of knowledge policy: implications for robotics, AI & Soc, 22, 331– outside their laboratories, but to deeply analyse the impact of 348 (2008). [12] M.P. Wellman, A. Greenwald, P. Stone, Autonomous Bidding robotic technologies and applications on traditional legal concepts Agents: Strategies and Lessons from the Trading Agent Competition, and constitutional rights. Cambridge, The Mit Press 2007. In this paper, in talking about Rights and Duties with respect to [13] L.S. Sterling, T. Taveter, The Art of Agent-Oriented Modeling, autonomous agents a few critical issues have been pointed out. Cambridge, The Mit Press, 2009. May we apply to current robots and to the RCs the [14] K. Stoy, D. Brandt, D.J. Christensen, Self-Reconfigurable Robots. (philosophical and legal, philosophically grounded) notion of duty? An Introduction, Cambridge, The Mit Press, 2010. They do not seem to support the set of conditions that pertains to [15] D. Floreano, C. Mattiussi, Bio-Inspired Artificial Intelligence: the notion of duty. Theories, Methods, and Technologies, Cambridge, The Mit Press, May we recognize them a legal subjectivity? It seems very hard 2008. to individuate a “reasonable rationale” that could ground this kind [16] R. Capurro, M. Nagenborn (ed.), Ethics and Robothics, Amsterdam, IOS Press, 2009. of choice, comparing robots with the other “legal subjects”, [17] A. Beavers, Robot Ethics and Human Ethics, Special Issue of Ethics different than natural persons, already existing in the Western legal and Information Technology, 2010, 12 (3). framework. Finally, awarding a legal status to robots companion [18] E. Datteri, G. Tamburrini, Ethical reflections on health care robotics, may be necessary according to a more functional perspective. If in R. Capurro and M. Nagenborg (Eds.): Ethics and Robotics. they operate in a living and therefore legal environment, rights and Heidelberg: Akad.Verlagsgesellschaft, (2009) (IOS Press). duties are simply a legal tool for implementing the technology and [19] J. Borenstein, Y. Pearson, Robot caregivers: harbingers of expanded better reaching the social goals to which it is devoted. freedom for all?, Ethics Inform. Technol. 12, 277–288 (2010). [20] N. Sharkey, A. Sharkey, The crying shame of robot nannies: an [52] A. Matthias, The responsibility gap: ascribing responsibility for the ethical appraisal, Interaction Studies, 11 (2), 161-190 (2010). actions of learning automata, Ethics Inform. Technol. 6, 175–183 [21] P. Sullins, RoboWarfare: can robots be more ethical than humans on (2004). the battlefield?, Ethics Inf Technol., 12, 263–275 (2010). [22] M. Guarini, P. Bello, Robotic Warfare: Some Challenges in Moving from Noncivilian to Civilian Theaters, in P. Lin, K. Abney, G.A. Bekey (eds), Robot Ethics. The Ethical and Social Implications of Robotics, Cambridge, The Mit Press 2012, 129–144. [23] L. Fonnesu, Dovere, Firenze, La Nuova Italia 1998. [24] Th. Reid’s Essays on the Active Powers of Man [1788], in D.D. Raphael (ed.), British Moralists, Indianapolis, Hackett 1991, Vol II. [25] C. Allen, G. Varner, J. Zinsen, Prolegomena to Any Future Artificial Moral Agent, «Journal of Experimental & Theoretical Artificial Intelligence», 2000, 12 (3), 251-261. [26] M. Anderson, S. Anderson, Machine Ethics: Creating an Ethical Intelligent Agent, «AI Magazine», 2007, 28 (4), 15-26. [27] M. Anderson, S. Anderson, Machine Ethics, New York, Cambridge University Press 2011. [28] R.M. Hare, Freedom and Reason, Oxford, Clarendon Press 1963. [29] A. Falzea, voce Capacità (teoria gen.), in Enc. dir., VI, Milano, 1960, p. 34 s. [30] G.L. Pellizzi, Soggettività giuridica, Enc. giur. Treccani, 1990. [31] L. Palazzani, Il concetto di persona tra bioetica e diritto, Torino, Giappichelli, 1996. [32] H.T. Engelhardt, Some Persons are Humans, Some Humans are Persons, and The World is What We Persons Make of It, in S.F. Spicker – T.H. Engelhardt (eds.), Philosophical medical ethics: nature and significance, Reidel, Dordrecht, 1977. [33] G. Oppo, Ancora su persona umana e diritto, in Riv. dir. civ., 2007, I, 259 ss. [34] G. Oppo, L’inizio della vita umana, in Riv. dir. civ., 1982, I, 499 ss. [35] F.D. Busnelli, L’inizio della vita umana, in Riv. dir. civ., 2004, I, 533 ss. [36] M. Basile, A. Falzea, Persona giuridica, in Enc. dir, XXXIII, Milano, 1983. [37] F. Riccobono, Soggetto, persona, diritti, Napoli, Terzo Millennio, 1999. [38] G. Gemma, Costituzione e tutela degli animali, in Forum Costituzionale, 2004. [39] Animal Welfare, Council of Europe, 2006. [40] I.J.H. Duncan, The changing concept of animal sentience, in Applied Animal Behaviour Science 100(2006), 11-19. [41] D. M. Weary, L. Niel, F. C. Flower, D. Fraser, Identifying and preventing pain in animals, in Applied Animal Behaviour Science 100 (2006), 64–76. [42] M. Nussbaum, Justice for Non-Human Animals, The Tanner Lectures on Human Values, November 13, 2002. [43] R. Scruton, Animal Rights and Wrongs, 3rd ed., London, Metro, 2000. [44] M. Scully, Dominion: The Power of Man, the Suffering of Animals, and the Call to Mercy, New York, St. Martin's Griffin, 2002. [45] J.M. Galvan, On Technoethics, in «IEEE Robotics and Automation Magazine», December, 2003. [46] Lin P., Abney K., and Bekey G.. ‘Robot ethics: Mapping the issues for a mechanized world, Artificial Intelligence 175.5-6, (2011). [47] G. Veruggio, F. Operto: Roboethics: Social and Ethical Implications of Robotics, Springer Handbook of Robotics, 2008. [48] Salvini P., Laschi C., Dario P., 'Roboethics and Biorobotics: Discussion of Case Studies', Proceedings of the ICRA 2005, IEEE International Conference on Robotics and Automation, Workshop on Robot-Ethics, Rome, Italy, April 10, 2005. [49] D. Marino and G. Tamburrini, Learning robots and human reponsibility, Int. Rev. Inform. Ethics 6, 46–51 (2006). [50] D. J. Calverley, Imagining a non-biological machine as a legal person, AI Soc. 22, 523–537 (2008). [51] S. N. Lehman-Wilzig, Frankenstein unbound: toward a legal definition of artificial intelligence, Futures 13, 442–457 (1981).