This work studies the support research on multiagent systems can give to the automation of computation of accountability (computational accountability), an din particular to accountability in organizational contexts. It introduces the idea of guaranteeing accountability as a design property of socio-technical systems, and explain a set of principles which a socio-technical system should respect. We explain the ADOPT protocol, which is a way that allows to realize computational accountability in an organizational setting, by means of multiagent systems technology.
1 In the sense that the same person may react in di erent ways to similar situations, depending on conditions that more often than not are not modeled in the system. them, as it does not support who is in charge to get a clear picture of how tasks are, or are not, being carried on. This aspect becomes aggravated when the STS involves principals who belong to di erent organizations and cross-organizational relationships have to be established and maintained.
It is precisely because of the principals' autonomy that accountability becomes
a critical feature for an enterprise. When something deviates from the expected
and desired behavior, an accountable enterprise is committed to understand
what went wrong and why, and to nd a proper course of repairing action. What
one expects from STSs is that such a process be, at least partially, automated.
In other words, STSs should provide the ground for a form of computational
accountability [
There are, indeed, attempts in business software applications to model the
distribution of tasks and the related responsibility. For instance, the BPMN
language for modeling work ows supplies the lane construct to distribute tasks
among di erent actors. The idea is that an actor is responsible for performing
the activities that lay in its lane, but a lot is left to intuition. Speci cally,
responsibility is not modeled explicitly, it is grasped from the work ow structure
and may concern an o ce, not necessarily individuals. Then, being responsible
for carrying out some activity does not mean being accountable for it. A more
explicit instrument is the RACI matrix [29] by which each activity is associated
with at least one responsible actor, exactly one accountable actor, and possibly
some consulted and informed actors. The limit is that a RACI matrix is
substantially an o -line resource that does not support actively an actor during her
activities. It cannot be used as a monitoring tool on-line, but just as piece of
data that a forum consults a posteriori to determine who was in charge of what.
In [
This paper reports the results of the two-year piloting phase of the AThOS 2
project. The ambitious, long-term objective of the AThOS project is the
design of a framework for computational accountability in the context of
crossorganizational business processes. From a methodological point of view, AThOS
aims at providing a modeling framework for STSs, where the focus is shifted from
a procedural representation of work ows to a declarative representation that is
centered around the key notions of interaction and social relationships among
actors. From a practical perspective, AThOS aims at providing a new enterprise
software engine that monitors on-going interactions (and processes), and
supports accountability. To achieve these results, the framework we are developing
takes advantage of previous declarative approaches to the modeling of business
processes, and integrates them with an explicit, rst-class representation of the
notion of \relationship", that relies on social commitments [27]. On the
practical side, relying on previous works in literature (among which [
The paper begins by explaining the kind of accountability the research focuses on (Section 2). Then, it introduces the notion of accountability by design, studying its feasibility in computational settings (Section 3). It continues by explaining the ADOPT protocol, which is a way to achieve computational accountability in multiagent systems (Section 4), and a rst direction of implementation (Section 5). A discussion ends the paper (Section 6). 2
Accountability is a wide-ranging concept, that can take on many di erent characteristics and meanings, depending on the discipline in which discussion takes place. So, the rst point that it is relevant to underline is that our focus is posed on accountability in organizational settings. We do not care whether the organization is persistent, meaning that it has long-lasting goals that will produce many interactions (like a selling company), or if it is speci cally created to achieve a once-in-a-time goal (like building a house), and will dissolve afterwards. We assume, however, the organization's functioning to be supported by an STS. In such a context, we aim at realizing accountability as computational process; more speci cally, a backward-looking, institutional process that permits one entity to be held to account by another.
The accountability process activates when a state of interest is reached, and
an authoritative entity wishes to hold to account those behind said state.
Following [
The realization of organizational accountability brings along practical advantages which motivate the purpose of studying accountablity as a system property. Evidence is provided by many studies and documents which promote the adoption of accountability frameworks. Among them, reports by the United Nations [33] and organizations like the OECD [26]. We brie y summarize the main reasons for realizing accountability frameworks and, thus, to study computational accountability. First of all, accountability within an organization is more than reporting information, disclosing reasons, or sanctioning the culprits. It is generally considered as a powerful feedback mechanism aimed at improving performance. An accountability framework, like the one of UNICEF [31] or WHO [32], explicitly establishes desired results, monitors and measures performance, uses the feedback obtained by the accountable parties to decide which changes should be made (which actions should be taken) to achieve the intended result. Such a process enables the evolution of an organization in a way that is functional to the ful llment of its objectives. It is helpful both in those situations where the objectives, to be achieved, need to rely on external actors after the stipulation of a covenant, and in those cases where objectives depend entirely on processes that are internal to the organization itself. Another positive e ect of organizational accountability is that it helps turning implicit expectations into explicit expectations. This is important because expectations that are not communicated create confusion and tensions inside an organization. Parties may be unaware of what they should deliver. Instead, a clear awareness of the expectations (and of the accountabilities) increases reliability, improves performance and trust, and helps to achieve common goals. 3
The research question that raises at this point is whether it is possible to create STSs that show accountability as a design property, that is, such that in every state the accountable parties are clearly identi able. Providing accountability by design requires decision of how to tackle certain situations, long discussed in philosophy as ethical dilemmas. One of these concerns is causal determinism, that in our context we can rephrase as whether or not an agent should be considered accountable for an adverse state if this state is inevitable, whatever action the agent will perform. We adopt the incompatibilist position to moral responsibility and conclude in a software setting that an agent cannot be held accountable in causal determinism but that the discovery of inevitable states lies with the agents. If an agent stipulates provisions for an inevitable adverse state, that same adversity would be represented to some degree in its provisions due to its inevitable nature. The agent would still be held accountable for the state, because it e ectively declares responsibility for a goal through its accepted stipulated conditions. Likewise if an agent o ers to realize a goal in the presence of a certain provisions, it is declaring for all intents and purposes that the goal is possible under certain conditions. Should that agent o er up an incorrect assessment and the goal be e ectively impossible even with stipulations, the agent will nevertheless be held accountable, because the organization \believes" an agent's declaration. We therefore conclude, thanks to the distributed nature of our vision of accountability, that an organization can consider absent both impossibilities and inevitabilities. Another ethical dilemma concerns knowledge of the consequences of one's actions. That is, can one be held accountable for an unforeseeable outcome?
As an example, consider a setting where a house is being built. The companies and the workers who contribute to the construction constitute an organization. In particular, the organization involves two members: one who should prepare a wall, wall-preparer, and another, painter, who should paint the wall white at some agreed price. The wall-preparer ful lls her/his task by spackling the wall but instead of using a standard white spackle some dark colored one (a remainder of a previous work) is used. Due to this unexpected action, painter has not the correct amount of materials and cannot ful ll the painting task at the agreed price. Though not coerced, painter cannot ful ll what expected of him/her, due to the choices of another. Clearly, despite the fact that wall-preparer did what assigned, the kind of spackle that was used hampers the painter 's work. In this case it is di cult to identify an accountable party: the painter might have provided adequate provisions instead of giving them as granted, the wall-preparer might have asked if an unusual choice would have had consequences before doing the work, the organization might have checked that the two members properly coordinated. Their sel shness prevents their understanding the assigned tasks as part of a greater work, and the lack of an explicit account of the relationships between the agents (and their tasks) contributes to the picture but. On the other hand, if expectations are not clear and shared, how could it be otherwise? For instance, wall-preparer does not know the circumstances under which the painter can achieve its goal, should it be held accountable for its decision of using a colored spackle? To foster the good functioning of the organization, then, there is the need of adequate support. 3.1
Accountability as a design property means that an STS is built in such a way that accountability can be determined from any future institutional state. Indeed, our goal lies in automating the entire process, that is, to create a structure that creates and collects contextualized information so that accountability can actually be determined from any future institutional state. We consider integral to this process the following steps: a forum must receive all information, including all causal actions, regarding a given situation under scrutiny, the forum must be able to contextualize actions to understand their adequacy and legitimacy, and nally the forum must be able to pass judgment on agents. One of the key di culties in realizing our goal lies with the notion of contextualized action. In our own societies, contextualizing might entail an examination of circumstances: for example, what should have a person done, why didn't she/he do that, what impact did her/his actions have, and given what the person had to work with, did she/he act in an exemplary fashion? The same process in a MAS would be guided by the same type of questions, though in order to facilitate the answers, we need to make use of di erent structures. In particular, we need structures that allow assessing who is accountable without actually infringing on the individual and private nature of agents.
We identify the following necessary-but-not-su cient principles a MAS must exhibit in order to support the determination of organizational accountability. Principle 1 All collaborations and communications subject to considerations of accountability among the agents occur within a single scope that we call organization.
Principle 2 An agent can enroll in an organization only by playing a role that is de ned inside the organization.
Principle 3 An agent willing to play a role in an organization must be aware of all the powers associated with such a role before adopting it.
Principle 4 An agent is only accountable, towards the organization or another agent, for those goals it has explicitly accepted to bring about.
Principle 5 An agent must have the leeway for putting before the organization the provisions it needs for achieving the goal to which it is committing. The organization has the capability of reasoning about the requested provisions and can accept or reject them.
Principle 1 calls for situatedness. Accountability must operate in a speci c context because individual actions take on their signi cance only in the presence of the larger whole. What constitutes a highly objectionable action in one context could instead be worthy of praise in another. Correspondingly, a forum can only operate in context and an agent's actions must always be contextualized. The same role in di erent contexts can have radically diverse impacts on the organization and consequently on accountability attribution. When determining attribution, thus, an organization will only take into account interactions that took place inside its boundaries. Placing an organizational limit on accountability determination serves multiple purposes. It isolates events and actors so that when searching for causes/e ects, one need not consider all actions from the beginning of time nor actions from other organizations. Agents are reassured that only for actions within an organization will they potentially be held accountable. Actions, thanks to agent roles (Principle 2), also always happen in context.
To adequately tackle accountability by categorizing action, we must deal with two properties within a given organization: 1) an agent properly completes its tasks and 2) an agent does not interfere with the tasks of others. The principles 2{5 deal more explicitly with the rst property, i.e., how to ensure that agents complete their tasks in a manner fair for both the agents and the organization. The second property is also partially satis ed by ensuring that, in the presence of goal dependencies, the rst agent in sequence not to complete its goal will bear accountability, not only for its incomplete goal, but for all dependent goals that will consequently remain incomplete. That is, should an agent be responsible for a goal on whose completion other agents wait, and should that agent not complete its goal, then it will be accountable for its incomplete goal and for that goal's dependents as well.
As an organizational and contextual aid to accountability, roles attribute
social signi cance to an agent's actions and can, therefore, provide a guide to the
severity of non-adherence. Following the tradition initiated by Hohfeld [
Going back to the painter example, before agreeing to be the organization's painter, painter would stipulate provisions for its role goals, in this case, white wall. An organization, accepting painter 's provisions, would then add white wall as a goal condition to the job description of wall-preparer. Wall-preparer would in turn accept its new goal condition. Come execution time, when wall-preparer adds the black stripe, not only will painter 's provisions not be met, but wallpreparer will also have violated its own goal conditions. Since wall-preparer knows what it did was wrong thanks to goal conditions, causally contributed to an adverse situation of work failure, and could have avoided causally contributing, it will be held accountable for the adverse state. 4
We now introduce our vision to achieve computational accountability via a proper design of the interactions between software components. The rst step is a paradigm shift: from the process-oriented view to the agent-oriented view. The process-oriented view is substantially activity-centric { that is, procedural. It focuses on the (business) process that is carried on by a software component, but overlooks the interaction between components, at the core of the realization of STSs. Instead, an STS can be conveniently thought of as a Multi-agent System (MAS) where software agents interact with each other on behalf of human principals. An advantage of the agent perspective is that it enables the use of some design (and programming) abstractions.These are, for instance, the environment where the agents operate, the norms governing organizations and institutions, the roles, and their powers, possibly de ned therein, and an explicit representation of interactions, for instance, via social commitments.
The agent abstraction allows us to think of software modules as goal-oriented components. That is, the focus moves from the activities within a process to the goal the process aims at reaching. The design process can therefore be modularized, because the interactions among agents (i.e., components) can be kept separated from the inner process of each agent. In fact, interactions can be speci ed as a consequence of the agents' goals. That is, the agents create and maintain interactions as a means for reaching their goals. The internal process of the agent lies at a di erent design level with respect to interaction. For the accountability purpose, only the interaction level is relevant since it makes explicit the engagements that agents have agreed upon. Monitoring the progress of these engagements is one the main objectives of an accountable system.
Our intuition is that an STS can be seen as an organization in which agents
(i.e., the software components of the STS at hand) can join and operate. More
precisely, we follow the ontological de nition of organizations given in [
The internal decision process, by which Org decides whether to accept the agent, are outside the scope of ADOPT. Here we are only interested in the nal decision of accepting the agent. Step (2) encodes such a case. Once Org has accepted Agi, this agent is obliged, by the engagement created at step (1), to commit to use Ri powers in case Org will ever need them. This is done with the commitments created at step (3). After these three steps, agent Agi is enrolled within the organization. Notably, these few steps are su cient to satisfy the rst three principles we have identi ed. In fact, the protocol imposes the existence of an organization within which all the relevant events must occur (Principle 1). The protocol allows an agent to be part of an organization only by playing a role within that organization (Principle 2), and nally, the protocol is such that an agent is aware of the powers it will possibly use within the organization as player of a speci c role (Protocol 3).
Goal-assignment Phase. The second phase of ADOPT considers the assignment of goals to some member of the organization. According to Principle 5, an agent should be able to negotiate with the organization the provisions it needs in order to accomplish a given goal. This negotiation is not part of the ADOPT proposal, and any negotiation algorithm can be used. Here, we just assume that after the negotiation, the agent and the organization have agreed upon the goal gi;k the organization wants Agi to perform, and the provision provi;k the agent demands to the organization as a prerequisite. Such an agreement is \sealed" by the agent and the organization in the ADOPT protocol as below outlined. (4) Org : commit to assign goal gi;k with provisions provi;k if Agi accepts (5) Agi : commit to bring about gi;k if Org assigns it and brings about provisions provi;k (6) Org : assign gi;k to Agi (7) Org : bring about provision provi;k (8) Agi : achieve goalgi;k In step (4), Org commits towards agent Agi that, if the agent will accept to accomplish the goal, it will assign the goal to the agent, and will supply Agi the agreed provisions. Thus, also Org takes on commitments towards its members. These commitments, in particular, assure that Org will never ask an agent to bring about a goal, for which either no agreement was reached, or provisions are missing. In case Org contravenes this expectation, it will be held accountable as any other player in the system. In step (5), Agi creates the commitment of bringing about goal gi;k provided that the provisions provi;k have been supplied, and the goal has been assigned to it. In the last three steps, the two parties operate so as to satisfy their commitments. Org will bring about the provisions4, and will assign goal gi;k to Agi. On its side, Agi will try achieve the assigned goal so as to satisfy its commitment towards Org.
ADOPT supports the realization of an accountable system through the
mutual commitments that exist between Org and any agent Agi, which are made
explicit. This feature realizes Principle 4, for which an agent is accountable only
for those goals it has explicitly accepted to achieve. In fact, in our proposal,
an organization cannot command one of its members to achieve a goal. Rather,
the organization will propose a goal, and the agent will have to decide whether
explicitly accepting that goal. Only in this way it becomes possible to assess
accountability. It is possible to verify that the ADOPT protocol satis es
Principle 4 via model checking. In fact, di erently from the other principles that are
directly satis ed by the structure of the organization we propose, the veri cation
of this principle demands consideration of the possible evolutions of the
protocol. Indeed, one has to guarantee that, in any possible run of the protocol, the
organization assigns a goal to an agent only when this goal is accepted. In [
Conditional antecedent fail
Expired
Active antecedent
Detached consequent
consequent fail Satis ed
Violated
One straightforward way for implementing ADOPT is to rely on the notion of
social commitment [
Commitments can be used to specify the semantics of message-based
protocols, or they can be a means for specifying protocols on their own, see for
instance the JaCaMo+ agent platform [
In the quest for the computational accountability, we have previously advocated
a shift from a procedural view to an agent-oriented view, which usually features
on a declarative approach. Think, for instance to BDI agents and the way they
are realized in many agent-based platforms (e.g. JaCaMo [
BALSA is extremely interesting, in particular because it introduces a novel perspective on the modeling of business processes. However, when it comes to the coordination of di erent business processes, the methodology refers to choreography languages and techniques proposed for service-oriented computing. Thence one of the major disadvantages of this approach and of its descendants, that is the lack of a clear separation of concerns between the coordination logic and the business logic. A business process will internalize the message exchange speci ed by the choreography at hand. Should the choreography be changed, it would be necessary to modify also the business processes accordingly. The happens because choreographies realize a form of subject coordination.
In [
The ADOPT protocol aims at supporting accountability at the organizational level, that is, ADOPT formalizes how an accountable relationship can be established between an organization and each of its members. Our long-term goal, however, is to realize an accountable system where the accountability property cuts across all aspects of an organization, and hence it holds not only between the organization and each member, but also between any two members of the organization. This is particularly important, and challenging, in an STS, where agents are autonomous in the achievement of their goals, and the interactions raising in this process may be problematic. The intuition, thus, is that every interaction that arises between any two agents in an organization should be accountable, in the sense that it should always be possible to identify an agent that is held to account towards another agent about its conduct. It is worth noting, however, that the accountability property must not be thought of as a means for limiting the autonomy of the agents. On the contrary, in our view, an accountable system supports the agents in their practical reasoning by making their (mutual) expectations explicit.
Commitments can play a fundamental role in realizing this vision since they
formalize a tie between the internal goal of an agent with the relationships that
the same agent may create in order to achieve that goal. In [30] this relation
between goals and commitments is formalized by an operational semantics given
as a set of patterns of practical reasoning rules. The Social-Continual Planning
(SCP) technique introduced in [
Another important aspect that we have to consider in the development of an
accountable system, is that agents are autonomous and can violate their
commitments on purpose. This eventuality must not be overlooked by a well-designed
accountable framework; nonetheless, an accountable system is not, in our view,
nalized to sanctioning the agents that violate their commitments (although
sanctioning mechanisms could be contemplated). Our goal, in fact, is in providing
agents with feedback about their accountability relationships. Under this respect,
an accountable framework should include some form of diagnostic reasoning (see
e.g., [
In this paper we have summarized the main results about computational accountability we have collected in the scope of the AThOS project. AThOS was a two-year, local project completed in 2017, whose main objective was to shed some light on the design and development of accountable Sociotechnical Systems (STSs). Current methodologies for the development of STSs, are mainly focused on procedural aspects, with the purpose of gaining in e ciency of execution. Many such systems rely on languages (like BPMN, YAML, or UML Activity Diagrams) and related tools for modeling work ows. The procedural perspective is inadequate for the computational accountability purpose. First of all, it results in a too rigid model where the possible lines of activity are all delineated a priori. Each actor is forced to follow a prede ned course of action, without the possibility to deviate for taking advantage of arising opportunities or for managing unexpected emergencies. More importantly, a procedural perspective, being activity-centric, overlooks the interaction dimension, which is central for the accountability reasoning. An accountability system, in fact, should in principle involve at least two actors: one which is held to account for its actions, and one (often referred to as the forum) which judges the conduct of the previous one. Both actors rely on the relationships they have created, and on the mutual expectations these relationships induce, to sustain their position (i.e., good vs. bad conduct). The AThOS project answers to these de ciencies by modeling STSs based on the typical abstractions provided by the agent-oriented paradigm (e.g., electronic institutions, organizations, norms, etc.). By leveraging on these abstraction, the main contribution of AThOS lies in the de nition of organizational accountability : the STS becomes the organization within which all the relevant interactions among the actors (i.e., the agents) occur, and are explicitly traced. We have introduced ve principles underpinning the accountability at the organizational level, and presented the ADOPT protocol as a means for satisfying these principles at the design of the STS. From a practical point of view, we have envisaged the use of social commitment for representing, and manipulating, the accountability relationships between the organization and its members.
Social commitments pave the way to further extensions that are sketched
in the paper. They provide a promising solution for dealing with accountability
of data and information management, overcoming some limits of the
artifactcentric proposals [
Another interesting line of research that stems from using social
commitments is about acting in an accountable system. In [
Intell. Law, 7(1):97{113, 1999. 28. Munindar P. Singh. Norms as a basis for governing sociotechnical systems. ACM
Transactions on Intelligent Systems and Technology, 5(1):21:1{21:23, 2013. 29. Michael L. Smith and James Erwin. Role and responsibility charting (RACI). https://pmicie.starchapter.com/images/downloads/raci_r_web3_1. pdf, 2005. 30. Pankaj R. Telang, Munindar P. Singh, and Neil Yorke-Smith. Relating Goal and Commitment Semantics. In Post-proc. of ProMAS, volume 7217 of LNCS. Springer, 2011. 31. United Nations Children's Fund. Report on the accountability system of UNICEF. https://www.unicef.org/about/execboard/files/ 09\discretionary{-}{}{}15\discretionary{-}{}{}accountability\ discretionary{-}{}{}ODS-English.pdf, 2009. E/ICEF/2009/15. 32. World Health Organization. WHO accountability framework. http://www.who.
int/about/who_reform/managerial/accountability-framework.pdf, 2015. 33. Mounir Zahran. Accountability Frameworks in the United Nations System. https://www.unjiu.org/en/reports-notes/JIU%20Products/JIU_REP_ 2011_5_English.pdf, 2011. UN Report.