<?xml version="1.0" encoding="UTF-8"?>
<TEI xml:space="preserve" xmlns="http://www.tei-c.org/ns/1.0" 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
xsi:schemaLocation="http://www.tei-c.org/ns/1.0 https://raw.githubusercontent.com/kermitt2/grobid/master/grobid-home/schemas/xsd/Grobid.xsd"
 xmlns:xlink="http://www.w3.org/1999/xlink">
	<teiHeader xml:lang="en">
		<fileDesc>
			<titleStmt>
				<title level="a" type="main">Characterizing Artificial Socio-Cognitive Technical Systems</title>
			</titleStmt>
			<publicationStmt>
				<publisher/>
				<availability status="unknown"><licence/></availability>
			</publicationStmt>
			<sourceDesc>
				<biblStruct>
					<analytic>
						<author>
							<persName><forename type="first">Rob</forename><surname>Christiaanse</surname></persName>
							<affiliation key="aff0">
								<orgName type="institution">Vrije Universiteit Amsterdam</orgName>
								<address>
									<country key="NL">The Netherlands</country>
								</address>
							</affiliation>
						</author>
						<author>
							<persName><forename type="first">Aditya</forename><surname>Ghose</surname></persName>
							<affiliation key="aff1">
								<orgName type="institution">University of Wollongong</orgName>
								<address>
									<country key="AU">Australia</country>
								</address>
							</affiliation>
						</author>
						<author>
							<persName><forename type="first">Pablo</forename><surname>Noriega</surname></persName>
							<affiliation key="aff2">
								<orgName type="institution">IIIA-CSIC</orgName>
								<address>
									<country key="ES">Spain</country>
								</address>
							</affiliation>
						</author>
						<author>
							<persName><forename type="first">Munindar</forename><forename type="middle">P</forename><surname>Singh</surname></persName>
							<affiliation key="aff3">
								<orgName type="institution">North Carolina State University</orgName>
								<address>
									<country key="US">USA</country>
								</address>
							</affiliation>
						</author>
						<title level="a" type="main">Characterizing Artificial Socio-Cognitive Technical Systems</title>
					</analytic>
					<monogr>
						<imprint>
							<date/>
						</imprint>
					</monogr>
					<idno type="MD5">CE00D2B175893114C56C6295D8D95F5D</idno>
				</biblStruct>
			</sourceDesc>
		</fileDesc>
		<encodingDesc>
			<appInfo>
				<application version="0.7.2" ident="GROBID" when="2023-03-25T04:04+0000">
					<desc>GROBID - A machine learning software for extracting information from scholarly documents</desc>
					<ref target="https://github.com/kermitt2/grobid"/>
				</application>
			</appInfo>
		</encodingDesc>
		<profileDesc>
			<abstract>
<div xmlns="http://www.tei-c.org/ns/1.0"><p>This paper is an invitation to examine a class of socio-technical systemsartificial socio-cognitive (ASCS)-whose distinctive nature is that they may involve humans as well as artificial agents who interact in a regulated milieu. We propose a characterization of these ASCS and build on that characterization to describe how these systems evolve.</p></div>
			</abstract>
		</profileDesc>
	</teiHeader>
	<text xml:lang="en">
		<body>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="1">Motivation</head><p>In recent years we have witnessed the appearance of several socio-technical systems like Facebook, eBay, Amazon Turk that have produced significant changes in the way everyday social coordination takes place. Changes that involve not only new types of coordination but, as is evident in the first two cases, coordination at a massive global extent. While these three examples may be paradigmatic of the highly visible and successful systems, the phenomenon includes a large amount of systems that share a number of cognate features and have a similar or potentially similar social and technological impact.</p><p>We believe it is worth taking a systematic look at these systems. This paper outlines a modest contribution in that direction. We point towards a characterization of these systems by introducing some terminological distinctions and an abstract descriptive framework. We, then outline the key elements of a particular aspect that we believe has not received yet systematic attention: the process through which these systems are created or updated.</p><p>Our interest is about socio-technical systems (STS) but will limit our scope to a particular kind that we call "artificial socio-cognitive systems" or ASCS. These matters were first discussed in <ref type="bibr" target="#b9">[10]</ref> and developed in <ref type="bibr" target="#b10">[11]</ref>. To illustrate some ideas we shall refer to electronic commerce and public health care, and use the system presented in <ref type="bibr" target="#b8">[9]</ref> for more specific references.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="2">A vocabulary for characterization</head><p>Socio-cognitive systems come in many different forms. Most are built around an IT platform that offers coordination capabilities, but the intents, structures and functionalities are typically widely divergent. Within the ambit of socio-cognitive systems, one would find systems as diverse as the ecologies built around social media platforms such as Facebook, Twitter or LinkedIn, B2C trading platforms such as eBay, B2B platforms such as Ariba or NASDAQ, crowdsourcing applications like Ushahidi or the Amazon Turk, mixed-level participatory social simulation, multiplayer online games, public health systems, or military command and control systems. There are also unlikely instances such as a flashmob or a criminal syndicate that leverages an IT infrastructure for coordination.</p><p>For researchers interested in developing a better understanding of such systems, it useful to explore what is common to these systems, and how they differ. A careful characterization of such systems using a common set of principles, or a common vocabulary can help. There is a long history of similar, principled approaches to characterize broad classes of systems. Kenneth Arrow's <ref type="bibr" target="#b2">[3]</ref> seminal work on characterizing social decision processes (an equally broad and variegated class as the one of interest here) is perhaps the earliest example. Arrow offered a set of postulates that formalized what are arguably intuitive requirements for social decision processes (specifically preference aggregation functions) and went on to establish the well-known result that no function existed that satisfied all of these properties. A similar exercise was undertaken by Alchourron, Gardenfors and Makinson <ref type="bibr" target="#b0">[1]</ref> to systematize what was at that time a highly varied class of belief revision operators.</p><p>Our intent here is similar. What we present below represents the first steps towards a comprehensive set of properties for characterizing the systems of interest. We will distinguish between two classes of properties: definitional and architectural. Definitional properties will offer a vocabulary for discriminating artificial socio-cognitive systems from other socio-technical systems. We shall use architectural properties to characterize sub-classes of such systems.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>Definitional properties:</head><p>Broadly speaking, our aim is to study systems that involve several rational participants who come together to perform a collective activity that they cannot accomplish on their own and such action does not occur directly between individuals but is mediated by technological artefacts. The following properties are a first attempt towards a top-down abstract characterisation of ASCS:</p><p>-System An artificial socio-cognitive system is composed by two ("first class") entities: a social space and the agents who act within that space. The system exists in the real world and there is a boundary that determines what is inside the system and what is out. -Agents Agents are entities who are capable of acting within the social space. They exhibit the following characteristics:</p><p>• Socio-cognitive Agents are presumed to base their actions on some internal decision model. The decision-making behaviour of agents, in principle, takes into account social aspects because the actions of agents may be affected by the social space or other agents and may affect other agents and the space itself <ref type="bibr" target="#b3">[4]</ref>. • Opaque The system, in principle, has no access to the decision-making models, or internal states of participating agents.</p><p>• Mixed Agents may be human or software entities (we'll simply call them "agents" or "participants where it is not necessary to distinguish). • Heterogeneous Agents may have different decision models, different motivations and respond to different principals. • Autonomous Agents are self-motivated, not necessarily competent or benevolent, hence they may fail to act as expected or demanded of them. -Social space. This is the environment or milieu where agent interactions take place.</p><p>It should have the following properties:</p><p>• Open Agents may enter and leave the social space and a priori, it is not known (by the system or other agents) which agents may be active at a given time, nor whether new agents will join or leave at some point or not. • Perceivable All interactions and events within the shared social space are mediated by technological artefacts-that is, as far as the system is concerned there are no direct interactions between agents outside the system and only those actions that are mediated by a technological artefact that is part of the system may have effects in the system-and although they might be described in terms of the five senses, they can collectively be considered percepts. • Constrained In order to coordinate actions, the space includes (and governs) regulations, obligations, norms or conventions that agents are in principle supposed to follow. • Persistent The social space may change over time in two ways: either by events that happen while the systems is enacted and by the actions of agents; or through some system functionalities that are part of the system design and are triggered while the system is enacted. <ref type="foot" target="#foot_0">5</ref>We see these systems as socio-technical systems because of the participation of humans and computational components <ref type="bibr" target="#b13">[14]</ref>, although they are better understood in the sense of <ref type="bibr" target="#b12">[13]</ref> where software agents may also be involved. The term artificial is used to evoke the existence of some external design of the system and the term socio-cognitive to suggest that in order to characterise or deploy them we need to " 'understand' and reproduce features of the human social mind like commitments, norms, mind reading, power, trust, 'institutional effects' and social macro-phenomena" <ref type="bibr" target="#b3">[4]</ref>. Because of the assumption of intrinsic constraint on agent interactions, the above assumptions characterise a type of normative multiagent system <ref type="bibr" target="#b1">[2]</ref>.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>Architectural properties:</head><p>These are properties that one would expect that all ASCS should, However have they would be achieved with different means and expressed in different ways and degrees. Thus we want to make them explicit for two main reasons, one is help us to classify systems into classes and get hold of appropriate expressive and functional means for modelling or deploying particular classes.</p><p>-Information decentralization: The extent to which information is decentralized varies across these systems. An email system would represent full decentralization (discounting settings where the email network manager obtains privileged access to all emails exchanged). In a public health system, patients own data that pertains to them, but clinicians have access to the records of all patients that they attend to, while adminstrators and insurers have access to even larger classes of medical records. A social media platform such as Facebook provides limited (policyregulated) access to data for specific users, while the providers of the platform have privileged access to all data from all users. Information decentralization in military command-and-control session would be similar. -Governance/control: Systems vary in the extent of autonomy afforded to constituent agents. An email system offers considerable autonomy, as would a flashmob. A B2B market offers a slightly lower level of autonomy, by requiring participants to conform to a set of market rules. A military command-and-control system would traditionally offer very little autonomy to the personnel under its command, but modern "'network-centric warfare"' offers considerably greater autonomy. -Fluidity of norms: A flashmob or an email system involves a minimal set of norms, but these are also relatively static. The market rules governing B2B or B2B market providers also tend to change relatively infrequently. Social media platforms frequently revise norms (specially those governing privacy, reacting to shifting user perceptions). Many online multi-player games support configurable games-for these the norms are clearly highly fluid. -Transparency: Market providers (specially B2B providers) are obliged to be very transparent in terms of the information available and the norms governing their behaviour (stock markets, for instance, need to comply with stringent transparency requirements imposed by market regulators). Public health systems are not always required to be transparent, both with the norms that govern their operations and the information they retain. Patient information clearly cannot be shared, but public health systems are often reluctant to share operational data for fear of being shown to be inefficient. Military command-and-control systems are by definition non-transparent. -Accountability: Public health systems and stock markets are typically held to very high standards of accountability. A social media platform might be required to be accountable to some degree in the event of privacy breaches or other adverse events. On the othe extreme, an email system or a flashmob represent examples of systems with very lax accountability requirements. -Nature of identity: A stock market (or other B2B market providers), a public health system and a military command-and-control system would insist on the true identity of each participating agent. A social media platform might permit the same agent to assume multiple identities. A flashmob would, in general, be not particularly concerned with the identities of the participating agents.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="3">The WIT framework</head><p>From the definitional properties mentioned above, one may see ASCS as systems where it is possible to govern the interaction of agents that are situated in a physical or artifi-cial "world" by means of technological artefacts. The key element, which is not usually included in other accounts of socio-technical systems, is the "governance" or "institutional" part that mediates between the "world" and the technological artefacts. The realization that one needs to account for the relationships between the institutional aspects of an ASCS and its associated technological and working aspects, motivates an abstract characterization of an ASCS from the point of view of each of those aspects. The relationships between these three components is explained in the following "notion" and illustrated in Fig. <ref type="figure" target="#fig_0">1</ref>. <ref type="foot" target="#foot_1">6</ref>Notion 1 The WIT framework: An artificial socio-cognitive system is composed by three interrelated elements:</p><p>View 1: The world system, W, as the agents (both human and software) see it and relate to it. View 2: An ideal institutional system, I, that stipulates the way the system should behave. View 3: The technological artefacts, T , that implement the ideal system and run the applications that enable users to accomplish collective actions in the real world, W, according to the rules set out in I. The normative institutional system, I; the technological artefacts that implement it, T , and the actual world where the system is used, W. After <ref type="bibr" target="#b9">[10]</ref>.</p><p>These three views are interrelated through three binary relationships:</p><p>-The institutional world corresponds with the real world through what is known as a "counts-as" relationship <ref type="bibr" target="#b11">[12,</ref><ref type="bibr" target="#b6">7]</ref> by which (brute) facts and (brute) actions in the real world correspond to institutional facts and actions in the institutional world I provided these comply with the institutional conventions. Suppose Alice sells her property to Bob by signing off a deed in Bob's favour. The ownership of property is an institutional fact; its sale is an institutional action. The presence of ink of a certain pattern on a piece of paper is a brute fact; placing that ink with a pen is a brute action. The signed transfer deed counts as proof of ownership; signing the deed counts as a sale. A brute action creates (or revises) a brute fact; in appropriate circumstances, it counts as an institutional action and creates (or revises) institutional facts. -The conventions prescribed in the institutional world have their counterpart in the technological world in the sense that institutional conventions constitute a specification of the requirements of the system that is implemented in T . -The system, as implemented in T , is what enables interactions (through a proper interface) in W.</p><p>It should be noted that each of these three binary relationships needs to satisfy certain integrity conditions:</p><p>-The corresponds relationship needs: (i) to guarantee that the objects and concepts involved in the descriptions and functioning in I are properly associated with entities in W; i.e., that there is a bijection between terms in the languages in I and objects and actions in W. (ii) that the identity of agents in W is properly reflected in their counterparts in I and is preserved as long as the agents are active in the system, (iii) that the agents that participate in W have the proper entitlements to be subject to the conventions that regulate their interactions and in particular to fulfil in W those commitments that they establish in I, and (iv) that the commitments that are established according to I are properly reflected in W. -The implements relationship needs to be a faithful programming of the institutional conventions so that actions and effects are well programmed, norms are properly represented and enforced, etc. -Finally, the controls relationship needs to make sure that: (i) the technological artefacts work properly (communication is not scrambled, data bases are not corrupted, etc.) and (ii) inputs and outputs are properly presented and captured in W, according to the implementation of the corresponding processes in I. (iii) Algorithms and data structures in T behave as the conventions in I prescribe.</p><p>This WIT framework is used in <ref type="bibr" target="#b10">[11]</ref> to clarify how changes in the social space are accounted for in each of the three views. The framework is further used to discuss how the normative view in I is mirrored in T through an institutional specification language (in I ) that is programmed with data structures and operations that are implemented as software in T .</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="4">Describing the innovation process</head><p>Next, we elaborate the abstract WIT structure above via an envisioned reference architecture and methodology. Specifically, consider the situation where some stakeholders come together to create or modify a sociotechnical system. In what follows we shall presume that there is already an existing ASCS, S 0 (the STS-as-is) and that those participant who are in its Wcomponent are in the process of designing a new S 1 (the STS-to-be) by specifying its I 1 component (Fig. <ref type="figure" target="#fig_1">2</ref>).</p><p>We assume that the stakeholders express some requirements that they would like the STS-to-be (adapting the terminology of requirements engineering <ref type="bibr" target="#b14">[15]</ref>) to address. In general, the stakeholders may not express their requirements explicitly, but their requirements might be elicited through discussion or understood through (e.g., ethnographic) observation.</p><p>These requirements would necessarily be framed in terms of the institution to which the stakeholders belong (I 0 ). Based on these requirements, the stakeholders would enact a potentially complex negotiation to create a specification for the STS-to-be. This specification in our terms would be a normative specification and thus serves as an abstract institution. That is, an STS-to-be does not initially include any principals or technical entities (i.e., resources or infrastructure). Principals, defined as socially autonomous entities, would decide whether to adopt various roles in this institution; their adoption would succeed provided they meet requirements such as the qualifications imposed by the institution. The principals would introduce the necessary technical entities, i.e., resources and infrastructure (T 1 ). The idea here is that the technical entities needed for the STS-to-be must come from somewhere. In our stance on socio-technical systems, this means that there must be a principal behind each technical entity. In introducing the requisite technical entities, the principals would instantiate and make the institution concrete, thereby instantiating the STS-to-be.</p><p>Following Chopra et al. <ref type="bibr" target="#b5">[6]</ref>, we distinguish between stakeholders and principals. Stakeholders are social entities involved in the design process that begins from requirements elicitation and ends in the creation of a specification of an STS. Principals are social entities who adopt roles in an institution being instantiated to produce an STS. The principals are not only a changing set but frequently are drawn form a different population than the stakeholders. That is, principals may have somewhat different requirements from the (original) stakeholders, though the principals are able to adapt the given STS to meet their requirements.</p><p>The above process incorporates some subtleties, which we explain via examples. A simple imagined scenario is of e-commerce. People already trade goods and they live in a world where institutional concepts such as ownership, transfer of ownership, and money are defined and infrastructure is available for shipping and delivery, and payment. Suppose that some prospective buyers and sellers or even one imaginative person (as in the case of eBay's founding) comes up with requirements for buying and selling. These requirements are framed in terms of the stakeholders of e-commerce, buyers, sellers, market enabler, and banks. The stakeholders could all be identified from the start or a few of them (e.g., buyers, sellers, and market enabler) might begin interacting and then recruit additional stakeholders (e.g., banks). The stakeholders figure out a solution that would support e-commerce via auctions. This solution is a normative specification of the STS-to-be. Let us say the market enabler (e.g., eBay) adopts the key unique role in this STS-to-be and provides the technical infrastructure. One or more banks join in for payment processing. Gradually the other roles are adopted by other principals, and the STS is fully instantiated. This STS carries out its specific interactions, e.g., placing a bid and determining a winner, according to its normative specification. The world outside remains present, e.g., to provide a venue for sanctioning via lawsuit if one of the principals claims another principal to have violated some norm. An example from the health care domain is where the stakeholders are clinical researchers, general practitioners (GPs), and patients, who wish to promote medical research. They would work within the institution of the existing health care STS, wherein concepts such as physician and patient are defined along with relationships such as treats and advises. In that STS, the stakeholders would identify a specification of an STS-to-be in which patients are recruited to participate in various medical studies.</p><p>In typical cases, the STS-to-be would not be created from scratch but would be a modification of an existing STS. For example, if clinical researchers want to conduct a new category of medical studies where patients carry sensors on their bodies or medical studies that engage families of patients, they would invite physicians as additional stakeholders and design new interactions whereby a patient may be recruited to wear a wrist band or a patient and the patient's spouse may both be recruited to survey them on their family's diet.</p><p>Summing up the above vision of requirements modeling and realization, we see that the process begins from an institution that functions as the social substrate or world for an STS-to-be. In this view, the existing world is indeed the STS-as-is <ref type="bibr" target="#b14">[15]</ref>. The process produces a normative specification of an STS-to-be, which upon instantiation becomes the STS-to-be. The STS-to-be need not cause any structural changes to the STS-as-is. For example, commerce remains defined and appropriate in the old world even after eBay, and the health care system continues to function despite the introduction of a clinical trials STS. However, if the requirements addressed by the new STS-to-be are significant, successful operation of the STS-to-be would affect the STS-as-is. For example, less commerce may occur in person when e-commerce is successful and fewer clinical trials would be planned and executed through a traditional way of recruiting patients.</p><p>We take the view that the technical infrastructure is not silently or secretly provided. That is, behind any component of the technical infrastructure, just as of any resource, there must be a principal. For example, for e-commerce, we require that the market enable (e.g., eBay, the company) provides the market website, including functionality for sellers listing items, buyers placing bids, and the market enabler determining the winning bid. Similarly, the clinical trials STS would require a principal providing the infrastructure, which could be the clinical trials company or the hospital where the patient is seen. Requiring a principal behind the infrastructure ensures that we can impose requirements on the infrastructure and make sure that there is a principal within the STS who is accountable for such requirements. In typical cases, some of the infrastructure would come from the STS-as-is. For example, the network connectivity needed for e-commerce is provided by the Internet Service Provider of each buyer and seller. It is not within the scope of the e-commerce market interactions but its functioning is a necessary assumption for success of the STS.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="5">Closing remarks</head><p>Towards a better characterization of ASCS. In Sec. 2 we introduced a few architectural properties to classify systems. A good compilation of examples and their scoring according to that list of properties should provide insights on the correctness and completeness of that list.</p><p>Another direction worth exploring (using the WIT framework) is the interplay between the means to specify the social space and its governance, on one side and, on the other, the technological "platform" that implements those means <ref type="bibr" target="#b10">[11]</ref>). Two lines of exploration are obvious:</p><p>-Focusing on the institutional view of ASCS, look into distinct classes of applicationslike on-line role-playing games, participatory simulation environments, prediction markets-and identify the expressive features that are common to the STS that fall in the class.</p><p>-In contrast, one may look into the class of socio-cognitive systems that may be built with particular existing platforms-for example wiki-based collaborative environments, Amazon Turk-based micro tasking aggregation systems, Repast-based simulations or CryEngine-based games-and abstract from these the affordances and the expressive and operational power of the devices that implement them.</p><p>Methodological outlook . The discussion of the process of moving from and existing STS to a new one in Sec. 4 is motivated by the aspiration of building ASCS in a principled manner. While the discussion in that section was limited to the identification of the main components and activities in the innovation process, we wanted to give an indication of the actual complexity of that process and the need to make a thorough analysis on which methodological guidelines may be founded.</p><p>One particular aspect that we believe deserves serious consideration in this respect, is to identify criteria to qualify as "adequate" the design and implementation processes of an ASCS, alongside the methodologies that guarantee that those criteria are met. (see <ref type="bibr" target="#b7">[8]</ref> for a related argument).</p><p>Practical significance of ASCS There are two reasons (beyond their characterization) for the empirical study of of ASCS. One is to provide an objective basis for theoretical and technological developments. The other is to understand-from economic, sociological, political and anthropological perspectives-how value is created through ACSC and how that value can be acquired for the benefit of society. This task is, evidently, a rather obvious challenge for interdisciplinary research.</p><p>An emerging scientific field. We share the view of Castelfranchi <ref type="bibr" target="#b4">[5]</ref>, that we are on the threshold of a new society where ASCS will be a pervasive reality. It is one that we do not fully understand and one of which we are becoming citizens through our use of ASCS. It is perhaps not an exaggeration to claim that it may be worth developing a scientific view of this reality and consequently develop the conceptual and theoretical constructs to explain what is happening and to have a crisper view of what may come next. Maybe, in a way not all that dissimilar to the zeitgeist of the early fifties that gave birth to artificial intelligence-with its "mind as processor" model for individual rationality-we are witnessing a new zeitgeist that may give birth to a new artificial social intelligence.</p></div><figure xmlns="http://www.tei-c.org/ns/1.0" xml:id="fig_0"><head>Fig. 1 :</head><label>1</label><figDesc>Fig.1:The WIT trinity view of artificial socio-cognitive systems: The normative institutional system, I; the technological artefacts that implement it, T , and the actual world where the system is used, W. After<ref type="bibr" target="#b9">[10]</ref>.</figDesc><graphic coords="5,203.93,469.82,53.23,53.78" type="bitmap" /></figure>
<figure xmlns="http://www.tei-c.org/ns/1.0" xml:id="fig_1"><head>Fig. 2 :</head><label>2</label><figDesc>Fig.2: The STS-as-is (S 0 ) and the definition of the institutional component (I 1 ) of the STS-to-be.</figDesc><graphic coords="8,224.68,136.01,170.61,170.61" type="bitmap" /></figure>
			<note xmlns="http://www.tei-c.org/ns/1.0" place="foot" n="5" xml:id="foot_0">Persistence, is a matter of convention. As Sec.4 illustrates, there may be situations when stakeholders may decide to change an active system ("the system as is") into a new one (the "system to be") that is different enough to deserve that exogenous intervention and labelling. While many features of the old system (including agents and commitments) "persist" in the new one, the old system itself is acknowledged to end.</note>
			<note xmlns="http://www.tei-c.org/ns/1.0" place="foot" n="6" xml:id="foot_1">Note that our discussion about innovation (Sec.<ref type="bibr" target="#b3">4)</ref> shows how what gives rise to a new ASCS, is the formulation within the existing ASCS of the institutional part of that new ASCS.</note>
		</body>
		<back>

			<div type="acknowledgement">
<div xmlns="http://www.tei-c.org/ns/1.0"><head>Acknowledgments</head><p>Pablo Noriega received support from the European Network for Social Intelligence, SINTELNET (FET Open Coordinated Action FP7-ICT-2009-C Project No. 286370) and Generalitat of Catalunya grant 2009-SGR-1434. Munindar Singh was partially supported by the U.S. Department of Defense (National Security Agency) under the Science of Security Lablet grant.</p></div>
			</div>

			<div type="references">

				<listBibl>

<biblStruct xml:id="b0">
	<analytic>
		<title level="a" type="main">On the logic of theory change: partial meet contraction and revision functions</title>
		<author>
			<persName><forename type="first">Carlos</forename><surname>Alchourron</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Peter</forename><surname>Głrdenfors</surname></persName>
		</author>
		<author>
			<persName><forename type="first">David</forename><surname>Makinson</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="j">Journal of Symbolic Logic</title>
		<imprint>
			<biblScope unit="volume">50</biblScope>
			<biblScope unit="page" from="510" to="530" />
			<date type="published" when="1985">1985</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b1">
	<monogr>
		<author>
			<persName><forename type="first">Giulia</forename><surname>Andrighetto</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Guido</forename><surname>Governatori</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Pablo</forename><surname>Noriega</surname></persName>
		</author>
		<title level="m">itors. Normative Multi-Agent Systems, volume 4 of Dagstuhl Follow-Ups</title>
				<editor>
			<persName><forename type="first">Leendert</forename><forename type="middle">W N</forename><surname>Van Der Torre</surname></persName>
		</editor>
		<imprint>
			<publisher>Schloss Dagstuhl -Leibniz-Zentrum fuer Informatik</publisher>
			<date type="published" when="2013">2013</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b2">
	<monogr>
		<author>
			<persName><forename type="first">J</forename><surname>Kenneth</surname></persName>
		</author>
		<author>
			<persName><surname>Arrow</surname></persName>
		</author>
		<title level="m">Social Choice and Individual Values</title>
				<imprint>
			<publisher>Yale University Press</publisher>
			<date type="published" when="2012">2012</date>
			<biblScope unit="volume">12</biblScope>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b3">
	<analytic>
		<title level="a" type="main">InMind and OutMind; Societal Order Cognition and Self-Organization: The role of MAS. Invited talk for the IFAA-MAS</title>
		<author>
			<persName><forename type="first">Cristiano</forename><surname>Castelfranchi</surname></persName>
		</author>
		<ptr target="http://www.slideshare.net/sleeplessgreenideas/castelfranchi-aamas13-v2?ref=httpMay2013" />
	</analytic>
	<monogr>
		<title level="m">Influential Paper Award</title>
				<meeting><address><addrLine>Saint Paul, Minn. US</addrLine></address></meeting>
		<imprint/>
	</monogr>
	<note>AAMAS 2013</note>
</biblStruct>

<biblStruct xml:id="b4">
	<analytic>
		<title level="a" type="main">Making visible the invisible hand. the mission of social simulation</title>
		<author>
			<persName><forename type="first">Cristiano</forename><surname>Castelfranchi</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="m">Interdisciplinary Applications of Agent-Based Social Simulation and Modeling</title>
				<editor>
			<persName><forename type="first">D</forename><forename type="middle">F</forename><surname>Adamatti</surname></persName>
		</editor>
		<editor>
			<persName><forename type="first">G</forename><forename type="middle">P</forename><surname>Dimuro</surname></persName>
		</editor>
		<editor>
			<persName><forename type="first">H</forename><surname>Coelho</surname></persName>
		</editor>
		<meeting><address><addrLine>Hershey, PA</addrLine></address></meeting>
		<imprint>
			<publisher>IGI Global</publisher>
			<date type="published" when="2014">2014</date>
			<biblScope unit="page" from="1" to="314" />
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b5">
	<analytic>
		<title level="a" type="main">Protos: Foundations for engineering innovative sociotechnical systems</title>
		<author>
			<persName><forename type="first">K</forename><surname>Amit</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Fabiano</forename><surname>Chopra</surname></persName>
		</author>
		<author>
			<persName><forename type="first">F</forename><surname>Dalpiaz</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Paolo</forename><surname>Bas ¸ak Aydemir</surname></persName>
		</author>
		<author>
			<persName><forename type="first">John</forename><surname>Giorgini</surname></persName>
		</author>
		<author>
			<persName><surname>Mylopoulos</surname></persName>
		</author>
		<author>
			<persName><forename type="first">P</forename><surname>Munindar</surname></persName>
		</author>
		<author>
			<persName><surname>Singh</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="m">Proceedings of the 18th IEEE International Requirements Engineering Conference (RE)</title>
				<meeting>the 18th IEEE International Requirements Engineering Conference (RE)<address><addrLine>Karlskrona, Sweden</addrLine></address></meeting>
		<imprint>
			<publisher>IEEE Computer Society</publisher>
			<date type="published" when="2014">2014</date>
			<biblScope unit="page" from="1" to="10" />
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b6">
	<analytic>
		<title level="a" type="main">A formal characterization of institutionalized power</title>
		<author>
			<persName><forename type="first">Andrew</forename><surname>Jones</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Marek</forename><surname>Sergot</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="j">Logic Journal of the IGPL</title>
		<imprint>
			<biblScope unit="volume">4</biblScope>
			<biblScope unit="issue">3</biblScope>
			<biblScope unit="page" from="427" to="446" />
			<date type="published" when="1996">1996</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b7">
	<analytic>
		<title level="a" type="main">The design of intelligent sociotechnical systems</title>
		<author>
			<persName><forename type="first">J</forename><forename type="middle">I</forename><surname>Andrew</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Alexander</forename><surname>Jones</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Jeremy</forename><surname>Artikis</surname></persName>
		</author>
		<author>
			<persName><surname>Pitt</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="j">Artif. Intell. Rev</title>
		<imprint>
			<biblScope unit="volume">39</biblScope>
			<biblScope unit="issue">1</biblScope>
			<biblScope unit="page" from="5" to="20" />
			<date type="published" when="2013">2013</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b8">
	<analytic>
		<title level="a" type="main">Multi-agent system for recruiting patients for clinical trials</title>
		<author>
			<persName><forename type="first">Samhar</forename><surname>Mahmoud</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Gareth</forename><surname>Tyson</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Simon</forename><surname>Miles</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Adel</forename><surname>Taweel</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Tjeerd</forename><surname>Vanstaa</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Michael</forename><surname>Luck</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Brendan</forename><surname>Delaney</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="m">Proceedings of the 2014 International Conference on Autonomous Agents and Multi-agent Systems, AAMAS &apos;14</title>
				<meeting>the 2014 International Conference on Autonomous Agents and Multi-agent Systems, AAMAS &apos;14</meeting>
		<imprint>
			<publisher>International Foundation for Autonomous Agents and Multiagent Systems</publisher>
			<date type="published" when="2014">2014</date>
			<biblScope unit="page" from="981" to="988" />
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b9">
	<analytic>
		<title level="a" type="main">Regulated MAS: Social Perspective</title>
		<author>
			<persName><forename type="first">Pablo</forename><surname>Noriega</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Amit</forename><forename type="middle">K</forename><surname>Chopra</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Nicoletta</forename><surname>Fornara</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Henrique</forename><forename type="middle">Lopes</forename><surname>Cardoso</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Munindar</forename><forename type="middle">P</forename><surname>Singh</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="m">Normative Multi-Agent Systems</title>
				<editor>
			<persName><forename type="first">Giulia</forename><surname>Andrighetto</surname></persName>
		</editor>
		<editor>
			<persName><forename type="first">Guido</forename><surname>Governatori</surname></persName>
		</editor>
		<editor>
			<persName><forename type="first">Pablo</forename><surname>Noriega</surname></persName>
		</editor>
		<editor>
			<persName><forename type="first">Leendert</forename><forename type="middle">W N</forename><surname>Van Der Torre</surname></persName>
		</editor>
		<meeting><address><addrLine>Germany</addrLine></address></meeting>
		<imprint>
			<publisher>Dagstuhl</publisher>
			<date type="published" when="2013">2013</date>
			<biblScope unit="volume">4</biblScope>
			<biblScope unit="page" from="93" to="133" />
		</imprint>
	</monogr>
	<note>of Dagstuhl Follow-Ups</note>
</biblStruct>

<biblStruct xml:id="b10">
	<analytic>
		<title level="a" type="main">The challenge of artificial socio-cognitive systems</title>
		<author>
			<persName><forename type="first">Pablo</forename><surname>Noriega</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Julian</forename><surname>Padget</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Harko</forename><surname>Verhagen</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Mark D'</forename><surname>Inverno</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="m">Pre-proceedings of Coordination, Organizations, Institutions, and Norms in Agent Systems COIN@AAMAS 2014</title>
		<title level="s">Lecture Notes in Computer Science</title>
		<meeting><address><addrLine>Paris</addrLine></address></meeting>
		<imprint>
			<publisher>Springer</publisher>
			<date type="published" when="2014-05">May 2014. 2014</date>
		</imprint>
	</monogr>
	<note>page tbd</note>
</biblStruct>

<biblStruct xml:id="b11">
	<analytic>
		<title level="a" type="main">What is an institution?</title>
		<author>
			<persName><forename type="first">John</forename><forename type="middle">R</forename><surname>Searle</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="j">Journal of Institutional Economics</title>
		<imprint>
			<biblScope unit="volume">1</biblScope>
			<biblScope unit="issue">01</biblScope>
			<biblScope unit="page" from="1" to="22" />
			<date type="published" when="2005">2005</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b12">
	<analytic>
		<title level="a" type="main">Norms as a basis for governing sociotechnical systems</title>
		<author>
			<persName><forename type="first">P</forename><surname>Munindar</surname></persName>
		</author>
		<author>
			<persName><surname>Singh</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="j">ACM Transactions on Intelligent Systems and Technology (TIST)</title>
		<imprint>
			<biblScope unit="volume">5</biblScope>
			<biblScope unit="issue">1</biblScope>
			<biblScope unit="page">23</biblScope>
			<date type="published" when="2013-12">December 2013</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b13">
	<analytic>
		<title level="a" type="main">The evolution of socio-technical systems</title>
		<author>
			<persName><forename type="first">Eric</forename><surname>Trist</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="j">Ontario Ministry of Labour</title>
		<imprint>
			<biblScope unit="volume">2</biblScope>
			<date type="published" when="1981">1981</date>
		</imprint>
	</monogr>
	<note>Occasional paper</note>
</biblStruct>

<biblStruct xml:id="b14">
	<analytic>
		<title level="a" type="main">From system goals to software architecture</title>
		<author>
			<persName><forename type="first">Axel</forename><surname>Van Lamsweerde</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="m">Formal Methods for Software Architectures</title>
		<title level="s">Lecture Notes in Computer Science</title>
		<imprint>
			<biblScope unit="volume">2804</biblScope>
			<biblScope unit="page" from="25" to="43" />
		</imprint>
	</monogr>
</biblStruct>

				</listBibl>
			</div>
		</back>
	</text>
</TEI>
