The Interplay of Social and Robotics Theories in AGI Alignment: Navigating the Digital City Through Simulation-based Multi-Agent Systems

Abstract

This study delves into the task of aligning Artificial General Intelligence (AGI) and Large Language Models (LLMs) to societal and ethical norms by using theoretical frameworks derived from social science and robotics. The expansive adoption of AGI technologies magnifies the importance of aligning AGI with human values and ethical boundaries. This paper presents an innovative simulation-based approach, engaging autonomous 'digital citizens' within a multi-agent system simulation in a virtual city environment. The virtual city serves as a platform to examine systematic interactions and decision-making, leveraging various theories, notably, Social Simulation Theory, Theory of Reasoned Action, Multi-Agent System Theory, and Situated Action Theory. The aim of establishing this digital landscape is to create a fluid platform that enables our AI agents to engage in interactions and enact independent decisions, thereby recreating life-like situations. The LLMs, embodying the personas in this digital city, operate as the leading agents demonstrating substantial levels of autonomy. Despite the promising advantages of this approach, limitations primarily lie in the unpredictability of real-world social structures. This work aims to promote a deeper understanding of AGI dynamics and contribute to its future development, prioritizing the integration of diverse societal perspectives in the process.

Keywords

• eol>Artificial General Intelligence
• Large Language Models
• Social Theories
• Robotics Theories
• Simulation-Based Approach

1. Introduction

LLMs are trained on diverse internet text content. They

have demonstrated performance in a wide range of tasks The increasingly pervasive role of AI, especially natural and languages without any task-specific training [ 4 ], a language processing (NLP), signifies a new frontier of capability that resonates with the concept of artificial technological development. AI-driven applications like general intelligence (AGI).

Generative Pretrained Transformers (GPT) pioneer trans- AGI refers to a type of AI with cognitive capabilities formations across society [ 1 ]. As reliance on such AI that can successfully understand, learn, and implement systems rises, so does the challenge of adapting these intellectual tasks equivalent to those of a human being [ 5 ]. models to human values, prompting deeper research and Contrary to traditional AI that is limited to expert-level development. competence in specific tasks, AGI can understand, learn,

Despite rapid advancements, achieving full controlla- and adapt to any intellectual task that can be performed bility and value alignment with AI is a notable hurdle, by humans [ 5 ] The universality of this ability in AGI is especially with large-scale neural networks [ 2 ]. The rise often considered as both beneficial and dangerous. While of powerful AI models like GPT further amplifies con- it promises extensive progress and eficiency in virtually cerns about their ethical alignment, controllability, and all fields of life, it also imposes significant risks related unpredictability [ 3 ]. This pressure intensifies the explo- to misuse and unintended consequences. ration of better testing and mitigation strategies [ 1 ]. Aside from text generation, sophisticated Large Lan

Large Language Models (LLMs) are artificial intelli- guage Models (LLMs) also exhibit the capacity to simulate gence (AI) programs capable of language generation, understanding of inquiries and perform complex cognitranslation, question answering, summarization, and tive tasks [ 6 ]. Among numerous platforms, OpenAI’s code generation [ 4 ]. Unlike traditional AI models, which LLMs stand out due to their potential for fine-tuning, are trained on specific datasets and for particular tasks, making them compatible with a wide range of use-cases. BISEC’23: 14th International Conference on Business Information This adaptability sets the stage for their comprehensive Security, November 24, 2023, Niš, Serbia influence and application across diverse fields. OpenAI * Corresponding author. continues the development of Artificial General Intel$ [e-mail] (L. Bojić); [e-mail] ligence publicly while devising strategies that ensure (V. 0Ð0a0p0i-ć0)002-5371-7975 (L. Bojić); 0000-0002-8661-0269 (V. Ðapić) AGI’s safety and alignment with human values [ 7 ]. On © 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License the other hand, LLMs can be given various degrees of CPWrEooUrckReshdoinpgs IhStpN:/c1e6u1r3-w-0s.o7r3g ACttEribUutRion W4.0oInrtekrnsahtioonpal (PCCroBYce4.0e).dings (CEUR-WS.org) autonomy while creating multiple agents with diferent prompts capable of interacting with each other [ 8 ]. Theory of Reasoned Action is considerable. Stemming

AI Alignment represents the proposition of ensuring from the Computational Social Science spectrum, Social that the behavior of AGI system is congruent with human Simulation Theory leverages computational methods for intentions and values. As Bostrom [ 9 ] argues in his book simulating and analyzing social dynamics, thus driving "Superintelligence," it is incredibly challenging to specify tests for large language models and better aligning AI what is meant by human values in a way that an AI behavior to social norms [17, 18]. However, representing can understand. The alignment of AGI is considered the unpredictable nature of real-world social systems in crucial due to multiple reasons. The development of abstract computational models is a significant challenge, AGI might lead to an intelligence explosion where AGI limiting the theory’s accuracy and applicability [19]. surpasses human intelligence. If such a situation arises, The Theory of Reasoned Action, from social psycholit is important to ensure that AGI is beneficially aligned ogy, asserts that intentions drive behavior, influenced and promotes the interests of humanity [ 9 ]. Moreover, by attitudes towards the behavior, norms, and perceived poorly aligned AI could result in negative ramifications if control [20]. While originally for understanding human it can impact significant resources or make autonomous behavior, it can guide AI behavior modeling, influencdecisions. Hence, dedicated research is needed to ensure ing AI intentions via programmed norms and attitudes, that AGI development is carried out responsibly and with and helping align AI actions with societal values [21]. necessary precautions. However, the challenge lies in replicating the complex

AI and AGI advancements come with benefits, com- nature of human emotions and irrational behavior in AI, plexities, risks, and ethical challenges. With traditional emphasizing the need for a multifaceted AI alignment risk management methodologies proving inadequate, approach. there’s a shift towards exploring more multi-layered Asimov’s Laws of Robotics and The Uncanny Valley methodologies [ 10 ]. The unpredictability of AI and AGI Hypothesis ofer insights for AI security, concerning systems poses risks, underpinning the necessity of em- human-AI interactions [22]. Asimov’s Laws provide ethibedding human values and ethics into AI systems [ 3 ]. cal guidelines enhancing AI system’s controllability and Transparent, accountable AI systems developed with pub- ethical behavior. Yet, ambiguity in AI behavior complilic involvement are advocated by scholars like Véliz [ 11 ] cates adherence to these laws [23]. and Whittlestone et al. [ 12 ], leading to the democrati- The Uncanny Valley Hypothesis highlights the comzation of technology. The unification of social science fort of users with human-like AI, stressing careful design theories and technology ofers a promising path for de- to ensure secure AI usage [24]. Despite the theory’s veloping socially-responsible AI and AGI [ 13, 14 ]. cultural subjectivity, considering such perceptions aug

This paper delves into the potentials and challenges of ments holistic AI system design, balancing advancement AI and social robotics theory convergence for aligning with ethical responsibility and security. Multi-Agent AGI and LLMs. It explores theories and their applica- System Theory ofers valuable insights for developing tion in AI alignment, demonstrating their relevance in autonomous systems and testing LLMs and AGI. Multisimulation-based approaches within a digital city envi- agent systems of AI agents, each with unique attributes ronment. The paper concludes with reflections on limi- and decisions in a simulated digital city, can reveal emertations and directions for future research, essential for gent behavior and systemic strengths or weak points. ensuring AGI technologies are efective, secure, and up- Challenges, though, include agent synchronization, conhold societal values lfict resolution, and handling competition [ 25]. Despite these, the theory provides crucial support for AI testing in simulated environments. Situated Action Theory 2. Theoretical framework encourages adaptive, situation-driven behavior, enhancing AI responses to digital environments. This theory Exploring social science and robotics theories can provide implies AI models should adapt dynamically to changes critical insights for testing and aligning Large Language rather than sticking to prescribed actions. This approach Models (LLMs) and artificial general intelligence (AGI). equips AI to navigate unpredictability inherent in large The complexity of LLMs and AGIs demand a stringent, networks. theory-based approach [ 13 ]. Social science theories aid However, translating these concepts into AI programin understanding and predicting AI behavior [ 15 ], while ming proves challenging due to reality’s multidimenrobotics theories provide essential insights on machine sional and ambiguous nature. Designing adaptive beethics and multi-agent system operation for AGI design havior based on Situated Action Theory helps decipher and refining [ 16 ]. cognitive functions in simulated environments, paving

Incorporating social science theories in AI research the way for advanced, reliable AI systems. grants a lens for understanding AI alignment and be- Next, we examine the practical implementation of havior. The relevance of Social Simulation Theory and these theories for AGI, focusing on developing a digital city. Subsequent section will reflect on the simulation’s ing as AI actors, these agents vary in personality, norms, results, ofering insights for alignment of AI models. and behaviors, enriching the simulation’s scenarios and insights. Autonomy, or the capacity to act independently, is critical for AI agents’ value and efectiveness [40]. 3. Towards simulation of a digital Various learning models, such as reinforcement learncity ing, are utilized for shaping digital citizens [41]. Interaction and responsiveness to their environment, other AI A simulation-based methodology enhances the reliability, agents, and external inputs is paramount [42]. Personeficacy, and safety of Large Language Models (LLMs) in ified digital citizens, complete with autonomy, natural AGI development [26]. The authors note that simula- language-processing capabilities, character traits, and tions provide controlled settings for testing AI behaviors unique behaviors, significantly enhance multi-agent simunder various scenarios. This digital city simulation, in- ulations [43]. Such enhancement underpins our objecspired by McEwan et al. [27], efectively mimics real-life tives for AGI development [ 9 ]. complex interactions in a controlled setting. As such, Our digital environment’s richness allows observation these tested procedures have become instrumental in and manipulation of variables influencing AI behavior, AGI development. with significant emphasis on interactions and decision

In this research, a virtual reality framework adds a making of digital citizens [44]. Interactions and decisions potent and immersive dimension to simulation studies, form the crux of our simulation, driving insights into AI a paradigm gaining wider acceptance [28]. Enhanced behavior under various scenarios. with AI, this approach ofers opportunities for in-depth Interactions can range from simple exchanges to conanalysis of AI interactions in realistic scenarios [29]. lfict resolutions and cooperative tasks [ 45]. Decision

By incorporating virtual reality, we tap into a broader making forms a crucial part of an autonomous agent’s context for AI implementation. Lending support to function, stretching from simple choices to complex Bolton et al. [30], the creation of a ’digital twin’ or ’mirror trade-ofs [ 41]. These interactions and decisions provide world’ facilitates dynamic AI learning. It triples as a plat- data useful in refining AI models and informing digital form for appreciating AI behaviors, an arena for future technology policies [46]. Our simulation-based approach social sciences research, and a toolkit for understanding provides invaluable insights for AGI and influences its social dynamics [31]. use [47]. The immersive environment ofers simulations

A simulation-based approach as noted by Bostrom & of significant clinical, social, and psychological interYudkowsky [32], enhances the evaluation of AI, espe- est [ 48 ]. These understandings, extending beyond AGI cially LLMs behavior. This methodology, bolstered by a performance, help anticipate and shape AGI’s potential virtual reality dimension, holds potential to remarkable societal impact [ 49 ]. breakthroughs in AGI understanding and enhancement. Data from the digital city facilitates bias addressing

Automated simulations for LLMs form the cornerstone in AGI systems [ 50 ]. Areas like autonomous vehicles, of our approach, ofering reproducible, scalable, and com- robotics, customer service, and translation would gain plex interactive environments [33]. Our digital city em- from information acquired in the digital city environploys a multi-agent-based simulation framework, mod- ment [ 51 ]. The virtual city also underlines the ethical eling a population of autonomous AI agents or ’digital considerations and value alignment issues concerning citizens’ [34]. Heath et al. [35] afirm the efectiveness AGI [ 9 ]. The use of a simulation approach in a digital of such agent-based models in understanding complex city enriches understanding of AGI dynamics, helping environments. society harness AGI innovations responsibly.

The development of this digital realm involves iterative Aligning AI models with human values is critical, escreation of autonomous agents operating within defined pecially in AGI, which has the potential to mimic humanparameter spaces [36]. Their autonomy determines their like reasoning, including ethical decision-making [ 9 ]. Obdynamics within the city [37]. A meticulously designed servations from interactions within our simulated apenvironment, where the AI agents function, necessitates proach assist in identifying and rectifying AGI’s anomaa thorough attention to interactions, constraints, and lies and misalignments. choices [38]. Continuity in learning behavior and re- Understanding how models encode knowledge is cruifnement of AI agents are ensured by a reinforcement cial for AI alignment [ 52 ]. Our simulation-based testing learning approach, as proposed by Leike et al. [ 2 ]. The ofers insights into AI’s cognitive understanding, givcreation of these simulations significantly influences the ing a better overview of its decision-making processes. lockdown approach’s efectiveness in providing real-life Decision-making in AGI leverages reinforcement learnscenario-based insights for AGI. ing, but it requires careful management to avoid endors

Describing the digital citizens, Bartneck et al. [39] un- ing undesired behaviors [46]. derscore their importance in our simulation strategy. Act- The behaviors and interactions of digital citizens within our simulation ofer rich data for AGI refinement [ 53 ]. This scenario-based data aids in developing safety measures, aligning AGI with human values, and mitigating the risks of AI integration into society. Consequently, this enables the creation of safer, controlled, and valuealigned AI systems.

4. Conclusion

AI growth necessitates innovative security solutions and alignment with human values. Through contriving a digital city with digital citizens, various societal interactions can be explored to gain insights into AI behavior. Key theories guiding our approach include social simulation and theory of reasoned action for studying AI behavior in social contexts. Robotics theories illuminate ethical considerations, informed by Asimov’s Laws of Robotics and the Uncanny Valley Hypothesis.

The application of Multi-Agent System Theory and Situated Action Theory helps manage AI behaviors, guiding interactions, and environment-response adaptations. This accentuates AI alignment with desired outcomes despite potential challenges. Our approach highlights automated simulations for exhaustive study of AI behavior. Autonomous citizens’ interactions provide rich data for understanding autonomy, crucial for AGI refinement and broader societal applications. Simulations also help design value-aligned AGIs. However, challenges exist with theory application to AI programming and replicating real-world efects. Nevertheless, simulation-based approaches show promise for aligning AI with human values, despite complexities.

Our approach also has limitations, primarily the dificulty in replicating complexities of real societies within a digital space. Translating theoretical concepts into AI programming presents additional challenges. Biases in AI models can be perpetuated from training environments, and defining "desirable" behavior for AI alignment proves complex.

Future research can enhance simulation realism using advanced VR and AR technology. Focus should also be on refining theory integration into AI programming and developing automated bias correction frameworks. There’s also the need to build definitions of AI alignment that respect the dynamism of values across cultures. This research is a starting point for harnessing theories and simulation-based approaches towards value-aligned AGI.

Acknowledgment

This paper monograph was realised with the support of the Ministry of Science, Technological Development and Innovation of the Republic of Serbia, according to the

Agreement on the realisation and financing of scientific research.

[17] C. Ciofi-Revilla, Introduction to computational so- security, Chapman and Hall/CRC, 2018, pp. 57–69.

cial science, Springer, 2014. [33] J. Banks, Discrete event system simulation, Pearson [18] M. W. Macy, R. Willer, From factors to actors: Com- Education India, 2005.

putational sociology and agent-based modeling, An- [34] E. E. Bertacchini, G. Jakob, E. Vallino, et al., Emernual review of sociology 28 (2002) 143–166. gence and evolution of property rights. an agent[19] B. Edmonds, S. Moss, From kiss to kids–an ‘anti- based perspective, WORKING PAPER SERIES 40 simplistic’modelling approach, in: International (2013). workshop on multi-agent systems and agent-based [35] B. Heath, R. Hill, F. Ciarallo, A survey of agentsimulation, Springer, 2004, pp. 130–144. based modeling practices (january 1998 to july [20] M. Fishbein, I. Ajzen, Predicting and changing be- 2008), Journal of Artificial Societies and Social Simhavior: The reasoned action approach, Taylor & ulation 12 (2009) 9.

Francis, 2011. [36] D. Silver, A. Huang, C. J. Maddison, A. Guez, [21] P. Sheeran, T. L. Webb, The intention–behavior L. Sifre, G. Van Den Driessche, J. Schrittwieser, gap, Social and personality psychology compass 10 I. Antonoglou, V. Panneershelvam, M. Lanctot, et al., (2016) 503–518. Mastering the game of go with deep neural net[22] I. Asimov, I, Robot., New York: Gnome Press., 1950. works and tree search, nature 529 (2016) 484–489. [23] J. J. Bryson, Robots should be slaves, Close En- [37] J. Lehman, J. Clune, D. Misevic, C. Adami, L. Algagements with Artificial Companions: Key social, tenberg, J. Beaulieu, P. J. Bentley, S. Bernard, psychological, ethical and design issues 8 (2010) G. Beslon, D. M. Bryson, et al., The surprising 63–74. creativity of digital evolution: A collection of anec[24] M. Mori, The uncanny valley, Energy 7 (1970) dotes from the evolutionary computation and ar33–35. tificial life research communities, Artificial life 26 [25] G. Weiss, Multiagent systems: a modern approach (2020) 274–306.

to distributed artificial intelligence, MIT press, 1999. [38] R. S. Olson, A. Hintze, F. C. Dyer, D. B. Knoester, [26] S. D. T. Kelly, N. K. Suryadevara, S. C. Mukhopad- C. Adami, Predator confusion is suficient to evolve hyay, Towards the implementation of iot for envi- swarming behaviour, Journal of The Royal Society ronmental condition monitoring in homes, IEEE Interface 10 (2013) 20130305.

sensors journal 13 (2013) 3846–3853. [39] C. Bartneck, D. Kulić, E. Croft, S. Zoghbi, Mea[27] G. F. McEwan, M. L. Groner, M. D. Fast, G. Gettinby, surement instruments for the anthropomorphism, C. W. Revie, Using agent-based modelling to predict animacy, likeability, perceived intelligence, and perthe role of wild refugia in the evolution of resistance ceived safety of robots, International journal of of sea lice to chemotherapeutants, PLoS One 10 social robotics 1 (2009) 71–81.

(2015) e0139128. [40] P. Stone, M. Veloso, Multiagent systems: A survey [28] R. C. A. Barrett, R. Poe, J. W. O’Camb, C. Woodruf, from a machine learning perspective, Autonomous S. M. Harrison, K. Dolguikh, C. Chuong, A. D. Robots 8 (2000) 345–383.

Klassen, R. Zhang, R. B. Joseph, et al., Comparing [41] R. S. Sutton, A. G. Barto, Reinforcement learning: virtual reality, desktop-based 3d, and 2d versions of An introduction, MIT press, 2018. a category learning experiment, Plos one 17 (2022) [42] S. Zhang, E. Dinan, J. Urbanek, A. Szlam, D. Kiela, e0275119. J. Weston, Personalizing dialogue agents: I have [29] J. Vora, S. Nair, A. K. Gramopadhye, A. T. a dog, do you have pets too?, arXiv preprint Duchowski, B. J. Melloy, B. Kanki, Using virtual re- arXiv:1801.07243 (2018). ality technology for aircraft visual inspection train- [43] J. Z. Leibo, V. Zambaldi, M. Lanctot, J. Marecki, ing: presence and comparison studies, Applied T. Graepel, Multi-agent reinforcement learning ergonomics 33 (2002) 559–570. in sequential social dilemmas, arXiv preprint [30] R. N. Bolton, J. R. McColl-Kennedy, L. Cheung, arXiv:1702.03037 (2017).

A. Gallan, C. Orsingher, L. Witell, M. Zaki, Cus- [44] A. M. Turing, Computing machinery and intellitomer experience challenges: bringing together dig- gence, mind 59 (1950) 433–460. ital, physical and social realms, Journal of service [45] N. R. Jennings, K. Sycara, M. Wooldridge, A management 29 (2018) 776–808. roadmap of agent research and development, Au[31] L. Bojic, Metaverse through the prism of power and tonomous agents and multi-agent systems 1 (1998) addiction: what will happen when the virtual world 7–38. becomes more attractive than reality?, European [46] D. Amodei, C. Olah, J. Steinhardt, P. Christiano, Journal of Futures Research 10 (2022) 1–24. J. Schulman, D. Mané, Concrete problems in ai [32] N. Bostrom, E. Yudkowsky, The ethics of artificial safety, arXiv preprint arXiv:1606.06565 (2016). intelligence, in: Artificial intelligence safety and [47] Y. Shoham, R. Perrault, E. Brynjolfsson, J. Clark,

References

[1] M. X. Chen , O. Firat , A. Bapna , M. Johnson , W. Macherey, G. Foster , L. Jones , N. Parmar , M. Schuster , Z. Chen , et al., The best of both worlds: Combining recent advances in neural machine translation , arXiv preprint arXiv: 1804 . 09849 ( 2018 ).

[2] J. Leike , M. Martic , V. Krakovna , P. A. Ortega , T. Everitt , A. Lefrancq , L. Orseau , S. Legg, Ai safety gridworlds , arXiv preprint arXiv:1711.09883 ( 2017 ).

[3] G. Irving , A. Askell , Ai safety needs social scientists , Distill 4 ( 2019 ) e14 .

[4] A. Radford , J. Wu , R. Child , D. Luan , D. Amodei , I. Sutskever , et al., Language models are unsupervised multitask learners , OpenAI blog 1 ( 2019 ) 9 .

[5] Gartner , Definition of Artificial general intelligence (AGI) , https://www.gartner. com/en/information-technology/glossary/ artificial-general - intelligence-agi, 2023 .

[6] G. Sartori , G. Orrù, Language models and psychological sciences , Frontiers in Psychology 14 ( 2023 ).

[7] S. Altman , Planning for agi and beyond , OpenAI Blog , February ( 2023 ).

[8] B. Lutkevich , Auto-GPT, https://www.techtarget. com/whatis/definition/Auto-GPT, 2023 .

[9] T. Mulgan , Superintelligence: Paths, dangers, strategies, 2016 .

[10] S. J. Russell , P. Norvig , Artificial intelligence a modern approach , London, 2010 .

[11] B. Rumbold , Privacy is power: Why and how you should take back control of your data, written by carissa véliz , Journal of Moral Philosophy 20 ( 2023 ) 585 - 587 .

[12] J. Whittlestone , R. Nyrup , A. Alexandrova , K. Dihal , S. Cave , Ethical and societal implications of algorithms, data, and artificial intelligence: a roadmap for research , London: Nufield Foundation ( 2019 ).

[13] I. Rafols , Knowledge integration and difusion: Measures and mapping of diversity and coherence , in: Measuring scholarly impact: Methods and practice , Springer, 2014 , pp. 169 - 190 .

[14] S. Cave , S. S. ÓhÉigeartaigh , Bridging near-and long-term concerns about ai , Nature Machine Intelligence 1 ( 2019 ) 5 - 6 .

[15] E. Ostrom , A general framework for analyzing sustainability of social-ecological systems , Science 325 ( 2009 ) 419 - 422 .

[16] P. Lin , K. Abney , G. A. Bekey , Robot ethics: the ethical and social implications of robotics , MIT press, 2014 . J. Manyika , J. C. Niebles , J. T. Etchemendy , et al., The ai index 2018 annual report - ai index steering committee, human-centered ai initiative , 2018 .

[48] C. Castelfranchi , Artificial liars: Why computers will (necessarily) deceive us and each other , Ethics and Information Technology 2 ( 2000 ) 113 - 119 .

[49] V. Mnih , K. Kavukcuoglu , D. Silver , A. A. Rusu , J. Veness , M. G. Bellemare , A. Graves , M. Riedmiller , A. K. Fidjeland , G. Ostrovski , et al., Human-level control through deep reinforcement learning , nature 518 ( 2015 ) 529 - 533 .

[50] A. Caliskan , J. J. Bryson , A. Narayanan , Semantics derived automatically from language corpora contain human-like biases , Science 356 ( 2017 ) 183 - 186 .

[51] A. Esteva , A. Robicquet , B. Ramsundar , V. Kuleshov , M. DePristo, K. Chou , C. Cui , G. Corrado, S. Thrun , J. Dean , A guide to deep learning in healthcare , Nature medicine 25 ( 2019 ) 24 - 29 .

[52] M. Ring , L. Orseau, Delusion, survival, and intelligent agents , in: Artificial General Intelligence: 4th International Conference, AGI 2011 , Mountain View, CA, USA, August 3- 6 , 2011 . Proceedings 4, Springer, 2011 , pp. 11 - 20 .

[53] P. F. Christiano , J. Leike , T. Brown, M. Martic, S. Legg , D. Amodei , Deep reinforcement learning from human preferences , Advances in neural information processing systems 30 ( 2017 ).