An open source perspective on AI and alignment with the EU AI Act

Abstract

Artificial intelligence systems based on deep learning have increasingly received interest due to their success in complex human tasks. A current trend in deep learning is to study how algorithms learn multiple new abilities as their size and training data increase. "General purpose AI" (GPAI), that is systems that can transfer the acquired knowledge to solve multiple tasks, are candidate to constitute the backbone of many AI algorithms applied in specific fields on industry, e.g. healthcare, customer support, administration. While various research laboratories express safety concerns on GPAI and do not openly share access to their algorithms, others advocate for their "democratization" and an increasing amount of open-source versions is available online. In this study we analyze this phenomenon from two perspectives and try to reconcile them. From one side, research communities support open collaborations, free access to knowledge and resources; on the other, political institutions, involved in the orchestration between the support for innovation and the control of societal impact, aim at preventing violations of fundamental human rights. We particularly focus on the European approach for risk assessment of AI systems. In our opinion, it greatly overlaps with work in ethics and law conducted by AI researchers (e.g. the Stanford Centre for Research on Foundation Models). Specifically we identify some necessary modifications to improve coordination between the two sides, while also discussing viable implementations in the technical field.

Keywords

• eol>open source AI
• general purpose AI
• European AI regulation
• AI social impact
• technological standards

1. Introduction

Fast-paced progress in deep learning research is currently followed by the proliferation of AI software to generate art [ 1 ], music [ 2 ] or to follow instructions for textual tasks [ 3 ]. Open access to these tools is favored by research organizations and communities (LAION, HuggingFace, EleutherAI), which are receiving increasing attention and funding for collaborative research. The pervasiveness of AI systems in society is unmatched by the progress of lawmakers in assessing their societal implications, alignment with fundamental human rights and safe development.

General-purpose AI. Originally, AI systems based on machine and deep learning were tuned to perform specific tasks ("fixed-purpose systems"[ 4 ]) but achieved poor results in others. More recently, research in natural language processing intersected with deep learning revealed the potential of language models, trained in specific tasks such as predicting the next word in a sentence [ 5 ][ 6 ] or filling the blanks [ 7 ], in solving more abstract problems such as processing common sense questions, solving simple maths or identifying patterns from scarce contextual information (in-context learning).

The ability of these systems to transfer knowledge and capabilities to tasks unseen at training time defines them as general purpose AI, as also stated in Aguirre et al. [ 4 ].

Foundation models. Bommasani et al. [ 8 ] analyze the implications of adopting GPAI as the backbone for many specialized systems, defined also as "foundation models". Transfer learning is the process of adaption of an already-trained AI to a new task with either further training (fine-tuning ), few examples (few-shot learning) or none (zero-shot learning). The majority of such systems has been originally applied to human language (Language Models), but applications in computer vision [ 9 ] and biology [ 10 ] resulted successful as well. Using such models as base for more specialized AI is defined in Bommasani et al. [ 8 ] as the process of homogenization: it could ease control and development with the risk, however, of propagating bias from design/data to all the downstream applications. organized in communities) on large-scale projects. This intelligence [ 12 ] focuses on the impact of AI systems principle has been recently transferred to AI research: in and it introduces a taxonomy of risks: unacceptable communities such as EleutherAI, OpenBioML, LAION, risks AI (enumerated fields of application, e.g. social openly accessible research projects are coordinated scoring, exploitation of social groups), high-risk AI by community members and AI models, datasets and (closed list with requirements, e.g. aircraft systems), development tools are publicly released. limited risk AI (open list, optional requirements), Open source also means free access to multimedia minimal risk AI. The proposal adopts a general definformation and knowledge: in the field of AI, free inition to cover a wide range of AI systems beyond courses on public streaming platforms or blogs are technical specifications. The regulation applies to AI made available by academic institutions such as MIT systems (including open source AI) that are put in OpenCourseWare, companies such as HuggingFace or the European market, so AI developed for the sole organizations such as FastAI. For open AI development, purpose of research is not afected by. On 14th June these online collections include documented software 2023, the European Parliament adopted its negotiating frameworks for beginners to start new AI projects. position on the Artificial Intelligence (AI) Act with 499 In the matter of safety and conditions of use, software votes in favour, 28 against and 93 abstentions ahead of developers adopt open-source licenses that vary in the talks with EU member states on the final shape of the law. level of freedom for multiple dimensions: modification, use, re-distribution, ownership, use case restrictions Council of the EU. On the 6th of December 2022, and downstream licensing. In the field of AI, policies the Council of the EU adopted the common position on related to these factors of use are heterogeneous among the EU AI Act [ 13 ], several changes were introduced to public and private entities: main companies such as the proposal. The requirements set out for high risk AI OpenAI have defined and enforced policies of use for systems in the EU AI Act are applicable to GPAI systheir models[ 11 ] (such as ChatGPT, GPT-4), which tems as a result of a risk assessment procedure described can be used by researchers and customers. However, in a specific implementing act. As originally proposed, this approach does not reflect the philosophy of high risk AI systems are required to undergo a conforopen source: no free access to the AI models’ code, mity assessment procedure followed by the emission of parameters and data documentation is given, precluding a certificate. Requirements for conformity include for opportunities for the verification of the capabilities instance: a constantly updated technical documentation, of these systems and open discussion on safety measures. a risk management system, an informative of use with reported assessment of robustness and security of the

Open research collectives. Tech companies with system, record-keeping of the operations of the AI. abundant resources were first in developing increasingly Similarly, Bommasani et al. [ 8 ] identify the source of larger language models. With respect to its predecessors harm of AI systems in development and training, where GPT-1 and GPT-2, GPT-3 (OpenAI) [ 7 ] has not been intrinsic bias, (e.g. patterns in data of toxic behavior and publicly released: on-demand remote access to use the associations with social groups), translates into extrinsic AI is restricted by OpenAI with paid options. Part of the harm (e.g. under-representation or misrepresentation, AI research community discussed whether discretion the generation of fake information or hate content). In to access to such powerful algorithms should be left our opinion, this well aligns with the view of the Euroto single private companies, it is arguable that this pean proposal, but the fields where the latter applies are choice could be also motivated by the non-negligible too strictly defined, potentially limiting its efectiveness. cost to train these models. Nonetheless, open-source AI research and industrial applications greatly overlap: versions of large language models started to emerge large language models developed with data and techamong research collectives: the Big Science Project is niques in research can become market products (e.g. GPTa global workshop along the lines of CERN or LHC, 3, GitHub CoPilot). Consequently, procedures to ensure it promotes joint collaboration on training open large transparency, safety and quality of systems should take language models applicable in science. Moreover, known part in the research and development process, as well academic conferences, such as NeurIPS, expect paper as they should not hamper them. We believe a potensubmissions to be accompanied with publicly released tial solution lies in research communities, where a unicode; "grassroot" research collectives such as EleutherAI fied collection of datasets and tools is open for testing (AI) or OpenBioML (computational biology), are funded and improvement. This allows to create a development by technical partners advocating for openness, such as framework that is widely adopted by researchers and if HuggingFace or StabilityAI. developed in synergy with political institutions, it could also ensure safety and alignment.

The EU AI Act. The European proposal for a Regulation laying down harmonised rules on artificial

2. The impact of open source AI

In this section we discuss the impacts generated by the

use of open source AI systems, for each field we suggest the behaviour to be maintained from the perspective of both users or providers. Our opinion reflects the necessity to develop open, safe, transparent and eficient AI.

2.1. Social impact.

Principal issues regard the misuse of private information

and ensuring fundamental rights from current law, that is right to ensure anonymity, the right to erasure, the right to be informed (and to know the interlocutor’s nature), the right to object and the right to access. Further critical issues regard liability for the model’s predictions: GPAI can be applied to an unpredictable amount of tasks. Given the consistent advantage in processing large amounts of historical data, model predictions may be overestimated. Moreover, since the mechanisms that drive such systems are in most cases not explainable or dificult to interpret, AI-assisted decisions can be dificult to understand. For instance, applying AI to healthcare and bio-medicine can bring remarkable advantages, although the problem of liability needs to be addressed as these models are not free of bias, and even in the optimal case they can have unacceptable margins of error [ 23 ].

Determining in advance a comprehensive list of social

implications of AI is challenging. Pre-trained models describe a recent paradigm shift and demonstrate remarkable capabilities of generalisation beyond training information. Before discussing social impacts, we must comprehend when such models represent some harm.

We believe that foundation models could have direct social impact when they are applied in public services or deployed in the market as products. Foundation models have the potential to be dangerous if they Inequities and malicious uses. Producing a dataset have been trained or adapted on questionable data. without a lack of diversity and biases has been considEspecially because such models can inherit biases from ered an endpoint for a while. Nonetheless, when we deal toxic information [ 14 ]. Disparities in the performance with a massive quantity of data, it is not straightforward of gender classification [ 15 ] and facial recognition to filter out toxic content [ 14 ]. Recently, LAION has resystems [ 16 ] have been found due to incomplete data. leased an open-source novel dataset consisting of 5.85 Additionally, it has been proved [ 17 ] that in large billions of image—text pairs [ 24 ], in which images that language models there are exploitable vulnerabilities. appear to contain harmful content have been removed. Nevertheless, these AI systems already brought tangible However, they briefly explained that even if they imbenefits in several fields. For instance, they could prove the overall safety of the data, there is no guarantee provide accurate medical diagnoses at a low charge that the generated dataset is completely safe. Evidence [ 18 ]. This could also give people living in areas afected showed that models trained on poor/biased data exhibit by severe poverty, the opportunity to access medical unexpected behaviours and tend to discriminate against treatment. Furthermore, these models with generative a few marginal social groups, by inherithing stereotypes. abilities could provide support to researchers working Moreover, Weidinger et al. [ 25 ], point out that not all on discovering new therapies to treat people [ 19 ]. In our social groups may have access to the services provided opinion, foundation models have shown potential for by these foundation models at the same quality. For inapplications in education: an AI can deliver lessons to stance, research on large language models in English students in a much more interactive way, depending and Chinese languages is extremely active, but it’s quite on the demands of the interlocutor, like a private tutor. diferent for some other languages spoken by a smaller Thus, it would be possible to grant everyone a low-cost amount of people. If the access to these services precustom education. Controlling AI development in a determined a competitive advantage, it would increase risk-based approach, allows to weight potential harms inequality between diferent social groups. Koenecke et with expected benefits, given a sound and transparent al. [ 26 ], found that speech recognition systems perform development procedure. better with white American English speakers rather than African American English speakers. Moreover, these gen

Law enforcement. Training large language models erative models are able to produce low-cost high-quality requires considerable amounts of data, which are usually content. This may facilitate disinformation campaigns web-scraped. The absence of a data filtering procedure and the dissemination of false information as outlined in prior to model fitting can lead to legal issues: models Goldstein et al.[27]: users might be over-relying on the AI put on the market such as GitHub CoPilot (based on system, unaware of biased model results. In fact language GPT-3) have shown to generate licensed software code models can replicate common misconceptions or they [ 20 ] [ 21 ]. Data curation is subject to existing regulations: can make unfounded statements [28]. Moreover, they The General Data Protection Regulation (Regulation could also suggest illegal activities rather than discourage (EU) 2016/679) (hereinafter “GDPR") and the California them; recent work in AI safety has focused on adapting Consumer Privacy Act of 2018 (hereinafter “CCPA”) [ 22 ]. generated text with human feedback [29]. Bai et. al. [29], have conducted some experiments on adapting the model for innovation in terms of costs, but at the current state to identify harmful outputs by revising its response with- of the regulation, specifications for general-purpose AI out human feedback. Furthermore, a model trained on systems are partially defined. The ALLAI organisation people’s private information may correctly infer sensible published some in-depth studies on the EU AI Act data of a person and use it in inappropriate contexts. In [32]: they point out what could actually hamper the this specific case, this might conflict with data privacy, use of such systems. By the Article 8(2), the provider non-discrimination, fairness and other ethical principles of an AI system is obliged to take into account its [30]. intended purpose when adhering to the requirements, but the ’intended purpose’, as defined in Article 3(12), is 2.2. Economics dificult to identify for foundation models due to their intrinsic nature. This way the burden of proving the compliance of those systems with the EU AI Act falls on "downstream users", resulting in stifling innovation for SMEs and micro enterprises that cannot handle the obligations. The solution ALLAI proposes is the definition of ’reasonably foreseeable use’, alongside the ’intended purpose’, that is "the use of an AI system in a way that is or should be reasonably foreseeable".

Open source AI ofers a great opportunity in terms of digitisation for European enterprises. It has been proven during the last decades that open strategies result in eficient normalisation of new technologies. Notably, when speaking of foundation models, whose field of application is very broad, the implementation of open source AI systems would be an important step towards a zero-cost digital innovation of the public sector.

Conversely, closed source systems are currently leading Work-force transformation. It is almost certain all over Europe, establishing a leading minority of tech that the introduction of AI in the European market will giants that hampers Small and Medium Enterprises’ have a significant impact on a wide range of fields of (SMEs) growth. The following paragraphs describe the occupation, e.g. foundation models applied on specific economical impact of the use of open-source AI systems. tasks could replace human workers due to a lower resource cost [ 25 ]. Moreover, they should be considered, as economists defined it, a form of general-purpose technology, that a new method of producing and inventing that is important enough to have a protracted aggregate impact. Foundations models, thanks to adaptation strategies like fine-tuning and prompting, might be able to solve a considerably large number of problems much more accurately than humans or even tasks that humans cannot perform. As a matter of fact, general-purpose AI systems, are released open-source online at no cost for the majority. Therefore this trend may lead to a significant shift in the labour market due to the fact that those models:

Digital Innovation. It is probably the greatest achievable benefit from the use of open foundation models. It is estimated that three in five European companies lack digitalisation [31], and that around most of SMEs lag behind in technological innovation because of the high costs. Foundation models can handle a several amount of diferent tasks, but only the writing skill will be considered for the following example. Most of the jobs have at least one secondary task in which writing is involved, leaving that part to an AI system would make the process much quicker, resulting in higher productivity in terms of produced outputs. The EU AI Act [ 12 ], in accordance to the Council of the EU position, defines and labels general purpose AI systems as a separate family of systems, outside the risk-based labeling approach (art. 4a, 4b, 4c). General-purpose systems that should be classified as high-risk systems due to their field of application (art.6, Annex III) are not necessarily open, therefore the exact cost of innovation is not fully predictable. Articles 53, 54 and 55 focus on measures for innovation of tech industries that will develop AI, through regulatory sandboxes and a priority system based on the enterprise scale. Nonetheless, a regulation to encourage the use of open systems applied to the public sector, especially to the fields listed in Annex III, is crucial to ensure faster and stronger innovation, even for non-tech companies.

Mitigating SMEs’ burdens. As stated in the previous paragraph, the uptake of foundation models is critical

1. perform functions with a zero marginal cost [33];

2. might increase productivity and profit; 3. achieve human-level performance.

Automation will replace and reshape millions of jobs, for instance: worker-less factories are fully automated factories empowered by automated systems [34]. In the mid-term, even knowledge workers like radiologists might be replaced by extremely precise AI systems [35]. To summarise, companies will need a novel work-force that masters the latest emerging skills. Thus, it is not about losing jobs, instead we are introducing a work-force transformation. It has been argued [36] that this process does result in employment increase.

Decentralization of power. As previously stated, some bigger AI providers hold power over smaller ones.

In the development of AI systems, power is defined by the control over data and models. Arguably the European harmful content, inconsistent performance depending market on AI systems is characterized by the oligopoly on the user. The EU AI Act addresses such issues with of tech giants, which overrule even on computational a series of requirements and procedures: Article 10 power. Indeed, open technologies introduce the possibil- concerns procedures for data governance in line with the ity to compete against those big companies, developing European law, including the GDPR; Article 15 introduces a broader, and less biased, community of experts. Open accuracy and robustness levels to be met for high-risk source AI in general provides better cybersecurity and AI, although strict compliance is not required, according transparency, but from an economical point of view, it to a recent review by the Council of the EU, as unified results in an eficient decentralization of power, which metrics are dificult to formalize. To summarize, Chapter in turn leads to a potential improvement in the public 2 of the EU AI Act [ 12 ] introduces standards, which have sector due to costs reduction and security enhancement. to be harmonised in compliance with Art. 40.

3. Risks in Foundation Models & the EU AI Act

Several principles for ethical development of AI systems, and specifically on foundation models, have been set out by Bommasani et al. [ 8 ]. The object of their paper is to describe risks and opportunities that foundation models bring with them. In this section we present how those highlighted issues are tackled in the EU AI Act [ 12 ], and for each of them we discuss our opinion.

3.1. Capabilities

Intrinsic bias & transparency. Due to the ingent amount of information processed by AI systems, contaminated data can introduce critical issues such as social under-representation (e.g. incoherent quality of medical diagnoses for members of a particular minority) or mis-representation (e.g. hate speech or inappropriate associations). In Article 11, the EU AI Act [ 12 ] introduces the obligation to write technical documentation, reporting architectural design choices, data pre-processing and task-specific model adaption. Further measures may consist in a more socially inclusive evaluation procedure with metrics developed to specifically measure bias and toxicity. Model Cards (Google [38]) provide with a template completely in line with requirements set out by the EU AI Act in Article 13, and further ensure the interpretability of AI systems. This level of documentation should be mandatory for open source AI systems since "Model cards also disclose the context under which models are intended to be used, details of the performance evaluation procedures"[38], which is crucial information when referring to foundation models.

Foundation models exhibited surprising capabilities in

language, vision, robotics, reasoning and search, user interaction in lots of downstream tasks [37]. The EU AI Act [ 12 ] does not consider general-purpose AI systems, although AI systems are divided following a risk-based approach, Articles 6, 7 establish some classification rules for high-risk systems. In particular Art. 7(2)(a) specifies that whenever a new high-risk area has to be assessed by the Commission, the intended purpose of the AI system must be taken in consideration. Article 3(12) defines intended purpose: "Intended purpose means the use for which an AI system is intended by the provider, including the specific context and conditions of use, [...]". By this definition, whatever the capabilities of the system, risk has to be assessed a posteriori by the Commission, based on the declared usage.

Legality & liability issues. There are three legality issues that stand out: accountability of AI system providers, output liability, and eventually copyright.

Given the broad foundation models’ field of application, it is probable that a defection of an AI system might harm people. In order to safeguard the right to an efective remedy and to a fair trial, the provider of the defective AI system should be traceable by the users and should be accountable for the damage caused by

Technology standards. Foundation models reach his product. Art. 62 of the EU AI Act [ 12 ] tackles this impressive performance across multiple downstream accountability requirement implying the obligation for tasks, through internet-size data. Hence, some consider- providers of high-risk AI systems to produce reports of ations about data: documentation, access, visualisation, serious incidents and malfunctionings. Finally, copyright curation and selection. Harmful behavior in AI models infringements are considered. They are composed of ifnd origin in the design process of the model and in two major issues, copyright protected data contained in data curation processes. Bommasani et al. [ 8 ] define the datasets, and copyright of model output. In the final the development of an AI system as a sequence of version of the AI Act there will be a separated section on defined stages, ranging from data processing to security Generative AI, or models capable of generating content assessments and deployment. Such standardised design such as images or text. Providers of such models will procedure tackles a variety of issues concerning AI: require to publish summaries of copyrighted data used robustness to adversarial attacks, the generation of for training, and to disclose that the content is generated

Auditing. AI systems should be periodically tested

against possible issues and shifts in data distributions. In order to fulfil these requirements a protocol for periodic testing should be defined, alongside a controlled environment for safe testing. The EU AI Act [ 12 ] handles this issue in two diferent steps. Title V, and in particular Art. 53, defines "AI Regulatory Sandboxes" to safely test high-risk AI systems before entering the market, this is also seen as an innovation enhancement tool, since it gives priority to "small scale" industries. Title VIII deifnes a post-market monitoring system, in which the Market Surveillance Authority watches over the European market for systems that present new risks, or break the requirements set by the EU AI Act. Wherever a provider establishes a causal link between an AI system and a malfunctioning, the provider is obliged to notify it within a period of 15 days. Notwithstanding the power given to the Market Surveillance Authority (Chapter 3 of EU AI Act), it acts only after notifications, drastically slowing down the process of law enforcement. However Art. 65 states that whenever a provider does not apply corrective actions within a reasonable period, commensurate with the nature of the risk, the "Market Surveillance Authority shall take all appropriate provisional measures to prohibit or restrict the AI system’s being made available on its national market, to withdraw the product from that market or to recall it".

4. Changes to the EU AI Act

Most of the changes that we wanted to bring into the

regulation have been already proposed by the European Council [ 13 ]. Nonetheless, we report ourselves a list of further additions to the EU AI Act to align it to an open-source perspective of AI development. Title 1a of European Council’s proposal is a collection of three articles concerning requirements and obligations for General Purpose AI Systems. However, we strongly believe that AI is not a product, but technology to build products. Thus, in Art. 55b we introduce "AIbased general-purpose technology", with the attempt of detaching the concept of product from General Purpose AI, which is a method of producing, see Art. 3(2a). Regarding the addition to ANNEX IV, we noted that data documentation for high-risk AI systems is mandatory only "where relevant" (ANNEX IV(2)(d)), thus we decided to strengthen the requirement for those systems whose use is related to individuals. It is possible to detect the presence of natural persons in the intended purpose of the AI system, because its documentation must include its context and condition of use.

*Art. 3(1a) - Foundation models. AI systems classified as general-purpose.

*Art. 3(2a) - General-purpose technology. A new method of producing and inventing that is important enough to have a protracted aggregate impact (e.g. Electricity or Information Technology).

*Art. 55b - Measures to enhance public sector. 1. General-purpose AI systems classified as highrisk AI systems by compliance with Annex III, shall be considered by the Member States as general-purpose technologies for public sector enhancement. 2. Member States shall undertake the following actions: (a) Whenever an AI-based general-purpose technology is chosen by the Member State to enhance innovation within the public sector, priority must be guaranteed to open source solutions in order to reduce innovation costs. (b) Exceptions to (a) are those AI systems developed from a closed source, that have proven to achieve better performance in terms of accuracy, transparency or security than any open source solution considered by the Member States, which are obliged to produce documentation explaining the choice of that AI system over the open solutions present in the

European market.

*ANNEX IV(2)(dd). Data requirements listed in ANNEX IV (2)(d) are mandatory for foundation models and whenever natural persons are involved in the intended purpose described in the documentation;

5. Compliance

In this chapter we review proposals to ensure trans

parency, accountability and fairness in AI systems. We discuss the applicability of software licenses, while underlining potential contributions from the open source AI community in developing sound technology standards.

Licensing. Contractor et al. [39] proposes Intellectual

Property (IP) tools for AI software to ensure liability and correctness of use. With respect to patents, licenses protect original creations over conceptual inventions.

A license is a legal agreement between an entity (the licensor) and a subject (the licensee), with defined rights and restrictions. By taking in consideration the 233,601 include a documentation, provided by the publisher, on AI models published on HuggingFace1, approximately the usage of the model, information on the composition 29% of these are published with a license, including: and processing of the training data, analysis on the limi14% with the Apache v2.0 License, fully permissive, tations, intended uses and biases of the model. However, which allows to use, modify and re-distribute a model these information largely vary in amount and detail at also under diferent licensing terms; 6% with the MIT the discretion of the publisher and the contributors. We License, permissive and in line with Apache v2.0; 3% believe these elements could be merged in standardized with OpenRAIL-M [39], permissive but it introduces the frameworks for development and deployment, with parobligation to propagate use restrictions defined by the ticular emphasis on data acquisition and curation that licensor or in line with the BigScience Ethical Charter involve ethical and societal choices. According to Article [40]. We denote two main principles in AI licensing: 40 of the EU AI Act: "conformity to the requirements for lfexibility of use, distribution and modification of the "high-risk AI" or GPAI is ensured through the adoption models; guidance in practical applications, in line with of harmonised standards or parts thereof published in the ethical principles and intended use. We believe the Oficial Journal of the European Union" . When deemed current licensing approaches are complementary, but necessary, the Commission can introduce "common spectheir union is not exhaustive: models could share a ifications" in addition to such standards, that is directives homogeneous set of guidelines for data processing, train- enforced through bodies of experts. Despite standards or ing [29] and output generation [41]; the enforcement of specifications allow to practically apply such provisions, restrictions on unethical use is fragmented or excessively few organisations are in charge for their development. delegated to the licensor. Licensor-defined restrictions The 2nd of February, 2022, the European Commission can result in legally incompatible models that could not announced a new approach to the standardisation sysbe combined in new systems, limiting both research and tem [46], including more active involvement of academic industrial development [42]. Finally, obligations such institutions in the process. as mandatory updates (BigScience RAIL, section IV.7 A major objective of this article is to promote communi[43]) are dangerous for task-specific applications, as cation between research organizations and the European undesired changes in performance can occur without Commission: we believe both parts share common conany power of control for the licensee. cerns in developing trustworthy AI, emerging eforts We believe a candidate solution consists in a permissive from researchers can bridge the gap between directives open AI license with emphasis on traceablity, account- and their practical application without conflicts of interability and limitations solely on use considered against est. reference codes of ethics such as the EU Charter of Fundamental Rights [44], the AI4People framework from Floridi et al. [41] or the EU Trustworthy AI 6. Conclusion Ethical Guidelines[30]. This could be achieved with the adoption of documentation standards for data and software [45][38]. The cooperation with political institutions, e.g. in the EU, becomes crucial to achieve efective enforcement, for instance through the legal validation of licenses for AI and standards for the deployment and monitoring of these systems in the public sector.

Open source AI allows for the collaboration of researchers on a large scale: we believe this setting is ideal to research robustness, reliability and safety in AI models scaling in complexity and capabilities. Openness should underlie the interplay between private research companies, which provide fundamental engineering resources, and researchers in academia and grassroots collectives.

Research from lawmakers and AI academics in AI ethics and safety well overlaps as the concerns are shared: improved communication between the two parts could be beneficial to support open scientific research on one hand, while also enforcing safety measures in the process of integration of increasingly complex AI in our society.

Standards development. A keyword for safe AI is transparency. In the development of trustworthy AI systems, three types of emerging initiatives from the open source AI community are reported by the BigScience initiative: best practices through guidelines and ethical standards, algorithm transparency with unified evaluation metrics and development toolkits, appropriateness of use through forms of documentation such as AI FactSheets [45] and Model Cards [38]. The HuggingFace platform hosts 233,601 models and 44,309 datasets1 organized in repositories: each model or data repository can

1Data updated up to June 18th, 2023

References

[1] A. B. e. a. R. Rombach , High-resolution image synthesis with latent difusion models , 2021 .

[2] R. M. e. a. Zalán Borsos , Audiolm: a language modeling approach to audio generation , 2022 .

[3] J. W. e. a. L. Ouyang , Training language models to recognition, Proceedings of the National Academy follow instructions with human feedback, 2022 . of Sciences 117 ( 2020 ) 7684 - 7689 .

[4] A. A. e. a. C. I. Gutierrez , A proposal for a definition [27] J. A. G. et al., Generative language models and autoof general purpose artificial intelligence systems, mated influence operations: Emerging threats and 2022 . potential mitigations, 2023 .

[5] A. R. et al., Language models are unsupervised mul- [28] J. H. e. a. S. Lin , Truthfulqa: Measuring how models titask learners , 2019 . mimic human falsehoods, 2022 .

[6] M. C. e. a. J. Devlin , Bert: Pre-training of deep bidi- [29] S. K. e. a. Y. Bai , Constitutional ai: Harmlessness rectional transformers for language understanding , from ai feedback , 2022 . 2018 . [30] Requirements of Trustworthy AI , EU Commission,

[7] B. M. e . a. T. B. Brown , Language models are few- 2018. shot learners , 2020 . [31] EIB , Who is prepared for the new digital age? : evi-

[8] R. Bommasani , H. et al., 2021 . dence from the EIB investment survey , Publications

[9] L. B. e. a. A. Dosovitskiy , An image is worth 16x16 Ofice, 2020. words: Transformers for image recognition at scale , [32] ALLAI , Aia in-depth ( 2022 ) 13 - 14 . 2020 . [33] J. Rifkin , The Zero Marginal Cost Society: The In-

[10] R. E. e. a. J. Jumper , Highly accurate protein struc- ternet of Things, the Collaborative Commons, and ture prediction with alphafold , 2021 . the Eclipse of Capitalism, St. Martin's Publishing

[11] Openai usage policy , 2023 . URL: https://openai.com/ Group, 2014 . policies/usage-policies. [34] J. Rifkin , Preparing students for" the end of work" .,

[12] EU , Proposal for a regulation of the eu parliament 1997 . and of the council laying down harmonised rules [35] The future of Artificial Intelligence and radiology on ai and amending certain union legislative acts , Hunimed , 2022 . 2023 . [36] T. Taylor , Some journal of economic perspectives

[13] C. of the European Union, Artificial intelligence act: articles recommended for classroom use , Journal Council calls for promoting safe ai that respects of Economic Perspectives 33 ( 2019 ) 243 - 48 . fundamental rights, 2022 . [37] Huge “foundation models” are turbo-charging AI

[14] V. U. Prabhu , A. Birhane , Large image datasets: A progress , 2022 . pyrrhic win for computer vision ?, 2020 . [38] S. W. e. a. M. Mitchell , Model cards for model re-

[15] J. Buolamwini , T. Gebru, Gender shades: Intersec- porting, 2018 . tional accuracy disparities in commercial gender [39] D. C. et al., Behavioral use licensing for responsible classification , 2018 . ai, 2022 .

[16] B. F. K. et al., Face recognition performance: Role [40] The big science ethical charter , 2022 . URL: of demographic information , IEEE Transactions on https://bigscience.huggingface.co/blog/ Information Forensics and Security ( 2012 ). bigscience-ethical-charter0.

[17] N. C. et al., Extracting training data from large lan- [41] J. C. e. a. L. Floridi , Ai4people-an ethical frameguage models, 2020. work for a good ai society: Opportunities, risks,

[18] L. P. e. a. V. Gulshan , Development and validation of principles, and recommendations , 2018 . a deep learning algorithm for detection of diabetic [42] R. Stallman , Why programs must not limit the freeretinopathy in retinal fundus photographs , 2016 . dom to run them, 2022 .

[19] C. Jee , The therapists using ai to make therapy bet- [43] BigScience , Bigscience rail license v1.0 , 2022 . ter, 2021 . [44] Requirements of Trustworthy AI , EU Commission,

[20] S. Ananya , Lawsuit Raises Copyright Concerns in 2012 . AI-Generated Work , 2022 . [45] D. P. e. a. J. Richards , A methodology for creating

[21] S. e. a. Barke, Grounded copilot: How programmers ai factsheets, 2020. interact with code-generating models , 2022 . [46] T. E. Commission , New approach to enable global

[22] Futurium | European AI Alliance - Trustworthy AI leadership of eu standards promoting values and a in Practice, 2022. resilient, green and digital single market , 2022 .

[23] D. E. Ho , A. Xiang , Afirmative algorithms: The legal grounds for fairness as awareness ( 2020 ).

[24] C. S . et al., Laion-5b: An open large-scale dataset for training next generation image-text models , 2022 .

[25] L. W. et al., Ethical and social risks of harm from language models , 2021 .

[26] A. K . et al., Racial disparities in automated speech