As Artificial Intelligence (AI) systems are increasingly deployed in critical sectors, ensuring their responsible and reliable operation has become a priority. This paper examines the importance of technical conformity verification as a mechanism for embedding principles of fairness, transparency, and safety within the AI development lifecycle. It distinguishes between commercial and research-grade systems, highlighting how their respective maturity levels influence the depth of conformity assessments. The study emphasizes the strategic placement of verification eforts after model evaluation and before deployment to mitigate risks related to bias, opacity, and lack of accountability. Drawing on international standards such as ISO/IEC TR 24027, the authors propose a structured approach to conformity verification that includes checklists and bias assessment protocols. The goal is to foster AI systems that are not only high-performing but also trustworthy, inclusive, and aligned with ethical norms. Future work will involve validating these practices through empirical case studies and fostering collaboration between academia, industry, and regulatory bodies.
CEUR ISSN1613-0073
Artificial Intelligence (AI) systems are being rapidly integrated into decision-making pipelines
across diverse sectors, including healthcare, finance, transportation, and public administration.
These technologies promise to enhance eficiency, reduce human error, and enable data-driven
policy and operational decisions. However, despite their technical advancements, AI systems
often carry risks related to bias, opacity, and limited accountability [
In light of these challenges, there is growing consensus around the need for AI systems to
conform to established technical standards that codify principles of fairness, transparency, and
reliability [
Yet the mere existence of such standards is not suficient to ensure ethical or robust outcomes. To translate principles into practice, they must be operationalized through concrete conformity assessment protocols—structured procedures that systematically evaluate whether an AI system meets the requirements defined by the standards. These protocols should be tailored to diferent stages of technological maturity and integrated directly into the AI development workflow, allowing developers to anticipate and correct potential failures before systems reach production environments. As demonstrated by real-world incidents, such as the temporary suspension of ChatGPT in Italy due to regulatory concerns over data processing and transparency, neglecting these safeguards can have immediate legal and reputational consequences.
In this context, embedding conformity verification into the AI lifecycle is not merely a regulatory formality but a critical component of building AI systems that are not only performant but also fair, transparent, and trustworthy by design.
Ensuring that AI systems meet established technical standards is essential for promoting safety, reliability, and public trust. This is particularly relevant for systems deployed in sensitive or high-impact environments. Technical conformity verification involves assessing whether an AI system adheres to normative requirements regarding its design, development, and behavior. This section outlines key considerations for integrating conformity assessment within the AI lifecycle, distinguishing between types of systems, and identifying the appropriate timing for such assessments.
AI systems difer significantly in their readiness for deployment. For this reason, it is important to distinguish between commercial products and research-grade systems, as the requirements for conformity verification vary accordingly.
A widely adopted framework for measuring technological maturity is the Technology Readiness Level (TRL) scale. Originally developed by NASA, TRL levels range from early conceptual stages (TRL 1–3) to fully operational systems (TRL 7–9). Research prototypes typically fall within TRL 1–6, while commercial products are generally situated at TRL 7 and above2. • TRL 1–3: Basic principles and proof-of-concept exploration, often limited to academic research. • TRL 4–6: Prototype development and validation in relevant settings, representing the so-called “valley of death” in innovation.
• TRL 7–9: Final stages of testing, system qualification, and commercial deployment.
While full conformity verification is mandatory for commercial AI solutions, research-oriented systems may undergo a lighter form of evaluation, primarily focusing on methodological validity and reproducibility. Nonetheless, even research systems can benefit from partial conformity assessment, especially when addressing fairness, safety, or replicability in applied contexts.
Conformity verification should be strategically integrated within the AI development process. The optimal point for intervention is after the model evaluation phase and before deployment. This allows for a comprehensive assessment of risks and limitations before the system is made available to end-users or integrated into operational environments.
Figure 1 illustrates the typical AI development pipeline and highlights where conformity assessment is most efectively applied. 2See NASA TRL definitions: https://www.nasa.gov/directorates/heo/scan/engineering/technology/txt_accordion1. html
Conducting verification before deployment is critical for minimizing legal, ethical, and reputational risks. Without it, issues such as inadequate bias control or insuficient data governance may only surface post-deployment, making remediation costly or infeasible. A notable example of premature deployment is the temporary suspension of ChatGPT in Italy due to concerns over user verification and data processing transparency 3—demonstrating the high stakes involved in releasing unverified AI systems.
A structured conformity process supports compliance with technical and ethical expectations, especially in applications involving sensitive user data or life-impacting decisions. In addition to addressing model performance and safety, conformity verification ensures: • User protection: Verifying mechanisms for access control and risk mitigation. • Data privacy: Assessing transparency in data usage and storage. • Bias detection: Identifying and addressing sources of algorithmic unfairness, in line with standards such as ISO/IEC TR 240274.
In this context, standards play a foundational role. Yet their efectiveness depends on implementation through structured, repeatable processes. The next sections propose a framework to operationalize this verification through detailed checklists and bias assessments based on international technical guidelines.
As AI systems become increasingly embedded in real-world applications, ensuring their alignment with technical standards is not merely a regulatory formality—it is a necessary safeguard for ethical, reliable, and accountable deployment. The complexity and autonomy of these systems demand proactive mechanisms for risk identification, bias control, and performance validation, particularly in high-stakes domains where the consequences of technical failures may afect human rights, public safety, or institutional trust.
In this paper we outlined the rationale for integrating conformity verification into the AI development lifecycle, emphasizing the importance of distinguishing between research-grade prototypes and commercial-grade systems. Such diferentiation enables the application of proportionate verification protocols tailored to the system’s level of technological maturity and intended deployment context. Importantly, we have highlighted the optimal placement of conformity checks—between model evaluation and deployment—where they can most efectively prevent costly or irreparable failures in production settings.
Beyond assessing performance, conformity assessment frameworks grounded in international standards such as ISO/IEC TR 24027 provide essential tools for the systematic identification 3See oficial press release from the Italian Data Protection Authority: https://www.garanteprivacy.it/home/docweb/ -/docweb-display/docweb/9870847 4ISO/IEC TR 24027:2021, “Information technology — Artificial intelligence — Bias in AI systems and AI aided decision making,” available at: https://www.iso.org/standard/77607.html and mitigation of algorithmic bias, data governance weaknesses, and opaque engineering choices. By incorporating structured verification steps—such as bias audits, documentation reviews, and checklists—developers can embed fairness, transparency, and safety directly into the development process, rather than treating these as post-hoc concerns.
The need for such frameworks is particularly pressing in critical applications involving sensitive data or life-altering decisions. In these contexts, the absence of reliable conformity verification not only exposes users to harm but also undermines public confidence in AI systems more broadly.
Looking ahead, future work should aim to refine and validate these verification protocols through empirical case studies and cross-sectoral collaboration. Engagement between standards bodies, regulatory institutions, industry practitioners, and academic researchers will be crucial to ensure that verification practices remain up-to-date, context-sensitive, and practically implementable. Only through such integrated, interdisciplinary eforts can we build AI systems that are not only powerful, but also just, inclusive, and trustworthy by design.
During the preparation of this work, the authors used GPT and DeepL to perform grammar and spelling check. After using these tools and services, the authors reviewed and edited the content as needed and take full responsibility for the publication’s content.