As the EU AI Act comes into force, AI systems in education, particularly those influencing learning outcomes and student pathways, can be considered 'high-risk' as these systems can significantly impact individuals' educational trajectories by profiling students and educators, predicting outcomes, and inadvertently perpetuating biases or limiting future opportunities. Given the rising integration of such systems into education, it becomes imperative to assess their accuracy, trustworthiness and fairness. This paper presents a comprehensive framework integrating legal, socio-ethical and technological dimensions to assess AI trustworthiness. We apply this framework to two case studies-an assessment platform (Thrively) and an adaptive learning tool (Century Tech)-and demonstrate their utility. Using a Natural Language Model to automate developer documentation analysis, we reveal gaps in fairness and data security. Our contribution ofers timely insights to guide the ethical development of educational AI and support compliance with the EU's AI Act (2024).
Artificial intelligence in education (AIED) is likely designated as ‘high-risk’ under EU AI Act (s.6(2),
Annex 3(3), and recital 56 of the AI Act) [
Under the EU´s AI Act (2024), AIED are classified ‘high risk’ if used to determine admission, evaluate
learning outcomes, assess appropriate education levels, and monitor behaviour (Annex III, Section 3
[
Responsibility lies with (1) AI providers—persons or entities that develop an AI system and place it on
the market or put into service (e.g., Century Tech, etc.) (Art 3(3) [
AI providers and deployers must document trustworthiness measures related to design, deployment, and standards. TAI-SDF supports automated analysis of this documentation using LLMs by flagging gaps across legal, ethical, and technical dimensions. Users can supplement automated outputs with manual inputs where needed. For example, a university developing and deploying an AI admissions system would need to assess transparency, fairness across socioeconomic groups, and appeal mechanisms. While some exceptions exist (e.g., for research-only systems), compliance becomes mandatory once systems enter the market. In a word, preparatory alignment with legal requirements becomes mandatory.
Developed under Horizon Europe ‘TRUSTEE’ project, TAI-SDF formalizes and streamlines humancentric trustworthiness assessments for AI systems. Based on ALTAI principles and other international standards, the framework is structured around four core components: • Personas – Diferent users who interact with TAI-SDF: – Model Provider (builds AI systems) – Deployer (uses AI systems in real-world settings) • Aspects of AI trustworthiness considered:
– Privacy, security, fairness, robustness and explainability • Lifecycle phases: – Scope and plan to understand the interfaces, types of AI feedback, and gather requirements in compliance with regulatory and ethics principles – Data and algorithm management – Verification and validation to test the system functionalities and security features • Evidence collection for compliance:
– Documentation and structured user prompts (tasks, constraints, examples)
TAI-SDF is not only a theoretical framework and a set of questionnaires covering diferent aspects and phases of AI products; it has been implemented as a suite of practical software tools that support the actions associated with the obligations of the AI Act for the High-Risk AI Systems (such as the AI based education applications). By combining structured questionnaires with AI-driven analysis, TAI-SDF enables compliance verification for AI systems in education and beyond. Its building blocks include: a questionnaire-based assessment, which allows the user to create questionnaires related to the following aspects: fairness, privacy, security, social impact and explainability; an AI-driven assistant that supports users in assessing compliance of documentation with trustworthiness questions and requirements. In particular, TAI-SDF tools support the Case Studies in the following: • A set of questions are created with the Questionnaire based assessment; • Deployer documentation is loaded on the AI based assistant; • The AI assistant provides answers/evidence related to the questions previously created.
The questionnaire-based assessment within TAI-SDF is a qualitative and evidence-based software tool designed for self-assessment (for AI developers) and third-party evaluations (for deployer, institutions or regulators). Specifically, the Model Provider Self-Assessment allows the AI developer to perform a Trustworthiness Analysis during the AI design and development lifecycle. This requires access to the source code, design documentation and test results. The Deployer Assessment allows evaluators or end-users to carry out a third-party assessment, relying only on the AI system user experience and any documentation that is shared by the Model Provider. From a deployer perspective (i.e., a school), TAI-SDF provides the capability to access predefined questionnaires compiled by domain experts or create new reusable ones. Both can benefit from the additional support of the AI driven assistant, which is described next in 2.3.2.
The TAI-SDF framework includes an AI-driven web service which allows the user (e.g., a school) to assess the completeness of product´s (technical) documentation against a set of requirements that can be retrieved from the TAI-SDF knowledge database itself, via an intuitive and essential user interface. Moreover, such a tool can assist the user with generating trustworthiness-related product documentation if missing in the first place. Writing software documentation compliant to AI trustworthiness standards must consider some typically overlooked considerations [15] which this tool aims to spot. Users can upload and edit one or multiple PDF files, refine them, and build structured prompts based on prompt engineering best practices to maximize the efectiveness of AI analysis [ 16]. This set-up aligns with advances in few-shot prompting [17], zero-shot reasoning [18] and retrieval-augmented generation for knowledge-intensive tasks [19]. While the use cases shown in the present work rely on publicly available documents, in real-world scenarios, third-party deployers should have access to provider documentation for a more thorough assessment. To input documentation and requirements, the user prompt includes: • Tasks: Instructs the AI model on what should be done; • Constraints: Instructs the AI model on the required response format; • Examples: Shows the model some examples to enhance its reasoning capabilities; • Additional Information: provide contextual details to enhance assessment accuracy. The AI-driven assessment can be conducted using a locally hosted private model (e.g., Ollama) for data security, or remote LLM services, (e.g. OpenAI), for scalability and ease of access. The result is a set of reports detailing compliance based on the available documentation against each of the selected trustworthiness requirements. Gaps in AI transparency, fairness and security policies are also generated as a result. Two examples of educational software documentation assessment are shown in Sections 3.1 and 3.2.
In this paper we aimed to demonstrate a novel tool for assessing trustworthiness and fairness of AIED.
Theoretically, the tool is informed by the new European legislature [
A fairness questionnaire (10 key questions) was generated using TAI-SDF. These focused on bias detection, fairness metrics and transparency in Thrively’s AI system. The AI-driven tool analysed Thrively’s publicly available documentation to determine whether fairness principles were addressed. The obligation to examine potential biases and to implement mitigation measures stems from Art 10(2)(f) and (g) AI Act and the Diversity, Non-discrimination and Fairness requirement of the ALTAI principles [11].
Can TAI-SDF support the Deployer Assessment? Yes, but with limitations. As described in the previous sections, the aim has been to assess the completeness of the documentation covering the given topic. The fairness assessment was based on publicly available documentation. The absence of developer documentation, model details or dataset descriptions prevented a fuller fairness evaluation, as shown in Table 1. An additional section, tailored to fairness assessment, has been added to the available documentation to show how the same questions can be compared against an updated documentation. Such documents are scanned by the natural language model wrapped by the tool to assimilate their content and assist the final user to check or generate product documentation by providing assessment reports analysing their semantic coverage of the TAI-SDF questions. The assessment results for each question, embedding the reasoning procedure of the model, are shown in the form of a long and verbose output text file [20].
The AI-driven assessment tool achieved a 100% true negative rate (TNR), meaning, the considered documentation does not address any of the considered TAI-SDF fairness questions. After fairness updates of the available documentation, TAI-SDF was able to detect fairness compliance, which confirmed its ability to enhance documentation quality when supported with relevant information. This simple comparison between the original true negative (from documentation without any fairness coverage) and the new true positive (from fairness-tailored simulation) shows that the approach of using a reasoning model to scan documentation for weak points (i.e. nonconformities with respect to TAI-SDF knowledge base) can assure a complete view on the potential strengths and vulnerabilities of the current documentation. Additionally, the tool ofers detailed explanations of the reasoning process (due to limited space, full findings can be found at [ 20]), which make the assessment assistance process transparent and objective, instead of serving as a mere black-box tool. Crucially, TAI-SDF successfully identified fairness gaps in the original documentation. Although assessment is limited when fairnessrelated disclosures are missing from AI providers, with updated documentation, AI models could confirm fairness compliance.
The second case study focused on the data security assessment of Century Tech (CT), chosen due to the relative availability of its public documentation. Again, the assessment involved two steps: generating 13-question security questionnaire with TAI-SDF—covering privacy-by-design, encryption, and anonymisation—and using the AI assistant to analyse CT’s public privacy policy for compliance. Each question was processed against the available documentation [20], in line with ensuring compliance with the Technical Robustness and Safety requirement of the ALTAI principles [11] while the corresponding provision of Art 15 AI Act obligates the developer and deployer of high-risk AI systems to provide a “appropriate level of accuracy, robustness, and cybersecurity, and that they perform consistently in those respects throughout their lifecycle.”
Can TAI-SDF support the Deployer Assessment? Yes, as in Thrively’s case, however with limitations due to the limited privacy and security documentation available, as highlighted in Table 2. Since CT’s developer documentation (regarding their AI models) was unavailable, the assessment was based on their security and privacy policy. Using what was available, TAI-SDF efectively identified privacy gaps (e.g., missing details on encryption, anonymisation, risk mitigation). Trustworthiness concerns were raised due to unclear security protocols and lack of explicit privacy-by-design measures.
A score of 100% for the TNR, the precision and the recall metrics have been obtained. The small number of true positives with respect to the number of true negatives shows that the considered CT privacy documentation does address some of the TAI-SDF privacy requirements ‘as-is’, but it can be definitively improved or integrated. In other words, two security elements were correctly identified (as a basic compliance mention); 11 security gaps were confirmed which includes missing encryption and details around data privacy. Lastly, no information relating to the AI functions of CT suggests that much improvement is needed to achieve transparency, fairness and trustworthiness.
The integration of AI into education introduces high stakes for fairness, trustworthiness, security and privacy for students, teachers, and society at large. This calls that key stakeholders urgently address measures to not only assess fast-encroaching AIED across fairness and trustworthiness principles but ensure that they fully meet them. This paper explored how the TAI-SDF framework—an AI trustworthiness evaluation tool—supports legal and ethical compliance, technical assessment, and contextual application in AI-powered educational technologies. Trustworthy AI systems in education require a multifaceted approach where legislation, technology, and contextual considerations are aligned. TAI-SDF is built on a multidisciplinary knowledge base with legal, ethical, industry, and academic experts, to align with current standards and technical best practices. The TAI-SDF framework shows its capabilities to support both model providers and deployers in assessing various aspects of the trustworthiness of AIED during their full lifecycle. The introduction of a local AI-based assessment assistant boosted by reasoning capabilities helps both self- and third-party assessors in spotting current documentation weak points and evaluating new documentation versions, all while fully demonstrating the logical process and guaranteeing no external leakage of product documentation. It is expected that this holistic and domain-agnostic approach can help both technical and non-technical professionals in a deeper understanding of the key concepts of trustworthiness of complex systems, improve the overall quality of the documentation and streamline the development of fair, ethical, and secure AI systems. By applying TAI-SDF to two real-world AI products-Thrively (for fairness in student assessment) and Century Tech (for data security and privacy in adaptive learning), we demonstrated both the potential and challenges of AI accountability measures in education. In Thrively’s case, TAI-SDF successfully identified missing fairness metrics in the company’s original documentation. Once fairness-related details were added, the AI assistant detected and validated fairness principles confirming that TAI-SDF can enhance transparency when AI providers disclose relevant information. For Century Tech, the tool flagged missing security and privacy details such as encryption methods and privacy-by-design practices. These flags should inform companies of the key actions to ensure their systems adhere to legal and socio-ethical standards and prevent the risk of harm in education. The findings have also demonstrated that TAI-SDF can help AI system providers and deployers assess and document the fairness of their AI systems. This structured assessment supports compliance with the AI Act, particularly Articles 10 and 11, by providing evidence of how fairness-related requirements are addressed and ensuring comprehensive technical documentation of such evidence. In doing so, the tool also aligns with the ALTAI principles, facilitating transparent documentation and enabling organisations to demonstrate that their AI systems uphold fairness and accountability standards. That said, a key limitation of this work is TAI-SDF’s reliance on available documentation. The framework and its tools can only assess what is documented; if AI providers fail to disclose fairness or security measures, a complete evaluation is not possible. While TAI-SDF helps deployers, end-users and even regulators lfag compliance gaps, this does not guarantee that an AI system is fair or trustworthy. A full audit remains essential, which necessitates direct access to internal development documents (e.g., source code, test reports, datasets diversity, etc.). Future work should focus on AI providers’ willingness to adopt structured documentation formats that support assessments of their systems’ fairness, trustworthiness, and security. Moreover, any future work shall also augment the relevance and robustness of this study by validating the results of these tools. Finally, TAI-SDF is adaptable to domains beyond education. Improving the AI model’s ability to interpret loosely structured or incomplete documentation is also necessary for better real-world assessments. Ultimately, the work around TAI-SDF along with AI systems in or outside the education domain will encourage AI developers to align with AI regulatory frameworks and prioritise their systems’ transparency and trustworthiness.
This work was produced under the European Project TRUSTEE (No: 101070214).
During the preparation of this work the authors used a small, locally-hosted, pretrained LLM, LLama3.1 8B Instruct. The LLM has been used in order to: • analyse the documentation to determine whether fairness principles were addressed (see Case study 1); • and analyse the documentation to determine whether security principles were addressed (see
Case study 2); Afterwards, the authors reviewed and edited the content as needed and take full responsibility for the paper’s content. [11] E. Commission, The assessment list for trustworthy artificial intelligence (altai)., 2020. URL: https://digital-strategy.ec.europa.eu/en/library/ assessment-list-trustworthy-artificial-intelligence-altai-self-assessment. [12] A. Koene, Ieee p7003tm standard for algorithmic bias considerations. 2018 ieee/acm international workshop on software fairness (fairware), 38–41, 2018. URL: https://doi.org/10.23919/FAIRWARE. 2018.8452919. [13] Thrively, 2025. URL: https://www.thrively.com/. [14] Century tech, 2025. URL: https://www.century.tech/. [15] F. Königstorfer, Software documentation is not enough! requirements for the documentation of ai. digital policy, regulation and governance, 23(5), 475–488., 2021. URL: https://doi.org/10.1108/ DPRG-03-2021-0047. [16] J. Wei, Chain of thought prompting elicits reasoning in large language models., 2022. [17] A. Fisch, Making pre-trained language models better few-shot learners. proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural langua gao ge processing, 3816–3830, 2021. [18] T. Kojima, Large language models are zero-shot reasoners., 2022. [19] P. Lewis, Retrieval-augmented generation for knowledge-intensive nlp tasks., 2021. [20] RINA-C, Results assessing ai trustworthiness of ai systems for education, 2022. URL: https://github. com/EPL-education/Assessing-AIeD.