In recent years, the increasing integration of AI within clinical software has led to the emergence of Software as a Medical Device (SaMD), a category of systems subject to strict regulatory oversight. A major challenge for developers in this domain is the continuous production of regulatory documentation, which remains largely manual and disconnected from development pipelines. This paper proposes a strategy to automate documentation generation by embedding MLOps principles-such as traceability, reproducibility, and continuous integration-into the development workflow. Applied to a representative healthcare AI project, the approach produced consistent, audit-ready artefacts with minimal manual efort, demonstrating its potential to narrow the gap between rapid innovation and regulatory compliance.
Artificial Intelligence (AI) is increasingly embedded in clinical software, giving rise to a new class of
regulated systems known as Software as a Medical Device (SaMD) [
From an MLOps perspective, the lack of integration between development pipelines and regulatory processes represents a major gap. Unlike traditional DevOps, MLOps for regulated AI systems must support not only traceability and reproducibility, but also generate artifacts required for audit and certification purposes. In this context, automation of documentation becomes a key enabler for scaling and operationalizing AI-based SaMD development.
MLOps (Machine Learning Operations) is an extension of DevOps tailored to the unique challenges
of machine learning systems. It focuses on automating and managing the entire ML lifecycle—from
data ingestion and model training to deployment and monitoring [
CEUR Workshop
ISSN1613-0073
Common tools like GitHub Actions, Docker, and MLflow support automation and versioning but
are not designed to produce the regulatory artefacts required by standards such as MDR 2017/745, ISO
13485, or IEC 62304. As a result, documentation and compliance tasks are often handled separately,
leading to ineficiencies and increased risk. This limitation has been explicitly highlighted by Granlund
et al., who studied the development of certified medical software and noted that CI/CD pipelines must
be augmented with regulatory-aware processes to ensure audit-ready artefact [
Software as a Medical Device (SaMD) is defined by the International Medical Device Regulators Forum
(IMDRF) as software intended to be used for medical purposes without being part of a physical medical
device [
Although documentation has long been acknowledged as vital in safety-critical domains, relatively
few eforts have focused on automating this process in the context of regulated machine learning
systems. Traditional DevOps tools, such as Sphinx and Doxygen, can generate documentation from
annotated code; however, they do not support regulatory requirements defined in standards like ISO
13485 or IEC 62304. MLOps platforms such as MLflow, Kubeflow, and Metaflow emphasize pipeline
orchestration, experiment tracking, and model versioning [
This section describes the architecture and design rationale of our system, which automates the generation of documentation artefacts required under the EU MDR 2017/745 regulatory framework. The solution is designed to integrate seamlessly into MLOps pipelines and focuses on two key documents: the Software List and the SOUP (Software of Unknown Provenance) List. The system is implemented in Python, emphasizing modularity, portability, and maintainability. Its design supports CI/CD workflows (e.g., GitHub Actions) and enables incremental development through distinct but interconnected stages, which are detailed in the following subsections.
The first phase involved identifying documentation requirements based on the European standards outlined in Section 2. From this analysis, we identified a subset of documents that are (i) critical to certification and (ii) potentially automatable. Notably, we focused on the Software List and the Software of Unknown Provenance (SOUP) List, which are essential for demonstrating traceability, component identification, and risk mitigation strategies.
The system is designed to operate on well-structured software repositories that follow best practices in software engineering. It is implemented in Python and relies on common tools such as GitHub, PyPI, and virtual environments. The architectural core is a documentation orchestrator—a component that orchestrates metadata extraction and document rendering via a set of scheduled or event-driven tasks. Among the key design principles: • Scalability: the system can be extended to include additional document types, enabling future expansion to cover a broader range of regulatory artefacts. • Maintainability: the documentation logic is modular and separated from the templates, allowing for independent updates and customisation. • Portability: the system is compatible with standard CI/CD infrastructures (e.g., GitHub Actions), ensuring ease of integration across projects. • Auditability: all automated processes are logged, and versioned outputs are maintained to support traceability and external audit requirements.
The data collection pipeline includes: • Static Source Tree Inspection: used to collect filenames, directory structures, software versions, and file paths within the project repository. • Dependency scanning: performed by parsing files such as requirements.txt, pyproject.toml, and environment descriptors to extract third-party packages and their versions. • PyPI Metadata Retrieval: each external package is queried on the Python Package Index to retrieve metadata such as license type, maintainer identity, version history, and repository URL.
This step uses the requests library to interface with PyPI. • Git inspection: Using Git commands, the system extracts repository metadata such as commit history, authorship, timestamps, and change frequency for each file. This allows for detailed temporal tracking and helps identify deprecated or inactive components. • Standard library discrimination: the sysconfig and pkgutil modules are used to diferentiate built-in Python modules from third-party dependencies.
All extracted data are structured and stored in JSON format, supporting persistence across runs and enabling reproducibility and audit readiness. Fields that cannot be resolved automatically, such as risk classification, are left as placeholders for manual input by domain experts.
We employed the Jinja22 templating engine to define customizable and regulatory-aligned document formats. For each document type (e.g., Software List, SOUP List), we created a structured template with placeholders populated dynamically from the collected metadata. This approach allows automatic updates whenever the source code changes, with minimal manual intervention. Furthermore, unresolved or ambiguous fields (e.g., risk classification) are flagged for manual input. The documentation generation process is fully integrated into the CI/CD pipeline and is triggered automatically upon each pull request to the repository. The flow, shown in Figure 1, consists of five main stages: • Pull Request: When a developer opens a pull request on GitHub, a workflow configured via GitHub Actions is executed. This ensures that documentation is updated synchronously with code modifications. • Data Analysis: The system extracts relevant metadata from the repository. This includes static code analysis, dependency extraction from configuration files, and querying the Python Package Index (PyPI) to retrieve licensing, versioning, and maintainer information. • Documents Generation: The extracted metadata is passed to the templating component to generate structured documents. This stage produces two regulatory artefacts: the Software List and the SOUP List. Any unresolved or unverifiable fields are flagged for expert review. • Store Documents: The generated documentation files are stored in a dedicated directory within the repository. These documents, along with auxiliary files such as historical software state snapshots and error reports, are committed and pushed to the pull request branch. • Final Output: The result is a version-controlled, machine-generated set of documentation files, ready for inclusion in the technical file required by regulatory audits and certification procedures.
GitHub Actions
PULL REQUEST
DATA ANALYSIS
DOCUMENTS GENERATION
DOCSTUOMREENTS
We tested the system on CT-COVID,3 a public Python-based SaMD project for COVID-19 detection using CT scans. The repository follows a clear structure, defines its dependencies explicitly, and includes a CI/CD setup, making it a suitable case study for our evaluation. The documentation tool was integrated into the project using GitHub Actions. Each pull request triggered automatic generation of the required documentation artefacts in Markdown format. The output captured detailed metadata for over 30 external packages, including names, versions, licenses, and maintainer information. To evaluate the results, we compared the generated documentation with the information available in the repository. The output demonstrated strong consistency, with most fields being filled in automatically. Manual input was only needed for specific elements, such as risk classification or component rationale, which naturally require expert judgment. This validation shows that the system can reliably extract and organize regulatory metadata with minimal manual efort and can be adapted to support other documents and use cases.
This work presented an approach to automating regulatory documentation for AI-based medical software by embedding document generation into MLOps workflows. Implemented with widely 2https://jinja.palletsprojects.com/en 3https://github.com/se4ai2122-cs-uniba/CT-COVID adopted tools such as Git, PyPI, and Jinja2, the system dynamically produces MDR-aligned documents that remain consistent with code changes and version-controlled within the repository. Validation on a real-world SaMD project demonstrated the ability to extract key metadata—such as dependencies, version history, and authorship—and populate documentation templates with high completeness. At the same time, expert oversight remains necessary for fields such as clinical validation. These results highlight the potential to reduce the manual burden of compliance while ensuring reproducibility and audit readiness. Although currently limited to Python-based projects, the approach can be extended to other document types and compliance frameworks. Overall, this study represents a first step towards integrating regulatory documentation into continuous MLOps pipelines, bridging the gap between rapid AI innovation and the stringent demands of medical software certification.
This research was co-funded by the Complementary National Plan PNC-I.1 (“DARE”, PNC0000002, CUP: B53C22006420001), the NRRP Initiative - Next Generation EU (“FAIR”, PE00000013, CUP: H97G22000210007), and (“SERICS”, PE0000014, CUP: H93C22000620001).
During the preparation of this work, the authors used ChatGPT to check grammar and spelling.