AI algorithms application to medical data has gained relevance due to their powerful benefits among different research tasks. However, medical data is heterogeneous and diverse, and these algorithms need technological support to tackle these data management challenges. KoopaML enables users to unify medical data, especially DICOM images and apply AI algorithms to them in a straightforward way through an online web application. This work presents a new feature in the KoopaML platform: a Machine Learning platform to assist non-expert users in defining and applying ML pipelines. The feature comprises the reception, storage, and management of DICOM images. These images are received through a connection with a PACS (Picture Archiving Communication System) system already configured by users on the platform and, after storing the images, it is possible to apply AI algorithms to them and make modifications or annotations.
Digital Imaging and Communications in Medicine (DICOM) is the standard for representing,
storing, and communicating medical images and related information. It has become one of the most
popular standards in medicine. Initially, DICOM was used to communicate image data between
different systems. Actual developments of the standardization enable increasingly more DICOM based
services for the integration of modalities and information systems [
The potential of artificial intelligence (AI) in healthcare has become evident in recent years with an increasing number of publications using Deep Learning (DL) and Machine Learning (ML) techniques for automated analysis of clinical data. Artificial intelligence systems have been shown to achieve performance quite like clinical experts by providing clinical decision support tools. For example, several AI applications are being proposed to store and manage DICOM images in any healthcare sector due to the motivation driven by the large variety and amount of clinical and imaging data and the potential benefits of AI solutions at different stages of patient care.
To be able to store and manage DICOM images in the healthcare sector, significant amounts of platforms have been created, with the goal of easing the exploitation of these new paradigms. This is the case of KoopaML, an application developed in previous research to allow users to train their own
ML models and analyze and interpret the results obtained without having expert knowledge on the
subject by creating and executing ML pipelines visually. Another of the examples is CARTIER-IA, a
platform for storing, managing, and unifying medical data and images [
Medical imaging ecosystems usually use imaging machinery and communication systems (PACS) as central medical imaging depots. However, most PACS configurations are optimized to support a single departmental workflow and do not easily adapt to new, retrospective, or prospective image analysis workflows for clinical or research purposes. Accessing a large collection of images from a PACS, as required for artificial intelligence (AI) modeling, often requires repetitive manual work to retrieve and post-process thousands of studies.
This work presents a new feature in the aforementioned KoopaML platform, where DICOM images will be stored, managed, and processed by providing an interoperable architecture with external PACS. One of the motivations for applying this feature to the application was to unify data from different sources and organize them into a more user-friendly structure.
The rest of this paper is structured as follows. Section 2 outlines related applications and works to assist users in DICOM images and artificial intelligence algorithms. Section 3 outlines the platform and its main features. Finally, section 4 presents the conclusions derived from the work.
Over the years, DICOM has established itself as the leading standard for imaging data management in Picture Archiving and Communication Systems (PACS). Whith DICOM software, a subject’s image data is viewed and analyzed by clinicians in hospitals as well as in clinical research.
Consequently, the poverty of interfaces for system integration has been identified as a key problem
in clinical trials where systems still operate independently [
Many of these software projects are used today by healthcare personnel in medical centers such as:
• OsiriX. This is a multidimensional image navigation and visualization software for the
visualization and interpretation of multidimensional and multimodality image sets, such as
combined PET-CT studies. The software design is based on leading the user to perform the
specific and complex tasks of navigating through large image data sets in a simpler way. An
important and challenging aspect of this project's development was integrating technologies
such as OpenGL, VTK, ITK, DICOM Offis, Papyrus and QuickTime, which are
crossplatform C/C++ toolsets [
And although more platforms focus on DICOM image management, this work will focus more indepth on two software tools framed in the Cardiology Department of the University Hospital of Salamanca, Spain. These applications are KoopaML and CARTIER-IA, the former being the one that will be the most emphasized and whose internal structure will be discussed. 2.1.
KoopaML is a system framed in the Cardiology Service of the University Hospital of Salamanca to ease the application of machine learning in the medical domain without the necessity of having programming skills [14].
The first version of the platform provides main features such as a graphical user interface to design and execute ML pipelines visually. To create a new pipeline, the system provides an empty workspace with a toolbar containing the potential tasks that can be included in the Machine Learning workflow. These tasks are connected by sockets related to their inputs and outputs. These connections need to be compatible to work if a task requires a dataset as an input, only those tasks that output a dataset can be connected. On the other hand, KoopaML also offers the possibility to visually explore the input dataset and model evaluation results. This feature takes advantage of the automatic generation of information dashboards by domain engineering and metamodeling [15]–[17].
However, possible improvements and validations of this platform were detected based on the new needs derived from the user-centered approach followed for its development. During the evaluation of this platform, users some features that could be included in the platform so that the use of Artificial Intelligence could be extended to DICOM images [18]. 2.2.
The Cartier-IA platform unifies structured data and medical images, specifically DICOM images, to
support researchers and physicians in the analysis associated with different studies, with a particular
focus on supporting the applications of artificial intelligence algorithms in images. The CARTIER-IA
platform can be seen as a technological ecosystem that supports all data-management related tasks
(including structured data and medical imaging collection) and enables both healthcare professionals
and data scientists to apply AI models to the stored images [
The platform relies on different technologies and frameworks which are integrated using a clientserver architecture. The technology employed to implement this client-server approach as a web application is Django, a Python-based web framework [19]. The web application is also connected through web requests to other services, such as a REDCap instance to manage additional projects and information [20].
Finally, to implement the integrated AI environment, the back end is supported by libraries such as OpenCV and TensorFlow, to enable the execution of deep learning models and other AI-related scripts.
To integrate the new functionality into KoopaML, a new workspace was devised, separating the Machine Learning workspace from the DICOM workspace as it was crucial to avoid a cluttered interface. In addition, the substantial differences between structured data require different treatments depending on the type of input.
Therefore, users will be able to choose between creating a Machine Learning project and a DICOM project. If the project created is of the second type it is necessary to add a configuration section to manage the Picture Archiving and Communication System (PACS) to allow KoopaML application to request images from the configured servers.
When a new DICOM project is created, users will be shown the images it has stored in that project and, to which it can apply the different Machine Learning algorithms found in the system. These algorithms can be stored by the application administrators and the users is asked for a series of characteristics such as weight files to be able to apply the algorithms to the DICOM images. These algorithms are loaded into the platform by authorized AI experts, making them accessible to non-expert users [21].
In addition, the DICOM workspace also allows users to modify images using different editing tools to measure, annotate, crop, zoom, pan, shift and segment, among others (Figure 1).
Regarding the connection between KoopaML and any of the PACS that would be configured in the tool, the following structured has been followed (Figure 2).
First, a connection has been made between the HTML template and the Python file of the KoopaML platform by making a request through the Django framework in JavaScript. Secondly, the different responses are made to finish the connection between KoopaML and the configured PACS. The first of the tools will request different data from the second one and this one will answer with the necessary data. Once the form data has been collected, it is broken down and then sent to the PACS so that it can receive the corresponding DICOM images, returning to the initial HTML template the number of images that this platform has received whit the data collected from the form.
In this way, a satisfactory connection is achieved between KoopaML and the different PACS that the users configure in the different projects and later storage of the DICOM images that the user has selected for later management of these.
To connect to these PACS, the first thing users must do is to configure this system. The platform contains the option to configure these external entities from which the images will be collected for subsequent management and storage. If a user wants to configure an entity, he/she must indicate its name, IP address, the port where it is located and, optionally, a description of the system.
Figure 3 shows the list of PACS entities configured in DICOM projects. As can be seen in each of the entries of the list the option to connect and delete the entities is given.
If the user wants to connect, user will be redirected to a screen where images collected from that PACS can be added to the project. In this screen it will be possible to search by patient data as well as by projects or studies to which the DICOM images we want to search belong. As you can see, the screen displays the information corresponding to the PACS entity to which the tool is connected. And once the search has been performed, a pop-up window will be displayed with the number of images that have been stored in the project you are in. 3.2.
One of the features that were proposed to be integrated in KoopaML was to provide a friendly interface to apply Artificial Intelligence algorithms and open its uses for non-expert users and offers benefits of these algorithms without the need to leave the platform to apply them.
This feature allows researchers to upload their Artificial Intelligence scripts to the platform and make them available to other users. Only researchers with privileges will be able to add new algorithms which must be thoroughly tested by the researcher who uploads them to the platform before integrating them to ensure reliable functionality, since leaving the functionality to upload Artificial Intelligence algorithms to any of the users have this capacity, and they must be Machine Learning experts. In this case, the algorithm metadata is very important to properly integrate the algorithms within the platform because it provides information about the algorithm output (a modified image, a set of measurements, a segmentation mask, …), its application (since its application may be limited to specific DICOM modalities) or other parameters depending on the output.
To integrate an algorithm into the platform, it is necessary to provide the model trained previously and the script that makes use of this model to allow its invocation by the platform’s Artificial Intelligence algorithm. Once an algorithm has been integrated into the platform, a button will be available in the image editor to apply an algorithm selected from those available for the current image (Figure 4). When the user confirms the request, the platform will produce the result, depending on the type of algorithm output, which could result in the display of a new image.
Figure 5 shows the application of a segmentation algorithm to a DICOM image. Image segmentation consists of dividing an image into as many regions as objects and background it contains.
This work presents the continuous improvement of a platform for creating AI-driven pipelines in the medical domain, namely KoopaML. The first version of the platform obtained good comments, but some new features were required to support other complex tasks in the medical context, one of which was the possibility of storing DICOM images received through PACS systems and their subsequent management.
Being able to receive DICOM images from other file systems and image communication (PACS) gives the platform several advantages such as increasing flexibility or making connections between others easier. In this way, users will have the possibility to analyze the DICOM images in a simple way and apply Artificial Intelligence algorithms or edit the images in a way that do not need help from the experts.
processing and visualisation,” NeuroImage, vol. 202, p. 116137, Nov. 2019, doi: 10.1016/j.neuroimage.2019.116137.
[14] F. García-Peñalvo et al., “KoopaML: A Graphical Platform for Building Machine Learning Pipelines Adapted to Health Professionals,” Int. J. Interact. Multimed. Artif. Intell., vol. In Press, no. In Press, p. 1, 2023, doi: 10.9781/ijimai.2023.01.006.
[15] A. Vázquez‐Ingelmo, A. García‐Holgado, F. J. García‐Peñalvo, and R. Therón, “Proof‐of‐ concept of an information visualization classification approach based on their fine‐grained features,” Expert Syst., vol. 40, no. 1, Jan. 2023, doi: 10.1111/exsy.12872.
[16] A. Vázquez-Ingelmo, F. J. García-Peñalvo, R. Therón, D. Amo Filvà, and D. Fonseca Escudero, “Connecting domain-specific features to source code: towards the automatization of dashboard generation,” Clust. Comput., vol. 23, no. 3, pp. 1803–1816, Sep. 2020, doi: 10.1007/s10586019-03012-1.
[17] A. Vázquez-Ingelmo, F. J. García-Peñalvo, and R. Therón, “Taking advantage of the software product line paradigm to generate customized user interfaces for decision-making processes: a case study on university employability,” PeerJ Comput. Sci., vol. 5, p. e203, 2019, doi: 10.7717/peerjcs.203.
[18] A. García-Holgado et al., “User-Centered Design Approach for a Machine Learning Platform for Medical Purpose,” in Human-Computer Interaction, P. H. Ruiz, V. Agredo-Delgado, and A. L. S. Kawamoto, Eds., in Communications in Computer and Information Science, vol. 1478. Cham: Springer International Publishing, 2021, pp. 237–249. doi: 10.1007/978-3-030-92325-9_18. [19] “Django,” Django Project. https://www.djangoproject.com/ (accessed Mar. 21, 2023). [20] P. A. Harris, R. Taylor, R. Thielke, J. Payne, N. Gonzalez, and J. G. Conde, “Research electronic data capture (REDCap)—A metadata-driven methodology and workflow process for providing translational research informatics support,” J. Biomed. Inform., vol. 42, no. 2, pp. 377–381, Apr. 2009, doi: 10.1016/j.jbi.2008.08.010.
[21] A. Vázquez-Ingelmo et al., “Testing and improvements of KoopaML: a platform to ease the development of Machine Learning pipelines in the medical domain,” presented at the Proceedings TEEM 2022: Tenth International Conference on Technological Ecosystems for Enhancing Multiculturality, Salamanca, Spain, Salamanca, Spain: Springer Nature Singapore, Oct. 2022.