<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>GRALENIA: Antimicrobial Resistance Management based on Natural Language and Artificial Intelligence</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Cristóbal Bernardo-Castiñeira</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff4">4</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Germán Bou</string-name>
          <email>german.bou.arevalo@sergas.es</email>
          <xref ref-type="aff" rid="aff2">2</xref>
          <xref ref-type="aff" rid="aff4">4</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Manuel Campos</string-name>
          <email>manuelcampos@um.es</email>
          <xref ref-type="aff" rid="aff3">3</xref>
          <xref ref-type="aff" rid="aff4">4</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Bernardo Cánovas- Segura</string-name>
          <email>bernardocs@um.es</email>
          <email>cristobal.bernardo@bahiasoftware.es</email>
          <xref ref-type="aff" rid="aff3">3</xref>
          <xref ref-type="aff" rid="aff4">4</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Sergio Figueiras Gómez</string-name>
          <email>sergio.figueiras@bahiasoftware.es</email>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff4">4</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Carlos Gómez-Rodríguez</string-name>
          <email>carlos.gomez@udc.es</email>
          <xref ref-type="aff" rid="aff1">1</xref>
          <xref ref-type="aff" rid="aff4">4</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Enrique Míguez Rey</string-name>
          <email>enrique.miguez.rey@sergas.es</email>
          <xref ref-type="aff" rid="aff2">2</xref>
          <xref ref-type="aff" rid="aff4">4</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Jesús Vilares</string-name>
          <email>jesus.vilares@udc.es</email>
          <xref ref-type="aff" rid="aff1">1</xref>
          <xref ref-type="aff" rid="aff4">4</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Bahía Software SLU</institution>
          ,
          <addr-line>Rúa das Hedras 4, L-1, Ames, 15895, A Coruña</addr-line>
          ,
          <country country="ES">Spain</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>CITIC - Universidade da Coruña, Depto. de Ciencias de la Computación y Tecnologías de la Información, Campus de Elviña</institution>
          ,
          <addr-line>15071, A Coruña</addr-line>
          ,
          <country country="ES">Spain</country>
        </aff>
        <aff id="aff2">
          <label>2</label>
          <institution>Instituto de Investigación Biomédica de A Coruña (INIBIC), Hospital Teresa Herrera</institution>
          ,
          <addr-line>15006, A Coruña</addr-line>
          ,
          <country country="ES">Spain</country>
        </aff>
        <aff id="aff3">
          <label>3</label>
          <institution>Medical Informatics and Artificial Intelligence Laboratory (MedAI Lab), University of Murcia</institution>
          ,
          <addr-line>30100, Murcia</addr-line>
          ,
          <country country="ES">Spain</country>
        </aff>
        <aff id="aff4">
          <label>4</label>
          <institution>SEPLN-CEDI2024: Seminar of the Spanish Society for Natural Language Processing at the 7th Spanish Conference on Informatics</institution>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2024</year>
      </pub-date>
      <abstract>
        <p>The objective of GRALENIA project is to develop a multidisciplinary, comprehensive and interoperable platform incorporating artificial intelligence algorithms and natural language processing techniques to improve the management of antimicrobial resistance (AMR) and reduce the impact of antimicrobial- or antibiotic-resistant microorganisms (aka superbugs) in hospitals. GRALENIA is supported through a Red.es grant for research and development projects in artificial intelligence and other digital technologies and their integration into value chains. Natural language processing, artificial intelligence, machine learning, deep learning, cloud services, healthcare, antimicrobial resistance, superbug, industrial research project</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>
        Over the last two decades, antimicrobial resistance
(AMR) has become a threat to public health systems
worldwide. World Health Organization (WHO)
considers this problem as one of its priorities [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. An
example of AMR is the resistance of certain pathogenic
bacteria to antibiotics, which causes some treatments
based on antibiotic prescription not to work, thus
resulting in the appearance of serious clinical
complications and a considerable increase in
healthcare costs. In Europe alone, AMR causes 33,000
deaths per year and a loss of €1.5 billion in terms of
additional treatment and social costs [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ].
      </p>
      <p>Furthermore, the uncontrolled transmission of
such superbugs1 between patients and healthcare
workers poses a serious risk to the healthcare system.</p>
      <p>0000-0003-0411-3056 (C. Bernardo-Castiñeira);
0000-00018837-0062 (G. Bou); 0000-0002-5233-3769 (M. Campos);
00000002-0777-0441 (B. Cánovas-Segura); 0009-0003-1680-9269 (S.
Figueiras); 0000-0003-0752-8812 (C. Gómez-Rodríguez);
00000003-2941-1834 (J. Vilares)
© 2023 Copyright for this paper by its authors. The use permitted under
Creative Commons License Attribution 4.0 International (CC BY 4.0).</p>
      <p>
        CEUR Workshop Proceedings (CEUR-WS.org)
1 Superbugs is a name given to harmful bacteria that have acquired
resistance to one or more of the antibiotics used to treat them.
Appropriate use of antimicrobials is very complicated
because of the complexity of infectious diseases and
the spread of antibiotic resistance. Because of this, the
Spanish National Plan for Antibiotic Resistance [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ] has
established the implementation of programs for
optimizing the use of antibiotics in hospitals, the
socalled Antimicrobial Stewardship Programs (ASP),
that carried out by multidisciplinary teams of
specialists, the ASP teams [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ].
      </p>
      <p>There is a wealth of information that can be
exploited to manage this serious problem, but the
analysis of all this information is a very complex
process. In many cases, such analysis is done manually,
which results in a work overload for professionals,
thus increasing hospital costs or, failing that,
worsening health care work.</p>
      <p>Artificial intelligence (AI) algorithms can improve
interventions and decision making by ASP and
healthcare-associated infection (HCAI) surveillance
teams. The information held by the different services
of a hospital is susceptible to be exploitable by AI
algorithms for the development of intelligent AMR and
HCAI control strategies, and for improving
evidencebased decision making. However, the exploitation of
such data by AI is very complex due to the
heterogeneity of the data and its frequent lack of
standardization and structuring. This fact has
hindered the development of AI models for dealing
with AMR and superbug infections, which are
generally still in a laboratory/research phase.</p>
      <p>In this context, GRALENIA constitutes an ambitious
industrial research project that seeks to develop
solutions based on the use of AI and natural language
processing (NLP) techniques to exploit this
information and provide innovative solutions to the
AMR challenge.</p>
      <p>The rest of the paper is organized as follows.
Section 2 describes the project, its objectives, the lines
of work involved in its development, the participant
teams and their contribution. Section 3 presents the
general architecture of the GRALENIA platform and
describes the modules that comprise it, with the
exception of the IA module, to be described in Section
4. Finally, Section 5 summarizes and closes the paper.</p>
    </sec>
    <sec id="sec-2">
      <title>2. The Project</title>
      <p>As a whole, the GRALENIA project2,3 aims to drive and
accelerate digital transformation in healthcare,
specifically in the area of AMR and superbug
management. Hospital information systems contain a
lot of information from various specialties directly or
indirectly related to AMR occurrences, such as the
possible inappropriate use of antibiotics (dosage
information can be provided by the Pharmacy service)
or diagnostic and characterization tests for superbugs
(provided by the Microbiology service). Nevertheless,
much of this information is not interrelated or
standardized, making it difficult to exploit it
automatically and to make decisions considering all
the available information.</p>
      <sec id="sec-2-1">
        <title>2.1. Objectives</title>
        <p>The GRALENIA project pursues the development of a
multidisciplinary, comprehensive and interoperable
platform that incorporates AI algorithms and NLP
techniques to improve the digital management of AMR,
and to reduce the impact of superbugs in hospitals.
Starting from this general objective, the project has the
following specific objectives:
1. The integration and standardization of
relevant clinical data held by different hospital
services. This allows its ulterior automatic
processing.
2. The development of a base infrastructure for
automated annotation of unstructured clinical
documents using NLP techniques.
3. The prediction of the risk of superbug
emergence in the hospital, and the identification of
groups of patients with high susceptibility to
superbug infection.
4. To improve the management and
visualization of integrated data for AMR
surveillance.</p>
        <p>GRALENIA platform is being developed to be
compatible with the data infrastructure of the A
Coruña University Hospital Complex (CHUAC). This
allows AI models to be designed using real-world
information, thus providing the hospital with a digital
solution to improve decision making regarding an
existing problem.</p>
        <p>To this end, GRALENIA not only focuses on R&amp;D
tasks for developing predictive models, but also
2 Spanish acronym for Gestión de Resistencias a Antimicrobianos
basada en Lenguaje Natural e Inteligencia Artificial (Antimicrobial
Resistance Management based on Natural Language and Artificial
Intelligence).
provide tools for structuring and standardizing both
structured and unstructured information from various
information silos. This makes it possible to integrate it
as a whole, in a structured way, into a common data
model on the platform. In turn, GRALENIA enables
easy data visualization and exploitation by AI
algorithms.</p>
      </sec>
      <sec id="sec-2-2">
        <title>2.2. Work Lines</title>
        <p>To achieve these ambitious objectives, considerable
R&amp;D effort is needed. Some of the main research lines
involved in this project are:
1. To determinate which type of information,
directly or indirectly related to AMR, is potentially
exploitable.
2. To design an NLP-based system for the
automatic extraction of AMR-related information
from unstructured clinical documents.
3. Definition and development of advanced AI
models for AMR-related risk prediction.
4. Validation of model results in a real-world
environment.</p>
        <p>In addition, it is also necessary to provide the
system with the necessary mechanisms so that it can
access information from the different hospital services
relevant for AMR management.</p>
      </sec>
      <sec id="sec-2-3">
        <title>2.3. Consortium</title>
        <p>These work lines are developed by a consortium of
four participants.</p>
        <p>Bahía Software.4 Coordinating team and
industrial partner of the consortium. Founded by
professionals with experience in the IT and Health &amp;
Healthcare sectors, the commercial activity of the
company is mainly focused on e-Health, with
important presence on e-Administration, industry and
banking too. The company is recognized as an
Innovative SME and as a national reference in medical
image, tumor committees and surgical block
management.</p>
        <p>LYS Group (CITIC-UDC). The second organization
involved in the project is the Language and
Information Society Group (LYS)5 of the ICT Research
Center (CITIC)6 of the University of A Coruña (UDC).
LYS is an interdisciplinary research group formed by
professors and researchers in AI and Linguistics, with
extensive experience in the fields of NLP and
Computational Linguistics. CITIC has an outstanding
activity in the transfer of research results to society
and industry. Based on its experience and expertise,
the contribution of the LYS research team to the
project will focus on the development of a base
infrastructure for the automatic annotation of
unstructured clinical information:
• The NLP-based clinical Information</p>
        <p>Extraction (IE) system for processing the
3 gralenia.es (visited on February 2024).
4 bahiasoftware.es (visited on February 2024).
5 www.grupolys.org (visited on February 2024).
6 citic.udc.es (visited on February 2024).
7 www.um.es/aike/ (visited on February 2024).
8 www.inibic.es (visited on February 2024).
•</p>
        <p>Co-design of the end-user tools (healthcare
professionals). This guarantees the usability
of tools and their algorithms, as well as the
highest possible added value.</p>
        <p>In addition, INIBIC coordinates the management of
access to data and infrastructures with the
Information Systems Service of CHUAC.</p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>3. General Architecture</title>
      <p>The user interface consists of a web interface, allowing
a simpler and more accessible use. This reduces the
learning curve of its handling and, therefore, makes it
more pleasant to use. Additionally, the front-end also
acts as a display window and information control
panel for the Dashboard module (see Section 3.4). The
front-end works as another service running within the
GRALENIA cluster.</p>
      <sec id="sec-3-1">
        <title>3.2. Data Ingestion</title>
        <p>The data ingestion process of the GRALENIA platform
includes several (sub)processes:
1. Extracting the data from the various
information silos of the hospital.
2. Transforming the data to adapt it to the
required format and structure.
9 aws.amazon.com (visited on February 2024).
Microservice that connects to the Amazon S3 and allows both accessing to the
information stored there and uploading new documents.</p>
        <p>Due to CHUAC internal regulations, privacy and
information security issues, and project deadlines, a
data integration strategy through CSV data downloads
has been chosen. Given the heterogeneous nature of
both the information systems of the hospital and the
variables required, specific extraction, anonymization
and transformation processes have been developed to
fit a common data model. The resulting data are
dumped into CSV files to be ingested into the platform
storage system.</p>
        <p>However, our platform also contemplates the
possibility of integration with hospital information
systems using HL7 standards.10</p>
      </sec>
      <sec id="sec-3-2">
        <title>3.3. Storage System</title>
        <p>GRALENIA platform implements a data lake. A data
lake is a data repository used to store large amounts of
both structured and unstructured data in its original
format. Unlike traditional database systems, that
require data to conform to a predefined schema, a data
lake allows the ingestion of data from various raw
sources and in various formats (text, images, sensor
data, logs, etc.). The use of this data lake allows data
from different information silos to be stored centrally
and securely.</p>
        <p>Despite being a centralized repository, a
threelayer design has been used:
1. Landing zone. This is the entry point of the
platform. In this first layer the data from the
different information silos is stored in raw form,
without any type of transformation.
2. Staging zone. An intermediate layer where
validations and transformations are performed to
adapt the data to the data model of the platform.
3. Refined zone. The last layer of the data lake.
This is where the already structured and curated
data is stored so that it can be consumed by the
systems and tools that require it.</p>
        <p>In the case of the first two layers, the landing and
staging zones, an Amazon Simple Storage Service (S3)
bucket11 is used. This is a versatile cloud object storage
service that stands out for its scalability, security and
accessibility. On the other hand, the last layer, the
refined zone, uses the Amazon RDS service12 to host
two relational databases:
1. An operational database to feed the various
functionalities of the platform. The operational
database is organized around hospital admissions,
which uniquely identify each patient stay in a
hospital.
2. An OMOP-CDM database to perform
observational data analysis. The Observational
Medical Outcomes Partnership (OMOP)13 is an
open community standard, designed to provide a
standardized way to represent data structure (i.e.
a Common Data Model or CDM) and content
(terminologies, vocabularies, coding scheme, etc.),
and to enable efficient analyses that can produce
reliable evidence.</p>
      </sec>
      <sec id="sec-3-3">
        <title>3.4. Information Control Panel</title>
        <p>The GRALENIA platform has a catalog of components
oriented to the visualization of the information in an
intuitive way so that it can be exploited by any user,
from generalist profiles to specialists in data analysis.
This catalog facilitates the correct assimilation of the
data and, when integrated with the AI module, it also
allows showing the results obtained by the algorithms.</p>
        <p>The dashboard designed in GRALENIA makes it
possible to quickly and easily analyze various Key
Performance Indicators (KPI) potentially related to the
emergence of superbugs in a hospital. The dashboard
integrates a set of filters that can be configured by the
user to run customized analyses such as, for example,
studying the behavior of a specific superbug in a given
time period and/or area of the hospital.</p>
      </sec>
      <sec id="sec-3-4">
        <title>3.5. Microservices (Back-End)</title>
        <p>The back-end is the part of the platform that runs on
the GRALENIA servers. We have decided to divide the
functionalities of the system into several
microservices. Thus, for each task of the platform,
there is a single microservice with that single
functionality. These microservices are described in
Table 1.
10 www.hl7.org/implement/standards/ (visited on February 2024).
11 aws.amazon.com/s3/storage-classes/ (visited on February 2024).
12 aws.amazon.com/rds/ (visited on February 2024).
13 www.ohdsi.org/data-standardization/ (visited on February 2024).</p>
      </sec>
      <sec id="sec-3-5">
        <title>3.6. Security, Management and</title>
      </sec>
      <sec id="sec-3-6">
        <title>Support Layer</title>
        <p>In order to guarantee the security of a system that
handles such sensitive data as this one, it is essential to
use secure user and role control tools. For this
purpose, the system uses Keycloak,14 an open source
and widely used identity and access management
(IAM) tool. Keycloak is deployed as another service
running on the GRALENIA cluster.</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>4. AI Module</title>
      <p>The purpose of the AI module is to identify the triggers
for the occurrence of multi-resistances. To this end, it
integrates three systems: (1) an NLP-based IE system
for clinical texts; (2) a system for predicting the risk of
superbug outbreaks in different areas of the hospital;
and (3) a system for identifying patient profiles with a
high susceptibility to superbug infection.</p>
      <sec id="sec-4-1">
        <title>4.1. Indication Extractor</title>
        <p>
          The NLP subsystem of the IA module consists of an IE
system to identify and extract relevant clinical
information on free text sources such as electronic
health records (EHR), nursing notes, etc. [
          <xref ref-type="bibr" rid="ref5 ref6">5, 6</xref>
          ]. In this
case, we are interested in mentions that may indicate
the possible existence or susceptibility of the patient to
the infections under analysis. Those indicators are
classified according to their typology (sign, symptom,
etc.). The extracted information is then structured,
homogenized and validated to complement that
information obtained from other sources of the
hospital. Finally, all this collected data feeds, in the
form of features, the rest of subsystems of the AI
module (outbreak prediction and phenotyping).
        </p>
        <p>
          The general architecture of the indication extractor
system is based on a pipeline, which gives it great
flexibility. Two different pipeline configurations are
provided:
1. A low-resource rule-based approach [
          <xref ref-type="bibr" rid="ref7">7</xref>
          ] that
relies on a fuzzy pattern matching process and
term expansion [
          <xref ref-type="bibr" rid="ref8">8</xref>
          ].
2. A state-of-the-art but resource-demanding
approach based on transformers [
          <xref ref-type="bibr" rid="ref9">9</xref>
          ]. In the
absence of a gold standard to train the system, the
output of the previous (and simpler) pattern-based
approach is used as a silver standard.
        </p>
        <p>With the international market in mind, both
approaches have been implemented for Spanish and
English languages.</p>
        <p>The IE system has been dockerized. for its final
deployment to facilitate its integration with the rest of
components of the platform.</p>
      </sec>
      <sec id="sec-4-2">
        <title>4.2. Outbreak Predictor</title>
        <p>
          The goal of this second component focuses on
predicting the risk of outbreaks in different areas of
the hospital at different times in the future [
          <xref ref-type="bibr" rid="ref10 ref11">10, 11</xref>
          ].
        </p>
        <p>The scope of the project has limited the analysis to the
most prevalent superbugs (MRSA, Klebsiella ESBL and
Klebsiella carbapenemase), as well as to the most
vulnerable areas of the hospital, those where an
outbreak of these characteristics could have
catastrophic consequences: the intensive care unit and
the recovery room. For each superbug, two
approaches have been, although a possible third
approach (based on time series) is also currently
under consideration.</p>
        <p>The first approach consists on predicting the risk
of occurrence of at least one superbug. Classification
algorithms are used for this purpose. A battery of
alternatives including logistic regression, decision
trees, random forest, support vector machines (SVM),
boosting and artificial neural network (ANN) based
algorithms are being evaluated.</p>
        <p>The second approach corresponds to the
prediction of the number of cases of superbug
contagion. In this case, due to the discrete nature of the
target variable, Poisson regression models are used.</p>
      </sec>
      <sec id="sec-4-3">
        <title>4.3. Phenotyping System</title>
        <p>The objective of this third and final system is to
identify patient profiles with high susceptibility to
trigger a superbug outbreak.</p>
        <p>
          For the phenotyping algorithms, an approach
based on subgroup discovery has been selected [
          <xref ref-type="bibr" rid="ref12">12</xref>
          ].
These mixed descriptive-predictive models aim to
obtain several sets of phenotypes to describe a
concept, but also different in form so that they can
show different perspectives of the database and,
possibly, some of them have a clinical interpretation.
        </p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>5. Conclusions</title>
      <p>As a whole, GRALENIA constitutes an industrial
research project with high added value for the
healthcare professionals of the A Coruña University
Hospital Complex (CHUAC), who are facing such
complex problems as antimicrobial resistance (AMR)
and superbug management. GRALENIA project lays
the groundwork for extending the technologies
developed within it not only to other hospital services,
but also to other hospitals and health services.</p>
      <p>It is also worth noting that many of the tools and
techniques based on artificial intelligence developed
within the project (e.g. predictive and spatial models,
natural language processing tools, etc.) lay the
foundations for their adaptation to other clinical areas,
further boosting the digital transformation of the
healthcare sector.</p>
    </sec>
    <sec id="sec-6">
      <title>Acknowledgements</title>
      <p>This work is partially funded by the GRALENIA project
(reference 2021/C005/00150055) supported by the
Spanish Ministry of Economic Affairs and Digital
Transformation, the Spanish Secretariat of State for
Digitization and Artificial Intelligence, Red.es and by
the NextGenerationEU funding.</p>
      <p>Research on CITIC is also financed by the Xunta de
Galicia through the collaboration agreement between
the Ministry of Culture, Education, Vocational Training
and Universities and the Galician universities to
reinforce the System's research centers University of
Galicia (CIGUS)</p>
      <p>We would also like to thank Dr. José María
Gutiérrez Urbón (CHUAC Pharmacy service), Dr. Laura
Gutiérrez Fernández (CHUAC Infectious Diseases
unit), Daniel Llamas Gómez (current Chief Information
Officer-CIO, A Coruña-Cee Health District, SERGAS)
and Guillermo Vázquez González (former CIO) for their
valuable contributions to the project.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>World</given-names>
            <surname>Health</surname>
          </string-name>
          <string-name>
            <surname>Organization</surname>
          </string-name>
          , Global Action Plan: Antimicrobial Resistance,
          <string-name>
            <surname>WHO</surname>
          </string-name>
          , Geneva,
          <year>2025</year>
          . ISBN 9789241509763. URL: https://www.who.
          <source>int/publications/i/item/978 9241509763 (visited on February</source>
          <year>2024</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>European</given-names>
            <surname>Commission Public Health</surname>
          </string-name>
          ,
          <source>EU Action on Antimicrobial Resistance</source>
          ,
          <year>2024</year>
          . URL: https://ec.europa.eu/health/antimicrobialresistance/eu-action-antimicrobialresistance_
          <source>en (visited on February</source>
          <year>2024</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          <article-title>[3] Plan Nacional frente a la Resistencia a los Antibióticos (PRAN</article-title>
          ),
          <year>2024</year>
          . URL: https://www.resistenciaantibioticos.
          <source>es (visited on February</source>
          <year>2024</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>J.</given-names>
            <surname>Rodríguez-Baño</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.R.</given-names>
            <surname>Paño-Pardo</surname>
          </string-name>
          , L. AlvarezRocha et al., ”Programas de optimización de uso de antimicrobianos (
          <article-title>PROA) en hospitales españoles: documento de consenso GEIH-SEIMC, SEFH y SEMPSPH”</article-title>
          ,
          <source>Enfermedades Infecciosas y Microbiología Clínica</source>
          ,
          <volume>30</volume>
          .1 (
          <year>2012</year>
          ):
          <fpage>22</fpage>
          .
          <fpage>e1</fpage>
          -
          <lpage>22</lpage>
          .
          <year>e23</year>
          . DOI:
          <volume>10</volume>
          .1016/j.eimc.
          <year>2011</year>
          .
          <volume>09</volume>
          .018.
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>K.</given-names>
            <surname>Kreimeyer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Foster</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Pandey</surname>
          </string-name>
          et al., “
          <article-title>Natural language processing systems for capturing and standardizing unstructured clinical information: A systematic review”</article-title>
          ,
          <source>Journal of Biomedical Informatics</source>
          ,
          <volume>73</volume>
          :
          <fpage>14</fpage>
          -
          <lpage>29</lpage>
          ,
          <year>2017</year>
          . DOI:
          <volume>10</volume>
          .1016/j.jbi.
          <year>2017</year>
          .
          <volume>07</volume>
          .012.
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>A.</given-names>
            <surname>Névéol</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Dalianis</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Velupillai</surname>
          </string-name>
          et al.,
          <article-title>“Clinical Natural Language Processing in languages other than English: opportunities and challenges”</article-title>
          ,
          <source>Journal of Biomedical Semantics</source>
          ,
          <volume>9</volume>
          :
          <fpage>12</fpage>
          ,
          <year>2018</year>
          . DOI:
          <volume>10</volume>
          .1186/s13326-018-0179-8.
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>G.</given-names>
            <surname>Napolitano</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Fox</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Middleton</surname>
          </string-name>
          et al., “
          <article-title>Patternbased information extraction from pathology reports for cancer registration”</article-title>
          ,
          <source>Cancer Causes Control</source>
          ,
          <volume>21</volume>
          :
          <fpage>1887</fpage>
          -
          <lpage>1894</lpage>
          ,
          <year>2010</year>
          . DOI:
          <volume>10</volume>
          .1007/s10552-010-9616-4.
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>F.</given-names>
            <surname>Prado-Valiño</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Santos-Rios</surname>
          </string-name>
          ,
          <string-name>
            <surname>C.</surname>
          </string-name>
          <article-title>GómezRodríguez</article-title>
          &amp;
          <string-name>
            <surname>J. Vilares</surname>
          </string-name>
          , “
          <article-title>Prototype of an Entity Recognition System for Antimicrobial Resistance Data Management”</article-title>
          ,
          <source>in: Proceedings of the 6th XoveTIC Conference (XoveTIC</source>
          <year>2023</year>
          ),
          <string-name>
            <given-names>A</given-names>
            <surname>Coruña</surname>
          </string-name>
          , Spain, forthcoming.
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>X.</given-names>
            <surname>Yang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Bian</surname>
          </string-name>
          ,
          <string-name>
            <given-names>W. R.</given-names>
            <surname>Hogan</surname>
          </string-name>
          &amp; Y. Wu, “
          <article-title>Clinical concept extraction using transformers”</article-title>
          ,
          <source>Journal of the American Medical Informatics Association (JAMIA)</source>
          ,
          <volume>27</volume>
          (
          <issue>12</issue>
          ):
          <fpage>1935</fpage>
          -
          <lpage>1942</lpage>
          ,
          <year>2020</year>
          . DOI:
          <volume>10</volume>
          .1093/jamia/ocaa189.
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>N.</given-names>
            <surname>Peiffer-Smadja</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.M.</given-names>
            <surname>Rawson</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Ahmad</surname>
          </string-name>
          et al., “
          <article-title>Machine learning for clinical decision support in infectious diseases: a narrative review of current applications”</article-title>
          ,
          <source>Clinical Microbiology and Infection</source>
          ,
          <volume>26</volume>
          (
          <issue>5</issue>
          ):
          <fpage>584</fpage>
          -
          <lpage>595</lpage>
          ,
          <year>2020</year>
          . DOI:
          <volume>10</volume>
          .1016/j.cmi.
          <year>2019</year>
          .
          <volume>09</volume>
          .009.
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>C.F.</given-names>
            <surname>Luz</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Vollmer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Decruyenaere</surname>
          </string-name>
          et al., “
          <article-title>Machine learning in infection management using routine electronic health records: tools, techniques, and reporting of future technologies”</article-title>
          ,
          <source>Clinical Microbiology and Infection</source>
          ,
          <volume>26</volume>
          (
          <issue>10</issue>
          ):
          <fpage>1291</fpage>
          -
          <lpage>1299</lpage>
          ,
          <year>2020</year>
          . DOI:
          <volume>10</volume>
          .1016/j.cmi.
          <year>2020</year>
          .
          <volume>02</volume>
          .003.
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>A.</given-names>
            <surname>Lopez-Martinez-Carrasco</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.M.</given-names>
            <surname>Proença</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.M.</given-names>
            <surname>Juarez</surname>
          </string-name>
          et al.,
          <article-title>“Novel Approach for Phenotyping Based on Diverse Top-K Subgroup Lists”</article-title>
          ,
          <source>in: Artificial Intelligence in Medicine, AIME 2023</source>
          , vol.
          <volume>13897</volume>
          of Lecture Notes in Computer Science, Springer, Cham,
          <year>2023</year>
          . DOI:
          <volume>10</volume>
          .1007/978-3-
          <fpage>031</fpage>
          - 34344-
          <issue>5</issue>
          _
          <fpage>6</fpage>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>