<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Data Product Designer: An Experimental Prototype to Support Data-Driven Decision-Making</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Susanna Kuper</string-name>
          <email>susanna.kuper@fokus.fraunhofer.de</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Juliane Schmeling</string-name>
          <email>juliane.schmeling@fokus.fraunhofer.de</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Philipp Martin</string-name>
          <email>ISSNc1e6u1r-3w-0s0.o7r3gphilipp.martin@fokus.fraunhofer.de</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Qi Kang Chen</string-name>
          <email>qi.kang.chen@fokus.fraunhofer.de</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Fraunhofer-Institute for open Communication Systems</institution>
          ,
          <addr-line>FOKUS, Kaiserin-Augusta-Allee 31, 10589 Berlin</addr-line>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2025</year>
      </pub-date>
      <fpage>0000</fpage>
      <lpage>0001</lpage>
      <abstract>
        <p>In this paper, we examine methods to align indicator systems with strategic goals, enhancing data-driven decision-making in public administration. Additionally, we leverage Large Language Models (LLMs) and Prompt Engineering techniques to optimize data analysis and visualization workflows. We explore the integration of the Data Mesh paradigm, which treats data as a product. This approach addresses challenges in traditional data architectures, such as a lack of democratization. The research problem focuses on the strategic alignment between data-driven decision-making and governmental goals, alongside the need for effective data governance. Our proposed solution, the Data Product Designer (DPD), encompasses four core areas: Strategy, Search, Data Gap, and Analysis. AI supports these areas with non-binding suggestions, adhering to the Human-in-the-Loop principle, ensuring user control over data product development. The DPD offers a unified method that strategically aligns indicators and enhances data exchange and visualization generation. By enriching brainstorming and facilitating data needs identification and visualization, our AI-assisted approach adapts the GQM method for public sector data management. Evaluation results indicate high user acceptance, providing insights for further refinement and contributing to robust data management in public administration.</p>
      </abstract>
      <kwd-group>
        <kwd>Data-driven decision-making</kwd>
        <kwd>data product</kwd>
        <kwd>data management</kwd>
        <kwd>AI assistance</kwd>
        <kwd>strategic alignment [1]</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>
        The advent of digitalization has offered unprecedented opportunities for public administration
worldwide, in particular, the potential to become data-driven [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. Data plays a crucial role in every
stage of the policy decision-making process throughout the entire policy cycle [
        <xref ref-type="bibr" rid="ref2 ref3">2, 3</xref>
        ]. Our focus is on
public administration as part of policy implementation, since the administration possesses a vast
wealth of data. Data-driven decision-making is increasingly shaped by emerging opportunities arising
from the integration of Artificial Intelligence (AI) [
        <xref ref-type="bibr" rid="ref4 ref5 ref6">4–6</xref>
        ]. However, this transition is not without its
challenges. One of the central problems is to maintain strategic alignment, resulting in ineffective use
of data in the decision-making process [
        <xref ref-type="bibr" rid="ref7 ref8 ref9">7–9</xref>
        ]. Additionally, the process of data collection is often
poorly digitized, further inhibiting the efficient use of data [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ].
      </p>
      <p>
        Data governance, the management and regulation of data usage, has become an essential
underpinning for data-driven decision-making in public administration [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ]. Research has indicated
that a data-driven governance depends on several vital components, such as a nationwide data
architecture, a digitally transformed public administration, and human resources equipped with the
necessary skills and expertise [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ]. Consequently, public administration is increasingly concentrating
on two primary areas to facilitate the transition towards data-driven governance. Firstly, they
leverage existing data from various sources through aggregation and analysis, and secondly, they are
building (real-time) data exchange networks to enhance public service delivery (Ibid).
      </p>
      <p>The research problem outlined is twofold: firstly, strategic alignment between data-driven
decision-making and the government's strategic goals is challenging public administration; secondly,
there is a need for effective data governance and management to be integrated with strategic and
organizational demands. This integration is essential to establish a robust foundation for successful
data-driven decision-making within complex data exchange networks. Therefore, the guiding
research question is: What organizational and technical methods can enhance the development of
data products in public administration?</p>
      <p>
        To approach the research question, this research adopted a problem-oriented and
practiceinspired approach within the framework of the Action Design Research Methodology [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ]. This
methodology was complemented by the use of human-centred design methods, including rapid
prototyping, which allows for the creation of solutions that are both user-friendly and effective [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ].
The human-centered design included different real-world use cases that we evaluated together with
a pilot partner in the public sector domain. The primary limitation of our research is that the DPD was
assessed exclusively within a single domain and organization, which may restrict the generalizability
of the findings.
      </p>
      <p>As a solution to the aforementioned research problem, the paper presents a modular data
governance software toolkit, named Data Product Designer (DPD), embedded within a conceptual
framework. The DPD toolkit is designed to promote an integrated, open, and analytics-driven
approach to public administration. Its innovative open architecture connects business intelligence
(BI) tools, different Large Language Models (LLM), data sources, and data governance applications,
effectively tackling both strategic alignment and operational data management as a foundation for
robust data-driven decision-making.</p>
      <p>The prototype was developed as part of a reference implementation project for data space
technology research between 2022 and 2025. It serves as an advanced smart service that can be
leveraged within complex and distributed data infrastructures. This paper presents the following
research contributions:




</p>
      <p>An AI-assisted methodology to enrich the strategically aligned data product development
process
A conceptual model that bridges the gap between operational data management and
strategic decision-making in the public administration
A lightweight design approach for low threshold entry to non-technical field experts and
strategists
A flexible architecture that is designed as an interoperable framework to integrate with
several data management and software stacks
An approach to identify and overcome data gaps through AI-driven synthetic data
generation</p>
      <p>The research paper begins with an overview of related work in the section 2. Section 3 outlines
the research design and overall methodology. Following this, the paper presents the DPD (section 4)
as the key innovation and the evaluation results (section 5). The discussion and future research
directions analyze the findings and suggest areas for future inquiry. Finally, the conclusion
summarizes the key findings and their significance, restates the importance of the research problem,
and highlights the contributions of the study to the field.</p>
    </sec>
    <sec id="sec-2">
      <title>2. Related Work</title>
      <p>In this section, we explore related work for the integration of data-driven frameworks and advanced
technologies to enhance decision-making processes. This discussion illustrates how strategic data
frameworks and cutting-edge AI technologies can be synergized to optimize decision-making
efficiency and effectiveness.</p>
      <sec id="sec-2-1">
        <title>2.1. Data Product Orientation</title>
        <p>
          This research was particularly inspired by the new data governance paradigm of Data Mesh,
especially in its emphasis on the concept of data as a product. The Data Mesh paradigm provokes
treating data as a product and organizing it according to business domains [
          <xref ref-type="bibr" rid="ref15 ref16">15, 16</xref>
          ]. This approach
enables teams to take distributed responsibility for data relevant to their areas, addressing challenges
of traditional monolithic architectures, such as poor data quality and lack of democratization [
          <xref ref-type="bibr" rid="ref16">16</xref>
          ]. By
prioritizing data as a product, Data Mesh shifts the focus from data pipelining and storage, promoting
a more product-oriented data management strategy. In this paper, we recognize data products as
crucial elements of indicator systems that enable effective data-driven decision-making.
        </p>
      </sec>
      <sec id="sec-2-2">
        <title>2.2. Goal-oriented Indicator Systems</title>
        <p>
          The governance model for trusted big data systems starts by adhering to important regulations,
such as data protection laws, which set the framework for developing policies, principles, and
procedures [
          <xref ref-type="bibr" rid="ref17">17</xref>
          ]. Policies are essentially rules that guide how users should interact with data.
Principles serve as the foundation for how data is managed, reflecting societal values and
expectations and emphasizing the treatment of data as a valuable asset. Procedures involve regular
activities like planning cycles, the establishment of ethics committees, and conducting audits to
ensure that operations are conducted smoothly and ethically. Professionals handling data are
expected to follow norms specific to their fields.
        </p>
        <p>
          It is essential for public administration to develop indicator systems and strategies that ensure
that data initiatives are closely integrated with their overarching goals to fully harness the benefits of
digitalization. Notable frameworks include the Balanced Scorecard [
          <xref ref-type="bibr" rid="ref18">18</xref>
          ], the Decision Maker Model
[
          <xref ref-type="bibr" rid="ref19">19</xref>
          ], the Goal Question Metric [
          <xref ref-type="bibr" rid="ref20">20</xref>
          ], or the Measurement Information Model in the ISO/IEC/IEEE
15939:2017 standard [
          <xref ref-type="bibr" rid="ref21">21</xref>
          ].
        </p>
        <p>
          The DPD utilizes the Goals-Questions-Metrics (GQM) method to establish an initial measurement
framework and remains flexible to adopt other frameworks as supplementary template applications.
GQM is a systematic approach to defining and interpreting a set of indicators or metrics to reach
strategic alignment. This method is primarily used in the fields of software engineering [
          <xref ref-type="bibr" rid="ref20">20</xref>
          ], skills
assessment [
          <xref ref-type="bibr" rid="ref22">22</xref>
          ], and cyber security [
          <xref ref-type="bibr" rid="ref23 ref24">23, 24</xref>
          ], but its principles can be applied in various contexts. The
GQM method thus provides a structured way to design an indicator system that aligns with the
organization's strategic and operational goals, ensuring that the collected data is relevant and
valuable.
        </p>
      </sec>
      <sec id="sec-2-3">
        <title>2.3. Large Language Model AI Application</title>
        <p>
          The emergence of LLMs has significantly expanded the scope of linguistic tasks that can be
automated and optimized. Such models are LLaMA 2 and GPT-4 [
          <xref ref-type="bibr" rid="ref25 ref26">25, 26</xref>
          ]. The effectiveness of these
models can be further enhanced through the application of Prompt Engineering techniques, which
refine input prompts to improve performance and accuracy [
          <xref ref-type="bibr" rid="ref27">27</xref>
          ]. Several key components of the DPD
leverage LLMs and integrate Prompt Engineering techniques. One such technique is Chain of Thought
prompting, which guides the model through a structured reasoning process by breaking down
complex tasks into a series of intermediate natural language steps, ultimately leading to more
accurate outputs [
          <xref ref-type="bibr" rid="ref28">28</xref>
          ]. Another commonly used technique is One-Shot and Few-Shot prompting,
which involves providing high-quality, hand-crafted input-output examples to enhance model
responses and improve task-specific accuracy [
          <xref ref-type="bibr" rid="ref29">29</xref>
          ]. The desk study provided a crucial basis for
generating ideas on potential AI functionalities for process support and for developing a concept for
lightweight, intuitive tool interaction and visual presentation. Two open-source tools that have
notably influenced our project are Microsoft LIDA [
          <xref ref-type="bibr" rid="ref30">30</xref>
          ] and Chat2Vis [
          <xref ref-type="bibr" rid="ref31">31</xref>
          ]. These tools generate
dataset summaries and use user-provided visualization queries to prompt an LLM, which then
produces a Python-based code for chart generation. By integrating LLM-driven services with Prompt
Engineering techniques, our methodology effectively improves data analysis workflows. These
advancements contribute to the broader adoption of AI-driven data processing tools, optimizing
efficiency and usability in data visualization and exploration. The visual representation of content and
the integration of AI features were significantly inspired by Maggie Appleton [
          <xref ref-type="bibr" rid="ref32">32</xref>
          ]. In her blog post,
Appleton presents several concepts illustrating how users can interact with AI beyond conventional
chat interfaces.
        </p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>3. Methodology</title>
      <p>
        For developing the proposed DPD framework, we adopted the Action Design Research (ADR)
methodology by Sein et al. [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ]. The approach conceptualized the research process as the creation
and evaluation of an IT artifact to solve organizational problems. ADR extends previous frameworks
like the Design Research framework from Peffers et al. [
        <xref ref-type="bibr" rid="ref33">33</xref>
        ] by emphasizing that artifacts emerge
through interactions with organizational elements and by employing an iterative process of recurring
evaluation and refinement. In our case, this fitted well with our problem-oriented, practice-inspired
research, conducted in constant collaboration with our pilot partners. The problem formulation
phase involved a stepwise exploration of several use cases with our pilot partners from the public
administration. The use cases included annually published data reports on public administration
personnel management, political inquiries, and open data releases. These were utilized to analyze
work processes, legal and organizational frameworks, and challenges, ultimately identifying
opportunities for enhancing data product creation in public administration. According to the ADR
framework, the artifact and its theoretical constructs were iteratively validated and refined.
      </p>
      <p>
        Between May 2023 and January 2025, we developed the prototype and conceptual model
iteratively and conducted six prototyping workshops with five field experts from different
departments, responsible for the development of various data products within the public
administration. The assessment of the protocols and transcripts from these workshops was
conducted qualitatively, applying an inductive approach [
        <xref ref-type="bibr" rid="ref34">34</xref>
        ]. To facilitate the ongoing evaluation, we
applied various human-centered design (HCD) methods [
        <xref ref-type="bibr" rid="ref35">35</xref>
        ]. HCD is recognized as a leading approach
for systematic, user-centered system development [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ]. Following HCD principles, prototyping was
central to our process as basic wireframes evolved into interactive prototypes that enabled
discussions on solution approaches, navigation, and design options. In order to evaluate the
prototype at a mature stage, user tests were carried out with three non-technical professionals from
public administration, each responsible for the development of different data products in this area.
To measure the perceived usability, we supplemented the tests by using the System Usability Scale
(SUS), developed by Brooke [
        <xref ref-type="bibr" rid="ref36">36</xref>
        ], a usability assessment tool based on a ten-item, five-point Likert
scale questionnaire.
      </p>
      <p>
        Complementing our practice-oriented research, we conducted a qualitative desk study to review
best practices, innovative market solutions, and emerging approaches in KPI system design, data
analysis, Large Language Models (LLMs), and data management. We also incorporated studies on AI
user interfaces and AI-based semantic search. Desk research is recognized as a vital methodological
tool in qualitative inquiry, as it provides access to a range of materials that are not originally gathered
by the researcher but are instead derived from secondary sources—including academic literature,
online databases, blogs, and industry reports [
        <xref ref-type="bibr" rid="ref37 ref38">37, 38</xref>
        ]. Although ADR is practice-oriented, such
research provided valuable insights that informed our artifact’s development and refinement.
      </p>
    </sec>
    <sec id="sec-4">
      <title>4. Data Product Designer</title>
      <p>The DPD has been developed as a research prototype to empirically explore the identified
challenges and potential solutions associated with data product development in public
administration. Rather than representing a finished product, the DPD serves as an experimental
platform designed to investigate practical research questions in data management, analytics, and the
creation of data products. The DPD is based on an AI-assisted methodology, a conceptual model, and
a technical framework. The DPD is characterized by a specific design concept, developed
collaboratively with our pilot partner to address the unique requirements of public administration.
Creating a data product is a time-consuming process involving multiple stakeholders and requiring
diverse expertise to address organizational, political, and legal parameters while ensuring data
quality. Recognizing this, we formulated a vision for the artifact: subject-matter experts should be
supported by an intuitive, lightweight tool that facilitates the standardized, cross-organizational
development of data products without requiring complex technical expertise.</p>
      <sec id="sec-4-1">
        <title>4.1. AI-assisted Methodology</title>
        <p>A standardized methodology for the development of data products in the public sector offers
great potential for improving harmonization and efficiency. It enables more systematic product
development while fostering alignment between data-driven decisions and overarching strategic
goals.</p>
        <p>
          The Data Product Designer (DPD) was developed to support this process through a flexible
template system that allows for the integration of various goal-oriented methods for indicator
development. While the Goal-Question-Metric (GQM) method has been prototypically implemented
due to its simplicity and structured logic from goals to questions to metrics [
          <xref ref-type="bibr" rid="ref20">20</xref>
          ], the DPD is explicitly
designed to accommodate alternative approaches such as Results-Based Management [
          <xref ref-type="bibr" rid="ref39">39</xref>
          ], Public
        </p>
      </sec>
      <sec id="sec-4-2">
        <title>Value Frameworks [40], or Balanced Scorecards [18].</title>
        <p>The current GQM-based template extends the method by linking strategic objectives to subgoals
and interrelated indicators, enabling traceability and strategic alignment. Additional AI-driven
support facilitates this process while reinforcing methodological consistency.</p>
      </sec>
      <sec id="sec-4-3">
        <title>4.2. Conceptual Model</title>
        <p>The DPD is built around four core areas—Strategy, Search, Data Gap, and Analysis—each supporting
key phases of data product development:



</p>
        <p>Strategy: Definition of goals, questions, and indicators, currently based on GQM but
extendable to other methods.</p>
        <p>Search: Identification of relevant datasets via metadata catalog integration.</p>
        <p>Data Gap: Documentation of missing datasets as data demands and optional generation of
synthetic data when real data is unavailable.</p>
        <p>Analysis: Creation of visualizations linked to indicators, using either real or synthetic data,
supporting iterative refinement.</p>
        <p>
          AI supports the entire process and is implemented in strict adherence to the Human-in-the-Loop
(HITL) principle [
          <xref ref-type="bibr" rid="ref41">41</xref>
          ]. At every stage, AI provides non-binding suggestions—including draft goals,
indicators, dataset recommendations, or visualization options—that users can review, edit, or discard
and therefore retain full control throughout the process.
        </p>
      </sec>
      <sec id="sec-4-4">
        <title>4.3. Technical Framework</title>
        <p>The technical framework of the DPD, illustrated in Figure 1, is designed as an open, modular
architecture compatible with various data management stacks. Based on a microservices approach,
it enables scalable, maintainable integration within broader data ecosystems. The system comprises
three main layers:


</p>
        <p>DPD Core (Figure 1: DPD Core Layer): This foundational layer includes key modules:
a. StrategyMap: A mind-map-style interface for building GQM models, enhanced by AI
suggestions.
b. DataDiscovery and SemanticSearch: Enable data exploration using traditional and
AIdriven vector searches, based on StrategyMap inputs.
c. VizGenerator: Transforms indicator definitions into visualizations and integrates them
into external BI tools.
d. Data Generator and Data Synthesizer: Create synthetic datasets based on identified
data needs, using schema-first generation and configurable AI support.</p>
      </sec>
      <sec id="sec-4-5">
        <title>AI Integration (Figure 1: AI Integration Layer): The PromptBroker middleware orchestrates</title>
        <p>LLM interactions across the platform, streamlining prompt engineering and enabling flexible,
model-agnostic AI integration.</p>
      </sec>
      <sec id="sec-4-6">
        <title>Data Management Infrastructure (Figure 1: Data Spaces): The DPD connects to open data</title>
        <p>catalogs (e.g., the EU Open Data Portal) and internal sources, using organizational data
warehouses and analytics databases.</p>
        <p>Overall, the DPD represents a scalable and AI-enhanced architecture tailored for public sector use,
enabling agile, interoperable data product development.</p>
      </sec>
      <sec id="sec-4-7">
        <title>4.4. Design Concept</title>
        <p>The DPD interface uses familiar interaction patterns, such as a mind-map canvas, to ensure ease
of use and foster methodological understanding. AI features like semantic search and data synthesis
are seamlessly integrated but remain transparent and controllable via visual cues and editable
suggestions. Complex functions are tucked into expandable sidebars, preserving workspace clarity.</p>
        <p>This user-centered design, combined with modular AI support, lowers entry barriers and supports
broader adoption of data-driven tools in public administration.</p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>5. Evaluation results</title>
      <sec id="sec-5-1">
        <title>5.1. Identified Problems</title>
        <p>To gain a deeper understanding of the problems of developing data products, we evaluated our
pilot partners' use cases.</p>
        <p>The first challenging area identified was the development of strategically aligned indicators. The
indicators were developed based on literature research, available data collection methods, and legal
and organizational parameters. However, the resulting indicators were not aligned with strategic
objectives or linked to specific measures, thereby reducing their effectiveness in guiding
decisionmaking. Moreover, the process involved extensive interdisciplinary and cross-organizational
collaboration and required approximately two years to complete the selection of indicators.</p>
        <p>A second challenging area was the exchange of data and data management. The data collection
process is characterized by a very low level of digitalization. Data is sourced from multiple origins,
facing particularly high barriers due to existing media disruptions and communication via emails and
third parties.</p>
        <p>The last key challenge was the creation of visualizations. The transition from PDF-based reports to
dashboard solutions was widely seen as crucial for delivering insights in a more dynamic,
userfriendly, and comprehensive manner. To date, the creation of visualizations has been an elaborate
process that requires technical expertise, and dashboard solutions are not available.</p>
      </sec>
      <sec id="sec-5-2">
        <title>5.2. Key Findings</title>
        <p>Through a comprehensive analysis of challenges and work processes, coupled with continuous
evaluation of the methodology, the conceptual model, and the design concept, key findings emerged.</p>
      </sec>
      <sec id="sec-5-3">
        <title>1. Expanding and Adapting the GQM Methodology for Public Administration Needs: The AI</title>
        <p>supported, standardized method for indicator development was seen by the pilot partner as
a valid approach to enhance quality, strategic alignment, and efficiency. While the GQM
method proved helpful for ensuring strategic alignment and streamlining the process, some
users preferred starting with questions rather than sub-goals. Its usefulness was closely linked
to AI support, which helped overcome formulation barriers and balance user preferences.
Although broadly applicable, it remains unclear whether GQM is the best methodological fit.
Flexibility to adapt the method to specific contexts was considered essential. Still, the adapted
GQM method in its traditional form does not fully address the needs of public administration
and data product creation. To address this gap, the GQM framework was extended to include
data discovery, data demand, data gap analysis, and synthesis. This resulted in an integrated
model tailored to the complexities of public governance.
2. The Need for a More Intuitive and Visual Approach: Previously used data and knowledge
management tools were insufficient for the complex demands of data product development
and required collaboration, often posing technical barriers for non-experts. To address this,
our design principles aimed to integrate key steps of data product creation into a single tool,
use a visual interface, and embed context-aware AI support. Evaluation showed that the
canvas-based, mind-map-like interface of the DPD, combined with AI suggestions and a data
sidebar, provided a low-threshold entry point to data product development. The DPD
achieved an average SUS score of 70, indicating good usability. The integration of AI into the
strategy map was seen as intuitive and helpful. Participants also expressed interest in more
flexible interaction, such as searchable data catalogs entering questions like political queries.</p>
      </sec>
      <sec id="sec-5-4">
        <title>3. AI-Powered Support for Data-Driven Workflows: AI support helped address the identified</title>
        <p>problems of indicator development, data management, and visualization:
a. Supporting Brainstorming and Methodological Adaptation: In sample runs of the
GQM-method, many AI suggestions were adopted or aligned with user ideas; even
unused suggestions supported brainstorming and eased method entry.
b. Improving Data Discovery and Data Demand Management: Locating relevant datasets
in the vast open data landscape remains challenging. The DPD improved data
discovery through AI-based semantic search across multiple catalogs, such as the
European Open Data Portal. However, the success of data discovery efforts depends
critically on the quality of the metadata associated with these datasets. Replacing the
initial method with a vector-based approach improved results and reduced reliance
on high-quality metadata. Still, human involvement remains essential to assess the
relevance and suitability of suggested data. Evaluation revealed the need for a clear
understanding of organizational data needs. The DPD contributes to this by facilitating
the documentation and communication of data demand across different
departments.
c. Reducing Technical Barriers and Addressing Data Gaps Through Synthesis:
AIgenerated visualizations made complex data more accessible, which enables a
broader range of stakeholders to contribute to data product elaboration. However, a
major obstacle was the need for pre-processed data compatible with BI tools.
Additionally, data gaps and restrictions on sharing sensitive data often limit analysis.
To avoid interrupting the creative process, the DPD was extended with an option to
generate synthetic data.</p>
        <p>Overall, user attitudes toward AI use in the DPD were shaped by subjective norms. While AI
integration was seen as central to perceived usefulness, skepticism remained and should be
further addressed. One proven measure to increase acceptance of AI in the public
administration was the integration of open source models such as LLaMA and Mixtral.</p>
        <p>Collectively, the evaluation of the DPD as a research prototype provides valuable insights into both
methodological and technological challenges associated with transitioning to data-driven
governance. Although the DPD remains a research prototype, its experimental deployment has
contributed significantly to the development of flexible and user-friendly data management
infrastructures to support the data product creation process in public administration.</p>
      </sec>
    </sec>
    <sec id="sec-6">
      <title>6. Discussion and future research directions</title>
      <p>The research aimed to tackle the core issue of maintaining strategic alignment, which leads to the
ineffective utilization of data in decision-making processes within public administration. In the course
of our evaluation of the DPD, we identified several challenges that shape future research directions.</p>
      <p>One significant issue was the lack of widely accepted methods for deriving indicators within public
administration. This challenge might be mitigated by concentrating on specific areas that utilize more
specialized monitoring systems, such as security or risk management. New monitoring systems would
also provide the basis for creating new templates that can be adapted to the strategy map.</p>
      <p>The flexible architecture of the DPD allows for seamless integration of existing IT components,
enhancing interoperability. A challenge lay in the integration and utilization of metadata catalogs.
The quality of metadata often varies, which can lead to inconsistencies and challenges in data
integration and usability. Therefore, a potential future extension of the DPD is the integration of
AIbased functionality to enhance and harmonize the quality of metadata. Further, connecting internal
data sources such as data warehouse applications and various relational databases presents a future
research challenge. Establishing robust connectors and ensuring seamless integration will be crucial
for leveraging existing data assets effectively within the public sector’s heterogeneous IT landscapes.</p>
      <p>AI plays a crucial role by facilitating the creation of data products. Further research should examine
how to improve user comprehension, validation, and interaction with AI-generated content in public
administration. At present, the DPD lacks a feedback mechanism that involves human input.
Domainspecific prompt customization is supported by the Prompt Broker and is currently carried out by
supervising experts in collaboration with relevant user groups. Nonetheless, the potential of human
feedback to improve the quality of AI suggestions remains an important area for investigation. A key
challenge identified is the need for users to verify the accuracy of visualizations. Future
enhancements could include explanatory text or additional supportive elements to facilitate
interpretation and ensure correctness. Moreover, future research should examine whether
governance and compliance standards may conflict with AI-generated suggestions and how such risks
can be proactively mitigated. We propose the integration of an AI-powered synthesizer as a Data
Generator component. One of the current research challenges is to generate synthetic data that
closely mirrors real-world data. In the future, the step of metadata generation and its requirements
in the public sector must be examined. For instance, specific types of administrative metadata could
be made available for selection to meet the unique needs of public administration.</p>
      <p>These challenges highlight the need for ongoing enhancements of the DPD framework to ensure
it remains adaptive and responsive to the evolving needs of data-driven governance in the public
sector.</p>
    </sec>
    <sec id="sec-7">
      <title>7. Conclusions</title>
      <p>This paper addresses the question of what organizational and technical methods can enhance the
development of data products in public administration with a problem-oriented and practice-inspired
approach. Our problem formulation identified three challenging areas: developing strategically
aligned indicators, data exchange and management, and generating visualizations. This paper
presents a conceptual model, design concept, and technical framework that illustrate the research
prototype DPD, which offers a solution to these challenges. The evaluation confirmed that these
challenges can be effectively tackled through a unified method that guides users through the process
and ensures the strategic alignment of indicators. Furthermore, utilizing AI enriches brainstorming,
assists in identifying relevant datasets and data needs, and facilitates the generation of visualizations
and synthetic data. Our work also contributed to investigate which methodological approach foster
data product development and integration to achieve high user acceptance in public administration.
This is achieved by further developing the GQM method into an AI-assisted approach specifically
adapted to the requirements of public sector data management.</p>
      <p>Evaluation results offered insights into the perceived usefulness and usability from the pilot
partner's perspective, forming the basis for further refinement of the DPD through the integration of
data demand capture and synthetic data generation to close data gaps. Overall, the DPD contributes
to establishing a robust and user-friendly data management infrastructure in the public
administration, bridging the gap between strategic data analytics and operational data management
and governance. However, the evaluation also highlighted the need for further in-depth
investigation, particularly in ensuring the trustworthiness and traceability of AI-generated results,
enhancing interoperability with internal data sources, and designing AI-driven generation of
highquality, domain-specific synthetic data in a targeted and demand-oriented manner while maintaining
privacy and security. While previous experiences in developing data products contributed to the
evolution of the DPD, the fact that the DPD was evaluated solely within one domain and organization
represents the greatest limitation of our project. Consequently, future studies should explore its
adaptation in other contexts and approaches for refining the indicator derivation method. Moreover,
future research should focus on the integration and standardization of metadata catalogs, enhancing
the semantic description of datasets to improve data quality and usability, and developing robust
connectors to seamlessly integrate diverse internal data sources within the public administration's IT
landscape.</p>
      <p>Overall, the DPD demonstrates its ability to enable collaborative data product creation, enhance
the strategic alignment of indicators and data exchange, and generate initial visualizations without
requiring technical expertise. This concept holds significant potential to strengthen the role of
subject-matter experts, expedite the collective decision-making process regarding focus indicators
and their visualization, and ensure that these indicators are aligned with the strategic needs of the
organization.</p>
    </sec>
    <sec id="sec-8">
      <title>8. Acknowledgements</title>
      <p>This work has been funded by the Federal Ministry for Economic Affairs and Climate Action under
grant no. 68GX21009B (“POSSIBLE: Phoenix open software stack for interoperable engagement in
dataspaces”).</p>
    </sec>
    <sec id="sec-9">
      <title>9. Declaration on Generative AI</title>
      <p>During the preparation of this work, the authors used GPT-4 in order to: Grammar and spelling check.
After using this tool, the authors reviewed and edited the content as needed and take full
responsibility for the publication’s content.
10. References</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <surname>Sangkachan</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Powintara</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          :
          <article-title>Data-driven government: Essential mechanisms to unleash the power of data</article-title>
          .
          <source>International Journal of Electronic Government Research (IJEGR)</source>
          , vol.
          <volume>18</volume>
          ,
          <fpage>1</fpage>
          -
          <lpage>19</lpage>
          (
          <year>2022</year>
          ). doi:
          <volume>10</volume>
          .4018/IJEGR.288070
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <surname>Höchtl</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Parycek</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Schöllhammer</surname>
          </string-name>
          , R.:
          <article-title>Big data in the policy cycle: Policy decision making in the digital era</article-title>
          .
          <source>J Org Comp Elect Com</source>
          , vol.
          <volume>26</volume>
          ,
          <fpage>147</fpage>
          -
          <lpage>169</lpage>
          (
          <year>2016</year>
          ). doi:
          <volume>10</volume>
          .1080/10919392.
          <year>2015</year>
          .1125187
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <surname>Mureddu</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Schmeling</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kanellou</surname>
          </string-name>
          , E.:
          <article-title>Research challenges for the use of big data in policymaking</article-title>
          .
          <source>Transform Gov-People</source>
          , vol.
          <volume>14</volume>
          ,
          <fpage>593</fpage>
          -
          <lpage>604</lpage>
          (
          <year>2020</year>
          ). doi:
          <volume>10</volume>
          .1108/TG-08-2019-0082
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <surname>Artene</surname>
            ,
            <given-names>A.E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Domil</surname>
            ,
            <given-names>A.E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ivascu</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          :
          <article-title>Unlocking Business Value: Integrating AI-Driven DecisionMaking in Financial Reporting Systems</article-title>
          . Electronics, vol.
          <volume>13</volume>
          ,
          <issue>3069</issue>
          (
          <year>2024</year>
          ). doi:
          <volume>10</volume>
          .3390/electronics13153069
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <surname>Giannoccaro</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Natarajan</surname>
            ,
            <given-names>A.K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Galety</surname>
            ,
            <given-names>M.G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Iwendi</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Das</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Shankar</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          :
          <article-title>Intersection of AI and business intelligence in data-driven decision-making</article-title>
          .
          <source>IGI Global</source>
          , Hershey, Pennsylvania (701
          <string-name>
            <given-names>E.</given-names>
            <surname>Chocolate</surname>
          </string-name>
          <string-name>
            <surname>Avenue</surname>
          </string-name>
          , Hershey, Pennsylvania,
          <volume>17033</volume>
          , USA) (
          <year>2024</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <surname>Wu</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Subramaniam</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Li</surname>
            ,
            <given-names>Z.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gao</surname>
            ,
            <given-names>X.</given-names>
          </string-name>
          :
          <article-title>Using AI Technology to Enhance Data-Driven DecisionMaking in the Financial Sector</article-title>
          . In: Dixit,
          <string-name>
            <given-names>S</given-names>
            . (ed.)
            <surname>Artificial</surname>
          </string-name>
          <string-name>
            <given-names>Intelligence-Enabled</given-names>
            <surname>Businesses</surname>
          </string-name>
          . How to Develop Strategies for Innovation, pp.
          <fpage>187</fpage>
          -
          <lpage>207</lpage>
          . John Wiley &amp; Sons Incorporated, Newark (
          <year>2025</year>
          ). doi:
          <volume>10</volume>
          .1002/9781394234028.ch11
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <surname>Jonathan</surname>
            ,
            <given-names>G.M.</given-names>
          </string-name>
          ,
          <string-name>
            <given-names>Kuika</given-names>
            <surname>Watat</surname>
          </string-name>
          , J.:
          <article-title>Strategic alignment during digital transformation</article-title>
          . In: Themistocleous,
          <string-name>
            <given-names>M.</given-names>
            ,
            <surname>Papadaki</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            ,
            <surname>Kamal</surname>
          </string-name>
          , M.M. (eds.).
          <source>Lecture notes in business information processing</source>
          , pp.
          <fpage>657</fpage>
          -
          <lpage>670</lpage>
          . Springer International Publishing, Dubai, United Arab Emirates (
          <year>2020</year>
          ). doi:
          <volume>10</volume>
          .1007/978-3-
          <fpage>030</fpage>
          -63396-7_
          <fpage>44</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <surname>Plantinga</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Voordijk</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Dorée</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          :
          <article-title>Clarifying strategic alignment in the public procurement process</article-title>
          .
          <source>IJPSM</source>
          , vol.
          <volume>33</volume>
          ,
          <fpage>791</fpage>
          -
          <lpage>807</lpage>
          (
          <year>2020</year>
          ). doi:
          <volume>10</volume>
          .1108/IJPSM-10-2019-0245
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <given-names>Alcaide</given-names>
            <surname>Muñoz</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            ,
            <surname>Alcaide</surname>
          </string-name>
          <string-name>
            <surname>Muñoz</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            ,
            <surname>Rodríguez</surname>
          </string-name>
          <string-name>
            <surname>Bolívar</surname>
          </string-name>
          ,
          <string-name>
            <surname>M.P.</surname>
          </string-name>
          :
          <article-title>Strategic alignment of open government initiatives in Andalusia</article-title>
          .
          <source>International Review of Administrative Sciences</source>
          , vol.
          <volume>89</volume>
          ,
          <fpage>685</fpage>
          -
          <lpage>702</lpage>
          (
          <year>2023</year>
          ).
          <source>doi: 10.1177/00208523221086125</source>
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <surname>Choi</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gil-Garcia</surname>
            ,
            <given-names>J.R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Aranay</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Burke</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Werthmuller</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          :
          <article-title>Using artificial intelligence techniques for evidence-based decision making in government: Random forest and deep neural network classification for predicting harmful algal blooms in New York State</article-title>
          . In: Lee,
          <string-name>
            <surname>J.</surname>
          </string-name>
          , Vilale Pereira,
          <string-name>
            <given-names>G.</given-names>
            ,
            <surname>Hwang</surname>
          </string-name>
          , S. (eds.), pp.
          <fpage>27</fpage>
          -
          <lpage>37</lpage>
          . Association for Computing Machinery, Omaha, Nebraska (
          <year>2021</year>
          ). doi:
          <volume>10</volume>
          .1145/3463677.3463713
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <surname>Brous</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Janssen</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Vilminko-Heikkinen</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          :
          <article-title>Coordinating Decision-Making in Data Management Activities: A Systematic Review of Data Governance Principles</article-title>
          . In: Scholl,
          <string-name>
            <given-names>H.J.</given-names>
            ,
            <surname>Glassey</surname>
          </string-name>
          ,
          <string-name>
            <given-names>O.</given-names>
            ,
            <surname>Janssen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            ,
            <surname>Klievink</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            ,
            <surname>Lindgren</surname>
          </string-name>
          ,
          <string-name>
            <given-names>I.</given-names>
            ,
            <surname>Parycek</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            ,
            <surname>Tambouris</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            ,
            <surname>Wimmer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.A.</given-names>
            ,
            <surname>Janowski</surname>
          </string-name>
          ,
          <string-name>
            <surname>T.</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Sa</given-names>
            <surname>Soares</surname>
          </string-name>
          ,
          <string-name>
            <surname>D</surname>
          </string-name>
          . (eds.).
          <source>Lecture Notes in Computer Science</source>
          , pp.
          <fpage>115</fpage>
          -
          <lpage>125</lpage>
          . Springer International Publishing, Guimarães, Portugal (
          <year>2016</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <surname>Yukhno</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          :
          <article-title>Digital Transformation: Exploring big data Governance in Public Administration</article-title>
          .
          <source>Public Organization Review</source>
          , vol.
          <volume>24</volume>
          ,
          <fpage>335</fpage>
          -
          <lpage>349</lpage>
          (
          <year>2024</year>
          ). doi:
          <volume>10</volume>
          .1007/s11115-022-00694-x
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13.
          <string-name>
            <surname>Sein</surname>
          </string-name>
          , Henfridsson, Purao, Rossi, Lindgren: Action Design Research.
          <source>MIS Quarterly</source>
          , vol.
          <volume>35</volume>
          ,
          <issue>37</issue>
          (
          <year>2011</year>
          ).
          <source>doi: 10.2307/23043488</source>
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <string-name>
            <surname>Paul</surname>
          </string-name>
          , M.O.:
          <article-title>Systemgestützte Integration von Usability-Methoden in den SoftwareEntwicklungsprozess (</article-title>
          <year>2014</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          15.
          <string-name>
            <surname>Joshi</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pratik</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rao</surname>
            ,
            <given-names>M.P.</given-names>
          </string-name>
          :
          <article-title>Data governance in data mesh infrastructures: the Saxo Bank case study</article-title>
          .
          <source>In: Proceedings of The 21st International Conference on Electronic Business</source>
          (
          <year>2021</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          16.
          <string-name>
            <surname>Machado</surname>
            ,
            <given-names>I.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Costa</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Santos</surname>
            ,
            <given-names>M.Y.</given-names>
          </string-name>
          :
          <article-title>Data Mesh: Concepts and Principles of a Paradigm Shift in Data Architectures</article-title>
          .
          <source>Procedia Computer Science</source>
          , vol.
          <volume>196</volume>
          ,
          <fpage>263</fpage>
          -
          <lpage>271</lpage>
          (
          <year>2022</year>
          ). doi:
          <volume>10</volume>
          .1016/j.procs.
          <year>2021</year>
          .
          <volume>12</volume>
          .013
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          17.
          <string-name>
            <surname>Janssen</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Brous</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Estevez</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Barbosa</surname>
            ,
            <given-names>L.S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Janowski</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          :
          <article-title>Data governance: Organizing data for trustworthy Artificial Intelligence</article-title>
          .
          <source>Government Information Quarterly</source>
          , vol.
          <volume>37</volume>
          ,
          <issue>101493</issue>
          [
          <fpage>1</fpage>
          -8] (
          <year>2020</year>
          ). doi:
          <volume>10</volume>
          .1016/j.giq.
          <year>2020</year>
          .101493
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          18.
          <string-name>
            <surname>Kaplan</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Norton</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          :
          <article-title>The Balanced Scorecard: Translating Strategy into Action by Robert</article-title>
          .
          <source>Harvard Business School</source>
          , Boston (
          <year>1996</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          19.
          <article-title>Practical software measurement</article-title>
          .
          <article-title>Objective information for decision makers</article-title>
          .
          <source>Addison-Wesley</source>
          , Boston, MA (
          <year>2002</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          20.
          <string-name>
            <surname>Basili</surname>
            ,
            <given-names>V.R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Weiss</surname>
            ,
            <given-names>D.M.:</given-names>
          </string-name>
          <article-title>A Methodology for Collecting Valid Software Engineering Data</article-title>
          .
          <source>IIEEE Trans. Software Eng.</source>
          , vol. SE-
          <volume>10</volume>
          ,
          <fpage>728</fpage>
          -
          <lpage>738</lpage>
          (
          <year>1984</year>
          ). doi:
          <volume>10</volume>
          .1109/TSE.
          <year>1984</year>
          .5010301
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          21.
          <string-name>
            <surname>International</surname>
          </string-name>
          <article-title>Standard: ISO/IEC 15939 Systems and software engineering - Measurement process</article-title>
          , vol. (
          <year>2017</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          22.
          <string-name>
            <surname>Muñoz</surname>
            ,
            <given-names>R.F.Z.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Alegría</surname>
            ,
            <given-names>J.A.H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Román-Gonzalez</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Robles</surname>
          </string-name>
          , G.:
          <article-title>Computational Thinking (CT) Measurement Model Based on Bloom's Taxonomy and the GQM Model</article-title>
          . In:
          <string-name>
            <surname>Duque-Méndez</surname>
            ,
            <given-names>N.D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Aristizábal-Quintero</surname>
            ,
            <given-names>L.Á.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Orozco-Alzate</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Aguilar</surname>
            ,
            <given-names>J</given-names>
          </string-name>
          . (eds.)
          <source>Advances in Computing. 18th Colombian Conference on Computing, CCC</source>
          <year>2024</year>
          , Manizales, Colombia, September 4-
          <issue>6</issue>
          ,
          <year>2024</year>
          ,
          <string-name>
            <given-names>Revised</given-names>
            <surname>Selected</surname>
          </string-name>
          <string-name>
            <surname>Papers</surname>
          </string-name>
          ,
          <source>Part II. Communications in Computer and Information Science</source>
          , vol.
          <volume>2209</volume>
          , pp.
          <fpage>134</fpage>
          -
          <lpage>149</lpage>
          . Springer Nature Switzerland; Imprint Springer, Cham (
          <year>2024</year>
          ). doi:
          <volume>10</volume>
          .1007/978-3-
          <fpage>031</fpage>
          -75236-0_
          <fpage>11</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          23.
          <string-name>
            <surname>Philippou</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Frey</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rashid</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          :
          <article-title>Contextualising and aligning security metrics and business objectives: A GQM-based methodology</article-title>
          .
          <source>Computers &amp; Security</source>
          , vol.
          <volume>88</volume>
          ,
          <issue>101634</issue>
          (
          <year>2020</year>
          ). doi:
          <volume>10</volume>
          .1016/j.cose.
          <year>2019</year>
          .101634
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          24.
          <string-name>
            <surname>Calvo</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Beltrán</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          :
          <article-title>Applying the Goal, Question, Metric method to derive tailored dynamic cyber risk metrics</article-title>
          .
          <source>Inf Comput Secur</source>
          , vol.
          <volume>32</volume>
          ,
          <fpage>133</fpage>
          -
          <lpage>158</lpage>
          (
          <year>2024</year>
          ). doi:
          <volume>10</volume>
          .1108/ICS-03-2023-0043
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          25.
          <string-name>
            <surname>Touvron</surname>
          </string-name>
          et al.,
          <source>H.: Llama</source>
          <volume>2</volume>
          :
          <string-name>
            <given-names>Open</given-names>
            <surname>Foundation</surname>
          </string-name>
          and
          <string-name>
            <surname>Fine-Tuned Chat Models</surname>
          </string-name>
          (
          <year>2023</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          26. OpenAI: GPT-4
          <source>Technical Report</source>
          (
          <year>2023</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          27.
          <string-name>
            <surname>Sahoo</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Singh</surname>
            ,
            <given-names>A.K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Saha</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Jain</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mondal</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chadha</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          :
          <article-title>A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications (</article-title>
          <year>2024</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          28.
          <string-name>
            <surname>Wang</surname>
            ,
            <given-names>X.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wei</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Schuurmans</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Le</surname>
            <given-names>Quoc</given-names>
          </string-name>
          , Chi,
          <string-name>
            <given-names>E.</given-names>
            ,
            <surname>Narang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            ,
            <surname>Chowdhery</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            ,
            <surname>Zhou</surname>
          </string-name>
          ,
          <string-name>
            <surname>D.</surname>
          </string-name>
          :
          <article-title>SelfConsistency Improves Chain of Thought Reasoning in Language Models (</article-title>
          <year>2022</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref29">
        <mixed-citation>
          29.
          <string-name>
            <surname>Brown</surname>
          </string-name>
          , T.B.,
          <string-name>
            <surname>Mann</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ryder</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Subbiah</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kaplan</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Dhariwal</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Neelakantan</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Shyam</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sastry</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Askell</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          , et al.:
          <article-title>Language Models are Few-Shot Learners (</article-title>
          <year>2020</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref30">
        <mixed-citation>
          30.
          <string-name>
            <surname>Dibia</surname>
          </string-name>
          , V.:
          <article-title>LIDA: A Tool for Automatic Generation of Grammar-Agnostic Visualizations and Infographics using Large Language Models (</article-title>
          <year>2023</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref31">
        <mixed-citation>
          31.
          <string-name>
            <surname>Maddigan</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Susnjak</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          :
          <article-title>Chat2VIS: Generating Data Visualisations via Natural Language using ChatGPT, Codex</article-title>
          and GPT-3
          <string-name>
            <surname>Large Language Models</surname>
          </string-name>
          (
          <year>2023</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref32">
        <mixed-citation>
          32.
          <string-name>
            <surname>Appleton</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          :
          <article-title>Language Model Sketchbook, or Why I Hate Chatbots</article-title>
          , https://maggieappleton.com/lm-sketchbook#daemons
        </mixed-citation>
      </ref>
      <ref id="ref33">
        <mixed-citation>
          33.
          <string-name>
            <surname>Peffers</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rothenberger</surname>
            ,
            <given-names>M.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tuunanen</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chatterjee</surname>
            ,
            <given-names>S.:</given-names>
          </string-name>
          <article-title>A design science research methodology for information systems research</article-title>
          .
          <source>Journal of Management Information Systems</source>
          , vol. ,
          <volume>45</volume>
          -
          <fpage>77</fpage>
          (
          <year>2007</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref34">
        <mixed-citation>
          34.
          <string-name>
            <surname>Miles</surname>
            ,
            <given-names>M.B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Huberman</surname>
            ,
            <given-names>A.M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Saldaña</surname>
          </string-name>
          , J.:
          <article-title>Qualitative data analysis. A methods sourcebook</article-title>
          .
          <source>SAGE</source>
          , Los Angeles, London, New Delhi, Singapore, Washington DC,
          <string-name>
            <surname>Melbourne</surname>
          </string-name>
          (
          <year>2020</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref35">
        <mixed-citation>
          35.
          <article-title>Ergonomics of human-system interaction</article-title>
          .
          <source>Part</source>
          <volume>210</volume>
          :
          <article-title>human-centred design for interactive systems</article-title>
          (ISO 9241-
          <fpage>210</fpage>
          :
          <year>2010</year>
          ), vol.
        </mixed-citation>
      </ref>
      <ref id="ref36">
        <mixed-citation>
          36. john Brooke: SUS:
          <article-title>A 'Quick and Dirty' Usability Scale Usability Evaluation In Industry</article-title>
          , pp.
          <fpage>207</fpage>
          -
          <lpage>212</lpage>
          . CRC Press (
          <year>1996</year>
          ). doi:
          <volume>10</volume>
          .1201/
          <fpage>9781498710411</fpage>
          -
          <lpage>35</lpage>
        </mixed-citation>
      </ref>
      <ref id="ref37">
        <mixed-citation>
          37.
          <string-name>
            <surname>Mayring</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          :
          <article-title>Qualitative Inhaltsanalyse</article-title>
          . In: Mey,
          <string-name>
            <given-names>G.</given-names>
            ,
            <surname>Mruck</surname>
          </string-name>
          ,
          <string-name>
            <surname>K</surname>
          </string-name>
          . (eds.)
          <source>Handbuch Qualitative Forschung in der Psychologie</source>
          , pp.
          <fpage>601</fpage>
          -
          <lpage>613</lpage>
          . VS Verlag für Sozialwissenschaften,
          <source>Wiesbaden</source>
          (
          <year>2010</year>
          ). doi:
          <volume>10</volume>
          .1007/978-3-
          <fpage>531</fpage>
          -92052-8_
          <fpage>42</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref38">
        <mixed-citation>
          38.
          <string-name>
            <surname>Mayring</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          :
          <article-title>Einführung in die qualitative Sozialforschung</article-title>
          . Beltz, Weinheim, Basel (
          <year>2023</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref39">
        <mixed-citation>
          39.
          <string-name>
            <surname>Oecd</surname>
          </string-name>
          <article-title>: Glossary of Key Terms in Evaluation and Results-Based Management for Sustainable Development (Second Edition) (</article-title>
          <year>2023</year>
          ), https://www.oecd.org/en/publications/glossary
          <article-title>-of-keyterms-in-evaluation-and-results-based-management-for-sustainable-development-secondedition_632da462-en-fr-es</article-title>
          .html
        </mixed-citation>
      </ref>
      <ref id="ref40">
        <mixed-citation>
          40.
          <string-name>
            <surname>Symes</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          :
          <article-title>Creating public value: Strategic management in government</article-title>
          (Cambridge, MA: Harvard University Press) by Mark
          <string-name>
            <surname>Moore. Int Public Manag</surname>
            <given-names>J</given-names>
          </string-name>
          , vol.
          <volume>2</volume>
          ,
          <fpage>158</fpage>
          -
          <lpage>167</lpage>
          (
          <year>1999</year>
          ). doi:
          <volume>10</volume>
          .1016/S1096-
          <volume>7494</volume>
          (
          <issue>00</issue>
          )
          <fpage>87438</fpage>
          -
          <lpage>3</lpage>
        </mixed-citation>
      </ref>
      <ref id="ref41">
        <mixed-citation>
          41.
          <string-name>
            <surname>Amershi</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Cakmak</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Knox</surname>
            ,
            <given-names>W.B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kulesza</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          :
          <article-title>Power to the People: The Role of Humans in Interactive Machine Learning</article-title>
          .
          <source>AI Magazine</source>
          , vol.
          <volume>35</volume>
          ,
          <fpage>105</fpage>
          -
          <lpage>120</lpage>
          (
          <year>2014</year>
          ). doi:
          <volume>10</volume>
          .1609/aimag.v35i4.
          <fpage>2513</fpage>
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>